Policy Analysis in Canada: The State of the Art

References

Analysis of Environmental Protection’ University of Pennsylvania Law Review 150(5), 1553-1584.

Adams, J. 1992. ‘Horse and Rabbit Stew.’ In A. Coker and C. Richards eds., Valuing the

Environment: Economic Approaches to Environmental Evaluation NY: John Wiley & Sons, 65-73.

Adler, Matthew D. 1998. ‘Incommensurability and Cost-Benefit Analysis.’ University of

Adler, Matthew D. and Eric A. Posner. 2000. ‘Implementing Cost-Benefit Analysis

When Preferences Are Distorted.’ Journal of Legal Studies. 29(2) Part 2, 1105-47.

Anderson, David A. 1999. ‘The Aggregate Burden of Crime’ Journal of Law and

ARA Consulting Group. 1995. ‘Prince George Timber Supply Area Socio-Economic

Analysis.’ Victoria: The Queen’s Printer.

Arrow, Kenneth J., Cropper, Maureen L., Eads, George C., Hahn Robert W., Lave, Lester

B., Noll, Roger, C., Portney, Paul R., Russell, Milton, Schmalensee, Richard, Smith, V. Kerry, and Stavins, Robert N. 1996. ‘Benefit-Cost Analysis in Environmental, Health and Safety Regulation: A Statement of Principles.’ Washington D.C: American Enterprise Institute for Public Policy Research.

Auditor General of Canada. 1996. Report of the Auditor General of Canada 1996.

‘Evaluation in the Federal Government,’ (Chapter 3, May), ‘The Canada Infrastructure Works Program; Lessons Learned,’ (Chapter 26, November).

- 1997. Report of the Auditor General of Canada 1997, ‘Reporting Performance in the

Expenditure Management System,’ (Chapter 5, April), ‘Moving Toward Managing for Results,’ (Chapter 11, April).

- 2000. Report of the Auditor General of Canada 2000, ‘Reporting Performance to

Parliament – Progress too Slow,’ (Chapter 19, December), ‘Managing Departments for Results...,’ (Chapter 20, December).

- 2003. Report of the Auditor General of Canada 2000. ‘Rating Departmental

Performance Reports,’ (Chapter 1).

Bergson, Abram. 1938. ‘A Reformulation of Certain Aspects of Welfare Economics.’

Birch, Stephen and Cam Donaldson. 2003.’Valuing the Benefits and Costs of Health

Care Programmes: Where’s the ‘Extra’ in Extra-Welfarism?’ Social Science & Medicine. 56(5), 1121-1133.

Boardman, Anthony, E., Aidan R. Vining and William Waters III. 1993. ‘Costs and

Benefits through Bureaucratic Lenses: Example of a Highway Project.’ Journal of Policy Analysis and Management. 12(3), 532-555.

Boardman, Anthony, E., Wendy L. Mallery and Aidan R. Vining. 1994. ‘Learning from

Boardman, Anthony E., David H. Greenberg, Aidan R. Vining and David L. Weimer.

1997. ‘Plug-In Shadow Price Estimates for Policy Analysis’ Annals of Regional Science. 31(4), 299-324.

Boardman, Anthony, E., David H. Greenberg, Aidan R. Vining and David L. Weimer.

2001. Cost-Benefit Analysis: Concepts and Practice 2^nd. Ed. Upper Saddle River, NJ: Prentice Hall.

Bowden, Gary K and Wally H. Malkinson. 1982. ‘Bt-Cost Analysis of the North East

Coal Development.’ Victoria, BC: Ministry of Industry and Small Business Development.

Buss, Terry F., and Laura C. Yancer. 1999. ‘Cost-Benefit Analysis: A Normative

Perspective’ Economic Development Quarterly. 13(1), 29-37.

Cavanagh, Sheila M., Robert W. Hahn and Robert N. Stavins. 2001. ‘National

Environmental Policy During the Clinton Years,’ Washington, DC: AEI-Brookings Joint Center Working Paper No. 01-09.

Congressional Budget Office (CBO). 1992. Auctioning Radio Spectrum Licenses,

Washington, D.C., March.

Crown Corporations Secretariat. 1993. Multiple Account Evaluation Guidelines, Victoria,

BC: Province of British Columbia.

Currie, Janet. 2001. ‘Early Childhood Education Programs.’ Journal of Economic

Davey, Anne, David Olson and Jyrki Wallenius. 1994.’The Process of Multiattribute

Decision Making: A Case Study of Selecting Applicants for a Ph. D. Program.’ European Journal of Operational Research. 72(3), 469-85.

Davis, H. Craig. 1990. Regional Economic Impact Analysis and Project Evaluation,

Vancouver, BC: University of British Columbia Press.

De Alessi, Louis. 1996. ‘Error and Bias in Benefit-Cost Analysis: HUD’s Case for the

Wind Rule.’ Cato Journal. 16(1), 129-147.

DiNardo, John and Justin L. Tobias. 2001. ‘Nonparametric Density and Regression

Estimation.’ Journal of Economic Perspectives. 15(4), 11-28.

Dodgson, John, Michael Spackman, Alan Pearman and Lawrence Phillips. 2001. Multi-

Dolan, Paul and Richard Edlin. 2002. ‘Is It Really Possible to Build a Bridge Between

Cost-Benefit Analysis and Cost-Effectiveness Analysis?’ Journal of Health Economics. 21(5), 827-843.

Dyer, James S., Peter C. Fishburn, Ralph E. Steuer, Jyrki Wallenius and Stanley Zionts.

1992. ‘Multiple Criteria Decision Making, Multiattribute Utility Theory: The Next Ten Years.’ Management Science. 38(5), 645-654.

Easton, Allan. 1973. Complex Managerial Decisions Involving Multiple Objectives,

N.Y.: John Wiley & Sons.

Edwards, Ward and J. Robert Newman (with the collaboration of Kurt Sapper and David

Server). 1982. Multiattribute Evaluation, Beverly Hills, CA: Sage Publications.

Flyvbjerg, Bent, Mette S. Holm and Soren Buhl. 2002. ‘Underestimating Costs in Public

Works Projects,’ Journal of the American Planning Association, 68(3), 279-295.

Friedlander, David, David H. Greenberg and Philip K. Robins. 1997. Journal of

Gray, John B. 2002. ‘The Economic Impact of the Winter Olympic & Paralympic

Games.’ Report Prepared by the Capital Projects Branch, Ministry of Competition, Science and Enterprise.

General Accounting Office (GAO). 1998. ‘Regulatory Reform: Agencies Could Improve

Development, Documentation, and Clarity of Regulatory Economic Analyses,’ Report to the Committee on Governmental Affairs, U.S. Senate, GAO/RCED-98-142, May.

Greenberg, David, H. 1992. ‘Conceptual Issues in Cost-Benefit Analysis of Welfare-to-

Work Programs.’ Contemporary Policy Issues. 10(4), 51-63.

Greene, Ian. 2002. ‘Lessons Learned from two Decades of Program Evaluation in

Canada.’ In Deitmar Braunig and Peter Eichorn eds., Evaluation and Accounting Standards in Public Management, Baden-Baden: Nomos Verlagsgesellschaft, 44-53.

Hahn, Robert W., Jason K. Burnett, Yee-Ho I. Chan, Elizabeth A. Mader, and Petrea R.

Moyle. 2000. ‘Assessing Regulatory Impact Analyses: The Failure of Agencies to

Comply with Executive Order 12,866.’ Harvard Journal of Law & Public Policy. 23(3), 859-885.

Hahn, Robert W. and Cass R. Sunstein. 2002. ‘A New Executive Order for Improving

Federal Regulation? Deeper and Wider Cost-Benefit Analysis.’ University of Pennsylvania Law Review. 150(5), 1489-1552.

Hahn, Robert W. 2000. ‘State and Federal Regulatory Reform: A Comparative Analysis.’

Journal of Legal Studies. 29(2), 873-912.

Harberger, Arnold C. 1978. ‘On the Use of Distributional Weights in Social Cost-

Benefit Analysis.’ Journal of Political Economy. 86(2), S87-120.

Harberger, Arnold C. 1997. ‘Economic Project Evaluation, Part 1: Some Lessons for the

1990s.’ Canadian Journal of Program Evaluation. Special Issue, 5-46.

Health Canada. 2004. ‘Economic Evaluation of Health Canada’s Regulatory Proposal for

Reducing Fire Risks from Cigarettes,’ Ottawa, ON: Health Canada.

Heckman James, L. 2001. ‘Micro Data, Heterogeneity, and the Evaluation of Public

Policy: Nobel Lecture.’ Journal of Political Economy. 109(4), 673-748.

Herfindahl, Orris C. and Allan V. Kneese. 1974. Economic Theory of Natural Resources,

Columbus, OH: Charles Merrill.

HLB Decision Economics (in association with ICF Consulting and PB Consult). 2002.

HM Treasury. 1997. Appraisal and Valuation in Central Government: Treasury

Hrudey, Steve E. (Chair), Vic Adamowicz, Alan Krupnick, John McConnell, Paolo

Renzi, Robert dales, Morton Lippmann and Beverley Hale. 2001. Report of an Expert Panel to Review the Socio-Economic Models and Related Components Supporting the Development of Canada-Wide Standards for Particulate Matter and Ozone to the Royal Society of Canada, RSC EPR 01-2, Ottawa, ON: Royal Society of Canada.

Hurley, J. 1998. ‘Welfarism, Extra-Welfarism and Evaluative Economic Analysis in

the Health Sector.’ In Morris Barer, T. Getzen and G. L. Stoddard eds., Health, Health Care and Health Economics: Perspectives on Distribution, Chichester, U.K: John Wiley, 373-395.

InterVISTAS Consulting Inc. 2002. ‘The Economic Impact of the 2010 Winter Olympic

and Paralympic Games.’ Prepared for Honourable Ted Bebbeling, Minister of State for Community Charter and 2010 Olympic Bid.

Joubert, Alison, R., Anthony Leiman, Helen M. de Klerk, Stephen Katua and J. Coenrad

Aggenbach. 1997. ‘Fynbos (Fine Bush) Vegetation and the Supply of Water: A

Comparison of Multi-Criteria Decision Analysis and Cost-Benefit Analysis.’ Ecological Economics. 22(2), 123-140.

Kaplan, Robert S. and David P. Norton. 1996. ‘Linking the Balanced Scorecard to

Strategy.’ California Management Review. 39(1), 53-79.

Keeney, Ralph L. and Raiffa, Howard. 1976. Decisions with Multiple Objectives:

Preferences and Value Tradeoffs, N.Y.: John Wiley & Sons.

KPMG. 1996. ‘Benefits and Costs of Treaty Settlements in British Columbia – A

Financial and Economic Perspective,’ Victoria, B.C.: Queen’s Printer.

LaLonde, Robert J. 1997. ‘The Promise of Public Sector-Sponsored Training Programs.’

Levelton, Paul, Lisa Kershaw and William Reid. 1966. ‘Estimating Economic Impacts

and Market Potential Associated with the Development and Production of Fuel Cells in British Columbia.’ KPMG Report Prepared for the Ministry of Environment, Lands and Parks, Victoria, BC: 60058/WPL/br.

Long, David, Charles D. Mallar and Craig V. Thornton. 1981. ‘Evaluating the Benefits

and Costs of the Jobs Corps.’ Journal of Policy Analysis and Management. 1(1), 55-76.

Martin, Fernand. 2001. ‘Evaluating a Large-Scale Infrastructure Project: A High-Speed

Railway Along the Quebec City to Windsor Corridor.’ In Aidan R. Vining and John Richards, eds., Building the Future: Issues in Public Infrastructure in Canada, Toronto, ON: C.D. Howe Institute, 195-213.

Marvin Shaffer & Associates Ltd. 1992. ‘Kispiox Resource Management Plan Socio-

Economic Impact Assessment,’ Final Report, Submitted to Economic and Trade Branch, Ministry of Forests.

Mayer, Igor S., C. Els van Daalen and Peter W.G. Bots. 2004. ‘Perspectives on Policy

Analyses: A Framework for Understanding and Design.’ International Journal of Technology Policy and Management. 4(2), 169-191.

Mayne, John. 1994. ‘Utilizing Evaluation in Organizations: The Balancing Act.’ In Frans

L. Leeuw, Ray C. Rist and Richard C. Sonnichsen, eds., Can Governments Learn? New Brunswick, N.J: Transaction Publishers, 17-44.

Ministry of Transportation. 2001. Highway 99 Alternative Routes Analysis Vancouver to

Moore, John L. 1995. ‘Cost-Benefit Analysis: Issues in its Use in Regulation.’

Committee for the National Institute for the Environment, June.

Mueller, Dennis C. 1989. Public Choice II, Cambridge: Cambridge University Press.

Muller-Clemm, Werner J. and Maria P. Barnes. 1997. ‘A Historical Perspective on

Federal Program Evaluation in Canada.’ Canadian Journal of Program Evaluation. 12(1), 47-70.

Myers, Samuel L. 2002. ‘Presidential Address—Analysis of Race as Policy Analysis.’

Nijkamp, Peter P. 1997. Theory and Application of Environmental Economics,

Amsterdam: North-Holland.

Nord, Erik, Jeff Richardson, Andrew Street Hega Kuhse and Peter Singer. 1995

‘Maximizing Health Benefits vs. Egalitarianism: An Australian Survey of Health Issues.’ Social Science & Medicine. 41(10), 1429-1437.

Nussbaum, Martha C. 2000. ‘The Costs of Tragedy: Some Moral Limits of Cost-Benefit

Analysis.’ Journal of Legal Studies. 29(2), Part 2, 1005-36.

OECD. 1995. Recommendation on Improving the Quality of Government Regulation.

Okun, Arthur, M. 1975. Equality and Efficiency: The Big Tradeoff, Washington D.C:

The Brookings Institution.

Pearce, David. 1998. ‘Cost-Benefit Analysis and Environmental Policy’ Oxford Review

of Economic Policy. 14(4), 84-100.

Phillips, Patti P. and Jack J. Phillips. 2004. ‘ROI in the Public Sector: Myths and

Realities.’ Public Personnel Management. 33(2), 139-149.

Privy Council Office. 1999. Government of Canada Regulatory Policy, Ottawa, ON.

Radin, Beverly. 2002. ‘The Government Performance and Results Act and the Tradition

of Federal Management Reform: Square pegs in Round Holes? Journal of Public Administration Research and Theory. 10(1), 111-135.

Rees, William E. 1998. ‘How Should a Parasite Value Its Host? Ecological Economics.

25(1), 49-52.

Rich, M.J. 1989. ‘Distributive Politics and the Allocation of Federal Grants.’ American

Political Science Review. 83(1), 193-213.

Reinke, William A. 1999. ‘A Multi-Dimensional Program Evaluation Model:

Considerations of Cost-Effectiveness, Equity, Quality, and Sustainability.’ Canadian Journal of Program Evaluation. 14(2), 145-160.

Richards, John, Aidan Vining, D. Brown, M. Krashinsky, W. Milne, E. Lightman, and S.

Hoy. 1995. Helping the Poor: A Qualified Case for ‘Workfare,’ Toronto, ON: C.D. Howe Institute.

Sanders, Heywood T. 2002. ‘Convention Myths and Markets: A Critical Review of

Convention Center Feasibility Studies.’ Economic Development Quarterly, 16(3), 195-210.

Schwindt, Richard, Aidan Vining and David Weimer. 2003. ‘A Policy Analysis of the BC

Salmon Fishery.’ Canadian Public Policy. 29(1), 73-93.

Svenson, Ola. 1979. ‘Process Description of Decision Making.’ Organizational Behavior

(TBS) Treasury Board of Canada Secretariat. 1976. Benefit-Cost Analysis Guide, Ottawa,

ON: Treasury Board of Canada Secretariat.

- 1997. Program Evaluation Methods, Third Edition, Ottawa, ON: Treasury Board of

Canada Secretariat, (www.tbs-sct.gc.ca).

- 1998. Benefit-Cost Analysis Guide, Ottawa, ON: Treasury Board of Canada Secretariat

(www.tbs-sct.gc.ca).

- no date. Study of the Evaluation Function in the Federal Government.

http://www.tbs-sct.gc.ca/eval/stud_etud/func-fonc-01_e.asp.

Vining, Aidan R. and David L. Weimer. 2001. ‘Criteria for Infrastructure Investment:

Normative, Positive, and Prudential Perspectives.’ In Aidan R. Vining and John Richards, eds., Building the Future: Issues in Public Infrastructure in Canada, Toronto: C.D. Howe Institute, 131-165.

1W.W. Waters, II. Undated. ‘A Reanalysis of the North East Coal Development,’ UBC:

mimeo.

Webber, David J. 1986. ‘Analyzing Political Feasibility: Political Scientists’ Unique

Weimer, David L. and Aidan R.Vining. 2005. Policy Analysis: Concepts and Practice.

4^th. Ed., Upper Saddle River, N.J: Pearson Prentice Hall.

Whitehead, Paul C. and William R. Avison. 1999. ‘Comprehensive Evaluation: The

Intersection of Impact Evaluation and Social Accounting.’ Canadian Journal of Program Evaluation, 14(1), 65-83.

Table 1

Metachoice Framework and Choice Classes

Benefit-Cost Analysis of the North East Coal Project

Source: (Bowden and Malkinson 1982, 108). All figures are present values in 1980 dollars, using a 10% discount rate and assuming no terminal value in 2003, the end of the discounting period.

Typology of Efficiency Analysis Methodologies

A Hypothetical Example of MNBA+ Analysis

Table 5

Monetized Net Benefit Analysis of Proposed Ignition Propensity Standard

Source: Derived from Health Canada 2004. All figures are present values in 2002 $CA millions, assuming impacts occur in perpetuity and a 3 percent discount rate.

An Example of Embedded NPV Analysis

Total Net Financial Benefits to British Columbia of Treaty Settlements

An Example of a Multi-Goal Analysis: B.C Salmon Fishery

		Alternative Policies
Goals	Criteria	Current Policy: Continued Implementation of the Mifflin Plan	Harvesting Royalties and License Auction (Pearse Plan)	River-Specific Exclusive Ownership Rights	Individual Transferable Quotas to Current License Holders
Efficient Resource Use	Impact on Rent Dissipation	Poor — large negative net present value	Good — considerable improvement over status quo	Very good — major improvement over status quo, lowest cost technology	Good — considerable improvement over status quo
Protection	Impact on the Number of Viable Runs	Poor — high risk	Poor — continued risk for vulnerable runs	Good — very good except possibly for Fraser River	Reasonable (provided ‘share’ quotas used)
Equitable Distribution	Fairness to Current License Holders	Good	Good	Depends upon compensation for licenses	Very good
	Fairness to Native Fishers	Neutral, good for incumbents	Neutral, good for incumbents	Good	Neutral, very good for incumbents
	Fairness to Taxpayers	Poor, large net costs	Excellent after phase-in	Excellent	Poor, but depends on fees

Table 10

Multi-Goal Analysis of Alternative Routes Between Vancouver and Squamish

COST and IMPACT FACTORS 1 = least preferable to 5 = most preferable	ROUTE OPTIONS
	Highway 99 North upgrade	Capilano River	Seymour River	Indian Arm /Indian River	Hybrid Seymour-Indian River
FINANCIAL ACCOUNT Capital cost
- route cost	5	3	1	1	2
- link to Provincial network	5	3	3	1	3
- network upgrade cost	4	4	4	2	4
Added operating/maintenance cost	5	4	3	1	4
Traffic disruption cost	1	4	5	4	5
CUSTOMER SERVICE ACCOUNT User travel time reduction	3	5	5	2	5
System integration	3	3	3	5	3
Mode shift potential	5	3	3	1	3
Timing of benefits	5	1	1	1	1
Vehicle operating cost reduction	5	4	3	3	3
Accident cost reduction	5	5	5	5	5
ECONOMIC DEVELOPMENT ACCOUNT developable land accessed	5	3	1	1	1
interior interaction	2	2	2	2	2
generated travel	3	3	3	5	3
SOCIAL ACCOUNT urban land use impact	3	3	3	2	3
park/recreation impact	4	4	3	1	3
First Nations impact	4	4	3	1	3
consistency with regional growth plans	5	3	3	1	3
emergency route	1	2	3	5	4
ENVIRONMENTAL ACCOUNT watershed impact	5	1	1	3	3
geotechnical concern	3	5	1	1	1
physical env. impact	3	2	2	1	2
avalanche concern	5	4	2	1	1
archaeology	5	3	2	1	2
TOTAL	94	78	65	51	69

Source: (Ministry of Transportation 2001,12).

An Example of A Multi-Goal Valuation Matrix

Sample Goals/Criteria For Metachoice Alternatives