Public Policy Analysis by William Dunn

Public Policy Analysis
Dunn

Fifth Edition

www.irpublicpolicy.ir

Pearson Education Limited
Edinburgh Gate
Harlow
Essex CM20 2JE
England and Associated Companies throughout the world
Visit us on the World Wide Web at: www.pearsoned.co.uk
© Pearson Education Limited 2014

All rights reserved. No part of this publication may be reproduced, stored in a retrieval system, or transmitted
in any form or by any means, electronic, mechanical, photocopying, recording or otherwise, without either the
prior written permission of the publisher or a licence permitting restricted copying in the United Kingdom
issued by the Copyright Licensing Agency Ltd, Saffron House, 6–10 Kirby Street, London EC1N 8TS.
All trademarks used herein are the property of their respective owners. The use of any trademark
in this text does not vest in the author or publisher any trademark ownership rights in such
trademarks, nor does the use of such trademarks imply any affiliation with or endorsement of this
book by such owners.

ISBN 10: 1-292-02021-0
ISBN 13: 978-1-292-02021-1

British Library Cataloguing-in-Publication Data
A catalogue record for this book is available from the British Library

Printed in the United States of America

www.irpublicpolicy.ir

PEARSON CUSTOM LIBRA RY

Table of Contents

1. The Process of Policy Analysis 1
31
William N. Dunn 65
117
2. Policy Analysis in the Policy-Making Process 189
247
William N. Dunn 311
339
3. Structuring Policy Problems 383
425
William N. Dunn 433
441
4. Forecasting Expected Policy Outcomes 451

William N. Dunn I

5. Prescribing Preferred Policies

William N. Dunn

6. Monitoring Observed Policy Outcomes

William N. Dunn

7. Evaluating Policy Performance

William N. Dunn

8. Developing Policy Arguments

William N. Dunn

9. Communicating Policy Analysis

William N. Dunn

10. Appendix: The Policy Issue Paper

William N. Dunn

11. Appendix: The Policy Memorandum

William N. Dunn

12. Appendix: Planning Oral Briefings

William N. Dunn
Index

www.irpublicpolicy.ir

II www.irpublicpolicy.ir

The Process of Policy
Analysis

From Chapter 1 of Public Policy Analysis, Fifth Edition. William N. Dunn. Copyright © 2012 by 1
Pearson Education, Inc. All rights reserved.

www.irpublicpolicy.ir

The Process of Policy
Analysis

OBJECTIVES

By studying this chapter, you should be able to

᭿ Define and illustrate phases of ᭿ Distinguish prospective and
policy analysis. retrospective policy analysis.

᭿ Describe elements of integrated ᭿ Describe the structure of a policy
policy analysis. argument and its elements.

᭿ Distinguish four strategies of policy ᭿ Understand the role of argument
analysis. mapping in critical thinking.

᭿ Contrast reconstructed logic and ᭿ Interpret scorecards, spreadsheets,
logic-in-use. influence diagrams, decision trees,
and argument maps.

Policy analysis is a process of multidisciplinary inquiry aiming at the
creation, critical assessment, and communication of policy-relevant infor-
mation. As a problem-solving discipline, it draws on social science methods,
theories, and substantive findings to solve practical problems.1

1For a sample of alternative definitions see Harold D. Lasswell, A Pre-view of Policy Sciences
(New York: American Elsevier Publishing, 1971); Yehezkel Dror, Ventures in Policy Sciences: Concepts
and Applications (New York: American Elsevier Publishing, 1971); Edward S. Quade, Analysis for
Public Decisions, 3d rev. ed., ed. Grace M. Carter (New York: North Holland Publishing, 1989); David
L. Weimer and Aidan R. Vining, Policy Analysis: Concepts and Practice, 2d ed. (Englewood Cliffs, NJ:
Prentice Hall, Inc., 1992); Duncan Mac Rae Jr., The Social Function of Social Science (New Haven, CT:
Yale University Press, 1976).

2 www.irpublicpolicy.ir

The Process of Policy Analysis 3

METHODOLOGY OF POLICY ANALYSIS

As used here, the word methodology refers to a process of reasoned inquiry
aimed at finding solutions to practical problems. The aim of methodology is to
help us understand not only the products of policy inquiry but also the processes
employed to create these products.2 The methodology of policy analysis is not
confined to the analytical routines of specialized social science fields—for
example, benefit-cost analysis in economics or implementation analysis in
political science—because none of these holds a privileged place in policy inquiry.
Nor is the methodology of policy analysis constrained by the doctrines and
principles of obsolescent philosophies of science such as logical positivism, which
mistakenly claimed that scientific knowledge, properly understood, is objective,
value free, and quantitative.3 On the contrary, policy analysis is methodologically
eclectic; its practitioners are free to choose among a wide range of scientific
methods, qualitative as well as quantitative, as long as these yield reliable
knowledge. In this context, policy analysis includes art, craft, and reasoned
persuasion, all of which are scientific to the extent that they succeed in producing
reliable knowledge.4 Ordinary commonsense knowing and well-winnowed
practical wisdom—both products of evolutionary learning across generations of

2Abraham Kaplan, The Conduct of Inquiry: Methodology for Behavioral Science (San Francisco, CA:
Chandler Publishing Company, 1964), pp. 23–24.
3Logical positivism (or logical empiricism) was abandoned by most philosophers of science more than
50 years ago, although its epistemological pillars—the correspondence theory of truth, the empirical
criterion of meaning, and quantificationism—are still venerated by many social scientists. For alterna-
tives to logical positivism in economics and political science see Daniel Bromley, Sufficient Reason:
Volitional Pragmatism and the Meaning of Economic Institutions (Princeton, NJ: Princeton
University Press, 2006); Henry E. Brady and David Collier, eds. Rethinking Social Inquiry: Diverse
Tools, Shared Standards (Lanham, MD: Rowman Littlefield, 2004); Deirdre N. McCloskey, The
Rhetoric of Economics, 2nd ed., Madison: University of Wisconsin Press, 1998; Stephen Thomas
Ziliak and Deirdre N. McCloskey, The Cult of Statistical Significance: How the Standard Error Costs
Us Jobs, Justice, and Lives. Ann Arbor: University of Michigan Press, 2008. Paul Diesing, How Does
Social Science Work? Reflections on Practice (Pittsburgh, PA: University of Pittsburgh Press, 1991);
and Mary Hawkesworth, Theoretical Issues in Policy Analysis (Albany: State University of New York
Press, 1988).
4Larry Laudan has argued that the demarcation between science and non-science, including art and
craft, is a pseudo-problem that should be replaced by focusing on the distinction between reliable and
unreliable knowledge. It is not necessary to ask whether knowledge is “scientific,” only whether it is
reliable. “The Demise of the Demarcation Problem,” in R.S. Cohen and L. Laudan, Physics, Philosophy
and Psychoanalysis: Essays in Honor of Adolf Grünbaum. Boston Studies in the Philosophy of Science,
Vol.76 (Dordrecht: D. Reidel, 1983), pp. 111–127. Aaron Wildavsky and others have used the terms art
and craft to characterize policy analysis. See Aaron Wildavsky, Speaking Truth to Power: The Art and
Craft of Policy Analysis (Boston, MA: Little Brown, 1979); and Iris Geva-May and Aaron Wildavsky,
An Operational Approach to Policy Analysis: The Craft, Prescriptions for Better Analysis (Boston, MA:
Kluwer, 1997). The term policy science(s) is Harold Lasswell’s. See the short methodological history of
the policy sciences in Ronald Brunner, “The Policy Movement as a Policy Problem,” in Advances in
Policy Studies since 1950, vol. 10, Policy Studies Review Annual, ed. W. N. Dunn and R. M. Kelly
(New Brunswick, NJ: Transaction Books, 1992), pp. 155–97 and contributions to Michael Moran,
Martin Rein, and Robert E. Goodin, eds. The Oxford Handbook of Public Policy (Oxford: Oxford
University Press, 2006).

www.irpublicpolicy.ir

The Process of Policy Analysis

problem solvers—often permit conclusions that are more trustworthy and
reliable than those produced by means of policy analysis and other specialized
forms of professional and scientific inquiry.5

The rationale for policy analysis is pragmatic. For this reason, it is unmistakably
different from social science disciplines that prize knowledge for its own sake. The
policy-relevance of these disciplines depends not on their status as sciences but on
the extent to which they are successful in illuminating and alleviating practical
problems, problems that come in complex bundles that are at once economic,
political, cultural, ethical, and more. Practical problems do not arrive in separate
disciplinary packages addressed to departments of economics and political
science—to name two of the most important policy disciplines. In today’s world,
multidisciplinary policy analysis seems to provide the best fit with the manifold
complexity of public policy making.

POLICY ANALYSIS—A MULTIDISCIPLINARY
FRAMEWORK

Policy analysis is partly descriptive. It relies on traditional social science disciplines
to describe and explain the causes and consequences of policies. But it is also
normative, a term that refers to value judgments about what ought to be, in con-
trast to descriptive statements about what is.6 To investigate problems of efficiency
and fairness, policy analysis draws on normative economics and decision analysis
as well as ethics and other branches of social and political philosophy—all of which
are about what ought to be. This normative orientation stems from the fact that
analyzing policies demands that we choose among desired consequences (ends) and
preferred courses of action (means). The choice of ends and means requires contin-
uing trade-offs among competing values of efficiency, equity, security, liberty, and
democracy.7 The importance of normative reasoning in policy analysis was well
stated by a former undersecretary in the Department of Housing and Urban
Development: “Our problem is not to do what is right. Our problem is to know
what is right.”8

5On the contrasts between scientific and professional knowledge on one hand, and ordinary
commonsense knowing on the other, see Charles E. Lindblom and David K. Cohen, Usable
Knowledge: Social Science and Social Problem Solving (New Haven, CT: Yale University Press,
1979). On the frequent soundness of evolved practical knowledge—but the periodic need for
supplemental scientific testing—see Donald T. Campbell, “Evolutionary Epistemology,” in
Methodology and Epistemology for Social Science: Selected Papers, ed. E. S. Overman (Chicago:
University of Chicago Press, 1989).
6One classic statement of the difference between positive and normative knowledge in economics is
Milton Friedman, Essays in Positive Economics (Chicago, IL: University of Chicago Press, 1953). This
same positive-normative distinction is present throughout the social sciences.
7Deborah Stone, Policy Paradox: The Art of Political Decision Making, rev ed. (New York: W. W.
Norton, 2001).
8Robert C. Wood, “Foreword” to The Study of Policy Formation, ed. Raymond A. Bauer and Kenneth
J. Gergen (New York: Free Press, 1968), p. v. Wood is quoting President Lyndon Johnson.

4 www.irpublicpolicy.ir

The Process of Policy Analysis 5

Policy-Relevant Information

Policy analysis is designed to provide policy-relevant information about five types of
questions:

᭿ Policy problems. What is the problem for which a potential solution is sought?
Is global warming a human-made consequence of aircraft and motor vehicle
emissions? Or is global warming a consequence of periodic fluctuations in the
temperature of the atmosphere? What alternatives are available to mitigate
global warming? What are the potential outcomes of these alternatives and what
is their value or utility?

᭿ Expected policy outcomes. What are the expected outcomes of policies designed
to reduce harmful emissions? Because periodic natural fluctuations are difficult or
impossible to control, what is the likelihood that emissions can be reduced by
raising the price of gasoline and diesel fuel, compared with requiring that aircraft
and motor vehicles use biofuels?

᭿ Preferred policies. Which policies should be chosen, considering not only
their expected outcomes in reducing harmful emissions, but also the value of
reduced emissions in terms of economic costs and benefits? Should distribu-
tional criteria involving environmental justice be used along with criteria of
economic efficiency?

᭿ Observed policy outcomes. What policy outcomes are observed, as distin-
guished from the outcomes expected before a preferred policy is implemented?
Did the preferred policy actually result in reduced emissions? Were other
factors such as political opposition to governmental regulation responsible for
the limited achievement of emissions targets?

᭿ Policy performance. To what extent do observed policy outcomes con-
tribute to the reduction of global warming through emissions controls? What
are the benefits and costs of government regulation to present and future
generations?

Answers to these questions yield five types of information, which are policy-
informational components. These components are shown as rectangles in
Figure 1.9

A policy problem is an unrealized need, value, or opportunity for improvement
attainable through public action.10 Knowledge of what problem to solve requires
information about a problem’s antecedent conditions (e.g., school dropouts as an
antecedent condition of unemployment), as well as information about values (e.g.,
safe schools or a living wage) whose achievement may lead to the problem’s
solution. Information about policy problems plays a critical role in policy analysis,

9The framework was originally suggested by Walter Wallace, The Logic of Science in Sociology
(Chicago: Aldine Books, 1971). Wallace’s framework addresses research methodology in sociology,
whereas Figure 1 addresses the methodology of policy analysis.
10Compare James A. Anderson, Public Policymaking: An Introduction, 7th ed. (Boston, MA:
Wadsworth, 2011); Charles O. Jones, An Introduction to the Study of Public Policy, 2d ed. (North
Scituate, MA: Duxbury Press, 1977), p. 15; and David Dery, Problem Definition in Policy Analysis
(Lawrence: University of Kansas Press, 1984).

www.irpublicpolicy.ir

The Process of Policy Analysis

Problem POLICY Forecasting
Structuring PROBLEMS
EXPECTED
Practical OUTCOMES
Inference

OBSERVED POLICY
OUTCOMES PERFORMANCE

Evaluation

Monitoring Prescription

PREFERRED
POLICIES

FIGURE 1
The process of integrated analysis

because the way a problem is defined shapes the search for available solutions.
Inadequate or faulty information may result in a fatal error: defining the wrong
problem.11

Expected policy outcomes are likely consequences of one or more policy alterna-
tives designed to solve a problem. Information about the circumstances that gave rise
to a problem is essential for producing information about expected policy outcomes.
Such information is often insufficient, however, because the past does not repeat itself
completely, and the values that shape behavior may change in the future. For this
reason, information about expected policy outcomes is not “given” by the existing
situation. To produce such information may require creativity, insight, and the use of
tacit knowledge.12

11Defining the wrong problem is a type III error, as contrasted with type I and type II errors committed
when the level of statistical significance (alpha) is set too high or too low in testing the null hypothesis.
An early statement of this contrast is Ian I. Mitroff and Thomas R. Featheringham, “On Systematic
Problem Solving and the Error of the Third Kind,” Behavioral Sciences 19, no. 6 (1974): 383–93.
12Dror, Ventures in Policy Sciences; Sir Geoffrey Vickers, The Art of Judgment: A Study of Policy
Making (New York: Basic Books, 1965); and C. West Churchman, The Design of Inquiring Systems;
Basic Concepts of Systems and Organization (New York: Basic Books, 1971).

6 www.irpublicpolicy.ir

The Process of Policy Analysis 7

A preferred policy is a potential solution to a problem. To select a preferred policy,
it is necessary to have information about expected policy outcomes as well as informa-
tion about the value or utility of these expected outcomes. Another way to say this is
that factual as well as value premises are required for policy prescriptions. Fact alone—
for example, the fact that one policy produces more of some quantity than another—do
not justify the choice of a preferred policy. Factual premises must be joined with value
premises involving efficiency, equality, security, democracy, or some other value.

An observed policy outcome is a present or past consequence of implementing a
preferred policy. It is sometimes unclear whether an outcome is actually an effect of
a policy, because some effects are not policy outcomes; many outcomes are the result
of other, extra-policy factors. It is important to recognize that the consequences of
action cannot be fully stated or known in advance, which means that many conse-
quences are neither anticipated nor intended. Fortunately, information about such
consequences can be produced ex post (after policies have been implemented), not
only ex ante (before policies are implemented).

Policy performance is the degree to which an observed policy outcome con-
tributes to the solution of a problem. In practice, policy performance is never perfect.
Problems are rarely “solved”; most often, problems are resolved, reformulated, and
even “unsolved.”13 To know whether a problem has been solved, resolved, reformu-
lated, or unsolved requires information about observed policy outcomes, as well as
information about the extent to which these outcomes contribute to the opportunities
for improvement that gave rise to a problem.

Policy-Informational Transformations

The five types of policy-relevant information are interdependent. The arrows connect-
ing each pair of components represent policy-informational transformations, whereby
one type of information is changed into another, so that the creation of information at
any point depends on information produced in an adjacent phase. Information about
policy performance, for example, depends on the transformation of prior information
about observed policy outcomes. The reason for this dependence is that any assess-
ment of how well a policy achieves its objectives assumes that we already have reliable
information about the outcomes of that policy. The other types of policy-relevant
information are dependent in the same way.

Information about policy problems is a special case. Information about policy
problems usually includes some problem elements—for example, potential solutions
or expected outcomes—and excludes others. What is included or excluded affects
which policies are eventually prescribed, which values are appropriate as criteria of
policy performance, and which potentially predictable outcomes warrant or do not
warrant attention. At the risk of being overly repetitious, it is worth stressing again
that a fatal error of policy analysis is a type III error—defining the wrong problem.14

13Russell L. Ackoff, “Beyond Problem Solving,” General Systems 19 (1974): 237–39.
14Type I and type II errors are also known as false positives and false negatives. Other sources on type III
errors include A. W. Kimball, “Errors of the Third Kind in Statistical Consulting,” Journal of the American
Statistical Association 52 (1957): 133–42; Howard Raiffa, Decision Analysis (Reading, MA: Addison-
Wesley, 1968), p. 264; and Ian I. Mitroff, The Subjective Side of Science (New York: Elsevier, 1974).

www.irpublicpolicy.ir

The Process of Policy Analysis

Policy-Analytic Methods

The five types of policy-relevant information are produced and transformed by
using policy-analytic methods. All methods involve judgments of different
kinds:15 judgments to accept or reject an explanation, to affirm or dispute the
rightness of an action, to prescribe or not prescribe a policy, to accept or reject a
prediction, and to formulate a problem in one way rather than another.

In policy analysis, these procedures have special names:

᭿ Problem structuring. Problem-structuring methods are employed to produce
information about which problem to solve. One example of problem-structuring
methods is the influence diagram and decision tree presented in Case 3 of this
chapter (The Influence Diagram and Decision Tree—Structuring Problems of
Energy Policy and International Security). Other examples of problem-structuring
methods include critical thinking tools such as argument mapping (Case 4:
The Argument Map—Problem Structuring in National Defense and Energy
Policy).

᭿ Forecasting. Forecasting methods are used to produce information about
expected policy outcomes. An example of a simple forecasting tool is the
scorecard described in Case 1 (The Goeller Scorecard—Monitoring and
Forecasting Technological Impacts). Scorecards, which are based on the
judgments of experts, are particularly useful in identifying expected
outcomes of science and technology policies.

᭿ Prescription. Methods of prescription are employed to create information
about preferred policies. An example of a prescriptive method is the spread-
sheet (Case 2: The Spreadsheet—Evaluating the Benefits and Costs of Energy
Policies). The spreadsheet goes beyond the identification of expected policy
outcomes by expressing consequences in terms of monetary benefits and
costs.

᭿ Monitoring. Methods of monitoring are employed to produce information
about observed policy outcomes. The scorecard (Case 1) is a simple method
for monitoring observed policy outcomes as well as for forecasting expected
policy outcomes.

᭿ Evaluation. Evaluation methods are used to produce information about the
value or utility of observed policy outcomes and their contributions to policy
performance. The spreadsheet (Case 2) may be used for evaluation as well as
prescription.

The first method, problem structuring, is about the other methods. For this reason, it
is a metamethod (method of methods). In the course of structuring a problem, analysts
typically experience a “troubled, perplexed, trying situation, where the difficulty is, as

15John O’Shaughnessy, Inquiry and Decision (London: George Allen & Unwin, 1972).

8 www.irpublicpolicy.ir

The Process of Policy Analysis 9

it were, spread throughout the entire situation, infecting it as a whole.”16 Problem
situations are not problems; problems are representations of problem situations.
Hence, problems are not “out there” in the world, but they stem from the interaction
of thought and external environments. Imagine a graph showing the growth of
defense expenditures as a percentage of gross domestic product. The graph represents
a problem situation, not a problem, because one analyst will see the graph as evidence
of increasing national security (more of the budget is allocated to defense), while
another interprets the graph as an indication of a declining budget for social welfare
(less of the budget can be allocated to social services). Problem structuring, a proce-
dure for testing different representations of a problem situation, is the central
guidance system of policy analysis.

Policy-analytic methods are interdependent. It is not possible to use one
method without first having used others. Thus, although it is possible to monitor
past policies without forecasting their future consequences, it is usually not possi-
ble to forecast policies without first monitoring them.17 Similarly, analysts can
monitor policy outcomes without evaluating them, but it is not possible to evaluate
an outcome without first establishing that it is an outcome in the first place. Finally,
to select a preferred policy requires that analysts have already monitored,
evaluated, and forecasted outcomes.18 This is yet one more way of saying that
policy prescription is based on factual as well as value premises.

Figure 1 supplied a framework for integrating methods from different policy-
relevant disciplines. Some methods are used solely or primarily in some disciplines,
and not others. Program evaluation, for example, employs monitoring to investigate
whether a policy is causally relevant to an observed policy outcome. Although
program evaluation has made extensive use of interrupted time-series analysis,
regression discontinuity analysis, causal modeling, and other techniques associated
with the design and analysis of field experiments,19 implementation research within
political science has not. Instead, implementation researchers have relied mainly on
techniques of case study analysis.20 Another example comes from forecasting.
Although forecasting is central to both economics and systems analysis, economics
has drawn almost exclusively on econometric techniques. Systems analysis has made
greater use of qualitative forecasting techniques for synthesizing expert judgment,
for example, the Delphi technique.21

16John Dewey, How We Think (Boston, MA: D.C. Heath and Company, 1933), p. 108. The original
statement of the difference between a problem and a problem situation is attributable to philosophical
pragmatists including Charles Sanders Peirce.
17An exception is predictions made on the basis of expert judgment. The explanation of a policy is not
necessary for predicting its future consequences. Strictly speaking, a prediction is a causal inference,
whereas a projection, extrapolation, or “rational forecast” is not.
18Causation may be assumed but not understood. Recipes claim only that a desired result is a conse-
quence of action. Joseph L. Bower, “Descriptive Decision Theory from the ‘Administrative’ Viewpoint,”
in The Study of Policy Formation, ed. Bauer and Gergen, p. 10.
19See, for example, William R. Shadish, Thomas D. Cook, and Donald T. Campbell, Experimental and
Quasi-Experimental Designs for Generalized Causal Inference (Boston, MA: Houghton Mifflin, 2002).
20Paul A. Sabatier and Hank C. Jenkins-Smith, “The Advocacy Coalition Framework: An Assessment,”
in Theories of the Policy Process, ed. P. A. Sabatier (Boulder, CO: Westview Press, 1999), pp. 117–66.
21See the chapter “Prescribing Preferred Policies.”

www.irpublicpolicy.ir

The Process of Policy Analysis

FOUR STRATEGIES OF ANALYSIS

Relationships among policy-informational components, policy-analytic methods,
and policy-informational transformations provide a basis for contrasting four
strategies of policy analysis (Figure 2).

Prospective and Retrospective Analysis

Prospective policy analysis involves the production and transformation of informa-
tion before policy actions are taken. This strategy of ex ante analysis, shown as the
right half of Figure 2, typifies the operating styles of economists, systems analysts,
operations researchers, and decision analysts.

The prospective strategy is what Williams means by policy analysis.22 Policy
analysis is “a means of synthesizing information to draw from it policy alternatives
and preferences stated in comparable, predicted quantitative and qualitative terms
as a basis or guide for policy decisions; conceptually, it does not include the
gathering of information [emphasis in original].” Policy research, by contrast,

RETROSPECTIVE (ex post): POLICY PROSPECTIVE (ex ante):
What happened and what PROBLEMS What will happen and
difference does it make? what should be done?

Problem Practical Forecasting
Structuring Inference
EXPECTED
PROBLEM FINDING: POLICY OUTCOMES
What problem PERFORMANCE
should be solved?

OBSERVED
OUTCOMES

PROBLEM SOLVING: Evaluation
What is the solution
to the problem? Prescription

Monitoring

PREFERRED
POLICIES

FIGURE 2
Forms strategies of policy analysis

22Walter Williams, Social Policy Research and Analysis: The Experience in the Federal Social Agencies
(New York: American Elsevier, 1971), p. 8.

10 www.irpublicpolicy.ir

The Process of Policy Analysis 11

refers to “all studies using scientific methodologies to describe phenomena and/or
determine relationships among them.” Prospective analysis often creates wide gaps
between preferred solutions and actual efforts to implement them. Perhaps no more
than 10 percent of the work actually required to achieve a desired set of policy
outcomes is carried out before policies are implemented: “It is not that we have too
many good analytic solutions to problems. It is, rather, that we have more good
solutions than we have appropriate actions.”23

Retrospective policy analysis is displayed as the left half of Figure 2. This strat-
egy of ex post analysis involves the production and transformation of information
after policies have been implemented. Retrospective analysis characterizes the oper-
ating styles of three groups of analysts:

᭿ Discipline-oriented analysts. This group, composed mainly of political
scientists, economists, and sociologists, seeks to develop and test discipline-
based theories that describe the causes and consequences of policies. This
group is not concerned with the identification of specific policy goals or with
distinctions between “policy” variables that are subject to policy manipula-
tion and those that are not.24 For example, the analysis of the effects of party
competition on government expenditures provides no information about
specific policy goals; nor is party competition a variable that policy makers
can manipulate to change public expenditures.

᭿ Problem-oriented analysts. This group, again composed mainly of political
scientists, economists, and sociologists, seeks to describe the causes and conse-
quences of policies. Problem-oriented analysts, however, are less concerned with
the development and testing of theories believed to be important in social science
disciplines than with identifying variables that may explain a problem. Problem-
oriented analysts are not overly concerned with specific goals and objectives,
primarily because the practical problems they analyze are usually general in
nature. For example, the analysis of aggregate data on the effects of gender,
ethnicity, and social inequality on national achievement test scores provides
information that helps explain a problem (e.g., inadequate test performance) but
does not provide information about policy variables that can be manipulated.

᭿ Applications-oriented analysts. A third group includes applied economists,
applied sociologists, applied psychologists, and applied anthropologists, as
well as analysts from professions such as public administration, social work,
and evaluation research. This group also seeks to describe the causes and
consequences of public policies and programs and is not concerned with the
development and testing of discipline-based theories. This group is concerned
not only with manipulable policy variables but also with the identification of
specific policy goals and objectives. Information about specific goals and
objectives provides a basis for monitoring and evaluating outcomes and

23Graham T. Allison, Essence of Decision: Explaining the Cuban Missile Crisis (Boston, MA: Little,
Brown, 1971), pp. 267–68.
24James S. Coleman, “Problems of Conceptualization and Measurement in Studying Policy Impacts,” in
Public Policy Evaluation, ed. Kenneth M. Dolbeare (Beverly Hills and London: Sage Publications,
1975), p. 25.

www.irpublicpolicy.ir

The Process of Policy Analysis

impacts of policies. For example, applications-oriented analysts may address
early childhood reading readiness programs that can be manipulated in order
to achieve higher scores on reading tests.

The operating styles of the three groups reflect their characteristic strengths
and limitations. Discipline-oriented as well as problem-oriented analysts seldom
produce information that is directly useful to policy makers. Even when problem-
oriented analysts investigate important problems such as educational opportunity,
energy conservation, crime control, or national security, the resultant information
is often macronegative. Macronegative information describes the basic (or “root”)
causes and consequences of policies, usually by employing aggregate data to show
why policies do not work. By contrast, micropositive information shows what
policies and programs do work under specified conditions.25 It is of little practical
value to policy makers to know that the crime rate is higher in urban than
rural areas, but it is practically important to know that a specific form of gun
control reduces the commission of serious crimes or that intensive police
patrolling is a deterrent.

Even when applications-oriented analysts provide micropositive information,
they may find it difficult to communicate with practitioners of ex ante policy
analysis, who in most cases are professional economists. In agency settings,
ex ante analysts, whose job it is to find optimally efficient solutions, often have
limited access to information about policy outcomes produced through retrospec-
tive analysis. For their part, practitioners of ex ante analysis often fail to specify
in sufficient detail the kinds of policy-relevant information that will be most
useful for monitoring, evaluating, and implementing their recommendations.
Often, the intended outcomes of a policy are so vague that “almost any evalua-
tion of it may be regarded as irrelevant because it missed the ‘problem’ toward
which the policy was directed.”26 Legislators, for example, usually formulate
problems in general terms in order to gain acceptance, forestall opposition, or
maintain neutrality.

Contrasts among the operating styles of policy analysts suggest that disci-
pline-oriented and problem-oriented analysis are inherently less useful than
applications-oriented analysis—that retrospective (ex post) analysis as a whole
is perhaps less effective in solving problems than prospective (ex ante) analysis.
Although this conclusion may have merit from the point of view of policy
makers who want advice on what actions to take, it overlooks several important
benefits of retrospective analysis. Retrospective analysis, whatever its shortcom-
ings, places primary emphasis on the results of action and is not content with
information about expected policy outcomes, as is the case with prospective
analysis. Discipline-oriented and problem-oriented analysis may offer new
frameworks for understanding policy-making processes, challenging conven-
tional formulations of problems, questioning social and economic myths, and
shaping the climate of opinion in a community or society. Retrospective analysis,

25Williams, Social Policy Research and Analysis, p. 8.
26Ibid. p. 13; and Alice Rivlin, Systematic Thinking for Social Action (Washington, DC: Brookings,
1971).

12 www.irpublicpolicy.ir

The Process of Policy Analysis 13

however, “has been most important in its impact on intellectual priorities and
understandings, and not nearly so effective in offering solutions for specific
political problems.”27

Descriptive and Normative Analysis

Figure 2 also captures another important contrast, the distinction between
descriptive and normative strategies of policy analysis. Descriptive policy analysis
parallels descriptive decision theory, which refers to a set of logically consistent
propositions that describe or explain action.28 Descriptive decision theories may be
tested against observations obtained through monitoring and forecasting.
Descriptive theories, models, and conceptual frameworks originate for the most part
in political science, sociology, and economics. The main function of these theories,
models, and frameworks is to explain, understand, and predict policies by identify-
ing patterns of causality. The principal function of approaches to monitoring such as
field experimentation is to establish the approximate validity of causal inferences
relating policies to their presumed outcomes.29 In Figure 2, the descriptive form of
policy analysis can be visualized as an axis moving from the lower left (monitoring)
to the upper right (forecasting).

Normative policy analysis parallels normative decision theory, which refers to a set
of logically consistent propositions that evaluate or prescribe action.30 In Figure 2,
the normative strategy of policy analysis can be visualized as an axis running from the
lower right (prescription) to upper left (evaluation). Different kinds of information are
required to test normative and descriptive decision theories. Methods of evaluation and
prescription provide information about policy performance and preferred policies, for
example, policies that have been or will be optimally efficient because benefits
outweigh costs or optimally equitable because those most in need are made better off.
One of the most important features of normative policy analysis is that its propositions
rest on disagreements about values such as efficiency, equity, responsiveness, liberty,
and security.

Problem Finding and Problem Solving

The upper and lower halves of Figure 2 provide another important distinction. The
upper half points to methods that are designed for problem finding, whereas the
lower designates methods for problem solving. The problem-finding strategy has to
do with the discovery of elements that go into the definition of problems, and not to
their solution. How well do we understand the problem? Who are the most impor-
tant stakeholders who affect and are affected by the problem? Have the appropriate
objectives been identified? Which alternatives are available to achieve objectives?

27Janet A. Weiss, “Using Social Science for Social Policy,” Policy Studies Journal 4, (Spring 1976): 237.
28Bower, “Descriptive Decision Theory,” p. 104.
29See Thomas D. Cook and Donald T. Campbell, Quasi-Experimentation: Design and Analysis Issues
for Field Settings (Boston, MA: Houghton Mifflin, 1979); Shadish, Cook, and Campbell, Experimental
and Quasi-Experimental Designs for Generalized Causal Inference.
30Bower, “Descriptive Decision Theory,” pp. 104–05.

www.irpublicpolicy.ir

The Process of Policy Analysis

Which uncertain events should be taken into account? Are we solving the “right”
problem rather than the “wrong” one?

Problem-solving methods, located in the lower half of Figure 2, are designed to
solve rather than find problems. The problem-solving strategy is primarily technical
in nature, in contrast to problem finding, which is more conceptual. Problem-solving
methods such as econometrics are useful in answering questions about policy causa-
tion, statistical estimation, and optimization. How much of the variance in a policy
outcome is explained by one or more independent variables? What is the probability
of obtaining a coefficient as large as that obtained? Another problem-solving method
is benefit-cost analysis. What are the net benefits of different policies? What is their
expected utility or payoff?

Segmented and Integrated Analysis

Integrated policy analysis links the four strategies of analysis displayed in Figure 2.
Retrospective and prospective strategies are joined in one continuous process.
Descriptive and normative strategies are also linked, as are methods designed to find
as well as solve problems. Practically speaking, this means that policy analysts bridge
the several main pillars of multidisciplinary policy analysis, especially economics and
political science. Today, this need is not being properly met by specialized social
science disciplines, which tend to practice segmented policy analysis. The job of
bridging segmented disciplines—to convert intellectual knowledge into practical
knowledge—is carried out by multidisciplinary professions including public adminis-
tration, planning, management, and policy analysis. The American Society for Public
Administration (ASPA), the National Association of Schools of Public Affairs and
Administration (NASPAA), the American Planning Association (APA), the
International Association of Schools and Institutes of Administration (IASIA), the
Academy of Management (AM), the Operations Research Society of America
(ORSA), and the Association for Public Policy and Management (APPAM) are
organizations that represent these professions. So far, these professions have been
more open to the disciplines of economics and political science than those disciplines
have been open to them, notwithstanding a consensus among policy scholars and
practitioners that the substance and methods of these and other disciplines are
essential for producing policy-relevant information.

In summary, the framework for integrated policy analysis (Figure 1) helps
examine the assumptions, strengths, and limitations of methods employed in disci-
plines that tend to be overly segmented and excessively specialized to be useful in
practical problem solving. The framework identifies and relates major elements of
policy analysis—policy-informational components, policy-analytic methods, and
policy-informational transformations—enabling us to see the particular roles
performed by methods of problem structuring, monitoring, evaluation, forecasting,
and prescription. The framework (Figure 2) identifies different strategies of policy
analysis: prospective (ex ante) and retrospective (ex post), descriptive and normative,
and problem finding and problem solving. The framework integrates these strategies
of analysis and explains why we have defined policy analysis as a problem-solving
discipline that links social science theories, methods, and substantive findings to solve
practical problems.

14 www.irpublicpolicy.ir

The Process of Policy Analysis 15

THE PRACTICE OF POLICY ANALYSIS

Reconstructed Logic versus Logic-in-Use

The process of integrated policy analysis is a logical reconstruction (reconstructed
logic). The process of actually doing policy analysis never completely conforms to this
reconstruction, because all logical reconstructions are abstract representations of
stylized practices endorsed by the scientific community.31 By contrast, the logic-in-use
of practicing analysts, as distinguished from the logical reconstruction of their use of
reason and evidence to solve practical problems, always varies from methodological
“best practices” due to personal characteristics of analysts, their professional social-
ization, and the institutional settings in which they work.

᭿ Cognitive styles. The personal cognitive styles of analysts predispose them
toward different modes of acquiring, interpreting, and using information.32
Corporations, nonprofit organizations, and public agencies such as the U.S.
Department of Corrections and the National Science Foundation use the
Myers-Briggs test as a training and personnel selection diagnostic.

᭿ Analytic roles. In agency settings, most analysts are largely insulated from
politics. As such, they are primarily “technicians.” Others perform roles that,
in addition to technical content, are political. These “politicians” are actively
committed to advancing the interests of political leaders or officials to whom
they report. Other activist analysts are “entrepreneurs” who seek greater
influence in policy making.33

᭿ Institutional incentive systems. Policy “think tanks” encourage different
orientations toward analysis, including the “humanistic-value-critical” and
the “scientific.”34 Institutional rewards and punishments affect the validity
of conclusions and recommendations.35

᭿ Institutional time constraints. Analysts working in governmental settings are
often subject to tight institutional time constraints (three to seven days is typical).
They work with much greater speed, and perhaps greater efficiency, than analysts
in academic settings or think tanks. Understandably, government analysts rarely
collect original data; nor do they employ complex and time-consuming
techniques.36

31On reconstructed logic and logic-in-use, see Kaplan, Conduct of Inquiry, pp. 3–11.
32Studies using the Myers-Briggs type indicator (Jungian personality types) suggest different cognitive
styles among scientists, managers, and analysts. References provided by the Myers and Briggs
Foundation at www.myersbriggs.org. See also Ian I. Mitroff and Ralph H. Kilmann, Methodological
Approaches to Social Science (San Francisco: Jossey-Bass, 1978).
33Arnold Meltsner, Policy Analysts in the Bureaucracy (Berkeley: University of California Press, 1976);
Robert A. Heineman, William T. Bluhm, Steven A. Peterson, and Edward N. Kearney, The World of the
Policy Analyst. Chatham, NJ: Chatham House, 1990.
34Pamela Doty, “Values in Policy Research,” in Values, Ethics, and the Practice of Policy Analysis, ed.
William N. Dunn (Lexington, MA: D.C. Heath, 1983).
35Donald T. Campbell, “Guidelines for Monitoring the Scientific Competence of Preventive Intervention
Research Centers: An Exercise in the Sociology of Scientific Validity,” Knowledge: Creation, Diffusion,
Utilization 8, no. 3 (1987): 389–430.
36See P. J. Cook and J. W. Vaupel, “What Policy Analysts Do: Three Research Styles,” Journal of Policy
Analysis and Management 4, no. 3 (1985): 427–28.

www.irpublicpolicy.ir

The Process of Policy Analysis

᭿ Professional socialization. The different disciplines and professions that make
up policy analysis socialize their members into different norms and values.
Analyses of published papers suggest that analysts employ formal-quantitative as
well as informal-narrative approaches, although sound policy recommendations
sometimes require formal-quantitative procedures.37

᭿ Multidisciplinary teamwork. Much of the analysis conducted in public
agencies is carried out by multidisciplinary teams. Some members have
primary responsibility for the particular types of analysis displayed in
Figure 2. Team members trained in economics and decision analysis are typ-
ically more qualified to perform prospective (ex ante) analysis, whereas
team members trained in applied sociology, applied political science, and
program evaluation are usually better at retrospective (ex post) analysis.
The effectiveness of teams depends on everyone acquiring an operational
understanding of analytic methods employed throughout the process of
integrated policy analysis.

Methodological Opportunity Costs

Integrated analysis has opportunity costs. Given limited time and resources, it is
difficult to conduct systematic economic, political, and organizational analyses
simultaneously. Multiple triangulation,38 or what Cook calls critical multiplism,39
responds to some of the inadequacies of logical positivism.40 Positivism now
appears as a one-sided methodology and epistemology claiming that true statements
about the world must be logically and empirically verifiable, expressed in a formal
(ideal) language such as mathematical statistics, and confirmed by means of state-
ments that correspond to objective reality. Objective reality, rather than a reality
constituted by subjective meaningful actions and institutions, is the foundation of
true statements. Logical positivism, as Cook argues, was the dominant methodology
of policy analysis and program evaluation during the era of President Lyndon
Johnson’s War on Poverty. The advantage of critical multiplism over logical
positivism is that multiplism provides a better approximation of what is true by
employing procedures that triangulate from a variety of perspectives on what is
worth knowing and what is known about policies.41

37An early but representative overview of approaches is Janet A. Schneider, Nancy J. Stevens, and Louis
G. Tornatzky, “Policy Research and Analysis: An Empirical Profile, 1975–1980,” Policy Sciences 15
(1982): 99–114.
38The methodology of triangulation is analogous to practices employed in geodesic surveys; cartography;
navigation; and, more recently, satellite tracking. The position or location of an object is found by means
of bearings from two or more fixed points or electronic signals a known distance apart.
39Cook advanced critical multiplism as an alternative to logical positivism. See Thomas D. Cook,
“Postpositivist Critical Multiplism,” in Social Science and Social Policy, ed. R. Lane Shotland and
Melvin M. Mark (Beverly Hills, CA: Sage Publications, 1985), pp. 21–62.
40A critical assessment of logical positivism is Mary E. Hawkesworth, “Epistemology and Policy
Analysis,” in Advances in Policy Studies since 1950, ed. Dunn and Kelly, pp. 293–328; and
Hawkesworth, Theoretical Issues in Policy Analysis.
41Cook, “Postpositivist Critical Multiplism,” p. 57.

16 www.irpublicpolicy.ir

The Process of Policy Analysis 17

A disadvantage of multiplism lies in its costs. Triangulation among multiple
disciplinary perspectives, along with the use of multiple methods, measures, and
data sources, involves trade-offs and opportunity costs.42 When single methods
such as econometric modeling are employed to achieve measurement precision
and statistical generalizability, analysts forgo opportunities to acquire a deeper
understanding of policies that is possible through ethnographic interviews, case
studies, and other qualitative methods. A leading econometrician, noting that
economists are unique among social scientists because they are trained only to
analyze data, not to collect it, observes that “empirical work can be greatly
enhanced by being sensitive to the context of the problem (the data-generating
process) and knowing a lot about one’s data.”43 Similar trade-offs apply to
methods of research synthesis, or meta-analysis, which purchase measurement
precision and generalized policy causation at the expense of a deeper understand-
ing of contexts of policy-making.44

Ethnographic interviews, by contrast, involve high information costs because
they require the collection of substantial primary data through interviews.
However, they also lack precision and seldom permit the generalization of policy
causation to other settings. Although greater precision and generalizability can be
obtained by means of field studies and field experiments, these are expensive,
especially when they are employed in conjunction with mixed (quantitative and
qualitative) methods. To be sure, triangulation among convergent (and divergent)
perspectives, methods, and measures may enhance the validity of policy analysis
and other applied social sciences.45 But the time and financial constraints make
trade-offs inevitable.

CRITICAL THINKING AND PUBLIC POLICY

The world of the policy analyst is complex. Analysts must sift through and evalu-
ate a large volume of available quantitative and qualitative data, make difficult
choices among sources of information, select appropriate methods and techniques,
and employ effective strategies for communicating the results of analysis through
oral briefings and documents. These practical challenges place a premium on
critical thinking—that is, the capacity to organize, synthesize, and evaluate diverse
sources of reasoning and evidence. One method available for this purpose is the

42See David Brinberg and Joseph E. McGrath, Validity and the Research Process (Beverly Hills, CA:
Sage Publications, 1985). For Brinberg and McGrath and other methodological pragmatists, the choice
of methods is similar to an optimization problem in decision analysis. See C. West Churchman,
Prediction and Optimal Decision: Philosophical Issues of a Science of Values (Englewood Cliffs, NJ:
Prentice Hall, 1961); and Russell Ackoff, Scientific Method: Optimizing Applied Research Decisions
(New York: John Wiley, 1962).
43Peter Kennedy, A Guide to Econometrics, 4th ed. (Cambridge, MA: MIT Press, 1998), pp. 83–84.
44See Lawrence Rudner, Gene V. Glass, David L. Evartt, and Patrick J. Emery, A User’s Guide to the
Meta-Analysis of Research Studies. ERIC Clearinghouse on Assessment and Evaluation, University of
Maryland, College Park, 2002. http://echo.edres.org
45The case for triangulation in its many forms is found in Campbell, Methodology and Epistemology
for Social Science, ed. Overman.

www.irpublicpolicy.ir

The Process of Policy Analysis

analysis of policy arguments. By analyzing policy arguments, we are able to iden-
tify and probe the assumptions underlying competing policy claims, recognize and
evaluate objections to these claims, and synthesize policy-relevant information
from different sources.

The Structure of Policy Arguments

Policy arguments are the main vehicle carrying debates about public policies.46
Although social scientists may rightly pride themselves on methodological special-
ization, they too often forget that “public policy is made of language. Whether in
written or oral form, argument is central to all stages of the policy process.”47

The structure of a policy argument can be represented as a set of seven elements
(Figure 3):48

Policy claim (C). A policy claim is the conclusion of a policy argument.
Arguments also include other elements, including policy-relevant infor-
mation (I), warrants (W), backings (B), qualifiers (Q), objections (O),
and rebuttals (R). The movement from policy-relevant information to
claim implies therefore, thus, or so. Policy claims are of different types.
Some are normative: “Congress should pass the amendments to the
Fair Employment Practices Act.” Some are descriptive: “The use of the
Internet will double in the next ten years.”

Policy-relevant information (I). Policy-relevant information provides the
grounds for a policy claim. These grounds may be statistical data, experi-
mental findings, expert testimony, common sense, or political judgments.
Policy-relevant information is a response to the question: What information
is relevant to the claim? Information is the starting point of a new argument
and the end of a previous one. Policy arguments may lead to complex
argument chains, trees, or cycles.

Warrant (W). The warrant is a reason to support a claim. Warrants may be
economic theories, ethical principles, political ideas, professional authority,
and so forth.49 A warrant answers the question: Why does this reason
support the claim? Different types of warrants are related to arguments

46See Frank Fischer and John Forester, ed., The Argumentative Turn in Policy Analysis and Planning
(Durham, NC: Duke University Press, 1993). Earlier works on policy argumentation are Ian I. Mitroff
and Richard O. Mason, Creating a Dialectical Social Science (Boston: D. Reidel, 1981); William N.
Dunn, “Reforms as Arguments,” Knowledge: Creation, Diffusion, Utilization 3 (1982): 293–326;
Donald T. Campbell, “Experiments as Arguments,” Knowledge: Creation, Diffusion, Utilization 3
(1982): 327–47; Giandomenico Majone, Evidence, Argument, and Persuasion in the Policy Process
(New Haven, CT: Yale University Press, 1989); and Stone, Policy Paradox and Political Reason.
47Majone, Evidence, Argument, and Persuasion, p. 1.
48This structural model of argument is part of the computer software called Rationale 2, which was de-
veloped by Tim van Gelder and his colleagues in Australia. URL: www.austhink.com. The classic struc-
tural model is presented in Stephen Toulmin, The Uses of Argument (Cambridge: Cambridge University
Press, 1958); and Stephen Toulmin, A. Rieke, and A. Janik, An Introduction to Reasoning (New York:
Macmillan, 1984).
49Different kinds of warrants yield the “modes” of policy argument presented in the chapter
“Developing Policy Arguments.”

18 www.irpublicpolicy.ir

The Process of Policy Analysis

CLAIM
A policy claim is the
conclusion of a policy
argument. There are four
types of policy claims:
definitional, descriptive,
evaluative, and
advocative.

INFORMATION WARRANT QUALIFIER

Policy-relevant information provides A warrant is a reason to A qualifier expresses the approximate
taken-for-granted facts to support a support a policy claim. truth of a claim, considering the strength
policy claim. Policy-relevant Warrants may be economic of information, warrants, backings,
information may be statistical data, theories, ethical principles, objections, and rebuttals. Qualifiers may
experimental findings, expert political ideas, authority, and be stated statistically (p < 0.01) or in
testimony, common sense, or so forth. Most arguments have everyday language (“probably,” “not
political judgments. multiple warrants. likely,” “apparently,” “unlikely”).

OBJECTION BACKING OBJECTION
An objection opposes A backing justifies or
or challenges “backs up” a warrant An objection opposes or challenges
information by by providing good a qualifier by identifying special
identifying special reasons for believing conditions or exceptions which
conditions or the warrant. reduce confidence in the strength
exceptions that of the qualifier.
reduce confidence in OBJECTION
the truth of the An objection opposes or challenges
information. a backing by identifying special
conditions or exceptions that
REBUTTAL reduce confidence in the truth of
A rebuttal opposes or the backing.
challenges an
objection by
identifying special
conditions or
exceptions that
reduce confidence
in the truth of the
objection,

FIGURE 3
Elements of a policy argument

Source: Created with Rationale 2. Melbourne: Austhink Consulting, 2010. www.austhink.com

made in different disciplines and professions. For example, law uses case
comparisons and rules of evidence, whereas economics uses theories and
component laws such as the law of diminishing utility of money. Policy
makers as well as social scientists employ causal warrants such as “Ethnic
cleansing will be deterred by air strikes that establish NATO’s credibility in

www.irpublicpolicy.ir 19

The Process of Policy Analysis

the region.” The warrant, which provides a justification for accepting a
claim, answers the question: Considering the information, what reasons
make the claim true?
Qualifier(Q). The qualifier expresses the degree to which a claim is approxi-
mately true, given the strength of the information, warrants, and backings,
as well as objections and rebuttals. Although social scientists may state
qualifiers in the language of formal probability (p ϭ 0.01 or t ϭ 2.24),
ordinary language is the normal mode of qualifying claims with such terms
as certainly, absolutely, necessarily, probably, in all likelihood, presumably,
apparently, and barring unforeseen circumstances. The qualifier answers the
question: How strong or credible is the claim? It is primarily through
processes of argumentation and debate that policy makers, policy analysts,
and other policy stakeholders adjust or even abandon arguments. Such
changes, when they occur, are motivated by the strength of objections and
rebuttals offered by those who have a stake in policies.
Backing (B). The backing is an additional reason to support or “back up” the
warrant. The backing answers the question: Why does the warrant support
the claim? with a more general reason, assumption, or argument that begins
with because. Different kinds of backings are characteristically employed by
members of different disciplines and professions. Backings may be scientific
laws, appeals to the authority of experts, or ethical and moral principles.
For example, consider the warrant presented earlier: “Ethnic cleansing will
be deterred by air strikes that establish NATO’s credibility in the region.”
The backing for warrants advocating the use of coercive force is frequently
an informal statement of the law of diminishing utility: “The greater the
cost of an alternative, the less likely it will be pursued.”
Objection (O). An objection opposes or challenges the information, warrant,
backing, or qualifier by identifying special conditions or exceptions that
reduce confidence in the truth of the information, warrant, backing, or
qualifier. An objection answers the question: Are there special circum-
stances or exceptions that threaten the credibility of the warrant? Analysts
who pay attention to objections are more likely to take a critical perspec-
tive toward a policy argument, identifying weak or hidden assumptions,
anticipating unintended consequences, or questioning possible rebuttals to
objections. Thereby, analysts can be self-critical, challenging their own
assumptions and arguments.
Rebuttal(R). A rebuttal is an objection to an objection. Rebuttals oppose or
challenge objections by identifying special conditions or exceptions that
reduce confidence in the truth of the objection. Rebuttals answer the ques-
tion: Are there special circumstances or exceptions that threaten the cred-
ibility of the objection? Most policy arguments have objections and rebut-
tals, because policy making involves bargaining, negotiation, competition,
and compromise among opponents and proponents of policies.

The frames of reference, perspectives, and reasons of policy makers and analysts
are found in their underlying warrants, backings, objections, and rebuttals.
Therefore, identical policy-relevant information is interpreted in distinctly different

20 www.irpublicpolicy.ir

The Process of Policy Analysis

ways. A decrease in crime rates in urban areas may be welcomed by the urban poor,
viewed with skepticism by owners of central city businesses, rejected by criminolo-
gists who attribute urban crime rates to changes in unemployment and homelessness,
and hailed as an achievement by elected officials. By examining contending
arguments and their underlying assumptions, analysts can uncover and critically
assess reasoning and evidence that otherwise goes unnoticed. Equally important is
what it brings to analysts themselves—they can probe their own assumptions by
examining the objections, qualifications, and exceptions to their own conclusions.

CHAPTER SUMMARY multidisciplinary inquiry. No one methodol-
ogy is appropriate for all or most problems.
This chapter has provided a framework for Given the need to choose among methods,
policy analysis that identifies the role of methodological choices can be viewed as an
policy-analytic methods in creating and trans- optimization problem involving trade-offs and
forming policy-relevant information. The four opportunity costs. The actual work of practic-
regions of this framework call attention to ing analysts demands critical thinking. The
similarities and differences among methods of analysis of policy arguments is well suited for
policy analysis and point to the origins of these this purpose.
methods in different social science disciplines
and professions, thus clarifying the meaning of

REVIEW QUESTIONS 6. Contrast retrospective and prospective analy-
sis. Which social science disciplines tend to spe-
1. What does it mean to define policy analysis as cialize in prospective analysis? Retrospective
a process of inquiry as distinguished from a set analysis?
of methods?
7. Discuss the strengths and limitations of critical
2. Describe the dynamics of policy-informational multiplism.
components, policy-analytic methods, and
policy-informational transformations. 8. Contrast the “logic-in-use” and the “recon-
structed logic” of policy analysis. Provide
3. Contrast segmented and integrated policy examples.
analysis. Give examples.
9. How can argumentation mapping assist analysts
4. How does normative decision theory differ to become critical thinkers?
from descriptive decision theory?

5. List some of the key differences between prob-
lem solving and problem finding.

DEMONSTRATION EXERCISES about the differences between monitoring and
evaluation in policy analysis? What are the
1. Scorecards provide a useful overview of the implications for the distinctions among different
observed and expected outcomes of different types of policy analysis?
policies. When using the scorecard for monitor- 2. Influence diagrams and decision trees are useful
ing and forecasting the outcomes of the two methods for structuring policy problems. The
speed limits (Case 2), the 55 mph speed limit diagram and tree displayed in Figure 2 help
seems preferable to the 65 mph speed limit. identify policy stakeholders, policy alternatives,
Compare the scorecard (Figure C2) with the uncertain outcomes and events, probabilities of
spreadsheet (Figure C3). Does the comparison these outcomes and events, and valued
change your conclusions about the performance
of the 55 mph speed limit? What does this tell us

www.irpublicpolicy.ir 21

The Process of Policy Analysis

outcomes (valued outcomes are objectives). gram), describe why these two problem repre-
Consider the influence diagram that represents sentations are good examples of descriptive and
the problem of energy supply in 1973 and normative decision theory.
1974, when the OPEC oil embargo posed sig- 3. Create an argument map based on the influ-
nificant challenges both to U.S. energy supply ence diagram presented in Case 3. Begin with
and national security. How does Figure C3 help the following claim: “The United States should
us formulate the problem? What do the arrows return to the 55 mph speed limit in order to
suggest about the causes of the energy shortage conserve fuel and save lives.” Include in your
as well as the causes of the decline in traffic fa- map as many warrants, backings, objections,
talities? What does the influence diagram sug- and rebuttals as you can. Assuming that the
gest about the conditions that gave rise to the original qualifier was certainly, indicate
55 mph speed limit and its effectiveness? After whether the qualifier changes as we move from
comparing the influence diagram with the deci- a simple, static, uncontested argument to a
sion tree (which is based on the influence dia- complex, dynamic, and contested argument.

BIBLIOGRAPHY Kaplan, Abraham. The Conduct of Inquiry:
Methodology for Behavioral Science. San
Campbell, Donald T. Methodology and Epistemology Francisco, CA: Chandler, 1964.
for Social Science: Selected Papers. Edited by
E. Samuel Overman. Chicago: University of Mac Rae, Duncan Jr. The Social Function of Social
Chicago Press, 1988. Science. New Haven, CT: Yale University Press,
1976.
Diesing, Paul. How Social Science Works: Reflections
on Practice. Pittsburgh, PA: Pittsburgh University Stone, Deborah. Policy Paradox: The Art of
Press, 1991. Political Decision Making. Rev Ed. New York:
W. W. Norton, 2001.
Dunn, William N., and Rita Mae Kelly. Advances
in Policy Studies since 1950. New Brunswick, Toulmin, Stephen R. Return to Reason. Cambridge,
NJ: Transactions Books, 1992. MA: Harvard University Press, 2001.

Fischer, Frank, and John Forester. The Argumentative Van Gelder, Tim. “The Rationale for Rationale.”
Turn in Policy Analysis and Planning. Durham, Law, Probability, and Risk 6 (2007): 23–42.
NC: Duke University Press, 1993.

Hawkesworth, Mary E. Theoretical Issues in
Policy Analysis. Albany: State University of
New York Press, 1988.

CASE 1 THE GOELLER SCORECARD—
MONITORING AND FORECASTING
TECHNOLOGICAL IMPACTS

When advanced technologies are used to achieve the resources to do so. Given the time constraints
policy goals, sociotechnical systems of considerable of policy making, many analyses are completed in a
complexity is created. Although it is analytically period of several days to a month, and in most
tempting to prepare a comprehensive economic cases policy analyses do not involve the collection
analysis of the costs and benefits of such policies, and analysis of new data. Early on in a project,
most practicing analysts do not have the time or policy makers and their staffs typically want an

22 www.irpublicpolicy.ir

The Process of Policy Analysis

overview of the problem situation and the potential income groups. In this case, as Quade observes, the
impacts of alternative policies. Under these large number of diverse impacts are difficult to
circumstances, the scorecard is appropriate. value in dollar terms, making a benefit-cost analysis
impractical and even impossible.50 Other impacts
The Goeller scorecard, named after Bruce Goeller involve financial and economic questions such as
of the RAND Corporation, is appropriate for this investments, jobs created, sales, and tax revenues.
purpose. Table C1 shows the impacts of alternative Other impacts are distributional because they
transportation systems. Some of the impacts involve involve the differential effects of transportation. ᭿
transportation services used by members of the
community, whereas others involve impacts on low- CASE 2 THE CASE 3

TABLE C1
Scorecard

Social Impacts CTOL VTOL TACV

TRANSPORTATION

Passengers (million miles) 7 4 9

Per trip time (hours) 2 1.5 2.5

Per trip cost ($) $17 $28 $20

Reduced congestion (%) 0% 5% 10%

FINANCIAL $150 $200 $200
0 0 90
Investment ($ millions)
Annual subsidy ($ millions)

ECONOMIC 20 25 100
50 88 500
Added jobs (thousands)
Added sales ($millions)

COMMUNITY 10 1 20
3% 9% 1%
Noise (households) 0% –20% 30%
Added air pollution (%) 0 20 500
Petroleum savings (%) 0.2
Displaced households 0 None 2
Taxes lost ($millions) None Fort X
Landmarks destroyed

DISTRIBUTIONAL 7% 1% 20%
2% 16% 40%
Low-income trips (%)
Low-income household
Noise annoyance (%)

SOurce: Goeller (1974); Quade, Analysis for Public Decisions (1975), p. 60.

NOte: Conventional takeoff and landing aircraft (CTOL); vertical takeoff and landing aircraft (VTOL);
tracked air-cushion vehicle (TACV).

50E.S. Quade, Analysis for Public Decisions (New York: American Elsevier, 1975), p. 65.

www.irpublicpolicy.ir 23

CASE 2 SPREADSHEET—EVALUATING THE
BENEFITS AND COSTS OF ENERGY POLICIES

In 1972 and 1973, the United States and other additional 600 to 1,000 deaths. The Washington Post
petroleum-dependent countries experienced the and the New York Times joined the opposition,
first of several oil crises precipitated by a reporting that, although fatalities would surely rise,
dramatic increase in the price of crude oil by the the savings in time was trivial. Later, Secretary of
Organization of Petroleum Exporting Countries Transportation Pena announced that the Clinton
(OPEC). The response of American and European administration was firmly opposed to abandoning the
leaders was to adopt maximum speed limits of 55 speed limit.
mph and 90 kph, respectively. In the United
States, the National Maximum Speed Limit This was the right moment for an evaluation of
(NMSL) was designed to reduce the consumption the benefits and costs of the NMSL. A spreadsheet is
of gasoline by requiring that all vehicles on a simple but powerful tool for doing so. The
interstate highways travel at a maximum of 55 scorecard, as we saw in Case 1, is a useful tool for
mph, a speed that would maximize fuel efficiency monitoring and forecasting impacts when benefit-
for most vehicles. cost analysis is not feasible or desirable. On the
scorecard, policy alternatives are arrayed in columns
Soon after the implementation of the 55 mph along the top of the matrix and policy impacts are
speed limit, it was discovered that the new policy listed in each row. Spreadsheets, by contrast, are
not only reduced fuel consumption, but apparently appropriate and useful for prescribing preferred
caused a dramatic decline in traffic fatalities and policies and evaluating their outcomes. Spreadsheets
injuries as well. Therefore, long after the OPEC oil display the benefits and costs of observed or
crisis was over, the speed limit was retained, expected policy outcomes, creating information
although it was no longer needed to respond to the about policy performance as well as preferred
energy crisis that prompted its passage in 1973. policies (see Figure 1).
Indeed, the 55 mph speed limit was retained for
more than 20 years until it was officially repealed Table C2 displays a spreadsheet used to
in November 1995.51 evaluate the effects of the 55 mph speed limit
at the end of 1974, one year after the policy was
Heated debates preceded the repeal. Senator implemented. To show the differences between
John C. Danforth of Missouri, an influential the spreadsheet and the scorecard, Table C2
advocate of the policy, argued that the repeal would also displays the same information as a
save one minute per day per driver but result in an scorecard. ᭿

51On April 2, 1987, Congress enacted the Surface Transportation and Uniform Relocation Assistance
Act, permitting 40 states to experiment with speed limits up to 65 mph.

24 www.irpublicpolicy.ir

The Process of Policy Analysis

TABLE C2

Scorecard and Spreadsheet

(a) Scorecard

OUTCOMES 65 MPH 55 MPH
(Base Case)*
45,196
Fatalities 54,052 1,281
Miles traveled (billions) 1,313 21.9
Hours driving (billions) 20.2 43.3
Gallons fuel consumed (billions) 46.8 16.1
Fuel efficiency (mpg) 14.9 7,425
Traffic citations (millions) 5,711 23.1
Property damage (million cases) 25.8

*The base case is the policy against which the new policy is compared.

OBJECTIVES (b) Spreadsheet Value $ Billions
65MPH 55MPH Difference

I Fatalities (000s) 54.1 45.2 8.856 $240,000.00 $ 2.13
II Hours driving (billions) 20.2 21.9 Ϫ1.7 5.05 Ϫ8.59
III Gallons fuel consumed 46.8 43.3 0.53
3.5 1.86
(billions) 5,711 7,425
IV Traffic citations (000s) 25,800 23,100 Ϫ1,714 3.94 Ϫ0.0068
V Property damage 2,700
363.00 0.98
cases (000s)
Benefits (I ϩ III ϩ V) 4.97
Costs (II ϩ IV) Ϫ8.60
Net Benefits (B–C) $ Ϫ3.63

CASE 3 THE INFLUENCE DIAGRAM AND
DECISION TREE—STRUCTURING PROBLEMS OF
ENERGY POLICY AND INTERNATIONAL SECURITY

Along with other policy-analytic methods discussed diagram (Figure C3) displays the policy, the
National Maximum Speed Limit, as a rectangle.
earlier in this chapter (Figure 1), the influence A rectangle always refers to a policy choice or
diagram and decision tree are useful tools for decision node, which in this case is the choice
structuring policy problems.52 The influence

(continued)

52The diagram and tree were created with the Decision Programming Language (DPL), which is available from Syncopation
Software at http://www.syncopation.com. Educational, professional, and commercial versions of DPL 7.0 are available.

www.irpublicpolicy.ir 25

The Process of Policy Analysis

(a) Influence Diagram

Employment Travel
Time

Recession

OPEC
Oil Crisis

NMSL Miles Fuel Net
Traveled Used Benefits

Injuries

Fatalities

(b) Decision Tree

OPEC Renew Recession Miles Fatalities
Oil Crisis NMSL Traveled
Yes Down 20% (8,900 fatalities)
Yes Yes No Down 10% (32 billion miles) $240,000 per Fatality Averted
No No Down 10%
$2,77 per 100 Miles Traveled $240,000 per Fatality Averted
Down 3% Down 5%
$240,000 per Fatality Averted
$2.77 per 100 Miles Traveled
Unchanged

$2.77 per 100 Miles Traveled

FIGURE C3
Influence diagram and decision tree

between adopting and not adopting the national such as the recession and unemployment, which
maximum speed limit of 55 mph. To the right and affect miles driven, which in turn affect all four
above the decision node are uncertain events, objectives. The “root cause” appears to be the
represented as ovals, which are connected to the OPEC oil embargo.
decision node with arrows showing how the speed
limit affects or is affected by them. The rectangles The decision tree is another representation of the
with shaved corners represent valued policy influence diagram. Whereas the influence diagram
outcomes or objectives. The objectives are to lower shows how policy choices and uncertain events affect
fuel consumption, reduce travel time, reduce the achievement of objectives, the decision tree displays
injuries, and avert traffic fatalities. To the right of the monetary value of these objectives. In this abridged
the objectives is another shaved rectangle, which and simplified decision tree, there are two branches
designates the net benefits (benefits less costs) of that represent the alternatives but also the OPEC oil
the four objectives. The surprising result of using embargo, the recession, the costs of miles traveled, and
the influence diagram for problem structuring is the dollar benefits of reducing fatalities. The bolded
the discovery of causally relevant economic events, branches show the events with the greatest likelihood
of occurring or that already have occurred. ᭿

26 www.irpublicpolicy.ir

CASE 4 THE ARGUMENT MAP—PROBLEM
STRUCTURING IN NATIONAL DEFENSE AND
TRANSPORTATION POLICY

The role of causal arguments in transforming policy- proposition, or law, within the rational policy model.
relevant information into policy claims may be After the objection (O) has successfully challenged
illustrated by Allison’s well-known study of foreign the warrant, the qualifier (Q) changes from absolutely
policy decision making during the Cuban missile to doubtful.
crisis of October 1962.53 Showing how different
explanatory models yield different conclusions, Allison’s account shows how the use of multiple
Allison argues that government policy analysts think competing explanations can facilitate critical
about problems of foreign policy in terms of implicit thinking. The use of multiple competing models
conceptual models that shape their thought; most moves the analysis from a simple uncontested
analysts explain the behavior of governments in argument (Figure C4.1) to a new argument that is
terms of a model that assumes the rationality of complex, contested, and dynamic (Figure C4.2).This
political choices (rational actor model); alternative change occurs because a serious objection has been
models, including those that emphasize raised about the warrant and the backing of the
organizational processes (organizational process model) claim. The objection states: “But Soviet leaders may
and bureaucratic politics (bureaucratic politics model), fail to convince their naval units to depart from
provide a basis for improved explanations. established organizational routines.” The warrant
for this objection is: “The bulk of research on
In 1962, the policy alternatives open to the organizations shows that major lines of
United States ranged from no action and diplomatic organizational behavior tend to be straight. Behavior
pressure to secret negotiations, invasion, surgical air at time t+1 differs little from behavior at time t.55
strikes, and blockade. Among the several claims The blockade will not work.” The warrant for the
made at the time of the Cuban missile crisis, let us objection is again a general proposition or law
consider the policy actually adopted by the United within the organizational process model, otherwise
States: “The United States should blockade Cuba.” known as the disjointed incremental theory of policy
In this case, the policy-relevant information (I) is change.
“The Soviet Union is placing offensive missiles in
Cuba.” The warrant states that “the blockade will Simple uncontested maps of arguments about
force the withdrawal of missiles by showing the the 55 mph speed limit are presented alongside of
Russians that the United States is determined to use the arguments about the Cuban missile crisis
force.” In providing reasons to accept the warrant, (Figures C4.1a and C4.1b). The comparisons show
the backing (B) supports the warrant by stating that that the simple argument maps represent uncritical
“an increase in the cost of an alternative reduces the thinking. By contrast, the complex, dynamic, and
likelihood of that alternative being chosen.”54 The contested maps of the same crises (Figures C4.2a
backing (B) represents a general theoretical and C4.2b) illustrate what is meant by critical
thinking. ᭿

(continued)

53Graham T. Allison, “Conceptual Models and the Cuban Missile Crisis,” American Political Science

Review 3002, no. 3 (1969): 689–718.
54Allison, “Conceptual Models,” p. 694.
55Ibid., p. 702.

www.irpublicpolicy.ir 27

The Process of Policy Analysis

(a) The Cuban Missile Crisis—Simple Uncontested Argument
C
The United States should force
the Soviet Union to withdraw the
missiles by blockading Cuba.

I supports W Q supports
Reliable intelligence A blockade will show Soviet Very
reports confirm that the leaders that the States means probably
Soviet Union is placing business and is prepared to
offensive missiles in Cuba. use force.

(b) The 55 mph Speed Limit—Simple Uncontested Argument

C
Congress should
reinstate the 55
MPH speed limit.

I W support
There was a decline The speed limit was
of 8,300 traffic fatalities responsible for the Q
in the year following the decline in traffic
implementation of the fatalities. Certainly
55 MPH speed limit.

FIGURE C4.1
Simple argument maps are static and uncontested

28 www.irpublicpolicy.ir

The Process of Policy Analysis

(a) The Cuban Missile Crisis—Dynamic Contested Argument

C
The United States should force
the Soviet Union to withdraw the
missiles by blockading Cuba.

I W Q weakly
Given that reliable intelli- Because a blockade will show Absolutely support
gence reports confirm that Soviet leaders that the States
the Soviet Union is placing means business and is
offensive missiles in Cuba. prepared to use force.

weakly O strongly
But considering the opposes
B support objections, the claim
is doubtful.
Because an increase
in the cost of an action
reduces the likelihood
that it will be taken.

O strongly
But Soviet leaders may fail to opposes
convince their naval units
to depart from established routines.

W strongly
supports
Because the bulk of research on
organizations shows that major lines of
behavior tend to be straight: Behavior at
time t + 1 differs little from behavior at
time t. The blockade will not work.

(continued)

www.irpublicpolicy.ir 29

The Process of Policy Analysis

(b) The 55 mph Speed Limit—Simple Uncontested Argument
C
Congress should
reinstate the 55
MPH speed limit.

I W Q support
Given that there was a decline of Because the speed Certainly
8,300 traffic fatalities in the year limit was responsible
following the implementation for the decline in
of the 55 MPH speed limit. traffic fatalities.

B support O opposes
But this is not
Since it is obvious that the decline is certain at all.
due to the speed limit. No other factors
can explain the decline, which is the
largest in U.S. history.

O opposes

But most of the decline was due to the 1974
recession, the rise in unemployment, the
doubling of gasoline prices, and the consequent
sharp decline in miles driven, which drastically
reduced the exposure to accidents.

FIGURE C4.2
Complex argument maps are dynamic and contested

30 www.irpublicpolicy.ir

Policy Analysis in the
Policy-Making Process

OBJECTIVES

By studying this chapter, you should be able to

᭿ Understand policy analysis as an composed of multiple phases
intellectual activity embedded in a ordered in time.
political process.
᭿ Distinguish competing explanations
᭿ Explain the historical development of policy change.
of policy analysis as a response to
practical problems and crises. ᭿ Contrast potential and actual uses
of analysis.
᭿ Describe policy making as a
complex, nonlinear process ᭿ Describe the composition, scope, and
expected effects of information use.

Policy analysis creates, critically assesses, and communicates information
about and in the policy-making process.1 This distinction between about and
in marks an essential difference between policy analysis, on one hand, and
political science and economics, disciplines that specialize in developing and testing
descriptive and normative theories of policy making. Although some members of
these disciplines do work on concrete problems facing policy makers, most are
motivated by incentive systems that demand the production of knowledge for its
own sake. By contrast, policy analysts work under incentives designed to promote
the creation and application of practical knowledge—that is, knowledge about

1Harold D. Lasswell, A Pre-view of Policy Sciences (New York: American Elsevier Publishing, 1971),
pp. 1–2. Information about refers to “systematic, empirical studies of how policies are made and put
into effect,” whereas information in refers to the fact that “the realism of a decision depends in part on
access to the stock of available information.”

From Chapter 2 of Public Policy Analysis, Fifth Edition. William N. Dunn. Copyright © 2012 by Pearson
Education, Inc. All rights reserved.

www.irpublicpolicy.ir 31

Policy Analysis in the Policy-Making Process

what works.2 Although policy analysis draws on social science methods, theories,
and substantive findings, the aim is to solve practical problems. This requires the
communication and use of information in as well as about the policy-making
process.3

THE HISTORICAL CONTEXT

Policy analysis is as old as civilization itself. It includes diverse forms of inquiry,
from mysticism and the occult to modern science. Etymologically, the term policy
comes from the Greek, Sanskrit, and Latin languages. The Greek polis (city-state)
and Sanskrit pur (city) evolved into the Latin politia (state) and later into the Middle
English policie. The latter referred to the conduct of public affairs or the administra-
tion of government. The etymological origins of policy are the same for two other
important words: police and politics. These multiple connotations are found in
Germanic and Slavic languages, which have only one word (Politik, politika, respec-
tively) to refer to both policy and politics. This is among the reasons for the porous
boundaries among political science, public administration, and policy analysis, and
the resultant confusion about their substance and aims.

Early Origins

The term policy analysis need not be restricted to its contemporary meaning, where
analysis refers to breaking problems into basic elements or parts, much as we
disassemble a clock or machine. This is the sense of “analysis” when it is argued that
decision problems may be decomposed into alternatives, outcomes, and objectives.
A related view is that policy analysis is a collection of quantitative techniques used
by systems analysts, decision analysts, and economists to examine the likelihood
and utility of policy outcomes.

Understood in a wider sense, however, policy analysis may be seen to have
emerged at a point in the evolution of human societies when practical knowledge
was consciously cultivated, thereby prompting an explicit and self-reflective exami-
nation of links between knowledge and action. The development of specialized
procedures for analyzing policies was related to the emergence of urban civilization

2On differences between intellectual and practical knowledge see the treatise by Fritz Machlup,
Knowledge: Its Creation, Distribution, and Economic Significance. Vol. 1: Knowledge and Knowledge
Production (Princeton, NJ: Princeton University Press, 1980). Machlup also discusses spiritual, pastime,
and unwanted knowledge.
3Carol H. Weiss’s Social Science Research and Decision Making (New York: Columbia University Press,
1980) is a comprehensive synthesis of research and theory on the uses of social science research by policy
makers. Other syntheses include William N. Dunn and Burkart Holzner, “Knowledge in Society: Anatomy
of an Emerging Field,” Knowledge in Society (later titled Knowledge and Policy) 1, no. 1 (1988): 1–26;
and David J. Webber, “The Distribution and Use of Policy Information in the Policy Process,” in Advances
in Policy Studies since 1950, ed. William Dunn and Rita Mae Kelly (New Brunswick, NJ: Transaction,
1991), pp. 415–41. Journals that focus on these problems include Science Communication (formerly
Knowledge: Creation, Diffusion, Utilization), Knowledge and Policy: The International Journal of
Knowledge Transfer and Utilization, and Science, Technology and Human Values.

32 www.irpublicpolicy.ir

Policy Analysis in the Policy-Making Process 33

out of scattered and largely autonomous tribal and folk societies.4 As a specialized
activity, policy analysis followed changes in social and, above all, political organiza-
tion that accompanied new production technologies and stable patterns of human
settlement.

One of the earliest recorded efforts to cultivate policy-relevant knowledge
occurred in Mesopotamia, in what is now the Basra region of southern Iraq. In the
ancient Mesopotamian city of Ur, one of the first legal codes was produced in the
twenty-first century B.C., some two thousand years before Aristotle (384–322 B.C.),
Confucius (551–479 B.C.), and Kautilya (ca. 300 B.C.) produced their classic treatises
on government and politics. In the eighteenth century B.C., the ruler of Babylon, with
the assistance of professionals whom we would now call policy analysts, created the
Code of Hammurabi. The Code was designed to establish a unified and just public
order in a period when Babylon was in transition from a small city-state to a large
territorial state. The Hammurabian Code, a set of policies that parallel Mosaic laws,
reflected the economic and social requirements of stable urban settlements where
rights and obligations were defined according to social position. The Code covered
criminal procedures, property rights, trade and commerce, family and marital
relations, physicians’ fees, and what we now call public accountability.5

The early Mesopotamian legal codes were a response to the growing complexity
of fixed urban settlements, where policies were needed to regulate the distribution of
commodities and services, organize record keeping, and maintain internal security
and external defense. A growing consciousness of relations between knowledge and
action fostered the growth of educated strata that specialized in the production of
policy-relevant information. These “symbol specialists,” as Lasswell called them,
were responsible for policy forecasting; for example, they were expected to foresee
crop yields at the onset of the planting season, or predict the outcomes of war.6
Because analysts used mysticism, ritual, and the occult to forecast the future, their
methods were unscientific by present-day standards. Although such procedures
were based in part on evidence acquired through experience, any reasonable defini-
tion of science requires that knowledge claims be assessed against observations that
are independent of the hopes of analysts, or of those who hire them.7 Then as now,
policy-relevant knowledge was ultimately judged according to its success (or failure)
in shaping better policies, not simply because special methods were used to produce
it. Even the ancients seemed to know what some contemporary analysts forget—
when methods are used for ritualistic purification, political persuasion, and
symbolic legitimation, analysts and their clients eventually must face the decisive
test of whether the methods produce reliable results.8 Although statements such as

4Lasswell, A Pre-view of Policy Sciences, pp. 9, 13.
5The Code of Hammurabi, trans. Robert F. Harper (Chicago: University of Chicago Press, 1904).
6Lasswell, A Pre-view of Policy Sciences, p. 11.
7Donald T. Campbell, “A Tribal Model of the Social System Vehicle Carrying Scientific Information,”
in Methodology and Epistemology for Social Science: Selected Papers, ed. E. Samuel Overman
(Chicago: University of Chicago Press, 1988), pp. 489–503.
8Edward A. Suchman, “Action for What? A Critique of Evaluative Research,” in Evaluating Action
Programs, ed. Carol H. Weiss (Boston, MA: Allyn and Bacon, 1972), p. 81; and Martin Rein and
Sheldon H. White, “Policy Research: Belief and Doubt,” Policy Analysis 3, no. 2 (1977): 239–71.

www.irpublicpolicy.ir

Policy Analysis in the Policy-Making Process

“drug policy is based on good science” are in vogue, the invocation of “science”
in these contexts may represent little more than ritualistic purification.

In India, Kautilya’s Arthashastra, written in the fourth century B.C., is a system-
atic guide to policy making, statecraft, and government administration. The
Arthashastra synthesized much that had been written up to that time on material
success, or what we now call economics. Kautilya, an adviser to the Mauryan
Empire in northern India, has been compared to Plato (427–327 B.C.), Aristotle
(384–322 B.C.), and Machiavelli (1469–1527). In addition to their work as political
theorists, all were deeply involved in the practical aspects of policy making. Plato
served as adviser to the rulers of Sicily, whereas Aristotle tutored Alexander of
Macedonia from the time Alexander was fourteen years old until he ascended the
throne at the age of twenty. Although Aristotle, like many social and behavioral
scientists, found practical politics repugnant, he seemed to have accepted the assign-
ment because he wanted to bring knowledge to bear on policy issues of the day. In
this respect, he followed his teacher Plato, who said that good government would
not occur until philosophers were kings, or kings philosophers. The opportunity to
influence policy by instructing the heir apparent was an offer that in good
conscience he could not refuse.9

These are examples of preeminent individual producers of specialized knowledge,
not of entire classes of educated persons who later would influence policy making
in Europe and Asia. In the Middle Ages, the gradual expansion and differentiation
of urban civilization brought with it an occupational structure that facilitated the
development of specialized knowledge. Princes and kings recruited policy specialists to
provide advice and technical assistance in areas where rulers were least able to make
effective decisions: finance, war, and law. German sociologist Max Weber described
the development of a class of educated policy specialists as follows:

In Europe, expert officialdom, based on the division of labor, has emerged in a
gradual development of half a thousand years. The Italian cities and
seigniories were the beginning, among the monarchies, and states of the
Norman conquerors. But the decisive step was taken in connection with
the administration of the finances of the prince. . . . The sphere of finance
could afford least of all a ruler’s dilettantism—a ruler who at that time was
still above all a knight. The development of war technique called forth the
expert and specialized officer; the differentiation of legal procedure called
forth the trained jurist. In these three areas—finance, war, and law—expert
officialdom in the more advanced states was definitely triumphant during the
sixteenth century.10

The growth of expert officialdom (what Weber called “professional politicians”)
assumed different forms in different parts of the world. In medieval Europe, India,
China, Japan, and Mongolia, the clergy were literate and therefore technically useful.

9J. A. K. Thompson, The Ethics of Aristotle: The Nichomachean Ethics Translated (Baltimore, MD:
Penguin Books,1955), p. D. 11.
10Max Weber, “Politics as a Vocation,” in From Max Weber: Essays in Sociology, ed. Hans C. Gerth
and C. Wright Mills (New York: Oxford University Press, 1946), p. 88.

34 www.irpublicpolicy.ir

Policy Analysis in the Policy-Making Process 35

Christian, Brahmin, Buddhist, and Lamaist priests, much like some modern social
and behavioral scientists, earned a reputation for impartiality and disinterestedness
insofar as they stood above practical politics and temptations of political power and
economic gain. Educated men of letters—whose modern counterpart is the special
presidential adviser—influenced policy making until court nobles, who later came to
dominate the political and diplomatic service, replaced them. In England, petty
nobles and urban rentiers (investors) were recruited without compensation to
manage local governments in their own interest. Jurists trained in Roman law and
jurisprudence had a strong influence on policy making, particularly in Continental
Europe. They were largely responsible for the transformation of the late medieval
state and the movement toward modern government.

The age of the Industrial Revolution was also that of the Enlightenment, a
period in which a belief in human progress through science and technology became
an ever more dominant theme among policy makers and their advisers. The develop-
ment and testing of scientific theories of nature and society gradually came to be
seen as the only objective means for understanding and solving social problems. For
the first time, policy-relevant knowledge was produced according to the canons of
empiricism and the scientific method.

The Nineteenth-Century Transformation

In nineteenth-century Europe, producers of policy-relevant knowledge began to
base their work on the systematic recording of empirical data. Earlier, philosophers
and statesmen had offered systematic explanations of policy making and its role in
society. Yet for several thousand years, there was an essential continuity in methods
for investigating and solving human social, economic, and political problems. If
evidence for a particular point of view was provided, it was typically based on appeals
to authority, ritual, or philosophical doctrine. What was new in the nineteenth
century was a basic change in the procedures used to understand society and its
problems, a change reflected in the growth of empirical, quantitative, and policy-
oriented research.11

The first censuses were conducted in the United States (1790) and England
(1801). It was at this time that statistics (“state arithmetic”) and demography
began to develop as specialized fields. The Manchester and London Statistical
Societies, established in the 1830s, helped shape a new orientation toward policy-
relevant knowledge. Organized by bankers, industrialists, and scholars, the soci-
eties sought to replace traditional ways of thinking about social problems with
empirical analyses of the effects of urbanization and unemployment on the lives
of workers and their families. In the Manchester Society, an enthusiasm for
quantification was coupled with a commitment to social reform, or “progress of
social improvement in the manufacturing population.”12 The London Society,

11Daniel Lerner, “Social Science: Whence and Whither?” in The Human Meaning of the Social Sciences,
ed. Daniel Lerner (New York: World Publishing, 1959), pp. 13–23.
12Nathan Glazer, “The Rise of Social Research in Europe,” in The Human Meaning of the Social
Sciences, ed. Lerner, p. 51.

www.irpublicpolicy.ir

Policy Analysis in the Policy-Making Process

under the influence of Thomas Malthus (1766–1834) and other academics, took
a more disinterested approach:

The Statistical Society will consider it to be the first and most essential rule of its
conduct to exclude carefully all opinions from its transactions and publications—
to confine its attention rigorously to facts—and, as far as it may be found possible,
to facts which can be stated numerically and arranged in tables.13

The London and Manchester societies used questionnaires to carry out studies,
and paid “agents” were the counterpart of today’s professional interviewer. There
were similar developments in France, Germany, and the Netherlands.

A preeminent contributor to the methodology of social statistics and survey
research was Adolphe Quetelet (1796–1874), a Belgian mathematician and astronomer
who was the major scientific adviser to the Dutch and Belgian governments.14 Quetelet
addressed most topics on survey design and analysis found in contemporary texts:
questionnaire design; data collection, analysis, and interpretation; data organization
and storage; and identification of conditions under which data are collected. In
the same period, Frederic Le Play (1806–82) wrote Les Ouvriers Europeans
[The European Workers], a detailed empirical investigation of family income and
expenditures of European workers in several countries. In Germany, Ernst Engel
(1821–96) sought to derive laws of “social economics” from empirical data expressed
in statistical form.

In England, the work of Henry Mayhew and Charles Booth, who studied the
life and employment conditions of the urban poor in natural (what we now call
“field”) settings, is representative of the new empirical approach to the study of
social problems. Mayhew’s London Labour and the London Poor (1851) described
the lives of the laborers, peddlers, performers, and prostitutes who comprised
London’s urban underclass. In writing Life and Labour of the People in London
(1891–1903), Booth employed school inspectors as key informants. Using what we
now call participant observation, Booth lived among the urban poor, gaining first-
hand experience of actual living conditions. A member of the Royal Commission on
the Poor Law, he was an important influence on the revision of policies on old-age
pensions. Booth’s work also served as something of a model for policy-oriented
research in the United States, including the Hull House Maps and Papers (1895)
and W. E. B. Dubois’s The Philadelphia Negro (1899), both of which sought to
document the scope and severity of poverty in urban areas.

The nineteenth-century transformation was not the result of declarations of
allegiance to canons of logical empiricism and the scientific method. Declarations to
this effect did not and could not occur until the next century, when Vienna Circle
philosophers engaged in the logical reconstruction of physics to propose formal
principles and rules of successful scientific practice (few natural or social scientists
have actually followed these principles and rules). The transformation came, rather,
from the uncertainty accompanying the shift from agrarian to industrial societies,

13Ibid., pp. 51–52.
14A significant history of statistics and statisticians is Stephen M. Stigler, The History of Statistics: The
Measurement of Uncertainty before 1900 (Cambridge, MA: Harvard University Press, 1990).

36 www.irpublicpolicy.ir

Policy Analysis in the Policy-Making Process 37

a shift that preceded the Industrial Revolution. Later, industry and industrialized
science required a politically stable order to operate efficiently. Political stability was
associated with profound social instability.15

Thus, for the most part science and technology were not responsible for the
growth of newly centralized systems of political control. Although science and
technology contributed to problems of a newly uprooted, uneducated, and displaced
class of urban workers and their families, rulers and dominant social groups valued
policy-oriented research as a means to achieve political and administrative control.
In the sphere of factory production, for example, the political organization of work
in assembly lines preceded scientific and technological developments that later
culminated in efficiency-enhancing machinery and the specialization of tasks.16 In
the sphere of public policy, we see a parallel development. Methods of empirical,
quantitative, and policy-oriented analysis were a product of the recognition by
bankers, industrialists, politicians, and the Victorian middle class that older methods
for understanding the natural and social world were no longer adequate. The key
questions of the day were practical: How much did members of the urban prole-
tariat need to earn to maintain themselves and their families? How much did they
have to earn before there was a taxable surplus? How much did they have to save
from their earnings to pay for medical treatment and education? How much should
capitalist owners and the state invest in day care facilities so that mothers might
put in an effective day’s work? How much investment in public works projects—
sanitation, sewage, housing, roads—was required to maintain adequate public
health standards, not only to maintain a productive workforce but also to protect
the middle and upper classes from infectious diseases cultivated in urban slums?

The Twentieth Century

An important feature of the twentieth century, as compared with the nineteenth, is
the institutionalization of the social sciences and professions. Twentieth-century
producers of policy-relevant knowledge were no longer the heterogeneous group of
bankers, industrialists, journalists, and academics who guided the early statistical
societies and other institutions of policy research. They were graduates with first
and advanced degrees in policy-relevant disciplines and professions who, along with
professors, occupied important positions in governments or worked as consultants
or researchers under grants and contracts. In background, experience, and motiva-
tion, they were members of established professions that, more or less, were guided
by commonly accepted scientific and professional norms.

The new professionals played an active role in the administration of Woodrow
Wilson, particularly during World War I. Later, under the Republican administration
of Herbert Hoover, social scientists carried out two major social surveys, Recent
Economic Trends and Recent Social Trends. The largest influx of social scientists into

15J. H. Plumb, The Growth of Political Stability in England, 1675–1725 (Baltimore, MD: Penguin
Books, 1973), p. 12.
16Stephen A. Marglin, “What Do Bosses Do? The Origins and Functions of Hierarchy in Capitalist
Production,” Review of Radical Political Economy 6, no. 2 (1974): 33–60.

www.irpublicpolicy.ir

Policy Analysis in the Policy-Making Process

government came, however, with Franklin Roosevelt’s New Deal. Large numbers of
social scientists staffed the numerous new agencies established during the Roosevelt
administration (e.g., National Recovery Administration, Work Projects Administration,
Public Works Administration, Securities and Exchange Commission, and Federal
Housing Administration).

The primary function of social scientists in the 1930s was to investigate policy
problems and broad sets of potential solutions, and not, as in later periods, to
employ economic modeling, decision analysis, or policy experimentation to
identify and select specific solutions to problems. The Roosevelt administration’s
National Planning Board (later the National Resources Planning Board), a majority
of whose members were professional social scientists, provides a good illustration
of the approach to policy questions characteristic of the 1930s. The board was
conceived as “a general staff gathering and analyzing facts, observing the
interrelation and administration of broad policies, proposing from time to time
alternative lines of national procedure, based on thorough inquiry and mature
consideration.”17 This same general orientation toward problems was evident
among economists working for the Department of Agriculture, political scientists
involved in the reorganization of the executive branch, and anthropologists
conducting studies for the Bureau of Indian Affairs. Social scientists also contributed
to methodological innovations; for example, the Department of Agriculture led in
further developing the sample survey as a research tool and instrument of govern-
ment census policy.18

World War II and the postwar readjustment that followed provided social scien-
tists with opportunities to demonstrate their value in solving practical problems.
Interwar achievements in the area of survey research had laid a foundation for the
use of interviews by the Office of War Information, the War Production Board, and
the Office of Price Administration. Military and civilian agencies relied on social
scientists to investigate problems of national security, social welfare, defense, war
production, pricing, and rationing. The activities of agencies such as the Office of
Strategic Services were continued after the war by the Office of Naval Research; by
the Department of the Air Force; and, later, by the Research and Development Board
(subsequently RAND) of the Department of Defense, and the Central Intelligence
Agency. Special research institutes were established by the federal government,
including the Operations Research Office at Johns Hopkins University and the
Human Resources Research Office at George Washington University. Among the
seminal contributions to policy research in this period was The American Soldier
(1950), a four-volume study produced by many of the most able applied social scien-
tists in the country. The director of the Army Morale Division commissioned this
large-scale project in 1941, under the general direction of sociologist Samuel Stouffer.
The project is significant, not only because of its scale but also because it was part of
an emerging pattern of extensive governmental support for policy research and
analysis. Military policy makers turned to the social researcher, not only for facts but

17Gene Lyons, The Uneasy Partnership: Social Science and the Federal Government in the Twentieth
Century (New York: Russell Sage Foundation, 1969), p. 65.
18Harry Alpert, “The Growth of Social Research in the United States,” in The Human Meaning of the
Social Sciences, ed. Lerner, pp. 79–80.

38 www.irpublicpolicy.ir

Policy Analysis in the Policy-Making Process 39

also for causal inferences and conclusions that would affect the lives of millions of
troops.19 This large research program contributed to the development and refine-
ment of multivariate analysis and other quantitative techniques that are now widely
used by researchers in social science disciplines.

After World War II, the first systematic effort to develop an explicit policy
orientation within the social and behavioral sciences was The Policy Sciences:
Recent Developments in Scope and Method (1951), edited by political scientists
Daniel Lerner and Harold D. Lasswell. The “policy sciences,” as stated by Lasswell
in the introduction, are not confined to theoretical aims of science; they also have a
fundamentally practical orientation. Moreover, their purpose is not simply to
provide a basis for making efficient decisions but also to provide knowledge
“needed to improve the practice of democracy. In a word, the special emphasis is
upon the policy sciences of democracy, in which the ultimate goal is the realization
of human dignity in theory and fact.”20

The systematic study of public policy also grew out of public administration,
then a field within political science. In 1937, Harvard University established the
Graduate School of Public Administration, which focused in part on public policy.
In the late 1940s, an interuniversity committee was established to develop public
policy curricular materials, a major product of which was Harold Stein’s Public
Administration and Policy Development: A Case-Book (1952). The interuniversity
committee, composed of professors and practitioners of public administration,
speaks for the close relationship between policy analysis and public administration
before and after World War II.21

The impetus for developing methods and techniques of policy analysis—as
distinguished from its theory and methodology—did not originate in political science
or public administration. The technical side of policy analysis rather grew out of
engineering, operations research, systems analysis, applied mathematics, and to a lesser
extent applied economics. Most of those responsible for developing methods and
techniques had received their formal training outside the social sciences. World War II
had prompted the involvement of specialists whose orientation toward policy was
primarily analytical, in the narrow sense of that term. The idea of “analysis” came to
be associated with efforts to separate or decompose problems into their fundamental
components, for example, decomposing problems of national defense into policy
alternatives (nuclear warheads, manned bombers, conventional ground troops) whose
consequences for the attainment of policy objectives could be estimated. This analycen-
tric perspective22 tends to preclude or restrict concerns with political, social, and
administrative aspects of public policy, for example, concerns with the political feasibility
of alternatives or their implications for democratic processes. Although the analycen-
tric turn represents a movement away from the multidisciplinary vision of Lasswell’s

19Howard E. Freeman and Clarence C. Sherwood, Social Research and Social Policy (Englewood Cliffs,
NJ: Prentice Hall, 1970), p. 25.
20Harold D. Lasswell, “The Policy Orientation,” in The Policy Sciences: Recent Developments in Scope and
Method, ed. Daniel Lerner and Harold D. Lasswell (Stanford, CA: Stanford University Press, 1951), p. 15.
21H. George Frederickson and Charles Wise, Public Administration and Public Policy (Lexington, MA:
D. C. Heath, 1977).
22Allen Schick, “Beyond Analysis,” Public Administration Review 37, no. 3 (1977): 258–63.

www.irpublicpolicy.ir

Policy Analysis in the Policy-Making Process

policy sciences,23 it has supplied more systematic procedures—from decision analysis
to applied microeconomics—for selecting policy alternatives.24

The analycentric turn was accompanied by the growing influence of nonprofit
research organizations (“think tanks”) such as the RAND Corporation, which fostered
the spread of systems analysis and related techniques to government agencies and the
academic community.25 The development of program-planning-budgeting systems
(PPBS) was due in large measure to the efforts of operations researchers and economists
working under Charles Hitch at RAND. The RAND group wanted to find out “how
the country could ‘purchase’ national security in the most efficient manner—how much
of the national wealth should be devoted to defense, how the funds allocated to defense
should be distributed among different military functions, and how to assure the most
effective use of these funds.”26 Although PPBS was introduced into the Department of
Defense in 1965 and was later mandated for use in all federal agencies, it was difficult
to implement. After 1971 it became discretionary and soon fell into disuse. Despite
mixed conclusions about its success as a tool of policy analysis, PPBS did capture the
attention of government and university analysts who value systematic procedures for
selecting and evaluating policy alternatives.27

The analycentric turn has been offset to some extent by the rapid growth of private
foundations whose mission is to support traditional lines of research in the social
sciences and humanities. More than three-fourths of these foundations were
established after 1950.28 In the same period, the federal government began to set aside
funds for applied and policy-related research in the social sciences, although the natural
sciences continued to receive the bulk of government research support. Although in
1972 the social sciences received approximately 5 percent of all available federal
research funds, in the period 1980–90 funding for applied and basic research in the
social sciences fell by approximately 40 percent in constant dollars.29 At the same time,
it is noteworthy that more than 95 percent of all research funded by governmental,
nonprofit, and private organizations is applied research on practical problems.

By the 1970s, many social science disciplines had established institutions expressly
committed to applied and policy-related research. These include the Policy Studies
Organization (political science), the Society for the Study of Social Problems (sociology),
and the Society for the Psychological Study of Social Issues (psychology). Each has its

23Peter de Leon, Advice and Consent: The Development of the Policy Sciences (New York: Russell Sage
Foundation, 1988), ch. 2.
24Martin Greenberger, Matthew A. Crenson, and Brian L. Crissey, Models in the Policy Process: Public
Decision Making in the Computer Era (New York: Russell Sage Foundation, 1976), pp. 23–46.
25Bruce L. R. Smith, The Rand Corporation: A Case Study of a Nonprofit Advisory Corporation
(Cambridge, MA: Harvard University Press, 1966).
26Greenberger, Crenson, and Crissey, Models in the Policy Process, p. 32.
27For example, Alice Rivlin, Systematic Thinking for Social Action (Washington, DC: Brookings
Institution, 1971); and Walter Williams, Social Policy Research and Analysis: The Experience in the
Federal Social Agencies (New York: American Elsevier Publishing, 1971).
28Irving Louis Horowitz and James E. Katz, Social Science and Public Policy in the United States
(New York: Praeger Publishers, 1975), p. 17.
29National Science Foundation, Federal Funds for Research, Development, and Other Scientific
Activities (Washington, DC: NSF, 1973); and National Science Board, Science and Engineering
Indicators—1989 (Washington, DC: NSB, 1989).

40 www.irpublicpolicy.ir

Policy Analysis in the Policy-Making Process 41

own journal of record. In the 1980s the process of institutionalizing policy-oriented
social science was carried a step further by the creation of multidisciplinary professional
associations such as the Association for Public Policy and Management, which holds
annual research conferences and publishes a journal of record, the Journal of Policy
Analysis and Management. The new journal brought a more technical focus than the
mainstream policy journals, including Policy Sciences, the Policy Studies Journal, and
Policy Studies Review. In addition to the mainstream journals were hundreds of others
that focused on specific issues involving health, welfare, education, criminal justice,
education, science and technology, and other areas.30

In the same period, universities in the United States and Europe founded new
graduate programs and degrees in policy analysis. In the United States, a number of
new programs were established with the support of the Ford Foundation’s Program
in Public Policy and Social Organization. Most research universities in the United
States have policy centers or institutes listed in the Encyclopedia of Associations,
along with thousands of freestanding nonprofit policy research organizations and
advocacy groups. Most were established after 1950. In Washington and most state
capitals, and in the European Union, “policy analyst” is a formal job description.
The National Governor’s Association and the National League of Cities have policy
analysis units. There are similar units throughout the U.S. government; in direc-
torates of the European Union; and in international organizations, including the
United Nations and the World Bank. Even a brief search of the World Wide Web
yields scores of policy think tanks in all regions of the world.31

Since the beginning of the twenty-first century, there has been increasing recognition
that the complexity of problems faced by governments today requires the systematic use
of natural and social scientists to help develop policies and assess their consequences.
The call for evidence-based policy making in the United Kingdom, the United States,
and the European Union is a response to this complexity; it is also a recognition that
ideological, religious, and political influences—usually hidden and lacking in trans-
parency—have exerted a harmful effect on policy making in areas ranging from health,
education, and welfare to national security and the environment. In the words of a recent
British House of Commons report titled Scientific Advice, Risk, and Evidence Based
Policy Making (2006), evidence-based policy making

has its roots in Government’s commitment to “what works” over ideologically
driven policy. . . . This Government expects more of policy makers. More new
ideas, more willingness to question inherited ways of doing things, better use of
evidence and research in policy making and better focus on policies that will
deliver long-term goals.32

30Michael Marien, editor of Future Survey (journal of record of the World Future Society), estimates the
number of policy journals to exceed four hundred. Marien, “The Scope of Policy Studies: Reclaiming
Lasswell’s Lost Vision,” in Advances in Policy Studies since 1950, vol. 10, Policy Studies Review Annual,
ed. William N. Dunn and Rita Mae Kelly (New Brunswick, NJ: Transaction, 1991), pp. 445–88.
31See, for example, www.nira.go.jp (World Directory of Think Tanks), a Japanese directory, and
www.policy.com.
32United Kingdom, House of Commons, Science and Technology Committee. Scientific Advice, Risk
and Evidence Based Policy Making. Seventh Report of Session 2005–06, Volume I (London: HMO
Printing House, 2006).

www.irpublicpolicy.ir

Policy Analysis in the Policy-Making Process

Evidence-based policy making in the United Kingdom and the European Union
takes several forms including regulatory impact assessment (RIA), which refers to
the use of scientific analyses to examine the benefits, costs, risks, and consequences
of newly introduced policies before they are adopted. In the United States,
evidence-based policy has been promoted by leading program evaluators and
policy analysts who founded the Coalition for Evidence-Based Policy of the
Council for Excellence in Government. Some procedures of the Office of
Management and Budget are based on evidence-based policy analysis, especially
methods and standards of program evaluation.33 Although some see the movement
toward evidence-based policy making as a continuation of an ostensibly harmful
logical positivist (scientistic) approach to questions of public policy and
democracy,34 as yet it is unclear whether this negative assessment is itself based on
sound reasoning and evidence.

THE POLICY-MAKING PROCESS

The development of policy analysis has been a response to practical problems.
Although recognition of these practical origins is important, historical awareness
alone does not tell us much about the characteristics of policy making and how
it works.

Policy analysis is a fundamentally intellectual activity embedded in a political
process. This process, which includes economic, cultural, and organizational factors,
is usually described as a policy-making process, or policy process for short. It is
useful to visualize this process as a series of interdependent activities arrayed through
time—agenda setting, policy formulation, policy adoption, policy implementation,
policy assessment, policy adaptation, policy succession, and policy termination
(Table 1).35 Depending on circumstances, analysts produce information relevant to
one, several, or all phases of policy making.

The policy process is composed of complex rounds or cycles (Figure 1). Each
phase of the policy cycle is linked to the next, in backward and forward loops, and
the process as a whole has no definite beginning or end. Individuals, interest
groups, bureaus, offices, departments, and ministries are involved in policy cycles

33Council for Evidence Based Policy. 1301 K Street, NW, Suite 450 West, Washington, DC 2005.
www.excelgov.org/evidence; www.evidencebasedprograms.org.
34Wayne Parsons, “From Muddling Through to Muddling Up: Evidence Based Policy-Making
and the Modernisation of British Government.” Unpublished paper (London: University of
London, 2004).
35The policy cycle, or stages approach, is summarized by Peter de Leon, “The Stages Approach
to the Policy Process: What Has It Done? Where Is It Going?” in Theories of the Policy
Process, ed. Paul A. Sabatier (Boulder, CO: Westview Press, 1999). See also Charles O. Jones,
An Introduction to the Study of Public Policy, 2d ed. (North Scituate, MA: Duxbury Press,
1977); James A. Anderson, Public Policy Making (New York: Praeger, 1975); Gary Brewer and
Peter de Leon, Foundations of Policy Analysis (Homewood, IL: Dorsey Press, 1983). The classic
work that influenced all works cited here is Harold D. Lasswell, The Decision Process: Seven
Categories of Functional Analysis (College Park: Bureau of Governmental Research, University
of Maryland, 1956).

42 www.irpublicpolicy.ir

Policy Analysis in the Policy-Making Process

TABLE 1

Phases of the Policy-Making Process

Phase Characteristics Illustration
Agenda Setting
Elected and appointed officials place A state legislator and her cosponsor prepare
Policy problems on the public agenda. Many a bill that goes to the Health and Welfare
Formulation problems are not acted on at all, whereas Committee for study and approval. The bill
others are addressed only after long delays. stays in committee and is not voted on.
Policy
Adoption Officials formulate alternative policies A state court considers prohibiting the
to deal with a problem. Alternative policies use of standardized achievement tests
assume the form of executive orders, court such as the SAT on grounds that the tests
decisions, and legislative acts. are biased against women and minorities.

A policy is adopted with the support of a In Roe v. Wade, Supreme Court justices
legislative majority, consensus among reach a majority decision that women
agency directors, or a court decision. have the right to terminate pregnancies
through abortion.

Policy An adopted policy is carried out by The city treasurer hires additional staff
Implementation administrative units that mobilize financial to ensure compliance with a new law
and human resources to comply with the that imposes taxes on hospitals that no
Policy policy. longer have tax-exempt status.
Assessment
Auditing and accounting units in The General Accounting Office monitors
Policy government determine whether executive social welfare programs such as Aid to
Adaptation agencies, legislatures, and courts are in Families with Dependent Children
compliance with statutory requirements of (AFDC) to determine the scope of
Policy a policy and achieving stated objectives. welfare fraud.
Succession
Auditing and evaluation units report A state Department of Labor and
Policy to agencies responsible for formulating, Industry evaluates an affirmative action
Termination adopting, and implementing policies that training program, finding that employees
adaptations in poorly written regulations, wrongly believe that complaints against
insufficient resources, inadequate training, discrimination should be made to
etc. are needed. immediates.

Agencies responsible for evaluating The National Highway Traffic Safety
policies acknowledge that a policy is no Administration (NHTSA) convinces
longer needed because the problem Congress to maintain the 55 mph speed
dissolved. Rather than terminate the limit because it is achieving the new goal
policy, it is maintained and redirected of reducing traffic fatalities, injuries, and
toward a new goal. property damage.

Agencies responsible for evaluation and The U.S. Congress terminates the Office of
oversight determine that a policy or an Technology Assessment (OTA) on grounds
entire agency should be terminated that other agencies and the private sector
because it is no longer needed. are able to assess the economic and social
effects of technologies.

www.irpublicpolicy.ir 43

Policy Analysis in the Policy-Making Process

Agenda AGENDAS Formulation
Setting PROPOSALS
Adaptation
Succession
Termination

OUTPUTS IMPACTS

Assessment

Implementation Adoption

POLICIES

FIGURE 1
The policy-making process has multiple functions and stages

through cooperation, competition, and conflict. One form of the cycle involves
policy adaptation, whereby a feedback loop connects later phases of the process to
earlier ones. Other forms of cycles are policy succession, where new policies and
organizations build on old ones, and policy termination. Policy termination may
mean the end of a policy or program, although even termination affects what is-
sues are placed on the public agenda, and in this sense represents another kind of
cycle.

In some cases, a policy is adopted first and then justified by working backward
to agenda setting, when a problem is formulated or reformulated to fit or justify the
policy. Adjacent phases may be linked, or skipped altogether, creating “short
circuits.” Solutions and problems are in continuous flux, creating a degree of
complexity that prompts metaphors of “garbage cans,” “primeval policy soups,”
and “organized anarchies.”36

36Michael Cohen, James March, and Johan Olsen, “A Garbage Can Model of Organizational Choice,”
Administrative Science Quarterly 17 (March 1972): 1–25; John W. Kingdon, Agendas, Alternatives, and
Public Policies, 2d ed. (New York: HarperCollins, 1995).

44 www.irpublicpolicy.ir

Policy Analysis in the Policy-Making Process 45

MODELS OF POLICY CHANGE

Metaphors such as garbage cans, primeval policy soups, and organized anarchies
are difficult to grasp, because it seems that policy making under these conditions
would have no structure or organization at all. That this is untrue may be illustrated
by a story:37

Some years ago, on a military base, there were reports of the possible theft of
military secrets. Every day, at about the same time, a worker pushing a
large wheelbarrow would attempt to leave the base by passing through the
security gate.

On the first day, the guards at the gate asked the worker what was in the
wheelbarrow. He replied “Just dirt.” The guard poked into the wheelbarrow
with his baton. Satisfied, he told the worker to pass through. The same proce-
dure was repeated for the rest of the week. By the second week, the guards
were growing increasingly suspicious. They made the worker empty the
contents of the wheelbarrow on the road. This time, the guards used a rake to
examine the contents. Again, they found nothing but dirt. This continued for
another two weeks.

In the third week, the guards called in a special investigations unit. They
not only emptied the contents; they also used a special scanning device. They
found nothing. Subsequently, the worker was allowed to pass freely through
the gate.

At the end of the month, dozens of wheelbarrows were reported missing.

Structures are just as important as their contents. Conceptual models of policy
making are mental structures or maps that help us understand policy-making
processes. These conceptual models are abstract representations based on metaphors
such as “garbage can,” “anarchy,” and “primeval soup.” Other metaphors are “policy
observatory,” “problems as infectious diseases,” “policy decay,” “war on poverty,”
and “rationality” itself.38

Comprehensive Rationality

The comprehensive rationality model portrays policy making as an exhaustive
striving for efficiency. A rational economic actor is seen as Homo economicus,
an individual or collective decision maker who weighs the costs and benefits of
all available alternatives and takes actions that are motivated by a concern with
the efficient use of resources. The fundamental proposition of the economic
rationality model is as follows: The greater the net efficiency (perceived benefits
less perceived costs) of an alternative selected from a full (comprehensive) set of

37John Funari, personal communication.
38Rein and Schon call them “generative metaphors” because they generate a framework for representing
policy problems. See Martin Rein and Donald A. Schon, “Problem Setting in Policy Research,” in
Using Social Research in Public Policymaking, ed. Carol Weiss (Lexington, MA: D.C. Heath, 1977),
pp. 240–43.

www.irpublicpolicy.ir

Policy Analysis in the Policy-Making Process

potential solutions, the more likely it will be chosen as a (rational) basis for policy
action and change.39 Policy change occurs when an individual or collective deci-
sion maker:

᭿ Identifies a policy problem on which there is sufficient consensus among
relevant stakeholders that the decision maker can act on their behalf.

᭿ Specifies and ranks consistently the objectives whose attainment would
constitute a resolution of the problem.

᭿ Identifies the policy alternatives that may best contribute to the attainment of
each goal and objective.

᭿ Forecasts the consequences that will result from the selection of each
alternative.

᭿ Compares the efficacy of these consequences in attaining each objective.
᭿ Chooses the alternative(s) that maximizes the attainment of objectives.
᭿ Intervenes in the policy-making process by acting on the choice.

Other versions of rational choice involve the incorporation of institutional trans-
action costs into choice situations;40 the redefinition of benefits and costs in political,
social, organizational, or moral terms (e.g., Homo politicus);41 and the proviso that
decision makers are fallible learners with imperfect information, limited computa-
tional capabilities, and a propensity for error in institutional contexts.42

Second-Best Rationality

An important criticism of the rational economic model is based on and known by
the name of Arrow’s impossibility theorem. The theorem states that it is impossible
for decision makers in a democratic society to meet the requirements of the
economic rationality model.43 Individual rational choices cannot be aggregated by
means of majority voting procedures to create a single best choice for all parties.
The impossibility of creating a collective decision that involves transitive preferences
(if A is preferred to B, and B is preferred to C, then A is preferred to C) is described
as the “voters’ paradox.”

Consider a committee composed of three members: Brown, Jones, and Smith.
The committee wants to decide which of three forms of energy—solar, coal, and
nuclear—should be adopted to resolve the energy crisis. Brown, the leader of an
energy rights organization, prefers solar to coal and coal to nuclear, reasoning that
this ranking creates least risk to citizens. Because Brown’s choice is transitive, it

39Rational choice models are reviewed by Elinor Ostrom, “Institutional Rational Choice: An
Assessment of the Institutional Analysis and Development Framework,” in Theories of the Policy
Process, ed. Sabatier, pp. 35–72. The classic critique of comprehensive rational choice is Charles E.
Lindblom, The Policy-Making Process (Englewood Cliffs, NJ: Prentice Hall, 1968).
40Oliver E. Williamson, The Economic Institutions of Capitalism (New York: Free Press, 1985).
41David J. Silverman, The Theory of Organisations (New York: Free Press, 1972).
42Elinor Ostrom, Governing the Commons: The Evolution of Institutions for Collective Action
(New York: Cambridge University Press, 1990); and Mancur Olson, The Logic of Collective Action:
Public Goods and the Theory of Groups (Cambridge, MA: Harvard University Press, 1965).
43Kenneth J. Arrow, Social Choice and Individual Values (New York: John Wiley, 1963).

46 www.irpublicpolicy.ir