Public Policy in an Uncertain World: Analysis and Decisions

by Charles F. Manski

Overview

Public policy advocates routinely assert that “research has shown” a particular policy to be desirable. But how reliable is the analysis in the research they invoke? And how does that analysis affect the way policy is made, on issues ranging from vaccination to minimum wage to FDA drug approval? Charles Manski argues here that current policy is based on untrustworthy analysis. By failing to account for uncertainty in an unpredictable world, policy analysis misleads policy makers with expressions of certitude. Public Policy in an Uncertain World critiques the status quo and offers an innovation to improve how policy research is conducted and how policy makers use research.

Consumers of policy analysis, whether civil servants, journalists, or concerned citizens, need to understand research methodology well enough to properly assess reported findings. In the current model, policy researchers base their predictions on strong assumptions. But as Manski demonstrates, strong assumptions lead to less credible predictions than weaker ones. His alternative approach takes account of uncertainty and thereby moves policy analysis away from incredible certitude and toward honest portrayal of partial knowledge. Manski describes analysis of research on such topics as the effect of the death penalty on homicide, of unemployment insurance on job-seeking, and of preschooling on high school graduation. And he uses other real-world scenarios to illustrate the course he recommends, in which policy makers form reasonable decisions based on partial knowledge of outcomes, and journalists evaluate research claims more closely, with a skeptical eye toward expressions of certitude.

Product Details

ISBN-13:	9780674066892
Publisher:	Harvard University Press
Publication date:	02/18/2013
Edition description:	New Edition
Pages:	224
Sales rank:	834,367
Product dimensions:	6.30(w) x 9.30(h) x 0.90(d)

About the Author

Charles F. Manski is Board of Trustees Professor of Economics at Northwestern University.

Table of Contents

Preface xiii

Introduction 1

Rumsfeld and the Limits to Knowledge 1

Using Policy Analysis to Inform Decisions 2

Organization of the Book 4

I Policy Analysis

1 Policy Analysis with Incredible Certitude 11

1.1 The Logic and Credibility of Policy Analysis 11

1.2 Incentives for Certitude 13

Support for Certitude in Philosophy of Science

1.3 Conventional Certitudes 15

CBO Scoring of Pending Legislation

Scoring the Patient Protection and Affordable Care Act of 2010; Credible Interval Scoring; Can Congress Cope with Uncertainty?

British Norms

1.4 Dueling Certitudes 23

The RAND and IDA Reports on Illegal Drug Policy

The National Research Council Assessment

1.5 Conflating Science and Advocacy 27

Friedman and Educational Vouchers

1.6 Wishful Extrapolation 30

Selective Incapacitation

Extrapolation from Randomized Experiments: The FDA Drug Approval Process

The Study Population and the Population of Interest

The Experimental Treatments and the Treatments of Interest

The Outcomes Measured in Experiments and the Outcomes of Interest

The FDA and Conventional Certitude

Campbell and the Primacy of Internal Validity

1.7 Illogical Certitude 37

Heritability

What Does "More Important" Mean?

Heritability and Social Policy

Gene Measurement

1.8 Media Overreach 44

"The Case for $320,000 Kindergarten Teachers" Peer Review and Credible Reporting

2 Predicting Policy Outcomes 47

2.1 Deterrence and the Death Penalty 48

Estimates Using Data on Homicide Rates across States and Years

2.2 Analysis of Treatment Response 51

Statistical Inference and Identification

2.3 Predicting Outcomes under Policies That Mandate a Treatment 53

Sentencing and Recidivism

Background

Our Analysis

Analysis Assuming Individualistic Treatment Response

Numerical Findings

Choosing a Policy

2.4 Identical Treatment Units 59

Before-and-After Studies

Difference-in-Differences Studies

Employment in Fast-Food Restaurants and the Minimum Wage

2.5 Identical Treatment Groups 63

Experiments with Random Assignment of Treatments

The "Gold Standard"

2.6 Randomized Experiments in Practice 67

Extrapolation

Compliance

The Illinois Unemployment Insurance Experiment

Random Compliance

Intention-to-Treat

The Mixing Problem

Extrapolation from the Perry Preschool Project

Social Interactions

Local and Global Interactions

Credible Analysis of Experimental Data

2.7 Random Treatment Choice in Observational Studies 77

Rational Treatment Choice and Selection Bias

Outcome Optimization with Perfect Foresight

Regression Discontinuity Analysis

2.8 Modeling Rational Treatment Choice 81

Outcome Optimization as a Model of Sentencing Distributional Assumptions

3 Predicting Behavior 85

3.1 Income Taxation and Labor Supply 86

The Theory of Labor Supply

Empirical Analysis

Basic Revealed-Preference Analysis

Illustration: Labor Supply under Progressive and Proportional Taxes

3.2 Discrete Choice Analysis 93

Random Utility Model Representation of Behavior

Attribute Representation of Alternatives and Decision Makers

Analysis with Incomplete Attribute Data

Practicality

College Choice in America

Predicting the Enrollment Effects of Student Aid Policy

Power and Price of the Analysis

Discrete Choice Analysis Today

3.3 Predicting Behavior under Uncertainty 99

How Do Youth Infer the Returns to Schooling?

How Do Potential Criminals Perceive Sanctions Regimes?

Measuring Expectations

Pill, Patch, or Shot?

3.4 Perspectives on Rational Choice 102

As-If Rationality

Bounded Rationality

Biases and Heuristics

Widespread Irrationality or Occasional Cognitive Illusions?

The Common Thread Is Certitude

II Policy Decisions

4 Planning with Partial Knowledge 115

4.1 Treating X-Pox 116

4.2 Elements of Decision Theory 116

States of Nature

The Welfare Function

Welfare Functions in Studies of Optimal Income Taxation

The Mirrlees Study

4.3 Decision Criteria 121

Elimination of Dominated Actions

Weighting States and the Expected Welfare Criterion

Criteria for Decision Making under Ambiguity

Maximin

Minimax Regret

Using Different Criteria to Treat X-Pox

4.4 Search Profiling with Partial Knowledge of Deterrence 125

4.5 Vaccination with Partial Knowledge of Effectiveness 127

Background

Internal and External Effectiveness

The Planning Problem

Partial Knowledge of External Effectiveness

Choosing a Vaccination Rate

4.6 Rational and Reasonable Decision Making 131

The Savage Argument for Consistency

Axiomatic Rationality and Actualist Rationality

Axiomatic and Actualist Perspectives on Subjective Probability

Ellsberg on Ambiguity

The Quest for Rationality and the Search for Certitude

5 Diversified Treatment Choice 139

Diversification and Profiling

5.1 Allocating a Population to Two Treatments 141

The Welfare Function

A Status Quo Treatment and an Innovation

Expected Welfare

Maximin

Minimax Regret

Choosing Sentences for Convicted Juvenile Offenders

Allocation of Wealth to a Safe and Risky Investment

Risk-Averse Planning

5.2 Diversification and Equal Treatment of Equals 148

Ex Ante and Ex Post Equal Treatment

Combining Consequentialism and Deontological Ethics

5.3 Adaptive Diversification 150

Adaptive Minimax Regret

Implementation in Centralized Health Care Systems

The AMR Criterion and the Practice of Randomized Clinical Trials

Fraction of the Population Receiving the Innovation

Group Subject to Randomization

Measurement of Outcomes

5.4 Diversification across Time or Space 154

Diversification by Cohort

Laboratories of Democracy

5.5 Adaptive Partial Drug Approval 156

The Present Approval Process

Binary versus Partial Approval

Adaptive Partial Licensing

5.6 Collective Decision Processes 160

Majority-Rule Voting with Single-Peaked Preferences

The Credibility of Single-Peaked Preferences

Strategic Interactions

Learning and Heterogeneity of Policy Preferences

Bilateral Negotiations

Pareto Optimal Allocations

Incentive-Compatible Processes

Teacher Evaluation in New York City

5.7 Laissez-Faire 169

Social Learning from Private Experiences

Laissez-Faire Learning and Adaptive Diversification

6 Policy Analysis and Decisions 173

Institutional Separation of Analysis and Decisions

Doing Better

Appendix A Derivations for Criteria to Treat X-Pox 177

Appendix B The Minimax-Regret Allocation to a Status Quo Treatment and an Innovation 179

Appendix C Treatment Choice with Partial Knowledge of Response to Both Treatments 181

References 183

Index 193

Editorial Reviews

To academic readers steeped in [economics and decision theory], [Manski's] account is likely to be of some interest. It includes many useful and important insights (for example, the distinctions among policies based on the principles of 'maximin,' 'minimax,' and 'adaptive mini-max' regret) that have substantial implications for real-world policy.

Science - Brian Baird