Public Program Evaluation: A Statistical Guide / Edition 2

by Laura Langbein

Overview

This readable and comprehensive text is designed to equip students and practitioners with the statistical skills needed to meet government standards regarding public program evaluation. Even those with little statistical training will find the explanations clear, with many illustrative examples, case studies, and applications.

Far more than a cookbook of statistical techniques, the book begins with chapters on the overall context for successful program evaluations, and carefully explains statistical methods—and threats to internal and statistical validity—that correspond to each evaluation design. Laura Langbein then presents a variety of methods for program analysis, and advise readers on how to select the mix of methods most appropriate for the issues they deal with— always balancing methodology with the need for generality, the size of the evaluator's budget, the availability of data, and the need for quick results.

Product Details

ISBN-13:	9780765626127
Publisher:	Taylor & Francis
Publication date:	07/15/2012
Pages:	264
Product dimensions:	6.90(w) x 9.90(h) x 0.60(d)

Table of Contents

Preface ix

1 What This Book Is About 3

What Is Program Evaluation? 3

Types of Program Evaluations 8

Basic Characteristics of Program Evaluation 13

Relation of Program Evaluation to the General Field of Policy Analysis 15

Assessing Government Performance: Program Evaluation and Performance Measurement 15

A Brief History of Program Evaluation 17

What Comes Next 19

Key Concepts 20

Do It Yourself 20

2 Defensible Program Evaluations: Four Types of Validity 26

Defining Defensibility 26

Types of Validity: Definitions 27

Types of Validity: Threats and Simple Remedies 28

Basic Concepts 47

Do It Yourself 48

3 Internal Validity 51

The Logic of Internal Validity 51

Making Comparisons: Cross Sections and Time Series 54

Threats to Internal Validity 55

Summary 63

Three Basic Research Designs 64

Rethinking Validity: The Causal Model Workhorse 66

Basic Concepts 68

Do It Yourself 69

A Summary of Threats to Internal Validity 70

4 Randomized Field Experiments 73

Basic Characteristics 73

Brief History 74

Caveats and Cautions About Randomized Experiments 76

Types of RFEs 79

Issues in Implementing RFEs 92

Threats to the Validity of RFEs: Internal Validity 96

Threats to the Validity of RFEs: External Validity 100

Threats to the Validity of RFEs: Measurement and Statistical Validity 101

Conclusion 101

Some Cool Examples of RFEs 102

Basic Concepts 103

Do It Yourself: Design a Randomized Field Experiment 104

5 The Quasi Experiment 110

Defining Quasi-Experimental Designs 110

The One-Shot Case Study 111

The Posttest-Only Comparison-Group (PTCG) Design 113

The Pretest-Posttest Comparison-Group (PTPTCG) (The Nonequivalent Control-Group) Design 119

The Pretest-Posttest (Single-Group) Design 123

The Single Interrupted Time-Series Design 125

The Interrupted Time-Series Comparison-Group (TTSCG) Design 131

The Multiple Comparison-Group Time-Series Design 134

Summary of Quasi-Experimental Design 135

Basic Concepts 136

Do It Yourself 137

6 The Nonexperimental Design: Variations on the Multiple Regression Theme 143

What Is a Nonexperimental Design? 143

Back to the Basics: The Workhorse Diagram 144

The Nonexperimental Workhorse Regression Equation 146

Data for the Workhorse Regression Equation 148

Interpreting Multiple Regression Output 149

Assumptions Needed to Believe That b Is a Valid Estimate of B [E(b) = B] 164

Assumptions Needed to Believe the Significance Test for b 184

What Happened to the R²? 190

Conclusion 191

Basic Concepts 192

Introduction to Stata 194

Do It Yourself: Interpreting Nonexperimental Results 197

7 Designing Useful Surveys for Evaluation 209

The Response Rate 210

How to Write Questions to Get Unbiased, Accurate, Informative Responses 217

Turning Responses into Useful Information 224

For Further Reading 233

Basic Concepts 233

Do It Yourself 234

8 Summing It Up: Meta-Analysis 239

What Is Meta-Analysis? 239

Example of a Meta-Analysis: Data 240

Example of a Meta-Analysis: Variables 241

Example of a Meta-Analysis: Data Analysis 242

The Role of Meta-Analysis in Program Evaluation and Causal Conclusions 243

For Further Reading 244

Index 247

About the Author 253