Developmental Evaluation: Applying Complexity Concepts to Enhance Innovation and Use

by Michael Quinn Patton PhD

Overview

Developmental evaluation (DE) offers a powerful approach to monitoring and supporting social innovations by working in partnership with program decision makers. In this book, eminent authority Michael Quinn Patton shows how to conduct evaluations within a DE framework. Patton draws on insights about complex dynamic systems, uncertainty, nonlinearity, and emergence. He illustrates how DE can be used for a range of purposes: ongoing program development, adapting effective principles of practice to local contexts, generating innovations and taking them to scale, and facilitating rapid response in crisis situations. Students and practicing evaluators will appreciate the book's extensive case examples and stories, cartoons, clear writing style, "closer look" sidebars, and summary tables. Provided is essential guidance for making evaluations useful, practical, and credible in support of social change.

See also Developmental Evaluation Exemplars, edited by Michael Quinn Patton, Kate McKegg, and Nan Wehipeihana, which presents 12 in-depth case studies.

Product Details

ISBN-13:	9781606238721
Publisher:	Guilford Publications, Inc.
Publication date:	06/15/2010
Pages:	375
Product dimensions:	6.90(w) x 9.90(h) x 0.80(d)

About the Author

Michael Quinn Patton, PhD, is an independent consultant who has been conducting program evaluations since the 1970s. Based in Minnesota, he was on the faculty of the University of Minnesota for 18 years and is a former president of the American Evaluation Association (AEA). His books include Blue Marble Evaluation, Developmental Evaluation, Developmental Evaluation Exemplars, Principles-Focused Evaluation, Qualitative Research and Evaluation Methods (now in its fourth edition), and Utilization-Focused Evaluation (now in its fourth edition), among others. He is a recipient of the Alva and Gunnar Myrdal Evaluation Practice Award and the Paul F. Lazarsfeld Evaluation Theory Award, both from AEA, as well as the Lester F. Ward Distinguished Contribution to Applied and Clinical Sociology Award from the Association for Applied and Clinical Sociology. Dr. Patton is an active trainer and workshop presenter who has conducted applied research and evaluation on a broad range of issues and has worked with organizations and programs at the international, national, state, provincial, and local levels.

 

Table of Contents

List of Exhibits xix

1 Developmental Evaluation Defined and Positioned 1

The Distinction Emerges 2

Facing Complexity and Facing Reality: Or, Facing the Realities of Complexity 4

Developmental Evaluation and Complexity Theory 7

Developmental Evaluation and Single-Loop versus Double-Loop Learning 11

The Importance of Interpretive Frameworks 12

Developmental Evaluation as Utilization-Focused 13

Charting the Developmental Journey: Overview of the Book 17

Five Developmental Evaluation Purposes and Uses 21

2 Developmental Evaluation as a Distinct Purpose and Niche 28

Beyond Planning: Jumping into the Fire 29

Evaluation Questions under Different Engagement Scenarios 30

Vision Encounters Reality: The Territory of Developmental Evaluation 30

Systems Issues: Boundary Management as a Developmental Evaluation Focus 34

Developing Innovations versus Improving and Testing Models: Developmental Evaluation Distinguished from Formative and Summative Evaluation 36

Ongoing Development Informed by Developmental Evaluation 41

Strategic Thinking and Developmental Evaluation 42

The Niche of Developmental Evaluation: Chapter Overview and Summary 51

3 Thinking Outside Evaluation's Boxes 53

Creative Evaluation 53

Of Plums and Prunes 54

History and Biography as Context: The Personal Factor 55

More History and Biography as Context 58

Fundamental Principles 59

Developmental Evaluation in Context 60

External Evaluators and Developmental Evaluation 64

Ten Key Points about Developmental Evaluation Illustrated by the Caribbean Example 75

4 Situation Recognition and Responsiveness: Distinguishing Simple, Complicated, and Complex 80

Detecting Patterns 80

Detecting Complexity as a Distinct Pattern and Territory for Inquiry 81

The Challenges of Situation Recognition 84

A Situation Recognition Heuristic: Distinguishing Simple, Complicated, and Complex 84

Situational Evaluation: Implications for Practice 97

Variations on a Theme: The Cynefin Framework 106

5 Systems Thinking and Complexity Concepts for Developmental Evaluation 111

From Train Brakeman to Locomotive Engineer to Evaluation Champion 112

Project SleepBetter 115

Systems Thinking 117

Complexities of Complexity 122

Useful Complexity Constructs for Developmental Evaluation 124

Complexity-Sensitizing Concepts 146

6 How the World Is Changed: A Dialectic with Thesis and Antithesis and Developmental Evaluation as the Synthesis 152

Competing Perspectives on How the World Is Changed 153

The Top-Down Approach 155

Adaptation versus Replication: Principles for Developmental Evaluation 160

Best Practices versus Effective Principles 167

From the Grass Roots to the Adaptive Middle 167

Alternative Approaches to Change 176

Developmental Evaluation: The Action in the Muddled Middle 177

Looking Back and Looking Forward 184

Appendix 6.1 Positioning and Contrasting Developmental Evaluation with Other Evaluation Approaches 187

7 The Adaptive Cycle and Developmental Evaluation 189

Some Context for Understanding the Adaptive Cycle: Poetry, Ecology, Sociology, Business, and Evaluation 190

The Complexity Doorframe Redux 193

Five Developmental Evaluation Purposes and Uses 194

Resilience and Developmental Evaluation 196

A Strategic Approach to Resilience: Engineering Resilience versus Ecosystem Resilience 198

The Adaptive Cycle 202

The Adaptive Cycle and Psychosocial Regimes 204

Evaluation and the Adaptive Cycle 206

Through the Looking Glass of the Adaptive Cycle: Examples of Organizational and Program Cycles 210

Transitions and Traps 212

Taking an Innovation to Scale: Challenges and Traps in Cross-Scale Change 218

Panarchy: Cross-Scale Interactions 220

Panarchy-Informed Evaluation: The Fact of a Doorframe 225

8 Developmental Evaluation Inquiry Frameworks 227

Frameworks for Developmental Evaluation Inquiry 228

Developmental Evaluation Inquiry Frameworks: Ten and Counting 261

9 Developmental Evaluation Bricolage: Reflective Practice, Sensitizing Concepts, Action Research, Abduction, Systems Change, and Retrospective Developmental Evaluation 264

Reflective Practice for Developmental Evaluation Inquiry and Engagement 265

Sensitizing Concepts as a Focus for Reflective Practice and Developmental Evaluation 270

Kia ora 274

Action Research and Developmental Evaluation 280

Intersections of Developmental Evaluation and Action Research: A Distinguished Evaluation Pioneer Reflects 280

Reasoning and Developmental Evaluation 283

In Praise of Methodological Diversity and Appropriateness: A Methods and Tools Rant 288

Systems Change and Developmental Evaluation 290

Retrospective Developmental Evaluation 294

Damiano's Retrospective Developmental Evaluation 298

Developmental Evaluation Bricolage 303

10 Utilization-Focused Developmental Evaluation: Engagement Practices, Diverse Designs, and Adaptive Methods 305

Different Kinds of Developmental Evaluation 306

Diversity of Design and Methods Options 307

Utilization-Focused Developmental Evaluation 315

Ten Utilization-Focused Developmental Evaluation Design Examples 315

Developmental Evaluation Engagement, Design, and Methods Summary 333

The First Developmental Evaluation: A Creation Story 337

References 341

Author Index 355

Subject Index 361

About the Author 375

Interviews

Instructors and graduate students in education, psychology, sociology, management, social work, nursing, and public policy; applied researchers who do evaluations. Will serve as a core or supplemental text in graduate-level courses in evaluation, social research methods, program planning, and management consulting.

Editorial Reviews

"Patton is the originator of developmental evaluation, and his decades of experience in the broader context of utilization-focused evaluation...radiate throughout the text, providing a rich source for evaluators at all levels of expertisethe reader is in good hands....A rich text, brimming with Patton's years of experience and the wisdom of his esteemed colleagues....Patton writes with emotion and effectively meshes storytelling with fundamentals to deepen the reader's understanding of developmental evaluation. For newcomers to the field, experienced evaluators, and social innovators alike, this book can serve to instill excitement about program evaluation and a broadened sense of its usefulness for creating social change."

Journal of Community and Applied Social Psychology

"Draws together sophisticated concepts and models from an extraordinary array of disciplines, such as ecology, business, sociology, and international development. It includes the work of thought leaders and practitioners from around the globe and is explicit about the need for more collaboration and input to continuously inform and develop this evaluation framework. Seasoned readers of Patton's earlier works will instantly recognize the classic Patton style, complete with rich narrative stories, wry wit, and clever cartoonstrappings purposefully used to not only engage the reader, but also bring about clearer understanding and meaning....This book is a must-read for any evaluator, as it will change the way you see the world around you and how you view and conduct your practice."

Canadian Journal of Program Evaluation

Patton's goal for this book is to teach evaluators about development evaluation (DE), a groundbreaking, complexity-based evaluation approach that aims to help social innovators working on extremely difficult problems in complex, dynamic environments....Patton uses engaging stories, conceptual frameworks, side-by-side comparison tables, extended evaluation case studies, menus of evaluation questions and methods, and cartoons to define, explain, and illustrate this new approach to evaluation. Dense with information and layers of insight....For evaluators who encounter situations that do not lend themselves to either formative or summative evaluation or who are working with social innovators in both program development and evaluation roles, Developmental Evaluation is invaluable.—American Journal of Evaluation

American Journal of Evaluation

"Proust wrote that 'the real voyage of discovery consists not in seeking new landscapes but in having new eyes.' In this book, Patton brings new eyes to evaluation landscapes. He illustrates the distinct contribution that developmental evaluation can make in addressing the dynamic complexity that often challenges evaluation efforts. Evaluators will see themselves among the stories Patton shares. The concepts and ideas are accessible and the case examples provide a diverse array of teachable vignettes, making the book ideal for classroom use. This is a most enjoyable read that offers lots of new learning, even for an evaluation veteran!"—Ann M. Doucette, PhD, Director, The Evaluators’ Institute, The George Washington University

"There is a real hunger for this book among social innovators, funders, policymakers, and educators. The book is sure to become dog-eared as it is read, used, and reread to help evaluators conduct their work in a manner consistent with the complexity of the challenges they are addressing."—Brenda Zimmerman, PhD, Director, Health Industry Management Program, Schulich School of Business, York University, Canada

"In true Michael Quinn Patton style, this book successfully extends both the theory and practice of evaluation in significant and timely ways. Solutions to the world’s most pressing social problems are neither predictable nor known; developmental evaluation is just what the field needs to evaluate the complex realities of today’s organizations and communities. This book is a 'must read' for anyone committed to understanding how, where, when, and for whom social innovations are achieving their goals."—Hallie Preskill, PhD, Executive Director, Strategic Learning and Evaluation Center, FSG Social Impact Advisors, Seattle, Washington
 
"Patton pulls back the curtain to reveal that there is no great Oz of evaluation. This book reminds us that when we are working in complex systems we are better off acknowledging how little we know from the outset—and then acting on continual feedback—rather than pretending we already have all the knowledge needed to succeed. Patton challenges evaluators to relentlessly adapt, react, change, and innovate to work toward the best outcomes."—John B. Bare, PhD, Vice President, The Arthur M. Blank Family Foundation, Atlanta, Georgia

From the Publisher