Bad Data: Why We Measure the Wrong Things and Often Miss the Metrics That Matter

by Peter Schryvers

Overview

Highlights the pitfalls of data analysis and emphasizes the importance of using the appropriate metrics before making key decisions. Big data is often touted as the key to understanding almost every aspect of contemporary life. This critique of "information hubris" shows that even more important than data is finding the right metrics to evaluate it. The author, an expert in environmental design and city planning, examines the many ways in which we measure ourselves and our world. He dissects the metrics we apply to health, worker productivity, our children's education, the quality of our environment, the effectiveness of leaders, the dynamics of the economy, and the overall well-being of the planet. Among the areas where the wrong metrics have led to poor outcomes, he cites the fee-for-service model of health care, corporate cultures that emphasize time spent on the job while overlooking key productivity measures, overreliance on standardized testing in education to the detriment of authentic learning, and a blinkered focus on carbon emissions, which underestimates the impact of industrial damage to our natural world. He also examines various communities and systems that have achieved better outcomes by adjusting the ways in which they measure data. The best results are attained by those that have learned not only what to measure and how to measure it, but what it all means. By highlighting the pitfalls inherent in data analysis, this illuminating book reminds us that not everything that can be counted really counts.

Product Details

ISBN-13:	9781633885905
Publisher:	Prometheus Books
Publication date:	01/10/2020
Pages:	352
Product dimensions:	6.48(w) x 9.18(h) x 1.27(d)

About the Author

Peter Schryvers is an urban planner in Calgary, Alberta and is the founder of the Beltline Urban Mural Project. A Registered Planning Professional with a master's in Environmental Design, he is a member of the Canadian Institute of Planners.

Table of Contents

Introduction ix

Chapter 1 Teaching to the Test: Goodhart's Law and the Paradox of Metrics 1

Chapter 2 The Ins and Outs: The Logic Model and Program Evaluation 39

Chapter 3 The Long and Short of It: Intertemporal Problems and Undervaluing Time 65

Chapter 4 The Problem of Per: Denominator Errors 95

Chapter 5 The Forest and the Trees: Simplifying Complex Systems 117

Chapter 6 Apples and Oranges: Ignoring Differing Qualities 141

Chapter 7 Not Everything That Can Be Counted Counts: The Lamppost Problem 163

Chapter 8 Not Everything That Counts Can Be Counted: Measuring What Matters 221

Chapter 9 The Measure of Metrics 261

Chapter 10 Gateways Not Yardsticks 283

Acknowledgments 295

Notes 297

Index 315

Editorial Reviews

08/26/2019

Schryvers, a senior planner for the city of Calgary, offers a debut that challenges the accepted wisdom about a host of data-driven programs and convincingly argues that people and institutions often rely on the wrong info. Under his scrutiny, various measures—fee-for-service billing in medical settings; the corporate emphasis on short-term earnings; bibliometrics, the evaluation of academics based on how many publications they are cited in—all fail to provide reliable data for credibly enumerated reasons. Schryvers points to Goodhart’s law, which holds that “when a measure becomes a target, the measure ceases to be a good measure.” Examining other pitfalls of data collection and analysis, Schryvers notes that some things simply defy measurement; here he points to the Vietnam War, where the American military’s reliance on the infamous body counts, for example, failed to account for the insurmountable North Vietnamese commitment to victory. Schryvers’s insights will provide valuable tools to readers, enabling them to think critically about the myriad uses of data today. Agent: Jeff Shreve, the Science Factory. (Oct.)

Publishers Weekly