What statistical evidence shows us about our misguided educational policies
Uneducated Guesses challenges everything our policymakers thought they knew about education and education reform, from how to close the achievement gap in public schools to admission standards for top universities. In this explosive book, Howard Wainer uses statistical evidence to show why some of the most widely held beliefs in education today—and the policies that have resulted—are wrong. He shows why colleges that make the SAT optional for applicants end up with underperforming students and inflated national rankings, and why the push to substitute achievement tests for aptitude tests makes no sense. Wainer challenges the thinking behind the enormous rise of advanced placement courses in high schools, and demonstrates why assessing teachers based on how well their students perform on tests—a central pillar of recent education reforms—is woefully misguided. He explains why college rankings are often lacking in hard evidence, why essay questions on tests disadvantage women, why the most grievous errors in education testing are not made by testing organizations—and much more.
No one concerned about seeing our children achieve their full potential can afford to ignore this book. With forceful storytelling, wry insight, and a wealth of real-world examples, Uneducated Guesses exposes today's educational policies to the light of empirical evidence, and offers solutions for fairer and more viable future policies.
Howard Wainer is distinguished research scientist at the National Board of Medical Examiners and adjunct professor of statistics at the Wharton School of the University of Pennsylvania. For twenty-one years, he was principal research scientist at Educational Testing Service. His many books include Picturing the Uncertain World: How to Understand, Communicate, and Control Uncertainty through Graphical Display and Graphic Discovery: A Trout in the Milk and Other Visual Adventures (both Princeton).
Table of Contents
Preface xi Introduction 1 Chapter 1: O n the Value of Entrance Exams: What Happens When the SAT Is Made Optional? 8 Chapter 2: O n Substituting Achievement Tests for Aptitude Tests in College Admissions 20 Chapter 3: O n Rigid Decision Rules for Scholarships 29 Chapter 4: The Aptitude-Achievement Connection: Using an Aptitude Test to Aid in Allocating Educational Resources 32 Chapter 5: C omparing the Incomparable: On the Importance of Big Assumptions and Scant Evidence 57 Chapter 6: O n Examinee Choice in Educational Testing 73 Chapter 7: What If Choice Is Part of the Test? 103 Chapter 8: A Little Ignorance Is a Dangerous Thing: How Statistics Rescued a Damsel in Distress 110 Chapter 9: A ssessing Teachers from Student Scores: On the Practicality of Value-Added Models 120 Chapter 10: S hopping for Colleges When What We Know Ain't 139 Chapter 11: O f CAT s and Claims: The First Step toward Wisdom 147 Epilogue 156 References 159 Index 165
"Uneducated Guesses is an insider's look at using test scores to make high stakes decisions in education. In this rigorous, refreshing rebuttal of conventional thinking, Wainer argues that in the world of education policy, we all would be better served by examining the evidence that demonstrates that our ideas will improve the systems we're trying to transform."—Dennis Van Roekel, president, National Education Association"With his usual verve, flair, and disdain for pious nonsense, Howard Wainer offers a refreshingly fact-based view of a complex problem: the use of tests in educational selection and evaluation. A must-read for anyone involved in these issues and a fun read for anyone who wishes to be educated and entertained at the same time."—Daniel Kahneman, Nobel Laureate in Economics, Princeton University"Howard Wainer's account of a selection of important scientific issues arising from educational testing is lucid, wise, and entertaining, and should be required reading for anyone interested in improving educational policy."—Stephen M. Stigler, University of Chicago"Uneducated Guesses is a must-read for enthusiasts of evidence-based decision making and for those who make public policy decisions without consulting the evidence. The former will be sobered by a real and random world that may not match their theoretical models. The latter will be surprised to learn from past research the power and limits of public policy decisions. Wainer lays it all out in engaging and accessible prose and numbers."—Arthur E. Wise, president emeritus, National Council for Accreditation of Teacher Education"Uneducated Guesses is a compelling, entertaining, and provocative book that elucidates some of the subtle and important issues of educational policy. In typical Wainer fashion, the graphics nicely complement and illuminate the text and tables, and I really enjoyed the variety of examples used in the book. I learned a lot about calibration, examinee choice, the history of testing, triathlon optimization, and the health of the professional football betting industry."—Nicholas Horton, Smith College"It is always a pleasure to read Wainer's work, and this book was certainly no exception. His carefully chosen and extremely interesting examples, his conversational tone, and his great sense of humor lead to a work that is hard to put down. Uneducated Guesses is a fabulous book, and one of great significance."—Karl W. Broman, University of Wisconsin-Madison
