The Power of Experiments: Decision Making in a Data-Driven World

by Michael Luca, Max H. Bazerman

Overview

How tech companies like Google, Airbnb, StubHub, and Facebook learn from experiments in our data-driven world—an excellent primer on experimental and behavioral economics
 
Have you logged into Facebook recently? Searched for something on Google? Chosen a movie on Netflix? If so, you've probably been an unwitting participant in a variety of experiments—also known as randomized controlled trials—designed to test the impact of different online experiences. Once an esoteric tool for academic research, the randomized controlled trial has gone mainstream. No tech company worth its salt (or its share price) would dare make major changes to its platform without first running experiments to understand how they would influence user behavior. In this book, Michael Luca and Max Bazerman explain the importance of experiments for decision making in a data-driven world.

Luca and Bazerman describe the central role experiments play in the tech sector, drawing lessons and best practices from the experiences of such companies as StubHub, Alibaba, and Uber. Successful experiments can save companies money—eBay, for example, discovered how to cut $50 million from its yearly advertising budget—or bring to light something previously ignored, as when Airbnb was forced to confront rampant discrimination by its hosts. Moving beyond tech, Luca and Bazerman consider experimenting for the social good—different ways that governments are using experiments to influence or “nudge” behavior ranging from voter apathy to school absenteeism. Experiments, they argue, are part of any leader's toolkit. With this book, readers can become part of “the experimental revolution.”

Product Details

ISBN-13:	9780262542272
Publisher:	MIT Press
Publication date:	03/02/2021
Series:	Mit Press
Pages:	232
Sales rank:	632,888
Product dimensions:	6.00(w) x 8.90(h) x 0.60(d)

About the Author

Michael Luca is Lee J. Styslinger III Associate Professor of Business Administration at Harvard Business School. His writing has appeared in publications including the Atlantic, Wall Street Journal, Harvard Business Review, and Slate.
Max H. Bazerman is Jesse Isidor Straus Professor of Business Administration at Harvard Business School. He is the author of The Power of Noticing and the coauthor of Blind Spots, Negotiation Genius, and other books.

Table of Contents

Preface vii

Acknowledgments xi

Part I Breaking Out of the Lab 1

1 The Power of Experiments 3

2 The Rise of Experiments in Psychology and Economics 17

3 The Rise of Behavioral Experiments in Policymaking 45

Part II Experiments in the Tech Sector 59

4 From the Behavioral Insights Team to Booking.com 61

5 #AirbnbWhileBlack 75

6 eBay's $50 Million Advertising Mistake 91

7 Deep Discounts at Alibaba 99

8 Shrouded Fees at StubHub 105

9 Market-Level Experiments at Uber 113

10 The Facebook Blues 121

Part III Experimenting for the Social Good 133

11 Behavioral Experiments for the Social Good 135

12 Healthy, Wealthy, and Wise 145

13 The Behavior Change for Good Project 161

14 The Ethics of Experimentation 173

15 A Final Case for Experiments and Some Concluding Lessons 177

Notes 189

Index 203

What People are Saying About This

From the Publisher

One of the great things about e-commerce is that it is far easier to run experiments online than offline. As more and more companies move online, they need to learn how to use this powerful tool. This book shows how to take advantage of experiments and how this will revolutionize business, both online and off.

—Hal Varian, Chief Economist, Google

This accessible and engaging book provides an excellent introduction to a subject that every young person entering the business world today should understand—experimentation. The case studies draw the reader into the challenges that arise in practice, highlighting issues ranging from bias to ethics to unintended consequences.

—Susan Athey, The Economics of Technology Professor, Stanford Graduate School of Business

Editorial Reviews

Driven by experimentation, a further transformation is in the air.”
—The New Yorker

“Luca and Bazerman balance their passion for experiments with a recognition of its limits.”
—Wall Street Journal
 
"For business economists, this book offers important insight into the use of experiments for organizations and their leadership. The benefits of experiments to an organization are the ability to test ideas, understand tradeoffs, and evaluate existing products and procedures. Experiments can also be a tool for discovery, with or without a theory to test. Leadership can use experiments to learn what they do not know, then adapt and grow their organizations with informed decisions."
—Business Economics

“'The Power of Experiments' is a terrific overview of how organizations, especially online companies but also government and other groups, can use A/B testing to optimize their results. Luca and Bazerman also discuss issues of ethics and consent that affect these efforts, perhaps limiting what some parties can or should do...an excellent book for managers and executives who oversee online marketing or other behavioral elements of an operation. While most professional marketers will be familiar with the concepts and examples presented in the book, Luca and Bazerman's easy style and approachable writing provide an excellent introduction or refresher for non-specialists at all levels. Highly recommended.”
—Technology and Society
 
“Filled with charming stories of experiments offering novel solutions to pressing social questions, 'The Power of Experiments: Decision Making in a Data-Driven World' is an enjoyable read that celebrates the power of experimentation to create social change.”
—Stanford Social Innovation Review
 
“'The Power of Experiments' is a fast, accessible read that offers a good overview of the applications, promise, and perils of experimentation. It's a good place for executives to start as they consider whether and how to use experiments inside their companies.”
—Strategy and Business
 
“One of the great things about e-commerce is that it is far easier to run experiments online than offline. As more and more companies move online, they need to learn how to use this powerful tool. This book shows how to take advantage of experiments and how this will revolutionize business, both online and off.”
—Hal Varian, Chief Economist, Google
 
“This accessible and engaging book provides an excellent introduction to a subject that every young person entering the business world today should understand—experimentation. The case studies draw the reader into the challenges that arise in practice, highlighting issues ranging from bias to ethics to unintended consequences.”
—Susan Athey, The Economics of Technology Professor, Stanford Graduate School of Business
 
“Leaders who create an environment for taking calculated risks get better results. This book reminds us that systemic experimentation and calculated risk taking has built great companies and great societies alike.”
—Deval Patrick, former Governor of Massachusetts
 
“Luca and Bazerman's The Power of Experiments will open your eyes about how to distill information from data.”
—Alvin E. Roth, Craig and Susan McCaw Professor of Economics, Stanford University; recipient of the 2012 Nobel Memorial Prize in Economics
 
“So often we don't know what we don't know until we try—and the best form of trying is often an experiment. This book is a brilliant tour through the past, present, and future of experimentation, packed full of insight and examples. Read it and learn—I did.”
—Tim Harford, author of The Undercover Economist and presenter of Cautionary Tales
 
“When testing and experimentation is at the heart of a culture, a team or company can be more responsive and nimble, driving a better product and more growth alongside a better employee experience.”
—Gillian Tans, Chairwoman, Booking.com
 
“A tremendous achievement, and inspiring to boot. Leaders in government and industry take note: Luca and Bazerman's book will guide you to better decisions in this age of experiments.”
—Cass R. Sunstein, Robert Walmsley University Professor, Harvard Law School; coauthor of Nudge
 
“Luca and Bazerman tell the fascinating story of the rise of experiments in the tech sector and beyond, and deliver a powerful message: experiments are transforming how businesses and governments make decisions. Going only by gut isn't enough—successful leaders need to move between data and decisions.”
—Charles Duhigg, author of The Power of Habit and Smarter Faster Better

“The authors have created a masterpiece, a manifesto showcasing the deepest revolution in the social sciences in the past twenty-five years—field experimentation. This book is a must-read for anyone interested in understanding policy, behavioral economics, technology, and life itself.”
—John A. List, Kenneth C. Griffin Distinguished Service Professor, University of Chicago

From the Publisher