Read an Excerpt
CHAPTER 1
Introduction
Singular causal claims are primary. This is true in two senses. First, they are a necessary ingredient in the methods we use to establish generic causal claims. Even the methods that test causal laws by looking for regularities will not work unless some singular causal information is filled in first. Second, the regularities themselves play a secondary role in establishing a causal law. They are just evidence — and only one kind of evidence at that — that certain kinds of singular causal fact have happened.
— Nancy Cartwright
This book addresses some of the core issues in doing (1) multimethod research, (2) causal mechanism analysis, and (3) case studies. Multimethod research has become very popular and almost a requirement for book-length studies. Multimethod can mean many things, but here it means combining case studies with statistics, qualitative comparative analysis (QCA), experiments, or game theory models. The purpose of case studies is to explore causal mechanisms at the heart of theories. One does case studies because cross-case methods give little purchase on the causal mechanisms (Mi) by which X produces Y. For multimethod researchers, showing a significant causal effect in a cross-case analysis is not sufficient; one needs to provide a causal mechanism and evidence for it. Demonstrating a causal effect is only half the job; the second half involves specifying the causal mechanism and empirically examining it, usually through case studies.
Causal mechanisms, cross-case analyses, and case studies form the research triad; see figure 1.1. This volume rests on the proposition that commitment to multimethod research is commitment to the research triad. Multimethod research typically is conceptualized as qualitative — within-case inference — along with quantitative cross-case inference. The research triad adds a third dimension to that, i.e., causal mechanisms. The research triad is an integrated approach because a commitment to multimethod research is also a commitment to a causal mechanism approach to explanation and social science research.
One way to see the integration is by looking at research that focuses, e.g., 90 percent of the research effort, on just one corner of the triad. For many experimentalists most of the effort is devoted to determining the treatment effect. They usually do not talk about causal mechanisms per se. While they usually have theories and hypotheses, these all boil down to the one treatment effect. Game theorists provide an nice example of those who focus a lot of attention on the model, i.e., causal mechanism. It is quite possible to publish articles where essentially the whole article is the model (this is very true in economics). Often historians focus on single events and the whole focus of the article is explaining some individual historical event. They are not interested in generalization and maybe only very implicitly interested in causal mechanisms.
The integrated approach rests on a central claim:
As you move away from one-corner-only research you embrace the research triad.
For example, as a game theorist moves away from exclusive interest in the model, she begins to be involved in multimethod research, cross-case analyses, and within-case analyses. As soon as a case study researcher moves from one case to more cases, he is asking about generalization. Finally, as the Cartwright epigraph forcefully states, all statistical, experimental analyses and generalization imply individual case causal inference.
Good multimethod and causal mechanism research means a relative balance between the three corners of the research triad. If 90 percent of the effort is in demonstrating significant causal effects via cross-case analysis then the research is not serious multimethod or serious about the investigation of the causal mechanism. If the case studies are only "illustrations," then there is little commitment to multimethod research. Conversely, research using case studies — e.g., the popular paired comparison — is weak on generalization. Doing five or six case studies — as is common in security studies books — deals poorly with cross-case analysis and generalization. The ideal is a fairly balanced effort on all three points of the triangle.
Case studies are often considered of questionable inferential value. Clarke and Primo illustrate this view of case studies. When they refer to case studies they almost always describe them as "exploratory": "one can also design exploratory models with an eye toward explaining the events surrounding a specific case" (2012, 92). By "exploratory" they strongly imply the endeavor is not about causal inference. For example, their discussion of the analytic narratives project (Bates et al. 1998) falls into this category. When they talk about "empirical models" they mean statistical models. So a model of the US Senate — statistical or formal — is not exploratory, but a case study of the American Civil War in analytic narratives is exploratory. In contrast, the research triad emphasizes that case studies are about causal inference.
In everyday life — along with virtually all natural sciences — people successfully make individual-case causal inferences, e.g., origins of the universe, origins of the human species, why a given person died, why the Challenger shuttle exploded. None of these inferential successes relies on randomized treatments assigned to subjects, nor do they depend on conditional probabilities.
A core philosophy motivating this study is that we want to explain individual outcomes. Statistical analyses do not provide explanations: "There is little argument in political science that statistical models cannot serve as explanations in and of themselves. This belief manifests itself in the relegation of statistical models to devices for testing explanations" (Clarke and Primo 2012, 154).
The research triad assumes that one accepts the importance and value of causal mechanism analysis. One can find statistical methodologists who do not believe this is possible or important: "The importance of searching for causal mechanisms is often overestimated by political scientists, and this sometimes leads to an underestimate of the importance of comparing conditional probabilities. We do not need to have much or any knowledge about mechanisms in order to know that a causal relationship exists. ... In general, as our understanding of an issue improves, studying individual cases becomes less important" (Sekhon 2004, 288–89; last sentence of the article). Gerring illustrates the skepticism about whether causal mechanism analysis is essential: "To clarify, this is not a polemic against mechanisms. It is a polemic against a dogmatic interpretation of the mechanismic mission. I argue that the analysis of causal mechanisms is best regarded as an important, but secondary, element of causal assessment — not a necessary condition" (Gerring 2010, 1500).
I do not survey or discuss how one goes about doing within-case causal inference in this book. There is a booming literature on the topic, e.g., Bennett and Checkel (2014), and Beach and Pedersen (2012; 2016) on process tracing, Mahoney (2012) on hoop and smoking gun tests, Goertz and Levy (2007), Levy (2008), and Harvey (2011) on counterfactuals. One can do statistical within-case causal inference. I take no position on how one does within-case inference. Similarly I do not cover how to do observational statistics, experiments, or QCA. As illustrated in figure 1.1 these methodologies provide input for the research triad but are not covered here.
"Qualitative" and "quantitative" are not very useful in describing or analyzing multimethod research. Instead of multi-method research as qualitative and quantitative, the research triad contrasts within-case causal inference (case studies) with cross-case causal inference (comparative case studies, statistical models, experiments, or QCA). This produces some surprising methodological bedfellows. Standard usage puts statistics in the quantitative category and set-theoretic approaches (e.g., QCA) in the qualitative. Experimenters spend a lot of time stressing the differences between experiments and observational research: here they are both cross-case methodologies. Similarly comparative case studies are cross-case analyses. I consider these all as versions of cross-case causal inference. In contrast, single case studies by their very nature are about what happens in individual cases. Case studies are fundamentally about within-case causal inference. The research triad means that multimethod research is multicausal inference analysis. The causal inference techniques, procedures, and methodology of each type, cross-case and within-case, serve different but complementary goals.
The research triad works from a basic principle:
Multimethod work involves cross-case causal inference AND within-case causal inference.
Multimethod in this book means complementary causal inference methodologies. How one does cross-case inference or within-case inference is less important than the causal inference goals.
To connect cross-case and within-case analysis means having a methodology for choosing cases for causal mechanism analysis. The practical problem of choosing cases runs as a bright red thread throughout the book. These decisions face all who connect case studies to other methodologies. In particular, I offer much specific guidance about case selection. This means a systematic set of guidelines for case selection including a list of criteria for getting to a final decision about which cases to choose.
McGuire (2010) illustrates typical multimethod research involving a statistical analysis and case studies. His dependent variable is health outcomes. The second chapter of his book is a large-N cross-national statistical analysis, which is followed by eight country case studies. These case studies focus on explaining outcomes in those countries, for example, the following:
What needs to be explained, then, is why Costa Rica's infant mortality rate was so low in 2005 (why it attained a certain level); why it fell so fast from 1960 to 2005 (why it achieved a certain amount of progress), and why it fell faster during the 1970s than at other times within this period (why it evolved at a certain tempo). The sustained and effective public provision of basic health services to the poor, this chapter finds, goes a long way toward explaining why Costa Rica from 1960 to 2005 achieved a rapid decline (and, eventually, a low level) of infant mortality. (McGuire 2010, 66)
McGuire is making claims about what happened in Costa Rica and why it happened, i.e., within-case causal inference.
The standard rationale for multimethod work involves looking at causal mechanisms via case studies: "Despite some claims to the contrary in the qualitative methods literature, case studies are not designed to discover or confirm empirical regularities. However they can be quite useful — indeed, essential — for ascertaining and assessing the causal mechanisms that give rise to empirical regularities in politics" (Fearon and Laitin 2008, 773). This is exactly what the research triad proposes: cross-case analyses for "empirical regularities" and case studies for causal mechanisms.
McAdam and Boudet give a similar rationale in their study of environmental social movements: "We conceived of the project as an attempt to develop an alternative to the methodological conventions of social movement research. Equally dissatisfied with 'thin,' large-N studies of protest events and rich but nongeneralizable case studies of this or that movement, we sought a middle ground between these two modal 'poles' of social movement scholarship" (2012, 52).
Pevehouse (2005) provides a nice example of why people want to do multimethod research. He argues that democratic IGOs (intergovermental organizations) can help establish democracy, make it more robust, and encourage transitions to democracy. As illustrated in figure 1.2a, there is a causal connection proposed between democratic IGOs and democracy in states. He shows that there is a significant correlation between democraticness of the IGO and democracy in its member states. Multimethod research comes into play because he thinks there are multiple — and not mutually exclusive — causal mechanisms that explain this significant correlation. Figure 1.2b adds the causal mechanisms that produce this statistical effect: (1) acquiescence effect, (2) legitimization, (3) pressure, and (4) financial assistance (Pevehouse 2005, table 5.2, 153). Here one sees Pevehouse going around the research triad, from the statistical analyses to case studies to causal mechanism analysis.
Many hypotheses, experiments, and the like propose multiple causal mechanisms connecting the treatment to the outcome, as illustrated in figure 1.2. For example, Helfer and Voeten list three causal mechanisms whereby the European Court of Human Rights influences state policy: (1) preempting future international court litigation, (2) persuasive authority, and (3) agenda-setting at the national level. They argue — like Pevehouse — that "these three mechanisms may work separately or in tandem" (Helfer and Voeten 2014, 82). Hence, one role of case studies is to explore which of these mechanisms is actually at work.
A central role of case studies is combining within-case causal inference with analyses of causal mechanisms. These two in fact cannot be separated. To do process tracing, for example, means to make causal claims about the case. Usually it means that the causal mechanism in question can explain or contributes to the outcome in the individual case. Some researchers offer case studies as "illustrations"; this is disingenuous. Good multimethod research means that one must be serious about the within-case causal claims and make sure that they stand up to scrutiny.
Why after all is one doing a case study in multimethod research? The answer is almost always "to explore causal mechanisms." This is why the research triad is integrated and tightly linked. Doing multimethod research means doing case studies, which means exploring causal mechanisms. Once you have started down the multimethod path you have to pass by causal mechanisms.
Causal mechanism by definition means causal complexity (see chapter 2). A commitment to multimethod research and the research triad is a commitment to causal complexity. Causal mechanisms involve causal complexity. The contrast is with those who are almost exclusively interested in the effect of X on Y. As the various figures in chapter 2 illustrate, causal mechanisms are complex entities. Most often the easiest — and maybe the only practical — way to empirically explore mechanisms is process tracing in individual cases.
One variable, single treatment analyses are by contrast causally simple. The classic examples are single-variable hypotheses empirically investigated via statistical analyses or experiments. At the other extreme are game-theoretic models, which often have numerous assumptions and complex features. Even simple game-theoretic models involve multiple assumptions about individuals, their beliefs, rules of the game, etc.
A quite popular move for game theorists is to move to the cross-case corner of the research triad. There have been extensive efforts — with substantial NSF (National Science Foundation) funding — to connect game theory and statistics, hence game theory and statistical cross-case inference. Notably the EITM program (Granato and Scioli 2004) has held summer workshops for a number of years where the central component has been connecting statistical methods with game-theoretic models. However, there has been very little done to connect game-theoretic models with qualitative methods and case studies (though see Lorentzen, Fravel, and Paine 2016), hence game theory and within-case causal inference.
Chapter 6 argues that there is a very natural connection between game theory and within-case causal inference. In addition to the cross-case analyses, one moves in the within-case inference direction. Many — if not almost all — of the crucial theoretical entities in game-theoretic models are hard to observe and measure in large-N settings. Factors like beliefs, information, uncertainty, and preferences are hard to determine even in one case, not to mention dozens or hundreds. As a result, statistical tests are virtually always indirect. For example, in the audience costs literature and debate (discussed in some detail in chapter 7) the usual proxy of audience costs is democracy. "Democracy" is some distance away from the theoretical mechanism developed by Fearon (1994). In contrast, there is some hope of assessing core model features in one or a few individual cases.
(Continues…)
Excerpted from "Multimethod Research, Causal Mechanisms, and Case Studies"
by .
Copyright © 2017 Princeton University Press.
Excerpted by permission of PRINCETON UNIVERSITY PRESS.
All rights reserved. No part of this excerpt may be reproduced or reprinted without permission in writing from the publisher.
Excerpts are provided by Dial-A-Book Inc. solely for the personal use of visitors to this web site.
