eBook
Available on Compatible NOOK devices, the free NOOK App and in My Digital Library.
Related collections and offers
Overview
Seven Rules for Social Research teaches social scientists how to get the most out of their technical skills and tools, providing a resource that fully describes the strategies and concepts no researcher or student of human behavior can do without.
Glenn Firebaugh provides indispensable practical guidance for anyone doing research in the social and health sciences today, whether they are undergraduate or graduate students embarking on their first major research projects or seasoned professionals seeking to incorporate new methods into their research. The rules are the basis for discussions of a broad range of issues, from choosing a research question to inferring causal relationships, and are illustrated with applications and case studies from sociology, economics, political science, and related fields. Though geared toward quantitative methods, the rules also work for qualitative research.
Seven Rules for Social Research is ideal for students and researchers who want to take their technical skills to new levels of precision and insight, and for instructors who want a textbook for a second methods course.
The Seven Rules
- There should be the possibility of surprise in social research
- Look for differences that make a difference, and report them.
- Build reality checks into your research.
- Replicate where possible.
- Compare like with like.
- Use panel data to study individual change and repeated cross-section data to study social change.
- Let method be the servant, not the master.
Product Details
| ISBN-13: | 9780691190433 |
|---|---|
| Publisher: | Princeton University Press |
| Publication date: | 06/26/2018 |
| Sold by: | Barnes & Noble |
| Format: | eBook |
| Pages: | 288 |
| File size: | 2 MB |
About the Author
Read an Excerpt
Seven Rules for Social Research
By Glenn Firebaugh Princeton University Press
Copyright © 2008 Princeton University Press
All right reserved.
ISBN: 978-0-691-12546-6
Chapter One The First Rule
THERE SHOULD BE THE POSSIBILITY OF SURPRISE IN SOCIAL RESEARCH
Social research differs fundamentally from advocacy research. Advocacy research refers to research that sifts through evidence to argue a predetermined position. Lawyers engage in advocacy research in attempting to prove the guilt or innocence of defendants. Special interest groups engage in advocacy research when trying to influence the vote of lawmakers on a particular bill. The aim of advocacy research is to convince others (juries, lawmakers) to think or act in a given way. Hence advocacy research presents evidence selectively, focusing on supporting evidence and suppressing contrary or inconvenient evidence.
Social research, in contrast, does not suppress contrary or inconvenient evidence. In the social sciences we may begin with a point of view, but (unlike advocacy research) we do not want that point of view to determine the outcome of our research. Rule 1 is not intended to suggest that you need to check your idealism at the door when you embark on a social research project. Far from that-if you don't want to change the world for the better, you might not have the doggedness needed to do good social research. But rule 1 is intended to warn that you don't want to be blinded by preconceived ideas so that you fail to look for contraryevidence, or you fail to recognize contrary evidence when you do encounter it, or you recognize contrary evidence but suppress it and refuse to accept your findings for what they appear to say.
You must be ready to be surprised by your research findings. It is the uncertainty that makes social research exciting and rewarding. If you always knew the answer beforehand, what is the point of doing the research?
For first-time researchers, the surprise very often is in the finding of small effects where large effects were expected. Rarely do new researchers obtain findings that were as strong as they anticipated. Sometimes results are even opposite to expectations. Perhaps you were absolutely certain, for example, that in the United States women are more likely to approve of abortion than men are (not true: see exercises at the end of this chapter), or that younger adults tend to be happier than older adults are (which is also false: see exercises at the end of chapter 2).
Although you might at first be disappointed in your results, it is important to keep in mind that a noneffect is not a nonfinding. Any result is a finding. Finding no effect could be more interesting than finding a big effect, especially when conventional wisdom holds that there is a big effect. Often the most interesting results in social research are those that fly in the face of conventional wisdom. So you should not be disappointed when you do not find the big effect you expected, or when your results are inconsistent with "what everyone knows."
Selecting a Research Question
The first step in a research project is to decide upon a research question. A research question in the social sciences is a question about the social world that you try to answer through the analysis of empirical data. The empirical data may be data that you collected yourself, or data collected by someone else.
There are two fundamental criteria for a research question: interest and researchability. In selecting a research question, your aim is to find a question that is (1) researchable and (2) interesting to you and to others in your field.
Researchable Questions
Some research questions in the social sciences are not researchable because they are simply unanswerable (see Lieberson 1985, chap. 1). In that respect economics, political science, psychology, and sociology are the same as astronomy, biology, chemistry, and physics. Some questions-such as the existence of God-are inherently unknowable with the scientific method. Other questions might be unanswerable as the questions are currently conceived. Other questions might be unanswerable with the knowledge and methods we currently have.
If you are a student, some questions that are answerable in principle might nonetheless be beyond your reach. The Minneapolis Domestic Violence Experiment, for example, sought to determine whether repeat incidents of domestic violence would decline if police arrested accused abusers on the spot (Sherman 1992; Sherman and Berk 1984). Typically students do not have the time, resources, or credentials required to carry out an experiment of this magnitude. The possibilities for student projects are not as restricted as one might think, however, because there are a surprising number of data sets that are available to students for analysis (the student exercises at the end of this chapter use just such a data set, the General Social Survey). Even if you plan to collect your own survey data, you should consult such prior surveys early in your project, if only for guidance on how to word your questions.
Other research is ruled out because it is unethical. To jump ahead to an example that is central in a subsequent discussion of causality (chapter 5), we did not know, until a few decades ago, whether or not smoking causes lung cancer-some experts said it did, some said that the evidence was inconclusive. Scientifically, the best way to settle the dispute would have been to assign individuals randomly to one of two groups, a smoking group and a nonsmoking group. In terms of science, then, the question could be answered by doing an experiment. In practice, however, such an experiment would have raised serious ethical questions regarding an individual's right to smoke (or not smoke). The important point here is that ethical considerations are important-important enough to render some types of research undoable.
What, then, does a researchable question look like? Researchable questions in the social sciences tend to be questions that are neither too specific (for example, about a specific individual or event) nor too grand ("Why are there wars?"). The question "Why is my uncle an alcoholic?" is not a good research question because in the final analysis it is impossible to predict the behavior of a given individual; social laws are not deterministic. (Besides, the question isn't very interesting to anyone outside your family.) It is possible to make that question researchable by generalizing it to alcoholics in general. We could ask, for example, why some people tend to be more prone to alcoholism than others are. To address that question, we first ask what characteristics distinguish alcoholics from nonalcoholics. In other words, we ask what individual or community traits correlate with alcoholism.
In the social sciences what questions generally are easier to answer than why questions. As Lieberson (1985, p. 219, italics removed) puts it, "Empirical data can tell us what is happening far more readily than they can tell us why it is happening." Consider alcoholism again. With appropriate data it is a straightforward matter to find the correlates of alcoholism, that is, to find the characteristics that distinguish alcoholics from others. Whether these correlates are causes of alcoholism is another matter (we take up the causality question under rule 5, "Compare like with like"). But that does not make the correlates uninformative. Even if we want to determine causes, generally the first step is to find correlates, since knowing the what gives us clues about the why.
Because "mere description" is sometimes devalued in the social sciences, it is important to underscore the point that "what" questions come before "why" questions. Facts come first, as Sherlock Holmes stated famously in his warning to Dr. Watson that "It is a capital mistake to theorize in advance of the facts" ("The Adventure of the Second Stain"). To punctuate the detective's point, later I describe an instance where social theorizing went astray by attempting to account for "rising global inequality" when in fact global inequality was not rising. Obviously we need to get the facts right about what there is to explain before we concoct an explanation. As the first maxim of Galileo's Discors states, "description first, explanation second" (see Pearl 2000, pp. 334-35).
Getting the facts right is critical, second, because accurate description sometimes is itself the end goal. Consider the problem of motion. Medieval attempts to understand motion grappled with the nature and origins of motion. Current understanding takes motion as given and attempts to describe it precisely and accurately. As Andrew Abbott (2003, p. 7) points out:
[Isaac Newton] solved the problem of motion by simply assuming that (a) motion exists and (b) it tends to persist. By means of these assumptions (really a matter of declaring victory, as we would now put it), he was able to develop and systematize a general account of the regularities of motion in the physical world. That is, by giving up on the why question, he almost completely answered the what question.
In this instance progress was stalled until the question was switched from the unknowable (why things move) to the knowable (how they move). We have returned, then, to the central point of this section: that a research question must first of all be answerable. The second requirement is that it is interesting.
Interesting Questions
Besides finding a question that is answerable, you want a question that is interesting-to you and to others. Research involves entering into a conversation. The issue is how your research will contribute to that conversation. A research question might be interesting because of its scientific relevance: In addressing this issue, you join an ongoing discussion in the social sciences. Or a question might be interesting because it is socially important; perhaps it bears on some important local or national social problem. Or a question might be interesting because of its timeliness; a question relating to Americans' attitudes about gun control might be particularly interesting, for example, when important legislation is pending on the subject.
"The heart of good work," Abbott (2003, p. xi) writes, "is a puzzle and an idea." An interesting research question encompasses both. The puzzle is the issue about the social world that bears investigation. The idea is the new twist that you bring to the investigation.
How do you find an interesting research question? Often personal experiences, or the experiences of others we know, provoke our curiosity about some aspect of human behavior. You can also obtain ideas by reading the research of others. New findings often lead to new questions, and research reports often conclude by noting further research that is needed, or by suggesting directions that subsequent research might take.
Unfortunately, there is no foolproof recipe for cooking up an interesting research question. In that light, it is important to keep in mind that the ideas below are intended to stimulate your thinking, not to provide a roadmap to discovery. No one knows where lightning will strike, but it is still a good idea to avoid sitting on a high metal roof in a thunderstorm. Inspiration is the same way: We cannot be certain where it will strike, but we know that the probabilities are greater in some places than in others.
If you have trouble thinking of a good question for a research project, most likely the problem is either that you have an idea that you like but others don't, or you cannot come up with an idea in the first place.
Suppose you have an idea for a project that enthuses you but no one else. One reaction-perhaps most common among students doing their first research project involving actual data analysis-is to ignore what others, such as your professors, think. That is generally not a good idea. The interest criterion for a research question dictates that you find an issue that is interesting to you and to others in your field. The "others in your field" requirement is important because the "so what?" question is the first challenge you will face when writing up the results of your research. Early in the report you must be able to explain, in a few sentences, why your results are of interest to your readers. Perhaps the question you address relates to an ongoing discussion in your field. Perhaps it extends prior research in new directions, or replicates findings using a new population. Or perhaps it sheds light on an empirical puzzle. The point is that you need some rationale for doing your research. The fact that you cannot get anyone else interested in your research is a telltale sign that you need to rethink the project. Rethinking the project does not necessarily mean that you need to abandon your idea. But it does mean that you need to rework your question to make it interesting to others.
The issue, then, is how to revise your research question to make it more interesting to others. (I assume here that you have done the necessary preliminary review of other research on the topic, so you have a basic idea of what others have found before you.) First you should discuss the project with others to find out why it is uninteresting to them. Most likely they will say that your question is uninteresting either because we already know the answer, or because we would not care about the answer even if we knew it. That is:
The no-surprises objection: We won't learn anything from the research because the answer either is already well documented by prior research or is preordained by the research method. In other words, the research is gratuitous, since we know the answer before we do the research. The "so what" objection: The answer to the research question either has no relevance for social science theory or for everyday life, or matters to such a small handful of people that the question is trivial.
"Why is my uncle an alcoholic?" is a good example of a trivial research question. No matter how important the answer might be to you, the question is trivial to social scientists because the answer matters to such a small handful of people. To make the question interesting we must generalize it. Instead of asking why your uncle is an alcoholic, we ask what general traits distinguish alcoholics from others. So the triviality problem generally is solved by casting a wider net in the research question.
Irrelevance is also solved by casting a wider net. In truth, of course, few questions are completely irrelevant; relevance is a matter of degree. As editor of the American Sociological Review I found that one of the most frequent complaints of reviewers was "Why should I care about the results of this paper?" As social scientists we care about research when the results speak to ongoing conversations and debates in our field. The more such connections we can make, the more interesting our results tend to be. So it is important to show how our research question links to theories and other empirical work.
The other objection (no-surprises) is consequential because, as rule 1 states, surprise is a hallmark of social research. It is the possibility of surprise that distinguishes social research from advocacy research. So if your social research project is uninteresting because it lacks the element of surprise, you need to make the case that the answer is not known beforehand. Sometimes results can be known ahead of time (making the research uninteresting) because the results are built in by the way concepts are measured. So if others say your project is uninteresting, you first need to make sure that your method is not to some degree preordaining your result. In examining the effect of marital satisfaction on overall life satisfaction, for example, it is important to find independent measures of the two concepts, so that the measures used do not guarantee a positive relationship between the two.
(Continues...)
Excerpted from Seven Rules for Social Research by Glenn Firebaugh
Copyright © 2008 by Princeton University Press. Excerpted by permission.
All rights reserved. No part of this excerpt may be reproduced or reprinted without permission in writing from the publisher.
Excerpts are provided by Dial-A-Book Inc. solely for the personal use of visitors to this web site.
Table of Contents
Preface xiThe First Rule: There Should Be the Possibility of Surprise in Social Research 1
Selecting a Research Question 2
Researchable Questions 2
Interesting Questions 4
Selecting a Sample 18
Samples in Qualitative Studies 23
Is Meaningful Social Research Possible? 26
Summary 29
Student Exercises on Rule 1 31
The Second Rule: Look for Differences That Make a Difference, and Report Them 36
You Can't Explain a Variable with a Constant 37
Maximizing Variance to Find the Effect of a Cause 39
Size versus Statistical Significance 41
Comparing Effects Where There Is a Common Metric 42
Calibration: Converting Explanatory Variables to a Common Metric 44
Substantive Profiling: The Use of Telling Comparisons 46
Visual Presentation of Results 51
Policy Importance 53
Importance for Theory 54
Conclusion 56
Student Exercises on Rule 2 58
The Third Rule: Build Reality Checks into Your Research 64
Internal Reality Checks 65
Reality Checks onData-Dubious Values and Incomplete Data 65
Reality Checks on Measures-Aim for Consistency in Conceptualization and Measurement 69
Reality Checks on Models-The Formal Equivalence Check 71
External Reality Checks: Validation with Other Data and Methods 76
Using Causal-Process Observations to Test Plausibility of Results 77
Using Ethnographic Data to Help Interpret Survey Results 79
Other Examples of Multiple-Method Research 81
Concluding Remark 82
Student Exercises on Rule 3 84
The Fourth Rule: Replicate Where Possible 90
Sources of Uncertainty in Social Research 91
Overview: From Population to Sample and Back to Population 93
Measurement Error as a Source of Uncertainty 100
Illustration: Two Methods for Estimating Global Poverty 101
Toward a Solution: Identical Analyses of Parallel Data Sets 105
Meta-analysis: Synthesizing Results Formally across Studies 106
Summary: Your Confidence Intervals Are Too Narrow 109
Student Exercises on Rule 4 111
The Fifth Rule: Compare Like with Like 120
Correlation and Causality 121
Types of Strategies for Comparing Like with Like 129
Matching versus Looking for Differences 130
The Standard Regression Method for Comparing Like with Like 131
Critique of the Standard Linear Regression Strategy 132
Comparing Like with Like Through Fixed-Effects Methods 134
First-Difference Models: Subtracting Out the Effects of Confounding Variables 134
Special Case: Growth-Rate Models 138
Sibling Models 140
Comparing Like with Like through Matching on Measured Variables 146
Exact Matching 146
Propensity-Score Method 147
Matching as a Preprocessing Strategy for Reducing Model Dependence 151
Comparing Like with Like through Naturally Occurring Random Assignment 152
Instrumental Variables: Matching through Partial Random Assignment 153
Matching Through Naturally Occurring Random Assignment to the Treatment Group 158
Comparison of Strategies for Comparing Like with Like 159
Conclusion 162
Student Exercises on Rule 5 165
The Sixth Rule: Use Panel Data to Study Individual Change and Repeated Cross-section Data to Study Social Change 172
Analytic Differences between Panel and Repeated Cross-section Data 173
Three General Questions about Change 175
Changing-Effect Models, Part 1: Two Points in Time 176
Changing-Effect Models, Part 2: Multilevel Models with Time as the Context 182
What We Want to Know 183
The General Multilevel Model 183
Convergence Models 185
The Sign Test for Convergence: Comparing Your [phi]s and [delta]s 186
Convergence Model versus Changing-Effect Model 191
Bridging Individual and Social Change: Estimating Cohort Replacement Effects 195
An Accounting Scheme for Social Change 197
Linear Decomposition Method 198
Summary 201
Student Exercises on Rule 6 203
The Seventh Rule: Let Method Be the Servant, Not the Master 207
Obsession with Regression 209
Naturally Occurring Random Assignment, Again 209
Decomposition Work in the Social Sciences 218
Decomposition of Variance and Inequality 220
Decomposition of Segregation Indexes 222
The Effects of Social Context 226
Context Effects as Objects of Study 227
Context Effects as Nuisance 230
Critical Tests in Social Research 231
Conclusion 235
Student Exercises on Rule 7 236
References 241
Index 253
What People are Saying About This
The audience for this book is great. Most graduate programs require second- or third-year students to write some type of research paper. This book is perfect for the task.
— Christopher Winship, Harvard University
"Anyone who wants to learn how to do social research better read this book. Written for the new student and the seasoned researcher alike (one is never too old, after all), Seven Rules for Social Research hits that sweet but till-now-neglected spot between overly simplified methods texts and advanced statistical manuals. Stick with Firebaugh's seven rules and you won't go wrong."—Dalton Conley, New York University"A valuable contribution. Firebaugh masterfully surveys a wide variety of key issues at the intersection of statistical theory, research design, and empirical analysis. His book can help improve the quality of social scientific research."—David Strang, Cornell University"The audience for this book is great. Most graduate programs require second- or third-year students to write some type of research paper. This book is perfect for the task."—Christopher Winship, Harvard University
Anyone who wants to learn how to do social research better read this book. Written for the new student and the seasoned researcher alike (one is never too old, after all), Seven Rules for Social Research hits that sweet but till-now-neglected spot between overly simplified methods texts and advanced statistical manuals. Stick with Firebaugh's seven rules and you won't go wrong.
— Dalton Conley, New York University
A valuable contribution. Firebaugh masterfully surveys a wide variety of key issues at the intersection of statistical theory, research design, and empirical analysis. His book can help improve the quality of social scientific research.
— David Strang, Cornell University