Hypothesis and Hypothesis Testing
Hypothesis and Hypothesis Testing
STRATEGIES OF HYPOTHESIS TESTING—QUANTITATIVE STRATEGIES
STRATEGIES OF HYPOTHESIS TESTING—QUALITATIVE STRATEGIES
FUNDAMENTAL CONTROVERSIES IN HYPOTHESIS TESTING
The scientific method is central to the identity of any self-described social science. Science is distinguished from nonscience not by the content or subject of study. Rather, the distinguishing characteristic of a science is the method of investigation. The scientific method relies on systematic, repeatable testing of expectations against the observed world. In their adoption of the scientific method, social sciences more closely resemble chemistry than its alchemical predecessors, which relied instead on the application of metaphysical rules to guide their work.
The physicist may use complex instruments to observe minute characteristics of subatomic particles. Similarly, the social scientist may use surveys to observe the characteristics of human behavior. The focus on observation as a method for testing expectations unites the physicist and the social scientist in their use of a common tool of the scientific method.
THE PURPOSE OF HYPOTHESES
Hypotheses are the central tool of scientific observation. Because the core method of scientific investigation is the comparison of expectations against observations of the world, scientists need to make clear statements about their expectations. A hypothesis is a concise, falsifiable statement that is subjected to observational testing as part of a scientific investigation.
Scientific research generally starts with a question about the observable world. In the social sciences research questions focus on human behavior—especially behavior related to groups (e.g., communities, countries, or societies). The scientific method says nothing about the origins of these research questions (just as it says nothing about the content of the areas of research). The scientific method simply requires that a scientist state an answer to this question (the hypothesis) that can be tested with observations (hypothesis testing).
There is a bewildering array of potential research questions—and thus hypotheses—in the domain of social science. Hypotheses can focus on expectations about voting behavior, the tendency of nations to go to war, or the factors that contribute to juvenile delinquency or to decisions about where to live (among many, many other hypotheses).
The purpose of the hypothesis is to ease the task of testing an expectation with observations of the world. A good hypothesis, then, is one that is easily tested. The ease of testing contributes to a second key aspect of the scientific method: reproducibility of testing. A clearly worded hypothesis can be tested repeatedly by a scientist and, maybe more important, by other scientists (King, Keohane, and Verba 1994, pp. 28–29).
Consider the following example. A social scientist may hypothesize that smaller class sizes in secondary schools will lead to higher performance on standardized tests. Because it is easy to observe the number of students in a class and the standardized tests scores are also easily observable (though there may be questions of the validity of the test as a measure of “intelligence” or even “academic achievement”), this hypothesis is easy to test. The test itself is also easy to replicate by the original social scientist or by other investigators. The hypothesis is sufficiently clear that any observer would be able to tell whether people in the smaller classes actually performed better on standardized tests. The judgment, then, is not a product of the specific observer but is instead independent of the identity of the scientist (a subject of some controversy that is discussed in a later section).
STRATEGIES OF HYPOTHESIS TESTING—QUANTITATIVE STRATEGIES
One of the major strategies for hypothesis testing is quantitative research. The focus of this approach is on the quantification of social science concepts for purposes of comparison and hypothesis testing. For example, a social scientist might ask whether the U.S. president’s approval ratings have gone down over the past year. This could give some sense of the power the president might have in promoting his or her legislative agenda or the chances of the president’s party in an upcoming election.
The hypothesis would be the social scientist’s guess as to how the candidate will fare against the proposed opponent. A good hypothesis will be one that is well grounded in the available theory on elections and that is testable against observable data (in this case a survey). The hypothesis would predict whether the approval ratings of the president have gone down over the past year. It would provide a preliminary answer to the stated research question. More ambitious hypotheses that predict specific levels of support (that the president has lost 8 percent of support from the previous year) are possible, but these require highly developed theories. One can take as an example the basic hypothesis that the president’s approval rating has gone down in the past year.
Armed with a hypothesis, the scientist will conduct a survey of a sample of potential voters to test the hypothesis. The scientist cannot, or would not want to, survey all citizens of the United States. Instead, the scientist will select a small sample out of the U.S. population. The scientist might send a survey to 1,000 citizens and see whether the hypothesis is correct within this sample of voters. The results of the survey will give the scientist a sense of the president’s current approval rating for comparison to previous approval ratings (Babbie 1995, pp. 190–193).
Quantitative hypothesis testing—the comparison of numerically represented measurements for purposes of hypothesis testing—allows for some detailed comparisons. One can say, for the sake of argument, that the survey suggests 43 percent of respondents said they approved of the job the president was doing. The previous year’s survey had reported that 47 percent of respondents had said they approved. At first glance, the evidence provides support for the hypothesis.
A number of questions remain about this test of the hypothesis. To what extent is the observed dip in approval indicative of a general trend in the U.S. population? To what extent is the dip indicative of a persistent change in approval? Tools of probability and statistics provide some opportunity to address these questions. Sampling theory provides some sense of how reliable the results are that come from a sample of a larger population (Babbie 1995, pp. 195–203). Such theory helps scientists describe the range of possible values in the population given the size of the sample surveyed. One could describe the probability that the actual approval rating was 47 percent (the previous rating) while the sample happened to be skewed toward lower approval ratings. In general, the larger the sample size, the lower the probability of these sorts of discrepancies. Such theory also provides insight into whether the variation one sees is relatively permanent or just part of the inherent variability in measuring people’s approval of a public figure. It is the ability to assess these issues of sampling and fundamental uncertainty that have convinced many of the utility of quantitative hypothesis testing techniques.
STRATEGIES OF HYPOTHESIS TESTING—QUALITATIVE STRATEGIES
Many scholars pursue an alternative style of hypothesis testing. These scholars tend to be unsatisfied with the techniques of measurement for social concepts employed in many quantitative research projects. In lieu of quantitative measurements of large samples of observations, qualitative hypothesis testing involves the careful study of a smaller number of observations with detailed treatment of the context and meaning of the social concepts themselves.
A qualitative hypothesis testing strategy follows the basic procedure of hypothesis testing. The social scientist generates a hypothesis in response to a research question. The social scientist then compares his or her expectation against the observed world. The difference between the qualitative approach and the quantitative approach reviewed earlier is in the strategy for getting reliable observations of the world.
Qualitative hypothesis testing tends to focus on detailed histories and culturally sensitive accounts of the social systems that are being studied. The detail and contextual knowledge provide the qualitative hypothesis testing strategies leverage on the challenges of hypothesis testing in two ways. First, the detailed knowledge of the subjects under study allow for careful selection of cases for study. As opposed to the quantitative strategy of multiplying the number of observations to avoid the possibility of drawing the wrong lessons from a study, qualitative hypothesis testing involves carefully selecting a few observations to achieve the ideal contrast. Second, the detailed knowledge of the subjects also allows for greater attention to the measurement of variables. Proponents of qualitative research focus on the ability to really get to know the subjects as a means to understand the nuances of the proposed effects of policies (e.g., Brady and Collier 2004).
Qualitative research tends to investigate different types of hypotheses than quantitative research, though the barriers between the two have eroded somewhat since the late twentieth century. Whereas quantitative hypotheses tend to involve statements of correlation, qualitative hypotheses have tended to focus on issues of necessary and sufficient conditions. These hypotheses focus on the conditions whose presence guarantees that an effect will be present (a sufficient condition) or whose absence will guarantee that an effect will not be present (a necessary condition) (Goertz and Starr 2003).
Theda Skocpol’s (1979) research on social revolutions exemplifies this approach. In States and Social Revolutions: A Comparative Analysis of France, Russia, and China, Skocpol studies the factors that are essential to the success of peasant revolutions in a selection of countries. To study complicated processes such as social revolution and its relation to the political structures of regimes, Skocpol focuses her attention on the contrast between France, Russia, and China. These detailed cases are contrasted with control cases such as England and Prussia. The control cases serve the comparative role of the prior approval ratings in the quantitative example given earlier. This approach allows Skocpol to study each of the cases in great detail and to have confidence in the measurement of such concepts as types of revolution and various aspects of regime structure. The result is a widely praised multidimensional account of the necessary conditions of success for peasant revolutions.
FUNDAMENTAL CONTROVERSIES IN HYPOTHESIS TESTING
While there is debate over the relative merits of qualitative and quantitative hypothesis testing, there are also more fundamental critiques of hypotheses testing. The quantitative and qualitative hypotheses are different strategies to accomplish the same goal. In both approaches observations are compared against hypotheses about the essential nature of the social world. In the qualitative example Skocpol is testing hypotheses about the underlying nature of social revolution. This assumes that there is an underlying nature of social revolutions. Some critics of the hypothesis testing contend that there is no singular underlying social nature. These authors, mostly associated with poststructuralism, argue that there is no singular structure of society about which one can generalize or that one can discover through repeated observation (for a famous statement of this argument, see Derrida 1978).
Other authors focus their criticism not on the absence of a stable world to observe, but instead on the tools that social scientists have to observe the world (assuming that such a stable world exists). These critics allege that social measurement is inherently filled with biases. Observation, these critics allege, is inseparable from the observer. If this is the case, especially given the importance of social values to humans, there is no such thing as neutral observation of the social world. The result is that all hypothesis tests are suspect. Many of these critics recommend exploring the social world through admittedly biased accounts and narratives rather than the “at-a-distance” observation implied by the typical hypothesis testing framework (Shank 2002).
SEE ALSO Scientific Method; Social Science
BIBLIOGRAPHY
Babbie, Earl. 1995. The Practice of Social Research. 7th ed. Belmont, CA: Wadsworth.
Brady, Henry E., and David Collier, eds. 2004. Rethinking Social Inquiry: Diverse Tools, Shared Standards. Lanham, NY: Rowman and Littlefield.
Derrida, Jacques. 1978. Writing and Difference. Trans. Alan Bass. Chicago: University of Chicago Press.
Goertz, Gary, and Harvey Starr, eds. 2003. Necessary Conditions: Theory, Methodology, and Applications. Lanham, MD: Rowman and Littlefield.
King, Gary, Robert O. Keohane, and Sidney Verba. 1994. Designing Social Inquiry: Scientific Inference in Qualitative Research. Princeton, NJ: Princeton University Press.
Shank, Gary D. 2002. Qualitative Research: A Personal Skills Approach. Upper Saddle River, NJ: Prentice Hall.
Skocpol, Theda. 1979. States and Social Revolutions: A Comparative Analysis of France, Russia, and China. New York: Cambridge University Press.
Scott E. Robinson