Campbell, Donald

views updated

Campbell, Donald 1916-1996

SOCIAL EXPERIMENTATION AND QUASI-EXPERIMENTATION

CRITICAL REALISM AND THE EPISTEMOLOGY OF PRAGMATIC ELIMINATIVE INDUCTION

METHODOLOGICAL TRIANGULATION AND THE MULTITRAIT-MULTIMETHOD MATRIX

Donald Thomas Campbell was a master methodologist whose intellectual passions and scholarly achievements spanned the fields of social science research methodology, epistemology and philosophy of science, and the sociology and psychology of science.

Campbell’s best-known works are on the methodology of social experimentation and quasi-experimentation (Campbell 1957; Campbell and Stanley 1963; Cook and Campbell 1979; Shadish et al. 2001). They are among the most influential contributions to the methodology of applied social science in the twentieth century. Although others had written about field experiments—for example, the preeminent statistician Ronald Fisher (1890–1962)—it was Campbell who introduced the methodology of quasi-experimentation, an inductive (but non statistical) methodology that seeks to understand the effects of the many unmanageable contingencies that lie beyond the control of experimenters. Although quasi-experimentation is “applied,” especially in contexts involving the evaluation of social programs, it is nevertheless “theoretical.” It is grounded in Campbell’s philosophy of critical realism, his epistemology of pragmatic eliminative induction, his methodology of triangulation and the multitrait-multimethod matrix, his theory of pattern matching in science, and his sociology of science (Campbell 1988).

SOCIAL EXPERIMENTATION AND QUASI-EXPERIMENTATION

Experimentation and quasi-experimentation are extensions of representative design, a theory of inquiry developed by Campbell’s two most influential teachers at Berkeley, Egon Brunswik (1903–1955) and Edward Tolman (1886–1959). Representative design requires that experiments be conducted in settings that are representative of an organism’s typical ecology. Representative design is a fundamental departure from the classical experiment, in which one independent (treatment) variable is manipulated in order to determine its effect on a dependent (outcome) variable, with all other factors held constant through random selection and statistical controls. Representative design is appropriate when social experiments are carried out in uncertain, unstable, and causally complex environments.

The classical experiment, which involves an independent (treatment) variable that is manipulated in order to determine its effect on a dependent (outcome) variable, requires a treatment and control group, random selection of subjects, and random assignment of subjects to treatment and control groups. If any of these requirements are absent, it is not a classical (“true”) experiment, but a quasi-experiment. Many of the most important social policy and program interventions are quasi-experiments because political, administrative, and ethical constraints rarely make it possible to satisfy the requirements of the classical experiment.

CRITICAL REALISM AND THE EPISTEMOLOGY OF PRAGMATIC ELIMINATIVE INDUCTION

Quasi-experimentation is expressly designed for research settings in which manifold contingencies are beyond the control of the experimenter. These contingencies give rise to many rival hypotheses (alternative explanations of the same outcome) that can threaten the validity of causal claims. Plausible rival hypotheses must be tested and, where possible, eliminated. This process of eliminative induction is an extension of John Stuart Mill’s (1806–1873) joint method of agreement and difference ; it also represents a (heavily qualified) version of Karl Popper’s (1902–1994) falsificationist principle that hypotheses can be falsified but never proved.

Pragmatic (not logical-analytic) eliminative induction is part of Campbell’s evolutionary critical-realist epistemology (Campbell 1974), according to which knowledge grows through processes of trial-and-error learning and selective retention. Quasi-experiments in education, welfare, health, and other social policy arenas embody such processes. When entire societies self-consciously and systematically engage in trial-and-error learning and selective retention, they are experimenting societies (see Campbell in Dunn 1998).

METHODOLOGICAL TRIANGULATION AND THE MULTITRAIT-MULTIMETHOD MATRIX

What we know as “truth” or objective knowledge is actually a product of theory interacting with data. Emblematic of Campbell’s methodological work is its synthesis of subjective and objective, perception and reality, theory and data. He accomplished this synthesis through methodological triangulation—a process of comparing two or more perspectives toward the “same” object. His work with D. W. Fiske (1916–2003) on the multitrait-multimethod matrix (Campbell and Fiske 1959) represents a form of theory and method triangulation, where traits posited by a theory interact with methods of observation. The product is some level of convergent and discriminant validity. Underlying the multitrait-multimethod matrix is an epistemology usually referred to as a coherence theory of truth. Campbell and Fiske’s 1959 paper is one of the most widely cited papers in the social and behavioral sciences.

PATTERN MATCHING IN SCIENCE

To the degree that products of the social and behavioral sciences merit the term knowledge, it is because they are grounded in representations of the social world achieved by matching observations of that world with abstract concepts of it. For Campbell, pattern matching is essential for attaining knowledge of everyday cultural objects and indispensable for achieving knowledge of inferred entities in science—the size and brightness of distant stars, the existence and magnitude of latent psychological variables such as alienation or intelligence, and the structure of complex social processes such as social mobility and economic growth. Campbell argued that these and other inferred entities are known indirectly and vicariously, through a process of pattern matching (Campbell 1966). He also contended that case studies can be a methodologically powerful form of pattern matching in the social sciences (Campbell 1975).

THE SOCIOLOGY OF SCIENTIFIC VALIDITY

Traditional sociology of knowledge, because it describes the social, political, and cultural conditions affecting science and other knowledge formations, is descriptive. Campbell’s sociology of knowledge, which goes beyond description, sought to explain how differences in the social structures and processes of sciences affect the degree of validity achieved by those sciences. He called this normative sociology of science the sociology of scientific validity (Campbell 1986). Among the social factors that affect scientific validity are institutional incentive systems that reward quality research and punish those whose research is flawed by violating norms of science.

SEE ALSO Experiments; Methodology; Natural Experiments; Philosophy of Science; Popper, Karl; Public Policy; Social Experiment; Sociology; Sociology, Knowledge in; Tolman, Edward

BIBLIOGRAPHY

PRIMARY SOURCES

Campbell, Donald T. 1957. Factors Relevant to the Validity of Experiments in Social Settings. Psychological Bulletin 54: 297–312.

Campbell, Donald T. 1966. Pattern Matching as an Essential in Distal Knowing. In The Psychology of Egon Brunswik, ed. Kenneth R. Hammond, 81–106. New York: Holt, Rinehart, and Winston.

Campbell, Donald T. 1974. Evolutionary Epistemology. In The Philosophy of Karl Popper, ed. Paul A. Schilpp, 413–463. La Salle, IL: Open Court.

Campbell, Donald T. 1975. “Degrees of Freedom” and the Case Study. Comparative Political Studies 8: 178–193.

Campbell, Donald T. 1986. Science’s Social System of Validity-Enhancing Collective Belief Change and the Problems of the Social Sciences. In Metatheory in Social Science: Pluralisms and Subjectivities, eds. D. W. Fiske and Richard A. Schweder, 108–135. Chicago: University of Chicago Press.

Campbell, Donald T. 1988. Methodology and Epistemology for Social Science: Selected Papers. Ed. E. Samuel Overman. Chicago: University of Chicago Press.

Campbell, Donald T., and Donald W. Fiske. 1959. Convergent and Discriminant Validation by the Multitrait-Multimethod Matrix. Psychological Bulletin 56: 81–105.

Campbell, Donald T., and Julian C. Stanley. 1963. Experimental and Quasi-Experimental Designs for Research. Chicago: Rand McNally.

Cook, Thomas D., and Donald T. Campbell. 1979. Quasi-Experimentation: Design & Analysis Issues for Field Settings. Boston: Houghton Mifflin.

Shadish, William R., Thomas D. Cook, and Donald T. Campbell. 2001. Experimental and Quasi-Experimental Designs for Generalized Causal Inference. Boston: Houghton Mifflin.