Prediction

Techniques of social prediction

Current problems

BIBLIOGRAPHY

In sociological writing the term “prediction” means a stated expectation about a given aspect of social behavior that may be verified by subsequent observation. Within this general meaning the term is used in two principal senses: for deductions from known to unknown events within a conceptually static system and for statements about future outcomes based on recurring sequences of events. This article is largely restricted to the latter sense, although the former usage, which covers significant forms of logical reasoning, is widely prevalent.

The estimate of a given variable from one or more concurrent variables as in regression analysis is conventionally referred to as a prediction. Similarly, the estimate of a population characteristic may be referred to as a prediction, although the sample from which the inference is drawn is not separated in time from the population that it represents. Still more common, especially in the writing on social systems, is the designation of the term y in the expression “If x, then y,” as a prediction, even though the x and y are often regarded as of simultaneous occurrence. Although such usage has been questioned on linguistic grounds, it is well established, and its currency is not likely to be affected by such arguments.

In its second principal sense the term “prediction” refers to assertions about future outcomes based on the observed regularities among consecutive events of the past. This category contains its own distinction, depending on whether the statement holds for a single, concrete instance, with due regard for the accidents of time and place, or abstractly holds for any case in a class satisfying stated conditions. If the statement is concrete and necessarily bound to the calendar, it carries the label “forecast”; otherwise, when it is not so restricted, it carries the more general term “prediction.” According to this distinction, the expected volume of crime in the United States in the next calendar year would constitute a forecast, whereas the expected success of the individual on parole under specified conditions would be a prediction. Although this terminology is useful for distinguishing between special and general formulations, it has not been consistently applied, even in technical writing, and many so-called predictions would have to be relabeled as forecasts if the distinction were to be strictly maintained.

Prediction as a social process. At least some form of prediction, in the broadest sense of the term, is practiced on all levels of culture (Tylor 1871, chapter 4). The contemporary emphasis in sociology, as described below, is thus consistent with traditional enterprise, answering to the same general purposes but differing in the process by which the foreknowledge is obtained. The social purpose of prediction, whether of physical or social events, is to secure a measure of control over what otherwise would be less manageable circumstances. The effects of such natural calamities as typhoons and floods may be mitigated, if not averted, by forehanded preparation; similarly, by stating the conditions under which a social upheaval can occur, steps may be taken to prevent the occurrence of one. Some of the Biblical prophecies were of this nature, urging the people to righteousness in order to avoid the wrath of God. At times such a statement may be more in the nature of a promise than a threat, setting forth the conditions to be met in order to achieve a desired objective, such as an annuity upon retirement. But however they differ in meaning, practically all predictions are potential instruments of social action, enabling the group either to facilitate a favorable outcome or to impede an unfavorable one.

The process of sociological prediction. Although all predictions are alike in broad social purpose, they differ in the process of their formulation, which will be more or less scientific according to the nature of the underlying analysis. The process of sociological prediction has in varying degree those elements common to all scientific prediction: some theory of behavior from which deductions may be drawn and some factual evidence that is relevant to the propositions of the theory. Sociological prediction has arisen naturally from the concerns of sociology itself, both theoretical and empirical, and thus sociologists now take prediction of the forms and processes of social life as one of their principal tasks. This commitment has its roots in the writings of Auguste Comte and has been regularly affirmed by leading representatives of the discipline since that time. Max Weber held that the purpose of sociology is to predict the patterns of social interaction, and Albion Small, one of the founders of American sociology, took very much the same position (1916).

Although sociological predictions ideally are to be drawn from theory, for the most part they have been little more than statistical projections based on compilations of empirical data within categories of perhaps little theoretical significance. But such compilations in the form of time series and actuarial tables have had their bearing on theory. For example, the hypothesis of “cultural lag” (Ogburn 1922) was derived in part from the empirical growth curve of inventions, and Edwin H. Sutherland’s theory of “differential association” has been refined on the basis of parole prediction studies (see Glaser 1954). In this way the construction of statistical trends and experience tables and their corresponding projections have had some impact on theory, in both extending and recasting it. Nevertheless, it is not the statistical materials and their manipulation that give sociological prediction its special character, for comparable series and their analysis are part of the natural sciences; rather, it is the underlying categories from which sociological predictions are derived.

Prediction research. Although many sociological investigations have a bearing on the predictability of social and cultural events, relatively few studies have had prediction as their primary goal. For the purposes of outlining these more specialized studies and citing examples, prediction research will be classified here according to whether its focus is the collective characteristics of the group or the characteristics of its constituent members.

In those studies analyzing the collective aspects of the group, the prediction has in some cases extended over several classes of events, whereas in various others the prediction has been restricted to a single outcome. The prediction of a relatively wide range of events is perhaps best represented by the work of William F. Ogburn, a consistent theme of which was the proposition that technological trends of the past provide a useful key to cultural trends of the future. This idea was developed in Ogburn’s work during the 1930s, which included reviews of selected social and economic trends in the United States from 1900 to 1930 (President’s Research Committee & 1933) and a government report on social and economic conditions affecting the rate of invention and the impact of invention on social life (U.S. National Resources Committee, Science Committee 1937), and it was later restated in The Social Effects of Aviation (Ogburn et al. 1946).

Not all studies of trends and cycles have been at the societal level; some have been concerned with the pattern of interaction and the sequence of its development within the small group. The work of Bales and his associates (summarized in Bales 1959) provides an example of this approach, and Bales’s writings contain an assessment of its potential for predictive knowledge.

The prediction of a single social outcome may be illustrated by the election forecast, since its problems are well denned (Mosteller et al. 1949) and its operating procedures are well standardized. Such a forecast rests on a succession of carefully designed and drawn sample polls taken at regular intervals shortly before the election. Based on the trend of these results, with due allowance for sampling and measurement error, the percentage of the vote for each candidate is predicted, and thus the probable winning candidate can be named. The critical matters bearing on the accuracy of such prediction include the correspondence between the sampled population and the population actually voting on election day as well as the stability of the observed trend, at least through the day of the election. Notwithstanding these and related difficulties, scientific polling agencies have been quite successful in predicting the results of political elections held in the United States since 1952. Students of this process have noted that such predictions may affect the election itself.

With minor exceptions, the prediction of individual behavior has been limited to those forms of personal adjustment whose variation is thought to be largely due to differences in social background and circumstance: for example, adjustment in the armed forces (Star 1950), postwar adjustment (Cottrell 1949), adjustment on parole (Burgess 1928), and adjustment in marriage (Burgess & Cottrell 1939).

The device by which the prediction of personal adjustment is usually effected is the experience table, which in principle is no different from the actuary’s life table that shows probabilities of death by age. Similarly, the table of social experience gives the odds of success for the several subclasses into which the population has been arranged, and it thereby yields a prediction for the individual case; obviously, the more nearly the probabilities approach zero or one, the more accurate is the prediction for each person. This method

can be illustrated by a table from Burgess’ early study of factors determining success on parole (see Table 1); this table, an adaptation of the original, shows the percentage of failures (that is, parole violators) for each of the classes into which 1,000 parolees had been grouped according to scores based on 21 factors correlated with outcome on parole. This table, which may be regarded as stating a set of empirical probabilities, is characteristic of practically all studies seeking to predict the behavior of the individual.

Recent studies of social adjustment are somewhat distinctive in their employment of refined statistical methods to differentiate between successes and failures. An important innovation consists in their attention to the possible discrepancy between decisions maximizing predictive accuracy and those minimizing social cost. Ohlin (1951) discusses the relevance of prediction tables for parole selection and analyzes their differing purposes, and Cronbach and Gleser (1957) provide a comparable discussion of personnel selection based on psychological test scores.

Techniques of social prediction

The derivation of a prediction is usually accomplished by general statistical methods, none of which is restricted in its application to social data. Despite their general familiarity, these methods will be briefly reviewed here, primarily to illustrate representative applications in social prediction. Although not all statistical predictions are derived in the same way, all share the requirement that both predictor and criterion variables be subject to reliable measurement. Except for this scant reference, and despite its importance, the problem of measurement will be ignored in the following discussion.

Extrapolation

Roughly speaking, extrapolation is the process of predicting a variable from itself— for example, predicting the future growth of a population from its past growth. By this technique it is possible to obtain a succession of expected values that are arrayed in the future from least to most distant. Such expected values will materialize only if the underlying social process or causal system that the curve expresses is constant for the period over which the prediction extends. Prediction by extrapolation will be in error to the degree that a given process changes; accordingly, prediction by this method will generally be more accurate for shorter rather than longer durations. An example of extrapolation is provided by Hart’s prediction of life expectancy on the basis of the trend in life expectancy during the last 75 years (1954).

[For other examples of extrapolation seePrediction and forecasting, economic; Population,article onpopulation growth.]

Correlation methods

Reduced to its lowest terms, prediction by correlation is based on measured linkages between earlier and later events in a given sequence–for example, between scholastic performance in high school and that in college, between adjustment in childhood and that in marriage. It goes without saying that such obtained relationships will have no predictive utility beyond the specific population for which they hold. Thus, if the correlation between type of infant feeding and social maturity (to take a hypothetical example) obtains only in the middle class, the former category will be worthless as a predictor of social maturity in the lower class.

In basing our prediction on correlated events, the guiding principle is that the errors of prediction must be minimized in some well-defined sense. To illustrate the application of this principle and also to mark points of entry into the pertinent literature, we will consider in barest detail the most common procedures for predicting a variable from a set of attributes or variables and those for predicting an attribute from a set of attributes or variables. When the criterion consists of more than a single element (Hotelling 1935) and/or when attributes and variates are employed together as predictors (Mannheim & Wilkins 1955), the precedures will be more complicated but no different in principle.

In predicting a variable from one or more variables, the practice is to predict so that the sum o the squared errors around the fitted curve is a minimum. When the fitted curve is linear, as is usually the case, the product-moment coefficient of correlation r, or an adaptation of it (partial or multiple), serves to gauge the relative accuracy of comparable predictions. For example, a succession of similar studies of marriage obtained the following correlations between scores considered to be prognostic of marital adjustment and scores of actual adjustment in marriage: Burgess and Cottrell, .51 (1939); Burgess and Wallin, .50 (1953); Terman, .54 (see Terman et al. 1938).

In predicting a quantitative variable from one or more attributes, the rule is to determine the predicted values so as to maximize the sum of squares between groups and correspondingly to minimize the sum of squares within groups, as in the analysis of variance; hence, the coefficient of intraclass correlation, or an equivalent, may be taken as a measure of predictive accuracy. The fact that no instance of such a measure appears in the bibliography to this article is intended to suggest the possibly limited value of attributes for purposes of predicting quantitative variables.

Where the prediction is from one or more attributes to a single attribute, the rule is to predict repeatedly whichever attribute has the largest subclass frequency, or conditional probability (Guttman 1941). The accuracy of such prediction will vary according to the difference between the conditional and marginal probabilities. Since measures of association such as C, T, or phi reflect that difference, they have been widely used to gauge the predictive accuracy of attributes. The examples given in Table 2, which are from one of the Gluecks’ early studies of recidivism, will serve to illustrate the form and usual magnitude of such coefficients.

Finally, where the prediction is from one or more variables to a single attribute the preferred procedure is to derive from the variables a composite variable that will yield the least overlap among the several within-class distributions and hence the least error in prediction. When the composite variable is a linear function of its components, the procedure is termed a discriminant function (Fisher 1936). The experience table for predicting recidivism among Borstal (i.e., reform school) lads (Mannheim & Wilkins 1955) was based on this method and is indicative of both its value and limitations; Kirby (1954) has also used this method, in a parole prediction study.

Markov chains

Although the Markov chain, as well as the general class of stochastic processes to which it belongs, is covered in most standard references on probability (Feller 1950–1966), its potentialities for sociological prediction have only recently been explored, and these efforts have been largely, if not exclusively, limited to the fitting of historical data to theoretical chains. Although such materials indicate whether conditions as observed at time tk might have been accurately predicted by Markov methods beginning at time t0, they contain no demonstration that a given process will repeat itself indefinitely and therefore give no indication whether it may be confidently used for predictive purposes. This remark, however, should not be construed as a criticism of the studies cited illustratively below, which purported to be more suggestive than conclusive.

The feasibility of predicting political attitudes has been considered by Anderson (1954) in a secondary analysis of panel data. His particular concern was with the probability that a person would hold the same political opinion at the end of a sequence that he held at the start. To determine whether such probabilities might be attained by Markovian methods, a comparison was drawn between distributions based on opinions expressed by a panel of voters in each of the six months immediately preceding the U.S. presidential election of 1940 and those expected on the basis of probability theory. Although the findings of this particular analysis were inconclusive, they do hint at the potentialities of the Markov chain as a device for analyzing the process of attitude change and as a tool for social prediction.

The correspondence between actual patterns of labor mobility and those obtained by treating the movement of workers as a Markov process has been examined by Blumen, Kogan, and McCarthy (1955). Although this investigation was concerned primarily with the dynamics of labor mobility, it has considerable relevance for prediction in its emphasis on statements expressing the probability that a worker in a given industrial group will be in that same group after k intervals of time. Apart from its substantive value for industrial sociology, this study is constructively important as a demonstration of the potential utility of Markov theory in the prediction of mobility, both social and geographical.

Current problems

During the past fifty years of American sociology, the problems, or problematics, of prediction have received at least as much attention as the predictions themselves, which, as suggested in the foregoing discussion, have been relatively few in number and circumscribed in content. The range of these problems is reflected in the kinds of issues that have been regularly debated in the sociological journals. Some of these issues are briefly presented here.

Actuarial approach versus case method

The issue of the actuarial approach versus the case method has arisen in the prediction of personal adjustment. This issue has two parts: whether it is possible to frame a prediction solely from case materials in total disregard of all probabilities, and whether such prediction, when there is no explicit reference to actuarial materials, is more accurate than that based on statistical averages, or rates. Most serious students of prediction would answer “no” to the first question and “possibly under some circumstances” to the second.

Prediction that claims to be wholly devoid of probabilities is regarded as logically impossible by some students (Social Science Research Council 1941). Briefly put, their argument is: notwithstanding the claims of the predictor to the contrary, the predicted case will be treated as a class member or located in a risk table although that table may be wholly based on the experience of the forecaster and may exist only in his mind. That the process is subjective and personal does not alter its essential nature as probabilistic; therefore—so runs the argument—all prediction is actuarial and, correspondingly, no prediction is wholly free of uncertainty.

The second answer, “possibly under some circumstances,” shows a recognition that predictions from extensive case materials may in some instances be more accurate than those based on group averages, by virtue of the analyst’s exceptional ability to assign cases to risk categories having probabilities very close to zero or one. However, such idiosyncratic accuracy embodies “procedures” that cannot be readily codified and transmitted to the public, as would be required of scientific methods. Hence, prediction from case materials, no matter how accurate, must be regarded as an expression of clinical insight and judgment rather than as the application of scientific law.

Prediction as feedback

Despite the antiquity of the idea that a stated prediction may affect its own fulfillment (Popper 1957) and despite the relevance of this idea to social prediction, few empirical studies have sought to measure the influence of predictions on subsequent events. However, an illustrative literature has emerged, including references to rumor, stereotyped expectation, the election forecast, and false prophecy (see especially Merton 1948; Simon 1954; Festinger et al. 1956). Students of voting behavior have noted the possible effects of a computer prediction of the election of specific candidates based on national election results in an earlier time zone on the pattern of voting in a later time zone.

Such materials lend themselves to systematic classification according to whether the outcome is favorable or unfavorable and according to whether the effect of prediction is positive or negative. Thus, the prognosis that the patient will recover may generate either a confidence that will hasten recovery or an overconfidence that will delay it; on the other hand, an unfavorable prognosis of chronic illness may create either an attitude of resignation or an attitude of defiance, with differing effects. Instances of this kind support the common opinion that by reason of the symbolic nature of human life, social prediction is reflexive and that in consequence the validity of social prediction is subject to greater uncertainty than physical prediction.

Efficiency of social prediction

Broadly speaking, the concept of the efficiency of prediction refers to the accuracy of a given method of prediction relative to that of an alternative that is taken as a standard or norm (Reiss 1951). If, for example, the chosen standard produces 20 errors per 100 trials and the alternative in question produces only 10, then the alternative may be said to be twice as efficient as the standard.

Usually the predictions compared are that derived from the joint distribution of a criterion and one or more predictors and that derived from the distribution of the criterion alone. Thus, given a male delinquency rate of 20 per cent, the prediction of nondelinquency on repeated trials would carry an error rate of 20 per cent; if, after introducing social class as a predictor, the error rate is reduced to 10 per cent, then the prediction from the two-way classification may be said to be twice as efficient as prediction from the single classification of children as delinquent or nondelinquent.

By and large, the measured efficiency of social prediction has been relatively low. For example, in criminological studies, where the identification of the prospective delinquent or recidivist would be of considerable practical importance, the gains resulting from the introduction of information thought to bear on such behavior have been negligible (Schuessler 1954). To achieve substantial gains in efficiency it will be necessary to identify and measure those variables that are closely correlated with the criterion.

Explanation versus prediction

Although explanation and prediction are not irreconcilable or even rival alternatives, there are differences of opinion about which of these should receive the greater emphasis. In general, this disagreement reflects the difference between a preference for theory and an emphasis on factual research.

In this disagreement, those placing greater emphasis on explanation usually argue as follows: Predictions (hypotheses) may be deduced from a general explanation (theory), but a collection of statistical predictions, no matter how accurate, does not constitute a theory; hence, explanation should be the first order of business. Moreover, obtained statistical regularities may be an accident of time or place and hence an unstable basis for prediction, whereas a scientific law is universal and therefore an unerring source of prediction. According to this line of reasoning, the poor showing of prediction in sociology is a reflection of crudities in general theory. Thus, the development of a set of valid explanations is considered to be a prerequisite for the improvement of predictions.

Those putting greater stress on prediction would probably not disagree with the premise of the foregoing argument, although they would emphasize the interplay between statistical association and theoretical explanation and the steady impact of the former on the latter. Furthermore, they would hold that if they are to frame hypotheses and to test them, they have no choice but to operate pragmatically within existing theory, reforming it as they proceed. Probably the rank and file of American sociologists are “agnostics” with regard to this issue, excepting those whose major interest lies in the philosophy of social science.

Limits of prediction

Another issue is the general question whether some classes of social events are inherently unpredictable. Although this question is valid, it necessarily admits of only a speculative answer. The judgment that the “future course of human affairs is unpredictable” (Toynbee 1934–1961, vol. 12) has to do with the broad question of cultural history or evolution and has little to do with prediction as a specialty within sociology. In general, sociologists have been primarily concerned with problems of much smaller compass : land use, migration, suburban growth, fertility rates of human populations, rates of assimilation, patterns of racial violence, and political movements. Nevertheless, the importance of predicting social mutations as well as recurring phenomena has been increasingly emphasized in sociological writings (e.g., Moore 1964), and it is to be anticipated that this problem will be studied empirically in the coming decades. Such factual studies will enable sociologists to set provisional limits to the range of social prediction (Bell 1965).

In conclusion, it should be noted that prediction and sociology have always been closely linked: sociology grew out of a concern with prediction (Comte’s savoir pour prevoir) and has always had the securing of predictive knowledge as one of its express aims. However, relatively few empirical studies have been specifically concerned with producing it. Thus, the importance of recent prediction studies lies as much in the methodological understandings that have grown out of them as in the substantive findings themselves. As these understandings become systematized and gain in currency, a wider variety of factual investigations will probably appear as a sequel to the pioneering work of the last several decades.

Karl F. Schuessler

[Directly related are the entriesCausation; Life Tables; Markov Chains; Prediction and Forecasting, Economic. Other relevant material may be found inLinear Hypotheses; Multivariate Analysis; Penology, article onProbation and Parole; Voting; and in the biographies ofComte; Ogburn; Small; Weber, Max.]

BIBLIOGRAPHY

Anderson, T. W. 1954 Probability Models for Analyzing Time Changes in Attitudes. Pages 17–66 in Paul F. Lazarsfeld (editor), Mathematical Thinking in the Social Sciences. Glencoe, III.: Free Press.

Bales, Robert F. 1959 Small-group Theory Research. Pages 293–305 in American Sociological Society, Sociology Today. Edited by Robert K. Merton, Leonard Broom, and Leonard S. Cottrell. New York: Basic Books.

Bell, Daniel 1965 Twelve Modes of Prediction. Pages 96–127 in Julius Gould (editor), Penguin Survey of the Social Sciences 1965. Baltimore: Penguin.

Blumen, Isadore; Kogan, Marvin; and Mccarthy, Philip 1955 The Industrial Mobility of Labor as a Probability Process. Cornell Studies in Industrial and Labor Relations, Vol. 6. Ithaca, N.Y.: Cornell Univ. Press.

Borgatta, Edgar F.; and Westoff, Charles F. 1954 The Prediction of Total Fertility. Milbank Memorial Fund Quarterly 32:383–419.

Bowerman, Charles E. 1964 Prediction Studies. Pages 215–246 in Harold T. Christensen (editor), Handbook of Marriage and the Family. Chicago: Rand McNally.

Burgess, Ernest W. 1928 Factors Determining Success or Failure on Parole. Pages 203–249 in Illinois, Committee on Indeterminate-sentence Law and Parole, The Workings of the Indeterminate-sentence Law an the Parole System in Illinois. Springfield, 111.: Division of Pardons and Paroles.

Burgess, Ernest W.; and Cottrell, Leonard S. 1939 Predicting Success or Failure in Marriage. New York: Prentice-Hall.

Burgess, Ernest W.; and Wallin, Paul 1953 Engagement and Marriage. Philadelphia: Lippincott. Cantril, Hadley 1938 The Prediction of Social Events. Journal of Abnormal and Social Psychology 33:364-389.

Chapin, Francis S. (1947)1955 Experimental Designs in Sociological Research. Rev. ed. New York: Harper.

Clausen, John A. 1950 Studies of the Postwar Plans of Soldiers: A Problem in Prediction. Pages 568–708 in Samuel A. Stouffer et al., Measurement and Prediction. Studies in Social Psychology in World War Ii, Vol. 4. Princeton Univ. Press.

Cottrell, Leonard S. 1949 The Aftermath of Hostilities. Pages 549–595 in Samuel A. Stouffer et al., The American Soldier. Studies in Social Psychology in World War ii, Vol. 2. Princeton Univ. Press.

Cronbach, Lee J.; and Gleser, Goldine C. (1957) 1965 Psychological Tests and Personnel Decisions. 2d ed. Urbana: Univ. of Illinois Press.

Dollard, John 1948 Under What Conditions Do Opinions Predict Behavior? Public Opinion Quarterly 12: 623–632.

Duncan, Otis Dudley et al. 1953 Formal Devices for Making Selection Decisions. American Journal of Sociology 58:573–584.

Feller, William 1950–1966 An Introduction to Probability Theory and Its Applications. 2 vols. New York: Wiley. → A second edition of the first volume was published in 1957.

Festinger, Leon; Riecken, H. W.; and Schachter, Stanley 1956 When Prophecy Fails. Minneapolis: Univ. of Minnesota Press.

Fisher, R. A. 1936 The Use of Multiple Measurements in Taxonomic Problems. Annals of Eugenics 7:179-188.

Glaser, Daniel 1954 A Reconsideration of Some Parole Prediction Factors. American Sociological Review 19:335–341.

Glueck, Sheldon; and Glueck, Eleanor 1937 Later Criminal Careers. New York: Commonwealth Fund.

Glueck, Sheldon; and Glueck, Eleanor 1959 Predicting Delinquency and Crime. Cambridge, Mass.: Harvard Univ. Press.

Goodman, Leo 1952 Generalizing the Problem of Prediction. American Sociological Review 17:609–612.

Guttman, Louis 1941 Mathematical and Tabulation Techniques. Pages 251–364 in Social Science Research Council, Committee on Social Adjustment, The Prediction of Personal Adjustment, by Paul Horst et al. New York: The Council.

Hart, Hornell 1954 Expectation of Life: Actual Versus Predicted Trends. Social Forces 33:82–85.

Hotelling, Harold 1935 The Most Predictable Criterion. Journal of Educational Psychology 26:139–142.

Kaplan, A.; Skogstad, A. L.; and Girshick, M. A. 1950 The Prediction of Social and Technological Events. Public Opinion Quarterly 14:93–110.

Kirby, Bernard C. 1954 Parole Prediction Using Multiple Correlation. American Journal of Sociology 59: 539–550.

Lavin, David E. 1965 The Prediction of Academic Performance: A Theoretical Analysis and Review of Research. New York: Russell Sage Foundation.

Lazarsfeld, Paul F.; and Franzen, Raymond H. 1945 Prediction of Political Behavior in America. American Sociological Review 10:261–273.

Mccormick, Thomas C. 1952 Toward Causal Analysis in the Prediction of Attributes. American Sociological Review 17:35–44.

Mannheim, Hermann; and Wilkins, Leslie T. 1955 Prediction Methods in Relation to Borstal Training. London: H.M. Stationery Office.

Meehl, Paul E. 1954 Clinical Versus Statistical Prediction. Minneapolis: Univ. of Minnesota Press.

Merton, Robert K. (1948) 1957 The Self-fulfilling Prophecy. Pages 421–436 in Robert K. Merton, Social Theory and Social Structure. Rev. & enl. ed. Glencoe, III.: Free Fress.

Moore, Wilbert E. 1964 Predicting Discontinuities in Social Change. American Sociological Review 29:331-338.

Mosteller, Frederick et al. 1949 The Pre-election Polls of 1948. New York: Social Science Research Council.

Ogburn, William F. (1922) 1950 Social Change, With Respect to Culture and Original Nature. New edition with supplementary chapter. New York: Viking.

Ogburn, William F. 1934 Studies in Prediction and the Distortion of Reality. Social Forces 13:224–229.

Ogburn, William F.; Adams, Jean L.; Gilfillan, S. C. 1946 The Social Effects of Aviation. Boston: Houghton Mifflin. → Includes an extended discussion of the methodology of prediction and a selected bibliography.

Ohlin, Lloyd E. 1951 Selection for Parole. New York: Russell Sage Foundation.

Popper, Karl R. 1957 The Poverty of Historicism. Boston: Beacon.

President’S Research Committee On Social Trends 1933 Recent Social Trends in the United States. 2 vols. New York: McGraw-Hill. ’ Incorporates the results of thirty studies of selected social and economic trends in the United States from 1900 to 1930.

Reiss, Albert J. Jr. 1951 The Accuracy, Efficiency, and Validity of a Prediction Instrument. American Journal of Sociology 56:552–561.

Sawyer, Jack T. 1966 Measurement and Prediction, Clinical and Statistical. Psychological Bulletin 66: 178–200.

Schuessler, Karl F. 1954 Parole Prediction: Its History and Status. Journal of Criminal Law, Criminology, and Police Science 45:425–431.

Simon, Herbert A. (1954) 1957 Bandwagon and Underdog Effects of Election Predictions. Pages 79–87 in Herbert A. Simon, Models of Man: Social and Ra tional; Mathematical Essays on Rational Human Behavior in a Social Setting. New York: Wiley.

Small, Albion W. 1916 Fifty Years of Sociology in the United States (1865–1915). American Journal of Sociology 21:721–864.

Social Science Research Council, Committee On So Cial Adjustment 1941 The Prediction of Personal Adjustment, by Paul Horst et al. New York: The Council.

Sorokin, Pitirim A. (1941) 1962 Social and Cultural Dynamics. Volume 4: Basic Problems, Principles and Methods. Totowa, N.J.: Bedminster Press.

Star, Shirley A. 1950 The Screening of Psychoneurotics in the Army: Technical Development of Tests. Pages 486–567 in Samuel A. Stouffer et al., Measurement and Prediction. Studies in Social Psychology in World War Ii, Vol. 4. Princeton Univ. Press.

Terman, L. M. et al. 1938 Psychological Factors in Marital Happiness. New York: McGraw-Hill.

Thorndike, Edward L. et al. 1934 Prediction of Vocational Success. New York: Commonwealth Fund.

Toynbee, Arnold J. 1934–1961 A Study of History. 12 vols. Oxford Univ. Press.

Tylor, Edward B. (1871) 1958 Primitive Culture: Researches Into the Development of Mythology, Philosophy, Religion, Art and Custom. Volume 1: Origins of Culture. Gloucester, Mass.: Smith.

U.S. National Resources Committee, Science Committee 1937 Technological Trends and National Policy, Including the Social Implications of New Inventions. Washington: Government Printing Office.

U.S. National Resources Committee, Science Committee 1938 The Problems of a Changing Population. Report of the Committee on Population Problems. Washington: Government Printing Office.

Westoff, Charles F.; Sagi, Philip C.; and Kelly, E.Lowell 1958 Fertility Through Twenty Years of Marriage: A Study in Predictive Possibilities. American Sociological Review 23:549–556.

PREDICTION

Prediction is a central concept in science and politics, with important ethical implications. Its meanings, however, differ in important if subtle ways in these different realms, which can cause confusion about the appropriate relationship between science and society. Distinguishing clearly between the meanings is crucial for understanding and prescribing an appropriate role for science in political and ethical decision making.

Prediction as Confirmation

One way of looking at science, championed especially by the influential twentieth-century philosopher Karl Popper, is to view knowledge acquisition as a process of first making, and then testing, falsifiable hypotheses (Popper 1992). According to this view, the first of these two activities generates predictions about the consequences of the hypotheses, and the second confirms or refutes the predictions. Perhaps the emblematic example of this type of prediction is how Albert Einstein's general theory of relativity, published in 1916, predicted that the path traveled by light would be bent by the force of gravity, and was later confirmed by Arthur Eddington's 1919 experiment in which this bending was observed during a solar eclipse. From this perspective, science is in its essence a prediction-generating-and-testing activity.

But the notion of prediction as inherent in science itself rests on a particular meaning of the word and a particular notion of what counts as science. Whereas common usage of the word prediction refers to the foretelling of future events, philosophers of science have viewed prediction as the process of deducing consequences from hypotheses independent of any sense of time. Indeed hypotheses that are temporally dependent for their correctness are said to have very little predictive power, because they are only true under limited circumstances.

The power of science, from this perspective, lies in its ability to make highly general, experimentally testable predictions about natural phenomena independent of time or other contexts external to the phenomena—light should always be bent by gravity. The temporal or locational power of a prediction (if one drops a paperweight from a particular desk at a particular time, one can predict that it will accelerate toward the center of the earth at thirty-two feet per second per second) is trivial compared to the more general, explanatory power of Isaac Newton's gravitational law (the attraction exerted by gravity between any two bodies is directly proportional to the masses of the bodies and inversely proportional to the square of the distance between them). The power of such generality is especially on display when scientific principles inform technological innovation, because the application of invariant laws ensures the uniform behavior of engineered devices from airplane wings and barometers to electronic circuits and clock pendulums.

Yet this view of science is problematic because it dismisses as nonscientific those disciplines that are not grounded in using experiments to test falsifiable predictions, such as most social sciences, paleontology and geology, system-level biology, ecology, and even some branches of the quantitative physical sciences dealing with complex, non-linear systems. Notably these are also the disciplines of science that most directly seek to understand the complexity of human experience and natural systems. More subtly, but of equal importance, generality in scientific prediction is almost always achieved through careful control of experimental conditions, or stripping away contextual complications. Newtonian mechanics, for example, operates as predicted in vacuums, in frictionless environments, and on rigid bodies, conditions that are often not met in the real world. Generality, as Nancy Cartwright (1983) argues, is often achieved at the expense of reality. Thus if science is in its essence a prediction-generating activity, where prediction means logical inference, then science can have only a limited capacity to inform about how the real world works.

Prediction as Foretelling

Yet it is in this real world that people make decisions about how to act. Indeed turning now to prediction in its more conventional sense, human decision making can be understood as an inherently predictive activity. Human action at every scale from the most mundane to the most ambitious aims at connecting the actions that one takes to some set of desired or expected outcomes. While such fields as philosophy, psychology, and economics have confronted the problem of how humans can make better decisions in light of existing knowledge and experience, it is only in the past several decades that science and technology have begun to offer the credible promise of actually predicting future events as an aid to decision making. Simultaneous advances in computer power, data acquisition technologies, and mathematical modeling are now being applied to predicting everything from economic trends and election results to the spread of diseases and the behavior of the ocean-climate system. This promise of a scientifically legitimate predictive capability has proven extremely attractive to decision makers interested in problems as diverse as selecting appropriate crops to plant in a specific region, assigning rates to insurance policies, and negotiating international environmental treaties. To serve such interests, each year billions of dollars are spent on science and technology aimed at improving predictive capabilities.

Prediction of future events is most familiar—and successful—in the area of short-term weather. Indeed there is a crucial connection between the familiarity and the success of weather predictions. Weather forecasters, who make upward of 10 million forecasts each year in the United States alone, are able to constantly test and refine their predictive skills because they can compare their predictions to the weather conditions that actually occur (realizations). At the same time, decision makers who use weather forecasts (everyone from individuals deciding whether to carry an umbrella to military generals deciding how to deploy their forces) are able to develop judgment about the reliability of weather forecasts based on their own multiple experiences, and integrate this judgment into their decision processes. Moreover the production of weather predictions is linked to their use by a dynamic and sophisticated enterprise, including the mass media and companies that sell weather predictions, whose goal is to communicate the forecasts to those who might benefit from them. Finally many decisions based on weather predictions—for example, whether to evacuate a town, mobilize snow plows, or ground a fleet of airplanes—entail significant costs, and if predictions turn out to be wrong, those issuing them may be held accountable for their mistakes.

One or more of these attributes—the ability to test predictions, gain experience in their use, communicate them effectively, and hold people accountable—are missing from most other areas of scientific prediction. The consequences of this distinction are especially important in public policy, for example, where decision makers turn to scientists to predict future costs of a public program, the health consequences of particular levels of chemicals in the environment, the future prospects of an endangered species, the regional climate impacts that can be expected as a result of greenhouse gas emissions, or the behavior of buried nuclear waste. Such predictive challenges are characterized by the fact that the event or condition to be predicted plays out over decades or even centuries, and may represent a temporally and spatially unique set of conditions within an open system. The learning necessary to improve predictive accuracy in such cases is thus very difficult to acquire, because (a) predictions cannot be compared to actual outcomes; (b) causes of error in predictions are contingent on specific conditions and thus cannot be generalized; or (c) in many cases, both.

Obstacles to Prediction

A well-known example of the first kind of difficulty are predictions of long-term climate change, which are the source of considerable scientific and political debate, yet cannot be confirmed in the time frame within which policy decisions about climate change will have to be made. Representative of the second problem was the inability of economic models to predict the change in the relationship between energy consumption and economic growth in the United States that occurred after the Arab oil embargoes of the 1970s. Economists had understood economic growth to be tightly coupled to rising energy use, and thus predicted that the embargo would cripple the U.S. economy. Actual events showed that existing energy technologies could be mobilized to significantly boost energy efficiency, and thus economic activity, in the absence of increased consumption (Schurr 1984). While this insight is important and revealed why past predictions were wrong, it is unlikely to add much to the ability to predict future economic growth trends, because such trends are influenced by innumerable variables of which energy use is only one.

Moreover it is likely that accurate predictions of the behavior of some types—perhaps most types—of natural and social systems are impossible even in theory, due to their complexity, nonlinearity, and openness. Frequentist approaches to prediction, which rely on probabilistic characterizations of past system behavior to predict future behavior, founder on the fact that, for an open system, there is no reason to think that past behavior (even if it has been correctly characterized) will continue unchanged into the future. Deterministic approaches to prediction seek to avoid the pitfalls of frequentist strategies by using first principles (described mathematically) to ascertain causal relations between past, present, and future conditions. Yet determinism has to confront the practical reality that choices must always be made about which aspects of the system are worth characterizing, and which are not. For an open system, such choices are always made on the basis of incomplete knowledge. For example, long-term behavior of complex systems are often dependent on small variations in initial conditions, which means that knowledge of present conditions would have to be characterized with complete accuracy to insure accurate predictions, while errors in this characterization would tend to compound over time. This is why weather forecasts, which depend on knowing the present state of the atmosphere and then projecting future behavior, are accurate to a maximum of about two weeks.

Alternatives to Prediction

In this light, one may well ask the following: Given the limitations, what good is all this science aimed at predicting complex systems?; and If the ability to predict the future is really so limited, how are people going to be able to make successful decisions? These questions are not unrelated.

In considering the first question, the important point is that insight about how complex systems behave may be valuable for reasons other than an ability to predict the future. Charles Darwin's theory of natural selection is one of the most powerful, influential, and enlightening theories of modern science, yet it is predictive in neither the explanatory sense (in that it is not easily falsifiable) nor the temporal sense (in that it can reveal little about how species will evolve in the future). Yet it offers enormous insight into how the natural world works, insight that can enhance understanding and appreciation of the interconnectedness of all things and inform decision making in light of this awareness. Similarly research on ecosystems, the climate system, social systems, and the connections among such systems can help explain causal relations among various system components, characterize past and present conditions, act as an alert to impending problems, and point toward potential solutions. But it is a very different thing to ask science to elucidate the general relations between greenhouse gas emissions and climate behavior (a difficult enough task), than it is to demand that science accurately predict how these relations will unfold on a regional level through the twenty-first century.

From this perspective it is important to recognize that, while decisions always carry with them some expectation of what the future will look like after the decision is made, good decisions—those that move in the direction desired by the decision makers—do not depend on accurate predictions. Numerous strategies exist for making effective decisions in the presence of scientific insight but the absence of accurate predictions.

One approach is prevention. For example, past experience shows that many areas of California are subject to earthquakes, and this knowledge has been sufficient to guide activities, such as better construction practices, that can reduce loss of life and property from earthquakes, without needing to predict them. Another approach is trial and error informed by understanding and monitoring. For example, the Federal Reserve Board modulates macroeconomic behavior in the United States by making small, incremental changes in interest rates and then seeing how those changes affect economic performance. Similarly, biologists and natural resource managers have increasingly been drawn to an adaptive, incremental approach for managing fragile ecosystems. The role of science here is to assess current conditions, suggest plausible cause-and-effect relations for guiding decisions, and then monitor the effects of actions taken to manage the system. This allows learning both from success and error, and it keeps the costs of errors relatively small, because decisions are incremental.

A third approach is to adopt hedging strategies for addressing future risks whose probabilities cannot be accurately predicted. For example, a no-regrets approach to global warming could mandate the adoption of energy efficient technologies whose lifetime costs are about the same as less-efficient technologies, and at the same time introduce reforms in land-use practices and insurance coverage that would reduce exposure to future climate events, whether or not they are caused by global warming. A fourth strategy is to introduce redundancy into the system, for example by combining geologic and engineering containment strategies for isolating nuclear waste, rather than depending on only one approach.

Political and Ethical Implications

Rather than making decisions in anticipation of a particular, predicted future, these sorts of approaches aim at building resilience into a system, a quality that allows for desired outcomes to be attained under a variety of plausible futures. Yet these approaches also demand that political commitments to action be made under conditions of uncertainty. This demand is not inherently problematic—indeed all decisions are made under conditions of uncertainty—but the rising expectation that science can provide accurate predictions may undercut the political motivation to actually take action, especially if such action entails political risk. The short-term benefits for both politicians and scientists of the predictive approach are clear: Politicians can avoid making tough decisions yet point to research as a step in the right direction, while scientists receive more funding to develop more accurate predictions. As a result, however, political discourse can shift from a discussion about the values and ethics that should inform action, to an endless debate about the technical merits of contesting scientific predictions. This dynamic is on stark display in a number of high-profile environmental controversies. The elusive promise of accurate scientific predictions may not only delay necessary action, but undermine the vitality of democratic debate.

DANIEL SAREWITZ

SEE ALSO Global Climate Change;Incrementalism.

BIBLIOGRAPHY

Cartwright, Nancy. (1983). How the Laws of Physics Lie. Oxford:, UK Oxford University Press.

Popper, Karl. (1992 [1935]). The Logic of Scientific Discovery. New York: Routledge. Hugely influential philosophical treatment of how reliable scientific knowledge is created.

Sarewitz, Daniel R.; Roger A. Pielke Jr.; and Radford Byerly Jr., eds. (2000). Prediction: Science, Decision Making, and the Future of Nature. Covelo, CA: Island Press. Contains expanded treatment of many of the arguments and examples in this article.

Schurr, Sam H. (1984). "Energy Use, Technological Change, and Productive Efficiency: An Economic-Historical Interpretation." Annual Review of Energy 9: 409–425.

Prediction

EMPIRICAL AND PRAGMATIC SIGNIFICANCE

CRITICISM OF METHODOLOGY

BIBLIOGRAPHY

Prediction can be defined as “to declare in advance: foretell on the basis of observation, experience, or scientific reason” (Webster’s 7th New Collegiate Dictionary). Predictions can be short-term, long-term, qualitative, quantitative, negative, positive, optimistic, or pessimistic. They can be based on hunches, guesses, folklore, astrology, palm reading, extrapolations from data, or scientific laws and theories or alleged laws of historical change. The term prediction overlaps with prophecy but has somewhat different connotations.

Not until the seventeenth and eighteenth centuries did prediction take on its modern connotations. Pierre-Simon Laplace (1749–1827), arguably its first major theorist, presented a systematic answer to the major theoretical and practical problem prediction raises: Why are we so successful in some areas of research (e.g., astronomy, physics, and chemistry) but conspicuously less successful in others, particularly human behavior? The most frequently given answers are that some phenomena are intrinsically unpredictable, or we are not using correct scientific methods, or the complexity of the phenomena rule it out, or it is due to limits of human ignorance, fallibility, and superstition. The twentieth century saw significant developments in the theory and practice of prediction, with simultaneous significant advances in predictive accuracy and a greater awareness of the limits of such accuracy.

EMPIRICAL AND PRAGMATIC SIGNIFICANCE

There are four major theories concerning the possibility of predictive success in the areas where, so far, they have been rather limited. The first is Laplacian determinism, named for Laplace, who asserted that an intelligence endowed with omniscience (who therefore would know the current position and velocity of all particles, as well as the laws that control their behavior) could predict every future event.

The second view, the covering law model, is based on the two principles involved in the Laplacian view. As developed by Karl Popper (1934), it requires that an adequate explanation be deduced from universal laws and initial conditions (a generalization of Laplace’s positions and velocity). The more general and more precise the prediction, the better it is. This is because it is more testable, which in Popper’s view means “falsifiable.”

The third position , probabilistic prediction, is more modest but still in accordance with Laplace’s view that our ignorance and fallibility forces us to rely on probability, not certainty, in our predictive endeavors. We can predict a 60 percent chance of rain this weekend, but not with the virtual certainty of astronomical predictions (e.g., that Venus will be in transit in 2112) or those based on (Newtonian) laws of gravity and motion. The quantum physics uncertainty principle is often seen as the paradigm case of this type, although weather predictions and many social phenomena may be better examples, given that quantum electrodynamics has the most numerically precise, accurate predictions in the history of science.

The final theory, pattern prediction, originated with Warren Weaver (1894–1978), a natural scientist, but was made widely known to economists and social scientists by Friedrich von Hayek (1899–1992). This position seems furthest from Laplacian determinism. Pattern prediction downplays precise quantitative predictions, and has a weakened type of testability, due to the theory of complex phenomena adapted from Weaver. It appears to harmonize well with modern chaos theory and with probabilistic reasoning about phenomena. However, the most recent developments in the theory and practice of prediction may be making it more susceptible to quantitative precision.

Any adequate theory of prediction should allow retrodiction as a special type. If one predicts that devaluing the currency will lead to inflation, then a study of such policies in ancient Rome, Egypt, or China should find these results. This can be defined as prediction, although it concerns not the society studied but what the scholars of such societies should find.

CRITICISM OF METHODOLOGY

B. F. Skinner’s behaviorism emphasized prediction and control; his position was extreme Laplacian determinism. The reason for our inability to be as successful at predicting human behavior as we are with predictions made in physics and astronomy is due to our failure to apply proper scientific method, primarily because of a lingering antiempirical belief in an inner man not subject to the laws of nature. But the history of science since Newton has postulated several unobservable entities (e.g., atoms, gravity, natural selection) with spectacular practical and empirical success.

Logical positivism (including Popper’s weakened version) associated predictions with testability, meaning both verifiability and falsifiability. But it has been argued on both logical and historical grounds that a false prediction does not always disprove a theory. Nor does verification of predictions prove the theory true for purely logical reasons (the fallacy of affirming the consequent).

Popper that, in contrast to unconditional historical prophecies, scientific predictions are conditional (1963). This is based on his model described above and the laws he argues are conditional not asserting the existence of the initial conditions. But the model only works for systems that are well isolated, stationary, and recurrent (Popper 1963). This is why predictions of eclipses and the regularity of the seasons can be made accurately, because the solar system is such a system. Such systems are not typical; they are special cases where scientific prediction becomes particularly impressive.

In addition, there are arguments based on the nondeterministic character of classical physics. The “three-body problem” undermines Laplacian determinism. The three-body problem arises out of the inability of physics (so far) to use Newton’s laws for more than two bodies.

The twentieth-century scientist Edward Lorentz, distinguished between two types of determinism thus: “one with clear rules that apply always and everywhere” so that repetition of the same conditions makes prediction possible and another where “small variations aggregate and amplify” (Kaplan and Kaplan, 2006, p. 221) and so repetition prevents prediction. The former will result in long-run variation canceling out, with a clear pattern emerging. The latter leads to what are now called “chaotic systems,” which are extremely sensitive to initial conditions.

Despite these impediments, progress has been made in improving meteorological and other types of predictions by the use of “ensemble prediction” as well as “threshold” and “pattern effects.” The former involves turning predictability itself into a variable, such as temperature or rainfall. The latter combines Weaver’s pattern prediction with the important idea of threshold effects, as in the second type of determinism described in the preceding paragraph. Threshold effects are best illustrated by the straw that broke the camel’s back.

Developments in the last two decades are improving efforts to predict possible disasters such as earthquakes, tornadoes, epidemics, and perhaps climate change, as well as the interactions of large groups of people, their societal effects, and their impact on the environment. (Cribb, 2006). Great use has been made of modern computers, often employing simulated effects over centuries (which leaves a great margin of error due to unreliable data), possibly unjustified assumptions in constructing the models, and human fallibility, yet there are good indications of improved predictive ability with computers.

Nonetheless, we are nowhere near Laplace’s omniscient intelligence, and for theoretical and practical reasons we are unlikely ever to be. The major reason is that the mathematics and data used are both so complicated (millions and millions of lines in a computer code) that no one person can master it. In addition, it relies on Baysean probability theory. The mathematics of that is not very complex, but it requires initial assignments of probability, which can then be modified by new evidence. However, there are numerous problems in deciding how to assign these so that they are not too arbitrary.

SEE ALSO Behaviorism; Determinism; Hayek, Friedrich August von; Popper, Karl; Positivism; Probability; Skinner, B. F.

BIBLIOGRAPHY

Cribb, Julian. 2006. Predicting the Future: It’s Becoming a Science. Cosmos (September 20).

Kaplan, Michael, and Ellen Kaplan. 2006. Chances Are. New York: Viking.

Kuhn, Thomas. 1962. The Structure of Scientific Revolutions. Chicago: University of Chicago Press.

Popper, Karl. [1934] 1959. The Logic of Scientific Discovery. New York: Basic Books.

Popper, Karl. 1963. Conjectures and Refutations. London: Routledge.

Skinner, B. F. 1953. Science and Human Behaviour. New York: Macmillan.

Calvin Hayes

pre·dic·tion / priˈdikshən/ • n. a thing predicted; a forecast: a prediction that the Greeks would destroy the Persian empire. ∎  the action of predicting something: the prediction of future behavior.

Prediction (Magazine)

British magazine founded in 1936 dealing with astrology and the occult. Brief features articles in each issue cover such topics as the tarot, palmistry, graphology, yoga, and magic. Astrological forecasts are featured. Prediction is published from Link House, Dingwall Ave., Croydon, CR9 2TA, England.