SENSITIVITY ANALYSES

Technically, a sensitivity analysis is a calculation or estimation, quantitative or not, in which all variables except one are held constant. This allows for a clear understanding of the effects of changes in that variable on the outcomes of the calculation or estimation. The methodologies of sensitivity analysis are well established in some areas of research, particularly those that employ methods of risk assessment and computer modeling (Satelli, Chan, and Scott 2000). However, the concept of sensitivity analysis has considerable potential for policy research, especially for understanding the role of different types of knowledge as factors contributing to particular value or ethical outcomes related to scientific research or technological change.

Potential use in Policy Making: Some Examples

In the context of research intended to support policy making a sensitivity analysis can help identify and frame the dimensions of a problem and thus clarify the potential efficacy of possible interventions. Consider a hypothetical example. There is a city in a desert that continually faces stress on its water resources. City officials invariably face finite time and budgets but have to make decisions about the community's water use. It is likely that they will hear from advocates proposing the development of new water projects such as dams and reservoirs as well as advocates who call for a reduction in water use in the community. Inevitably a question will arise: To what degree should the city consider limiting the use of water, for example, through conservation, versus increasing supply, for example, by building a new dam?

A sensitivity analysis can help policy makers understand the source of stresses on the community's water resources. Specifically, does stress result primarily from a growing population or from limited storage of water? From drought and climate? From a combination? If so, to what degree? The following idealized example shows how a sensitivity analysis might be organized in this case.

1. A valued outcome is identified. In this instance the variable is water availability as measured by reservoir storage. Of course, other valued outcomes might be selected, and other measures might be selected.
2. The existing literature is surveyed to assess the range of factors expected to influence the valued outcome over a period of time that is relevant to the decision context. For water resources the period of concern might be the upcoming decade. The two factors identified to be the most important influences affecting water availability might be rainfall and municipal water usage.
3. With the two factors identified, the next step is to return to the literature to identify the distribution of views on the effects of rainfall and water use on water availability. The goal here is to identify the range of perspectives on the independent influence of (a) rainfall and (b) municipal water use on water availability.
4. With a quantitative understanding of 3(a) and 3(b), it will be possible to compare the sensitivity of water availability to each of the two factors, with possible implications for decision making.

For example, if a sensitivity analysis showed that water use was expected to grow faster than variations in existing storage related to climate, policy makers might consider managing water use. Similarly, if a sensitivity analysis showed that reservoir storage was largely insensitive to accumulated rainfall, perhaps because there was far more rainfall than storage capacity, policy makers might consider building new reservoirs. A sensitivity analysis cannot determine what means and ends are worth pursuing, but it can shed some light on the connection of different means and ends.

The point of a sensitivity analysis is to identify factors that may be influenced by decision making in order to make desired outcomes more likely than undesired outcomes. Because the process of framing a problem (for example, using too much water versus not having enough water) necessarily implies some valued outcomes, a sensitivity analysis can help make those values explicit and demonstrate the prospects that different policy interventions might lead to desired outcomes.

More generally, in light of the multicausal nature of most phenomena that are of interest to policy makers (for instance, all the factors implicated in the supply of and demand for water in a large urban setting) and the large uncertainties typically associated with efforts to quantify the relationships between a particular cause (such as the challenges associated with projecting water supply over a period of decades) and an impact (for example, the difficulties of understanding who will be affected the most by water shortages and oversupply decades in the future), one obvious approach to guiding policy decisions is to look for areas of relative strength in relationships between causes and impacts and focus research to support decision making in those areas.

In a somewhat less idealized example Pielke et al. (2000) show that in light of scientific understanding as reported by the Intergovernmental Panel on Climate Change, demographic and socioeconomic change will be twenty to sixty times more important than climate change in contributing to economic losses related to tropical cyclones over the next fifty years. This sensitivity analysis suggests that (1) even if all losses resulting from climate change were prevented, the overall benefit would be dwarfed by increasing losses caused by the growth of populations and economies, and (2) research priorities relevant to the tropical cyclone threat could reflect those relationships by focusing on issues of preparation, planning, infrastructure, development, and resilience. The order-of-magnitude difference between these two sources of tropical cyclone impacts strongly suggests that more research on the sensitivity of tropical cyclones to climate changes is not likely to change the implications for decision making.

In another example one might consider the changing incidence and impacts of tropical diseases such as malaria to understand how predictions of the influence of climate change compare with other causal factors, such as growth in resistance to antibiotics, changes in healthcare delivery systems, migration and growth of populations, and annual-to-interannual climate variability.

Goals of Sensitivity Analyses

The goal is not to predict but to provide information about the relative sensitivity of impacts to various causal factors. That information can enhance the bases for effective decision making in the context of values and ethics as well as decisions about science priorities intended to support the generation of knowledge useful in pursuing desired outcomes without additional reduction in or characterization of scientific uncertainty.

In a policy setting sensitivity analysis does not attempt to resolve scientific disputes about causes of societal impacts but to compare and assess existing quantified predictions and observations of the multiple causes of such impacts to identify strong causal links. As the examples of water resources and tropical cyclones show, a sensitivity analysis approach can lessen the perceived need for reduction of uncertainty about future behavior as a prerequisite for decision making and point toward research avenues that can provide knowledge that can be useful in addressing high-priority sources of environmental change and societal vulnerability. Thus, sensitivity analysis can be an important tool for science policy decision makers in their attempt to enhance the societal value of their portfolios.

ROGER A. PIELKE, JR.

SEE ALSO Science Policy.

BIBLIOGRAPHY

Pielke, Roger A., Jr., Roberta A. Klein; and Daniel Sarewitz. (2000). "Turning the Big Knob: Energy Policy as a Means to Reduce Weather Impacts." Energy and Environment 11(3): 255–276.

Satelli, Andrea; Karen Chan; and E. Marian Scott, eds. (2000). Sensitivity Analysis. New York: Wiley. Focuses on the methodologies of sensitivity analysis and with emphasis on computer models. Written for a reader with a high level of expertise.

Saltelli, Andrea; Stefano Tarantola; Francesca Campolongo; and Marco Ratto. (2004). Sensitivity Analysis in Practice: A Guide to Assessing Scientific Models. New York: Wiley. Targeted at the non-expert, this book discusses the selection of sensitivity analysis methods and their applications.