EVALUATION OF PUBLIC HEALTH PROGRAMS

Public health programs are essential to protect and enhance the health of the population and are, by definition, publicly funded. It is therefore essential that they be evaluated. The benefits of public health programs often extend well beyond the immediate target population—reductions in communicable diseases or environmental hazards may benefit the whole world. Different disciplines approach evaluation in different ways.

Epidemiologists distinguish between an intervention's efficacy (its ability to work under ideal circumstances), its effectiveness (results obtained under real-world circumstances), and its efficiency (results obtained relative to the resources expended). An intervention cannot be effective unless it is efficacious. Randomized studies are the gold standard for studies of efficacy, since they ensure that study groups are comparable (in the long run). They thereby help to ensure that the results are due to the intervention. But they are more feasible for single interventions directed at individuals than for complex programs directed at groups, so their potential for evaluating public health programs is limited. Community trials, in which groups are the unit of randomization, are difficult and expensive. Quasi-experimental approaches try to obtain the advantages of randomization through careful selection of comparison groups (often multiple groups) and sophisticated statistical analysis.

Administrative sciences tend to assume that a program's efficacy has been demonstrated and concentrate on other dimensions of evaluation. Using the World Health Organization's 1991 Health Programme Evaluation as a basis, J. E. Veney and A. D. Kaluzny (1991) recommend that five aspects of a health program be evaluated: its relevance (to the needs of the population and to social priorities), progress (implementation), efficiency (outcomes achieved in relation to resources expended), effectiveness (extent to which predetermined objectives are met), and impact (long-term outcomes). Approaches for measuring these dimensions include monitoring, case studies, and various epidemiologic designs.

Economists compare a particular program to a possible alternative. Costs are expressed in monetary units and should be marginal (cost of producing the last unit of output) rather than average (across all units of output). Outcomes may be assumed to be identical and therefore not measured (cost-minimization analysis) or expressed in natural units like cases prevented (costeffectiveness), monetary units (cost-benefit), or universal units like quality-adjusted life years (costutility).

Public health managers often use logic models for evaluating public health programs, setting out the program's expected operation in matrix format and showing the activities, target groups, and short-and long-term outcomes for each component. Much more highly developed is the PRECEDE-PROCEED framework of L. W. Green and M. W. Kreuter, a nine-step process for the design of a health-promotion program (social, epidemiologic, behavioral/environmental, educational/organization, and administrative/policy diagnoses) and its evaluation (implementation and process, impact, and outcome evaluation). Less systematically, a program can be illustrated as a flow chart. All these approaches force program developers to think through a program systematically, and they show program evaluators what to measure. By sampling the program process at various stages evaluators can distinguish between an implementation failure (a potentially good idea that was never properly tested) and a program failure (a bad idea). Governments often produce standards against which local public health programs can be tested. The National Public Health Performance Standards Program is the standard being developed in the United States.

Robert A. Spasoff

(see also: Case-Control Study; Centers for Disease Control and Prevention; Epidemiology; Health Measurement Scales; Statistics for Public Health )

Bibliography

Drummond, M. F.; O'Brien, B.; Stoddart, G. L.; and Torrance, G. W. (1997). Methods for the Economic Evaluation of Health Care Programmes, 2nd edition. Oxford: Oxford University Press.

Gray, J. A. M. (1997). Evidence-based Healthcare: How to Make Health Policy and Management Decisions. New York: Churchill Livingstone.

Green, L. W., and Kreuter, M. W. (1999). Health Promotion Planning: An Education and Ecological Approach, 3rd edition. Mountain View, CA: Mayfield Publishing Company.

Public Health Practice Program Office (2000). National Public Health Performance Standards Program. Atlanta, GA: Centers for Disease Control and Prevention, Division of Public Health Systems.

Rundall, T. G. (1992). "Health Planning and Evaluation." In Maxcy-Rosenau-Last Public Health & Preventive Medicine, 13th edition. Norwalk, CT: Appleton & Lange.

Veney, J. E., and Kaluzny, A. D. (1991). Evaluation and Decision Making for Health Services, 2nd edition. Ann Arbor, MI: Health Administration Press.

World Health Organization (1981). Health Programme Evaluation: Guiding Principles. Geneva: Author.