CRIMINOLOGY AND CRIMINAL JUSTICE RESEARCH: METHODS

Those interested in the study of criminology and criminal justice have at their disposal a wide range of research methods. Which of the particular research methods to use is entirely contingent upon the question being studied. Research questions typically fall into four categories of research: (1) descriptive, (2) exploratory, (3) explanatory, and (4) evaluative (Schutt). Descriptive research attempts to define and describe the social phenomena under investigation. Exploratory research seeks to identify the underlying meaning behind actions and individual behavior. Explanatory research seeks to identify the cause-(s) and effect(s) of social phenomena. Evaluation research seeks to determine the effects of an intervention on individual behavior. These four areas of research are not mutually exclusive; rather, they are designed to be used interactively in order to gain a deeper understanding of the question under investigation.

With this background, the purpose of this entry will be to introduce the reader to the two major research paradigms and issues that organize the field of criminology and criminal justice: quantitative and qualitative research strategies. After describing the different research methodologies several issues related to internal and external validity are identified that are important to bear in mind when assessing the adequacies of distinct research methodologies. The entry closes by highlighting what appears to be the most promising research strategy for criminology and criminal justice.

Quantitative research methods

Quantitative research methods are typically concerned with measuring criminological or criminal justice reality. To understand this process several terms must first be identified. Concepts are abstract tags placed on reality that are assigned numerical values, thus making them variables. Variables are then studied to examine patterns of relation, covariation, and cause and effect. At the most basic level, there exists at least one dependent variable and one independent variable. The dependent variable is commonly referred to as the outcome variable. This is what the researcher is attempting to predict. The independent variable is commonly referred to as the predictor variable, and it is the variable that causes, determines, or precedes in time the dependent variable (Hagan). Consider the following examples.

Criminological theorists may be interested in studying the relationship between impulsivity (independent variable) and criminal behavior (dependent variable). In studying such a relationship, scholars create a summated scale of items that is designed to indirectly measure the concept of impulsivity. Then, this impulsivity scale is used to predict involvement in criminal behavior. Criminal justice scholars may be interested in studying the effects of a mandatory arrest policy (independent variable) on future patterns of domestic violence (dependent variable). In studying such a question, scholars typically evaluate the effect of an arrest, compared to some other sanction, on the future criminal behavior of the arrestee. Thus, quantitative research methods involve a pattern of studying the relationship(s) between sets of variables to determine cause and effect.

Three criteria are needed to establish causality. The first is association. That is, the independent and dependent variables must be related to one another. The second is time order; the independent variable must precede the dependent variable in time. Finally, there is the issue of nonspuriousness. This occurs if the relationship between the independent and dependent variables is not due to variation in some unobserved third variable.

There are a number of different quantitative research methods available to researchers, most of which fall under the rubric of a research design, which loosely can be defined as the plan or blueprint for a study that includes the who, what, where, when, why and how of an investigation (Hagan). These research methods include: survey research, experimental and quasi-experimental research, cross-sectional research, longitudinal research, time series research, and meta-analysis.

Survey research. Serving as the most frequently used mode of observation within the social sciences, including criminology (Maxfield and Babbie), survey research involves the collection of information from a sample of individuals through their responses to questions (Schutt). Survey research is generally carried out via mail, telephone, computer, or in person.

Typically, surveys contain a combination of open- and closed-ended questions. Open-ended questions ask the respondent to provide an answer to a particular question. For example, the respondent may be asked: "What do you think is the most important problem facing residents in your neighborhood today?" Then in their own words, the respondent would provide his or her answer. On the other hand, closed-ended questions ask the respondents to select an answer from a list of choices provided. For example, the question asked above would read exactly the same only now respondents are provided with a list of options to choose from: "What do you think is the most important problem facing residents in your neighborhood today? (a) crime, (b) drugs, (c) education, (d) employment, (e) family structure, (f ) poverty, (g) health care, (h) child care, (i) extracurricular activities, ( j) other."

Surveys offer a number of attractive features that make them a popular method of doing research. They are versatile, efficient, inexpensive, and generalizable. At the same time, survey methods may be limited due to problems in sampling, measurement, and overall survey design. When creating a survey, researchers should take care in making sure that the items in the survey are clear and to the point.

Experimental and quasi-experimental research. Some scholars believe that experimental research is the best type of research to assess cause and effect (Sherman; Weisburd). True experiments must have at least three features: (1) two comparison groups (i.e., an experimental group and a control group); (2) variation in the independent variable before assessment of change in the dependent variable; and (3) random assignment to the two (or more) comparison groups (Schutt).

Many experiments contain both a pre-test and a post-test. The former test measures the dependent variable prior to the experimental intervention while the latter test measures the outcome variable after the experimental group has received the treatment. Randomization is what makes the comparison group in a true experiment a powerful approach for identifying the effects of the treatment (Schutt). Assigning groups randomly to the experimental and comparison groups ensures that systematic bias does not affect the assignment of subjects to groups. This is important if researchers wish to generalize their findings regarding cause and effect among key variables within and across groups.

The classic experimental design is one in which there is a pre-test for both groups, an intervention for one group (i.e., the experimental group), and then a post-test for both groups. Consider the following criminal justice example. Two police precincts alike in all possible respects are chosen to participate in a study that examines fear of crime in neighborhoods. Both precincts would be pre-tested to obtain information on crime rates and citizen perceptions of crime. The experimental precinct would receive a treatment (i.e., increase in police patrols), while the comparison precinct would not receive a treatment. Then, twelve months later, both precincts would be post-tested to determine changes in crime rates and citizen perceptions.

There have been several experimental designs in criminology and criminal justice including the Domestic Violence Experiment (Sherman), where offenders were randomly assigned to one of three interventions (arrest, mediation, separation). The Jersey City Police Department's Program to Control Violent Places also utilized an experimental design (Braga et al.). For this study, twenty-four high-activity, violent crime places were matched into twelve pairs and one member of each pair was allocated to treatment conditions in a randomized block field experiment.

On the other hand, quasi-experimental research lacks the random assignment to experimental and control groups, but can be approximated by close and careful matching of subjects across the two groups on several key variables. The two major types of quasi-experimental designs are: (1) nonequivalent control group designs, which have experimental and comparison groups that are designated before the treatment occurs and are not created by random assignment; and (2) before-and-after designs, which have both a pre- and post-test but no comparison group (Schutt).

An example of a nonequivalent control group design is a study of the effect of police actions on seat-belt law violations. For example, Watson selected two communities of comparable size where police enforcement of the law was low. In the experimental community, Watson instituted a media campaign to increase seat-belt usage, followed by increased police enforcement of the seat-belt law. Watson found that the percentage of drivers using seat belts increased in the experimental community but remained stable or declined slightly in the comparison community.

An example of the before-and-after design is the Pierce and Bowers analysis of the impact of the Massachusetts Bartley-Fox gun law. This law carried a one-year minimum prison sentence for the unlicensed carrying of firearms. Their early evaluation showed a decrease in gun-related assaults, robberies, and homicides, but was offset by increases in nongun assaults and robberies using other weapons.

Cross-sectional research. Cross-sectional designs involve studies of one group at one point in time. Therefore, they offer a quick glimpse or snapshot of the phenomena being studied. Typically, they refer to a representative sample of the group and thus allow researchers to generalize their findings (Hagan). Cross-sectional research designs permeate criminology and criminal justice research. Hirschi's famous study of causes of delinquency utilized a cross-sectional design in which he asked male respondents a series of questions related to involvement in delinquent activities and emotional ties to social bonds.

Longitudinal research. There are two commonly used longitudinal research designs, panel and cohort studies. Both study the same group over a period of time and are generally concerned with assessing within- and between-group change. Panel studies follow the same group or sample over time, while cohort studies examine more specific populations (i.e., cohorts) as they change over time. Panel studies typically interview the same set of people at two or more periods of time. For example, the National Crime Victimization Survey (NCVS) randomly selects a certain number of households from across the United States and interviews a member from each a series of seven times at six-month intervals. Cohort studies follow individuals or specific cohorts as they change over time. One classic example of a cohort study was conducted by Marvin Wolfgang and his colleagues in Philadelphia. The authors traced the criminal records of all boys born in Philadelphia in 1945 through the age of eighteen. Similarly, Tracy, Wolfgang and Figlio tracked the criminal history of males and females born in Philadelphia in 1958.

Time-series designs. Time-series designs typically involve variations of multiple observations of the same group (i.e., person, city, area, etc.) over time or at successive points in time. Typically, they analyze a single variable (such as the crime rate) at successive time periods, and are especially useful for studies of the impact of new laws or social programs (Schutt). An example of a time-series design would be to examine the murder rate in the United States over the last twenty years or to compare the murder rate of the United States and Canada over the same period of time.

An interrupted time-series design analyzes a single variable at successive time periods with measures taken prior to some form of interruption (i.e., intervention) and other observations taken after the intervention. An example of an interrupted time-series design may be found in Spelman and Eck (1987). These authors studied the number of larcenies from automobiles in Newport News, Virginia. The intervention in this study was a problem-oriented policing program that consisted of special tracking and investigation of crime incidents. The results showed that the number of larcenies dropped significantly immediately after the intervention took place and remained significantly small for over one year after the intervention. In another interrupted time series study, D'Alessio and Stolzenberg investigated the impact of Minnesota sentencing guidelines on jail incarceration. They found that the onset of the sentencing guidelines increased judicial use of the jail sanction beyond the effect of preexisting trends.

Although time-series designs are especially useful in studying trends over time and how such trends are influenced by some sort of intervention, researchers should be aware of one key feature of time-series designs: the inability to control for all potential spurious effects. Consider the following example. Suppose that a researcher is studying the effect on robberies of a mandatory convenience store law that requires stores to have at least two clerks working during hours of operation. After examining the number of robberies before and after the law took effect, the researcher observed that the number of robberies significantly decreased after the law was instituted. Therefore, the researcher claimed that the law led to the decrease in the number of robberies committed and concluded that the law should be generalized to other locales. However, what the researcher may have failed to consider was the recent capture of two offenders who were committing 75 percent of all convenience store robberies, and who just happened to be captured about the time the law took effect. In sum, researchers need to be careful in making sure that their interpretations of interrupted time-series analyses take into consideration as much information, both empirical and nonempirical, as possible.

Meta-analysis. A recent advent in research methodology is the use of meta-analysis. This research approach is the quantitative analysis of findings from multiple studies. At its core, meta-analysis involves researchers pulling together the results of several studies and making summary, empirical statements about some cause and effect relationship. A classic example of meta-analysis in criminology was performed by Wells and Rankin and concerned the relationship between broken homes and delinquency.

After observing a series of findings showing that the broken-homes-causes-delinquency hypothesis was inconclusive, Wells and Rankin identified fifty studies that tested this hypothesis. After coding the key characteristics of the studies, such as the population sampled, age range, measures (both independent and dependent) used, the authors found that the average effect of broken homes across the studies was to increase the probability of delinquency by about 10 to 15 percent. Perhaps more importantly, they found that the different methods used across the studies accounted for much of the variation in estimating the effect of broken homes. For example, the effect of broken homes on delinquency tended to be greater in studies using official records rather than self-report surveys.

Although the research community has not spoken with one voice regarding the usefulness of meta-analysis, one thing is clear: meta-analysis makes the research community aware that it is inappropriate to base conclusions on the findings of one study. It is because of this important lesson that meta-analysis has become a popular technique in criminological and criminal justice research (Lipsey and Wilson).

Threats to validity

Validity refers to the accuracy of measurement or whether the instrument is in fact measuring what it is suppose to measure (Hagan). While quantitative research methods have permeated criminological and criminal justice research, they are not without problems. Threats to validity are perhaps the most profound and should be acknowledged. Some of these threats are internal and are concerned with whether the observational process itself produced the findings, while external threats are concerned with whether the results were unique and applicable only to the group or target studied (Hagan).

Internal threats. According to Campbell and Stanley, a number of internal threats need to be considered, including: (1) history, (2) maturation, (3) testing, (4) instrumentation, (5) statistical regression, (6) selection bias, (7) experimental mortality, and (8) selectionmaturation interaction. In determining whether a particular design rules out threats to internal validity, Cook and Campbell suggest that "estimating the internal validity of a relationship is a deductive process in which the investigator has to systematically think through how each of the internal validity threats can be ruled out" (p. 55).

External threats. Campbell and Stanley also identify several threats to external validity, including: (1) testing effects, (2) selection bias, (3) reactivity or awareness of being studied, and (4) multiple-treatment interference. These threats are greater for experiments conducted under more carefully controlled conditions (Maxfield and Babbie). Perhaps one of the best methods for assessing threats to external validity is replication, or the repetition of experiments or studies utilizing the same methodology. By replication of key findings, researchers can gain confidence that the results observed in one study may not be due to external validity threats. One of the key examples of replication occurred in the late 1980s when the Minneapolis Domestic Violence Experiment was replicated in six cities throughout the United States (Sherman). Importantly, these replications yielded both similar and contradictory conclusions to those observed in the initial experiment.

Qualitative research methods

Unlike quantitative research methods, qualitative approaches are designed to capture life as participants experience it, rather than in categories predetermined by the researcher. These methods typically involve exploratory research questions, inductive reasoning, an orientation to social context and human subjectivity, and the meanings attached by participants to events and to their lives (Schutt). There are a number of distinctive research designs under this paradigm: (1) participant observation, (2) intensive interviewing, (3) focus groups, and (4) case studies and life histories. Each of these will be discussed in turn.

Participant observation. At its most basic level, participant observation involves a variety of strategies in data gathering in which the researcher observes a group by participating, to varying degrees, in the activities of the group (Hagan). Gold discusses four different positions on a continuum of roles that field researchers may play in this regard: (1) complete participant, (2) participant-as-observer, (3) observer-as-participant, and (4) complete observer. Complete participation takes place when the researcher joins in and actually begins to manipulate the direction of group activity. In the participant-as-observer strategy, the researcher usually makes himself known and tries to objectively observe the activities of the group. The observer-as-participant strategy is very much like a one-visit interview, where the interviewees are also short-term participant observers. Typically, these interviews are conducted with individuals who are known to participate in a designated activity. For example, Jacobs interviewed known active drug dealers in order to gain a better understanding of how the crack business actually operates on the streets. Finally, the complete observer strategy relies on sole observation absent participation from the researcher.

Although several issues must be confronted when engaging in this sort of research, two are of vital importance: (1) objectivity, and (2) "going native." The former deals with the researcher's ability to avoid not only overidentification with the study group, but also aversion to it (Hagan). The latter deals with a situation in which the researcher identifies with and becomes a member of the study group, and in the process abandons his or her role as an objective researcher (Hagan). Even with these cautions, a number of important participant observation studies have been undertaken in criminology and criminal justice including Polsky's study of pool hustlers and con artists, as well as Marquart's study of prison life.

Intensive interviewing. Intensive interviewing consists of open-ended, relatively unstructured questioning in which the interviewer seeks in-depth information on the interviewee's feelings, experiences, or perceptions (Schutt, 1999). Unlike the participant observation strategy, intensive interviewing does not require systematic observation of respondents in their natural setting. Typically, interviewing sample members, and identification and interviewing of more sample members, continues until the saturation point is reached, the point when new interviews seems to yield little additional information (Schutt).

A prominent example of the intensive interviewing technique can be found in a series of studies with active residential burglars (Wright and Decker, 1994) and robbers (Wright and Decker, 1997) in St. Louis. These authors have conducted in-depth interviews with active criminals in their natural environment. Some of these interviews have yielded important theoretical insights that perhaps may not have been garnered via traditional survey methods. Other prominent examples may be found in Fagan and Wilkinson's study of gun-related violence in New York and Jacobs's study of crack addicts in St. Louis.

Focus groups. Focus groups are groups of unrelated individuals that are formed by a researcher and then led in group discussions of a topic (Schutt). Typically, the researcher asks specific questions and guides the discussion to ensure that group members address these questions, but the resulting information is qualitative and relatively unstructured (Schutt).

Although generalizations from focus groups to target populations cannot be precise (Maxfield and Babbie), research suggests that focus group information, combined with survey information, can be quite consistent under certain conditions (Ward et al.). One such criminal justice example is provided by Schneider and her colleagues. These authors examined the implementation process and the role of risk/need assessment instruments for decisions about the proper level of supervision among parolees and probationers. Their use of focus group was able to provide a context for a more complete understanding of the survey results from the probation officers interviewed.

Case studies and life histories. In general, case studies and life histories are in-depth, qualitative studies of one or a few illustrative cases (Hagan). Several criminological examples using this approach exist, and a few in particular have produced some of the most important, baseline information in the discipline today. The classic example is Sutherland's The Professional Thief (1937). In this case study, Sutherland's informant, Chic Conwell, described the world of the professional thief. Other examples include Shaw's The Jack-Roller (1930), which tells the autobiographical story of a delinquent's own experiences, influences, attitudes, and values. Finally, Horatio Alger's tale of street life in New York tells the story of Young Dick, a street boy who is involved in a delinquent life but who is also honest and hardworking. Life-history methods generally involve the analysis of diaries, letters, biographies, and autobiographies to obtain a detailed view of either a unique or representative individual (Hagan). A classic example of the life-history method is Teresa and Renner's My Life in the Mafia (1973).

Future of research methods in criminology and criminal justice

Although the preceding discussion has portrayed the two main research paradigms, quantitative and qualitative research methods, as two ends of the research continuum, it was not meant to imply that the two are mutually exclusive. On the contrary, the future of research methods in criminology and criminal justice lies in the combination of quantitative and qualitative research approaches. Illustrated below are two successful integrations.

The first, by Eric Hirsch, used a combination of methods, including participant observation, intensive interviewing, and a standardized survey, to study the 1985 student movement that attempted to make Columbia University divest its stock in companies dealing with South Africa. Hirsch believed that the combination of research methodologies provided a more comprehensive picture of student's motivations.

The second example is from John Laub and Robert Sampson. For quite some time, these two scholars have been working on the reanalysis of one of the classic data sets in criminology, the Unraveling Juvenile Delinquency (UJD) study that was initiated by Sheldon and Eleanor Glueck in 1940. The data contain the original case records of all one thousand sample members as well as detailed archival life records that included information from the "home investigation," which consisted of an interview with family members and offered an opportunity for the investigators to observe the home and family life of sample members. Furthermore, the UJD study included interviews with key informants such as social workers, settlement house workers, clergymen, schoolteachers, neighbors, employers, and criminal justice and social welfare officials. When this detailed information is combined with the statistical information on criminal behavior and other life events, one can begin to appreciate the richness with which Laub and Sampson have been able to document these one thousand lives and contribute much needed information regarding crime over the life course.

The future of criminological and criminal justice research will likely come full circle. Early studies of crime and criminality began with qualitative observations almost to the exclusion of quantitative research. New research topics were observed and highlighted by scholars who wished to forge ahead in the understanding of crime and criminality. Once these topics were brought to the forefront of the field, quantitative research became the choice method of analysis. The future of criminological research must focus on the blending of the two. As John Clausen notes, both case history and statistical data are required "if we are to understand the influences on the lives of persons who have lived through a particular slice of American history" (p. 43).

Alex R. Piquero

Nicole Leeper Piquero

See also Criminology and Criminal Justice Research: Organization.

BIBLIOGRAPHY

Alger, Jr., Horatio. Ragged Dick or, Street Life in New York with the Boot-blacks (1867). New York: Signet Classic, 1990.

Braga, Anthony. A.; Weisburd, David L.; Waring, Elin J.; Mazerolle, Lorraine G.; Spelman, William; and Gajewski, Francis. "Problem-Oriented Policing in Violent Crime Places: A Randomized Controlled Experiment." Criminology 37 (1999): 541–580.

Campbell, Donald T., and Stanley, Julian C. Experimental and Quasi-Experimental Designs for Research. Chicago: Rand McNally, 1963.

Clausen, John. American Lives: Looking Back at the Children of the Great Depression. New York: Free Press, 1993.

Cook, Thomas D., and Campbell, Donald T. Quasi-Experimentation: Design and Analysis Issues for Field Settings. Chicago: Rand McNally, 1979.

D'Alessio, Stewart J., and Stolzenberg, Lisa. "The Impact of Sentencing Guidelines on Jail Incarceration in Minnesota." Criminology 33 (1995): 283–302.

Fagan, Jeffrey, and Wilkinson, Deanna L. "Guns, Youth Violence, and Social Identity in Inner Cities." In Youth Violence, Crime and Justice: An Annual Review of Research, Volume 24. Edited by Michael Tonry and Mark H. Moore. Chicago: University of Chicago Press, 1998. Pages 105–188.

Gold, Raymond L. "Roles in Sociological Field Observations." Social Forces 36 (1958): 217–223.

Hagan, Frank E. Research Methods in Criminal Justice and Criminology. 3d ed. New York: Macmillan, 1994.

Hirschi, Travis. Causes of Delinquency. Berkeley: University of California Press. Hirsch, Eric L. "Sacrifice for the Cause: Group Processes, Recruitment, and Commitment in a Student Social Movement." American Sociological Review 55 (1990): 243–254.

Jacobs, Bruce A. Dealing Crack: The Social World of Streetcorner Selling. Boston: Northeastern University Press, 1999.

Laub, John H., and Sampson, Robert J. "Integrating Quantitative and Qualitative Data." In Methods of Life Course Research: Qualitative and Quantitative Approaches. Edited by Janet Z. Giele and Glenn H. Elder, Jr. Newbury Park, Calif.: Sage, 1998. Pages 213–230.

Lipsey, Mark W., and Wilson, David B. "Effective Intervention for Serious Juvenile Offenders: A Synthesis of Research." In Serious and Violent Juvenile Offenders: Risk Factors and Successful Interventions. Edited by Rolf Loeber and David P. Farrington. Newbury Park, Calif.: Sage, 1998. Pages 313–345.

Marquart, James W. "Doing Research in Prison: The Strengths and Weaknesses of Full Participation as a Guard." Justice Quarterly 3 (1986): 15–32.

Maxfield, Michael G., and Babbie Earl. Research Methods for Criminal Justice and Criminology. Belmont, Calif.: Wadsworth, 1995.

Pierce, Glen, and Bowers, William. "The Impact of the Bartley-Fox Gun Law on Crime in Massachusetts." Boston: Northeastern University Press, 1979.

Polsky, Ned. Hustlers, Beats and Others. Chicago: University of Chicago Press, 1967.

Schneider, Anne L.; Ervin, Laurie; and Snyder-Joy, Zoann. "Further Exploration of the Flight from Discretion: The Role of Risk/Need Instruments in Probation Supervision Decisions." Journal of Criminal Justice 24 (1996): 109–121.

Schutt, Russell K. Investigating the Social World: The Process and Practice of Research, 2d ed. Thousand Oaks, Calif.: Pine Forge Press, 1999.

Shaw, Clifford R. The Jack-Roller: A Delinquent Boy's Own Story. Chicago: University of Chicago Press, 1930.

Sherman, Lawrence W. Policing Domestic Violence. New York: Free Press, 1992.

Spelman, William, and Eck, John E. "Problem-Solving: Problem-Oriented Policing in Newport News." Research in Brief. National Institute of Justice. Washington, D.C., 1987.

Sutherland, Edwin H. The Professional Thief. Chicago: University of Chicago Press, 1937.

Teresa, Vincent, and Renner, Thomas C. My Life in the Mafia. New York: Doubleday, 1973.

Tracy, Paul E.; Wolfgang, Marvin E.; and Figlio, Robert M. Delinquency Careers in Two Birth Cohorts. New York: Plenum, 1990.

Ward, Victoria M.; Bertrand, Jane T.; and Brown, Lisanne F. "The Comparability of Focus Group and Survey Results: Three Case Studies." Evaluation Review 15 (1991): 266–283.

Watson, Roy E. "The Effectiveness of Increased Police Enforcement as a General Deterrent." Law and Society Review 20 (1986): 293–299.

Weisburd, David. "Design Sensitivity in Criminal Justice Experiments." In Crime and Justice: A Review of Research. Vol. 17. Edited by Michael Tonry. Chicago: University of Chicago Press, 1993.

Wells, L. Edward, and Rankin, Joseph H. "Families and Delinquency: A Meta-Analysis of the Impact of Broken Homes." Social Problems 38 (1991): 71–93.

Wolfgang, Marvin E.; Figlio, Robert M.; and Sellin, Thorsten E. Delinquency in a Birth Cohort. Chicago: University of Chicago Press, 1972.

Wright, Richard T., and Decker, Scott H. Burglars on the Job. Boston: Northeastern University Press, 1994.

——. Armed Robbers in Action. Boston: Northeastern University Press, 1997.