Self-Report Method
Self-Report Method
DISADVANTAGES OF SELF-REPORT AND POSSIBLE SOLUTIONS
METHODS OTHER THAN SELF-REPORT
Social scientists use many methods to collect data. The most common method is self-report, in which people respond to questions about themselves regarding a wide variety of issues such as personality traits, moods, thoughts, attitudes, preferences, and behaviors. In fact, much of social science knowledge and theory are based largely on self-report data.
ADVANTAGES OF SELF-REPORT
The main advantage of self-report is that it is a relatively simple way to collect data from many people quickly and at low cost. A second advantage is that self-report data can be collected in various ways to suit the researcher’s needs. Questionnaires can be completed in groups or individually and can be mailed to respondents or made available on the Internet. Self-report data can also be collected in an interview format, either in person or over the telephone. Researchers can thus obtain data from respondents across a large geographic area or to whom they do not have direct access.
Furthermore, researchers can collect data regarding behaviors that cannot be observed directly or are unethical to simulate in the laboratory (i.e., activities typically done in private and behaviors that would cause embarrassment if done in public). The only person with direct access to mental events and some behaviors is the self; therefore, the self is the best person to report on these variables. Also, respondents completing pencil-and-paper questionnaires with the assurance of confidentiality and anonymity might be more willing to accurately report on a variety of behaviors and characteristics.
DISADVANTAGES OF SELF-REPORT AND POSSIBLE SOLUTIONS
There are several disadvantages of self-report that threaten the reliability and validity of measurement. Researchers must ensure that measures are reliable—meaning the outcomes of measurements are repeatable—and valid— meaning the intended variable is measured.
Some threats to validity derive from the way the measure is designed, such as using ambiguous words or words that are not appropriate for the reading level of the respondents. This is especially problematic when different groups (e.g., men and women, adolescents and adults) are likely to interpret words differently or have different reading levels. Researchers must choose words carefully to convey their desired meaning precisely and at an appropriate reading level. Another way to ensure consistent interpretation of items is to provide respondents with a reference group. For example, respondents may be instructed to indicate their level of shyness in relation to people of their same age and gender. It may also be important to provide additional information to ensure that respondents interpret and use response scales correctly, such as providing examples of portion sizes when asking questions about eating habits.
The order of items may also influence how a person responds. For example, people may report different levels of happiness depending on whether they answer this question before or after reporting how many dates they had last month. Researchers should consider how questions might influence answers to subsequent questions. Furthermore, when asking about controversial or potentially embarrassing information, researchers might plan to ask less controversial questions first so that respondents feel comfortable at the outset and thus will be more likely to provide honest responses to more difficult questions.
Other threats to the validity of self-report are due to respondent characteristics. One such well-known threat is socially desirable responding, or the attempt by respondents to make themselves look good. Researchers avoid this problem by using neutral items; by using the Q-sort method of response, in which the respondents are allowed to rate only a certain number of items as highly descriptive of themselves; and by informing respondents that answers are anonymous and/or confidential, thereby encouraging honest responding. Finally, researchers may choose not to use data from respondents who score high on measures of social desirability.
Respondents may have other biases unrelated to item content. They might tend to embrace or avoid extreme responses, or they might exhibit acquiescence, a tendency to agree with statements. These threats can be reduced by requiring respondents to choose one answer from a list. To reduce acquiescence bias, researchers can use measures with items keyed in different directions so that agreeing with some items lowers the total score.
An especially harmful threat to self-report occurs when respondents are intentionally dishonest. Dishonest responding can be decreased by ensuring confidentiality (and anonymity, if possible). Data from people who score high on measures of lying or faking (in both negative and positive directions) can also be excluded from further analyses.
Experimenters may pose threats to the validity of self-report by unintentionally influencing how people respond to questions, which is especially likely with an interview format. Typically, this threat results from experimenters expecting people to respond in line with hypotheses. To lessen this threat, experimenters and interviewers should be unaware of the research hypotheses, and several interviewers should be used to reduce systematic influences of any one interviewer.
The situation and location of interviews may also influence self-report measures. For example, people interviewed on college campuses may agree with the statement “The government should give more money to education” in greater numbers than people interviewed in a park. The situation may serve as a cue to the respondents about the desirable answer, and they may respond accordingly. Even when the respondent is not consciously aware of contextual cues, he or she may still be influenced by them on a subconscious level.
A final threat to validity can occur if all data have been collected with self-report because this is likely to artificially inflate correlations; this problem is known as method variance. In fact, collecting data with any single method has its pitfalls. Researchers who use a single method for collecting data need to be aware of method variance so they can correctly interpret the magnitudes of the correlations found between various measures.
METHODS OTHER THAN SELF-REPORT
Multiple methods can be used to gain a multifaceted understanding of variables of interest and to determine consistency between self-reports and other measurements. Other-report (asking another person about the person of interest) is increasingly being used to examine variables that were previously primarily measured with self-report. Behavioral observation is another method for gathering information about variables commonly assessed with self-report, although this method is more time-consuming and expensive than self-report. To check for accuracy, self-reports can be compared with archival data (e.g., criminal records).
Researchers must keep in mind the purpose of their research to determine which method(s) of data collection to use. If researchers are interested in people’s subjective experience of their own thoughts and behaviors, then self-report is appropriate. However, if researchers are interested in more than people’s subjective experience of themselves, then a multimethod approach should be used to ensure reliable and valid measurement. Social science researchers are often concerned with more than subjective experience and are beginning to embrace multimethod approaches to data collection. This shift in data collection methods is likely to increase the quality of data available to test and revise social science theories.
SEE ALSO Ethics in Experimentation; Informed Consent; Mood; Personality; Privacy; Reliability, Statistical; Self-Disclosure; Self-Presentation; Self-Serving Bias; Survey; Validity, Statistical
BIBLIOGRAPHY
Block, Jack. 1977. Advancing the Psychology of Personality: Paradigmatic Shift or Improving the Quality of Research. In Personality at the Crossroads: Current Issues in Interactional Psychology, eds. David Magnusson and Norman S. Endler, 37–63. Hillsdale, NJ: Erlbaum.
Moskowitz, D. S. 1986. Comparison of Self-Reports, Reports by Knowledgeable Informants, and Behavioral Observation Data. Journal of Personality 54 (1): 294–331.
Paulhus, Delroy L. 1991. Measurement and Control of Response Bias. In Measures of Personality and Social Psychological Attitudes, eds. John P. Robinson, Philip R. Shaver, and Lawrence S. Wrightsman, pp. 17–59. San Diego, CA: Academic Press.
Tera D. Letzring