FACULTY TEACHING, ASSESSMENT OF

A comprehensive model of evaluating teaching advocates the use of multiple sources of information to confirm decisions. Each source–students, self-reports, colleagues and chairs, and evidence of learning–has particular strengths and limitations. The weight of the accumulated results leads to the most valid personnel decisions. Using a mixture of evaluation sources can also lead to greater improvement in teaching because different sources are helpful to different teachers and can help identify weaknesses in different areas of instruction.

Four major sources of information are used to determine the effectiveness of an individual teacher: student evaluations, teacher self-reports, colleague/department chair evaluations, and evidence of student learning. Each of these approaches can be useful in making personnel decisions (salary, promotion, and tenure) or in improving teaching. Each approach has strengths and weaknesses that must be kept in mind and that make it imperative to combine evidence for the best judgments.

Student Evaluations

Evaluations of teaching by students have become important and a frequently used method of assessment, the rationale being that students, and only students, are constant observers of what happens in the classroom. Moreover, only students can answer questions about the effects of instruction on them. Research over the past forty-five years has generally demonstrated their validity, reliability, and utility in improving instruction; this same body of research has shown that systematic course evaluations by students can provide useful information in assessing teachers for salary, promotion, and tenure decisions.

Although typically administered during the last week or two of the semester (but prior to final exams and grades), rating forms are on occasion given at midsemester so the instructors may make immediate adjustments for a particular course. Machine-scored forms are frequently used in order to process the large amounts of college and department data, but the open-ended comments solicited by most rating forms often provide teachers with more specific suggestions for improvement. The questions asked about teaching or the course commonly fall into these categories: organization or planning, teacher–student interaction or rapport, clarity or communication skill, workload and course difficulty, grading and assignments, and student self-reported learning. In addition there are always questions asking students to provide a global or overall rating of the course, the instructor, or the instruction received.

Many colleges have assembled their own student rating forms and some allow students to make their ratings by computer. Commercial forms published by the Educational Testing Service (SIR II) and Kansas State University (IDEA) have for many years provided colleges with score summaries, comparison information, and research reports.

Any use of student evaluations should take into account the vast amount of evidence from research (more than two thousand studies in the Educational Resource Information Center (ERIC) system since 1971). This evidence provides the foundation for guidelines for the proper use of student evaluations. Some of the guidelines that institutions should keep in mind follow.

1. Use several sets of evaluation results. Because an individual course may not accurately reflect a teacher's performance, a set of results based on several classes should be used for personnel decisions. Some research suggests using at least five classes.
2. Have a sufficient number of students evaluate each course. Averaging responses from enough students will minimize the effects of a few divergent opinions. Generally, fifteen students is a sufficient number, assuming they represent at least half of the enrolled students in a class. Research also has shown that student evaluations are consistent over short periods of time.
3. Consider some course characteristic in interpreting ratings. Although any single characteristic does not have a great effect, a combination could effect a teacher's evaluation. Research shows, for example, that small classes receive slightly higher ratings and that subject areas such as natural sciences and mathematics receive somewhat lower ratings. Courses that are college required, but not to satisfy the requirements of a major or minor, are also rated lower.
4. For personnel decisions, emphasize global ratings and estimates of learning. Research has shown that an overall or global rating correlated best with measured student achievement–more highly than ratings dealing with different teaching styles and presentation methods. Likewise, student estimates of their learning in a course are good reflections of instructional effectiveness.
5. Student evaluations can improve instruction, depending on how instructors use the results. Good evaluation forms help teachers diagnose their strengths and weaknesses. Studies indicate that some teachers can use the results directly, while others may need to discuss the results with a colleague or a professional consultant.
6. Give those being evaluated an opportunity to respond to evaluation results and to describe their teaching in writing. Teachers being evaluated for personnel purposes should have a chance to describe what they were trying to accomplish in the course and how their teaching methods fit those objectives. The self-report of teaching, which can be part of a teaching portfolio, gives teachers an opportunity to make their own best case.

Teacher Self-Reports

Teacher self-reports are descriptive information about teaching, which generally becomes part of a teacher's annual report. Self-ratings or self-evaluations can be included in self-reports but they should not be given much emphasis in personnel decisions because they lack validity and objectivity. In studies in which self-ratings were compared to ratings of the teacher made by students or colleagues, there was little agreement between teacher's view of themselves and others view of them. For personnel decisions, self-reports are important because they give the teacher the opportunity to make their own best case. Annual reports, sometimes called "brag sheets," are the most common type of self-report. The reports usually include information about the following activities.

• Teaching: teaching load, advising load, honors received, evaluations by students or others.
• Scholarship and creative endeavors: publications completed or in press, works in progress, grants, awards, presentations at conferences, performances, exhibitions.
• Service: service to the institution (e.g. committee work), service to the government or local community, service to the profession.

The Teaching Portfolio

The teaching portfolio has been heralded as an important new contribution to teaching evaluation because it allows teachers to provide continuous documentation of their performance. Borrowed from such fields as art and architecture, where the practice is for professionals to display samples of their work to perspective clients or employers, the teaching portfolio (or teaching dossier) contains three kinds of information:

• Products of good teaching (e.g. student workbooks or logs, student preand postexamination results).
• Materials developed by the teacher (course materials, syllabi, descriptions of how materials were used in teaching and in innovations attempted, and curriculum development materials).
• Assessments or comments by others (students, colleagues, alumni).

Most writers, including Russell Edgerton and colleagues, believe that a portfolio should include not only those items that present the teachers' views about their teaching but also examples and artifacts that they or others contribute. They also argue that the portfolio should be reflective and explain the teachers' thoughts and hopes as they made instructional decisions. During the 1990s many colleges have used teaching portfolios for both instructional improvement and personnel decisions. Some institutions also encourage graduate students to use teaching portfolios to document their experience and to demonstrate their potential for prospective employers.

Colleague and Department Chair Evaluations

Colleagues and department chairs can provide information that is not available from any other source. Neither students, who lack the background and perspective, nor deans, who lack the time, can contribute the kind of information that colleagues and chairs can. Colleagues and chairs can make judgments about the following areas:

• Organization of the subject matter and course. (Does the content appear to be appropriate and relevant?)
• Effective communication. (Are student assignments well defined?)
• Knowledge of subject matter and teaching.
• Fairness in examinations and grading (Do exams test course objectives?)
• Appropriateness of teaching methodology (Do instructional approaches suggest creativity and flexibility?)
• Appropriate student learning outcomes (Are student-produced documents and examinations consistent with course goals and objectives?)

Colleagues can make their judgments about the effectiveness of the teacher by examining course syllabi, assignments, and other documentary evidence. Colleagues in the same subject field as the person being evaluated can best judge several of the areas. In small institutions or in small departments, however, it is virtually impossible to limit evaluators to those in the same departments. In fact, given possible friendships or rivalries within departments, a more balanced evaluation may be achieved by including colleagues from other departments.

For tenure or promotion decisions, colleagues and chairs usually prepare formal written recommendations for candidates from their departments. Although tenure and promotion committees include faculty representatives, these committee members usually do not have the time to obtain their own information on a candidate's teaching, scholarship, or service performance. Instead, they must rely on other sources. For teaching an ad hoc faculty sub-committee could review each candidate's dossier or teaching portfolio and collect supplementary evidence of teaching effectiveness. Their report and recommendation then is made to the tenure and promotion committee. At institutions in which teaching is of primary importance, this approach provides a way for colleagues to have greater influences in personnel decisions. At least one study by Lewis S. Root in 1987 has shown that a small group of colleagues relying on a variety of evidence on teaching performance, such as that suggested for the ad hoc committees on teaching, do make reliable and valid assessments.

Colleagues can also play an important role in improving teaching through their evaluations and collaboration. One example is a faculty-mentoring program, in which senior faculty members provide intellectual, emotional, and career guidance for young, untenured colleagues. Another example is the so-called buddy system, in which two or three colleagues agree to collaborate on a teaching improvement program. Activities include mutual classroom visitations, student interviews, and discussions among colleagues. Although faculty members vary in their ability to offer useful suggestions to each other, colleagues can still provide a perspective that students or others cannot.

The department chair should meet annually with any faculty member whose teaching is substantially below the department's expectations. Such meetings should be made well before final personnel decisions are to be made so that the candidate has ample opportunity to develop a plan to improve his or her teaching.

Evidence of Student Learning

Assessing student learning as a way of evaluating teaching performance has always been an important faculty responsibility. Teachers ask students questions in class, administer examinations, and evaluate projects and performance in laboratories and field settings. Some faculty members feel that student scores on final examinations in their courses provide a valid measure of student learning and that this measure should be used to assess their effectiveness as a teacher. Yet many factors other than the faculty member's teaching competence can affect examination results. Prior knowledge and their ability, interest, and skills in the subject area can also contribute greatly to how students score on examinations.

Another way to assess student learning is to test students at the beginning and then again at the end of a course and inspect the "gain scores." This procedure also is valuable to improve instruction: if significant numbers of students do not understand an important concept, instructional changes are needed. But because gain scores are easily misinterpreted, manipulated and may not be statistically reliable, they should not be used for examining student learning for tenure, promotion or salary decisions.

The Importance of Evaluation

Tenure decisions have important financial as well as educational implications for a college. Because a faculty member may spend thirty to thirty-five years at an institution, granting tenure amounts to more than a two million-dollar commitment at early twenty-first century salary and fringe benefits rates. When an institution makes this decision, they must consider their own goals and the faculty member's performance related to those goals. At community colleges and at many four-year colleges, teaching is the primary activity; so faculty members are evaluated largely as teachers, although their service and scholarly activities can also receive emphasis. At universities and certain four-year colleges, evidence of research and scholarship is primary, though teaching and to some extent service are also considered. In fact some doctoral-granting universities have made concerted efforts to evaluate and improve teaching performance.

See also: College Teaching; Faculty Roles and Responsibilities.

bibliography

Centra, John A. 1973. "Self-Ratings of College Teachers: A Comparison with Student Ratings." Journal of Education Measurement 10 (4):287–295.

Centra, John A. 1993. Reflective Faculty Evaluation, Enhancing Teaching and Determining Faculty Effectiveness. San Francisco: Jossey-Bass.

Edgerton, Russell; Hutchings, Patricia; and Quinlan, Katherine. 1991. The Teaching Portfolio: Capturing the Scholarship of Teaching. Washington DC: The American Association of Higher Education.

Feldman, Kenneth A. 1989. "Instructional Effectiveness of College Teachers as Judged by Teachers Themselves, Current and Former Students, Colleagues, Administrators and External (Neutral) Observers." Research in Higher Education 30:137–189.

Marsh, Herbert W. 1987. "Student Evaluations of University Teaching: Research Findings, Methodological Issues, and Directions for Future Research." International Journal of Educational Research 11:253–388.

McKeachie, Wilbur J. 1979. "Student Ratings of Teaching: A Reprise." Academe 65:384–397.

Root, Lewis S. 1987. "A Faculty Evaluation: Reliability of Peer Assessments of Research, Teaching, and Service." Research in Higher Education 26:71–84.

John A. Centra