Problem–based Learning in Medical Education: Curriculum Reform and Alignment of Expected Outcomes

Matthew C. E. Gwee

The Central Mission of Medical Education

The central mission of medical education is to improve the quality of health care delivered by doctors and we must never fail to remember the central role played by patients as the ultimate recipients of our skills—what doctors do, and how and when they do it, depends on the quality of medical education. We need to get it right (Bligh & Parsell, 2000).

Whether patients will live or die, and the quality of life they will have upon recovery from an illness or disease, is critically dependent upon what doctors do and how and when they do it. Moreover, the degree of faith, trust, and respect that patients have for their doctors depends much upon what doctors do and how they do it in initiating and maintaining the doctor-patient relationship. In a similar context, what medical teachers teach and how and when they teach it impact strongly on the quality of the education that they provide to students and, consequently, on the quality of medical graduates when they take on their professional roles in practice. Thus, in preparing the medical students of today to become the competent and caring doctors of tomorrow, medical teachers need to get it right.

The Traditional Medical Curriculum: Entrenchment, Entrapment, and Soul Searching

Medical education, with its intensive pattern of basic science lectures followed by an equally exhausting clinical programme, was rapidly becoming an ineffective and inhumane way to prepare students, given the explosion in medical education and the rapidly changing demands of future practice (Boud & Feletti, 1997).

Undergraduate medical education has been deeply entrenched and entrapped in tradition over many decades. Major critics of traditional medical education have often come from within the medical profession. This has prompted medical educators to engage in periodic soul searching for over a century in their attempts to design an effective and relevant curriculum for preparing medical students for their future practice. More importantly, the criticisms also have often provided the impetus for curriculum reforms in medical education (General Medical Council, U.K., 1993).

Identifying Major Shortcomings: Pedagogical Sins of Commission and Omission

Three pedagogical limitations (the unholy trinity) have been identified as the major shortcomings in traditional undergraduate medical education:

1. Highly lecture-based instruction
2. Highly discipline-specific curricula
3. Highly teacher-centered education

A criticism of the highly lecture-based instruction is that “the scarcely tolerable burden of information that is imposed taxes the memory but not the intellect. The emphasis is on the passive acquisition of knowledge, much of it to become outdated or forgotten, rather than on its discovery through curiosity and experiment” (General Medical Council, 1993). In such an educational environment, students become passive recipients of abundant and often inert content knowledge that causes information overload. Student learning and thinking, therefore, are often not curiosity-driven nor grounded in understanding (deep learning). Instead, students resort to rote learning (memorize, recall, regurgitate) as a survival skill to pass examinations that test mainly recall of facts. Knowledge acquisition is therefore the main outcome of this mode of instruction. As Meyers and Jones (1993) have commented, “a steady diet of lecturing leads to intellectual anaemia.”

Miller (1961) has clearly pointed out that “each department is responsible for some part of the education of a medical student, but no department should forget that it is no more than a part of the whole school which is responsible for the education of a whole student and the fulfillment of the overall objectives.” Unfortunately, the highly discipline-specific traditional medical curriculum created the great preclinical-clinical divide leading to the departmentalization of the medical disciplines, which vie for time and dominance without due regard for the goals of the overall curriculum. Abrahamson (1978) characterized this situation as “curriculosclerosis,” one of nine diseases of the medical curriculum, which tends to stifle and inhibit curriculum development and function with a design that reflects “more a power-struggle than an educational venture.”

Students on such a curriculum undertake compartmentalized learning that borders on the education of “mini-specialists” in multiple disciplines!Consequently, medical students often lack the ability to integrate and apply knowledge from different disciplines, especially in the transfer of basic science knowledge to the disease process and to patient care. The serious consequences of pedagogically separating knowledge and understanding from action and practice in medical education have been clearly pointed out by Margetson (1999). The need to ensure greater unity (integration) in diversity of the undergraduate medical curriculum is now well recognized as a priority in curriculum reforms implemented in medical schools worldwide.

In the traditional teacher-centered education, the dominance of the teacher as the know-all and the sage in center stage creates in students a high dependency state, relying on their teachers to decide for them what, when, and how to learn, including a strong predisposition toward an addiction to lectures. Medical students, therefore, often lack the skills, the confidence, and the motivation to undertake self-directed learning, for which they need to take greater initiative for their own learning and to develop skills for appraising the accuracy of information obtained from various sources. Boud (2001) has cautioned that we teachers can potentially “deskill” students by unrealistically mediating all that they need to know.

The Need for Medical Education Reform

…we can at best strive to educate doctors capable of adaptation to change, with minds that can encompass new ideas and developments and with attitudes to learning that inspire the continuation of the educational process throughout professional life (General Medical Council, 1993).

Global events emerging in this digital age of information explosion, along with the rapid advances made in science, technology, and the information and communication field, have had a major impact on the setting of priorities and strategies at all levels of education. More importantly, education today must enable students to meet the growing challenges and the increasingly more complex demands of the work environment and of daily living in this new millennium. Similarly, major reform of the undergraduate medical curriculum is necessary to cope with changes in the medical field, such as the changing patterns of diseases (including the emergence of more chronic ailments of old age) and of health-care delivery; exponential growth in medical knowledge and rapid advances in medical technology; higher expectations of more educated patients, who are better informed about health matters; and greater emphasis on preventive and community health care. Thus, student learning in the knowledge, skills, and attitudinal domains in medical education need to be redefined to closely reflect the changing needs and demands required of good medical practice in the years ahead (Dent, 2001).

It is imperative for educators to reappraise the undergraduate medical curriculum to align the education of students with the desired outcomes in the context of the medical and health-care needs in this new millennium. An educational strategy known as the SPICES model (Harden, Snowden, & Dunn, 1984) offers a useful approach to planning and designing an undergraduate medical curriculum that can overcome many of the major shortcomings identified in traditional medical education. Several of the major educational elements of the SPICES model—which is a s tudent-centered, p roblem-based, i ntegrated, c ommunity-based, e lective, s ystematic approach—are incorporated in the Problem–based learning (PBL) strategy. Moreover, incorporating the PBL strategy in medical education will more effectively address the concerns expressed by the General Medical Council (1993) that “the doctors of tomorrow will be applying knowledge and deploying skills which are at present unforeseen,” a philosophy that must guide educators in designing the undergraduate medical curriculum of today.

Problem–based Learning: An Innovative Pathway to Learning in Medical Education

Problem–based learning (PBL) is grounded in the belief that learning is most effective when students are actively involved and learn in the context in which the knowledge is to be used (Boud & Feletti, 1997).

The contextual theory has often been claimed to be the main pedagogical underpinning of PBL. The basic premise is that when material is learned in the context in which it is to be applied, learning—and therefore the ability to use that information—is enhanced. More recently, Albanese (2000) has criticized this theory as one of the least compelling theories in support of PBL, since almost the entire clinical education is in the relevant context of patient care. Albanese proposed four other theories as the major pedagogical underpinnings of PBL:

• The information processing theory, which relates to prior knowledge activation, encoding specificity (i.e., learning in context), and knowledge elaboration (i.e., creating opportunities for active discussion).
• The cooperative learning theory, which relates to mutual cooperation and support in group learning with the aim of achieving common goals.
• The self-determination theory, which relates to conditions that are more likely to promote intrinsic (autonomous) motivation, as in a PBL environment (in contrast to the extrinsic or controlled motivation that is likely to occur in a traditional curriculum).
• The control theory, which relates to influencing behavior through satisfying an individual's basic needs for survival, belonging and love, power, freedom, and fun. From a pedagogical viewpoint, the PBL small-group learning design seems to correlate well with this theory.

The Role of PBL in Medical Education

PBL is a learning system design that incorporates several complementary pedagogical elements in a strategic alliance that is employed in medical education to enhance and optimize student learning, not only in the acquisition of knowledge but also in the development and acquisition of more enduring and transferable process and professional skills required for medical practice. In its operational context, PBL can be defined as the learning that results from the process of working toward the understanding or resolution of a problem (Spaulding, 1969; Barrows & Tamblyn, 1980;Barrows, 1996).

Essentially, then, PBL represents a major shift in educational paradigm to Problem–based, process-oriented, discipline-integrated, and student-centered learning in collaborative small groups. It creates excellent opportunities for students to engage in regular practice in the problem-solving process so essential to medical practice. PBL in medical education also focuses on learning, not only about a patient's disease but also about the patient's “disease” (discomfort) and about the impact and implications of the disease on the patient's family, the community, and the population. PBL, therefore, offers a more holistic approach to medical education that incorporates learning about the p opulation, the b ehavior, and the l ife sciences in relation to diseases.

Expected Educational Outcomes of PBL in Medical Education

K nowledge acquisition

An important intended outcome of PBL in medical education is to enhance the problem-solving skills of students in preparation for their future role as doctors; and this, according to Norman (1997), will require students to acquire an optimum knowledge base. An immediate aim of PBL, therefore, must be to ensure that learning results in enhanced knowledge acquisition. Earlier studies reported that, when compared to students from conventional medical curricula, PBL students did not perform much better, or even slightly worse in some cases (Albanese & Mitchell, 1993; Berkson, 1993; Vernon & Blake, 1993; Norman & Schmidt, 2000). However, a more recent study provided more convincing evidence that medical students from PBL curricula performed better than students from traditional curricula in the United States Medical Licensing Examination (USMLE) (Blake, Hosokawa, & Riley, 2000).

B eyond knowledge acquisition: E nhancing process, professional, and life skills

Apart from knowledge transfer, medical students should also be inculcated with attitudes of mind and of behavior that befit a doctor, together with qualities appropriate to their future responsibilities to patients, colleagues, and the community in general (General Medical Council, 1993). Learning in small social groups (or learning communities) is the main instructional design used in the delivery of PBL curricula. Such a structure creates opportunities for students to practice and develop transferable skills useful to their future practice and professional development. Central to the goals of PBL in medical education is the nurturing and enhancement of the problem-solving skills of students with the expectation of the transfer of these skills to the clinical management of patients in their future practice. The PBL strategy creates excellent opportunities for students to regularly practice and learn the problem-solving process early in the course.

PBL also fosters the development of self-directed learning skills, which will lay the foundation for, and is fundamental to, nurturing the habit of lifelong continuing self-education so essential to ensuring continued professional competence in medical practice. Medical students consistently report that they enjoy their learning experience in a PBL environment, and Albanese (2000) suggested that such an environment is more likely to motivate students to undertake lifelong learning. The need to inculcate in students the habit of lifelong engagement in learning is clearly stated in the recommendations of the General Medical Council (1993).

Medical students need to develop critical thinking skills, which are vital for the clinical decision-making process. PBL is expected to help students learn how to analyze, evaluate, integrate, and apply information through their active involvement in learning together in small groups. The PBL tutorial, therefore, creates an “all teach, all learn” educational setting for students to engage in peer teaching and learning through constructive dialogue (brainstorming, argumentation, exchanging viewpoints, seeking clarification and justification), which, according to Abercrombie (1960), promotes clinical judgment. “Discussion in a group does for thinking what testing on real objects does for seeing. … Instead of seeing our own mistakes by contrast with the statements of an unquestioned authority as in the traditional pupil-teacher relationship, we see a variety of interpretations of the same stimulus pattern, and the usefulness of each must be tested in its own right” (Abercrombie, 1960). Moreover, Brookfield (1987) pointed out that, when like-minded students learn together through mutual exploration and critical discussion, the process provides “a powerful psychological ballast to critical thinking efforts.”

PBL also fosters the development of reflective learning through feedback sessions, and this helps develop self-awareness and hence encourages self-initiated remedial action (i.e., metacognition). Consequently, students can develop personal insights of their strengths and limitations relating to a given area of learning, an attribute that characterizes critical thinkers (Halpern, 1996; Maudsley & Strivens, 2000).

The PBL small-group learning also provides students with opportunities to develop their communication skills. Doctors have to be able and willing to communicate effectively, an attribute that the General Medical Council (1993) recommends must be developed throughout the undergraduate course and beyond. There is now an even greater need for doctors to communicate effectively with and display a more caring attitude toward patients. Today's patients are more educated and better informed about health matters, as such information is now readily available at the click of a button, and they also have greater expectations about what doctors can and should do for them. Skills in effective communication are critical to initiating and maintaining an optimum doctor-patient relationship, and the General Medical Council (1993) observed that a high proportion of complaints and misunderstandings result from the lack of communication skills among doctors. This lack of communication, resulting in misunderstanding, could lead to unnecessary litigation (Medical Alumni Newsletter, 2001).

Interpersonal and teamwork skills are also essential for doctors working in a health-care team. Small-group learning in PBL promotes the development of such skills, as it is underpinned by cooperative learning principles characterized by “joint goals, mutual rewards, shared resources, and complementary roles among the members of a group” (Albanese, 2000). Qin, Johnson, and Johnson (1995) concluded from a meta-analysis that higher-quality problem solving is achieved through cooperation than through competition (effect size = 0.55): “The average person (at the 50th percentile) in the cooperation condition solved problems better than 72.5% of the participants in the competitive condition.”

A consistent finding in the medical literature is that students and teachers express more joy in learning together through the PBL process, when compared to those learning through more conventional curricula, and that PBL provides a more collegial environment for learning (Albanese, 2000). In his commentary on PBL in medical education, Federman (1999) made the point that “the benefits of contact between student and student, and between student and tutor, need no statistical confirmation.” The PBL strategy, therefore, creates a conducive learning environment that builds p artnerships and b onding in l earning between students and their tutors and among students. Students learn through social interaction in an educational setting, and they also learn to comply with the “code of practice” for working in groups. In the process, they develop mutual trust and respect as well as provide support to one another. Consequently, students can be expected to develop useful interpersonal skills required of health-care practitioners working as a team. According to a Canadian study, PBL graduates from McMaster Medical School showed a greater desire for affiliation in their choice of medical careers and specialties, thus providing evidence that PBL does have a strong impact on the learning environment (Woodward, Ferrier, Cohen, & Goldsmith, 1990).

Return on Investment in PBL in Medical Education

The strategic alliance of complementary pedagogical elements employed in the PBL approach offers great expectations. The PBL way to medical education is expected to lead to more effective acquisition and transfer of knowledge, skills, and attitudes that will equip students with desirable abilities and attributes required of a competent and caring doctor throughout his or her professional life. However, there appears to be a serious mismatch between the high expectations of the outcomes and the results of outcome measures of PBL in medical education.

Two meta-evaluations (Albanese & Mitchell, 1993; Vernon & Blake, 1993) and a study (Berkson, 1993) showed that only small, short-term gains in medical knowledge and clinical skills were achieved through PBL. Colliver (2000) interpreted the findings as clear evidence that PBL does not improve knowledge base and clinical performance to the extent that is to be expected considering the commitment of resources. The main thrust of Colliver's arguments is that the theoretical basis (of the contextual learning theory) underpinning PBL is weak and that the effect sizes produced by studies of PBL approaches in medical education were less than 0.8 and thus did not live up to expectations (of 0.8 to 1.0).

Citing the review of Nendaz and Tekian (1999) for support, Rothman (2000) pointed out that assessment in PBL is still unsatisfactory, without any specific model to follow, and lacking in psychometric scrutiny. He also raised doubts about the value of implementing PBL in the clinic, since there would be difficulties encountered as a consequence primarily of differences in the culture and environment of the preclinical and the clinical contexts, as described by Gresham and Philip (1996). Rothman concluded that it may be time to acknowledge that there are serious limitations of the PBL approach.

The arguments so far seem to suggest that the PBL approach in medical education has little pedagogical advantage to offer in terms of educational outcomes, especially in view of the demands made on the resources of an institution in the implementation of PBL. Thus, the return on investment in PBL would seem rather uncertain in medical education, raising some genuine concerns among educators that need to be addressed urgently.

Insights and Perspectives on the Educational Outcomes of PBL in Medical Education

Several counter-arguments have been advanced that criticized Colliver's (2000) interpretation and conclusion that there is no convincing evidence for the educational benefits of PBL in medical education. Agreeing with Colliver that the contextual theory is weak, Albanese (2000) proposed four other theories (information processing theory, cooperative learning theory, self-determination theory, and control theory—which were described earlier in this chapter) that he considered could provide stronger theoretical underpinnings for PBL and better explain and predict the effectiveness of the various elements of PBL. He also offered some plausible explanations for the apparent mismatch between the expected educational outcomes of PBL and the results of outcome measures.

Albanese (2000) considered it unrealistic to expect knowledge gain by medical students on a PBL curriculum to reach statistical significance with a large effect size of 0.8 or greater as perceived by Colliver since, firstly, “many very commonly used and accepted medical procedures and therapies are based upon studies that had effect sizes even below 0.5.” Moreover, medical students generally are already very high achievers and, consequently, top medical students who receive high grades in examinations cannot be expected to go much higher whatever innovative curriculum they may be subjected to. The operation of such “ceiling effects” will therefore make it difficult to show significant gains in knowledge in such comparative studies.

Albanese also drew attention to “system issues” related to the prior learning experience of students when in school, as this would have a considerable influence on the outcome measures of such comparative studies. He pointed out that, throughout their schooling, medical students have been nurtured and selected by a traditional educational approach in which they have excelled and, therefore, expecting them “to suddenly excel in a different type of milieu seems to be overly optimistic.” Albanese also commented that the selection process of medical students itself guarantees that they will survive well in a traditional curriculum, and to expect such students to perform better in PBL would seem like “transporting a T. rex from the Jurassic period to modern times and expecting it to thrive in a petting zoo”!

Additionally, Albanese pointed out that it would be difficult to interpret with any accuracy the results of the outcome measures used in evaluating the effectiveness of PBL in medical education, since the assessment strategies used to measure performance generally have been designed for the conventional curriculum. Consequently, performance in PBL would mean that it has not only to outperform the former curriculum in meeting the old goals but to meet the goals of the new curriculum too. Such confounding and conflicting aims and procedures cannot therefore generate the effect sizes expected. Moreover, as discussed above, the effect sizes are also likely to be attenuated by system issues.

Similar arguments were put forth by Norman and Schmidt (2000), who observed that “educational trials are ill-founded and ill-advised” since it is not possible to ensure blinding, either among students or among teachers, in such interventions. Thus, success or failure cannot be attributed solely to a given intervention. The authors also highlighted the futility of seeking cause and effect relationships in respect of the PBL curriculum, as PBL represents not just a single curricular intervention but an alliance of several curriculum components (e.g., group size and dynamics, quality of problem case, tutor characteristics, overall curriculum goals). Thus, in estimating the effectiveness of PBL, it is important to take into account the possibility of complex interactions among the various components.

Toward a New Generation of PBL Strategies in Medical Education

PBL has been one of the most successful innovations in medical education and has established credibility. It is time to step back to reflect on “accepted practice” and to develop the next generation of PBL (Kaufman, 2000).

While recognizing some of the shortcomings of its expected educational deliverables, the PBL strategy's inherent pedagogical strength still makes a strong case for reflection, retention, and its renewal in medical education. Albanese (2000) drew attention to the fact that there is compelling evidence that PBL is doing something useful, in view of the fact that it has spread so widely within and beyond medicine. Kaufman (2000) also provided strong support to several of the arguments advanced by Albanese. The most consistent benefit, and the ultimate outcome, of PBL relates to the creation of a conducive learning environment enjoyed by both students and teachers (Norman & Schmidt, 2000; Albanese, 2000). “PBL does provide a more challenging, motivating and enjoyable approach to medical education. That may be a sufficient raison d'être, providing that the cost of implementation is not too great” (Norman & Schmidt, 2000).

A more recent study comparing graduates from the traditional and from the PBL curricula clearly showed gains in scores for the PBL group, based on their performance in the USMLE (Blake et al., 2000). It is noteworthy that the design strategy of this examination has changed: questions now increasingly test knowledge application in the clinical context. Thus, this study clearly demonstrates the need to ensure that the design of the quality control system for appraising the effectiveness of any innovative curriculum change employs performance measures that are appropriate for and relevant and sensitive to (and not distort) the intended outcomes to be measured (Albanese, 2000).

Kaufman (2000) has therefore proposed that we critically reflect on the pedagogical issues in PBL that impact strongly on the achievement of the intended outcomes of PBL in medical education, especially in regard to enhancing the psychometric scrutiny of assessment in PBL through more relevant and more insightful research, as well as “bridging the gap between the PBL approach used in the pre-clinical and clinical settings with actual patient problems seen by learners” with the specific aim of developing the next generation of PBL strategies.

Conclusion

Even if knowledge acquisition and clinical skills are not improved by PBL, the enhanced work environment for students and faculty that has been consistently found with PBL is a worthwhile goal (Albanese, 2000).

PBL has certainly endured the test of time since its inception more than three decades ago. It continues to flourish and to spread widely beyond the discipline of medicine and in many continents, including Asia, which is now experiencing a rapid rise in the acceptance of PBL, especially in medical schools. More recently, the return on educational investment in PBL has been questioned, giving rise to serious concerns among educators about the value and benefits of PBL in medical education. At the same time, there are strong arguments that support the continued use of PBL in medical education. There is good reason to be optimistic: as noted by Albanese (2000), “change may be in the offing,” in his reference to the compelling evidence of PBL students scoring better in the USMLE. The need for more relevant and insightful research in PBL that will further enhance its status as a value-added, quality-enhancing, and more holistic pedagogical strategy in medical education is also well recognized.

It is indeed likely that the best is yet to come from PBL in medical education, as “it will probably take a change in the entire educational process leading up to medical school before PBL (or any similar innovation) will be likely to reach its true potential” (Albanese, 2000). The serious concerns notwithstanding, PBL can certainly claim to provide an innovative pathway to learning in medical education with clear accruing benefits in the humanistic aspects of medicine: “But isn't that, enriched by science, what medicine is?” as Federman (1999) so aptly observed.

References

Abercrombie, M. L. J. (1960). The anatomy of judgement. Harmondsworth: Penguin. Abrahamson, S. (1978). Diseases of the curriculum. Journal of Medical Education, 53, 951–57.

Albanese, M. A. (2000). Problem–based learning: Why curricula are likely to show little effect on knowledge and clinical skills. Medical Education, 34, 729–38.

Albanese, M. A., & Mitchell, S. (1993). Problem–based learning: A review of literature on its outcomes and implementation issues. Academic Medicine, 68, 52–81.

Barrows, H. S. (1996). Problem–based learning in medicine and beyond: A brief overview. In L. Wilkerson & W. H. Gijselaers (Eds.), Bringing Problem–based learning to higher education: Theory and practice (pp. 3–12). New directions for teaching and learning, No. 68. San Francisco: Jossey-Bass.

Barrows, H. S., & Tamblyn, R. N. (1980). Problem–based learning: An approach to medical education. New York: Springer.

Berkson, L. (1993). Problem–based learning: Have the expectations been met? Academic Medicine, 68 (Suppl.), 79–88.

Blake, R. L., Hosokawa, M. C., & Riley, S. L. (2000). Student performances on Step 1 and Step 2 of the United States Medical Licensing Examination following implementation of a Problem–based learning curriculum. Academic Medicine, 75, 66–70.

Bligh, J., & Parsell, G. (2000). Taking stock … Medical Education, 34, 416–17.

Boud, D. (2001). Introduction: Making the move to peer learning. In D. Boud, R. Cohen, & J. Sampson (Eds.), Peer learning in higher education. London: Kogan Page.

Boud, D., & Feletti, G. I. (Eds.) (1997). The challenge of Problem–based learning (2nd ed.). London: Kogan Page.

Brookfield, S. D. (1987). Developing critical thinkers: Challenging adults to explore alternative ways of thinking and acting. San Francisco: Jossey-Bass.

Colliver, J. A. (2000). Effectiveness of Problem–based learning curricula: Research and theory. Academic Medicine, 75, 259–66.

Dent, J. A. (2001). Teaching and learning medicine. In J. A. Dent & R. M. Harden (Eds.), A practical guide for medical teachers. Edinburgh: Churchill Livingstone.

Federman, D. D. (1999). Little-heralded advantages of Problem–based learning. Academic Medicine, 74, 93–94.

General Medical Council, U.K. (1993). Tomorrow's doctors: Recommendations on undergraduate medical education. London.

Gresham, C. L., & Philip, J. R. (1996). Problem–based learning in clinical medicine. Teaching and Learning in Medicine, 8, 111–15.

Halpern, D. F. (1996). Thought and knowledge: An introduction to critical thinking (3rd ed.). Mahwah, NJ: Erlbaum.

Harden, R. M., Snowden, S., & Dunn, W. R. (1984). Some educational strategies in curriculum development: The SPICES model. Medical Education, 18, 284–97.

Kaufman, D. M. (2000). Problem–based learning—Time to step back? Medical Education, 34, 510–11.

Margetson, D. B. (1999). The relation between understanding and practice in Problem–based learning. Medical Education, 33, 359–64.

Maudsley, G., & Strivens, J. (2000). “Science,” “critical thinking” and “competence” for tomorrow's doctors: A review of terms and concepts. Medical Education, 34, 53–60.

Medical Alumni Newsletter, Singapore (2001). Communication, May, 1–2.

Meyers, C., & Jones, T. B. (1993). Promoting active learning strategies for the college classroom. San Francisco: Jossey-Bass.

Miller, G. E. (Ed.) (1961). The objectives of medical education. In Teaching and learning in medical school. Cambridge, MA: Harvard University Press.

Nendaz, M. R., & Tekian, A. (1999). Assessment in Problem–based learning medical schools: A literature review. Teaching and Learning in Medicine, 11, 232–43.

Norman, G. R. (1997). Assessment in Problem–based learning. In D. Boud & G. I. Feletti (Eds.), The challenge of Problem–based learning. London: Kogan Page.

Norman, G. R., & Schmidt, H. G. (2000). Effectiveness of Problem–based learning curricula: Theory, practice and paper darts. Medical Education, 34, 721–28.

Qin, Z., Johnson, D. W., & Johnson, R. T. (1995). Cooperative versus competitive efforts and problem solving. Review of Educational Research, 65, 129–43.

Rothman, A. I. (2000). Problem–based learning—Time to move forward? Medical Education, 34, 509–10.

Spaulding, W. B. (1969). The undergraduate medical curriculum (1969 model): McMaster University. Canadian Medical Association Journal, 100, 659–64.

Vernon, D. T. A., & Blake, R. L. (1993). Does Problem–based learning work? A meta-analysis of evaluative research. Academic Medicine, 68, 550–63.

Woodward, C. A., Ferrier, B. M., Cohen, M., & Goldsmith, C. (1990). A comparison of the practice patterns of general practitioners and family physicians graduating from McMaster and other Ontario medical schools. Teaching and Learning Medicine, 2, 79–88.