Advances in Health Sciences Education 1: 83-94, 1996. ) 1996 Kluwer Academic Publishers. Printedin the Netherlands.
83
Problem-Based Learning in Medical Education: Developing a Research Agenda CHRISTEL A. WOODWARD Depart. of ClinicalEpidemiology &Biostatistics, Health Sciences Centre 3H4, McMaster University, 1200 Main Street West, Hamilton, OntarioL8N 3Z5, Canada Abstract. While the use of problem-based learning (PBL) methods continues to increase in medical education, three literature reviews of PBL have appeared in the past several years which come to different opinions about their merits. This analysis summarizes the research evidence regarding PBL by examining how well it has met its originators' goals, what we know about how PBL works, and how PBL fares in a goal-free comparison with conventional curricula. A research agenda is suggested to refine our understanding of well-documented effects of PBL, to probe for other possible longer term PBL outcomes, and to examine if and how PBL affects knowledge acquisition and retention. Consistency of evidence from a variety of PBL implementations can help decide whether the effects seen can be attributed to PBL or are the results of other curricular features unique to one setting. Key words: medical education, problem-based learning, evaluation
Three review papers have been written about the outcomes of problem-based learning in the past several years [1-3]. Although the authors review a highly similar set of studies, their conclusions differ. They ranged from a ringing endorsement of PBL [3], the suggestion that PBL (even given whole curriculum innovation) produces graduates who are indistinguishable from their traditional counterparts [2] to concern that PBL may have adverse effects on the learner [1]. Why does this diversity of opinion exist? Is it simply in the eye of the beholder? Are more substantive issues involved which may help define a research agenda for medical educators interested in improving the education process? This analysis attempts to put these three reviews in perspective and outline a research agenda for medical education researchers interested in examining PBL. Defining PBL What do we mean by Problem-Based Learning (PBL)? What are its necessary and sufficient conditions? The invariants in any definition of PBL are that a set of problems is used to engage the learner in the learning task and that discussion among learners occurs to identify the features of the problem, the areas in which they lack information needed to understand the nature of the problem and how it can best be approached and managed. The learners then engage in a variety of independent learning activities which help them explore the constructs, issues, theories, mecha-
84
CHRISTEL A. WOODWARD
nisms involved in obtaining a deeper understanding of the problem and how it can be managed. The results of these learning activities are again brought to the group for further discussion to elaborate the problem and its implications. Faculty have major responsibilities in the learning process, but they differ significantly from their responsibilities in a traditional role. They must develop the problems used to ensure that issues, ideas, etc. discussed are relevant to the overall learning objectives. They must facilitate group process. Finally, faculty (aside from the tutor) may act as expert resources and/or help ensure such resources are available for students to use. How Should PBL Be Evaluated? The field of program evaluation suggests that we review the objectives of the program to decide how to assess outcomes. To establish the objectives of PBL, we must examine the reasons given by the "founding fathers" at McMaster for designing such a curriculum which was a radical departure from their own medical education. Discussion with them indicates that they wished to make education more enjoyable for the learner. They sought to put the emphasis on learning rather than teaching by a) making the student an active partner in the learning process, b) increasing the perceived relevance of what is being learned, c) focusing more on conceptual or deep understanding than rote memorization, and d) having less scheduled time in the curriculum. In their deliberations about how to achieve these objectives, they noted that medicine had become a great deal more fun for them once they had moved beyond the initial education period which had been largely lecture-based and entered a phase of their education when learning centred around the problems presented by patients. Relatively unschooled in education theory, these innovators then decided to create a curriculum that mirrored the types of learning they found most rewarding, which for them had mainly come in their post-graduate education and beyond. Not surprisingly, they hit upon using patient problems as the cornerstone of curriculum planning. In thinking about problem-based learning, they postulated that it might increase the problem-solving skills of the learner, make the curriculum more flexible and open to change, and allow integration of basic and clinical sciences education. Further, it might produce physicians who, because they learned around the problems they encountered, would continue to seek out new information to solve problems that they encountered in practice and become "life-long self-directed learners". (For a more detailed account, see Spaulding and Cochrane [4].) What Is the Evidence That PBL Has Met the Goals Set By Its Originators? Let us examine first the evidence relevant to assessing the overriding goal of PBL: to put the emphasis on learning rather than teaching and thus make learning more enjoyable. All of the studies reviewed by Albanese and Mitchell suggest
PROBLEM-BASED LEARNING IN MEDICAL EDUCATION
85
"that students generally perceive PBL environments in a positive light; certainly these curricula are not perceived as less humane"[3, p. 63]. Vernon and Blake, who formally developed effect size and vote counting procedures to categorize the studies that they reviewed, agree. "No sample was found in which the students' attitudes did not favour PBL to some degree" [3, p. 554]. Some more recent studies not included in this review continue to concur with this assessment [5, 6]. Students in PBL curricula report spending far less of their time engaged in rote learning without conceptual understanding [6, 7]. An examination of the time formally scheduled with classes and labs for traditional curriculum and PBL students, invariably finds that PBL students have more time to engage in self-initiated learning activities [7, 8]. A final bit of evidence that medical education is more enjoyable is in the satisfaction of the learners with their education. Satisfaction, however, can occur even when little learning has been achieved. Thus, greater satisfaction with the learning experience among PBL learners when learners had had an opportunity to assess PBL and traditional curricula in light of their subsequent education and current professional activities provides more compelling evidence than satisfaction during or immediately after the experience. In a recent study of all Ontario-based certificants of the College of Family Physicians of Canada in 1989-1991 who graduated from an Ontario medical school (N = 320), significant differences were noted among physicians in their satisfaction with "the extent to which medical school has prepared you for practice"[9]. PBL graduates (McMaster) were much more likely to report being satisfied or very satisfied (65.7% compared to 39.7%) and fewer (10.4%) were dissatisfied than graduates of the other four schools (25.5%). This question was buried among many other questions as it forms just one of sixteen items found on a Professional Satisfaction Scale for physicians developed at the Rand Corporation [10]. But, meeting the original goals may not be sufficient; the justifications expounded for using PBL methods should also hold up to careful scrutiny. Here, the evidence for PBL becomes much more mixed and how the objectives are operationalized and measured becomes much more varied. In fact, one rationalization, that PBL would increase problem-solving skills, has not been demonstrated [11]. Both in medicine and other fields, educators no longer believe in the existence of a general problem-solving skill that can be taught and learned [11], a notion that was popular among educators in the 1960s. The importance of content-specificity, the variability in problem-solving performance across problems which is related to the extent of prior knowledge of the specific problem area, is now clear [12]. There is no evidence that one curriculum or another can enhance problem-solving skills independent of knowledge acquisition. Measurement conundrums face researchers when they try to operationally define constructs important to PBL such as 'life-long learners'. Studies which have attempted to measure 'life-long learning' will be used to illustrate the problem. Some investigators have reasoned that if PBL is to create life-long learners, we
86
CHRISTEL A.WOODWARD
must first show that students actually exhibit self-directed learning skills during their education (to assume that they are acquiring such skills). They studied the library behaviour of students in PBL and traditional curricula or tracks [13-16]. The range and number of resources consulted by PBL students is larger than that of students in traditional curricula who are more likely to use course notes, syllabi and assigned texts to study. Such studies have been criticized for assuming that students successful in gaining medical school entry have not already acquired self-directed learning skills [2]. Further, most PBL programs offer their students short courses in how to use the library. Could such a course alone be responsible for any difference seen? (See Berkson [2] for an elaboration of these arguments.) However, the weight of the available evidence suggests "a greater degree of independent study in the PBL programs than in traditional programs" [3, p. 554]. Two studies, done after physicians entered practice, have also tried to address 'life-long learning'. One study asked how much formal continued education (CE) activities graduates of a traditional medical school (Ottawa) and a PBL school (McMaster) engaged in by surveying them about this [17]. No difference was observed. The graduates participated equivalently in CE. Here, the question becomes whether the amount of formal CE activities is the best way, or even a good way, to operationally define a life-long learner. Do you think a life-long learner will do more or less CE? It can be argued either way. If PBL graduates do less, it may be because they use more informal CE or because they do CE more efficiently. If they do more, it may be because they are more motivated to keep current in a number of areas. Even if they spend the same amount of time, the crucial questions remain: What prompts CE? How well do their search strategies work? Does CE lead to change in behaviour? A second study tried to operationalize life-long learners as physicians who are current in knowledge of areas of medicine where practice had changed since their medical graduation [18]. It compared primary care physicians who had graduated from a PBL school (McMaster) or a traditional school (Toronto) five to ten years earlier. The McMaster graduates were more familiar with new developments in the management of hypertension. While we are unlikely to quibble with this definition of a PB learner, the area of medicine chosen for study (hypertension) has raised some questions about generalizability of study findings. Cardiovascular research is one of the fields where McMaster University's medical school faculty has distinguished itself. Would similar results be likely using other medical conditions? Do the results need to be consistent across all fields? Further, the reason for the effect may also be more complex. We know that McMaster graduates display a higher interest in providing education to learners as part of their professional activities [19]. Even if they are more up-to-date than a comparison group, is being up-to-date a directresult of enhanced life-long learning skills or is it an indirect effect, mediated by being more involved in teaching? Clearly, operationalizing, defining and measuring a relatively simply construct as 'life-long learner' is a major challenge.
PROBLEM-BASED LEARNING IN MEDICAL EDUCATION
87
Deciding whether to attribute any differences in knowledge, if found, to life-long learning skills or propensities also must be done with caution. The studies that have appeared which attempt to address whether PBL curricula allow better integration of basic and clinical science learnings also provide a mixed review for PBL. One study [20, 21 ] asked PBL (McMaster) and conventional school students (McGill) to solve a clinical problem and then integrate three passages of relevant basic science knowledge in their explanations. While PBL students advanced more causal explanations and were better able to integrate clinical with basic science knowledge at all educational levels, because they made more clinical inferences (when in some cases the other group of students made none), a higher proportion of the hypotheses made by PBL students were incorrect. In a second study of PBL and conventional curriculum effects on knowledge integration [22], pre-clinical students from two Dutch medical schools were asked to explain how a specific metabolic deficiency and a specific disease might be related, along with an expert group of biochemists and internists. The PBL students and biochemists usually first explored the problem's biochemical aspects and later linked these to the clinical aspects. A more memory-based approach was noted among conventional curriculum students and internists, who tended to search their memories for a direct answer to the question. This latter strategy worked less well, producing more errors. The two studies, taken together, suggest PBL students tend to offer more casual explanations about pathophysiological process underlying disease which tie together basic science and clinical knowledge. Why overall errors in problem solution were higher for the PBL group in one instance and lower in the other is unclear. It may relate to the problems chosen or other features of measurement. Another possible explanation is that the difference observed is a product of the way PBL and conventional education are operationalized in the curricula examined. Replication of these studies in different settings with a greater number of problems would help to sort out how consistent the finding is. Little empirical work has been done to examine whether or not the contents within PBL curricula are easier to change than that of conventional curricula. The notion has face validity - change in a PBL curriculum may be as easy as changing the relative emphasis given to identifiable issues in a problem, adding a new wrinkle to a problem or substituting one problem for another. If this is so, there may be cost-saving involved in a PBL curriculum during the maintenance/updating phase. To my knowledge, this has not been carefully explored. Why Does PBL Work? An interesting line of research has been gaining popularity ... examining 'how
come' PBL works - how it relates to theories of learning. Three studies suggest PBL may promote long-term (3 months-2 years later) recall of information studied. The mechanisms by which it might do so are unclear [23-25]. The immediate knowledge of PBL students was somewhat less in all studies. Eisenstadt, Barry
88
CHRISTEL A. WOODWARD
and Glanz invited randomly selected students to participate in a PBL course rather than a lecture-based hematology-transfusion medicine segment of a second year pathophysiology course [23]. Over a three year period fifty-nine; students participated in the PBL course. When compared to their classmates, PBL students and consenters to PBL/nonconsenters were not different in socio-demographic characteristics or prior performance. On the objective examination at end of course, PBL students scored lower but their performance remained near that level two years later, by which time the control groups performance had declined and matched the PBL group. Similar results are reported by Tans [24] and colleagues, who studied physiotherapy students randomly assigned to PBL or lectures for a muscle physiology course. PBL students scored significantly poorer on a multiple-choice exam directly after the course. When asked to recall core knowledge gained in the course in a free-recall situation six months later, PBL students remembered up to five times more concepts than the lecture group. Coulsen [25] reported somewhat poorer initial performance by PBL students, but a more precipitous drop off in information retained among conventionally taught students. A study by Martenson [26] and colleagues had also been used to suggest better retention for PBL. However, the way PBL is put into operation does not meet the usual definition of PBL. Further, in measuring longer term recall, length of time since exposure to the material was confounded with type of instruction. Two possible reasons for lower immediate recall by PBL students are posited. The phenomenon observed may be a measurement artifact; it is difficult to test exactly what students have learned in PBL because they may venture far from the "prescribed" curriculum which is more reliably covered by lectures. On the other hand, PBL students may learn less but retain more. Allowing free recall of information rather than constraining recall to multiple choice or other structured questions both initially and later would help us decide which of these hypotheses, if either, is correct. Norman and Schmidt have described current knowledge about how memory is enhanced and suggest aspects of PBL which may be responsible for improved long-term recall [11]. Noting that the activation of priorknowledge facilitates the subsequentprocess of new information, they suggest that small-group discussion
of a problem, a feature of PBL, may be a powerful method of activating relevant prior knowledge. Yet, the evidence is lacking that small group discussion is a better or more efficient way of activating prior knowledge than methods often used by lecturers. They also note that elaboration of knowledge at the time of learning
enhances subsequentretrieval.Certainly, tutorial discussion and use of knowledge to solve a problem can facilitate knowledge elaboration in a PBL curriculum. Yet, such activities as note-taking, answering the occasional question, preparing for examinations, etc. prevalent in conventional curricula also may facilitate knowledge elaboration. Finally, they note that similar contextfacilitatesrecall.Here, at least on the surface, PBL should increase an advantage. However, how discrete aspects of PBL may promote better long-term recall than memory prompts used by teaching
PROBLEM-BASED LEARNING IN MEDICAL EDUCATION
89
in conventional curricula has not yet been studied. We are far from being able to illuminate how the 'black box' of PBL works. Norman and Schmidt also hypothesize that PBL may facilitate the formation of a larger number of mental images (instances) of prototypical problem situations in medicine, essential to the competent diagnostician. That is, PBL students develop a larger repertoire of patterns which they can use to assist them in the solution of new (similar) problems. Much of problem-solving is actually pattern recognition. It is rare that the expert must resort to reasoning from basic principles, although when called for by the situation, experts do so with facility. Their hypothesis has yet to be tested.
Goal-Free PBL Evaluation Not all studies examining the effects of PBL have focused on seeing whether or not PBL has met its original goals or tried to analyze the way PBL works. In the spirit of Scriven [27], who advocated goal-free evaluation, studies have been done which use either (1) measures that exist and are considered benchmarks by which conventional curricula measure themselves or (2) other measures that might illuminate differences, or lack thereof, between PBL and conventional curricula not previously considered. Perhaps because these are the easiest types of studies to mount, a number of investigators have examined PBL and conventional curricula or tracks, using examinations required for graduation or licensure as the common measure. Berkson's [2] review suggests no difference is seen in academic achievement as measured by such tests. Albanese and Mitchell [1 ] separated basic science knowledge from clinical knowledge in their review. They found no consistency across studies (and perhaps implementations of PBL) in whether or not PBL students differed from conventional students in basic science knowledge, although when a difference was found, it often favoured the conventional curriculum. Vernon and Blake, whose review also separated consideration of basic science knowledge from clinical knowledge, report greater heterogeneity in results on basic science tests exists for PBL curricula than for conventional ones, again suggesting implementation differences rather than PBL itself, is responsible for any difference (positive or negative) in basic science knowledge [3]. Both reviews concur that there is a slight trend for PBL curricula students to outperform conventional students on clinical examinations. Others have sought information about the career choices or practice patterns of their medical school's graduates [eg., 19, 28, 29]. These studies have been done to examine the output of a particular medical school program. Often, the effects of PBL cannot be isolated from effects of major differences in admission criteria, types of clinical placements or predominant philosophy of care. Here, caution must be used so that all observed differences are not attributed to PBL or PBL alone.
90
CHRISTEL A. WOODWARD
Some studies have been erroneously used in arguments about the effects of PBL when it is more likely that the effects found are due to selection and philosophy of care differences. For example, one study [29] showed that McMaster graduates who are primary care physicians, compared to their matched contemporaries from other Ontario medical schools, bill the health care system less, see fewer patients, do fewer minor assessments (very quick visits), and more psychotherapy and are less likely to practice in emergency rooms. They appear to spend more time with their patients. This approach to primary care medicine is consistent with the approach advocated throughout their medical education, which emphasized the importance of understanding illness in the context of their patients' lives, the role of psychosocial issues in health and well-being, and the primacy of the caring function in primary care medicine. If spending more time with people to understand the context in which their symptoms present and to provide a listening ear in the long run leads to better or worse health outcomes, or is more efficient or inefficient health care delivery, can be debated. However, this debate is not about PBL's effects.
Setting the PBL Research Agenda What do we know about PBL versus conventional medical curricula? (See Table I). The evidence is consistent that PBL students like their education more and that their approach to learning is somewhat different. They use the library more, are involved in more self-learning activities and report more deep or conceptual learning as opposed to surface or rote learning. Their knowledge of basic medical sciences may be better or worse than that of the conventional curriculum students. Here both selection and implementation differences likely account for the difference, not PBL itself. Clinical knowledge and skills of PBL students are usually judged slightly better than students from conventional curricula. PBL students identify PBL features as program strengths, rate their education as more relevant and are generally more satisfied with their educational experiences, even when viewed from a practice perspective. Neither curriculum type teaches problem-solving skills more effectively. Problem-solving skills do not seem to be generic, teachable skills aside from content [11]. What kinds of information about PBL are important but currently lacking? (See Table II). Several studies suggest information gained through PBL is better retained. However, more evidence is required to be confident of this finding. There is some question as to whether PBL originally covers less content or whether PBL students, when measured using tests of content coverage devised for lectures, appear to acquire less content knowledge initially. More studies to sort these issues are needed that rely less on highly structured measures of knowledge acquisition/retention and provide the opportunity to measure free recall. There is also a set of questions related to understanding how come PBL works, when it produces differences from conventional curricula. If PBL does indeed lead to better long-term knowledge retention, why? What features are the key ingredients
PROBLEM-BASED LEARNING IN MEDICAL EDUCATION
91
Table I. Summary of our knowledge about problem-based learning (PBL) compared to conventional curricula Does PBL students' approach to learning differ? Yes * Use library more * Do more self-learning * Report more conceptual learning and less memorization Does basic science knowledge differ? No - not necessarily * Can be better or worse than conventional curriculum: Appears to be an implementation effect Does clinical knowledge differ? Yes * PBL students' clinical knowledge and skills are judged slightly better Does satisfaction with education differ? Yes * PBL students and graduates are more satisfied with their education * PBL students identify features of PBL as strengths * Perceive learning as relevant Neither PBL nor Conventional Curricula directly improve generic problem-solving skills. Table II. Questions about problem-based learning (PBL) which research should address Questions that would aid theory development * How does PBL affect knowledge acquisition and retention? * How come PBL works? * What aspect(s) are the key ingredients? * Why do some implementations produce different results from others? Questions about effects In the longer term does PBL: * Make the transition from education to practice easier? * Facilitate greater interpersonal skills, especially communication skills, among the learners? * Produce physicians who approach their patients in a more egalitarian manner?
in this finding? The suggestion that some PBL implementations produce different results than others (e.g., vis a vis basic science knowledge) is fascinating. Are there differences due to how PBL is implemented or to selection? Questions which seek causal explanation regarding how come PBL works and questions related to longer term outcomes of PBL have been most poorly addressed to date. Both measurement issues and other methodological/technical issues such as the choice of comparison groups, the confounding effects of other educational experiences and the socio-political realities in which physicians practice, make it
92
CHRISTEL A. WOODWARD
difficult to measure the longer term outcomes of PBL with certainty. It has been suggested that only effects of considerable size should be apparent in the practice period, given the numerous other influences on practice [30]. Yet, researchers are still interested in discerning whether PBL produces more or better life-long learners. If PBL-educated physicians were found to have a larger repertoire of usable instances which increase their pattern recognition skills, this would be seen as an important finding. If PBL physicians provided better quality of care, as measured by the health outcomes of their patients over a 10-15 year span, PBL would receive a major boost. Yet, PBL will survive whether these questions are answered affirmatively or not. It is popular with students and faculty. Witness the growth of PBL. Three other important potential longer term outcomes of PBL may be of interest if unbiased evidence can be obtained about them. First, is it possible that PBL might make the transition from education to practice easier or less stressful? The evidence is scanty but compelling enough to want to take a look at this issue. As a faculty advisor to medical students who grapple with PBL, I have noted that they initially complain that they never know if/when they know enough about a topic. These students eventually settle on their own comfort level about information related to a problem and usually this happens within the first year of education. About a year ago, while conducting a focus group of family physicians then 5-8 years in practice about their early practice years, several physicians mentioned their initial anxiety about not knowing enough to practice effectively. The two McMaster graduates in the group demurred. For them, this type of anxiety had faded within the first year of medical school. Is this observation generalizable to other PBL students and schools? No information is available. This observation needs to be explored in other groups and settings to see if it is repeatedly linked to PBL. Second, a very underplayed and underrated aspect of PBL is the interpersonal learning that goes on in small PBL discussion groups. Students often interact within ten or more different work groups of peers during their medical education. This is likely to help them explore facets of their personality that they would otherwise simply take for granted. Remember when you last entered a new work situation? Didn't people respond to you differently than folks you encountered previously? The small group experiences in PBL may refine students' communication skills and make them much more aware of how they react to others and how people react to them. Measurement of such awareness and how it impacts on the relationships PBL graduates form with patients is an area which is unexplored. One PBL graduate, when asked what was the most important learning she attributed directly to the PBL experience, said that it helped her learn to get along with people she did not like, an important skill. Research that probes such interpersonal learning is needed. Finally, PBL sets up a situation where the learner/teacher relationship is construed in a much more egalitarian way than is typically observed in conventional curricula. Does this have impact on the kinds of relationships PBL physicians form with their patients? No evidence is available but the question is important.
PROBLEM-BASED LEARNING IN MEDICAL EDUCATION
93
Conclusions To summarize, the educational experience that PBL provides is different from conventional medical education and is often valued more highly by learners and teachers. We would like to understand how a PBL curriculum facilitates learning and if and how such a curriculum produces longer lasting effects. The search for lasting effects will be difficult, given the crudeness of our measuring devices, the passing of time and number of confounding intervening variables (events) that have occurred. Further, we should not discount the ability of bright people to learn what they need to know despite of any curriculum. These factors add to the challenge facing researchers. An important and large research agenda remains if we are to understand what PBL does and does not do, and how come it does what it does. The answers to the questions posed are likely to be complex. They are unlikely to emerge from a single study or site and require greater cooperation among medical education researchers across educational programs. While looking for consistency of evidence from a variety of studies, we can also learn from inconsistencies which are likely to help us sharpen our questions and refine our theories. References 1. Albanese MA, Mitchell S. Problem-based learning: A review of literature on its outcomes and implementation issues. Academic Medicine 1993; 68(1): 52-81. 2. Berkson L. "Problem-based learning" Have the expectations been met? Academic Medicine 1993; 68(10): S79-S88. 3. Vernon DTA, Blake RL. Does problem-based learning work? A meta-analysis of evaluative research. Academic Medicine 1993; 68(7): 950-63. 4. Spaulding WB, Cochran J. Revitalizing medical education: McMaster medical school, the early years 1965-1974. Philadelphia: BC Decker Inc., 1991. Chapter 4, Establishing Goals: 27-34. 5. Vasconez HC, Donnelly MB, Mayo P, Schwartz RW. Student perceptions of the effectiveness of a problem-based surgery curriculum. Academic Medicine 1993; 68(10 Supplement): 528-30. 6. Regan-Smith MG, Obenshain SS, Woodward CA, Richards B, Zeitz HJ, Small PA. Rote Learning in Medical School. Journal of the American Medical Association 1994; 17: 1380-1381. 7. Moore GT, Black S, Mitchell R. A randomized trial evaluating the impact of the New Pathway Curriculum at Harvard Medical School. Report to the Fund for the Improvement of PostSecondary Education. Cambridge, MA: Harvard Medical School, 1990. 8. Kaufman A, editor. Implementing problem-based medical education: Lessons from successful innovations. New York: Springer, 1985. 9. Woodward CA, Cohen M, Ferrier BM, Williams AP. The relative importance of undergraduate and postgraduate education to the practice decisions and attitudes of young family physicians practising in Ontario. ACMC Medical Resources Conference, Vancouver, April 24, 1994. 10. McGlynn E. Physicians job satisfaction: Its measurement and use as an indicator of system performance. Unpublished doctoral dissertations. Santa Monica, CA: Rand Graduate School, 1988. 11. Norman GR, Schmidt HG. The psychological basis of problem-based learning: A review of the evidence. Academic Medicine 1992; 67(9): 557-65. 12. Elstein AS, Shulman LS, Sprafka SA. Medical problem solving: An analysis of clinical reasoning. Cambridge, MA: Harvard University Press, 1978. 13. Blumberg P, Michael JA. The development of self directed learning behaviours in a partially teacher-centred, problem-based learning curriculum. Teaching and Learning in Medicine 1991; 4: 3-8.
94
CHRISTEL A. WOODWARD
14. Anderson S, Camp MG, Phillip JR. Library utilization by medical students in a traditional or problem-based curriculum. In: Bender W, Hioemstra RJ, Scherpbier A, Zwierstra R, editors. Teaching and assessing clinical competence. Groningen, The Netherlands: Boekwerk Publications, 1990: 77-80. 15. Marshall JG, Fitzgerald D, Busby L, Heaton G. Study of library use in problem-based and traditional medical curricula. Bulletin of the Medical Library Association 1993; 81(3): 299-305. 16. Saunders K, Northrup D, Mennin S. The library in a problem-based curriculum. In: Kaufman A, editor. Implementing problem-based medical education: Lessons from successful innovations. New York: Springer Publishing Company, 1985: 71-88. 17. Tolnai S. Continuing medical education and career choice among graduates of problem-based and traditional curricula. Medical Education 1991; 25: 414-20. 18. Shin JH, Haynes RB, Johnston M. The effect of a problem-based self-directed undergraduate education on life-long learning. Clinical Investigative Medicine 1991; 14: A82. 19. Ferrier BM, Woodward CA. Career choices of McMaster University medical graduates and contemporary Canadian medical graduates. Canadian Medical Association Journal 1987; 136: 39-44. 20. Patel VL, Groen GJ, Norman GR. Effects of conventional and problem-based medical curricula on problem-solving. Academic Medicine 1991; 66: 380-9. 21. Patel VL, Groen GJ, Norman GR. Reasoning and instruction in medical curricula. Cognition and Instruction 1993; 10: 335-78. 22. Boshuizen HP, Schmidt HG, Wassamer I. Curriculum style and the integration of biomedical and clinical knowledge. Paper presented at the Second International Symposium on Problem Based Learning, Yokyakarta, Indonesia, 1990. 23. Einusenstadt RS, Barry WE, Glanz K. Problem-based learning: Cognitive retention and cohort traits of randomly selected participants and decliners. In: Research in Medical Education, 1990. Proceedings of the 29th Annual Conference (MB Anderson, compiler). Washington, DC: AAMC, 1990: S1-S12. 24. Tans TW, Schmidt HG, Schade-Hoogeveen BEJ, Gijselaers WH. Sturing van her Onderswijsleerproces Door Model van Problemen: Een Veldexperiment. (Directing the learning process by means of problems: a field experiment.) Tijdschrift voor Onderwijs Research, 1986; 11: 35-46. 25. Coulsen RL. Problem-based student-centred learning of the cardiovascular system using the problem-based learning module (PBLM). Physiologist 1983; 26: 220-4. 26. Martensen D, Eriksson H, Ingelman-Sundberg M. Medical chemistry: evaluation of active and problem-oriented teaching methods. Medical Education 1985; 19: 34-42. 27. Scriven M: Pros and cons about goal-free evaluations. In: Popham WJ, editor. Evaluation in education: Current applications. Berkeley, CA: McCutchan, 1974. 28. Kaufman A, Mennin S, Waterman R, Duban S, et al. The New Mexico experiment: Educational innovation and institutional change. Academic Medicine 1989; 64: 285-94. 29. Woodward CA, Ferrier BM, Cohen M, Goldsmith A. A comparison of the practice patterns of general practitioners and family physicians graduating from McMaster and other Ontario medical schools. Teaching and Learning in Medicine 1990; 2: 79-88. 30. Friedman CP, deBliek R, Greer DS, Mennin SP, et al. Charting the winds of change: Evaluating innovative medical curricula. Academic Medicine 1990; 65: 8-14.
83
Problem-Based Learning in Medical Education: Developing a Research Agenda CHRISTEL A. WOODWARD Depart. of ClinicalEpidemiology &Biostatistics, Health Sciences Centre 3H4, McMaster University, 1200 Main Street West, Hamilton, OntarioL8N 3Z5, Canada Abstract. While the use of problem-based learning (PBL) methods continues to increase in medical education, three literature reviews of PBL have appeared in the past several years which come to different opinions about their merits. This analysis summarizes the research evidence regarding PBL by examining how well it has met its originators' goals, what we know about how PBL works, and how PBL fares in a goal-free comparison with conventional curricula. A research agenda is suggested to refine our understanding of well-documented effects of PBL, to probe for other possible longer term PBL outcomes, and to examine if and how PBL affects knowledge acquisition and retention. Consistency of evidence from a variety of PBL implementations can help decide whether the effects seen can be attributed to PBL or are the results of other curricular features unique to one setting. Key words: medical education, problem-based learning, evaluation
Three review papers have been written about the outcomes of problem-based learning in the past several years [1-3]. Although the authors review a highly similar set of studies, their conclusions differ. They ranged from a ringing endorsement of PBL [3], the suggestion that PBL (even given whole curriculum innovation) produces graduates who are indistinguishable from their traditional counterparts [2] to concern that PBL may have adverse effects on the learner [1]. Why does this diversity of opinion exist? Is it simply in the eye of the beholder? Are more substantive issues involved which may help define a research agenda for medical educators interested in improving the education process? This analysis attempts to put these three reviews in perspective and outline a research agenda for medical education researchers interested in examining PBL. Defining PBL What do we mean by Problem-Based Learning (PBL)? What are its necessary and sufficient conditions? The invariants in any definition of PBL are that a set of problems is used to engage the learner in the learning task and that discussion among learners occurs to identify the features of the problem, the areas in which they lack information needed to understand the nature of the problem and how it can best be approached and managed. The learners then engage in a variety of independent learning activities which help them explore the constructs, issues, theories, mecha-
84
CHRISTEL A. WOODWARD
nisms involved in obtaining a deeper understanding of the problem and how it can be managed. The results of these learning activities are again brought to the group for further discussion to elaborate the problem and its implications. Faculty have major responsibilities in the learning process, but they differ significantly from their responsibilities in a traditional role. They must develop the problems used to ensure that issues, ideas, etc. discussed are relevant to the overall learning objectives. They must facilitate group process. Finally, faculty (aside from the tutor) may act as expert resources and/or help ensure such resources are available for students to use. How Should PBL Be Evaluated? The field of program evaluation suggests that we review the objectives of the program to decide how to assess outcomes. To establish the objectives of PBL, we must examine the reasons given by the "founding fathers" at McMaster for designing such a curriculum which was a radical departure from their own medical education. Discussion with them indicates that they wished to make education more enjoyable for the learner. They sought to put the emphasis on learning rather than teaching by a) making the student an active partner in the learning process, b) increasing the perceived relevance of what is being learned, c) focusing more on conceptual or deep understanding than rote memorization, and d) having less scheduled time in the curriculum. In their deliberations about how to achieve these objectives, they noted that medicine had become a great deal more fun for them once they had moved beyond the initial education period which had been largely lecture-based and entered a phase of their education when learning centred around the problems presented by patients. Relatively unschooled in education theory, these innovators then decided to create a curriculum that mirrored the types of learning they found most rewarding, which for them had mainly come in their post-graduate education and beyond. Not surprisingly, they hit upon using patient problems as the cornerstone of curriculum planning. In thinking about problem-based learning, they postulated that it might increase the problem-solving skills of the learner, make the curriculum more flexible and open to change, and allow integration of basic and clinical sciences education. Further, it might produce physicians who, because they learned around the problems they encountered, would continue to seek out new information to solve problems that they encountered in practice and become "life-long self-directed learners". (For a more detailed account, see Spaulding and Cochrane [4].) What Is the Evidence That PBL Has Met the Goals Set By Its Originators? Let us examine first the evidence relevant to assessing the overriding goal of PBL: to put the emphasis on learning rather than teaching and thus make learning more enjoyable. All of the studies reviewed by Albanese and Mitchell suggest
PROBLEM-BASED LEARNING IN MEDICAL EDUCATION
85
"that students generally perceive PBL environments in a positive light; certainly these curricula are not perceived as less humane"[3, p. 63]. Vernon and Blake, who formally developed effect size and vote counting procedures to categorize the studies that they reviewed, agree. "No sample was found in which the students' attitudes did not favour PBL to some degree" [3, p. 554]. Some more recent studies not included in this review continue to concur with this assessment [5, 6]. Students in PBL curricula report spending far less of their time engaged in rote learning without conceptual understanding [6, 7]. An examination of the time formally scheduled with classes and labs for traditional curriculum and PBL students, invariably finds that PBL students have more time to engage in self-initiated learning activities [7, 8]. A final bit of evidence that medical education is more enjoyable is in the satisfaction of the learners with their education. Satisfaction, however, can occur even when little learning has been achieved. Thus, greater satisfaction with the learning experience among PBL learners when learners had had an opportunity to assess PBL and traditional curricula in light of their subsequent education and current professional activities provides more compelling evidence than satisfaction during or immediately after the experience. In a recent study of all Ontario-based certificants of the College of Family Physicians of Canada in 1989-1991 who graduated from an Ontario medical school (N = 320), significant differences were noted among physicians in their satisfaction with "the extent to which medical school has prepared you for practice"[9]. PBL graduates (McMaster) were much more likely to report being satisfied or very satisfied (65.7% compared to 39.7%) and fewer (10.4%) were dissatisfied than graduates of the other four schools (25.5%). This question was buried among many other questions as it forms just one of sixteen items found on a Professional Satisfaction Scale for physicians developed at the Rand Corporation [10]. But, meeting the original goals may not be sufficient; the justifications expounded for using PBL methods should also hold up to careful scrutiny. Here, the evidence for PBL becomes much more mixed and how the objectives are operationalized and measured becomes much more varied. In fact, one rationalization, that PBL would increase problem-solving skills, has not been demonstrated [11]. Both in medicine and other fields, educators no longer believe in the existence of a general problem-solving skill that can be taught and learned [11], a notion that was popular among educators in the 1960s. The importance of content-specificity, the variability in problem-solving performance across problems which is related to the extent of prior knowledge of the specific problem area, is now clear [12]. There is no evidence that one curriculum or another can enhance problem-solving skills independent of knowledge acquisition. Measurement conundrums face researchers when they try to operationally define constructs important to PBL such as 'life-long learners'. Studies which have attempted to measure 'life-long learning' will be used to illustrate the problem. Some investigators have reasoned that if PBL is to create life-long learners, we
86
CHRISTEL A.WOODWARD
must first show that students actually exhibit self-directed learning skills during their education (to assume that they are acquiring such skills). They studied the library behaviour of students in PBL and traditional curricula or tracks [13-16]. The range and number of resources consulted by PBL students is larger than that of students in traditional curricula who are more likely to use course notes, syllabi and assigned texts to study. Such studies have been criticized for assuming that students successful in gaining medical school entry have not already acquired self-directed learning skills [2]. Further, most PBL programs offer their students short courses in how to use the library. Could such a course alone be responsible for any difference seen? (See Berkson [2] for an elaboration of these arguments.) However, the weight of the available evidence suggests "a greater degree of independent study in the PBL programs than in traditional programs" [3, p. 554]. Two studies, done after physicians entered practice, have also tried to address 'life-long learning'. One study asked how much formal continued education (CE) activities graduates of a traditional medical school (Ottawa) and a PBL school (McMaster) engaged in by surveying them about this [17]. No difference was observed. The graduates participated equivalently in CE. Here, the question becomes whether the amount of formal CE activities is the best way, or even a good way, to operationally define a life-long learner. Do you think a life-long learner will do more or less CE? It can be argued either way. If PBL graduates do less, it may be because they use more informal CE or because they do CE more efficiently. If they do more, it may be because they are more motivated to keep current in a number of areas. Even if they spend the same amount of time, the crucial questions remain: What prompts CE? How well do their search strategies work? Does CE lead to change in behaviour? A second study tried to operationalize life-long learners as physicians who are current in knowledge of areas of medicine where practice had changed since their medical graduation [18]. It compared primary care physicians who had graduated from a PBL school (McMaster) or a traditional school (Toronto) five to ten years earlier. The McMaster graduates were more familiar with new developments in the management of hypertension. While we are unlikely to quibble with this definition of a PB learner, the area of medicine chosen for study (hypertension) has raised some questions about generalizability of study findings. Cardiovascular research is one of the fields where McMaster University's medical school faculty has distinguished itself. Would similar results be likely using other medical conditions? Do the results need to be consistent across all fields? Further, the reason for the effect may also be more complex. We know that McMaster graduates display a higher interest in providing education to learners as part of their professional activities [19]. Even if they are more up-to-date than a comparison group, is being up-to-date a directresult of enhanced life-long learning skills or is it an indirect effect, mediated by being more involved in teaching? Clearly, operationalizing, defining and measuring a relatively simply construct as 'life-long learner' is a major challenge.
PROBLEM-BASED LEARNING IN MEDICAL EDUCATION
87
Deciding whether to attribute any differences in knowledge, if found, to life-long learning skills or propensities also must be done with caution. The studies that have appeared which attempt to address whether PBL curricula allow better integration of basic and clinical science learnings also provide a mixed review for PBL. One study [20, 21 ] asked PBL (McMaster) and conventional school students (McGill) to solve a clinical problem and then integrate three passages of relevant basic science knowledge in their explanations. While PBL students advanced more causal explanations and were better able to integrate clinical with basic science knowledge at all educational levels, because they made more clinical inferences (when in some cases the other group of students made none), a higher proportion of the hypotheses made by PBL students were incorrect. In a second study of PBL and conventional curriculum effects on knowledge integration [22], pre-clinical students from two Dutch medical schools were asked to explain how a specific metabolic deficiency and a specific disease might be related, along with an expert group of biochemists and internists. The PBL students and biochemists usually first explored the problem's biochemical aspects and later linked these to the clinical aspects. A more memory-based approach was noted among conventional curriculum students and internists, who tended to search their memories for a direct answer to the question. This latter strategy worked less well, producing more errors. The two studies, taken together, suggest PBL students tend to offer more casual explanations about pathophysiological process underlying disease which tie together basic science and clinical knowledge. Why overall errors in problem solution were higher for the PBL group in one instance and lower in the other is unclear. It may relate to the problems chosen or other features of measurement. Another possible explanation is that the difference observed is a product of the way PBL and conventional education are operationalized in the curricula examined. Replication of these studies in different settings with a greater number of problems would help to sort out how consistent the finding is. Little empirical work has been done to examine whether or not the contents within PBL curricula are easier to change than that of conventional curricula. The notion has face validity - change in a PBL curriculum may be as easy as changing the relative emphasis given to identifiable issues in a problem, adding a new wrinkle to a problem or substituting one problem for another. If this is so, there may be cost-saving involved in a PBL curriculum during the maintenance/updating phase. To my knowledge, this has not been carefully explored. Why Does PBL Work? An interesting line of research has been gaining popularity ... examining 'how
come' PBL works - how it relates to theories of learning. Three studies suggest PBL may promote long-term (3 months-2 years later) recall of information studied. The mechanisms by which it might do so are unclear [23-25]. The immediate knowledge of PBL students was somewhat less in all studies. Eisenstadt, Barry
88
CHRISTEL A. WOODWARD
and Glanz invited randomly selected students to participate in a PBL course rather than a lecture-based hematology-transfusion medicine segment of a second year pathophysiology course [23]. Over a three year period fifty-nine; students participated in the PBL course. When compared to their classmates, PBL students and consenters to PBL/nonconsenters were not different in socio-demographic characteristics or prior performance. On the objective examination at end of course, PBL students scored lower but their performance remained near that level two years later, by which time the control groups performance had declined and matched the PBL group. Similar results are reported by Tans [24] and colleagues, who studied physiotherapy students randomly assigned to PBL or lectures for a muscle physiology course. PBL students scored significantly poorer on a multiple-choice exam directly after the course. When asked to recall core knowledge gained in the course in a free-recall situation six months later, PBL students remembered up to five times more concepts than the lecture group. Coulsen [25] reported somewhat poorer initial performance by PBL students, but a more precipitous drop off in information retained among conventionally taught students. A study by Martenson [26] and colleagues had also been used to suggest better retention for PBL. However, the way PBL is put into operation does not meet the usual definition of PBL. Further, in measuring longer term recall, length of time since exposure to the material was confounded with type of instruction. Two possible reasons for lower immediate recall by PBL students are posited. The phenomenon observed may be a measurement artifact; it is difficult to test exactly what students have learned in PBL because they may venture far from the "prescribed" curriculum which is more reliably covered by lectures. On the other hand, PBL students may learn less but retain more. Allowing free recall of information rather than constraining recall to multiple choice or other structured questions both initially and later would help us decide which of these hypotheses, if either, is correct. Norman and Schmidt have described current knowledge about how memory is enhanced and suggest aspects of PBL which may be responsible for improved long-term recall [11]. Noting that the activation of priorknowledge facilitates the subsequentprocess of new information, they suggest that small-group discussion
of a problem, a feature of PBL, may be a powerful method of activating relevant prior knowledge. Yet, the evidence is lacking that small group discussion is a better or more efficient way of activating prior knowledge than methods often used by lecturers. They also note that elaboration of knowledge at the time of learning
enhances subsequentretrieval.Certainly, tutorial discussion and use of knowledge to solve a problem can facilitate knowledge elaboration in a PBL curriculum. Yet, such activities as note-taking, answering the occasional question, preparing for examinations, etc. prevalent in conventional curricula also may facilitate knowledge elaboration. Finally, they note that similar contextfacilitatesrecall.Here, at least on the surface, PBL should increase an advantage. However, how discrete aspects of PBL may promote better long-term recall than memory prompts used by teaching
PROBLEM-BASED LEARNING IN MEDICAL EDUCATION
89
in conventional curricula has not yet been studied. We are far from being able to illuminate how the 'black box' of PBL works. Norman and Schmidt also hypothesize that PBL may facilitate the formation of a larger number of mental images (instances) of prototypical problem situations in medicine, essential to the competent diagnostician. That is, PBL students develop a larger repertoire of patterns which they can use to assist them in the solution of new (similar) problems. Much of problem-solving is actually pattern recognition. It is rare that the expert must resort to reasoning from basic principles, although when called for by the situation, experts do so with facility. Their hypothesis has yet to be tested.
Goal-Free PBL Evaluation Not all studies examining the effects of PBL have focused on seeing whether or not PBL has met its original goals or tried to analyze the way PBL works. In the spirit of Scriven [27], who advocated goal-free evaluation, studies have been done which use either (1) measures that exist and are considered benchmarks by which conventional curricula measure themselves or (2) other measures that might illuminate differences, or lack thereof, between PBL and conventional curricula not previously considered. Perhaps because these are the easiest types of studies to mount, a number of investigators have examined PBL and conventional curricula or tracks, using examinations required for graduation or licensure as the common measure. Berkson's [2] review suggests no difference is seen in academic achievement as measured by such tests. Albanese and Mitchell [1 ] separated basic science knowledge from clinical knowledge in their review. They found no consistency across studies (and perhaps implementations of PBL) in whether or not PBL students differed from conventional students in basic science knowledge, although when a difference was found, it often favoured the conventional curriculum. Vernon and Blake, whose review also separated consideration of basic science knowledge from clinical knowledge, report greater heterogeneity in results on basic science tests exists for PBL curricula than for conventional ones, again suggesting implementation differences rather than PBL itself, is responsible for any difference (positive or negative) in basic science knowledge [3]. Both reviews concur that there is a slight trend for PBL curricula students to outperform conventional students on clinical examinations. Others have sought information about the career choices or practice patterns of their medical school's graduates [eg., 19, 28, 29]. These studies have been done to examine the output of a particular medical school program. Often, the effects of PBL cannot be isolated from effects of major differences in admission criteria, types of clinical placements or predominant philosophy of care. Here, caution must be used so that all observed differences are not attributed to PBL or PBL alone.
90
CHRISTEL A. WOODWARD
Some studies have been erroneously used in arguments about the effects of PBL when it is more likely that the effects found are due to selection and philosophy of care differences. For example, one study [29] showed that McMaster graduates who are primary care physicians, compared to their matched contemporaries from other Ontario medical schools, bill the health care system less, see fewer patients, do fewer minor assessments (very quick visits), and more psychotherapy and are less likely to practice in emergency rooms. They appear to spend more time with their patients. This approach to primary care medicine is consistent with the approach advocated throughout their medical education, which emphasized the importance of understanding illness in the context of their patients' lives, the role of psychosocial issues in health and well-being, and the primacy of the caring function in primary care medicine. If spending more time with people to understand the context in which their symptoms present and to provide a listening ear in the long run leads to better or worse health outcomes, or is more efficient or inefficient health care delivery, can be debated. However, this debate is not about PBL's effects.
Setting the PBL Research Agenda What do we know about PBL versus conventional medical curricula? (See Table I). The evidence is consistent that PBL students like their education more and that their approach to learning is somewhat different. They use the library more, are involved in more self-learning activities and report more deep or conceptual learning as opposed to surface or rote learning. Their knowledge of basic medical sciences may be better or worse than that of the conventional curriculum students. Here both selection and implementation differences likely account for the difference, not PBL itself. Clinical knowledge and skills of PBL students are usually judged slightly better than students from conventional curricula. PBL students identify PBL features as program strengths, rate their education as more relevant and are generally more satisfied with their educational experiences, even when viewed from a practice perspective. Neither curriculum type teaches problem-solving skills more effectively. Problem-solving skills do not seem to be generic, teachable skills aside from content [11]. What kinds of information about PBL are important but currently lacking? (See Table II). Several studies suggest information gained through PBL is better retained. However, more evidence is required to be confident of this finding. There is some question as to whether PBL originally covers less content or whether PBL students, when measured using tests of content coverage devised for lectures, appear to acquire less content knowledge initially. More studies to sort these issues are needed that rely less on highly structured measures of knowledge acquisition/retention and provide the opportunity to measure free recall. There is also a set of questions related to understanding how come PBL works, when it produces differences from conventional curricula. If PBL does indeed lead to better long-term knowledge retention, why? What features are the key ingredients
PROBLEM-BASED LEARNING IN MEDICAL EDUCATION
91
Table I. Summary of our knowledge about problem-based learning (PBL) compared to conventional curricula Does PBL students' approach to learning differ? Yes * Use library more * Do more self-learning * Report more conceptual learning and less memorization Does basic science knowledge differ? No - not necessarily * Can be better or worse than conventional curriculum: Appears to be an implementation effect Does clinical knowledge differ? Yes * PBL students' clinical knowledge and skills are judged slightly better Does satisfaction with education differ? Yes * PBL students and graduates are more satisfied with their education * PBL students identify features of PBL as strengths * Perceive learning as relevant Neither PBL nor Conventional Curricula directly improve generic problem-solving skills. Table II. Questions about problem-based learning (PBL) which research should address Questions that would aid theory development * How does PBL affect knowledge acquisition and retention? * How come PBL works? * What aspect(s) are the key ingredients? * Why do some implementations produce different results from others? Questions about effects In the longer term does PBL: * Make the transition from education to practice easier? * Facilitate greater interpersonal skills, especially communication skills, among the learners? * Produce physicians who approach their patients in a more egalitarian manner?
in this finding? The suggestion that some PBL implementations produce different results than others (e.g., vis a vis basic science knowledge) is fascinating. Are there differences due to how PBL is implemented or to selection? Questions which seek causal explanation regarding how come PBL works and questions related to longer term outcomes of PBL have been most poorly addressed to date. Both measurement issues and other methodological/technical issues such as the choice of comparison groups, the confounding effects of other educational experiences and the socio-political realities in which physicians practice, make it
92
CHRISTEL A. WOODWARD
difficult to measure the longer term outcomes of PBL with certainty. It has been suggested that only effects of considerable size should be apparent in the practice period, given the numerous other influences on practice [30]. Yet, researchers are still interested in discerning whether PBL produces more or better life-long learners. If PBL-educated physicians were found to have a larger repertoire of usable instances which increase their pattern recognition skills, this would be seen as an important finding. If PBL physicians provided better quality of care, as measured by the health outcomes of their patients over a 10-15 year span, PBL would receive a major boost. Yet, PBL will survive whether these questions are answered affirmatively or not. It is popular with students and faculty. Witness the growth of PBL. Three other important potential longer term outcomes of PBL may be of interest if unbiased evidence can be obtained about them. First, is it possible that PBL might make the transition from education to practice easier or less stressful? The evidence is scanty but compelling enough to want to take a look at this issue. As a faculty advisor to medical students who grapple with PBL, I have noted that they initially complain that they never know if/when they know enough about a topic. These students eventually settle on their own comfort level about information related to a problem and usually this happens within the first year of education. About a year ago, while conducting a focus group of family physicians then 5-8 years in practice about their early practice years, several physicians mentioned their initial anxiety about not knowing enough to practice effectively. The two McMaster graduates in the group demurred. For them, this type of anxiety had faded within the first year of medical school. Is this observation generalizable to other PBL students and schools? No information is available. This observation needs to be explored in other groups and settings to see if it is repeatedly linked to PBL. Second, a very underplayed and underrated aspect of PBL is the interpersonal learning that goes on in small PBL discussion groups. Students often interact within ten or more different work groups of peers during their medical education. This is likely to help them explore facets of their personality that they would otherwise simply take for granted. Remember when you last entered a new work situation? Didn't people respond to you differently than folks you encountered previously? The small group experiences in PBL may refine students' communication skills and make them much more aware of how they react to others and how people react to them. Measurement of such awareness and how it impacts on the relationships PBL graduates form with patients is an area which is unexplored. One PBL graduate, when asked what was the most important learning she attributed directly to the PBL experience, said that it helped her learn to get along with people she did not like, an important skill. Research that probes such interpersonal learning is needed. Finally, PBL sets up a situation where the learner/teacher relationship is construed in a much more egalitarian way than is typically observed in conventional curricula. Does this have impact on the kinds of relationships PBL physicians form with their patients? No evidence is available but the question is important.
PROBLEM-BASED LEARNING IN MEDICAL EDUCATION
93
Conclusions To summarize, the educational experience that PBL provides is different from conventional medical education and is often valued more highly by learners and teachers. We would like to understand how a PBL curriculum facilitates learning and if and how such a curriculum produces longer lasting effects. The search for lasting effects will be difficult, given the crudeness of our measuring devices, the passing of time and number of confounding intervening variables (events) that have occurred. Further, we should not discount the ability of bright people to learn what they need to know despite of any curriculum. These factors add to the challenge facing researchers. An important and large research agenda remains if we are to understand what PBL does and does not do, and how come it does what it does. The answers to the questions posed are likely to be complex. They are unlikely to emerge from a single study or site and require greater cooperation among medical education researchers across educational programs. While looking for consistency of evidence from a variety of studies, we can also learn from inconsistencies which are likely to help us sharpen our questions and refine our theories. References 1. Albanese MA, Mitchell S. Problem-based learning: A review of literature on its outcomes and implementation issues. Academic Medicine 1993; 68(1): 52-81. 2. Berkson L. "Problem-based learning" Have the expectations been met? Academic Medicine 1993; 68(10): S79-S88. 3. Vernon DTA, Blake RL. Does problem-based learning work? A meta-analysis of evaluative research. Academic Medicine 1993; 68(7): 950-63. 4. Spaulding WB, Cochran J. Revitalizing medical education: McMaster medical school, the early years 1965-1974. Philadelphia: BC Decker Inc., 1991. Chapter 4, Establishing Goals: 27-34. 5. Vasconez HC, Donnelly MB, Mayo P, Schwartz RW. Student perceptions of the effectiveness of a problem-based surgery curriculum. Academic Medicine 1993; 68(10 Supplement): 528-30. 6. Regan-Smith MG, Obenshain SS, Woodward CA, Richards B, Zeitz HJ, Small PA. Rote Learning in Medical School. Journal of the American Medical Association 1994; 17: 1380-1381. 7. Moore GT, Black S, Mitchell R. A randomized trial evaluating the impact of the New Pathway Curriculum at Harvard Medical School. Report to the Fund for the Improvement of PostSecondary Education. Cambridge, MA: Harvard Medical School, 1990. 8. Kaufman A, editor. Implementing problem-based medical education: Lessons from successful innovations. New York: Springer, 1985. 9. Woodward CA, Cohen M, Ferrier BM, Williams AP. The relative importance of undergraduate and postgraduate education to the practice decisions and attitudes of young family physicians practising in Ontario. ACMC Medical Resources Conference, Vancouver, April 24, 1994. 10. McGlynn E. Physicians job satisfaction: Its measurement and use as an indicator of system performance. Unpublished doctoral dissertations. Santa Monica, CA: Rand Graduate School, 1988. 11. Norman GR, Schmidt HG. The psychological basis of problem-based learning: A review of the evidence. Academic Medicine 1992; 67(9): 557-65. 12. Elstein AS, Shulman LS, Sprafka SA. Medical problem solving: An analysis of clinical reasoning. Cambridge, MA: Harvard University Press, 1978. 13. Blumberg P, Michael JA. The development of self directed learning behaviours in a partially teacher-centred, problem-based learning curriculum. Teaching and Learning in Medicine 1991; 4: 3-8.
94
CHRISTEL A. WOODWARD
14. Anderson S, Camp MG, Phillip JR. Library utilization by medical students in a traditional or problem-based curriculum. In: Bender W, Hioemstra RJ, Scherpbier A, Zwierstra R, editors. Teaching and assessing clinical competence. Groningen, The Netherlands: Boekwerk Publications, 1990: 77-80. 15. Marshall JG, Fitzgerald D, Busby L, Heaton G. Study of library use in problem-based and traditional medical curricula. Bulletin of the Medical Library Association 1993; 81(3): 299-305. 16. Saunders K, Northrup D, Mennin S. The library in a problem-based curriculum. In: Kaufman A, editor. Implementing problem-based medical education: Lessons from successful innovations. New York: Springer Publishing Company, 1985: 71-88. 17. Tolnai S. Continuing medical education and career choice among graduates of problem-based and traditional curricula. Medical Education 1991; 25: 414-20. 18. Shin JH, Haynes RB, Johnston M. The effect of a problem-based self-directed undergraduate education on life-long learning. Clinical Investigative Medicine 1991; 14: A82. 19. Ferrier BM, Woodward CA. Career choices of McMaster University medical graduates and contemporary Canadian medical graduates. Canadian Medical Association Journal 1987; 136: 39-44. 20. Patel VL, Groen GJ, Norman GR. Effects of conventional and problem-based medical curricula on problem-solving. Academic Medicine 1991; 66: 380-9. 21. Patel VL, Groen GJ, Norman GR. Reasoning and instruction in medical curricula. Cognition and Instruction 1993; 10: 335-78. 22. Boshuizen HP, Schmidt HG, Wassamer I. Curriculum style and the integration of biomedical and clinical knowledge. Paper presented at the Second International Symposium on Problem Based Learning, Yokyakarta, Indonesia, 1990. 23. Einusenstadt RS, Barry WE, Glanz K. Problem-based learning: Cognitive retention and cohort traits of randomly selected participants and decliners. In: Research in Medical Education, 1990. Proceedings of the 29th Annual Conference (MB Anderson, compiler). Washington, DC: AAMC, 1990: S1-S12. 24. Tans TW, Schmidt HG, Schade-Hoogeveen BEJ, Gijselaers WH. Sturing van her Onderswijsleerproces Door Model van Problemen: Een Veldexperiment. (Directing the learning process by means of problems: a field experiment.) Tijdschrift voor Onderwijs Research, 1986; 11: 35-46. 25. Coulsen RL. Problem-based student-centred learning of the cardiovascular system using the problem-based learning module (PBLM). Physiologist 1983; 26: 220-4. 26. Martensen D, Eriksson H, Ingelman-Sundberg M. Medical chemistry: evaluation of active and problem-oriented teaching methods. Medical Education 1985; 19: 34-42. 27. Scriven M: Pros and cons about goal-free evaluations. In: Popham WJ, editor. Evaluation in education: Current applications. Berkeley, CA: McCutchan, 1974. 28. Kaufman A, Mennin S, Waterman R, Duban S, et al. The New Mexico experiment: Educational innovation and institutional change. Academic Medicine 1989; 64: 285-94. 29. Woodward CA, Ferrier BM, Cohen M, Goldsmith A. A comparison of the practice patterns of general practitioners and family physicians graduating from McMaster and other Ontario medical schools. Teaching and Learning in Medicine 1990; 2: 79-88. 30. Friedman CP, deBliek R, Greer DS, Mennin SP, et al. Charting the winds of change: Evaluating innovative medical curricula. Academic Medicine 1990; 65: 8-14.
