Advances in Health Sciences Education 1: 141-151, 1997. ) 1997 Kluwer Academic Publishers. Printedin the Netherlands.
141
Comparative Study of Medical Education as Perceived by Students at Three Dutch Universities JAMIU O. BUSARI, ALBERT J.J.A. SCHERPBIER and HENNY P.A. BOSHUIZEN Correspondenceand requestsfor reprintsshould be addressed to A.J.J.A Scherpbier, Skills Lab, MaastrichtUniversity, PO Box 616, 6200 MD Maastricht, The Netherlands. E-mail:[email protected]
Abstract. Objective. The aim of the present study was to identify the differences that may exist in professional satisfaction and skills (clinical patient management and psychosocial skills) in students and graduates from two traditional medical schools and their counterparts from a problem-based medical school in the Netherlands. Also their satisfaction for their training was investigated. Method. Questionnaires were designed containing items reflecting the earlier mentioned differences between the students. Following an initial pilot study, the questionnaire used in this study was constructed accordingly. Sample. The questionnaires were sent to 180 near and recent graduates of these three schools. Results. A total of 127 questionnaires were returned (response rate of 70.5%). The students of the PBL curriculum felt better prepared in psychosocial and interpersonal skills. They felt more satisfied with their training, but were less enthusiastic with the profession. The results also suggest that in their own opinion students and graduates from the three schools do not differ in clinical patient management skills acquired in medical school. Conclusion. A limitation of this study lies in the fact that results are based on actual self-assessment of the students. One must be aware that self-assessment does not always provide for objective information. However, since this limitation applied equally to respondents of all three medical schools, the differences between the schools can be considered indicative of the effects resulting from the different curricula. Key words: comparison of curricula, professional satisfaction, preparation for medical practice
Introduction Medical education has been subjected to intense scrutiny and reassessment in recent times. A lot of attention has been paid to innovation of curricula and how to carry out objective evaluations of the effects of such innovation. An important innovation is the introduction of problem-based learning (PBL), being implemented into medical curricula in different forms. In some cases it has been adopted in its entirety as the main medical curriculum, while in other cases it has only been implemented in the form of a course within a traditional curriculum. Also the PBL curriculum has been implemented in some institutions as "parallel tracks" for selected student-groups alongside their counterparts in the conventional curriculum.
142
JAMIU 0. BUSARI ET AL.
The introduction of PBL necessitated evaluation studies that attempted to prove, as well disprove, the advantages of this curriculum approach over the traditional curriculum. Review of studies comparing both forms of medical education show that students who undergo the PBL curriculum tend to acquire better problem solving and clinical skills, while there counterparts in traditonal medical institutions possess better knowledge of basic and clinical facts when compared on academic achievement scores (Schmidt, Dauphinee, and Patel, 1987). The observed differences could be associated with the increased emphasis laid on problem solving and less fact finding in PBL curricula and the emergence of skill laboratories where better social and clinical skills are taught. Patel, Groen and Norman (1991) reported in their study that PBL students exhibited methodological and systematic approaches to organising clinical information, and also integrated clinical findings with their knowledge of basic medical sciences. Moore-West and O'Donnell (1985) and Kaufman, Mennis, Waterman et al. (1989) found that PBL students at the University of New Mexico Scool of Medicine received higher clinical subscores than their counterparts in the traditional course, while Goodman, Brueschky, Bone et al. (1991) found that PBL students received higher ratings than their lecture-based learning colleagues in three out of five categories of an oral examination during the first month of their junior year at Rush medical college of Rush University. Moore, Block, Briggs Style et al. (1994) however showed that there were no differences found between traditional and PBL third-year students at Harvard Medical school on supervisory ratings in internal medicine. Richards, Ober, Cariaga-Lo et al. (1996) reported that PBL students scored higher in clinical clerkship ratings then their counterparts in a traditional lecture-based curriculum at the same institution. There have been published several reviews (Albanese and Mitchell, 1993; Berkson, 1993; Vernon and Blake, 1993) on the outcomes of problem-based learning. A conclusion of the reviews is that it is difficult to compare and thus draw conclusions on the effects of conventional and innovative problem-based medical curricula on clinical problem solving in students. Schmidt (1990) explained that because it takes quite long before the effect of a particular curriculum is reflected in students' performance, extraneous variables that are not controlled for, influence the results. He described these extraneous variables as changing admission procedures within medical institutions, differential attrition, the use of volunteers in the study groups, low response rates, difficulty establishing truly comparable control groups and differential exposure to the evaluation measures by the comparison group(s). In their survey of comparative studies in medical education, Friedman, De Blieks, Greer et al. (1990) suggested that in evaluating the effect(s) of different forms of medical curricula on students' performance, such evaluation studies should be capable of emphasizing the outcome of educational effects. Also, since conventional and PBL curricula probably produce (noticeable) differences in certain domains only, other areas remain unaffected. Friedman et al. argue therefore that any form of research or comparative study to be conducted should concern both
STUDENTS FROM CONVENTIONAL VS PBL CURRICULA
143
domains where differences are and domains where differences are not expected to be found. In this article we present the results of a study aimed at identifying the differences that (may) exist in professional satisfaction, clinical patient management and psychosocial skills in students and graduates from two conventional and one PBL medical institution in the Netherlands. It was assumed that the respondents from the PBL school, due to their problem-based and problem-solving system of education, would be more satisfied with their respective clinical clerkship or professional functions. We expect that since communicative, psychosocial and interpersonal skills are inherent aspects of the curriculum of UM, differences should exist in these areas. Final year medical students and recent graduates of the University of Maastricht, Maastricht (UM), the Catholic University Nijmegen (KUN), and the University of Groningen (RUG) were chosen for this study. For the past 20 years the UM has had an established problem-based learning medical curriculum, while the KUN and RUG have run conventional medical curricula. In the past few years, the medical curricula in the KUN and RUG have been undergoing changes, but the present graduates chosen in this study passed through the traditional curriculum. The twoyear clinical clerkship programmes in these three universities are, however, similar. Since the clinical patient management skills focused on in this comparison were encountered only during clinical clerkships, it was presumed that there would be little or no difference in the perceived level of preparedness from respondents of the three universities in this respect. The admission of students into medical schools in the Netherlands is centralised. The characteristics of the Dutch scholarship system harbours admission of students from identical age groups, who have successfully passed (nation-wide) high school exams. The students are selected through a national "weighted" lottery system based on their various performances in the high school exams. It is therefore possible for comparisons to be made amongst medical students in Dutch universities with a considerable degree of confidence. Studies conducted have further shown that there are no significant differences identifiable in the knowledge of medical students from the three medical schools used in this study (Bender, Cohen, Imbos et al., 1984). However, the homogeneity of the study groups in this study with respect to age, admission requirements, admission procedures, and their knowledge-base after admission, made their comparison suitable for this study. Instrument In conducting comparative studies among students from different medical schools, different research tools can be made use of. In this study questionnaires were considered to be the most feasible tool. They were designed in a way that would reflect the professional satisfaction and level of preparedness of respondents, in clinical patient management and psychosocial skills. The recommendations of Friedman et
144
JAMIU . BUSARI ET AL.
Table I. An example of a question on psychosocial skills How well were/are you prepared by your training in/to:
poor
fair
adequate
well
very well
Do you think you should have better prepared in this skill during your medical training?
1
2
3
4
5
no
yes
1
2
3
4
5
no
yes
Listen to the patients complaints a. when the complaints appear relevant to the current illness b. when the complaints appear irrelevant to the current illness
al. (1990) were taken into consideration, questions were formulated to address the areas where differences were expected to be found (professional statisfaction and preparedness in psychosocial skills), as well as those where differences were not expected to be found (preparedness in clinical patient management skills). After an initial pilot study and modification of some of the questions, the questionnaire was constructed to suit the different groups of respondents, with separate questionnaires being prepared for the students and recent graduates. The content of the items in the questionnaires were nonetheless the same. The questionnaire comprised five items on the personal data of respondents. There were three items on professional satisfaction addressing broad domains. The respondents had to respond using open answers to how satisfied they were with their respective functions as final year medical students and recent graduates. They were again to respond, using open answers, to how satisfied they were with their respective medical training. Finally, they were to rate their overal satifaction for the profession based on their open-ended responses on a scale of - (1) Yes, (2) Averagely well, (3) Uncertain, and (4) Not satisfied with the profession. There were 19 questions on the perceived level of preparedness of the respondents in psychosocial skills and 11 more in clinical patient management skills. Each question from both domains were respectively answered by the respondents on a five-point Likert-type scale of (1) Poor, (2) Fair,(3) Adequate, (4) Well, (5) Very well prepared. Table 1 shows examples of questions used in the questionnaire. The respondents were also requested to answer yes or no to the question on whether they felt they should have been better prepared in the different skill(s) during their medical training. Table 2 provides a list of the various questions that were included in the questionnaire.
STUDENTS FROM CONVENTIONAL VS PBL CURRICULA
145
Table II. The list of questions included in the questionnaire How well were/are you prepared in/to: Psychosocial and interpersonal skills: -
-
-
-
-
Beginning and establishing a continuous conversation with patients Listening to what patients complain of; a. When they appear to be relevant to the current illness b. When they do not appear to be relevant to the current illness Dealing with emotional and personal problems of patients sometimes encountered Obtaining clinical history from; a. Difficult patients b. Patients with difficult illnesses Explain the medical problem(s) of a patient to him/her; a. When asked (by the patient) b. Without being asked (by the patient) Inform patients when you are unable to help; a. With acute illnesses b. With chronic illnesses c. Terminal illnesses (e.g Cancer) Involving patients in the management of their illnesses Explain the rights and responsibilities of the patients to them Accept to fulfil the wish(es) of patients in certain instances e.g choice of medication Breaking bad news to relatives of patients e.g Death of a child or loved one Share responsibility with other members of the health team Receiving unpleasant criticisms from colleagues (and/or superiors) Receiving unpleasant criticisms from patients Give unpleasant criticisms to colleagues
Clinical patient management skills: How well were/are you prepared in/to; - Perform standard* physical examinations before making a diagnosis - Perform tailored* physical examinations before making a diagnosis Handle clinical problems with unclear indicators of a likely diagnosis(es) - Perform basic biochemical investigations in the ward side-laboratory or bedside (e.g Urine tests, Blood smears) - Handle emergency situations e.g Fractures, Coma, Bums. - Making use of results of laboratory investigations in diagnosis and treatment - Identifying associated clinical problems unmentioned by the patient Invite other clinical specialities in management of your patient Prescribing appropriate treatment (drugs/medication) in patient care Handle the management of a patient with a contagious and/or deadly disease (e.g AIDS) Admitting when a case is beyond your capability
146
JAMIU 0. BUSARI ET AL.
Table II. Continued Satisfaction for the profession: - Having learnt so much in medical school, seen and participated in various clinical training activities, would you say you are/were satisfied with your level of preparedness for medical practice? Yes ( ) Averagely well ( ) Uncertain ( ) No( ) Give reasons (not more than 3 sentences) for your choice made above. -
-
Having learnt so much in medical school, seen and participated in various clinical training activities, would you say you are/were satisfied with the profession? Yes ( ) Averagely well ( ) Uncertain ( ) No ( ) Give reasons (not more than 3 sentences) for your choice made above.
Standard here refers to conducting a detailed clinical or physical examination irrespective of complaints of the patient or the likely diagnosis a doctor may have in mind in the case of the former, and tailored to clinical examination conducted based solely on the complaints or suspected problem a patient may have.
Method In May 1994, questionnaires were distributed amongst 180 final year students and posted to recent graduates (doctors in practise for not more than 12 months) from the medical schools of the University of Groningen (RUG) N = 66, the Catholic University Nijmegen (KUN) N = 54, and the University of Maastricht (UM) N = 60, Maastricht respectively. The responses between the final year medical students and the recent graduates per university showed negligible differences and therefore the various data were combined respectively per university in the analysis of results. Result A total of 127 questionnaires (UM=36 (60%), KUN=25 (46%), RUG=66 (100%)) were returned by the respondents, a total response rate of 70.5%. The questionnaire's reliability on psychosocial and clinical patient management skills was carried out using Cronbach's alpha test of reliability. 92 (72%) of the 127 questionnaires returned were completely filled in without any missing values. Coefficient alpha computed to test the reliability of the various item clusters produced alpha's of 0.95 on psychosocial and interpersonal skills, 0.83 on clinical patient management skills. Cronbach's alpha was an unsuitable method for testing the items on professional satisfaction due to their construct. The reliability coefficient for the questionnaire produced an alpha of 0.85. It could therefore be concluded that the questionnaire was a reliable instrument for this study.
147
STUDENTS FROM CONVENTIONAL VS PBL CURRICULA Table III. Responses from graduates and students on the satisfaction with their respective activities/functions (four-point scale; yes = 1, no = 4)
UM KUN RUG
Satisfaction with function/activity Mean SD
Satisfaction with medical training Mean SD
1.88 1.72 1.50
1.52 1.88 1.81
0.66 0.67 0.63
0.69 0.66 0.72
PROFESSIONAL SATISFACTION
In general, results revealed that the graduates and students from the three institutions felt fairly satisfied with their various activities/functions as students and doctors (Table 3). However the analysis of variance of the results showed significant differences between the institutions (F(2,124) = 4.27, p = 0.02). The respondents from the UM felt less satisfied with their activities than their colleagues from the RUG and the KUN. The respondents felt fairly satisfied (average mean = 1.74) with the medical training they all received from their respective institutions. Between institutions, the UM respondents appear to have felt slightly better prepared than their colleagues from the KUN and RUG. ANOVA however, only showed marginally significant differences (F(2,124) = 2.53, p = 0.08). An inventory on the answers to the open-ended questions in the questionnaire, revealed the Maastricht respondents expressing more dissatisfaction with irregular working shifts, lack of enthusiasm from superiors, inadequate supervision, heavy workload, too much bureaucracy and budgetary cuts. Responses from Groningen and Nijmegen centred mainly around inadequate training in psychosocial and interpersonal skills, theory-oriented rote-learning preclinical education, and inadequate training in communication skills.
PSYCHOSOCIAL SKILLS
The graduates' and students' views on their level of preparedness in psychosocial skills are provided in Table 4. The respondents from the RUG and the KUN felt they were inadequately prepared in psychosocial skills, while their counterparts from the UM felt that they had been well prepared by their training. Analysis of variance (ANOVA) of the results showed a significant difference (F(2,124) = 31.29, p = 0.00). Post hoc comparisons of means (Scheff6) also revealed significant differences between the students and recent graduates of the UM and their counterparts from the RUG and KUN. Maastricht students and graduates felt better prepared than their counterparts from Groningen and Nijmegen in psychosocial skills.
148
JAMIU 0. BUSARI ET AL. Table IV. Means and standard deviations of the 19 questions on psychosocial skills, split into "How well were/are you prepared by your medical training in/to?" and "Do you think you should have been better prepared in this skill during your training?" Level of preparedness (1 = poor, 5 = very well) Mean SD UM KUN RUG
3.34 . 2.24 2.46
0.47 0.68 0.65
Should you have been better prepared (0 = no, 1 = yes) Mean SD 0.29 0.51 0.49
0.16 0.29 0.24
Table V. Means and standard deviations of the 11 questions on clinical patient management skills, split into "How well were/are you prepared by your medical training in/to?" and "Do you think that you should have been better prepared in this skill during your training?"
UM KUN RUG
Level of preparedness (1 = poor, 5 = very well) Mean SD
Should you have been better prepared (0 = no, 1 = yes) Mean SD
3.09 2.89 2.94
0.40 0.42 0.37
0.58 0.50 0.67
0.67 0.17 0.22
Their views on the question if their training in psychosocial skills should have been better, revealed significant differences (F(2,124) = 9.58, p = 0.00). Post hoc comparisons also showed significant differences between the UM and the RUG and KUN. The respondents from the RUG and KUN felt more strongly that they should have better been prepared in their psychosocial skills than their colleagues from the UM. CLINICAL PATIENT MANAGEMENT SKILLS
The results with respect to how well prepared the respondents from the three institutions felt in clinical patient management skills revealed no remarkable difference (Table 5). Their respective levels of preparedness in these skills were perceived as more or less adequate (F(2,124) = 0.92, p = 0.40). The results also showed no remarkable difference in the adequacy of the training provided, by the respective curricula (F(2,124) = 0.82,p = 0.44). Nevertheless, 60% of the respondents indicate that clinical patient management training in the respective curricula could still be improved.
STUDENTS FROM CONVENTIONAL VS PBL CURRICULA
149
Discussion The results obtained in this study agreed with our hypothesis that differences are expected to be found in psychosocial skills between students and graduates from PBL and traditional medical schools. Respondents from the UM were quite satisfied with their psychosocial skills training, while the others were not. This is not surprising since the training of psychosocial and communicative skills has been incorporated into the problem-based curriculum of the UM. The results on professional satisfaction did not support the hypothesis that the PBL students and graduates would be more satisfied with the medical profession than their counterparts in traditional medical schools. UM respondents, contrary to our expectations appeared less satisfied with the profession than their counterparts from RUG and KUN. On the other hand, the KUN and RUG respondents felt less satisfied with their medical training, though ANOVA showed a marginally significant difference. Answers to the open-ended questions in the questionnaire showed that the KUN and RUG respondents used this medium to express their dissatisfaction with their respective medical training. The respondents from the UM, on the other hand, used the medium to express criticisms on their activities as students and doctors and the respective learning and working conditions associated with these activities. There is unfortunately no single and clear-cut answer to explain these findings. The results also showed, as expected, that there were no remarkable differences in clinical patient management skills, due to different curricula. Students and graduates from the three institutions felt equally (dis)satisfied with their clinical patient management skills training. This may however surprise the reader who according to Barrows (1984) might have expected that since PBL furthers clinical problem-solving skills the UM respondents should possess better clinical patient management skills, and therefore start to wonder if this present finding proves otherwise. The explanation for this finding can be found in the skills referred to in the questionnaire, which were actually practical aspects of diagnosis and patient management in a clinical setting (see Table 2), and not the cognitive problem-solving skills Barrows (1984) referred to. PBL students may very well have better cognitive problem solving skills, but do not necessarily excel their colleagues in practical clinical problem solving skills, referred to as clinical patient management skills in this study. Furthermore, many of the issues investigated in this part of the questionnaire were learnt during the clinical rotations which are very similar in the PBL and traditional medical schools. It is our impression that the present result, although in line with our expectations, can not be taken for granted since we are aware that self-assessments do not always provide objective evaluation of the mastery of skills (Gordon, 1991).
150
JAMIU O. BUSARI ET AL.
However, since these limitations in our study apply equally to the different groups of respondents, the results can be considered to be indicative of the effects of the different curricula at the three universities. The perceived preparedness of the respondents in this study corresponds with the respective trainings they had all received in their different curricula. Though these differences may not appear large enough to compel the adoption of PBL into medical curricula, our study strengthens the recommendations of Friedman et al. (1990), and there is enough support for further studies in which the skills of students from different schools will be tested. Acknowledgements The research reported in this article was done in partial fulfilment of the requirements for the Masters Health Professions Education course by the first author. The authors wish to thank Prof. J. Metz Ph.D., Catholic University Nijmegen, and R. Hiemstra, MD, University of Groningen, for their assistance. Jamiu Busari is a Resident doctor, department of Paediatrics, Academic Hospital Maastricht, and holds a masters degree in health professions education (MHPE), from the Maastricht University. Albert Scherpbier is assistant professor. He is the managing director of the Skills Lab of the medical school of the Maastricht University. Henny Boshuizen is associate professor in the Department of Educational Research and Development. She is the Program Director of the international course Masters of Health Professions Education (MHPE), Maastricht University Maastricht, The Netherlands. References Albanese, M.A. & Mitchell, S. (1993). Problem Based Learning, a Review of Literature on Its Outcomes and Implementation Issues. Academic Medicine 68: 52-81. Barrows, H.S. (1984). A Specific Problem - Based, Self-Directed Learning Method Designed to Teach Medical Problem-Solving Skills, and Enhance Knowledge Retention and Recall. In Schmidt, H.G. & de Volder, M.L. (eds.) Tutorials in Problem-Based Learning, 16-32. Van Gorcum: Assen/Maastricht. Bender, W., Cohen-Schotanus, J., Imbos, T.J., Versfelt, W.A. & Verwijnen, G.M. (1984). Medische kennis bij studenten uit verschillende faculteiten: van hetzelfde laken een pak? (Medical Knowledge of Students from Different Universities; of a Similar Kind?) Nederlandse Tijdschrijft van Geneeskunde 128: 917-921. Berkson, L. (1993). Problem Based Learning. "Have the Expectations Been Met?", Academic Medicine 68: S79-S88. Goodman, L., Brueschky, E., Bone, R., Rose, W., Williams, J. & Paul H. (1991). An Experiment in Medical Education. Journalof the American Medical Association 265: 2373-2376. Gordon, M.J. (1991). A Review of the Validity and Accuracy of Self-Assessments in Health Professions Training. Academic Medicine 66: 762-769. Friedman, C.P., De Bliek, R., Greer, D.S., Mennin, S.P., Norman, G.R., Sheps, C.G., Swanson, B.D. & Woodward, C.A. (1990). Charting the Winds of Change: Evaluating Innovative Medical Curricula. Academic Medicine 65: 8-14.
STUDENTS FROM CONVENTIONAL VS PBL CURRICULA
151
Kaufman, A., Mennin, S., Waterman, R., Duban, S., Hansbarger, C., Silverblatt, H., Obenshain, S., Kantrowitz, M., Becker, T., Samet, J. & Wiese W. (1989). The New Mexico Experiment: Educational Innovation and Institutional Change. Academic Medicine 64: 285-294. Moore, G.T., Block, S.D., Briggs Style, C. & Mitchell, R. (1994). The Influence of the New Pathway Curriculum on Harvard Medical Students. Academic Medicine 69: 983-989. Moore-West, M. & O'Donnell, M.J. (1985). Program Evaluation. In Kaufman, A. (ed.) Implementing Problem-BasedMedical Education, 180-206. Springer Publishing: New York. Patel, V.L., Groen, G.J. & Norman, G.R. (1991). Effects of Conventional and Problem-Based Medical Curricula on Problem solving. Academic Medicine 66: 380-389. Richards, B.F., Ober, P.K., Cariaga-Lo, L., Camp, G., Philp, J., Mcfarlane, M., Rupp, R. & Zaccaro D.J. (1996). Ratings of Students' Performances in a Third-Year Internal Medicine Clerkship: A Comparison Between Problem-Based and Leture-Based Curricula. Academic Medicine 71: 187-189. Schmidt, H.G., Dauphinee, W.D. & Patel, V.L. (1987). Comparing the Effects of Problem-Based and Conventional Curricula in an International Sample. Journalof Medical Education 62: 305-315. Schmidt, H.G. (1990). Innovative and Conventional Curricula Compared: What Can Be Said About Their Effects? In Nooman, Z., Schmidt, H.G. &Ezzat, E. (eds.) Innovation in Medical Education; An Evaluation of Its Present Status, 1-7. Springer Publishing: New York. Vernon, D.T.A., Blake, R.L. (1993). Does Problem-Based Learning Work? A Meta-Analysis of Evaluative Research. Academic Medicine 68: 550-563.
141
Comparative Study of Medical Education as Perceived by Students at Three Dutch Universities JAMIU O. BUSARI, ALBERT J.J.A. SCHERPBIER and HENNY P.A. BOSHUIZEN Correspondenceand requestsfor reprintsshould be addressed to A.J.J.A Scherpbier, Skills Lab, MaastrichtUniversity, PO Box 616, 6200 MD Maastricht, The Netherlands. E-mail:[email protected]
Abstract. Objective. The aim of the present study was to identify the differences that may exist in professional satisfaction and skills (clinical patient management and psychosocial skills) in students and graduates from two traditional medical schools and their counterparts from a problem-based medical school in the Netherlands. Also their satisfaction for their training was investigated. Method. Questionnaires were designed containing items reflecting the earlier mentioned differences between the students. Following an initial pilot study, the questionnaire used in this study was constructed accordingly. Sample. The questionnaires were sent to 180 near and recent graduates of these three schools. Results. A total of 127 questionnaires were returned (response rate of 70.5%). The students of the PBL curriculum felt better prepared in psychosocial and interpersonal skills. They felt more satisfied with their training, but were less enthusiastic with the profession. The results also suggest that in their own opinion students and graduates from the three schools do not differ in clinical patient management skills acquired in medical school. Conclusion. A limitation of this study lies in the fact that results are based on actual self-assessment of the students. One must be aware that self-assessment does not always provide for objective information. However, since this limitation applied equally to respondents of all three medical schools, the differences between the schools can be considered indicative of the effects resulting from the different curricula. Key words: comparison of curricula, professional satisfaction, preparation for medical practice
Introduction Medical education has been subjected to intense scrutiny and reassessment in recent times. A lot of attention has been paid to innovation of curricula and how to carry out objective evaluations of the effects of such innovation. An important innovation is the introduction of problem-based learning (PBL), being implemented into medical curricula in different forms. In some cases it has been adopted in its entirety as the main medical curriculum, while in other cases it has only been implemented in the form of a course within a traditional curriculum. Also the PBL curriculum has been implemented in some institutions as "parallel tracks" for selected student-groups alongside their counterparts in the conventional curriculum.
142
JAMIU 0. BUSARI ET AL.
The introduction of PBL necessitated evaluation studies that attempted to prove, as well disprove, the advantages of this curriculum approach over the traditional curriculum. Review of studies comparing both forms of medical education show that students who undergo the PBL curriculum tend to acquire better problem solving and clinical skills, while there counterparts in traditonal medical institutions possess better knowledge of basic and clinical facts when compared on academic achievement scores (Schmidt, Dauphinee, and Patel, 1987). The observed differences could be associated with the increased emphasis laid on problem solving and less fact finding in PBL curricula and the emergence of skill laboratories where better social and clinical skills are taught. Patel, Groen and Norman (1991) reported in their study that PBL students exhibited methodological and systematic approaches to organising clinical information, and also integrated clinical findings with their knowledge of basic medical sciences. Moore-West and O'Donnell (1985) and Kaufman, Mennis, Waterman et al. (1989) found that PBL students at the University of New Mexico Scool of Medicine received higher clinical subscores than their counterparts in the traditional course, while Goodman, Brueschky, Bone et al. (1991) found that PBL students received higher ratings than their lecture-based learning colleagues in three out of five categories of an oral examination during the first month of their junior year at Rush medical college of Rush University. Moore, Block, Briggs Style et al. (1994) however showed that there were no differences found between traditional and PBL third-year students at Harvard Medical school on supervisory ratings in internal medicine. Richards, Ober, Cariaga-Lo et al. (1996) reported that PBL students scored higher in clinical clerkship ratings then their counterparts in a traditional lecture-based curriculum at the same institution. There have been published several reviews (Albanese and Mitchell, 1993; Berkson, 1993; Vernon and Blake, 1993) on the outcomes of problem-based learning. A conclusion of the reviews is that it is difficult to compare and thus draw conclusions on the effects of conventional and innovative problem-based medical curricula on clinical problem solving in students. Schmidt (1990) explained that because it takes quite long before the effect of a particular curriculum is reflected in students' performance, extraneous variables that are not controlled for, influence the results. He described these extraneous variables as changing admission procedures within medical institutions, differential attrition, the use of volunteers in the study groups, low response rates, difficulty establishing truly comparable control groups and differential exposure to the evaluation measures by the comparison group(s). In their survey of comparative studies in medical education, Friedman, De Blieks, Greer et al. (1990) suggested that in evaluating the effect(s) of different forms of medical curricula on students' performance, such evaluation studies should be capable of emphasizing the outcome of educational effects. Also, since conventional and PBL curricula probably produce (noticeable) differences in certain domains only, other areas remain unaffected. Friedman et al. argue therefore that any form of research or comparative study to be conducted should concern both
STUDENTS FROM CONVENTIONAL VS PBL CURRICULA
143
domains where differences are and domains where differences are not expected to be found. In this article we present the results of a study aimed at identifying the differences that (may) exist in professional satisfaction, clinical patient management and psychosocial skills in students and graduates from two conventional and one PBL medical institution in the Netherlands. It was assumed that the respondents from the PBL school, due to their problem-based and problem-solving system of education, would be more satisfied with their respective clinical clerkship or professional functions. We expect that since communicative, psychosocial and interpersonal skills are inherent aspects of the curriculum of UM, differences should exist in these areas. Final year medical students and recent graduates of the University of Maastricht, Maastricht (UM), the Catholic University Nijmegen (KUN), and the University of Groningen (RUG) were chosen for this study. For the past 20 years the UM has had an established problem-based learning medical curriculum, while the KUN and RUG have run conventional medical curricula. In the past few years, the medical curricula in the KUN and RUG have been undergoing changes, but the present graduates chosen in this study passed through the traditional curriculum. The twoyear clinical clerkship programmes in these three universities are, however, similar. Since the clinical patient management skills focused on in this comparison were encountered only during clinical clerkships, it was presumed that there would be little or no difference in the perceived level of preparedness from respondents of the three universities in this respect. The admission of students into medical schools in the Netherlands is centralised. The characteristics of the Dutch scholarship system harbours admission of students from identical age groups, who have successfully passed (nation-wide) high school exams. The students are selected through a national "weighted" lottery system based on their various performances in the high school exams. It is therefore possible for comparisons to be made amongst medical students in Dutch universities with a considerable degree of confidence. Studies conducted have further shown that there are no significant differences identifiable in the knowledge of medical students from the three medical schools used in this study (Bender, Cohen, Imbos et al., 1984). However, the homogeneity of the study groups in this study with respect to age, admission requirements, admission procedures, and their knowledge-base after admission, made their comparison suitable for this study. Instrument In conducting comparative studies among students from different medical schools, different research tools can be made use of. In this study questionnaires were considered to be the most feasible tool. They were designed in a way that would reflect the professional satisfaction and level of preparedness of respondents, in clinical patient management and psychosocial skills. The recommendations of Friedman et
144
JAMIU . BUSARI ET AL.
Table I. An example of a question on psychosocial skills How well were/are you prepared by your training in/to:
poor
fair
adequate
well
very well
Do you think you should have better prepared in this skill during your medical training?
1
2
3
4
5
no
yes
1
2
3
4
5
no
yes
Listen to the patients complaints a. when the complaints appear relevant to the current illness b. when the complaints appear irrelevant to the current illness
al. (1990) were taken into consideration, questions were formulated to address the areas where differences were expected to be found (professional statisfaction and preparedness in psychosocial skills), as well as those where differences were not expected to be found (preparedness in clinical patient management skills). After an initial pilot study and modification of some of the questions, the questionnaire was constructed to suit the different groups of respondents, with separate questionnaires being prepared for the students and recent graduates. The content of the items in the questionnaires were nonetheless the same. The questionnaire comprised five items on the personal data of respondents. There were three items on professional satisfaction addressing broad domains. The respondents had to respond using open answers to how satisfied they were with their respective functions as final year medical students and recent graduates. They were again to respond, using open answers, to how satisfied they were with their respective medical training. Finally, they were to rate their overal satifaction for the profession based on their open-ended responses on a scale of - (1) Yes, (2) Averagely well, (3) Uncertain, and (4) Not satisfied with the profession. There were 19 questions on the perceived level of preparedness of the respondents in psychosocial skills and 11 more in clinical patient management skills. Each question from both domains were respectively answered by the respondents on a five-point Likert-type scale of (1) Poor, (2) Fair,(3) Adequate, (4) Well, (5) Very well prepared. Table 1 shows examples of questions used in the questionnaire. The respondents were also requested to answer yes or no to the question on whether they felt they should have been better prepared in the different skill(s) during their medical training. Table 2 provides a list of the various questions that were included in the questionnaire.
STUDENTS FROM CONVENTIONAL VS PBL CURRICULA
145
Table II. The list of questions included in the questionnaire How well were/are you prepared in/to: Psychosocial and interpersonal skills: -
-
-
-
-
Beginning and establishing a continuous conversation with patients Listening to what patients complain of; a. When they appear to be relevant to the current illness b. When they do not appear to be relevant to the current illness Dealing with emotional and personal problems of patients sometimes encountered Obtaining clinical history from; a. Difficult patients b. Patients with difficult illnesses Explain the medical problem(s) of a patient to him/her; a. When asked (by the patient) b. Without being asked (by the patient) Inform patients when you are unable to help; a. With acute illnesses b. With chronic illnesses c. Terminal illnesses (e.g Cancer) Involving patients in the management of their illnesses Explain the rights and responsibilities of the patients to them Accept to fulfil the wish(es) of patients in certain instances e.g choice of medication Breaking bad news to relatives of patients e.g Death of a child or loved one Share responsibility with other members of the health team Receiving unpleasant criticisms from colleagues (and/or superiors) Receiving unpleasant criticisms from patients Give unpleasant criticisms to colleagues
Clinical patient management skills: How well were/are you prepared in/to; - Perform standard* physical examinations before making a diagnosis - Perform tailored* physical examinations before making a diagnosis Handle clinical problems with unclear indicators of a likely diagnosis(es) - Perform basic biochemical investigations in the ward side-laboratory or bedside (e.g Urine tests, Blood smears) - Handle emergency situations e.g Fractures, Coma, Bums. - Making use of results of laboratory investigations in diagnosis and treatment - Identifying associated clinical problems unmentioned by the patient Invite other clinical specialities in management of your patient Prescribing appropriate treatment (drugs/medication) in patient care Handle the management of a patient with a contagious and/or deadly disease (e.g AIDS) Admitting when a case is beyond your capability
146
JAMIU 0. BUSARI ET AL.
Table II. Continued Satisfaction for the profession: - Having learnt so much in medical school, seen and participated in various clinical training activities, would you say you are/were satisfied with your level of preparedness for medical practice? Yes ( ) Averagely well ( ) Uncertain ( ) No( ) Give reasons (not more than 3 sentences) for your choice made above. -
-
Having learnt so much in medical school, seen and participated in various clinical training activities, would you say you are/were satisfied with the profession? Yes ( ) Averagely well ( ) Uncertain ( ) No ( ) Give reasons (not more than 3 sentences) for your choice made above.
Standard here refers to conducting a detailed clinical or physical examination irrespective of complaints of the patient or the likely diagnosis a doctor may have in mind in the case of the former, and tailored to clinical examination conducted based solely on the complaints or suspected problem a patient may have.
Method In May 1994, questionnaires were distributed amongst 180 final year students and posted to recent graduates (doctors in practise for not more than 12 months) from the medical schools of the University of Groningen (RUG) N = 66, the Catholic University Nijmegen (KUN) N = 54, and the University of Maastricht (UM) N = 60, Maastricht respectively. The responses between the final year medical students and the recent graduates per university showed negligible differences and therefore the various data were combined respectively per university in the analysis of results. Result A total of 127 questionnaires (UM=36 (60%), KUN=25 (46%), RUG=66 (100%)) were returned by the respondents, a total response rate of 70.5%. The questionnaire's reliability on psychosocial and clinical patient management skills was carried out using Cronbach's alpha test of reliability. 92 (72%) of the 127 questionnaires returned were completely filled in without any missing values. Coefficient alpha computed to test the reliability of the various item clusters produced alpha's of 0.95 on psychosocial and interpersonal skills, 0.83 on clinical patient management skills. Cronbach's alpha was an unsuitable method for testing the items on professional satisfaction due to their construct. The reliability coefficient for the questionnaire produced an alpha of 0.85. It could therefore be concluded that the questionnaire was a reliable instrument for this study.
147
STUDENTS FROM CONVENTIONAL VS PBL CURRICULA Table III. Responses from graduates and students on the satisfaction with their respective activities/functions (four-point scale; yes = 1, no = 4)
UM KUN RUG
Satisfaction with function/activity Mean SD
Satisfaction with medical training Mean SD
1.88 1.72 1.50
1.52 1.88 1.81
0.66 0.67 0.63
0.69 0.66 0.72
PROFESSIONAL SATISFACTION
In general, results revealed that the graduates and students from the three institutions felt fairly satisfied with their various activities/functions as students and doctors (Table 3). However the analysis of variance of the results showed significant differences between the institutions (F(2,124) = 4.27, p = 0.02). The respondents from the UM felt less satisfied with their activities than their colleagues from the RUG and the KUN. The respondents felt fairly satisfied (average mean = 1.74) with the medical training they all received from their respective institutions. Between institutions, the UM respondents appear to have felt slightly better prepared than their colleagues from the KUN and RUG. ANOVA however, only showed marginally significant differences (F(2,124) = 2.53, p = 0.08). An inventory on the answers to the open-ended questions in the questionnaire, revealed the Maastricht respondents expressing more dissatisfaction with irregular working shifts, lack of enthusiasm from superiors, inadequate supervision, heavy workload, too much bureaucracy and budgetary cuts. Responses from Groningen and Nijmegen centred mainly around inadequate training in psychosocial and interpersonal skills, theory-oriented rote-learning preclinical education, and inadequate training in communication skills.
PSYCHOSOCIAL SKILLS
The graduates' and students' views on their level of preparedness in psychosocial skills are provided in Table 4. The respondents from the RUG and the KUN felt they were inadequately prepared in psychosocial skills, while their counterparts from the UM felt that they had been well prepared by their training. Analysis of variance (ANOVA) of the results showed a significant difference (F(2,124) = 31.29, p = 0.00). Post hoc comparisons of means (Scheff6) also revealed significant differences between the students and recent graduates of the UM and their counterparts from the RUG and KUN. Maastricht students and graduates felt better prepared than their counterparts from Groningen and Nijmegen in psychosocial skills.
148
JAMIU 0. BUSARI ET AL. Table IV. Means and standard deviations of the 19 questions on psychosocial skills, split into "How well were/are you prepared by your medical training in/to?" and "Do you think you should have been better prepared in this skill during your training?" Level of preparedness (1 = poor, 5 = very well) Mean SD UM KUN RUG
3.34 . 2.24 2.46
0.47 0.68 0.65
Should you have been better prepared (0 = no, 1 = yes) Mean SD 0.29 0.51 0.49
0.16 0.29 0.24
Table V. Means and standard deviations of the 11 questions on clinical patient management skills, split into "How well were/are you prepared by your medical training in/to?" and "Do you think that you should have been better prepared in this skill during your training?"
UM KUN RUG
Level of preparedness (1 = poor, 5 = very well) Mean SD
Should you have been better prepared (0 = no, 1 = yes) Mean SD
3.09 2.89 2.94
0.40 0.42 0.37
0.58 0.50 0.67
0.67 0.17 0.22
Their views on the question if their training in psychosocial skills should have been better, revealed significant differences (F(2,124) = 9.58, p = 0.00). Post hoc comparisons also showed significant differences between the UM and the RUG and KUN. The respondents from the RUG and KUN felt more strongly that they should have better been prepared in their psychosocial skills than their colleagues from the UM. CLINICAL PATIENT MANAGEMENT SKILLS
The results with respect to how well prepared the respondents from the three institutions felt in clinical patient management skills revealed no remarkable difference (Table 5). Their respective levels of preparedness in these skills were perceived as more or less adequate (F(2,124) = 0.92, p = 0.40). The results also showed no remarkable difference in the adequacy of the training provided, by the respective curricula (F(2,124) = 0.82,p = 0.44). Nevertheless, 60% of the respondents indicate that clinical patient management training in the respective curricula could still be improved.
STUDENTS FROM CONVENTIONAL VS PBL CURRICULA
149
Discussion The results obtained in this study agreed with our hypothesis that differences are expected to be found in psychosocial skills between students and graduates from PBL and traditional medical schools. Respondents from the UM were quite satisfied with their psychosocial skills training, while the others were not. This is not surprising since the training of psychosocial and communicative skills has been incorporated into the problem-based curriculum of the UM. The results on professional satisfaction did not support the hypothesis that the PBL students and graduates would be more satisfied with the medical profession than their counterparts in traditional medical schools. UM respondents, contrary to our expectations appeared less satisfied with the profession than their counterparts from RUG and KUN. On the other hand, the KUN and RUG respondents felt less satisfied with their medical training, though ANOVA showed a marginally significant difference. Answers to the open-ended questions in the questionnaire showed that the KUN and RUG respondents used this medium to express their dissatisfaction with their respective medical training. The respondents from the UM, on the other hand, used the medium to express criticisms on their activities as students and doctors and the respective learning and working conditions associated with these activities. There is unfortunately no single and clear-cut answer to explain these findings. The results also showed, as expected, that there were no remarkable differences in clinical patient management skills, due to different curricula. Students and graduates from the three institutions felt equally (dis)satisfied with their clinical patient management skills training. This may however surprise the reader who according to Barrows (1984) might have expected that since PBL furthers clinical problem-solving skills the UM respondents should possess better clinical patient management skills, and therefore start to wonder if this present finding proves otherwise. The explanation for this finding can be found in the skills referred to in the questionnaire, which were actually practical aspects of diagnosis and patient management in a clinical setting (see Table 2), and not the cognitive problem-solving skills Barrows (1984) referred to. PBL students may very well have better cognitive problem solving skills, but do not necessarily excel their colleagues in practical clinical problem solving skills, referred to as clinical patient management skills in this study. Furthermore, many of the issues investigated in this part of the questionnaire were learnt during the clinical rotations which are very similar in the PBL and traditional medical schools. It is our impression that the present result, although in line with our expectations, can not be taken for granted since we are aware that self-assessments do not always provide objective evaluation of the mastery of skills (Gordon, 1991).
150
JAMIU O. BUSARI ET AL.
However, since these limitations in our study apply equally to the different groups of respondents, the results can be considered to be indicative of the effects of the different curricula at the three universities. The perceived preparedness of the respondents in this study corresponds with the respective trainings they had all received in their different curricula. Though these differences may not appear large enough to compel the adoption of PBL into medical curricula, our study strengthens the recommendations of Friedman et al. (1990), and there is enough support for further studies in which the skills of students from different schools will be tested. Acknowledgements The research reported in this article was done in partial fulfilment of the requirements for the Masters Health Professions Education course by the first author. The authors wish to thank Prof. J. Metz Ph.D., Catholic University Nijmegen, and R. Hiemstra, MD, University of Groningen, for their assistance. Jamiu Busari is a Resident doctor, department of Paediatrics, Academic Hospital Maastricht, and holds a masters degree in health professions education (MHPE), from the Maastricht University. Albert Scherpbier is assistant professor. He is the managing director of the Skills Lab of the medical school of the Maastricht University. Henny Boshuizen is associate professor in the Department of Educational Research and Development. She is the Program Director of the international course Masters of Health Professions Education (MHPE), Maastricht University Maastricht, The Netherlands. References Albanese, M.A. & Mitchell, S. (1993). Problem Based Learning, a Review of Literature on Its Outcomes and Implementation Issues. Academic Medicine 68: 52-81. Barrows, H.S. (1984). A Specific Problem - Based, Self-Directed Learning Method Designed to Teach Medical Problem-Solving Skills, and Enhance Knowledge Retention and Recall. In Schmidt, H.G. & de Volder, M.L. (eds.) Tutorials in Problem-Based Learning, 16-32. Van Gorcum: Assen/Maastricht. Bender, W., Cohen-Schotanus, J., Imbos, T.J., Versfelt, W.A. & Verwijnen, G.M. (1984). Medische kennis bij studenten uit verschillende faculteiten: van hetzelfde laken een pak? (Medical Knowledge of Students from Different Universities; of a Similar Kind?) Nederlandse Tijdschrijft van Geneeskunde 128: 917-921. Berkson, L. (1993). Problem Based Learning. "Have the Expectations Been Met?", Academic Medicine 68: S79-S88. Goodman, L., Brueschky, E., Bone, R., Rose, W., Williams, J. & Paul H. (1991). An Experiment in Medical Education. Journalof the American Medical Association 265: 2373-2376. Gordon, M.J. (1991). A Review of the Validity and Accuracy of Self-Assessments in Health Professions Training. Academic Medicine 66: 762-769. Friedman, C.P., De Bliek, R., Greer, D.S., Mennin, S.P., Norman, G.R., Sheps, C.G., Swanson, B.D. & Woodward, C.A. (1990). Charting the Winds of Change: Evaluating Innovative Medical Curricula. Academic Medicine 65: 8-14.
STUDENTS FROM CONVENTIONAL VS PBL CURRICULA
151
Kaufman, A., Mennin, S., Waterman, R., Duban, S., Hansbarger, C., Silverblatt, H., Obenshain, S., Kantrowitz, M., Becker, T., Samet, J. & Wiese W. (1989). The New Mexico Experiment: Educational Innovation and Institutional Change. Academic Medicine 64: 285-294. Moore, G.T., Block, S.D., Briggs Style, C. & Mitchell, R. (1994). The Influence of the New Pathway Curriculum on Harvard Medical Students. Academic Medicine 69: 983-989. Moore-West, M. & O'Donnell, M.J. (1985). Program Evaluation. In Kaufman, A. (ed.) Implementing Problem-BasedMedical Education, 180-206. Springer Publishing: New York. Patel, V.L., Groen, G.J. & Norman, G.R. (1991). Effects of Conventional and Problem-Based Medical Curricula on Problem solving. Academic Medicine 66: 380-389. Richards, B.F., Ober, P.K., Cariaga-Lo, L., Camp, G., Philp, J., Mcfarlane, M., Rupp, R. & Zaccaro D.J. (1996). Ratings of Students' Performances in a Third-Year Internal Medicine Clerkship: A Comparison Between Problem-Based and Leture-Based Curricula. Academic Medicine 71: 187-189. Schmidt, H.G., Dauphinee, W.D. & Patel, V.L. (1987). Comparing the Effects of Problem-Based and Conventional Curricula in an International Sample. Journalof Medical Education 62: 305-315. Schmidt, H.G. (1990). Innovative and Conventional Curricula Compared: What Can Be Said About Their Effects? In Nooman, Z., Schmidt, H.G. &Ezzat, E. (eds.) Innovation in Medical Education; An Evaluation of Its Present Status, 1-7. Springer Publishing: New York. Vernon, D.T.A., Blake, R.L. (1993). Does Problem-Based Learning Work? A Meta-Analysis of Evaluative Research. Academic Medicine 68: 550-563.
