2014; 36: 164–168
Encouraging an environment to nurture lifelong learning: An Asian experience JOSHUA L. JACOBS, DUJEEPA D. SAMARASEKERA, LIANG SHEN, K. RAJENDRAN & SHING CHUAN HOOI National University of Singapore, Singapore
Abstract Introduction: Within an Asian context, this study examines the effect of changing from traditional course grades to a distinction/ pass/fail (D/P/F) grading system on medical student self-perceived stress levels and on student exam performance. Methods: At the end of the 2010–2011 academic year, the Perceived Stress Scale-10 (PSS-10) was administered to the cohort of students finishing their first year of medical studies. For the academic year 2011–2012, the grading system was changed to D/P/F for the first year of medical school. The PSS-10 was also administered to the subsequent cohort of first-year medical students at the same point in the academic year as previous. Qualitative comments were collected for both cohorts. Results: Stress as measured by the PSS-10 was significantly lower in the cohort that went through the year with the D/P/F grading system in place. Thematic analysis of qualitative responses showed a shift in sources of student stress away from peer-competition. There were no significant differences in overall exam performance. Discussion: Within an Asian context, switching to a D/P/F grading system can alleviate stress and peer competition without compromising knowledge. This may help foster a ‘‘learning orientation’’ rather than an ‘‘exam orientation,’’ and contribute to inculcating lifelong learning skills.
Introduction
Practice points
Pursuit of lifelong learning is one of the hallmarks of the medical professional. It requires intrinsic motivation and must be inculcated and nurtured throughout the formal education and training period to maximize the likelihood that it becomes ingrained in the persona of the trainee. This goal of medical education is challenging to realize given that most curricula are organized such that ‘assessment drives performance’ (Godfrey 1995). In Asia and many other parts of the world, an exam-oriented focus is often rewarded with more opportunities for career development, progression, and advancement (Li & Li 2010). The results of high-stakes exams during primary and secondary school help shape the trajectory of education the pupil will be eligible for. Examples of these types of exams include the Primary School Leaving Examination in Singapore, the General Certificate of Examination in the United Kingdom, the Stanford Achievement Test (SAT) in the USA, International Baccalaureate (IB) exams, and others. A poor performance at this stage in life can strongly affect the student’s chances of obtaining access to a medical education in the local system. The number of applicants seeking to gain admission to medical school (the ‘‘demand’’) vastly outnumbers the slots available (the ‘‘supply’’). Due to this fact, getting into medical school is a highly competitive process. In countries with medical schools matriculating students from secondary school (aka ‘‘school-leavers’’), getting into medical school can be a major focus for parents and students alike. Most medical
. Stress over peer-competition and exam-focus can prevent medical students from maximizing learning. . In an Asian context in the first year of undergraduate medical education, changing from traditional grades to a Distinction/Pass/Fail grading system resulted in lower stress. . Student knowledge acquisition as measured by exam performance was preserved in the new grading system.
20 14
school admissions processes give significant weightage to academic performance, as evaluated by grades and exam performance (Dunleavy et al. 2011). Therefore, most students who are successful in gaining entry have been highly focused on exam performance, grade achievement, and out-competing their peers. This orientation is usually further reinforced by the secondary school teachers. Schools with more successful medical school applicants can use this statistic to enhance their reputation and desirability in the community. Teachers who successfully train their students to do well on standardized exams often get rewarded based on these metrics (Baker et al. 2010). Despite the intense focus on exams and grade performance as a gating item to entry into medical school and to progression within medical school, some studies show performance on these measures is not well correlated with performance as a doctor (Wingard & Williamson 1973; Taylor & Albo 1993; Probert et al. 2003). Furthermore,
Correspondence: Joshua L. Jacobs, MD, Yong Loo Lin School of Medicine, National University of Singapore, 1E Kent Ridge Road, NUHS Tower Block Level 11 Dean’s Office, Singapore 119228, Singapore. Tel: þ65 6772 3824; fax: þ65 6778 5743; email: [email protected]
164
ISSN 0142–159X print/ISSN 1466–187X online/14/020164–5 ß 2014 Informa UK Ltd. DOI: 10.3109/0142159X.2013.852168
Encouraging lifelong learning
exam-related stress can cause excessive cognitive load that, according to cognitive load theory, can result in less absorption, understanding, or retention of material (Sweller et al. 1998). As a lifelong learning endeavor, medicine is ideally collaborative, not competitive. Healthcare providers work in teams to address the needs of their patients (Hopkins & Baker 2010). Given the nature of their future practice, there is a need to re-orient medical student learners to learning, rather than exam performance. Instead of hiding deficiencies to show a good exam result or course grade, students should be encouraged to partner with their teachers to explore areas of relative weakness in order to improve and ultimately provide better, safer patient care and do better research. In order to minimize exam-orientation in school-leaver students matriculating into medical school from a highly competitive, exam-oriented setting, the National University of Singapore Yong Loo Lin School of Medicine (YLLSoM) switched year 1 grading from traditional grades (A–F) to a grading system of Distinction/Pass/Fail (D/P/F). It was hypothesized that this change would reduce stress among the students overall, and help shift their learning orientation from being exam-focused to more learning-focused. The YLLSoM curriculum is a five-year course of study that is direct-entry from secondary school (i.e. school-leavers) that culminates in the awarding of a Bachelor of Medicine, Bachelor of Surgery (MBBS) degree. The first two years of the course have clinical components but focus primarily on pre and paraclinical sciences basic to medicine. Year 1 focusses on normal structure and function of the body, while year 2 focusses on abnormal structure and function. The remaining three years of teaching and learning are based primarily in clinical settings and focus on the clinical sciences. There is a high-stakes barrier exam at the end of each year of study in addition to periodic summative continual assessments. Upon completing medical school, graduates compete for postgraduate clinical training posts in the various specialties and pursue an undifferentiated year as a house officer, or a specialty-track residency program accredited by the Accreditation Council for Graduate Medical Education International (ACGME-I).
Methods The rationale, proposed workflow and operational implications of switching year 1 to D/P/F grading was discussed over several months with approvals obtained from all stakeholders including appropriate departments, faculty, and university committees. Appropriate ethical approvals for the study were obtained from the Institutional Review Board at the National University of Singapore. The instrument chosen to measure stress was the perceived stress scale 10 (PSS-10; Cohen et al. 1983). As all students are native English speakers, no translation of the scale was needed. In addition to the 10 items that are based on a quantitative scale, three free-text response items were included to gather qualitative data. The 10-item tool has been validated in many settings and cultures, with age and gender norms well established (Malarkey et al. 1995; Lane et al. 2007; Leon et al. 2007; McAlonan et al. 2007).
The free-text response items included were: ‘‘What are the most stressful things for you in the curriculum/learning environment?’’, ‘‘List 3 things that you enjoyed in Phase I [Year 1] learning’’, and ‘‘Do you think grading or ranking leads to unwanted levels of student stress or poor performance? If yes please explain?’’ The survey instrument was administered at the same time point in the curriculum timetable for year 1 students in the academic year before the change in grading (graduating class 2016), and after the change (graduating class 2017). There were no changes to the curriculum for year 1 students from one year to the next. After the change in grading, year 1 students were unable to view the marks performance on continual assessments and final exams. Students who performed at a borderline pass level and those who failed were given individualized feedback and remediation. A two-tailed t-test for equality of means was used to compare exam performance data between the two cohorts. General linear regression was used to compare the Perceived Stress Scale 10 between the two cohorts with adjusting the gender effect. A p-value 50.05 was considered significant. Data analysis was performed using SPSS version 20 (IBM SPSS Statistics, NY).
Results Of the 255 students in the graduating class of 2016 who completed all exams in year 1, a total of 238 (93%) completed the scale. Of the 282 students in the class of 2017 who completed all exams in year 1, a total of 253 (90%) completed the scale. Respondent characteristics are shown in Table 1. Comparison of performance on final examinations (worth 60% of total marks for the year), total marks for the year, and continual assessment (worth 40% of total marks for the year) of the two cohorts of students is shown in Table 2. The results show no significant difference in total marks for the year or in marks on the final exams between the two cohorts. There is a significant difference in marks for the continual assessment component between the cohorts, with the 2017 cohort having higher marks. The mean difference was 0.6 (95% CI 0.1–1.1). The mean (SD) Perceived Stress Scale 10 score of 2016 cohort and 2017 cohort was 18.98 (5.69) and 17.58 (6.07), respectively. The difference was significant ( p ¼ 0.009, two sample t test). Moreover, the mean Perceived Stress Scale 10 score of females (mean 19.37, SD 5.45) was significantly higher than that of males (mean 17.12, SD 6.18) ( p 5 0.0005, two sample t test). General linear model showed there is no significant interaction between cohort and gender ( p ¼ 0.572), hence only main effects of cohort and gender were included into the final model. It showed that cohort 2016 had significantly higher mean PSS 10 score compared with the cohort 2017 ( p ¼ 0.004, mean difference 1.52, 95% CI 0.49– 2.55). Females had significantly higher mean PSS 10 score compared with males ( p 5 0.0005, mean difference 2.33, 95% CI 1.30–3.35) (Table 3). A thematic analysis of the qualitative feedback revealed the main source of stress for the 2016 cohort to be ‘‘competition with peers.’’ Illustrative quotes from respondents include
165
J. L. Jacobs et al.
Comparing my pace of studying with others when I know I shouldn’t and feeling like there is insufficient time to complete all my revision; The number of geniuses present in my cohort and the extreme competiveness; A lot of people are quite competitive so you suddenly feel the need to study like crazy just to keep up. For the 2017 cohort, the main source of stress was ‘‘uncertainty about residency’’ [post-graduate training], as exemplified by these quotes from respondents: ‘‘In today’s society, especially in Singapore, competiveness is cyclonical and it causes unintended levels of stress because everyone is fighting for the limited slots of residency’’; ‘‘The future struggle for limited residency slots. Suddenly, studying is not enough and we will have to do research, hold leadership positions etc. all in a bid to boost our CV to compete with other applicants and overseas colleagues.’’ A subset of the 2017 cohort also asked for ‘‘more granularity’’ to know if they passed or ‘‘just barely’’ passed. Of note, subsequent to the finish of the year 1 curriculum using the D/P/F grading, the medical students of the 2017 cohort submitted results to a poll initiated by students and that Table 1. Characteristics of graduating class year 2016 cohort and graduating class year 2017 cohort.
Completed PSS-10 (% of cohort that completed all exams)
Completed all exams 2016 cohort Male Female 2017 cohort Male Female
255 136 119 282 134 148
238 124 114 253 119 134
(93%) (91%) (96%) (90%) (89%) (91%)
Table 2. Comparison of performance on marks during year 1 for the graduating class year 2016 cohort and graduating class year 2017 cohort.
Final exam marks (of 100) Year-end total marks (of 100) Continual assessment marks (of 40)
Grad class year
N
2016 2017 2016 2017 2016
255 282 255 282 255
71.8 71.82 72 72.08 28.57
2017
282
29.17 (3.03)
Mean (SD) (8.06) (8.11) (7.45) (7.48) (2.88)
p-Value 0.98 0.9 0.02
Bold value is statistically significant ( p 5 0.05).
was not solicited by the school. The poll asked students to support either the statement ‘‘I support the continuation of Pass/Fail for Phase One’’, or the statement ‘‘I do not support the continuation of Pass/Fail for Phase One’’. One hundred seventy-eight of the 280 students (64%) responded to the poll. Of those, 165 of the 178 (93%) supported the continuation, and 136 of the 178 (76%) supported extending this grading system to year 2 of the curriculum.
Discussion With no other changes to the curriculum in year 1, the stress level of the cohort of students that studied under a grading framework of D/P/F was lower than the preceding cohort of students that studied under a traditional grading framework (A–F). Marks for performance on the final exam and total marks for the year were not significantly different between cohorts. There was a significant difference in marks for continual assessment, with better performance in the 2017 cohort. This finding is intriguing, and may warrant further study of the possible link between grade reporting, and test performance. The finding suggests that introducing D/P/F grading does not result in poorer exam performance. Examination of the qualitative comments indicates that the major source of stress for the 2016 cohort was peer competition, presumably from comparing exam performance grades. With the removal of grades, there appears to be a shift away from a peer-competition mindset, with the 2017 cohort requesting to know roughly where they stand in relation to the pass mark, rather than in relation to each other. A direct quote from a 2017 cohort respondent highlights If you don’t want stress or competition, give feedback to ALL students in terms of percentage, marked scripts. Perhaps in a single line ‘‘you are fine’’ or ‘‘you need to buck up". At least something to tell us where our knowledge stands. A better communication campaign may help alleviate the fear of uncertainty. Upon further discussion with students, it became clear that many were not aware that ‘‘no news is good news’’ and that if they were showing areas of weakness they would be approached by the Dean’s Office for remediation. These findings suggest that D/P/F grading may be linked to alleviation of stress among medical students. This alleviation of stress may help them focus on learning, rather than exam-taking. This shift in orientation will help medical schools foster the attributes of lifelong learning. The poll from the medical students of the 2017 cohort further supports the themes gathered from the qualitative comments in the study, and points to the high level
Table 3. Comparison of results on the Perceived Stress Scale 10 for the graduating class year 2016 cohort and graduating class year 2017 cohort.
Grad class year 2016 2017 Gender Male Female
166
N
Unadjusted mean difference (95% CI)
Unadjusted p-value
Adjusted mean difference (95% CI)
Adjusted p-value
238 253
1.40 (0.36–2.45) –
0.009
1.52 (0.49–2.55) –
0.004
243 248
– 2.25 (1.22–3.28)
50.0005
– 2.33 (1.3–3.35)
50.0005
Encouraging lifelong learning
of student acceptability of and desire for this type of grading framework. Stress levels in both cohorts (graduating class year 2016 and 2017) were higher than the reference values for age and gender in the general population (Cohen & Williamson 1988). Additionally, stress as measured by the PSS-10 was higher in females than in males across both cohorts examined. Causes for this were not elucidated in the current study. Strengths of this study include the large cohorts with good response rates, use of a validated instrument, and the addition of qualitative data to aid in interpretation of the quantitative results. To our knowledge, this is the first such English language study of medical student stress in an Asian context. The experience and manifestations of stress may be quite different in cultures with strong Confucian, Islamic, Hindu, and/or Buddhist principles compared to many Western cultures with traditional JudeoChristian value systems (Thang 2004; Aminuddin et al. 2011; Chia 2013). This study is not without its limitations. The cohort study design limits the conclusions that can be definitively drawn from the data. The qualitative comments comport with the conclusion from the quantitative data that the differences between cohorts can largely be explained by the change to the grading system. However, there may have been other contributing factors that influenced the cohort’s experience or expression of stress. For example, the 2016 and 2017 cohorts may have communicated with each other about this topic, with the 2016 cohort giving a message that the D/P/F grading system is ‘‘better’’ or causes less stress than the system they experienced. This may have influenced the 2017 cohort’s perception of stress induced by the grading system, resulting in something akin to a placebo effect. An additional area that was not examined for effect on stress scores was the communication of grades versus a change to the grading system. It is possible that simply masking the traditional letter grades from students would result in a similar decrement in stress scores as changing to the D/P/F system. Although instituting D/P/F in year 1 appears to be linked to lower stress levels, we cannot definitively assume causation. Additionally, the results cannot be generalized to other years of study. Finally, although the results show lower levels of stress and a shift away from peercompetition, they do not demonstrate conclusively a shift away from exam-orientation. The results are encouraging. In the preamble to the unsolicited student poll described in the results section, the students pointed out the benefits of the D/P/F that support the qualitative themes that emerged from the PSS-10 survey. Namely, the preamble cited the benefits of less competition and comparison between students. A direct quote from the preamble states ‘‘We strongly believe that the Pass/Fail system has been and will be greatly beneficial to us in shaping an ideal learning environment where learning is done not for the pursuit of grades but for attaining knowledge so that we may be better doctors in the future.’’
retention (Sweller et al. 1998). To study for a traditional knowledge-based medical school exam, students may adopt a study strategy to load several facts for short-term retrieval on the exam. By removing the high-stakes of the exam, students may adopt a learning strategy that consolidates, integrates and solidifies slightly fewer concepts, but for a deeper understanding and retention. Further study in this area should be done to investigate this effect. In retaining a ‘‘distinction’’ category, peer-competition and exam-focus have not been entirely removed. Homuth has suggested that retaining this category may even defeat the intent of switching to a pass/fail system (Homuth 2008). The effect of retaining the ‘‘distinction’’ category should be further investigated. Measurement of stress in medical students should continue to confirm and extend the findings of the current study. The lower stress levels in the 2017 cohort are encouraging. The negative effects of stress on medical student health has been repeatedly and well documented (Tyssen et al. 2000, 2001; Dyrbye et al. 2006; Voltmer et al. 2012), and measures that reduce stress should be supported. YLLSoM will be evaluating this and other data to determine if the D/P/F scheme should be extended to other years. Based partly on the successful implementation at the medical school, other faculties on the university campus are now considering a similar change to their grading systems. If this occurs, studies of the effects on stress, performance, and exam-orientation should be conducted on the different student cohorts to verify generalizability.
Glossary Stress: A state of mental or emotional strain or tension resulting from adverse or demanding circumstances http://oxforddictionaries.com/definition/english/stress
Notes on contributors All authors work at the National University of Singapore Yong Loo Lin School of Medicine. Dr. JOSHUA L. JACOBS, MD, is the Associate Professor, Department of Medicine, Division of Family Medicine. He serves as Assistant Dean (Education) and is a Senior Consultant in Family Medicine at the National University Hospital. Dr. DUJEEPA D. SAMARASEKERA, MBBS, is the Director, Medical Education Unit. He also consults for the Singapore Ministry of Health on medical education matters. Dr. LIANG SHEN, PhD, is the Manager, Dean’s Office. She is a statistician supporting health services research and other research programs at the school.
Next steps
Dr. K RAJENDRAN, MBBS, is Associate Professor, Department of Anatomy. He is the Phase I (Year 1) Lead.
Keeping in mind cognitive load theory, a shift from exam orientation to learning orientation may enhance learning and
Prof. SHING CHUAN HOOI, MBBS, is the Professor, Department of Physiology. He is the Vice Dean (Education) overseeing all aspects of undergraduate medical education for the school.
167
J. L. Jacobs et al.
Acknowledgements The authors wish to thank the staff of the Dean’s Office and the Phase I Committee at the Yong Loo Lin School of Medicine for their support in this project. Declaration of interest: The authors report no conflicts of interest. The authors alone are responsible for the content and writing of this article. Ethical approval has been granted for the study described in this manuscript by the Institutional Review Board of the National University of Singapore – NUS IRB Code12-492E.
References Aminuddin H, Abiddin NZ, Anuar NAAK. 2011. Viewing the philosophy of education from the perspectives of different schools of thought. Int Rev Soc Sci Human 1(2):55–64. Baker EL, Barton PE, Darling-Hammond L, Haertel E, Ladd HF, Linn RL, Ravitch D, Rothstein R, Shavelson RJ, Shepard LA. 2010. Problems with the use of student test scores to evaluate teachers. Economic Policy Institute. Washington, DC. Briefing Paper No. 278. Chia MO. 2013. Major differences between eastern and western philosophies as the basis for adult education - The Singapore experience. European Association for the Education of Adults. [Accessed 10 February 2013]. Available from http://www.eaea.org/index.php?k=12117. Cohen S, Kamarck T, Mermelstein R. 1983. A global measure of perceived stress. J Health Soc Behav 24(4):385–396. Cohen S, Williamson GM. 1988. Perceived stress in a probability sample of the United States. In: Spacapan S, Oskamp S, editors. The social psychology of health. Newbury Park, CA: Sage. pp 31–67. Dunleavy D, Sondheimer H, Bletzinger R, Castillo-Page L. 2011. Medical school admissions: More than grades and test scores. AAMC Newsletter 11(6). Dyrbye LN, Thomas MR, Shanafelt TD. 2006. Systematic review of depression, anxiety, and other indicators of psychological distress among U.S. and Canadian medical students. Acad Med 81:354–373. Godfrey RC. 1995. Undergraduate examinations – A continuing tyranny. Lancet 765:3. Homuth D. 2008. How honours/pass/fail grading is failing students. UTMJ 85(2):99–102.
168
Hopkins D, Baker PG. 2010. Framework for action on interprofessional education and collaborative practice. Geneva, Switzerland: World Health Organisation Press. Lane JD, Seskevich JE, Pieper CF. 2007. Brief meditation training can improve perceived stress and negative mood. Alternat Ther Health Med 13(1):38–44. Leon KA, Hyre AD, Ompad D, DeSalvo KB, Muntner P. 2007. Perceived stress among a workforce 6 months following hurricane Katrina. Soc Psych Psych Epid 42(12):1005–1011. Li W, Li Y. 2010. An analysis on social and cultural background of the resistance for China’s education reform and academic pressure. IES 3(3):211. Malarkey WB, Pearl DK, Demers LM, Kiecolt-Glaser JK, Glaser R. 1995. Influence of academic stress and season on 24-hour mean concentrations of ACTH, cortisol, and beta-endorphin. Psychoneuroendocrino 20(5):499–508. McAlonan GM, Lee AM, Cheung V, Cheung C, Tsang KW, Sham PC, Chua SE, Wong JG. 2007. Immediate and sustained psychological impact of an emerging infectious disease outbreak on health care workers. Can J Psychiatr 52(4):241–247. Probert CS, Cahill DJ, McCann GL, Ben-Shlomo Y. 2003. Traditional finals and OSCEs in predicting consultant and self-reported clinical skills of PRHOs: A pilot study. Med Educ 37:597–602. Sweller J, Van Merrie¨nboer J, Paas F. 1998. Cognitive architecture and instructional design. Educ Psychol Rev 10(3):251–296. Taylor CW, Albo D. 1993. Measuring and predicting the performances of practicing physicians: An overview of two decades of research at the university of Utah. Acad Med 68(2):S65–S67. Thang SM. 2004. Student approaches to studying: Identifying the Malaysian constructs and comparing them with those in other contexts. J Further Higher Educ 28(4):359–371. Tyssen R, Vaglum P, Gronovold NT, Ekeberg O, Tyssen R. 2000. The impact of job stress and working conditions on mental health problems among junior house officers. A nationwide Norwegian prospective cohort study. Med Educ 34:374–384. Tyssen R, Vaglum P, Gronovold NT, Ekeberg O. 2001. Factors in medical school that predict postgraduate mental health problems in need of treatment. A nationwide and longitudinal study. Med Educ 35:110–120. Voltmer E, Kotter T, Spahn C. 2012. Perceived medical school stress and the development of behavior and experience patterns in German medical students. Med Teach 34(10):840–847. Wingard JR, Williamson JW. 1973. Grades as predictors of physicians’ career performance: An evaluative literature review. J Med Educ 48:311–322.
Copyright of Medical Teacher is the property of Taylor & Francis Ltd and its content may not be copied or emailed to multiple sites or posted to a listserv without the copyright holder's express written permission. However, users may print, download, or email articles for individual use.
Encouraging an environment to nurture lifelong learning: An Asian experience JOSHUA L. JACOBS, DUJEEPA D. SAMARASEKERA, LIANG SHEN, K. RAJENDRAN & SHING CHUAN HOOI National University of Singapore, Singapore
Abstract Introduction: Within an Asian context, this study examines the effect of changing from traditional course grades to a distinction/ pass/fail (D/P/F) grading system on medical student self-perceived stress levels and on student exam performance. Methods: At the end of the 2010–2011 academic year, the Perceived Stress Scale-10 (PSS-10) was administered to the cohort of students finishing their first year of medical studies. For the academic year 2011–2012, the grading system was changed to D/P/F for the first year of medical school. The PSS-10 was also administered to the subsequent cohort of first-year medical students at the same point in the academic year as previous. Qualitative comments were collected for both cohorts. Results: Stress as measured by the PSS-10 was significantly lower in the cohort that went through the year with the D/P/F grading system in place. Thematic analysis of qualitative responses showed a shift in sources of student stress away from peer-competition. There were no significant differences in overall exam performance. Discussion: Within an Asian context, switching to a D/P/F grading system can alleviate stress and peer competition without compromising knowledge. This may help foster a ‘‘learning orientation’’ rather than an ‘‘exam orientation,’’ and contribute to inculcating lifelong learning skills.
Introduction
Practice points
Pursuit of lifelong learning is one of the hallmarks of the medical professional. It requires intrinsic motivation and must be inculcated and nurtured throughout the formal education and training period to maximize the likelihood that it becomes ingrained in the persona of the trainee. This goal of medical education is challenging to realize given that most curricula are organized such that ‘assessment drives performance’ (Godfrey 1995). In Asia and many other parts of the world, an exam-oriented focus is often rewarded with more opportunities for career development, progression, and advancement (Li & Li 2010). The results of high-stakes exams during primary and secondary school help shape the trajectory of education the pupil will be eligible for. Examples of these types of exams include the Primary School Leaving Examination in Singapore, the General Certificate of Examination in the United Kingdom, the Stanford Achievement Test (SAT) in the USA, International Baccalaureate (IB) exams, and others. A poor performance at this stage in life can strongly affect the student’s chances of obtaining access to a medical education in the local system. The number of applicants seeking to gain admission to medical school (the ‘‘demand’’) vastly outnumbers the slots available (the ‘‘supply’’). Due to this fact, getting into medical school is a highly competitive process. In countries with medical schools matriculating students from secondary school (aka ‘‘school-leavers’’), getting into medical school can be a major focus for parents and students alike. Most medical
. Stress over peer-competition and exam-focus can prevent medical students from maximizing learning. . In an Asian context in the first year of undergraduate medical education, changing from traditional grades to a Distinction/Pass/Fail grading system resulted in lower stress. . Student knowledge acquisition as measured by exam performance was preserved in the new grading system.
20 14
school admissions processes give significant weightage to academic performance, as evaluated by grades and exam performance (Dunleavy et al. 2011). Therefore, most students who are successful in gaining entry have been highly focused on exam performance, grade achievement, and out-competing their peers. This orientation is usually further reinforced by the secondary school teachers. Schools with more successful medical school applicants can use this statistic to enhance their reputation and desirability in the community. Teachers who successfully train their students to do well on standardized exams often get rewarded based on these metrics (Baker et al. 2010). Despite the intense focus on exams and grade performance as a gating item to entry into medical school and to progression within medical school, some studies show performance on these measures is not well correlated with performance as a doctor (Wingard & Williamson 1973; Taylor & Albo 1993; Probert et al. 2003). Furthermore,
Correspondence: Joshua L. Jacobs, MD, Yong Loo Lin School of Medicine, National University of Singapore, 1E Kent Ridge Road, NUHS Tower Block Level 11 Dean’s Office, Singapore 119228, Singapore. Tel: þ65 6772 3824; fax: þ65 6778 5743; email: [email protected]
164
ISSN 0142–159X print/ISSN 1466–187X online/14/020164–5 ß 2014 Informa UK Ltd. DOI: 10.3109/0142159X.2013.852168
Encouraging lifelong learning
exam-related stress can cause excessive cognitive load that, according to cognitive load theory, can result in less absorption, understanding, or retention of material (Sweller et al. 1998). As a lifelong learning endeavor, medicine is ideally collaborative, not competitive. Healthcare providers work in teams to address the needs of their patients (Hopkins & Baker 2010). Given the nature of their future practice, there is a need to re-orient medical student learners to learning, rather than exam performance. Instead of hiding deficiencies to show a good exam result or course grade, students should be encouraged to partner with their teachers to explore areas of relative weakness in order to improve and ultimately provide better, safer patient care and do better research. In order to minimize exam-orientation in school-leaver students matriculating into medical school from a highly competitive, exam-oriented setting, the National University of Singapore Yong Loo Lin School of Medicine (YLLSoM) switched year 1 grading from traditional grades (A–F) to a grading system of Distinction/Pass/Fail (D/P/F). It was hypothesized that this change would reduce stress among the students overall, and help shift their learning orientation from being exam-focused to more learning-focused. The YLLSoM curriculum is a five-year course of study that is direct-entry from secondary school (i.e. school-leavers) that culminates in the awarding of a Bachelor of Medicine, Bachelor of Surgery (MBBS) degree. The first two years of the course have clinical components but focus primarily on pre and paraclinical sciences basic to medicine. Year 1 focusses on normal structure and function of the body, while year 2 focusses on abnormal structure and function. The remaining three years of teaching and learning are based primarily in clinical settings and focus on the clinical sciences. There is a high-stakes barrier exam at the end of each year of study in addition to periodic summative continual assessments. Upon completing medical school, graduates compete for postgraduate clinical training posts in the various specialties and pursue an undifferentiated year as a house officer, or a specialty-track residency program accredited by the Accreditation Council for Graduate Medical Education International (ACGME-I).
Methods The rationale, proposed workflow and operational implications of switching year 1 to D/P/F grading was discussed over several months with approvals obtained from all stakeholders including appropriate departments, faculty, and university committees. Appropriate ethical approvals for the study were obtained from the Institutional Review Board at the National University of Singapore. The instrument chosen to measure stress was the perceived stress scale 10 (PSS-10; Cohen et al. 1983). As all students are native English speakers, no translation of the scale was needed. In addition to the 10 items that are based on a quantitative scale, three free-text response items were included to gather qualitative data. The 10-item tool has been validated in many settings and cultures, with age and gender norms well established (Malarkey et al. 1995; Lane et al. 2007; Leon et al. 2007; McAlonan et al. 2007).
The free-text response items included were: ‘‘What are the most stressful things for you in the curriculum/learning environment?’’, ‘‘List 3 things that you enjoyed in Phase I [Year 1] learning’’, and ‘‘Do you think grading or ranking leads to unwanted levels of student stress or poor performance? If yes please explain?’’ The survey instrument was administered at the same time point in the curriculum timetable for year 1 students in the academic year before the change in grading (graduating class 2016), and after the change (graduating class 2017). There were no changes to the curriculum for year 1 students from one year to the next. After the change in grading, year 1 students were unable to view the marks performance on continual assessments and final exams. Students who performed at a borderline pass level and those who failed were given individualized feedback and remediation. A two-tailed t-test for equality of means was used to compare exam performance data between the two cohorts. General linear regression was used to compare the Perceived Stress Scale 10 between the two cohorts with adjusting the gender effect. A p-value 50.05 was considered significant. Data analysis was performed using SPSS version 20 (IBM SPSS Statistics, NY).
Results Of the 255 students in the graduating class of 2016 who completed all exams in year 1, a total of 238 (93%) completed the scale. Of the 282 students in the class of 2017 who completed all exams in year 1, a total of 253 (90%) completed the scale. Respondent characteristics are shown in Table 1. Comparison of performance on final examinations (worth 60% of total marks for the year), total marks for the year, and continual assessment (worth 40% of total marks for the year) of the two cohorts of students is shown in Table 2. The results show no significant difference in total marks for the year or in marks on the final exams between the two cohorts. There is a significant difference in marks for the continual assessment component between the cohorts, with the 2017 cohort having higher marks. The mean difference was 0.6 (95% CI 0.1–1.1). The mean (SD) Perceived Stress Scale 10 score of 2016 cohort and 2017 cohort was 18.98 (5.69) and 17.58 (6.07), respectively. The difference was significant ( p ¼ 0.009, two sample t test). Moreover, the mean Perceived Stress Scale 10 score of females (mean 19.37, SD 5.45) was significantly higher than that of males (mean 17.12, SD 6.18) ( p 5 0.0005, two sample t test). General linear model showed there is no significant interaction between cohort and gender ( p ¼ 0.572), hence only main effects of cohort and gender were included into the final model. It showed that cohort 2016 had significantly higher mean PSS 10 score compared with the cohort 2017 ( p ¼ 0.004, mean difference 1.52, 95% CI 0.49– 2.55). Females had significantly higher mean PSS 10 score compared with males ( p 5 0.0005, mean difference 2.33, 95% CI 1.30–3.35) (Table 3). A thematic analysis of the qualitative feedback revealed the main source of stress for the 2016 cohort to be ‘‘competition with peers.’’ Illustrative quotes from respondents include
165
J. L. Jacobs et al.
Comparing my pace of studying with others when I know I shouldn’t and feeling like there is insufficient time to complete all my revision; The number of geniuses present in my cohort and the extreme competiveness; A lot of people are quite competitive so you suddenly feel the need to study like crazy just to keep up. For the 2017 cohort, the main source of stress was ‘‘uncertainty about residency’’ [post-graduate training], as exemplified by these quotes from respondents: ‘‘In today’s society, especially in Singapore, competiveness is cyclonical and it causes unintended levels of stress because everyone is fighting for the limited slots of residency’’; ‘‘The future struggle for limited residency slots. Suddenly, studying is not enough and we will have to do research, hold leadership positions etc. all in a bid to boost our CV to compete with other applicants and overseas colleagues.’’ A subset of the 2017 cohort also asked for ‘‘more granularity’’ to know if they passed or ‘‘just barely’’ passed. Of note, subsequent to the finish of the year 1 curriculum using the D/P/F grading, the medical students of the 2017 cohort submitted results to a poll initiated by students and that Table 1. Characteristics of graduating class year 2016 cohort and graduating class year 2017 cohort.
Completed PSS-10 (% of cohort that completed all exams)
Completed all exams 2016 cohort Male Female 2017 cohort Male Female
255 136 119 282 134 148
238 124 114 253 119 134
(93%) (91%) (96%) (90%) (89%) (91%)
Table 2. Comparison of performance on marks during year 1 for the graduating class year 2016 cohort and graduating class year 2017 cohort.
Final exam marks (of 100) Year-end total marks (of 100) Continual assessment marks (of 40)
Grad class year
N
2016 2017 2016 2017 2016
255 282 255 282 255
71.8 71.82 72 72.08 28.57
2017
282
29.17 (3.03)
Mean (SD) (8.06) (8.11) (7.45) (7.48) (2.88)
p-Value 0.98 0.9 0.02
Bold value is statistically significant ( p 5 0.05).
was not solicited by the school. The poll asked students to support either the statement ‘‘I support the continuation of Pass/Fail for Phase One’’, or the statement ‘‘I do not support the continuation of Pass/Fail for Phase One’’. One hundred seventy-eight of the 280 students (64%) responded to the poll. Of those, 165 of the 178 (93%) supported the continuation, and 136 of the 178 (76%) supported extending this grading system to year 2 of the curriculum.
Discussion With no other changes to the curriculum in year 1, the stress level of the cohort of students that studied under a grading framework of D/P/F was lower than the preceding cohort of students that studied under a traditional grading framework (A–F). Marks for performance on the final exam and total marks for the year were not significantly different between cohorts. There was a significant difference in marks for continual assessment, with better performance in the 2017 cohort. This finding is intriguing, and may warrant further study of the possible link between grade reporting, and test performance. The finding suggests that introducing D/P/F grading does not result in poorer exam performance. Examination of the qualitative comments indicates that the major source of stress for the 2016 cohort was peer competition, presumably from comparing exam performance grades. With the removal of grades, there appears to be a shift away from a peer-competition mindset, with the 2017 cohort requesting to know roughly where they stand in relation to the pass mark, rather than in relation to each other. A direct quote from a 2017 cohort respondent highlights If you don’t want stress or competition, give feedback to ALL students in terms of percentage, marked scripts. Perhaps in a single line ‘‘you are fine’’ or ‘‘you need to buck up". At least something to tell us where our knowledge stands. A better communication campaign may help alleviate the fear of uncertainty. Upon further discussion with students, it became clear that many were not aware that ‘‘no news is good news’’ and that if they were showing areas of weakness they would be approached by the Dean’s Office for remediation. These findings suggest that D/P/F grading may be linked to alleviation of stress among medical students. This alleviation of stress may help them focus on learning, rather than exam-taking. This shift in orientation will help medical schools foster the attributes of lifelong learning. The poll from the medical students of the 2017 cohort further supports the themes gathered from the qualitative comments in the study, and points to the high level
Table 3. Comparison of results on the Perceived Stress Scale 10 for the graduating class year 2016 cohort and graduating class year 2017 cohort.
Grad class year 2016 2017 Gender Male Female
166
N
Unadjusted mean difference (95% CI)
Unadjusted p-value
Adjusted mean difference (95% CI)
Adjusted p-value
238 253
1.40 (0.36–2.45) –
0.009
1.52 (0.49–2.55) –
0.004
243 248
– 2.25 (1.22–3.28)
50.0005
– 2.33 (1.3–3.35)
50.0005
Encouraging lifelong learning
of student acceptability of and desire for this type of grading framework. Stress levels in both cohorts (graduating class year 2016 and 2017) were higher than the reference values for age and gender in the general population (Cohen & Williamson 1988). Additionally, stress as measured by the PSS-10 was higher in females than in males across both cohorts examined. Causes for this were not elucidated in the current study. Strengths of this study include the large cohorts with good response rates, use of a validated instrument, and the addition of qualitative data to aid in interpretation of the quantitative results. To our knowledge, this is the first such English language study of medical student stress in an Asian context. The experience and manifestations of stress may be quite different in cultures with strong Confucian, Islamic, Hindu, and/or Buddhist principles compared to many Western cultures with traditional JudeoChristian value systems (Thang 2004; Aminuddin et al. 2011; Chia 2013). This study is not without its limitations. The cohort study design limits the conclusions that can be definitively drawn from the data. The qualitative comments comport with the conclusion from the quantitative data that the differences between cohorts can largely be explained by the change to the grading system. However, there may have been other contributing factors that influenced the cohort’s experience or expression of stress. For example, the 2016 and 2017 cohorts may have communicated with each other about this topic, with the 2016 cohort giving a message that the D/P/F grading system is ‘‘better’’ or causes less stress than the system they experienced. This may have influenced the 2017 cohort’s perception of stress induced by the grading system, resulting in something akin to a placebo effect. An additional area that was not examined for effect on stress scores was the communication of grades versus a change to the grading system. It is possible that simply masking the traditional letter grades from students would result in a similar decrement in stress scores as changing to the D/P/F system. Although instituting D/P/F in year 1 appears to be linked to lower stress levels, we cannot definitively assume causation. Additionally, the results cannot be generalized to other years of study. Finally, although the results show lower levels of stress and a shift away from peercompetition, they do not demonstrate conclusively a shift away from exam-orientation. The results are encouraging. In the preamble to the unsolicited student poll described in the results section, the students pointed out the benefits of the D/P/F that support the qualitative themes that emerged from the PSS-10 survey. Namely, the preamble cited the benefits of less competition and comparison between students. A direct quote from the preamble states ‘‘We strongly believe that the Pass/Fail system has been and will be greatly beneficial to us in shaping an ideal learning environment where learning is done not for the pursuit of grades but for attaining knowledge so that we may be better doctors in the future.’’
retention (Sweller et al. 1998). To study for a traditional knowledge-based medical school exam, students may adopt a study strategy to load several facts for short-term retrieval on the exam. By removing the high-stakes of the exam, students may adopt a learning strategy that consolidates, integrates and solidifies slightly fewer concepts, but for a deeper understanding and retention. Further study in this area should be done to investigate this effect. In retaining a ‘‘distinction’’ category, peer-competition and exam-focus have not been entirely removed. Homuth has suggested that retaining this category may even defeat the intent of switching to a pass/fail system (Homuth 2008). The effect of retaining the ‘‘distinction’’ category should be further investigated. Measurement of stress in medical students should continue to confirm and extend the findings of the current study. The lower stress levels in the 2017 cohort are encouraging. The negative effects of stress on medical student health has been repeatedly and well documented (Tyssen et al. 2000, 2001; Dyrbye et al. 2006; Voltmer et al. 2012), and measures that reduce stress should be supported. YLLSoM will be evaluating this and other data to determine if the D/P/F scheme should be extended to other years. Based partly on the successful implementation at the medical school, other faculties on the university campus are now considering a similar change to their grading systems. If this occurs, studies of the effects on stress, performance, and exam-orientation should be conducted on the different student cohorts to verify generalizability.
Glossary Stress: A state of mental or emotional strain or tension resulting from adverse or demanding circumstances http://oxforddictionaries.com/definition/english/stress
Notes on contributors All authors work at the National University of Singapore Yong Loo Lin School of Medicine. Dr. JOSHUA L. JACOBS, MD, is the Associate Professor, Department of Medicine, Division of Family Medicine. He serves as Assistant Dean (Education) and is a Senior Consultant in Family Medicine at the National University Hospital. Dr. DUJEEPA D. SAMARASEKERA, MBBS, is the Director, Medical Education Unit. He also consults for the Singapore Ministry of Health on medical education matters. Dr. LIANG SHEN, PhD, is the Manager, Dean’s Office. She is a statistician supporting health services research and other research programs at the school.
Next steps
Dr. K RAJENDRAN, MBBS, is Associate Professor, Department of Anatomy. He is the Phase I (Year 1) Lead.
Keeping in mind cognitive load theory, a shift from exam orientation to learning orientation may enhance learning and
Prof. SHING CHUAN HOOI, MBBS, is the Professor, Department of Physiology. He is the Vice Dean (Education) overseeing all aspects of undergraduate medical education for the school.
167
J. L. Jacobs et al.
Acknowledgements The authors wish to thank the staff of the Dean’s Office and the Phase I Committee at the Yong Loo Lin School of Medicine for their support in this project. Declaration of interest: The authors report no conflicts of interest. The authors alone are responsible for the content and writing of this article. Ethical approval has been granted for the study described in this manuscript by the Institutional Review Board of the National University of Singapore – NUS IRB Code12-492E.
References Aminuddin H, Abiddin NZ, Anuar NAAK. 2011. Viewing the philosophy of education from the perspectives of different schools of thought. Int Rev Soc Sci Human 1(2):55–64. Baker EL, Barton PE, Darling-Hammond L, Haertel E, Ladd HF, Linn RL, Ravitch D, Rothstein R, Shavelson RJ, Shepard LA. 2010. Problems with the use of student test scores to evaluate teachers. Economic Policy Institute. Washington, DC. Briefing Paper No. 278. Chia MO. 2013. Major differences between eastern and western philosophies as the basis for adult education - The Singapore experience. European Association for the Education of Adults. [Accessed 10 February 2013]. Available from http://www.eaea.org/index.php?k=12117. Cohen S, Kamarck T, Mermelstein R. 1983. A global measure of perceived stress. J Health Soc Behav 24(4):385–396. Cohen S, Williamson GM. 1988. Perceived stress in a probability sample of the United States. In: Spacapan S, Oskamp S, editors. The social psychology of health. Newbury Park, CA: Sage. pp 31–67. Dunleavy D, Sondheimer H, Bletzinger R, Castillo-Page L. 2011. Medical school admissions: More than grades and test scores. AAMC Newsletter 11(6). Dyrbye LN, Thomas MR, Shanafelt TD. 2006. Systematic review of depression, anxiety, and other indicators of psychological distress among U.S. and Canadian medical students. Acad Med 81:354–373. Godfrey RC. 1995. Undergraduate examinations – A continuing tyranny. Lancet 765:3. Homuth D. 2008. How honours/pass/fail grading is failing students. UTMJ 85(2):99–102.
168
Hopkins D, Baker PG. 2010. Framework for action on interprofessional education and collaborative practice. Geneva, Switzerland: World Health Organisation Press. Lane JD, Seskevich JE, Pieper CF. 2007. Brief meditation training can improve perceived stress and negative mood. Alternat Ther Health Med 13(1):38–44. Leon KA, Hyre AD, Ompad D, DeSalvo KB, Muntner P. 2007. Perceived stress among a workforce 6 months following hurricane Katrina. Soc Psych Psych Epid 42(12):1005–1011. Li W, Li Y. 2010. An analysis on social and cultural background of the resistance for China’s education reform and academic pressure. IES 3(3):211. Malarkey WB, Pearl DK, Demers LM, Kiecolt-Glaser JK, Glaser R. 1995. Influence of academic stress and season on 24-hour mean concentrations of ACTH, cortisol, and beta-endorphin. Psychoneuroendocrino 20(5):499–508. McAlonan GM, Lee AM, Cheung V, Cheung C, Tsang KW, Sham PC, Chua SE, Wong JG. 2007. Immediate and sustained psychological impact of an emerging infectious disease outbreak on health care workers. Can J Psychiatr 52(4):241–247. Probert CS, Cahill DJ, McCann GL, Ben-Shlomo Y. 2003. Traditional finals and OSCEs in predicting consultant and self-reported clinical skills of PRHOs: A pilot study. Med Educ 37:597–602. Sweller J, Van Merrie¨nboer J, Paas F. 1998. Cognitive architecture and instructional design. Educ Psychol Rev 10(3):251–296. Taylor CW, Albo D. 1993. Measuring and predicting the performances of practicing physicians: An overview of two decades of research at the university of Utah. Acad Med 68(2):S65–S67. Thang SM. 2004. Student approaches to studying: Identifying the Malaysian constructs and comparing them with those in other contexts. J Further Higher Educ 28(4):359–371. Tyssen R, Vaglum P, Gronovold NT, Ekeberg O, Tyssen R. 2000. The impact of job stress and working conditions on mental health problems among junior house officers. A nationwide Norwegian prospective cohort study. Med Educ 34:374–384. Tyssen R, Vaglum P, Gronovold NT, Ekeberg O. 2001. Factors in medical school that predict postgraduate mental health problems in need of treatment. A nationwide and longitudinal study. Med Educ 35:110–120. Voltmer E, Kotter T, Spahn C. 2012. Perceived medical school stress and the development of behavior and experience patterns in German medical students. Med Teach 34(10):840–847. Wingard JR, Williamson JW. 1973. Grades as predictors of physicians’ career performance: An evaluative literature review. J Med Educ 48:311–322.
Copyright of Medical Teacher is the property of Taylor & Francis Ltd and its content may not be copied or emailed to multiple sites or posted to a listserv without the copyright holder's express written permission. However, users may print, download, or email articles for individual use.
