Workforce, Learners, Competencies, and the Learning Environment: Research in Medical Education 2014 and the Way Forward Daniel C. West, MD, Lynne Robins, PhD, and Larry D. Gruppen, PhD
Abstract Medicine in the United States is changing as a result of many factors, including the needs and demands of 21st-century society. In this commentary, the authors review the 2014 Research in Medical Education (RIME) articles in the context of these changes and with an eye toward the future. The authors organized the 12 RIME articles into four broad themes: career development and workforce issues;
The world of medicine is changing
around us. This evolution is driven by many factors, including advances in medicine, an aging U.S. population, changes in the way health care is financed, and the needs and demands of our society.1 In this commentary, we highlight how this year’s Research in Medical Education (RIME) publications reflect responses to these changes. We have organized the publications into four broad themes: (1) career development and workforce issues; (2) competency and assessment; (3) admissions, wellness, and the learning environment; and (4) intended and unintended learning. Career Development and Workforce Issues
With the prospect of a shortage of primary care physicians, especially in Dr. West is professor, Department of Pediatrics, UCSF Benioff Children’s Hospital and University of California, San Francisco, San Francisco, California. Dr. Robins is professor, Departments of Biomedical Informatics and Medical Education, Family Medicine and Pediatric Dentistry, University of Washington, Seattle, Washington. Dr. Gruppen is professor, Department of Learning Health Sciences, University of Michigan, Ann Arbor, Michigan. Correspondence should be addressed to Dr. West, Department of Pediatrics, UCSF, 505 Parnassus Ave., Box 0110, San Francisco, CA 94143-0110; telephone: (415) 476-6136; e-mail: [email protected]
Acad Med. 2014;89:1432–1435. First published online September 23, 2014 doi: 10.1097/ACM.0000000000000504
competency and assessment; admissions, wellness, and the learning environment; and intended and unintended learning. Although the articles represent a broad range of issues, the authors identified three key take-home points from the collection: (1) Schools may be able to address the looming shortage of primary care physicians through admission selection criteria and targeted curricular
activities; (2) better understanding of the competencies required to perform complex physician tasks could lead to more effective ways to teach and assess these tasks; and (3) the intended and unintended learning that take place in the medical learning environment require careful attention in order to produce physicians who are both skilled enough and well enough to meet the needs of society.
rural and underserved areas,2 two RIME articles shed new light on why students choose to pursue primary care careers and to work with underserved populations. Clinite and colleagues3 surveyed nearly 1,000 senior medical students after they completed the Match to determine what factors were important in their career choice. When asked to identify the single most important factor contributing to having a good physician lifestyle, most students chose “enjoying the type of work,” particularly for those choosing primary care fields. “Having enough time off work” and “having control of work schedule” were also highly rated, but “financial compensation” was ranked lowest of all the lifestyle factors. For those choosing primary care, the desire to work with underserved populations was also important. Boscardin and colleagues4 linked the Association of American Medical Colleges Matriculating Student Questionnaire and Graduation Questionnaire to create a longitudinal sample of over 7,000 students to ask what factors reinforce or change student intentions to work with underserved populations. They found that, while personal characteristics were important (e.g., older age, being female, and a member of an underrepresented minority), training experiences did matter. Community field experiences, cultural competence training, learning a second language, and training at a school with a strong social mission were all associated with intention to practice in underserved areas.
These two studies tell us that, in addition to personal characteristics, students’ experiences and perceptions related to primary care and underserved populations are important in promoting primary care careers. The good news is that we have the ability to influence career choice during medical school through purposeful curricular activities. However, it is one thing to match into a primary care residency position—it is another ultimately to practice primary care medicine, especially in an underserved area. Much more work needs to be done to better understand how experiences in residency training can reinforce or undermine graduating medical students’ intentions to enter primary care if we are to successfully address looming shortages. Competency and Assessment
Competency-based medical education has the potential to standardize outcomes and produce physicians who are better prepared to meet the needs of 21stcentury society. Three RIME articles focus on competencies and assessment. Kawamura and colleagues5 and Hubinette and colleagues6 studied two important physician tasks: complex clinical problem solving and health advocacy for patients. Gingerich and colleagues7 examined sources of variance among faculty ratings of clinical competence. Kawamura and colleagues used semistructured interviews of a small sample of expert and novice
Academic Medicine, Vol. 89, No. 11 / November 2014
developmental pediatricians to model how they created case formulations (i.e., explanations of patient problems). They showed that experts developed case formulations collaboratively with parents and other members of the health care team and refined them over time. Interestingly, experts strategically chose the collaborative approach with the expectation that a shared understanding would result in better patient outcomes. Novices were less collaborative and did not recognize the longitudinal and dynamic aspects of case formulation. Hubinette and colleagues interviewed a sample of family physician clinical preceptors and found that clinicians had widely divergent and sometimes conflicting understanding of health advocacy. This finding helps explain why learners have such varied experiences around health advocacy and why this topic is difficult to teach and assess. The authors argue for developing a conceptual model of health advocacy that is explicit, clinically relevant, feasible, and meaningful in practice and that can be used to design curricula and assessments of health advocacy competencies. In the move toward a medical education system where advancement decisions are based on demonstrated competence, assessment by clinical supervisors will play a vital role. Thus, understanding the variability of clinical performance ratings of supervisors becomes very important. It is commonly assumed that different judgments of the same performance reflect rater biases and thus should be treated as error variance. However, Gingerich and colleagues turn this concept upside down. They asked physicians to rate videotaped trainee– patient interactions from a selection of mini-CEX assessments and to respond to a set of questions designed to elicit social judgments of the trainee they observed. They found that a small, defined set of social judgments based on inferred explanations for observed behaviors for the same resident explained a significant amount of variance in performance ratings. This finding suggested that at least some of the “error” variance was systematic and may be measuring behaviors or skills that are relevant to some patient interactions. In other words, some trainees may exhibit behaviors that lead certain raters, but not others, to draw social judgments about the trainee that are meaningful in patient–
physician interactions. Understanding this is important for educators to draw accurate conclusions about this type of observation data. In the end, what really matters is how effectively trainees perform the day-to-day tasks of caring for patients. All three of these articles reinforce what is known about other physician tasks—they represent a complex synthesis of numerous competencies that we do not yet understand well.8 Future research in this area should focus on validating these models and their generalizability to other tasks and settings with the goal of identifying better ways to teach and assess these skills. Admissions, Wellness, and the Learning Environment
Two RIME articles raise compelling questions about the learning environment in medical education and the kinds of information that admissions committees should collect to select medical students who can emerge from medical school equipped to provide compassionate, high-quality care. Brazeau and colleagues9 compared measures of burnout, depression, and quality of life (QOL) indicators in matriculating students at six U.S. medical schools and age-matched college graduates from the general U.S. population and found that matriculating medical students had lower rates of burnout and depression symptoms as well as higher scores on QOL indicators. In other words, medical students begin medical school with better mental health indicators than age-similar college graduates in the general population. This finding is both surprising and alarming, since a study of graduating medical students, residents, and early physicians showed that after medical school, burnout is more prevalent among physicians than in the general population.10 Although Brazeau and colleagues’ study is not longitudinal, it suggests the possibility that the learning environment may be responsible for the increase in burnout, and it raises the provocative question of whether we should admit students with characteristics (e.g., high resilience scores) that might enable them to better weather the physician training process. Instead, perhaps we need to change the
Academic Medicine, Vol. 89, No. 11 / November 2014
training environment to prevent this increase in burnout. Caulfield and colleagues11 assembled a dataset of the responses of over 13,000 entering medical students to the Tolerance for Ambiguity (TFA) scale and used a variety of analytic techniques to establish the scale’s internal structure and internal consistency. Among the more intriguing findings was that lower TFA scores were associated with higher perceived stress levels, and higher TFA scores were associated with a desire to work in an underserved area. Although more research is needed, this finding raises the intriguing and perhaps controversial possibility that admissions committees could use TFA scores to identify applicants who would be more likely to enter primary care fields. Intended and Unintended Learning
The final five RIME articles address our assumptions about intended and unintended learning that happens in our training environment. Each is a reminder of the value of looking at “routine” practices through different cultural lenses and of “making the familiar strange” in order to gain new perspectives on seemingly neutral, taken-for-granted phenomena. A great example of the tension between intended and unintended learning is the article by Ho and colleagues12 in which the authors studied the effects of hospital accreditation on medical students in Taiwan. In this qualitative study, Ho and colleagues describe how “taken-forgranted” administrative requirements of an accreditation process can inadvertently interfere with intended learning and create unintended learning that promotes student cynicism and unprofessional behavior. Interviews with senior clinical-year medical students at all 11 medical schools in Taiwan revealed that, despite some benefits, students identified significant negative effects of the accreditation process, including decreased clinical learning and increased trivial workload. Alarmingly, students reported being asked to participate in forging documents and otherwise creating an idealized “performance instead of a reflection of reality.” In another example, Cristancho and colleagues13 used a combination of interviews, observations, and drawing
sessions to investigate surgeons’ perspectives on the complexities of the systems in which they practice. The visual perspectives provided unique insight into the nonprocedural complexities of surgical practice, while the interviews were dominated by descriptions of the more familiar procedural challenges. The novel use of drawings allowed for a more robust understanding of the learning opportunities in the surgical environment and supports the expansion of intended learning in team dynamics, trust, emotions, and external pressures. The observational study by Satterfield and colleagues14 identified missed opportunities to teach “soft” topics in social and behavioral science (SBS) that impact “hard” clinical outcomes. They observed bedside teaching on internal medicine and pediatric inpatient services to determine how frequently SBS topics were discussed and how the team responded to this. SBS topics arose in nearly every patient encounter, and it is important to note that the number of SBS topics raised was strongly associated with the patient-centeredness of the team. SBS topics were more likely to be addressed if teams had sufficient time per patient, if they had a service-appropriate team size, and if they paid careful attention to more patient-centered interactions. Holtzman and colleagues’15 international comparisons of USMLE Step 2 Clinical Knowledge exam scores challenge our assumptions about whether what gets taught or omitted in medical school is value-free or context-neutral. The authors compared subscores from international medical graduate (IMG) first-time exam takers over three years with scores from graduates of U.S. medical schools and found that IMGs performed best in Surgery and Understanding Mechanisms of Disease, and worst in Psychiatry and Promoting Preventive Medicine & Health Maintenance. The strength of this pattern varied with the region of the world, and, seen through a global lens, the pattern suggests that cultural or health system differences might influence intended learning. On a practical level, U.S. residency programs that train international graduates could use this information to inform their curricula. The final article by Webb and colleagues16 was based on the broad consensus that effective leadership training for
physicians is vital in our changing health care system. To better understand how best to provide such training, they conducted a systematic review of the published medical literature on leadership training at the undergraduate medical education level. They found that leadership curricula focused on a wide range of competency markers but were usually not aligned with established leadership competency frameworks. Additionally, most published curricula received low scores on effectiveness and quality of evidence. The bottom line is that leadership curricula should be better aligned with existing leadership frameworks and should take advantage of many areas of medical school curricula that, with a small frame shift, provide opportunities for leadership training. The key to integrating leadership training into medical school curricula is to take advantage of overlap with existing curricular content. RIME Wrap-up 2014: Putting It All Together
This year’s RIME articles offer new information about a broad range of issues that relate to our changing health care system and what society needs from their future physicians. We identified three key take-home points from this collection of articles: 1. We may be able to address the looming shortage of primary care physicians through admission selection criteria and targeted curricular activities. 2. Tasks that physicians need to perform on a day-to-day basis are complex and involve the integration of multiple competencies. Better understanding of the competency frameworks that define these tasks could allow us to teach and assess them better. 3. The medical learning environment is rich with learning opportunities— intended and unintended. We need to tend carefully to this environment to ensure that medical students emerge not only with skills they need for practice but also with the personal well-being required to provide the compassionate, high-quality care that patients need. Acknowledgments: The authors thank the members and staff of the Research in Medical Education (RIME) Planning Committee for
their review of the papers discussed in this manuscript. RIME Planning Committee members include Maryellen E. Gusic, MD (chair), Liselotte N. Dyrbye, MD (past chair), Anthony R. Artino, Jr, PhD, John Boulet, PhD, Larry D. Gruppen, PhD, Monica L. Lypson, MD, MHPE, Kathleen Mazor, EdD, Lynne S. Robins, PhD, and Daniel C. West, MD. Staff members include Katherine McOwen, MSEd, and Jada Greene. Funding/Support: None reported. Other disclosures: None reported. Ethical approval: Reported as not applicable.
References 1 Berwick DM, Finkelstein JA. Preparing medical students for the continual improvement of health and health care: Abraham Flexner and the new “public interest.” Acad Med. 2010; 85(9 suppl):S56–S65. 2 Association of American Medical Colleges. AAMC Physician Workforce Policy Recommendations. https://www.aamc.org/ download/304026/data/2012aamcworkforce policyrecommendations. Accessed August 8, 2014. 3 Clinite KL, DeZee KJ, Durning SJ, et al. Lifestyle factors and primary care specialty selection: Comparing 2012–2013 graduating and matriculating medical students’ thoughts on specialty lifestyle. Acad Med. 2014;89:1483–1489. 4 Boscardin CK, Grbic D, Grumbach K, O’Sullivan P. Educational and individual factors associated with positive change in and reaffirmation of medical students’ intention to practice in underserved areas. Acad Med. 2014;89:1490–1496. 5 Kawamura AA, Orsino A, Mylopoulos M. Integrating competencies: Exploring complex problem solving through case formulation in developmental pediatrics. Acad Med. 2014;89:1497–1501. 6 Hubinette MM, Ajjawi R, Dharamsi S. Family physician preceptors’ conceptualizations of health advocacy: Implications for medical education. Acad Med. 2014;89:1502–1509. 7 Gingerich A, van der Vleuten CPM, Eva KW, Regehr G. More consensus than idiosyncrasy: Categorizing social judgments to examine variability in mini-CEX ratings. Acad Med. 2014;89:1510–1519. 8 ten Cate O, Scheele F. Competency-based postgraduate training: Can we bridge the gap between theory and clinical practice? Acad Med. 2007;82:542–547. 9 Brazeau CMLR, Shanafelt T, Durning SJ, et al. Distress among matriculating medical students relative to the general population. Acad Med. 2014;89:1520–1525. 10 Dyrbye LN, West CP, Satele D, et al. Burnout among U.S. medical students, residents, and early career physicians relative to the general U.S. population. Acad Med. 2014;89:443–451. 11 Caulfield M, Andolsek K, Grbic D, Roskovensky L. Ambiguity tolerance of students matriculating to U.S. medical schools. Acad Med. 2014;89:1526–1532. 12 Ho MJ, Chang HH, Chiu YT, Norris JL. Effects of hospital accreditation on medical
Academic Medicine, Vol. 89, No. 11 / November 2014
Commentary students: A national qualitative study in Taiwan. Acad Med. 2014;89:1533–1539. 13 Cristancho SM, Bidinosti SJ, Lingard LA, Novick RJ, Ott MC, Forbes TL. What’s behind the scenes? Exploring the unspoken dimensions of complex and challenging surgical situations. Acad Med. 2014;89: 1540–1547.
14 Satterfield JM, Bereknyei S, Hilton JF, et al. The prevalence of social and behavioral topics and related educational opportunities during attending rounds. Acad Med. 2014;89:1548–1557. 15 Holtzman KZ, Swanson DB, Ouyang W, Dillon GF, Boulet JR. International variation in performance by clinical discipline and
task on the United States Medical Licensing Examination Step 2 Clinical Knowledge component. Acad Med. 2014;89:1558–1562. 16 Webb AMB, Tsipis NE, McLellan TR. A first step toward understanding best practices in leadership training in undergraduate medical education: A systematic review. Acad Med. 2014;89:1563–1570.
Cover Art Artist’s Statement: Med Students Since 2009, I have worked at the Albany Medical Center (AMC) as a standardized patient. It has been a great part-time job to help subsidize my income as a freelance artist and designer. Working with the students and other standardized patients has been a wonderful experience. Part of what makes it a great job for me is being able to sketch on breaks or while observing student–patient encounters. The state-of-the-art facilities at AMC have two cameras per room, offering a great vantage point for observing students and patients participating in encounters. Although most encounters are under 15 minutes, there are 5 to 8 minutes where the student asks the patient questions about his or her medical history, including history of present illness, past medical history, and
family history. During that time, I am able to quickly scribble down a likeness of the student with pencil on paper while viewing the encounter on a computer monitor in a separate viewing room.
Academic Medicine, Vol. 89, No. 11 / November 2014
Although all of the students wear a standard lab coat and dress professionally, I am often struck by how individual they appear. It may be their posture on the stool, how they hold the clipboard and pen for jotting down notes, or it may be each student’s stylish accessories, such as a pair of pointy shoes or glasses that set them apart from each other. I’ve accumulated a number of these drawings and have turned some of them into finished paintings that hang in the standardized patient break room at AMC. One of these paintings appears on the cover of this issue. Mark Gregory Mr. Gregory is an artist living and working in Albany, New York. For more information, visit www. markgregoryart.com; [email protected]