The Journal of Emergency Medicine, Vol. 46, No. 5, pp. 701–705, 2014 Copyright Ó 2014 Elsevier Inc. Printed in the USA. All rights reserved 0736-4679/$ - see front matter

http://dx.doi.org/10.1016/j.jemermed.2013.08.132

Education DEVELOPING A CURRICULUM FOR EMERGENCY MEDICINE RESIDENCY ORIENTATION PROGRAMS Raymond Lucas, MD,* Tina Choudhri, MD,* Colleen Roche, MD,* Claudia Ranniger, MD,* and Larrie Greenberg, MD† *Department of Emergency Medicine, The George Washington University, Washington, District of Columbia and †Department of Pediatrics, The George Washington University and Children’s National Medical Center, Washington, District of Columbia Reprint Address: Raymond Lucas, MD, Department of Emergency Medicine, George Washington University, 2120 L Street NW, Suite 450, Washington, DC 20037

, Abstract—Background: New residents enter emergency medicine (EM) residency programs with varying EM experiences, which makes residency orientation programs challenging to design. There is a paucity of literature to support best practices. Objective: We report on a curriculum development project for EM residency orientation using the Kern Model. Curriculum: Components of the revised curriculum include administrative inculcation into the program; delivering skills and knowledge training to ensure an entering level of competence; setting expectations for learning in the overall residency curriculum; performing an introductory performance evaluation; and socialization into the program. Results: Post-implementation resident surveys found the new curriculum to be helpful in preparing them for the first year of training. Conclusions: The Kern Model was a relevant and useful method for redesigning a new-resident orientation curriculum. Ó 2014 Elsevier Inc.

to EM in medical school, the heterogeneity of their medical school EM experience potentially yields an incoming class with a wide variety of skills and knowledge across all the core competencies. In a recent survey of EM clerkship directors, less than half of EM clerkship directors required students to perform basic procedures, and some reported as little as 6 h of lectures and 2 h of labs during the entire rotation (1). Only 7% of respondents reported using an objective standardized clinical evaluation (OSCE), even though this has become a standard evaluation method in many residency programs. This lack of a standardized curriculum and experience before residency makes preparing new EM interns for clinical responsibilities challenging. At the residency level, many EM programs provide an orientation program (unpublished data, review of posted online residency curricula by study authors). Brillman et al. surveyed EM program directors on the prevalence and composition of their orientation program (2). Most consisted of a combination of didactic instruction, emergency department (ED) clinical work, and certificate courses, such as Advanced Cardiac Life Support (ACLS). The mean number of didactic instruction hours was 98 and the mean duration of orientation was 2.8 weeks. However, the duration, content, and goals among EM residency orientation programs varied greatly across responding programs.

, Keywords—emergency medicine residency; orientation; curriculum development

INTRODUCTION The transition from medical school to residency has the potential to be difficult, and is especially so for emergency medicine (EM) residents, given the wide range of EM experiences that exist in medical schools. Although incoming residents typically have had significant exposure

RECEIVED: 12 July 2013; FINAL SUBMISSION RECEIVED: 19 August 2013; ACCEPTED: 27 August 2013 701

702

R. Lucas et al.

In our institution, we previously developed a 4-week orientation program consisting of didactic instruction, ED clinical shifts with resident mentors, special courses such as ACLS and a 2-day ultrasound course, and procedural skills labs. The curriculum was developed a priori by the program director, faculty, and chief residents. Based on feedback from our residents and faculty over time, we recognized the importance of revising our orientation curriculum. The purpose of this article is to report that curriculum development process in the IDCRD (introduction, development, curriculum, results, and discussion) format and to offer a suggested EM orientation curriculum for use by others (3). This project was reviewed and approved for exemption by the Institutional Review Board at The George Washington University. DEVELOPMENT PROCESS We based our process to revise our residency orientation curriculum on the Kern Model of curriculum development (4). This process involves six steps: problem identification and general needs assessment, needs assessment for targeted learners, establishing goals and objectives, selecting educational strategies, implementation, and evaluation and feedback. A focus group of key educational faculty and chief residents met to define the general problems associated with integration of new interns into the program, and to develop an integration needs assessment that the orientation program should address (5). The focus group identified skills, experiences, and expectations that were best suited for an EM-focused intern orientation, and excluded those already covered in the general orientation for incoming residents given by the graduate medical education office at our institution, and those that could be delayed until later in residency training. To complement the findings of our focus group, a more formalized needs assessment was undertaken with both the faculty and current residents to target the learners’ needs. The faculty participated in an anonymous electronic survey asking their opinions on specific skills and knowledge areas they believed an intern should possess early in their training. Likewise, current residents were surveyed with a copy of the existing orientation curriculum, asking them to evaluate which portions they found helpful as they progressed through training, and what types of content delivery format they preferred (lecture, laboratory, simulation, small group work, etc.). Lastly, the residency leadership, with knowledge of past intern performance and evaluations, identified specific competency domains that should be addressed during intern orientation. The skills most highly rated by faculty as important for interns to possess early in training included effective

communication, introductory-level procedural competence, interpreting electrocardiograms (ECG) and chest x-rays, developing a differential diagnosis and early management plan for common ED chief complaints, finding answers to clinical questions quickly, prioritizing tasks and workflow in the ED, and dealing with shift work. Based on resident feedback, most content areas covered in the existing orientation were believed to be important, however, trainees strongly preferred skills labs, simulation sessions, and small-group learning over lectures. Residents also emphasized the need for orientation to make them comfortable and ‘‘functional’’ in the ED, to counteract the stress of learning their new job. The residency leadership recognized that the recent postgraduate year 1 (PGY-1) classes had difficulty integrating a reading plan and independent study into their learning. This process confirmed the need for a robust orientation program to prepare interns with diverse prior EM experience for clinical practice, and provided a framework of orientation goals: 1. To ensure a minimum level of knowledge for common ED chief complaints (i.e., chest pain) and their initial management. 2. To provide a general approach to ED procedures and introductory-level training in common procedures. 3. To provide sufficient skills training for interns independently to institute basic life-saving interventions, while awaiting assistance from a supervising physician. 4. To teach the use of evidence-based medicine (EBM) skills in the clinical setting, and to support patient-centered, self-directed learning very early in training. 5. To set expectations for group and self-directed learning and describe the evaluation process applied during residency. 6. To socialize the interns into the EM community at our hospital. 7. To provide feedback on each learner’s current level of EM skills and knowledge and use as a basis to create an individualized learning plan. NEW CURRICULUM The newly developed orientation curriculum was crafted to address the goals using appropriate strategies: 1. Administrative orientation sessions: With the goal of improving intern functionality, these sessions include tours of the clinical sites, covering the ‘‘where, what, and how’’ issues of each location, identification badging, and credentialing. Mandatory training in the use of the electronic medical

Developing a Curriculum for EM Residency Orientation Programs

2.

3.

4.

5.

6.

record is included. During an introductory session, the overall curriculum and learning objectives of the residency are reviewed. Assessment of incoming interns’ knowledge, skills, and learning styles: New residents are evaluated using an internally developed general EM medical knowledge examination, the Fresno test of EBM skills, the Kolb Learning Style Inventory, and two OSCEs with standardized patients (6,7). Each resident meets with the program director to review the results and form an individualized learning plan. The use of semi-annual reviews to reinforce adherence to learning and reading plans is emphasized. Simulation sessions: The interns manage standardized simulation cases in small groups to reinforce knowledge and management of common ED chief complaints, including chest pain, abdominal pain, trauma, undifferentiated shock, neurological complaints, and vaginal bleeding. These casebased learning sessions are coupled with EBM discussions on the same topic, detailed here later. Didactic sessions: Medical topics believed to be important, but not covered in enough depth in the simulation sessions, are addressed in smallgroup, case-based, discussion sessions. Topics include basics of ECG interpretation, chest x-ray interpretation, principles of acid/base balance, pediatric fever, how to interact with consultants, and end-of-life issues. Skills workshops: Residents must complete a comprehensive central-line training program previously developed by a multidisciplinary group in our hospital, and are subsequently expected to perform an ultrasound-guided internal jugular line on a mannequin as per a hospital-developed checklist. Residents also participate in procedure labs for suturing, abscess drainage, splinting, lumbar puncture, endotracheal intubation, procedural sedation, and slit-lamp examination of the eye. These other skills labs are formative and are considered introductory-level learning experiences. EBM: After a 1-h orientation to the medical library by a reference librarian, residents receive an EBM workshop designed specifically to teach how to form PICO (population, intervention, comparison, outcome) questions, develop search strategies, and learn to find clinically relevant information in real time. At the end of the workshop, each intern is given a clinical case with specific management questions and is assigned to use their newly taught skills to find evidence-based answers. The cases and questions are similar in

703

7.

8.

9.

10.

content area to cases managed in the simulation lab during orientation. To model and reinforce the integration of EBM into clinical practice, during the debriefing of each simulation case, an intern will review their assigned questions, share their search strategy, and offer ways that the EBM information they learned can be used to guide care in similar cases. Special courses: Interns receive a 16-h introductory ultrasound course with both lecture and hands-on training components. An ultrasound skills evaluation is conducted later in the curriculum. An ACLS course is provided with the expectation that all interns become certified. Reading plan: The program director leads a reading plan workshop. After a discussion on the importance of reading and independent learning in residency, common ED textbooks and other learning resources, such as the EM Council of Residency Directors Test Bank, are reviewed. A reading plan is provided and expectations for reading goals (e.g., completion of a standard EM textbook in the first year of training) are explicitly stated. Clinical shifts: During the 4 weeks of orientation, new residents work eight clinical shifts in the ED. The first three shifts are ‘‘teaching shifts,’’ in which each intern is paired with an upper-level resident who provides close observation, teaches ED logistics, patient-flow management, and supports the new electronic medical record user. Social events: The department sponsors an off-site team-building activity for all residents during the orientation month, and provides both resident and faculty-sponsored social events for new interns and their immediate families to foster social ties.

Novel Curricular Components There are two components of our orientation that we believe are novel innovations in EM residency orientation. The first is assessing baseline knowledge and skills as part of the orientation process. Our primary intent was to use the information to help develop a customized, individual learning plan for each new resident. Our prior experience suggests that many interns are unsure of how to approach learning in residency, and often do not get any meaningful feedback with which to focus their learning efforts until several months into residency. By getting timely feedback on their medical knowledge and clinical performance scores on the OSCEs, the interns have objective information with which to structure their learning effort at the outset of training. In addition, the Kolb learning style inventory enables our interns to

704

R. Lucas et al.

Table 1. Resident Rating of Usefulness of Orientation Program Components Baseline Skills and Knowledge Tests

Mean Score*

Medical knowledge test Fresno test OSCE with standardized patients Kolb Learning Style Inventory Follow-up meeting with PD/ILP Session on developing a reading plan Lectures Calling consultants ED patient flow Acid base Assessing the ill child ECG interpretation CXR interpretation Training sessions/labs ACLS course Ultrasound course Central-line training Lumbar puncture Airway Suture lab Simulation cases/patient care scenarios Other Intro to EBM EBM questions/review ED clinical teaching shifts

2.9 3.4 3.9 3.1 3.8 3.9 4.3 2.6 4.5 4.3 4.6 4.5 4.7 5.0 4.6 4.3 4.6 3.9 4.8 3.8 4.1 4.5

ACLS = Advanced Cardiac Life Support; CXR = chest x-ray; EBM = evidence-based medicine; ECG = electrocardiogram; ED = emergency department; ILP = individualized learning plan; OSCE = objective standardized clinical evaluation; PD = program director. * 1 = not helpful to 5 = very helpful.

find strategies that best suit their own learning style. We believe this comprehensive knowledge gap and learning styles assessment will lead to improved performance. Secondary benefits of instituting early evaluation are to set expectations for evaluation in residency, and to emphasize the importance of using feedback for reflection and self-improvement. Lastly, on-arrival testing can identify residents who have considerable knowledge and skills deficiencies early in their training, before the first semi-annual review, and enable early remediation. This concept is consistent with work done by others who demonstrated that EM intern performance on an OSCE in the first month of training correlates with future resident performance (8). The second novel component of our curriculum is the integration of EBM skills with small-group learning and active case management in the simulation lab. Our intent is to teach clinically relevant EBM skills and reinforce the link between EBM and clinical care early in training. This differs from the typical approach of teaching EBM through journal club and other conferences, which are removed from an immediate clinical context (9). The ‘‘application of research findings to individual patients’’ was the most highly ranked EBM skill for emergency physicians, as reported in a recent survey of EM faculty

and we believe this approach reinforces that skill (9). To achieve this, at the end of a formal EBM workshop each intern is given a clinical question that is linked to a case-based problem. An example question is ‘‘For patients with pleuritic chest pain, when is a D-dimer useful? What is its sensitivity and specificity for pulmonary embolism?’’ At a later point in the simulation lab, when the case for evaluating of a patient with chest pain is completed, the intern presents an EBM answer, including the search strategy used to find the main references. Each intern also prepares a one-page handout to share with their classmates, so that by the end of orientation each intern has evidence-based information on how to approach 10 common clinical scenarios seen in the ED. RESULTS We evaluated the success of the new curriculum with a survey of our PGY-1 residents at the end of the academic year, with instructions to reflect on how they now felt each item in the orientation prepared them for the first year of training. A survey outlining the basic components of the orientation curriculum was administered utilizing a 5-point Likert scale ranging from ‘‘not helpful’’ to ‘‘extremely helpful.’’ These surveys were completed anonymously by all 11 PGY-1 residents. Mean score for the components of the orientation program are listed in Table 1. The most highly rated components were the simulation lab sessions, the ultrasound course, many of the lecture and procedure labs, and the teaching shifts in the ED. Feedback on the baseline assessment of their skills and knowledge was mixed. The interns believed the OSCEs with feedback were helpful, yet they did not rate taking a written examination as helpful. Most found the follow-up session with the program director to review the results and an individualized learning plan to be useful. Given the low number of participants, we performed no statistical analyses on the data. DISCUSSION The Kern Model of curriculum development was both relevant and useful for revising our orientation program. In previous years, orientation had been planned by chief residents who relied on what had ‘‘always been done’’ over the years without a formal problem statement or needs assessment (4). The more formal curriculum development approach allowed us to deliver an orientation that addresses the needs of our new learners and is more thoughtfully integrated with later portions of the general residency curriculum. For example, the simulation sessions and lectures give some ‘‘just in time’’ training for entry-level clinical competence of learners from varied

Developing a Curriculum for EM Residency Orientation Programs

backgrounds, and the ultrasound and procedural training is done at an introductory level and sets the stage for the ultrasound rotation later in the year and for our ongoing procedural competency program. From a resource perspective, the Kern Model allowed us to work largely with existing educational and evaluation resources. The needs assessment relied heavily on existing data from our ongoing residency evaluation process and required one additional survey of the residents and faculty (4). With regard to content, some of the existing orientation lectures, simulation sessions, skills workshops, and special courses were retained and modified to more closely address the identified learning needs, rather than developed de novo. We did, however spend considerable time developing our process for assessing incoming knowledge and skills for each new intern and on the EBM components of the orientation. We have used resident feedback as the primary way to evaluate our orientation curriculum and it was largely positive. The administrative aspects, clinical teaching shifts, small-group learning, and EBM teaching received positive reviews and were left largely intact. Although the residents did not assess ‘‘taking a medical knowledge test’’ to be very useful, the program leadership feels that this approach best assesses baseline fund of knowledge and contributes to development of individualized learning plans. We have continued all components of the baseline testing part of the orientation curriculum, yet reframed the testing experience to our interns as a means to help them target specific content areas of their self-directed learning rather than focusing on their overall score and how they compared to the group mean. Feedback on this change is pending. Although the specifics of our curriculum described here may be unique to our institution, the curriculum development process has identified important components of residency orientation that may serve as a useful framework for others. Those components are: 1. Administrative inculcation into the program with regards to policies, clinical logistics, credentialing, information technology training, etc. 2. Delivering key medical knowledge and skills training to ensure that all new residents are at a baseline level of competence in their new role. 3. Reviewing the overall curriculum and learning objectives and setting expectations for self-learning, self-assessment, and program evaluation at the beginning of residency

705

4. Introducing the new resident into the social fabric of the program. 5. Introducing performance evaluations at the beginning of the program. These evaluations serve the dual role of identifying special-needs learners early in residency and, when paired with a structure that teaches learning plan development, improves resident self-assessment, reflection, and self-directed learning. CONCLUSIONS There is a paucity of literature on residency orientation programs in general, and little to none related to EM since the Brillman et al. article published in 1995 (2). Residency orientation programs remain important as a means to transition new residents from various backgrounds into the complex and challenging residency learning environment. New resident orientation has the potential to be a defining activity early in residency and can set the stage for the success of the individual and the program if done well. This report describes our successful use of a curriculum development process to plan residency orientation and offers a model template for use by other residency programs. REFERENCES 1. Wald DA, Manthey DE, Kruus L, Trip M, Barrett J, Amoroso B. The state of the clerkship: a survey of emergency medicine clerkship directors. Acad Emerg Med 2007;14:629–34. 2. Brillman JC, Sklar DP, Viccellio P. Characteristics of emergency medicine resident orientation programs. Acad Emerg Med 1995;2: 225–31. 3. Reznich CB, Anderson WA. A suggested outline for writing curriculum development journal articles: the IDCRD format. Teach Learn Med 2001;13:4–8. 4. Kern DE, Thomas PA, Hughes MT. Curriculum development for medical education: a six step approach. 2nd ed. Baltimore, MD: The Johns Hopkins University Press; 2009. 5. Grant J. Learning needs assessment: assessing the need. Br Med J 2002;324:156–9. 6. Ramos KD, Schaefer S, Tracz SM. Validation of the Fresno test of competence in evidence based medicine. Br Med J 2003;326:319–21. 7. Kolb DA. Kolb Learning style inventory. Boston, MA: The Hay Group; 1999. 8. Wallenstein J, Heron S, Santen S, Shayne P, Ander D. A CoreCompetency Based Objective Structured Clinical Examination (OSCE) can predict future resident performance. Acad Emerg Med 2010;17(Suppl. 2):S67–71. 9. Carpenter C, Kane B, Carter M, Lucas R, Wilbur L, Graffeo C. Incorporating evidence-based medicine into resident education: a CORD survey of faculty and resident expectations. Acad Emerg Med 2012;17(Suppl. 2):S54–61.

Developing a curriculum for emergency medicine residency orientation programs.

New residents enter emergency medicine (EM) residency programs with varying EM experiences, which makes residency orientation programs challenging to ...
102KB Sizes 1 Downloads 0 Views