Bedside Ultrasound Curriculum for Medical Students: Report of a Blended Learning Curriculum Implementation and Validation Uch e Blackstock, MD, RDMS, Jaclyn Munson, BA, Demian Szyld, MD, EdM NYU School of Medicine/Bellevue Hospital Center, Department of Emergency Medicine, 462 First Avenue, OBVA349A, NY, New York 10016 Received 21 June 2013; accepted 22 July 2014
ABSTRACT: Background. Medical students on clinical rotations rarely receive formal bedside ultrasound (BUS) training. We designed, implemented, and evaluated a standardized BUS curriculum for medical students on their Emergency Medicine (EM) rotation. Teaching was aimed toward influencing four cognitive and psychomotor learning domains: BUS instrumentation knowledge, image interpretation, image acquisition, and procedural guidance. Methods. Participants viewed three instructional Web-based tutorials on BUS instrumentation, the Focused Assessment for Sonography in Trauma (FAST) examination and ultrasound-guided central venous catheter (CVC) placement. Subsequently, participants attended a 3-hour hands-on training session to discuss the same content area and practice with faculty coaches. A Web-based, multiple-choice questionnaire was administered before and after the session. During the final week of the rotation, students returned for skills assessments on FAST image acquisition and CVC placement. Results. Forty-five medical students on an EM rotation were enrolled. Sonographic knowledge overall mean score improved significantly from 66.6% (SD 611.2) to 85.7% (SD 610.0), corresponding to a mean difference of 19.1% (95% CI 15.5–22.7; p < 0.001). There were high pass rates for FAST (89.0%, 40/45) and CVC (96.0%, 43/45) skills assessments. There was no significant difference between medical student posttest and EM resident test scores 85.7% (SD 610.0) and 88.1% (SD 6 7.6) (p 5 0.40), respectively. Conclusions. A formal BUS curriculum for medical students on EM rotation positively influenced performance in several key learning domains. As BUS competency is required for residency in EM and other Correspondence to: U. Blackstock C 2014 Wiley Periodicals, Inc. V
VOL. 43, NO. 3, MARCH/APRIL 2015
specialties, medical schools could consider routinely incorporating BUS teaching into their clinical rotation C 2014 Wiley Periodicals, Inc. J Clin Ultracurricula. V sound 43:139–144, 2015; Published online in Wiley Online Library (wileyonlinelibrary.com). DOI: 10.1002/ jcu.22224 Keywords: bedside ultrasound; undergraduate medical education; curriculum development
INTRODUCTION
E
mergency Medicine (EM) physicians, surgeons, and intensivists are expected to accurately perform bedside ultrasound (BUS) for diagnostic and therapeutic purposes in order to maintain the highest quality of patient care and safety.1 The Focused Assessment for Sonography in Trauma (FAST) examination, to assess for internal bleeding in trauma patients, and ultrasound-guided central venous catheter (CVC) placement, are among two of the core EM BUS applications.2–10 BUS has become such an integral part of EM practice that EM residency programs require BUS training for graduation.11 Despite the prevalent clinical use of BUS, medical students rarely receive formal BUS hands-on instruction while on clinical rotations.12 There are limited but increasing reports in the literature of formal BUS during preclinical and clinical training.13–21 The lack of formal BUS instruction in many medical school curricula, despite its increasing use in multiple specialties, underscores the need to create 139
BLACKSTOCK ET AL
opportunities to expose medical students to BUS and train them in the fundamentals and basic applications prior to beginning residency training. The American Board of Emergency Medicine and the Accreditation Council for Graduate Medical Education of Graduate jointly released EM milestones in 2012; two of them call for evaluation of ultrasound knowledge and skill (“Goal-Directed Focused Ultrasound” and “Vascular Access”).22 We designed, implemented, and evaluated a 4-week-long, formal curriculum for medical students during the EM rotation consisting of narrated Web-based tutorials and hands-on BUS training developed to influence four cognitive and psychomotor learning domains: (1) BUS instrumentation knowledge; (2) image interpretation; (3) image acquisition; and (4) procedural guidance. Our study goes further than prior studies by implementing a formal, blended curriculum within the EM rotation, assessing multiple learning domains and comparing students’ abilities to that of EM residents. It is expected the outcomes of this study will help inform the implementation of formal BUS curriculums during EM and other clinical rotations in order to better address medical students’ training needs.
MATERIALS AND METHODS
We prospectively enrolled a convenience sample of medical students on their EM rotation at an urban tertiary medical center from January 2012 to June 2012. The medical center’s Institutional Review Board approved the study. Verbal informed consent was obtained from all participants. Medical students participating in the 4-week EM elective clinical rotation were enrolled. Students can take the elective during their clerkship year (after basic sciences) or during their final year of medical school. Clerkship-level students were expected to function at the level of a medical student and the fourth-year students were expected to function more independently (at the level of a new intern) in the clinical setting. Sample Size Calculation The primary outcome of the study was defined as an increase in BUS instrumentation knowledge and image interpretation ability after the educational intervention. We calculated that 14 subjects are required to have a 80% chance of detecting, as significant at the 5% level, an 140
increase in the primary outcome measure from a mean score of 70 in the preintervention group to a mean score of 85 in the postintervention group using a paired design with an estimated SD of the outcome of 10% using a two-tailed paired design analysis. Pretest Assessment Prior to the educational intervention, all students completed a 43-item Web-based multiplechoice questionnaire, which contained items assessing baseline demographics (seven items), confidence with BUS (three items), followed by 33 multiple-choice questions assessing knowledge of BUS instrumentation (17/33) and image interpretation (16/33). The image interpretation questions consisted of normal and pathologic FAST images, as well as normal neck vascular ultrasound anatomy images. Students were asked to identify anatomic structures and pathology in still images and video clips. Educational Intervention The blended curriculum consisted of narrated Web-based video tutorials and hands-on instructional sessions. All students viewed three Web-based tutorials at their own pace, covering (1) ultrasound physics and instrumentation (5 minutes), (2) the FAST examination (20 minutes), and (3) ultrasound-guided CVC placement (15 minutes). Next, students were randomly assigned and divided into three groups for their hands-on instructional sessions. In order to maintain a low student-to-faculty ratio, sessions were held during the first, second, and third weeks of the rotation and two to three students attended each session. Each student attended one 3-hour BUS instructional session, led by an EM BUS faculty, where they applied concepts and techniques discussed during the Web-based tutorials. Faculty demonstrated correct technique, reviewed concepts, coached, and gave feedback to the students. Curriculum was standardized to ensure that during this session, students learned how use the ultrasound machine, how to perform the FAST examination on each other, and how to cannulate the vein on an internal jugular vein task trainer using ultrasound guidance (CentralLineMan System; Simulab Corporation, Seattle, WA). All students received identical instruction from the BUS instructor regardless of which session the student attended. The ultrasound equipment used during the instructional session was the Sonosite JOURNAL OF CLINICAL ULTRASOUND
BEDSIDE US CURRICULUM FOR MEDICAL STUDENTS
M-Turbo (Sonosite, Inc., Bothell, WA) equipped with 5–1-MHz and 10–5-Hz transducers.
residency program soliciting anonymous participation in the study.
Posttest Assessment Subsequent to the instructional session, students completed a 49-item Web-based multiplechoice questionnaire, containing 13 questions asking them about their perception of the utility of the training and technique, as well as confidence in applying BUS use, 33 questions assessing their knowledge of BUS instrumentation and image interpretation questions (same as pretest), and 3 closing questions regarding interest in additional ultrasound education. Psychomotor skills were formally assessed during the last week of using trained raters (1 to 3 weeks after the initial session). Students were asked to perform the FAST examination on a healthy male volunteer and ultrasoundguided CVC placement on an internal jugular task trainer. Student psychomotor skills during performance of the FAST examination and ultrasoundguided CVC placement were assessed using behaviorally specific standardized checklists. The FAST examination checklist was developed using American College of Emergency Physicians imaging criteria, and the CVC checklist was developed based on a literature review of previously validated skills checklists.23–25 The FAST examination checklist contained items assessing appropriate use of gain and depth functions, probe indicator positioning, and adequacy of right upper quadrant, left upper quadrant, cardiac, bladder, and lung views. The CVC checklist contained items assessing appropriate use of gain and depth functions, probe indicator positioning, number of cannulation attempts, number of carotid artery punctures, and number of times the needle tip was inserted to the hub, a proxy for pneumothorax, a known complication of CVC placement. A passing score of 70% on each checklist was chosen prior to study initiation based on pilot study data.
DATA ANALYSIS
We performed descriptive analyses using SPSS software (IBM Corp., Armonk, NY) and calculated BUS instrumentation knowledge and image interpretation test scores by dividing total correct responses by total number of questions. The student pre- versus postquestionnaire performance comparison was performed using a paired t test. The student posttest versus resident posttest performance comparison was performed using a two-sample t test. For the demographic and confidence questions, we calculated descriptive statistics and report frequencies using percentages. For the FAST and CVC skills checklists, overall performance and subscore performance were reported as percentages based on number of correct responses divided by total number of items. For performance comparison between year of study, a repeated measures ANOVA was used with the between subjects factor of year and within subjects factor of pre- versus posttest physics, instrumentation knowledge, and image interpretation ability.
RESULTS
Forty-five medical students (second-year [16; 35.6%], third-year [21; 46.7%], and fourth-year [8; 17.7%]) on clinical rotation in the Department of Emergency Medicine were prospectively and consecutively enrolled. None were lost to follow-up. Only 2% (1/45) of participants had completed a previous ultrasound rotation and 35.6% (16/ 45) had received prior ultrasound training outside of a formal ultrasound rotation (ie, onetime hands-on instructional session).
Validation
Cognitive Skills: BUS Instrumentation and Image Interpretation Knowledge Test
Evidence of construct validity was obtained by applying the 33-item knowledge questionnaire on BUS instrumentation and image interpretation questions to a convenience sample of 15 EM residents of varying postgraduate years (five PGY-1; three PGY-2; five PGY-3; two PGY4).26 The number of EM residents who participated was based on the residents who voluntarily responded to an e-mail sent to the entire
On the pretest, consisting of questions about BUS instrumentation knowledge and image interpretation, the students’ overall mean score was 66.6 6 11.2%, comprising a BUS instrumentation score of 70.7 6 13.7% and mean image interpretation score of 62.2 6 15.4% (Table 1). On the posttest, the students’ overall mean score was 85.7 6 10.0%, with an instrumentation knowledge mean score of 88.9 6 9.6% and
VOL. 43, NO. 3, MARCH/APRIL 2015
141
BLACKSTOCK ET AL TABLE 1 Comparison of Students’ Pretest Scores with Students’ Posttest Scores and EM Residents’ Posttest Scores
Item Score* Overall BUS instrumentation (physics) Image interpretation
EM Residents’ Posttest Scores,% (n 5 15)
Mean Difference Between Students’ Posttest and EM Residents’ Posttest Scores, %†
Students’ Pretest Scores, % (n 5 45)
Students’ Posttest Scores, % (n 5 45)
Mean Difference Between Students’ Pretest and Posttest Scores, %†
66.6 6 11.2 70.7 6 13.7
85.7 6 10.0 88.9 6 9.6
19.1 (15.5–22.7; p < 0.001) 18.2 (14.2–22.1; p < 0.001)
88.1 6 7.6 92.2 6 6.2
22.4 (28.1 to 3.3; p 5 0.40) 23.3 (28.8 to 2.0; p 5 0.22)
62.2 6 15.4
82.2 6 15.2
20.0 (14.6–25.4; p < 0.001)
83.8 6 12.9
21.6 (210.4 to 7.2; p 5 0.73)
Abbreviations: EM, Emergency Medicine; BUS, bedside ultrasound. *All scores are presented as means 6 SD. † Parenthetic numbers are 95% confidence intervals with p values.
image interpretation mean score of 82.2 6 15.2%. There was a significant overall mean score difference of 19.1% (95% CI 15.5–22.7; p < 0.001), BUS instrumentation score difference of 18.2% (95% CI 14.2–22.1; p < 0.001), and image interpretation score difference of 20.0% (95% CI14.6–25.4; p < 0.001) when comparing pre- and posttest performance. Additionally, there was no statistically significant difference in pre- and posttest scores as a function of year of study (p 5 0.39). Construct Validity for Cognitive Skills There were no statistically significant differences between students’ and residents’ overall posttest scores (p 5 0.40) or in any subcategory (instrumentation knowledge; p 5 0.22, image interpretation; p 5 0.73) (Table 1). Psychomotor Skills: Image Acquisition and Procedural Guidance Eighty-nine percent (40/45) of participants passed the FAST examination skills assessment performed on the volunteer patient. The mean FAST skills score was 86.6 6 12.8%. Participants were successful in obtaining most of the required FAST examination views, yielding a right upper quadrant mean score of 90.6%, left upper quadrant score of 88.3%, bladder view score of 97.2%, and lung sliding score of 90.6%, but students had the most difficulty obtaining an adequate subxiphoid cardiac view (72.2%). There was no statistical difference in FAST examination skills performance as a function of year of study (p 5 1.0). Ninety-six percent (43/45) of participants passed the CVC skills assessment performed on the internal jugular task trainer. The mean CVC skills score was 90.0 6 14.3% (minimum 40%, maximum 100%). Eighty-four percent 142
(38/45) of participants placed successful CVC within three attempts, with 64.4% (29/45) achieving success on the first attempt. Ninetyone percent (41/45) avoided inadvertent carotid artery puncture on the task trainer. Only 15.6% (7/45) of participants inserted the needle tip to the hub during their attempts, a surrogate for pneumothorax complication. There was no statistical difference in CVC skills performance as a function of year of study (p 5 0.08). Confidence One hundred percent (45/45) of students reported that they were slightly or not confident obtaining the required images for a FAST examination in the pretest assessment. However, after exposure to the curriculum, 73% (33/45) of the students went from being slightly or not confident to being at least moderately confident with obtaining the required images for the FAST examination (95% CI 59%, 84%). Ninety-six percent (43/45) of students reported that they were slightly or not confident interpreting FAST examination images in the pretest. However, after exposure to the curriculum, 77% (33/43) of students went from being slightly or not confident to being at least moderately confident at interpreting FAST examination images (95% CI 62%, 87%). Ninety-one percent (41/45) of students reported that they were slightly or not confident placing ultrasound-guided vascular access in the pretest. After exposure to the curriculum, 71% (29/41) of students went from being slightly or not confident to being at least moderately confident (95% CI 56%, 82%). At the end of the rotation, 93% (42/45) of participants were interested in participating in additional ultrasound learning sessions during the rotation and 95.6% (43/45) were interested JOURNAL OF CLINICAL ULTRASOUND
BEDSIDE US CURRICULUM FOR MEDICAL STUDENTS
in ultrasound learning sessions during their preclinical years.
DISCUSSION
Over the last two decades, several authors have implemented and reported on BUS curricula in a variety of undergraduate medical education contexts. Teichgraber et al and Ivanusic et al integrated BUS teaching into anatomy courses, which enhanced students’ understanding of anatomy.13,21 Barloon et al demonstrated that using BUS real-time as an aid to the physical examination improved preclinical students’ accuracy in measuring liver size.14 Kobal et al observed that after a brief echocardiographic training, medical students were able to detect cardiac valvular murmurs with more diagnostic accuracy than when compared with experienced ndez-Frackelton et al precardiologists.17 Ferna sented that an introductory course for medial students on clinical rotation was effective at significantly improving students’ image acquisition and interpretation ability.18 Prior BUS literature has described interventions demonstrating findings in one or two learning domains, did not leverage Web-based and blended learning strategies, ideal for teaching facts and knowledge, and did not offer evidence of construct validity.26 In this study, we implemented and evaluated a standardized, formal BUS curriculum for medical students on an EM rotation, consisting of narrated Web-based tutorials and a 3-hour hands-on training designed to target four learning domains: (1) BUS instrumentation knowledge, (2) image interpretation, (3) image acquisition, and (4) procedural guidance. Our study goes further than prior work in the field by validating the curriculum at two levels. We have demonstrated that this curriculum significantly improved students’ BUS instrumentation knowledge and image interpretation ability and that the level of knowledge attained by students was on par with current EM residents. Students also achieved high passing rates on the relevant psychomotor tasks: image acquisition (FAST) and procedural guidance (CVC) skills assessments. Finally, student confidence with image acquisition and procedural guidance increased after exposure to the curriculum. Students also expressed substantial interest in additional ultrasound sessions and in learning ultrasound during the preclinical medical school curriculum. The novel BUS blended curriculum preVOL. 43, NO. 3, MARCH/APRIL 2015
sented in this article adds significantly to this emerging field of BUS and encourages medical educators to consider BUS training as part of the formal medical school curriculum. Despite this study’s achievements, several limitations exist. A convenience sample was used for this study. The curriculum consisted of only 4 hours of instruction: 1-hour Web-based and a single 3-hour hands-on session with the BUS instructor. The investigators were also involved in the student psychomotor assessments, which could have introduced bias into the study. Although this curriculum influenced several important BUS learning domains, we were only able to assess short-term knowledge retention and learners were not tested in the clinical setting. Last, students acquired FAST examination images on a healthy volunteer and learned to place CVC on a task trainer, which may limit the generalizability of the results. In summary, we report on the successful design, implementation, evaluation, and validation of a standardized, formal, blended BUS curriculum to improve rotating EM students’ instrumentation knowledge, image acquisition and interpretation ability, and procedural guidance skills. These results provide promising support that medical students on clinical rotation can acquire a variety of BUS skills relatively quickly and with relatively minimal exposure. As BUS competency is required for residency in EM and other specialties, medical schools should consider formally incorporating BUS teaching into medical school clinical rotations.27,28
REFERENCES 1. Moore CL, Copel JA. Point-of-care ultrasonography. N Engl J Med 2011;364:749. 2. Randolph AG, Cook DJ, Gonzales CA, et al. Ultrasound guidance for placement of central venous catheters: a meta-analysis of the literature. Crit Care Med 1996;24:2053. 3. Hind D, Calvert N, McWilliams R, et al. Ultrasonic locating devices for central venous cannulation: meta-analysis. Br Med J 2003;327:361. 4. Kumar A, Chuan A. Ultrasound guided vascular access: efficacy and safety. Best Pract Res Clin Anaesthesiol 2009;23:299. 5. Ortega R, Song M, Hansen CJ, et al. Ultrasoundguided internal jugular vein cannulation. N Engl J Med 2010;362:e57. 6. Healey MA, Simons RK, Winchell RJ, et al. A prospective evaluation of abdominal ultrasound in blunt trauma: is it useful? J Trauma 1996;40:875. 7. Scalea TM, Rodriguez A, Chiu WC, et al. Focused Assessment with Sonography for Trauma (FAST):
143
BLACKSTOCK ET AL
8.
9.
10.
11.
12.
13.
14.
15.
16.
17.
18.
results from an international consensus conference. J Trauma 1999;46:466. Jehle D, Heller M, eds. Ultrasonography in Trauma: The FAST Exam. Dallas, TX: American College of Emergency Physicians; 2003, p 40. Melniker LA, Leibner E, McKenney MG, et al. Randomized controlled clinical trial of point-ofcare, limited ultrasonography for trauma in the emergency department: the first Sonography Outcomes Assessment Program trial. Ann Emerg Med 2006;48:227. ACEP Section of Emergency Ultrasound. ACEP Policy Statement on Emergency Ultrasound Guidelines. Ann Emerg Med 2009;53:550. Accreditation Council for Graduate Medical Education. ACGME Program Requirements for Graduate Medical Education in Emergency Medicine. September 30, 2012. Available at: http://www. acgme.org/acgmeweb/Portals/0/PFAssets/2013-PRFAQ-PIF/110_emergency_medicine_ 07012013.pdf. Accessed on February 19, 2014. Wald DA, Manthey DE, Kruus L, et al. The state of the clerkship: a survey of emergency medicine clerkship directors. Acad Emerg Med 2007;14:629. Teichgraber U, Meyer J, Poulsen C, et al. Ultrasound anatomy: a practical teaching system in human gross anatomy. Med Educ 1996;30:296. Barloon TJ, Brown BP, Abu-Youssef MM, et al. Teaching physical examination of the adult liver with use of real-time sonography. Acad Radiol 1998;5:101. Shapiro RS, Ko PK, Jacobson S. A pilot project to study the use of ultrasonography for teaching physical examination to medical students. Comput Biol Med 2002;32:403. Yoo MC, Villegas L, Bones JB et al. Basic ultrasound curriculum for medical students: validation of content and phantom. J Laparoendosc Adv Surg Tech A 2004;14:374. Kobal SL, Trento L, Baharami S, et al. Comparison of effectiveness of hand-carried ultrasound to bedside cardiovascular physical examination. Am J Cardiol 2005;96:1002. Fern andez-Frackelton M, Peterson M, Lewis RJ, et al. A bedside ultrasound curriculum for medical students: prospective evaluation of skill acquisition. Teach Learn Med 2007;19:14.
144
19. Wright SA, Bell AL. Enhancement of undergraduate rheumatology teaching through the use of musculoskeletal ultrasound. Rheumatology 2008;47: 1564. 20. Rao S, Van Holsbeeck L, Musial JL, et al. A pilot study of comprehensive ultrasound education at the Wayne State University School of Medicine: a pioneer year review. J Ultrasound Med 2008;27:745. 21. Ivanusic J, Cowie B, Barrington M, et al. Undergraduate student perception of the use of ultrasonography in the study of “living anatomy”. Anat Sci Educ 2010;3:318. 22. The Accreditation Council for Graduate Medical Education and the American Board of Emergency Medicine. The Emergency Medicine Milestones Project. Available at: https://www.abem.org/public/ docs/default-source/migrated-documents-and-files/ em-milestones.pdf?sfvrsn54. Accessed on February 19, 2014. 23. American College of Emergency Physicians. ACEP Emergency Ultrasound Imaging Criteria Compendium. April 2006. Available at: http://www.acep.org/ MobileArticle.aspx?id532886&coll_id556&parent id5. Accessed on February 19, 2014. 24. Sullivan M, Sullivan M, Baker C, et al. A Cognitive-Task-Analysis Informed Central Venous Catheter Placement Curriculum. MedEdPORTAL. April 5, 2012. Available at: https://www.mededportal.org/publication/9135. Accessed on February 19, 2014. 25. Woo MY, Frank J, Lee CA et al. Effectiveness of a novel training program for emergency medicine residents in ultrasound-guided insertion of central venous catheter. CJEM 2009;11:343. 26. Cook DA, Beckman TJ. Current concepts in validity and reliability for psychometric instruments: theory and application. Am J Med 2006;119:166.e7. 27. Cook T, Hunt P, Hoppmann R. Emergency Medicine leads the way for training medical students in clinician-based ultrasound: a radical paradigm shift in patient imaging. Acad Emerg Med 2007; 14:558. 28. Hoppmann RA, Rao VV, Poston MB, et al. An integrated ultrasound curriculum (iUSC) for medical students: 4-year experience. Crit Ultrasound J 2011;3:1.
JOURNAL OF CLINICAL ULTRASOUND
ABSTRACT: Background. Medical students on clinical rotations rarely receive formal bedside ultrasound (BUS) training. We designed, implemented, and evaluated a standardized BUS curriculum for medical students on their Emergency Medicine (EM) rotation. Teaching was aimed toward influencing four cognitive and psychomotor learning domains: BUS instrumentation knowledge, image interpretation, image acquisition, and procedural guidance. Methods. Participants viewed three instructional Web-based tutorials on BUS instrumentation, the Focused Assessment for Sonography in Trauma (FAST) examination and ultrasound-guided central venous catheter (CVC) placement. Subsequently, participants attended a 3-hour hands-on training session to discuss the same content area and practice with faculty coaches. A Web-based, multiple-choice questionnaire was administered before and after the session. During the final week of the rotation, students returned for skills assessments on FAST image acquisition and CVC placement. Results. Forty-five medical students on an EM rotation were enrolled. Sonographic knowledge overall mean score improved significantly from 66.6% (SD 611.2) to 85.7% (SD 610.0), corresponding to a mean difference of 19.1% (95% CI 15.5–22.7; p < 0.001). There were high pass rates for FAST (89.0%, 40/45) and CVC (96.0%, 43/45) skills assessments. There was no significant difference between medical student posttest and EM resident test scores 85.7% (SD 610.0) and 88.1% (SD 6 7.6) (p 5 0.40), respectively. Conclusions. A formal BUS curriculum for medical students on EM rotation positively influenced performance in several key learning domains. As BUS competency is required for residency in EM and other Correspondence to: U. Blackstock C 2014 Wiley Periodicals, Inc. V
VOL. 43, NO. 3, MARCH/APRIL 2015
specialties, medical schools could consider routinely incorporating BUS teaching into their clinical rotation C 2014 Wiley Periodicals, Inc. J Clin Ultracurricula. V sound 43:139–144, 2015; Published online in Wiley Online Library (wileyonlinelibrary.com). DOI: 10.1002/ jcu.22224 Keywords: bedside ultrasound; undergraduate medical education; curriculum development
INTRODUCTION
E
mergency Medicine (EM) physicians, surgeons, and intensivists are expected to accurately perform bedside ultrasound (BUS) for diagnostic and therapeutic purposes in order to maintain the highest quality of patient care and safety.1 The Focused Assessment for Sonography in Trauma (FAST) examination, to assess for internal bleeding in trauma patients, and ultrasound-guided central venous catheter (CVC) placement, are among two of the core EM BUS applications.2–10 BUS has become such an integral part of EM practice that EM residency programs require BUS training for graduation.11 Despite the prevalent clinical use of BUS, medical students rarely receive formal BUS hands-on instruction while on clinical rotations.12 There are limited but increasing reports in the literature of formal BUS during preclinical and clinical training.13–21 The lack of formal BUS instruction in many medical school curricula, despite its increasing use in multiple specialties, underscores the need to create 139
BLACKSTOCK ET AL
opportunities to expose medical students to BUS and train them in the fundamentals and basic applications prior to beginning residency training. The American Board of Emergency Medicine and the Accreditation Council for Graduate Medical Education of Graduate jointly released EM milestones in 2012; two of them call for evaluation of ultrasound knowledge and skill (“Goal-Directed Focused Ultrasound” and “Vascular Access”).22 We designed, implemented, and evaluated a 4-week-long, formal curriculum for medical students during the EM rotation consisting of narrated Web-based tutorials and hands-on BUS training developed to influence four cognitive and psychomotor learning domains: (1) BUS instrumentation knowledge; (2) image interpretation; (3) image acquisition; and (4) procedural guidance. Our study goes further than prior studies by implementing a formal, blended curriculum within the EM rotation, assessing multiple learning domains and comparing students’ abilities to that of EM residents. It is expected the outcomes of this study will help inform the implementation of formal BUS curriculums during EM and other clinical rotations in order to better address medical students’ training needs.
MATERIALS AND METHODS
We prospectively enrolled a convenience sample of medical students on their EM rotation at an urban tertiary medical center from January 2012 to June 2012. The medical center’s Institutional Review Board approved the study. Verbal informed consent was obtained from all participants. Medical students participating in the 4-week EM elective clinical rotation were enrolled. Students can take the elective during their clerkship year (after basic sciences) or during their final year of medical school. Clerkship-level students were expected to function at the level of a medical student and the fourth-year students were expected to function more independently (at the level of a new intern) in the clinical setting. Sample Size Calculation The primary outcome of the study was defined as an increase in BUS instrumentation knowledge and image interpretation ability after the educational intervention. We calculated that 14 subjects are required to have a 80% chance of detecting, as significant at the 5% level, an 140
increase in the primary outcome measure from a mean score of 70 in the preintervention group to a mean score of 85 in the postintervention group using a paired design with an estimated SD of the outcome of 10% using a two-tailed paired design analysis. Pretest Assessment Prior to the educational intervention, all students completed a 43-item Web-based multiplechoice questionnaire, which contained items assessing baseline demographics (seven items), confidence with BUS (three items), followed by 33 multiple-choice questions assessing knowledge of BUS instrumentation (17/33) and image interpretation (16/33). The image interpretation questions consisted of normal and pathologic FAST images, as well as normal neck vascular ultrasound anatomy images. Students were asked to identify anatomic structures and pathology in still images and video clips. Educational Intervention The blended curriculum consisted of narrated Web-based video tutorials and hands-on instructional sessions. All students viewed three Web-based tutorials at their own pace, covering (1) ultrasound physics and instrumentation (5 minutes), (2) the FAST examination (20 minutes), and (3) ultrasound-guided CVC placement (15 minutes). Next, students were randomly assigned and divided into three groups for their hands-on instructional sessions. In order to maintain a low student-to-faculty ratio, sessions were held during the first, second, and third weeks of the rotation and two to three students attended each session. Each student attended one 3-hour BUS instructional session, led by an EM BUS faculty, where they applied concepts and techniques discussed during the Web-based tutorials. Faculty demonstrated correct technique, reviewed concepts, coached, and gave feedback to the students. Curriculum was standardized to ensure that during this session, students learned how use the ultrasound machine, how to perform the FAST examination on each other, and how to cannulate the vein on an internal jugular vein task trainer using ultrasound guidance (CentralLineMan System; Simulab Corporation, Seattle, WA). All students received identical instruction from the BUS instructor regardless of which session the student attended. The ultrasound equipment used during the instructional session was the Sonosite JOURNAL OF CLINICAL ULTRASOUND
BEDSIDE US CURRICULUM FOR MEDICAL STUDENTS
M-Turbo (Sonosite, Inc., Bothell, WA) equipped with 5–1-MHz and 10–5-Hz transducers.
residency program soliciting anonymous participation in the study.
Posttest Assessment Subsequent to the instructional session, students completed a 49-item Web-based multiplechoice questionnaire, containing 13 questions asking them about their perception of the utility of the training and technique, as well as confidence in applying BUS use, 33 questions assessing their knowledge of BUS instrumentation and image interpretation questions (same as pretest), and 3 closing questions regarding interest in additional ultrasound education. Psychomotor skills were formally assessed during the last week of using trained raters (1 to 3 weeks after the initial session). Students were asked to perform the FAST examination on a healthy male volunteer and ultrasoundguided CVC placement on an internal jugular task trainer. Student psychomotor skills during performance of the FAST examination and ultrasoundguided CVC placement were assessed using behaviorally specific standardized checklists. The FAST examination checklist was developed using American College of Emergency Physicians imaging criteria, and the CVC checklist was developed based on a literature review of previously validated skills checklists.23–25 The FAST examination checklist contained items assessing appropriate use of gain and depth functions, probe indicator positioning, and adequacy of right upper quadrant, left upper quadrant, cardiac, bladder, and lung views. The CVC checklist contained items assessing appropriate use of gain and depth functions, probe indicator positioning, number of cannulation attempts, number of carotid artery punctures, and number of times the needle tip was inserted to the hub, a proxy for pneumothorax, a known complication of CVC placement. A passing score of 70% on each checklist was chosen prior to study initiation based on pilot study data.
DATA ANALYSIS
We performed descriptive analyses using SPSS software (IBM Corp., Armonk, NY) and calculated BUS instrumentation knowledge and image interpretation test scores by dividing total correct responses by total number of questions. The student pre- versus postquestionnaire performance comparison was performed using a paired t test. The student posttest versus resident posttest performance comparison was performed using a two-sample t test. For the demographic and confidence questions, we calculated descriptive statistics and report frequencies using percentages. For the FAST and CVC skills checklists, overall performance and subscore performance were reported as percentages based on number of correct responses divided by total number of items. For performance comparison between year of study, a repeated measures ANOVA was used with the between subjects factor of year and within subjects factor of pre- versus posttest physics, instrumentation knowledge, and image interpretation ability.
RESULTS
Forty-five medical students (second-year [16; 35.6%], third-year [21; 46.7%], and fourth-year [8; 17.7%]) on clinical rotation in the Department of Emergency Medicine were prospectively and consecutively enrolled. None were lost to follow-up. Only 2% (1/45) of participants had completed a previous ultrasound rotation and 35.6% (16/ 45) had received prior ultrasound training outside of a formal ultrasound rotation (ie, onetime hands-on instructional session).
Validation
Cognitive Skills: BUS Instrumentation and Image Interpretation Knowledge Test
Evidence of construct validity was obtained by applying the 33-item knowledge questionnaire on BUS instrumentation and image interpretation questions to a convenience sample of 15 EM residents of varying postgraduate years (five PGY-1; three PGY-2; five PGY-3; two PGY4).26 The number of EM residents who participated was based on the residents who voluntarily responded to an e-mail sent to the entire
On the pretest, consisting of questions about BUS instrumentation knowledge and image interpretation, the students’ overall mean score was 66.6 6 11.2%, comprising a BUS instrumentation score of 70.7 6 13.7% and mean image interpretation score of 62.2 6 15.4% (Table 1). On the posttest, the students’ overall mean score was 85.7 6 10.0%, with an instrumentation knowledge mean score of 88.9 6 9.6% and
VOL. 43, NO. 3, MARCH/APRIL 2015
141
BLACKSTOCK ET AL TABLE 1 Comparison of Students’ Pretest Scores with Students’ Posttest Scores and EM Residents’ Posttest Scores
Item Score* Overall BUS instrumentation (physics) Image interpretation
EM Residents’ Posttest Scores,% (n 5 15)
Mean Difference Between Students’ Posttest and EM Residents’ Posttest Scores, %†
Students’ Pretest Scores, % (n 5 45)
Students’ Posttest Scores, % (n 5 45)
Mean Difference Between Students’ Pretest and Posttest Scores, %†
66.6 6 11.2 70.7 6 13.7
85.7 6 10.0 88.9 6 9.6
19.1 (15.5–22.7; p < 0.001) 18.2 (14.2–22.1; p < 0.001)
88.1 6 7.6 92.2 6 6.2
22.4 (28.1 to 3.3; p 5 0.40) 23.3 (28.8 to 2.0; p 5 0.22)
62.2 6 15.4
82.2 6 15.2
20.0 (14.6–25.4; p < 0.001)
83.8 6 12.9
21.6 (210.4 to 7.2; p 5 0.73)
Abbreviations: EM, Emergency Medicine; BUS, bedside ultrasound. *All scores are presented as means 6 SD. † Parenthetic numbers are 95% confidence intervals with p values.
image interpretation mean score of 82.2 6 15.2%. There was a significant overall mean score difference of 19.1% (95% CI 15.5–22.7; p < 0.001), BUS instrumentation score difference of 18.2% (95% CI 14.2–22.1; p < 0.001), and image interpretation score difference of 20.0% (95% CI14.6–25.4; p < 0.001) when comparing pre- and posttest performance. Additionally, there was no statistically significant difference in pre- and posttest scores as a function of year of study (p 5 0.39). Construct Validity for Cognitive Skills There were no statistically significant differences between students’ and residents’ overall posttest scores (p 5 0.40) or in any subcategory (instrumentation knowledge; p 5 0.22, image interpretation; p 5 0.73) (Table 1). Psychomotor Skills: Image Acquisition and Procedural Guidance Eighty-nine percent (40/45) of participants passed the FAST examination skills assessment performed on the volunteer patient. The mean FAST skills score was 86.6 6 12.8%. Participants were successful in obtaining most of the required FAST examination views, yielding a right upper quadrant mean score of 90.6%, left upper quadrant score of 88.3%, bladder view score of 97.2%, and lung sliding score of 90.6%, but students had the most difficulty obtaining an adequate subxiphoid cardiac view (72.2%). There was no statistical difference in FAST examination skills performance as a function of year of study (p 5 1.0). Ninety-six percent (43/45) of participants passed the CVC skills assessment performed on the internal jugular task trainer. The mean CVC skills score was 90.0 6 14.3% (minimum 40%, maximum 100%). Eighty-four percent 142
(38/45) of participants placed successful CVC within three attempts, with 64.4% (29/45) achieving success on the first attempt. Ninetyone percent (41/45) avoided inadvertent carotid artery puncture on the task trainer. Only 15.6% (7/45) of participants inserted the needle tip to the hub during their attempts, a surrogate for pneumothorax complication. There was no statistical difference in CVC skills performance as a function of year of study (p 5 0.08). Confidence One hundred percent (45/45) of students reported that they were slightly or not confident obtaining the required images for a FAST examination in the pretest assessment. However, after exposure to the curriculum, 73% (33/45) of the students went from being slightly or not confident to being at least moderately confident with obtaining the required images for the FAST examination (95% CI 59%, 84%). Ninety-six percent (43/45) of students reported that they were slightly or not confident interpreting FAST examination images in the pretest. However, after exposure to the curriculum, 77% (33/43) of students went from being slightly or not confident to being at least moderately confident at interpreting FAST examination images (95% CI 62%, 87%). Ninety-one percent (41/45) of students reported that they were slightly or not confident placing ultrasound-guided vascular access in the pretest. After exposure to the curriculum, 71% (29/41) of students went from being slightly or not confident to being at least moderately confident (95% CI 56%, 82%). At the end of the rotation, 93% (42/45) of participants were interested in participating in additional ultrasound learning sessions during the rotation and 95.6% (43/45) were interested JOURNAL OF CLINICAL ULTRASOUND
BEDSIDE US CURRICULUM FOR MEDICAL STUDENTS
in ultrasound learning sessions during their preclinical years.
DISCUSSION
Over the last two decades, several authors have implemented and reported on BUS curricula in a variety of undergraduate medical education contexts. Teichgraber et al and Ivanusic et al integrated BUS teaching into anatomy courses, which enhanced students’ understanding of anatomy.13,21 Barloon et al demonstrated that using BUS real-time as an aid to the physical examination improved preclinical students’ accuracy in measuring liver size.14 Kobal et al observed that after a brief echocardiographic training, medical students were able to detect cardiac valvular murmurs with more diagnostic accuracy than when compared with experienced ndez-Frackelton et al precardiologists.17 Ferna sented that an introductory course for medial students on clinical rotation was effective at significantly improving students’ image acquisition and interpretation ability.18 Prior BUS literature has described interventions demonstrating findings in one or two learning domains, did not leverage Web-based and blended learning strategies, ideal for teaching facts and knowledge, and did not offer evidence of construct validity.26 In this study, we implemented and evaluated a standardized, formal BUS curriculum for medical students on an EM rotation, consisting of narrated Web-based tutorials and a 3-hour hands-on training designed to target four learning domains: (1) BUS instrumentation knowledge, (2) image interpretation, (3) image acquisition, and (4) procedural guidance. Our study goes further than prior work in the field by validating the curriculum at two levels. We have demonstrated that this curriculum significantly improved students’ BUS instrumentation knowledge and image interpretation ability and that the level of knowledge attained by students was on par with current EM residents. Students also achieved high passing rates on the relevant psychomotor tasks: image acquisition (FAST) and procedural guidance (CVC) skills assessments. Finally, student confidence with image acquisition and procedural guidance increased after exposure to the curriculum. Students also expressed substantial interest in additional ultrasound sessions and in learning ultrasound during the preclinical medical school curriculum. The novel BUS blended curriculum preVOL. 43, NO. 3, MARCH/APRIL 2015
sented in this article adds significantly to this emerging field of BUS and encourages medical educators to consider BUS training as part of the formal medical school curriculum. Despite this study’s achievements, several limitations exist. A convenience sample was used for this study. The curriculum consisted of only 4 hours of instruction: 1-hour Web-based and a single 3-hour hands-on session with the BUS instructor. The investigators were also involved in the student psychomotor assessments, which could have introduced bias into the study. Although this curriculum influenced several important BUS learning domains, we were only able to assess short-term knowledge retention and learners were not tested in the clinical setting. Last, students acquired FAST examination images on a healthy volunteer and learned to place CVC on a task trainer, which may limit the generalizability of the results. In summary, we report on the successful design, implementation, evaluation, and validation of a standardized, formal, blended BUS curriculum to improve rotating EM students’ instrumentation knowledge, image acquisition and interpretation ability, and procedural guidance skills. These results provide promising support that medical students on clinical rotation can acquire a variety of BUS skills relatively quickly and with relatively minimal exposure. As BUS competency is required for residency in EM and other specialties, medical schools should consider formally incorporating BUS teaching into medical school clinical rotations.27,28
REFERENCES 1. Moore CL, Copel JA. Point-of-care ultrasonography. N Engl J Med 2011;364:749. 2. Randolph AG, Cook DJ, Gonzales CA, et al. Ultrasound guidance for placement of central venous catheters: a meta-analysis of the literature. Crit Care Med 1996;24:2053. 3. Hind D, Calvert N, McWilliams R, et al. Ultrasonic locating devices for central venous cannulation: meta-analysis. Br Med J 2003;327:361. 4. Kumar A, Chuan A. Ultrasound guided vascular access: efficacy and safety. Best Pract Res Clin Anaesthesiol 2009;23:299. 5. Ortega R, Song M, Hansen CJ, et al. Ultrasoundguided internal jugular vein cannulation. N Engl J Med 2010;362:e57. 6. Healey MA, Simons RK, Winchell RJ, et al. A prospective evaluation of abdominal ultrasound in blunt trauma: is it useful? J Trauma 1996;40:875. 7. Scalea TM, Rodriguez A, Chiu WC, et al. Focused Assessment with Sonography for Trauma (FAST):
143
BLACKSTOCK ET AL
8.
9.
10.
11.
12.
13.
14.
15.
16.
17.
18.
results from an international consensus conference. J Trauma 1999;46:466. Jehle D, Heller M, eds. Ultrasonography in Trauma: The FAST Exam. Dallas, TX: American College of Emergency Physicians; 2003, p 40. Melniker LA, Leibner E, McKenney MG, et al. Randomized controlled clinical trial of point-ofcare, limited ultrasonography for trauma in the emergency department: the first Sonography Outcomes Assessment Program trial. Ann Emerg Med 2006;48:227. ACEP Section of Emergency Ultrasound. ACEP Policy Statement on Emergency Ultrasound Guidelines. Ann Emerg Med 2009;53:550. Accreditation Council for Graduate Medical Education. ACGME Program Requirements for Graduate Medical Education in Emergency Medicine. September 30, 2012. Available at: http://www. acgme.org/acgmeweb/Portals/0/PFAssets/2013-PRFAQ-PIF/110_emergency_medicine_ 07012013.pdf. Accessed on February 19, 2014. Wald DA, Manthey DE, Kruus L, et al. The state of the clerkship: a survey of emergency medicine clerkship directors. Acad Emerg Med 2007;14:629. Teichgraber U, Meyer J, Poulsen C, et al. Ultrasound anatomy: a practical teaching system in human gross anatomy. Med Educ 1996;30:296. Barloon TJ, Brown BP, Abu-Youssef MM, et al. Teaching physical examination of the adult liver with use of real-time sonography. Acad Radiol 1998;5:101. Shapiro RS, Ko PK, Jacobson S. A pilot project to study the use of ultrasonography for teaching physical examination to medical students. Comput Biol Med 2002;32:403. Yoo MC, Villegas L, Bones JB et al. Basic ultrasound curriculum for medical students: validation of content and phantom. J Laparoendosc Adv Surg Tech A 2004;14:374. Kobal SL, Trento L, Baharami S, et al. Comparison of effectiveness of hand-carried ultrasound to bedside cardiovascular physical examination. Am J Cardiol 2005;96:1002. Fern andez-Frackelton M, Peterson M, Lewis RJ, et al. A bedside ultrasound curriculum for medical students: prospective evaluation of skill acquisition. Teach Learn Med 2007;19:14.
144
19. Wright SA, Bell AL. Enhancement of undergraduate rheumatology teaching through the use of musculoskeletal ultrasound. Rheumatology 2008;47: 1564. 20. Rao S, Van Holsbeeck L, Musial JL, et al. A pilot study of comprehensive ultrasound education at the Wayne State University School of Medicine: a pioneer year review. J Ultrasound Med 2008;27:745. 21. Ivanusic J, Cowie B, Barrington M, et al. Undergraduate student perception of the use of ultrasonography in the study of “living anatomy”. Anat Sci Educ 2010;3:318. 22. The Accreditation Council for Graduate Medical Education and the American Board of Emergency Medicine. The Emergency Medicine Milestones Project. Available at: https://www.abem.org/public/ docs/default-source/migrated-documents-and-files/ em-milestones.pdf?sfvrsn54. Accessed on February 19, 2014. 23. American College of Emergency Physicians. ACEP Emergency Ultrasound Imaging Criteria Compendium. April 2006. Available at: http://www.acep.org/ MobileArticle.aspx?id532886&coll_id556&parent id5. Accessed on February 19, 2014. 24. Sullivan M, Sullivan M, Baker C, et al. A Cognitive-Task-Analysis Informed Central Venous Catheter Placement Curriculum. MedEdPORTAL. April 5, 2012. Available at: https://www.mededportal.org/publication/9135. Accessed on February 19, 2014. 25. Woo MY, Frank J, Lee CA et al. Effectiveness of a novel training program for emergency medicine residents in ultrasound-guided insertion of central venous catheter. CJEM 2009;11:343. 26. Cook DA, Beckman TJ. Current concepts in validity and reliability for psychometric instruments: theory and application. Am J Med 2006;119:166.e7. 27. Cook T, Hunt P, Hoppmann R. Emergency Medicine leads the way for training medical students in clinician-based ultrasound: a radical paradigm shift in patient imaging. Acad Emerg Med 2007; 14:558. 28. Hoppmann RA, Rao VV, Poston MB, et al. An integrated ultrasound curriculum (iUSC) for medical students: 4-year experience. Crit Ultrasound J 2011;3:1.
JOURNAL OF CLINICAL ULTRASOUND
