FOCUSED CARDIAC ULTRASOUND
Development and Evaluation of Methodologies for Teaching Focused Cardiac Ultrasound Skills to Medical Students Thomas R. Cawthorn, MD, Curtis Nickel, MD, Michael O’Reilly, MD, Henryk Kafka, MD, James W. Tam, MD, Lynel C. Jackson, Anthony J. Sanfilippo, MD, and Amer M. Johri, MD, Kingston, Ontario, Canada; Winnipeg, Manitoba, Canada
Background: Handheld ultrasound is emerging as an important tool for point-of-care cardiac assessment. Although cardiac ultrasound skills are traditionally introduced during postgraduate training, the optimal time and methodology to initiate training in focused cardiac ultrasound (FCU) are unknown. The objective of this study was to develop and evaluate a novel curriculum for training medical students in the use of FCU. Methods: The study was conducted in two phases. In the first phase, 12 first-year medical students underwent FCU training over an 8-week period. In the second phase, 45 third-year medical students were randomized to one of three educational programs. Program 1 consisted of a lecture-based approach with scan training by a sonographer. Program 2 coupled electronic education modules with sonographer scan training. Program 3 was fully self-directed, combining electronic modules with scan training on a high-fidelity ultrasound simulator. Image interpretation skills and scanning technique were evaluated after each program. Results: First-year medical students were able to modestly improve interpretation ability and acquire limited scanning skills. Third-year medical students exhibited similar improvement in mean examination score for image interpretation whether a lecture-based program or electronic modules was used. Students in the selfdirected group using an ultrasound simulator had significantly lower mean quality scores than students taught by sonographers. Conclusions: Third-year medical students were able to acquire FCU image acquisition and interpretation skills after a novel training program. Self-directed electronic modules are effective for teaching introductory FCU interpretation skills, while expert-guided training is important for developing scanning technique. (J Am Soc Echocardiogr 2014;27:302-9.) Keywords: Handheld ultrasound, Medical education, Electronic learning, Simulation
Recent developments in ultrasound technology have enabled pointof-care cardiac assessment of patients using portable, handheld ultrasound (HHU) units. The American Society of Echocardiography (ASE) has recognized that these devices are capable of performing focused cardiac ultrasound (FCU) assessments as an adjunct to the physical examination. The ASE has also noted that comprehensive From the Queen’s University, School of Medicine, Kingston, Ontario, Canada (T.R.C., C.N., L.C.J.); Cardiovascular Imaging Network at Queen’s Research Group, Queen’s University, Kingston, Ontario, Canada (T.R.C., C.N., A.M.J.); Division of Cardiology, Queen’s University, Kingston, Ontario, Canada (M.O.R., H.K., A.J.S., A.M.J.); and Section of Cardiology, University of Manitoba, Winnipeg, Manitoba, Canada (J.W.T.). Drs Cawthorn and Nickel contributed equally to this work. Funded by the Cardiovascular Imaging Network at Queen’s University. Reprint requests: Amer M. Johri, MD, MSc, FRCPC, FASE, Kingston General Hospital, Division of Cardiology, FAPC3, 76 Stuart Street, Kingston, ON K7L 3N6, Canada (E-mail: [email protected]). 0894-7317/$36.00 Copyright 2013 by the American Society of Echocardiography. http://dx.doi.org/10.1016/j.echo.2013.12.006
302
echocardiographic examination requires image acquisition by trained sonographers and image interpretation by skilled echocardiographers.1 Although the opportunity for widespread use of HHU exists, the appropriate clinical applications of FCU are yet to be clearly defined. The potential clinical utility of FCU has been well established.2-8 However, no studies have attempted to elucidate the optimal timing and methodology to educate trainees in the use of FCU. Current research has focused on the application of FCU in clinical settings; published training programs have not been well validated. Several international bodies have recognized the need to train nonexpert imagers; however, a consensus does not exist regarding the most appropriate timing and method of training.1,9,10 Considering that this technology is available for use in a broad range of settings, a lack of standardized training is concerning. The majority of studies examining FCU have focused exclusively on the proficiency of attending staff members and postgraduate trainees. There has been comparatively little research focusing on introduction of FCU training during undergraduate medical education. Because medical school is the stage at which fundamental clinical examination skills are introduced, training in FCU may be most effective during this period. Considering the increasing prevalence of
Cawthorn et al 303
Journal of the American Society of Echocardiography Volume 27 Number 3
ultrasound use, competence in sonographic examination may A5C = Apical five-chamber become a core competency required of graduating medical ASE = American Society of students. Echocardiography The objective of this study A2C = Apical two-chamber was to develop and evaluate three novel curricula for training FCU = Focused cardiac medical students in the acquisiultrasound tion and interpretation of FCU. HHU = Handheld ultrasound The educational programs were LV = Left ventricular designed to provide students with a basic skill set for the assessPLX = Parasternal long-axis ment of cardiac patients using an PSX = Parasternal short-axis eight-view FCU protocol. The study was conducted in two phases to (1) assess the time at which the introduction of FCU skills would be appropriate and (2) assess the optimal methodology (didactic teaching, electronic modules, or simulation training) to deliver this fundamental knowledge. Abbreviations
Table 1 Cardiac pathologies that medical students were taught to recognize and identify Pathology
Interpretation Method
Aortic valve dysfunction (moderate to severe regurgitation or stenosis)
Abnormal leaflet thickening, mobility Conventional color flow with visual assessment
Mitral valve dysfunction (moderate to severe regurgitation or stenosis) LV hypertrophy (moderate to severe)
Conventional color flow with visual assessment
LV systolic dysfunction (moderate to severe) Pericardial effusion (moderate to severe)
Visual estimate of LV systolic contractility Visual estimate of pericardial space for presence/absence of effusion
Qualitative analysis on the basis of visual estimate of wall thickness
METHODS The study was conducted in two phases. The first phase was designed to establish the feasibility of first-year medical students to acquire FCU interpretation and image acquisition skills. The second phase was conducted in a larger cohort of third-year medical students and investigated different methods of delivering knowledge and skills, including electronic and simulation-based modules. The Queen’s University Research Ethics Board approved the study protocols, and written consent was obtained from all participants. Phase 1 Study Population. Twelve medical students from the Queen’s University School of Medicine were recruited to participate in this project during the summer break after their first year of medical education. Students had not completed their core training in cardiology. Exclusion criteria included any preexisting ultrasound or echocardiography experience. No students met these criteria, thus none were excluded. This study was extracurricular, and no compensation, monetary or academic credit, was provided for participation. HHU Device. The Vscan device (GE Healthcare, Horton, Norway) used in this study consists of a display unit and a broad-bandwidth, phased-array probe (total weight, 390 g). It provides twodimensional imaging and conventional color flow echocardiographic images. It has limited controls for adjusting image depth and gain. Electronic calipers and a touchpad enable the user to make distance and area measurements. All images can be frozen and scrolled for review, but there is no electrocardiographic gating. Images may be saved in still or video format. Cardiac Pathologies. Five cardiac pathologies were examined: left ventricular (LV) systolic dysfunction, LV hypertrophy, aortic valve dysfunction (regurgitation and stenosis), mitral valve dysfunction (regurgitation and stenosis), and pericardial effusion (Table 1). The cardiac pathologies were selected on the basis of consensus by three ASE level III echocardiographers at our institution. All FCU images provided to students for assessment depicted abnormalities rated as moderate or severe by ASE level III echocardiographers; this level
of pathology was considered representative of clinically relevant abnormalities. Students were not expected to grade pathologies on the basis of severity; assessment of images was based exclusively on identification of the presence of a moderate or severe abnormality. Preintervention Evaluation. Participants underwent an evaluation of FCU interpretation knowledge before commencing the educational intervention. This evaluation consisted of 15 online case-based multiple-choice questions. Each question consisted of a one-line clinical vignette and a video showing an FCU assessment. Participants were then asked to identify the presence of major cardiac pathology if any was present. Three examples of each of the five cardiac pathologies were presented. Educational Intervention. The educational intervention was created in consultation with National Board of Echocardiography, ASE level III, and FASE certified echocardiographers and American Registry for Diagnostic Medical Sonography certified sonographers. The program was designed to instruct the participants in image acquisition and interpretation of focused cardiac imaging using HHU units. The educational intervention included instruction in image acquisition using a focused protocol consisting of seven specific views: apical two-chamber (A2C), apical four-chamber, apical five-chamber (A5C), parasternal long-axis (PLX), and parasternal short-axis (PSX) with cuts at the aortic valve, mitral valve, and midchamber left ventricle (PSX-LV). The intervention occurred over an 8-week period with a 2-hour session each week. Each 2-hour session was divided into a didactic and a practical component. The eight didactic lectures focused on instruction surrounding general cardiac anatomy, general cardiac ultrasound, device use, and the five pathologies described in Table 1. All lectures were delivered by ASE level III echocardiographers. Six 1-hour practical instructional sessions focused on FCU scanning technique. Practical sessions were taught by both ASE level III echocardiographers and American Registry for Diagnostic Medical Sonography certified sonographers. Participants were instructed in the use of the HHU unit and proper sonography technique to improve their image accuracy and quality. Participants worked in pairs
304 Cawthorn et al
or small groups with the HHU units to develop their sonographic technique under the direct supervision of the instructors. The remaining two 1-hour practical sessions focused on teaching cases developed from FCU examinations of hospital inpatients. The final element of the educational intervention involved self-directed learning; participants were provided unlimited access to the HHU units and dedicated examination rooms for self-directed practice sessions. The self-directed portion was not a required element of the intervention and thus was not formally quantified. Postintervention Evaluation. Evaluation of the education intervention was composed of two elements: (1) interpretation knowledge and ability and (2) image acquisition accuracy and quality. Interpretation knowledge and ability was evaluated using an online 15-question, case-based, multiple-choice examination that was identical in structure to the preintervention examination described above. Image acquisition was assessed via an objective structured clinical examination, during which participants were required to obtain seven defined views within a 15-min time period on a healthy volunteer patient. These views were captured, downloaded, and compiled for scoring. Each view was evaluated by an ASE level III echocardiographer blinded to the HHU unit operator. Each view was assigned a composite score on the basis of subjective image quality (1 = excellent, 2 = good, 3 = fair, 4 = poor). Data Analysis. Multiple-choice examination scores were compiled and tabulated. Data were exported into SPSS version 17.0 for Windows (SPSS, Inc, Chicago, IL) for analysis. Mean scores were calculated for each individual, pathology, and the cohort for both the preintervention and postintervention evaluations. Mean improvement was calculated for each of these categories. Paired t tests were used to compare improvement from preintervention to postintervention for individuals and the cohort. P values < .05 were deemed significant. Improvement in scores by pathology was compared from before to after the intervention. Exact P values were obtained using Bowker’s test of symmetry. Mean scanning scores were tabulated for each participant, view, and the cohort and compared using repeated-measures analysis of variance. Phase 2 Study Population. Forty-five third-year medical students from the Queen’s University School of Medicine were recruited to participate in this study. All students were in their third year of medical school and had completed their core teaching unit in cardiology before the study. Exclusion criteria were identical to those of phase 1; none of the phase 2 students had participated in phase 1. Students were randomized into one of three educational programs using a random-number generator and were assigned unique identification numbers for the duration of the study. Cardiac Pathologies. The same cardiac pathologies and interpretation methodology used in phase 1 were used in phase 2 (Table 1). Preintervention Assessment. After randomization, baseline knowledge of FCU was assessed using an online multiple-choice examination comprising 20 questions, each based on a different FCU cine movie. Ten cases involved normal cases and required students to identify the FCU view shown or identify an aspect of cardiac anatomy. The remaining 10 cases involved two examples of each of the five pathologies; for aortic and mitral valve dysfunction, one example of regurgitation and one example of stenosis was used for each valve
Journal of the American Society of Echocardiography March 2014
(Table 1). Students were required to report their confidence in each answer (on a scale ranging from 0% to 100% in 10% intervals). Educational Interventions. A novel 4-week FCU curriculum was developed through collaboration with ASE level III echocardiographers and experienced medical educators. This curriculum was delivered via three distinct educational programs (Table 2). Each education program consisted of two components: a ‘‘fundamental education unit’’ that focused on teaching the fundamental principles of cardiac ultrasound and image interpretation and a ‘‘scanning education unit’’ that focused on instruction of FCU scanning technique and image acquisition. Scanning education focused on eight views: the seven views used in phase 1 and a PSX cut of the LV apex. All training sessions were voluntary and conducted entirely outside of the standard medical school curriculum. Sessions were conducted in the School of Medicine building at Queen’s University. To evaluate compliance with the educational protocol, attendance was taken at each training session; however, compliance with self-directed learning components of the curriculum was not formally assessed. Didactic Education. The didactic component of the educational programs was delivered in four 2-hour lectures over the course of 2 weeks. These sessions were conducted by two expert (ASE level III) staff echocardiographers with extensive experience in medical education. Each session was designed to address specific topics and meet predefined objectives that aligned with the overall educational program goals (Table 3). Didactic content was delivered using interactive PowerPoint presentations (Microsoft Corporation, Redmond, WA) and case-based discussion in a small-group format (15 students). Electronic Learning Modules. Three interactive online electronic learning modules were designed specifically for this study to deliver identical content as the didactic lectures. These modules were created using the MEdTech platform, an electronic interface purpose-built for the Queen’s University School of Medicine. The modules were designed by study investigators in conjunction with medical educators and an expert MEdTech programmer and Web developer. Modules were accessible to students in educational programs 2 and 3 for a defined 2-week period during the study. Students were able to access the modules at any time during the study period (either on campus or from remote locations using their institutional accounts). The first module introduced the basic fundamentals of FCU, including physics, clinical applications, and FCU scanning theory. It was designed to provide foundational knowledge regarding FCU to the participants, who had no previous ultrasound or echocardiography experience. This module was intended to be completed over approximately 2 hours. The second module focused on developing image acquisition skills with the HHU device. The basic functions of the device were outlined using diagrams, video, and text. The standard echocardiographic views to be included in the focused cardiac assessment were described with text and example videos obtained using the HHU device. A major component of this module was a series of nine tutorial videos that led the students through a complete FCU assessment on a patient using the HHU device. This included instructional videos on patient positioning, positioning of the probe for each view, cardiac anatomy using FCU, and troubleshooting. This module was intended to be completed over approximately 4 hours.
Cawthorn et al 305
Journal of the American Society of Echocardiography Volume 27 Number 3
Table 2 Comparison of the components of each educational program used in phase 2
Fundamental education Scanning education
Educational program 1
Educational program 2
Educational program 3
Didactic sessions Sonographer training
Electronic modules Sonographer training
Electronic modules Simulation training
Table 3 Outline of didactic lectures sessions (phase 2, educational program 1). Session
Topic/content
Objective
1 (2 h)
Introduction to fundamental principles of FCU Review of cardiac anatomy
Develop an understanding of basic FCU principles and theory Establish baseline knowledge of cardiac anatomy and physiology as related to FCU
2 (2 h)
Basic echocardiographic views Introduction to scanning technique
3 (2 h)
Valvular abnormalities (aortic and mitral)
Recognize eight standard echocardiographic views (A2C, A4C, A5C, PLX, PSX-AV, PSX-MV, PSXLV, PSX-A) Develop fundamental scanning technique using the HHU device Develop an approach to recognizing basic valvular abnormalities on FCU (aortic stenosis/regurgitation and mitral stenosis/ regurgitation)
4 (2 h)
LV systolic dysfunction and hypertrophy Pericardial effusion
Develop an approach to recognizing global LV systolic dysfunction and LV hypertrophy Learning to recognize signs of pericardial effusion
A4C, Apical four-chamber; PSX-A, PSX view with a cut at the LV apex; PSX-AV, PSX view with a cut at the aortic valve; PSX-LV, PSX view with a cut at the left ventricle; PSX-MV, PSX view with a cut at the mitral valve.
The third module provided education on the interpretation of the predetermined pathologies. A systematic approach to image interpretation was described. In addition, a brief review of the various imaging views was provided. The predefined pathologies were described in terms of anatomy, etiologies, and pathophysiology. Annotated figures and videos of specific pathology were provided as the main instructional tool. Finally, video examples with annotations were provided for self-assessment. This module was intended to be completed over approximately 4 hours. Sonographer-Directed Scanning Tutorials. Students participated in two hands-on training sessions (2 hours each) conducted by an American Registry for Diagnostic Medical Sonography certified sonographer. The sonographers were instructed by study coordinators to educate the students within the parameters of the study (focusing on specific views and techniques). Sonographers provided demonstrations in scanning technique and troubleshooting methods.
Students developed their scanning technique by practicing on one another in small groups, under the supervision and guidance of the sonographer using a HHU unit. Simulation-Based Scanning Sessions. The simulation sessions were not led by an expert instructor; all sessions were self-directed after a brief orientation of the simulation system by a study coordinator. Students participated in two self-directed hands-on training sessions (2 hours each) using a CAE Vimedix Ultrasound Simulator System (CAE Healthcare, Montreal, Quebec, Canada). During these sessions, students used the simulation system to develop their scanning technique and ability to identify pathologies. This high-fidelity simulation system has multiple learning modalities and settings, enabling students to progressively learn the basics of scanning technique. The simulation system allows students to practice scanning on ‘‘normal’’ patients and patients with a variety of preprogrammed cardiac pathologies, including LV systolic dysfunction, valvular dysfunction, and pericardial effusion. The simulation system uses a software-to-software probe-phantom system and also allows for simultaneous ‘‘cutaway’’ anatomic illustrations on a second viewing screen adjacent to the screen on which the ultrasound image is acquired. Self-Directed Learning. As in phase 1, throughout the 4-week study, students in all educational programs were able to access HHU units to conduct self-directed practice in scanning. Participants were provided unlimited access to the HHU units and dedicated examination rooms for self-directed practice sessions. The self-directed portion was not a required element of the intervention and thus was not formally quantified. Postintervention Assessment. Upon completion of the educational programs, students were assessed in two areas: interpretation of FCU images and scanning technique and image acquisition. Ability to interpret FCU images was assessed using an online multiple-choice examination, structurally similar to the preintervention examination (20 questions or cases). Ten questions focused on basic cardiac anatomy and recognition of standard echocardiographic views and 10 questions on identification of one of the five defined pathologies. Different images from the preintervention test were used to ensure that any improvements in scores were a result of the teaching intervention and not specifically learning the test items. Students were required to report their confidence in each answer (on a scale ranging from 0% to 100% in 10% intervals). The technical ability of the students to accurately acquire highquality FCU images was assessed by a practical examination. Each student conducted an FCU examination on a healthy male patient within a 10-min time period. Students were instructed to acquire images using our eight-view FCU protocol: A2C, apical four-chamber, A5C, PLX, and PSX with cuts at the aortic valve, mitral valve, left ventricle, and LV apex. Data Analysis. Online examination scores were automatically tabulated and reported using our institutional database program
306 Cawthorn et al
(MEdTech). Each student was required to sign into the system to access the examinations, and their scores from each question were compiled for analysis. Confidence reports were collected, and the data were compiled. All data were analyzed by individual examination question. Compiled data sets were exported into SPSS version 17.0 for Windows for analysis. Student FCU scans were evaluated independently by three expert (ASE level III) staff echocardiographers who were blinded to the teaching intervention. Each FCU scan was evaluated using two distinct parameters: scan quality and accuracy. Image quality scores were used to evaluate students’ technical ability to acquire satisfactory images. Image accuracy scores were used as a supplementary measure to further assess students’ fundamental knowledge of scanning principles (i.e., probe location required to acquire particular FCU views). Image accuracy was not considered a measure of evaluation of scanning technique. Scan quality was assessed using a continuous scale with scores ranging from 0 (unacceptable/unreadable study) to 9 (ideal/best possible study). Accuracy was defined as a binary measure of whether the correct FCU view was acquired. To ensure a clear distinction between these two parameters, images that received unacceptable quality scores (zero) were automatically classified as inaccurate. Thus, only images that were of sufficient quality were further evaluated for accuracy. Each student was assigned an overall accuracy percentage score by determining the number of correctly identified views and dividing by the total views required (eight). Scores from each of the three evaluators were combined for analysis. Mean scores were calculated for each individual, pathology, and the cohort for both the preintervention and postintervention evaluations. Mean improvement was calculated for each of these categories. Paired t tests were used to compare improvement from preintervention to postintervention for individuals and the cohort. P values < .05 were deemed significant. An analysis of covariance allowed comparison between groups while adjusting for baseline scores. All tests were two sided. Interrater Reliability. To improve reliability, the mean of three assessors was used to obtain the student rating for each view and for the total number of correct views. The reliability coefficient of the raters’ mean was estimated as the between-student variance divided by the sum of the between-student variance and one third of the within-student variance.11 The variance components were estimated using an analysis-ofvariance model with random effects for students and raters.
Journal of the American Society of Echocardiography March 2014
Scanning Technique. The overall mean composite scanning score across the participants was 2.14 (using a scale on which 1 = excellent, 2 = good, 3 = fair, and 4 = poor). Using the repeated-measures analysis of variance, there were no statistically significant differences in overall score across the seven FCU views assessed. Phase 2 Participant Completion. All 45 students completed the preintervention examination; however, one student from educational program 1 did not complete the intervention and the postintervention examinations. Two students from educational program 3 did not complete the postintervention scanning assessment. Image Interpretation. There was a significant increase in mean examination scores (preintervention to postintervention) across all three educational programs (Table 4). The mean improvement in examination score across all three educational programs was 137%. No significant difference in examination score improvement was observed among the three educational programs (P = .65). Mean confidence scores also improved significantly across all three educational groups; no significant differences observed among the three programs (Table 5). Scanning Technique and Knowledge. The mean quality score of imaging by students in educational program 3 (2.58) was significantly lower than the mean quality scores from programs 1 and 2 (3.87 and 3.15, respectively) (Table 6). The overall mean accuracy score for all students was 82.6%. Mean accuracy scores were not significantly different between the three educational programs (Table 6). Across the eight FCU views assessed, there were significant differences in overall mean accuracy and quality for all students (Table 7). The two views with the highest mean quality scores were the PLX view and the PSX view with a cut at the left ventricle (3.98 and 3.73 respectively). The two views with the lowest mean quality scores were the A2C and A5C views (2.14 and 2.70 respectively). The two views that were imaged with the highest overall accuracy were the PSX view with a cut at the mitral valve and the PLX view (91.3% and 90.5%, respectively). The two views with the lowest accuracy scores were the PSX view with a cut at the aortic valve and the A2C view (78.6% and 67.5%, respectively). Interrater Reliability. The reliability coefficient of the mean rating of the eight views ranged from 0.73 to 0.92 (mean, 0.82). The reliability coefficient of the average assessment of the total number of correct views was also 0.82.
RESULTS Phase 1 Image Interpretation. All 12 first-year medical students demonstrated improvement in interpretation ability after the educational intervention. There was a significant improvement in mean examination score (preintervention to postintervention), with a mean improvement across the entire cohort of 85% (Table 4). There were differences in interpretation across the pathologies examined. Using the exact P value from Bowker’s test of symmetry, we found differences between pretest and posttest scores for the following pathologies: mitral regurgitation (P = .0039), LV hypertrophy (P = .0214), and recognition of absence of pathology (P = .0214). There were no significant differences for the other three pathologies (LV systolic dysfunction, pericardial effusion, and aortic valve dysfunction).
DISCUSSION Phase 1 of the study was conducted to determine the feasibility of improving FCU interpretation and image acquisition skills of firstyear medical students using a seven-view protocol. Our results demonstrated that first-year medical students were able to improve their ability to acquire and identify pathology using FCU, but this improvement was modest. There are no previous studies available with which to compare and contrast this level of improvement in first-year students. Although each individual student did show improvement in image interpretation, the mean postintervention examination score was quite low (61%). Furthermore, improvement was not observed across all
Cawthorn et al 307
Journal of the American Society of Echocardiography Volume 27 Number 3
Table 4 Mean percentage interpretation test scores from preintervention and postintervention examinations conducted in phase 1 and phase 2 Cohort
Preintervention
Postintervention
Improvement
P
Phase 1 cohort (n = 12) Phase 2 cohort (n = 44) Program 1 (n = 14) Program 2 (n = 15) Program 3 (n = 15)
33.0% 34.8% 37.7% 34.1% 32.6%
61.3% 82.6% 85.0% 82.1% 80.7%
86% 137% 125% 141% 148%
Development and Evaluation of Methodologies for Teaching Focused Cardiac Ultrasound Skills to Medical Students Thomas R. Cawthorn, MD, Curtis Nickel, MD, Michael O’Reilly, MD, Henryk Kafka, MD, James W. Tam, MD, Lynel C. Jackson, Anthony J. Sanfilippo, MD, and Amer M. Johri, MD, Kingston, Ontario, Canada; Winnipeg, Manitoba, Canada
Background: Handheld ultrasound is emerging as an important tool for point-of-care cardiac assessment. Although cardiac ultrasound skills are traditionally introduced during postgraduate training, the optimal time and methodology to initiate training in focused cardiac ultrasound (FCU) are unknown. The objective of this study was to develop and evaluate a novel curriculum for training medical students in the use of FCU. Methods: The study was conducted in two phases. In the first phase, 12 first-year medical students underwent FCU training over an 8-week period. In the second phase, 45 third-year medical students were randomized to one of three educational programs. Program 1 consisted of a lecture-based approach with scan training by a sonographer. Program 2 coupled electronic education modules with sonographer scan training. Program 3 was fully self-directed, combining electronic modules with scan training on a high-fidelity ultrasound simulator. Image interpretation skills and scanning technique were evaluated after each program. Results: First-year medical students were able to modestly improve interpretation ability and acquire limited scanning skills. Third-year medical students exhibited similar improvement in mean examination score for image interpretation whether a lecture-based program or electronic modules was used. Students in the selfdirected group using an ultrasound simulator had significantly lower mean quality scores than students taught by sonographers. Conclusions: Third-year medical students were able to acquire FCU image acquisition and interpretation skills after a novel training program. Self-directed electronic modules are effective for teaching introductory FCU interpretation skills, while expert-guided training is important for developing scanning technique. (J Am Soc Echocardiogr 2014;27:302-9.) Keywords: Handheld ultrasound, Medical education, Electronic learning, Simulation
Recent developments in ultrasound technology have enabled pointof-care cardiac assessment of patients using portable, handheld ultrasound (HHU) units. The American Society of Echocardiography (ASE) has recognized that these devices are capable of performing focused cardiac ultrasound (FCU) assessments as an adjunct to the physical examination. The ASE has also noted that comprehensive From the Queen’s University, School of Medicine, Kingston, Ontario, Canada (T.R.C., C.N., L.C.J.); Cardiovascular Imaging Network at Queen’s Research Group, Queen’s University, Kingston, Ontario, Canada (T.R.C., C.N., A.M.J.); Division of Cardiology, Queen’s University, Kingston, Ontario, Canada (M.O.R., H.K., A.J.S., A.M.J.); and Section of Cardiology, University of Manitoba, Winnipeg, Manitoba, Canada (J.W.T.). Drs Cawthorn and Nickel contributed equally to this work. Funded by the Cardiovascular Imaging Network at Queen’s University. Reprint requests: Amer M. Johri, MD, MSc, FRCPC, FASE, Kingston General Hospital, Division of Cardiology, FAPC3, 76 Stuart Street, Kingston, ON K7L 3N6, Canada (E-mail: [email protected]). 0894-7317/$36.00 Copyright 2013 by the American Society of Echocardiography. http://dx.doi.org/10.1016/j.echo.2013.12.006
302
echocardiographic examination requires image acquisition by trained sonographers and image interpretation by skilled echocardiographers.1 Although the opportunity for widespread use of HHU exists, the appropriate clinical applications of FCU are yet to be clearly defined. The potential clinical utility of FCU has been well established.2-8 However, no studies have attempted to elucidate the optimal timing and methodology to educate trainees in the use of FCU. Current research has focused on the application of FCU in clinical settings; published training programs have not been well validated. Several international bodies have recognized the need to train nonexpert imagers; however, a consensus does not exist regarding the most appropriate timing and method of training.1,9,10 Considering that this technology is available for use in a broad range of settings, a lack of standardized training is concerning. The majority of studies examining FCU have focused exclusively on the proficiency of attending staff members and postgraduate trainees. There has been comparatively little research focusing on introduction of FCU training during undergraduate medical education. Because medical school is the stage at which fundamental clinical examination skills are introduced, training in FCU may be most effective during this period. Considering the increasing prevalence of
Cawthorn et al 303
Journal of the American Society of Echocardiography Volume 27 Number 3
ultrasound use, competence in sonographic examination may A5C = Apical five-chamber become a core competency required of graduating medical ASE = American Society of students. Echocardiography The objective of this study A2C = Apical two-chamber was to develop and evaluate three novel curricula for training FCU = Focused cardiac medical students in the acquisiultrasound tion and interpretation of FCU. HHU = Handheld ultrasound The educational programs were LV = Left ventricular designed to provide students with a basic skill set for the assessPLX = Parasternal long-axis ment of cardiac patients using an PSX = Parasternal short-axis eight-view FCU protocol. The study was conducted in two phases to (1) assess the time at which the introduction of FCU skills would be appropriate and (2) assess the optimal methodology (didactic teaching, electronic modules, or simulation training) to deliver this fundamental knowledge. Abbreviations
Table 1 Cardiac pathologies that medical students were taught to recognize and identify Pathology
Interpretation Method
Aortic valve dysfunction (moderate to severe regurgitation or stenosis)
Abnormal leaflet thickening, mobility Conventional color flow with visual assessment
Mitral valve dysfunction (moderate to severe regurgitation or stenosis) LV hypertrophy (moderate to severe)
Conventional color flow with visual assessment
LV systolic dysfunction (moderate to severe) Pericardial effusion (moderate to severe)
Visual estimate of LV systolic contractility Visual estimate of pericardial space for presence/absence of effusion
Qualitative analysis on the basis of visual estimate of wall thickness
METHODS The study was conducted in two phases. The first phase was designed to establish the feasibility of first-year medical students to acquire FCU interpretation and image acquisition skills. The second phase was conducted in a larger cohort of third-year medical students and investigated different methods of delivering knowledge and skills, including electronic and simulation-based modules. The Queen’s University Research Ethics Board approved the study protocols, and written consent was obtained from all participants. Phase 1 Study Population. Twelve medical students from the Queen’s University School of Medicine were recruited to participate in this project during the summer break after their first year of medical education. Students had not completed their core training in cardiology. Exclusion criteria included any preexisting ultrasound or echocardiography experience. No students met these criteria, thus none were excluded. This study was extracurricular, and no compensation, monetary or academic credit, was provided for participation. HHU Device. The Vscan device (GE Healthcare, Horton, Norway) used in this study consists of a display unit and a broad-bandwidth, phased-array probe (total weight, 390 g). It provides twodimensional imaging and conventional color flow echocardiographic images. It has limited controls for adjusting image depth and gain. Electronic calipers and a touchpad enable the user to make distance and area measurements. All images can be frozen and scrolled for review, but there is no electrocardiographic gating. Images may be saved in still or video format. Cardiac Pathologies. Five cardiac pathologies were examined: left ventricular (LV) systolic dysfunction, LV hypertrophy, aortic valve dysfunction (regurgitation and stenosis), mitral valve dysfunction (regurgitation and stenosis), and pericardial effusion (Table 1). The cardiac pathologies were selected on the basis of consensus by three ASE level III echocardiographers at our institution. All FCU images provided to students for assessment depicted abnormalities rated as moderate or severe by ASE level III echocardiographers; this level
of pathology was considered representative of clinically relevant abnormalities. Students were not expected to grade pathologies on the basis of severity; assessment of images was based exclusively on identification of the presence of a moderate or severe abnormality. Preintervention Evaluation. Participants underwent an evaluation of FCU interpretation knowledge before commencing the educational intervention. This evaluation consisted of 15 online case-based multiple-choice questions. Each question consisted of a one-line clinical vignette and a video showing an FCU assessment. Participants were then asked to identify the presence of major cardiac pathology if any was present. Three examples of each of the five cardiac pathologies were presented. Educational Intervention. The educational intervention was created in consultation with National Board of Echocardiography, ASE level III, and FASE certified echocardiographers and American Registry for Diagnostic Medical Sonography certified sonographers. The program was designed to instruct the participants in image acquisition and interpretation of focused cardiac imaging using HHU units. The educational intervention included instruction in image acquisition using a focused protocol consisting of seven specific views: apical two-chamber (A2C), apical four-chamber, apical five-chamber (A5C), parasternal long-axis (PLX), and parasternal short-axis (PSX) with cuts at the aortic valve, mitral valve, and midchamber left ventricle (PSX-LV). The intervention occurred over an 8-week period with a 2-hour session each week. Each 2-hour session was divided into a didactic and a practical component. The eight didactic lectures focused on instruction surrounding general cardiac anatomy, general cardiac ultrasound, device use, and the five pathologies described in Table 1. All lectures were delivered by ASE level III echocardiographers. Six 1-hour practical instructional sessions focused on FCU scanning technique. Practical sessions were taught by both ASE level III echocardiographers and American Registry for Diagnostic Medical Sonography certified sonographers. Participants were instructed in the use of the HHU unit and proper sonography technique to improve their image accuracy and quality. Participants worked in pairs
304 Cawthorn et al
or small groups with the HHU units to develop their sonographic technique under the direct supervision of the instructors. The remaining two 1-hour practical sessions focused on teaching cases developed from FCU examinations of hospital inpatients. The final element of the educational intervention involved self-directed learning; participants were provided unlimited access to the HHU units and dedicated examination rooms for self-directed practice sessions. The self-directed portion was not a required element of the intervention and thus was not formally quantified. Postintervention Evaluation. Evaluation of the education intervention was composed of two elements: (1) interpretation knowledge and ability and (2) image acquisition accuracy and quality. Interpretation knowledge and ability was evaluated using an online 15-question, case-based, multiple-choice examination that was identical in structure to the preintervention examination described above. Image acquisition was assessed via an objective structured clinical examination, during which participants were required to obtain seven defined views within a 15-min time period on a healthy volunteer patient. These views were captured, downloaded, and compiled for scoring. Each view was evaluated by an ASE level III echocardiographer blinded to the HHU unit operator. Each view was assigned a composite score on the basis of subjective image quality (1 = excellent, 2 = good, 3 = fair, 4 = poor). Data Analysis. Multiple-choice examination scores were compiled and tabulated. Data were exported into SPSS version 17.0 for Windows (SPSS, Inc, Chicago, IL) for analysis. Mean scores were calculated for each individual, pathology, and the cohort for both the preintervention and postintervention evaluations. Mean improvement was calculated for each of these categories. Paired t tests were used to compare improvement from preintervention to postintervention for individuals and the cohort. P values < .05 were deemed significant. Improvement in scores by pathology was compared from before to after the intervention. Exact P values were obtained using Bowker’s test of symmetry. Mean scanning scores were tabulated for each participant, view, and the cohort and compared using repeated-measures analysis of variance. Phase 2 Study Population. Forty-five third-year medical students from the Queen’s University School of Medicine were recruited to participate in this study. All students were in their third year of medical school and had completed their core teaching unit in cardiology before the study. Exclusion criteria were identical to those of phase 1; none of the phase 2 students had participated in phase 1. Students were randomized into one of three educational programs using a random-number generator and were assigned unique identification numbers for the duration of the study. Cardiac Pathologies. The same cardiac pathologies and interpretation methodology used in phase 1 were used in phase 2 (Table 1). Preintervention Assessment. After randomization, baseline knowledge of FCU was assessed using an online multiple-choice examination comprising 20 questions, each based on a different FCU cine movie. Ten cases involved normal cases and required students to identify the FCU view shown or identify an aspect of cardiac anatomy. The remaining 10 cases involved two examples of each of the five pathologies; for aortic and mitral valve dysfunction, one example of regurgitation and one example of stenosis was used for each valve
Journal of the American Society of Echocardiography March 2014
(Table 1). Students were required to report their confidence in each answer (on a scale ranging from 0% to 100% in 10% intervals). Educational Interventions. A novel 4-week FCU curriculum was developed through collaboration with ASE level III echocardiographers and experienced medical educators. This curriculum was delivered via three distinct educational programs (Table 2). Each education program consisted of two components: a ‘‘fundamental education unit’’ that focused on teaching the fundamental principles of cardiac ultrasound and image interpretation and a ‘‘scanning education unit’’ that focused on instruction of FCU scanning technique and image acquisition. Scanning education focused on eight views: the seven views used in phase 1 and a PSX cut of the LV apex. All training sessions were voluntary and conducted entirely outside of the standard medical school curriculum. Sessions were conducted in the School of Medicine building at Queen’s University. To evaluate compliance with the educational protocol, attendance was taken at each training session; however, compliance with self-directed learning components of the curriculum was not formally assessed. Didactic Education. The didactic component of the educational programs was delivered in four 2-hour lectures over the course of 2 weeks. These sessions were conducted by two expert (ASE level III) staff echocardiographers with extensive experience in medical education. Each session was designed to address specific topics and meet predefined objectives that aligned with the overall educational program goals (Table 3). Didactic content was delivered using interactive PowerPoint presentations (Microsoft Corporation, Redmond, WA) and case-based discussion in a small-group format (15 students). Electronic Learning Modules. Three interactive online electronic learning modules were designed specifically for this study to deliver identical content as the didactic lectures. These modules were created using the MEdTech platform, an electronic interface purpose-built for the Queen’s University School of Medicine. The modules were designed by study investigators in conjunction with medical educators and an expert MEdTech programmer and Web developer. Modules were accessible to students in educational programs 2 and 3 for a defined 2-week period during the study. Students were able to access the modules at any time during the study period (either on campus or from remote locations using their institutional accounts). The first module introduced the basic fundamentals of FCU, including physics, clinical applications, and FCU scanning theory. It was designed to provide foundational knowledge regarding FCU to the participants, who had no previous ultrasound or echocardiography experience. This module was intended to be completed over approximately 2 hours. The second module focused on developing image acquisition skills with the HHU device. The basic functions of the device were outlined using diagrams, video, and text. The standard echocardiographic views to be included in the focused cardiac assessment were described with text and example videos obtained using the HHU device. A major component of this module was a series of nine tutorial videos that led the students through a complete FCU assessment on a patient using the HHU device. This included instructional videos on patient positioning, positioning of the probe for each view, cardiac anatomy using FCU, and troubleshooting. This module was intended to be completed over approximately 4 hours.
Cawthorn et al 305
Journal of the American Society of Echocardiography Volume 27 Number 3
Table 2 Comparison of the components of each educational program used in phase 2
Fundamental education Scanning education
Educational program 1
Educational program 2
Educational program 3
Didactic sessions Sonographer training
Electronic modules Sonographer training
Electronic modules Simulation training
Table 3 Outline of didactic lectures sessions (phase 2, educational program 1). Session
Topic/content
Objective
1 (2 h)
Introduction to fundamental principles of FCU Review of cardiac anatomy
Develop an understanding of basic FCU principles and theory Establish baseline knowledge of cardiac anatomy and physiology as related to FCU
2 (2 h)
Basic echocardiographic views Introduction to scanning technique
3 (2 h)
Valvular abnormalities (aortic and mitral)
Recognize eight standard echocardiographic views (A2C, A4C, A5C, PLX, PSX-AV, PSX-MV, PSXLV, PSX-A) Develop fundamental scanning technique using the HHU device Develop an approach to recognizing basic valvular abnormalities on FCU (aortic stenosis/regurgitation and mitral stenosis/ regurgitation)
4 (2 h)
LV systolic dysfunction and hypertrophy Pericardial effusion
Develop an approach to recognizing global LV systolic dysfunction and LV hypertrophy Learning to recognize signs of pericardial effusion
A4C, Apical four-chamber; PSX-A, PSX view with a cut at the LV apex; PSX-AV, PSX view with a cut at the aortic valve; PSX-LV, PSX view with a cut at the left ventricle; PSX-MV, PSX view with a cut at the mitral valve.
The third module provided education on the interpretation of the predetermined pathologies. A systematic approach to image interpretation was described. In addition, a brief review of the various imaging views was provided. The predefined pathologies were described in terms of anatomy, etiologies, and pathophysiology. Annotated figures and videos of specific pathology were provided as the main instructional tool. Finally, video examples with annotations were provided for self-assessment. This module was intended to be completed over approximately 4 hours. Sonographer-Directed Scanning Tutorials. Students participated in two hands-on training sessions (2 hours each) conducted by an American Registry for Diagnostic Medical Sonography certified sonographer. The sonographers were instructed by study coordinators to educate the students within the parameters of the study (focusing on specific views and techniques). Sonographers provided demonstrations in scanning technique and troubleshooting methods.
Students developed their scanning technique by practicing on one another in small groups, under the supervision and guidance of the sonographer using a HHU unit. Simulation-Based Scanning Sessions. The simulation sessions were not led by an expert instructor; all sessions were self-directed after a brief orientation of the simulation system by a study coordinator. Students participated in two self-directed hands-on training sessions (2 hours each) using a CAE Vimedix Ultrasound Simulator System (CAE Healthcare, Montreal, Quebec, Canada). During these sessions, students used the simulation system to develop their scanning technique and ability to identify pathologies. This high-fidelity simulation system has multiple learning modalities and settings, enabling students to progressively learn the basics of scanning technique. The simulation system allows students to practice scanning on ‘‘normal’’ patients and patients with a variety of preprogrammed cardiac pathologies, including LV systolic dysfunction, valvular dysfunction, and pericardial effusion. The simulation system uses a software-to-software probe-phantom system and also allows for simultaneous ‘‘cutaway’’ anatomic illustrations on a second viewing screen adjacent to the screen on which the ultrasound image is acquired. Self-Directed Learning. As in phase 1, throughout the 4-week study, students in all educational programs were able to access HHU units to conduct self-directed practice in scanning. Participants were provided unlimited access to the HHU units and dedicated examination rooms for self-directed practice sessions. The self-directed portion was not a required element of the intervention and thus was not formally quantified. Postintervention Assessment. Upon completion of the educational programs, students were assessed in two areas: interpretation of FCU images and scanning technique and image acquisition. Ability to interpret FCU images was assessed using an online multiple-choice examination, structurally similar to the preintervention examination (20 questions or cases). Ten questions focused on basic cardiac anatomy and recognition of standard echocardiographic views and 10 questions on identification of one of the five defined pathologies. Different images from the preintervention test were used to ensure that any improvements in scores were a result of the teaching intervention and not specifically learning the test items. Students were required to report their confidence in each answer (on a scale ranging from 0% to 100% in 10% intervals). The technical ability of the students to accurately acquire highquality FCU images was assessed by a practical examination. Each student conducted an FCU examination on a healthy male patient within a 10-min time period. Students were instructed to acquire images using our eight-view FCU protocol: A2C, apical four-chamber, A5C, PLX, and PSX with cuts at the aortic valve, mitral valve, left ventricle, and LV apex. Data Analysis. Online examination scores were automatically tabulated and reported using our institutional database program
306 Cawthorn et al
(MEdTech). Each student was required to sign into the system to access the examinations, and their scores from each question were compiled for analysis. Confidence reports were collected, and the data were compiled. All data were analyzed by individual examination question. Compiled data sets were exported into SPSS version 17.0 for Windows for analysis. Student FCU scans were evaluated independently by three expert (ASE level III) staff echocardiographers who were blinded to the teaching intervention. Each FCU scan was evaluated using two distinct parameters: scan quality and accuracy. Image quality scores were used to evaluate students’ technical ability to acquire satisfactory images. Image accuracy scores were used as a supplementary measure to further assess students’ fundamental knowledge of scanning principles (i.e., probe location required to acquire particular FCU views). Image accuracy was not considered a measure of evaluation of scanning technique. Scan quality was assessed using a continuous scale with scores ranging from 0 (unacceptable/unreadable study) to 9 (ideal/best possible study). Accuracy was defined as a binary measure of whether the correct FCU view was acquired. To ensure a clear distinction between these two parameters, images that received unacceptable quality scores (zero) were automatically classified as inaccurate. Thus, only images that were of sufficient quality were further evaluated for accuracy. Each student was assigned an overall accuracy percentage score by determining the number of correctly identified views and dividing by the total views required (eight). Scores from each of the three evaluators were combined for analysis. Mean scores were calculated for each individual, pathology, and the cohort for both the preintervention and postintervention evaluations. Mean improvement was calculated for each of these categories. Paired t tests were used to compare improvement from preintervention to postintervention for individuals and the cohort. P values < .05 were deemed significant. An analysis of covariance allowed comparison between groups while adjusting for baseline scores. All tests were two sided. Interrater Reliability. To improve reliability, the mean of three assessors was used to obtain the student rating for each view and for the total number of correct views. The reliability coefficient of the raters’ mean was estimated as the between-student variance divided by the sum of the between-student variance and one third of the within-student variance.11 The variance components were estimated using an analysis-ofvariance model with random effects for students and raters.
Journal of the American Society of Echocardiography March 2014
Scanning Technique. The overall mean composite scanning score across the participants was 2.14 (using a scale on which 1 = excellent, 2 = good, 3 = fair, and 4 = poor). Using the repeated-measures analysis of variance, there were no statistically significant differences in overall score across the seven FCU views assessed. Phase 2 Participant Completion. All 45 students completed the preintervention examination; however, one student from educational program 1 did not complete the intervention and the postintervention examinations. Two students from educational program 3 did not complete the postintervention scanning assessment. Image Interpretation. There was a significant increase in mean examination scores (preintervention to postintervention) across all three educational programs (Table 4). The mean improvement in examination score across all three educational programs was 137%. No significant difference in examination score improvement was observed among the three educational programs (P = .65). Mean confidence scores also improved significantly across all three educational groups; no significant differences observed among the three programs (Table 5). Scanning Technique and Knowledge. The mean quality score of imaging by students in educational program 3 (2.58) was significantly lower than the mean quality scores from programs 1 and 2 (3.87 and 3.15, respectively) (Table 6). The overall mean accuracy score for all students was 82.6%. Mean accuracy scores were not significantly different between the three educational programs (Table 6). Across the eight FCU views assessed, there were significant differences in overall mean accuracy and quality for all students (Table 7). The two views with the highest mean quality scores were the PLX view and the PSX view with a cut at the left ventricle (3.98 and 3.73 respectively). The two views with the lowest mean quality scores were the A2C and A5C views (2.14 and 2.70 respectively). The two views that were imaged with the highest overall accuracy were the PSX view with a cut at the mitral valve and the PLX view (91.3% and 90.5%, respectively). The two views with the lowest accuracy scores were the PSX view with a cut at the aortic valve and the A2C view (78.6% and 67.5%, respectively). Interrater Reliability. The reliability coefficient of the mean rating of the eight views ranged from 0.73 to 0.92 (mean, 0.82). The reliability coefficient of the average assessment of the total number of correct views was also 0.82.
RESULTS Phase 1 Image Interpretation. All 12 first-year medical students demonstrated improvement in interpretation ability after the educational intervention. There was a significant improvement in mean examination score (preintervention to postintervention), with a mean improvement across the entire cohort of 85% (Table 4). There were differences in interpretation across the pathologies examined. Using the exact P value from Bowker’s test of symmetry, we found differences between pretest and posttest scores for the following pathologies: mitral regurgitation (P = .0039), LV hypertrophy (P = .0214), and recognition of absence of pathology (P = .0214). There were no significant differences for the other three pathologies (LV systolic dysfunction, pericardial effusion, and aortic valve dysfunction).
DISCUSSION Phase 1 of the study was conducted to determine the feasibility of improving FCU interpretation and image acquisition skills of firstyear medical students using a seven-view protocol. Our results demonstrated that first-year medical students were able to improve their ability to acquire and identify pathology using FCU, but this improvement was modest. There are no previous studies available with which to compare and contrast this level of improvement in first-year students. Although each individual student did show improvement in image interpretation, the mean postintervention examination score was quite low (61%). Furthermore, improvement was not observed across all
Cawthorn et al 307
Journal of the American Society of Echocardiography Volume 27 Number 3
Table 4 Mean percentage interpretation test scores from preintervention and postintervention examinations conducted in phase 1 and phase 2 Cohort
Preintervention
Postintervention
Improvement
P
Phase 1 cohort (n = 12) Phase 2 cohort (n = 44) Program 1 (n = 14) Program 2 (n = 15) Program 3 (n = 15)
33.0% 34.8% 37.7% 34.1% 32.6%
61.3% 82.6% 85.0% 82.1% 80.7%
86% 137% 125% 141% 148%
