Providing Initial Transthoracic Echocardiography Training for Anesthesiologists: Simulator Training Is Not Inferior to Live Training Thomas Edrich, MD, PhD,* Raghu R. Seethala, MD,† Benjamin A. Olenchock, MD,‡ Annette K. Mizuguchi, MD, PhD,* Jose M. Rivero, MD,‡ Sascha S. Beutler, MD, PhD,* John A. Fox, MD,* Xiaoxia Liu, MS,* and Gyorgy Frendl, MD, PhD* Objective: Transthoracic echocardiography (TTE) is finding increased use in anesthesia and critical care. Efficient options for training anesthesiologists should be explored. Simulator mannequins allow for training of manual acquisition and image recognition skills and may be suitable due to ease of scheduling. The authors tested the hypothesis that training with a simulator would not be inferior to training using a live volunteer. Design: Prospective, randomized trial. Setting: University hospital. Participants: Forty-six anesthesia residents, fellows, and faculty. Interventions: After preparation with a written and video tutorial, study subjects received 80 minutes of TTE training using either a simulator or live volunteer. Practical and written tests were completed before and after training to assess improvement in manual image acquisition skills and theoretic knowledge. The written test was repeated 4 weeks later.

Measurements and Main Results: Performance in the practical image-acquisition test improved significantly after training using both the live volunteer and the simulator, improving by 4.0 and 4.3 points out of 15, respectively. Simulator training was found not to be inferior to live training, with a mean difference of –0.30 points and 95% confidence intervals that did not cross the predefined noninferiority margin. Performance in the written retention test also improved significantly immediately after training for both groups but declined similarly upon repeat testing 4 weeks later. Conclusions: When providing initial TTE training to anesthesiologists, training using a simulator was not inferior to using live volunteers. & 2014 Elsevier Inc. All rights reserved.

P

Randomization of subjects into groups Sim and Live was performed by first numbering the participants in order of registration. Then, by 1 toss of a coin, all even-numbered subjects were assigned to either the simulator or the live training only, leaving all odd-numbered subjects to train using the other modality. Prescripted instruction was provided by 2 instructors on each course date who were pulled from a pool of 3 instructors as available (TE, RS, BO). All instructors had formal training in TTE. (TE completed the Certificate in Critical Care Ultrasonography from the American College of Chest Physicians, RS is certified by the American College of Emergency Physicians, and BO is a cardiologist.) The simulator and live training arms both had 1 instructor at all times. Instructors were assigned randomly by the toss of a coin to start with either the simulator or the live training and then to switch at half time (40 minutes) to provide equal exposure for all subjects. Simulator training was provided using an echocardiography simulator (Vimedix, CEA Healthcare Inc., Montreal, Canada), which allows the user to manipulate a TTE transducer over the chest of a mannequin and visualize the resulting 2-dimensional image on the screen.4 Live training was provided using a volunteer with normal stature and heart. The written and video tutorial were designed to teach focused TTE to physicians who already have had exposure to basic principles of ultrasonography (but not specifically to TTE). The 18-page written tutorial reviews basic principles of echocardiography and presents 5 basic 2-dimensional TTE views (FATE views).5 Examples of patients with hemodynamic compromise (eg, hypovolemia, LV, or RV failure)

OINT-OF-CARE transthoracic echocardiography (TTE) for diagnosis and monitoring of critically ill patients is finding increasing use across different specialties1,2 including the perioperative period.3 Efficient and scalable methods are needed to train anesthesiologists to meet this increasing demand. Recently, sophisticated mannequin-based simulators of TTE have evolved allowing the user to practice the manual and cognitive skill of image acquisition in the setting of both normal and pathologic findings.4 Yet, it is unclear whether simulator training can replace traditional training using a live volunteer when teaching basic TTE to novices. The authors’ primary hypothesis was that simulator training would not be inferior to live training with regard to the manual skill of image acquisition. The secondary hypothesis was that theoretic knowledge gained during each type of training modality would be similar. METHODS Approval of this prospective interventional study was granted by the institutional review board of the authors’ institution. Consent was implied when subjects registered for the study. Inclusion criteria included any resident, fellow, or staff physician in the department of anesthesia. Physicians who previously had performed more than 2 full TTE exams were excluded. Likewise, any physician with formal training in transesophageal echocardiography was excluded. Fig 1 outlines the design of the study. Up to 8 subjects were recruited for each training session. They received a written and video tutorial 2 weeks before the TTE course for self study. On the day of the course, all participants completed both a written and a practical pre-test. Subsequently, participants were randomized equally into 2 groups to receive training on either the simulator (designated group Sim) or the live volunteer (group Live) with a dedicated instructor for each group. Immediately after training, both groups repeated a written and practical post-test. Four weeks after the course, another written test was administered to assess the retention of theoretic knowledge.

KEY WORDS: echocardiography, point-of-care, ultrasound, education

From the Department of *Anesthesia, Perioperative and Pain Medicine, †Emergency Medicine, and ‡Medicine Division of Cardiology, Brigham and Women’s Hospital, Boston, MA. Address reprint requests to Thomas Edrich, MD, PhD, Brigham Women’s Hospital, Department of Anesthesia, Perioperative and Pain Medicine, 75 Francis Street, Boston, MA 02115. E-mail: [email protected] © 2014 Elsevier Inc. All rights reserved. 1053-0770/2601-0001$36.00/0 http://dx.doi.org/10.1053/j.jvca.2013.07.011

Journal of Cardiothoracic and Vascular Anesthesia, Vol 28, No 1 (February), 2014: pp 49–53

49

50

EDRICH ET AL

practical test, the scores for all 5 required views were added for a total score between 0 and 15. The power analysis involving the primary hypothesis was based on preliminary experience in teaching and testing 6 inexperienced individuals, finding a standard deviation of 2.2 total score points. Based on this observed variability and the clinical judgment of the authors, a difference in improvement of image acquisition skills in excess of 2 total score points (out of 15 possible points) was declared to be clinically meaningful. Thus, the a priori non-inferiority margin was declared to be 2. Using a 1-sided, 2-sample t-test, a sample size of 17 subjects would be required for each group to detect an inferiority of group Sim compared to group Live beyond 2 points with a power of 0.8. All statistical analyses were performed using Matlab (MathWorks, Natick, MA). For data with normal distributions as verified by the Jarque-Bera test, comparisons were performed using 2-sample t-tests or ANOVA with Bonferroni adjustment for multiple comparisons. Wilcoxon Rank Sum test and Kruskal-Wallis ANOVA were used for nonparametric data. Klippendorff’s alpha coefficient was calculated to assess for inter-rater concordance.7 A significance level of p o 0.05 was used for all comparisons.

RESULTS

Fig 1. Schematic of the study including numbers of subjects completing each stage.

are included in this tutorial as well as a limited refresher on ultrasound physics, resolution optimization, and common artifacts. The 12-minute video tutorial6 demonstrates the acquisition of 5 basic imaging views, first using the echocardiography simulator mannequin and then on a live volunteer using a Sonosite M-Turbo ultrasound machine with a P21, 1-5 mHz phased-array cardiac transducer (Sonosite Inc., Bothell, WA). The written pre-, post-, and retention tests were designed as follows: One 40-question multiple-choice examination was designed by 4 coauthors who hold diplomate certification for advanced perioperative transesophageal echocardiography by the National Board of Echocardiography (TE, AM, SB and JF). Three versions of this written test were created by randomizing both the order of questions and the order of multiple-choice answers within each question to make test versions A, B, and C. During the closed-book written tests, subjects received either version A or B for the pre-test and then the other version for the post-test. All subjects received version C for the retention test. The practical pre- and post-tests were designed to test the ability to rapidly acquire each of the 5 standard TTE views. Using only 1 volunteer for all pre- and post-tests (TE), the test administrator would prompt the subject to acquire each of the 5 required TTE views (parasternal long-axis view, parasternal short-axis view, apical 4chamber view, subcostal 4-chamber view, or inferior vena cava [IVC] view). The time to achieve each view was limited to 20 seconds. Four-second clips were stored by the test administrator as soon as the subject announced that the image was of appropriate quality or at the conclusion of 20 seconds. The de-identified clips were scored separately and in random order by TE and RS after the conclusion of the study according to a scoring system that was developed by the authors (Table 1). The score for each view ranged from 0 to 3. For each

Forty-six subjects who had no significant prior experience with any form of echocardiography were enrolled and were instructed in 6 study sessions over a 6-month period. For each session, groups Sim and Live ranged from 3 to 4 participants each. Thus, the instructor-to-trainee ratio was 1:3 to 1:4 for the 80-minute training period, allowing for 20 to 25 minutes of hands-on time for each trainee while the other trainees observed. Over the course of the 6-month period, 23 subjects were enrolled into groups Live and Sim, respectively, as indicated in Fig 1. The median postgraduate training years of groups Live and Sim were 4 and 5 years, which was not significantly different by Wilcoxon Rank Sum test (p ¼ 0.39). Groups Live and Sim contained 52% and 61% males, respectively. This was not significantly different by Fisher exact test (p ¼ 0.21). No subjects previously had completed any formal training or full TTE exams as defined by having performed all of the views tested in this study. No subject had significant training in transesophageal echocardiography beyond the exposure expected during anesthesia residency. All subjects had viewed the video and written tutorial within 2 weeks of the study date. All subjects completed both the practical and written pre- and post-tests. Three subjects did not complete the written retention test (1 in group Live and 2 in Sim). After conclusion of the study, all clips were independently scored by 2 examiners (TE and RS). Agreement between the 2 examiners was assessed using the Krippendorff test for interrater reliability, finding an alpha of 0.85 with 95% confidence intervals of 0.81 to 0.88 based on 10,000 bootstrap samples.7 This is categorized as “almost-perfect agreement.”7,8 Thus, subsequent statistical analysis was performed using the average scores of both examiners. Fig 2 shows the image-acquisition scores for each group before and after training given as means and 95% confidence intervals as analyzed with ANOVA. The baseline scores of groups Live and Sim did not differ significantly. After training, groups Live and Sim improved their scores by an average of

51

SIMULATOR V LIVE TRANSTHORACIC ECHOCARDIOGRAPHY TRAINING

Table 1. Image Quality Criteria for Basic TTE Views (“Quality Points”) TTE View

Parasternal long-axis view

Parasternal shortaxis view

Apical 4-chamber view

Subcostal 4-chamber view

Subcostal view of IVC

Criteria

Points

Similar to “gold standard” quality. RVOT, Ao, LA, and LV seen well. One chamber missing or severely foreshortened, aortic or mitral valve not visualized well. Poor quality, cannot assess LV contractility reasonably well (not enough LV visualized), cannot assess RVOT reasonably well. Structures not recognizable Similar to “gold standard” quality. Round LV with papillary muscles, RV visualized similarly to “gold standard” view. View is too basal or too apical without proper visualization of papillary muscles, not proper short axis (ie, oval LV), or with significant lateral wall dropout (of LV). And At least part of RV visualized Poor quality. Both LV and RV not visualized properly. Structures not recognizable. Similar to “gold standard” quality. 4 chambers visualized. Not more dropout of RV of LV free walls than in “gold standard,” no foreshortening of LV. One atrium missing or aortic outflow added (5-chamber view), LV foreshortened, apices or portion of LV free wall missing, ventricular septum not close to vertical (deviated by 415%). Poor quality, LV or RV not visualized properly. Structures not recognizable. Similar to “gold standard” quality. 4 chambers visualized. One atrium missing, 5-chamber view, significantly foreshortening of LV. Poor quality, LV or RV not visualized properly. Structures not recognizable. Similar to “gold standard” quality. IVC seen to connect to RA. No clear connection of IVC to RA. Poor quality, IVC not clearly visualized in longitudinal fashion. Structures not recognizable, aorta imaged instead of IVC.

3 2 1 0 3 2

1 0 3 2 1 0 3 2 1 0 3 2 1 0

NOTE. The points for the 5 required views are added for a maximum of 15 points. Note that the “gold standard” refers to a TTE performed by the sonographer (JR) on the live volunteer used for all tests (TE). Abbreviations: Ao, aorta; IVC, inferior vena cava; LA, left atrium; LV, left ventricle; RA, right atrium; RVOT, right ventricular outflow tract; TTE, transthoracic echocardiography.

4.0 and 4.3 points, respectively. The difference between the average improvement of groups Live and Sim was assessed with a 2-sample t-test finding a mean difference of -0.30 points with a 95% confidence interval of −2.02 to 1.41. Fig 3 presents these results in relation to the apriori non-inferiority margin of 2.0 in accordance with the CONSORT Statement.9 When subtracting the score improvement after training of group Live from the improvement in group Sim, the mean and 95% confidence interval lie entirely to the left of the non-inferiority margin and span zero, denoting that simulation training is not inferior to live training. Fig 4 shows the results of the written pre-, post-, and retention tests. The medians of the delays between the study

Fig 2. Practical image acquisition skills tested before and after training for groups undergoing live training (Live) or simulator training (Sim). The scores are the sums of the 5 basic TTE views, allowing for a maximal score of 15. Non-overlapping 95% confidence intervals indicate significant improvements in groups Sim and Live.

date and the retention test were 44 and 34 days for groups Live and Sim, respectively, and these were not significantly different (p ¼ 0.44 by Wilcoxon Rank Sum test). In both groups, the performance in the written test improved significantly immediately after training as shown by non-overlapping 95% confidence intervals in Fig 4. Both groups demonstrated a similar decline in performance in the retention test. DISCUSSION

In this study, training of basic image-acquisition skills using a simulator was found to be non-inferior to traditional training using a live volunteer when providing initial TTE training to

Fig 3. The difference between the improvement in groups Live and Sim presented as a mean with 95% confidence intervals. Note that Live minus Sim spans zero and lies entirely to the left of the non-inferiority margin of 2.0. This indicates non-inferiority of simulator training compared to live training based on the a priori noninferiority margin.9

52

EDRICH ET AL

Fig 4. Performance in the written pre-, post-, and retention tests shown as means and 95% confidence intervals. Non-overlapping intervals indicate a significant improvement immediately after training (p o 0.05). However, there was a deterioration of theory knowledge when tested 4 weeks later with the retention test. Note that all subjects completed the pre- and post-tests, but some did not complete the retention test as indicated by the percentages on the right.

anesthesia physicians. In addition, groups Live and Sim both improved significantly between the written pre- and post-tests, which measured familiarity with ultrasound physics and image acquisition. The retention of the theory knowledge gained during the course declined similarly in both groups. This suggests that continued training and exposure to TTE is necessary to maintain the benefit derived from a short training course. The authors’ training course was designed to be completed in 1 evening with most time spent either performing or observing others perform exams with a high ratio of instructor to trainee (1:3 or 1:4). Preparatory “self-study” training was expected of each trainee before the course utilizing both a written and video tutorial. This made the overall training program as efficient as possible. Simulation training for echocardiography recently was shown to have the potential to satisfy increasing training demands as outlined by Matyal et al.4 Simulation training also was compared to lecture-based training by Bose et al, finding that simulation was better at teaching theory regarding chamber identification and placement of the transducer.10 Only 1 study is known to the authors that compares simulator to live training for TTE. This study by Neelankavil et al compared simulator training first to a lecture-based didactic, finding that residents in the simulator arm were superior at image acquisition and anatomic identification. Then, a subgroup underwent further training but with the control group receiving hands-on training on a live volunteer. Again, the simulator group maintained superior performance.11 However, the different study design with more overall hands-on training in the simulator group than in the control group makes a direct comparison to the authors’ study difficult as the authors focused on a single training

session instead of sequential teaching methods. This may explain the authors’ divergent finding that simulator and live training were not significantly different. One limitation of the authors’ study was the inherently subjective method of quantifying the image acquisition skills. Although a qualitative scoring system has been published using expert evaluation,11 no quantitative score is known to the authors. The scoring system developed for this study focused on the ability to acquire clips with sufficient quality to be useful for assessment of hemodynamic status. It was developed by cardiac anesthesiologists (TE, AM, SB, JF) with certification by the American National Board of Echocardiography together with 2 cardiologists (BO, JR). A high level of concordance between the grading by the 2 examiners was demonstrated by a Krippendorff test, thus demonstrating the consistency of this score. The trainees were required to obtain high-quality images in a short period of time. This time restraint is consistent with the current use of point-of-care cardiac ultrasound, which is useful for rapid guidance of hemodynamic management according to the focused echocardiography entry level (FEEL) concept12 rather than to deliver precise diagnostic images. For example, the use of point-of-care TTE during hemodynamic resuscitation in hospital13 or pre-hospital settings14 has been described recently with protocols that require image acquisition within 10 seconds so as not to interfere with ongoing cardiopulmonary resuscitation. The improved but relatively low post-test scores achieved after training (Fig 2) may be a reflection of the challenge imposed by this time limit. The predictable and stylized images delivered by the simulator are intended to reinforce the skills for obtaining basic TTE views; however, they have limited value for practicing imaging for challenging conditions encountered with some patients such as those with obesity or emphysema. Finally, the training delivered in this study encompassed only the 5 basic 2-dimensional views described by the FEEL concept and included only normal findings. Future training in the setting of cardiac pathology may be greatly facilitated with the use of simulator mannequins since volunteers or patients with corresponding pathology may be more difficult to recruit for a large number of such training sessions. The acquisition cost of the simulator used in this study was similar to the cost of a single portable ultrasound as described above. The authors’ study provides evidence that simulation training is not inferior to traditional teaching of basic TTE skills to physicians without prior experience. Considering the challenges of scheduling trainees and volunteers for traditional teaching and considering the increasing availability of highfidelity simulators, training with a simulator may be an efficient alternative method to acquire this skill.

REFERENCES 1. Beaulieu Y: Bedside echocardiography in the assessment of the critically ill. Crit Care Med 35:S235-S249, 2007 2. Schmidt GA: ICU ultrasound. The coming boom. Chest 135: 1407-1408, 2009 3. Cowie B: Focused cardiovascular ultrasound performed by anesthesiologists in the perioperative period: Feasible and alters patient management. J Cardiothorac Vasc Anesth 23:450-456, 2009

4. Matyal R, Bose R, Warraich H, et al: Transthoracic echocardiographic simulator: Normal and the abnormal. J Cardiothorac Vasc Anesthesia 25:177-181, 2011 5. Jensen MB, Sloth E, Larsen KM, et al: Transthoracic echocardiography for cardiopulmonary monitoring in intensive care. Eur J Anaesthesiol 21:700-707, 2004 6. Edrich T: Basic video TTE tutorial. http://youtu.be/GcjFyRPlISw. Accessed June 11, 2013

SIMULATOR V LIVE TRANSTHORACIC ECHOCARDIOGRAPHY TRAINING

7. Hayes AF, Krippendorff K: Answering the call for a standard reliability measure for coding data. Commun Methods Meas 1:77-89, 2007 8. Landis JR, Koch GG: The measurement of observer agreement for categorical data. Biometrics 33:159-174, 1977 9. Piaggio G, Elbourne DR, Pocock SJ, et al: Reporting of noninferiority and equivalence randomized trials: Extension of the CONSORT 2010 statement. JAMA 308:2594-2604, 2012 10. Bose RR, Matyal R, Warraich HJ, et al: Utility of a transesophageal echocardiographic simulator as a teaching tool. J Cardiothorac Vascular Anesth 25:212-215, 2011 11. Neelankavil J, Howard-Quijano K, Hsieh TC, et al: Transthoracic echocardiography simulation is an efficient method to train

53

anesthesiologists in basic transthoracic echocardiography skills. Anesth Analg 115:1042-1051, 2012 12. Breitkreutz R, Uddin S, Steiger H, et al: Focused echocardiography entry level: New concept of a 1-day training course. Minerva Anestesiol 75:285-292, 2009 13. Oren-Grinberg A, Gulati G, Fuchs L, et al: Hand-held echocardiography in the management of cardiac arrest. Anesth Analg 115: 1038-1041, 2012 14. Breitkreutz R, Price S, Steiger HV, et al: Focused echocardiographic evaluation in life support and peri-resuscitation of emergency patients: A prospective trial. Resuscitation 81:1527-1533, 2010

Providing initial transthoracic echocardiography training for anesthesiologists: simulator training is not inferior to live training.

Transthoracic echocardiography (TTE) is finding increased use in anesthesia and critical care. Efficient options for training anesthesiologists should...
421KB Sizes 0 Downloads 0 Views