The American Journal of Surgery (2014) 207, 170-178
Association for Surgical Education
The impact of brief team communication, leadership and team behavior training on ad hoc team performance in trauma care settings Nicole K. Roberts, Ph.D.a,*, Reed G. Williams, Ph.D.b, Cathy J. Schwind, R.N., M.S.b, John A. Sutyak, M.D.b, Christopher McDowell, M.D.b, David Griffen, M.D.b, Jarrod Wall, M.D.b, Hilary Sanfey, M.B., B.Ch.b, Audra Chestnut, M.S.c, Andreas H. Meier, M.D.b, Christopher Wohltmann, M.D.b, Ted R. Clark, M.D.b, Nathan Wetter, B.S.b a
Department of Medical Education, Southern Illinois University School of Medicine, PO Box 19681, Springfield, IL 62794-9681, USA; bDepartment of Surgery, Southern Illinois University School of Medicine, Springfield, IL, USA; c Memorial Medical Center, Springfield, IL, USA KEYWORDS: Trauma care; Communication; Leadership; Team training; Interprofessional education
Abstract BACKGROUND: Communication breakdowns and care coordination problems often cause preventable adverse patient care events, which can be especially acute in the trauma setting, in which ad hoc teams have little time for advanced planning. Existing teamwork curricula do not address the particular issues associated with ad hoc emergency teams providing trauma care. METHODS: Ad hoc trauma teams completed a preinstruction simulated trauma encounter and were provided with instruction on appropriate team behaviors and team communication. Teams completed a postinstruction simulated trauma encounter immediately afterward and 3 weeks later, then completed a questionnaire. Blinded raters rated videotapes of the simulations. RESULTS: Participants expressed high levels of satisfaction and intent to change practice after the intervention. Participants changed teamwork and communication behavior on the posttest, and changes were sustained after a 3-week interval, though there was some loss of retention. CONCLUSIONS: Brief training exercises can change teamwork and communication behaviors on ad hoc trauma teams. Ó 2014 Elsevier Inc. All rights reserved.
Evidence obtained over the past decade suggests that communication breakdowns are among the most frequent
contributors to adverse events in medicine, and they waste health care providers’ time and increase costs because of
This research was supported by a contract from the Telemedicine and Advanced Technology Research Center of the US Army Medical Research and Materiel Command. The opinions expressed in this report are those of the authors and do not necessarily reflect the opinions of the sponsoring agency. The authors declare no conflicts of interest. * Corresponding author. Tel.: 11-217-545-2103; fax: 11-217-545-0210. E-mail address: [email protected] Manuscript received January 23, 2013; revised manuscript June 11, 2013 0002-9610/$ - see front matter Ó 2014 Elsevier Inc. All rights reserved. http://dx.doi.org/10.1016/j.amjsurg.2013.06.016
N.K. Roberts et al.
Impact of brief training
duplication of efforts and orders for medical procedures.1 Data collected by the Joint Commission on Accreditation of Healthcare Organizations suggest that poor communication contributed to nearly 70% of sentinel events reported during 2005.2 Kitch et al3 found that 59% of internal medicine and surgery residents reported that R1 patient had been harmed during their most recent rotations because of problematic handoffs. This rate is similar to the rate of adverse drug events each year in the same hospital.4 Greenberg et al5 found that 14% of 444 medical malpractice claims involved communication problems resulting in harm to patients. Ninety-two percent of these involved verbal communication problems. The most prevalent contributing factors were asymmetry in the status of the involved health care providers (74%) and ambiguity about the roles and responsibilities of the involved providers (58%). A number of curricula have been developed to ameliorate the impact of poor teamwork on patient care. The most extensive and systematic effort has been the joint US Department of Defense and Agency for Healthcare Research and Quality collaboration to develop and implement the Team Strategies and Tools to Enhance Performance and Patient Safety (TeamSTEPPS) curriculum.6 The TeamSTEPPS curriculum includes a number of elements that are oriented toward improving health care team communication, coordination, and adaptation.7 Emergency medical care teams have many features in common with other health care teams but differ in that they have little time for deliberate planning and elaborate communication while providing care. Second, such teams are generally ad hoc, that is, assigned to work together in ever changing compositions.8,9 Effective performance is ensured only if the roles and tasks of the team members are clearly defined and communication and leadership aspects are regulated.10,11 Unfortunately, this may not happen in all emergency situations. Marsch et al11 found that two thirds of the rapid response teams they studied failed to convert their patient to sinus rhythm, because of a lack of communication and leadership within the cardiac arrest setting. These problems also arise in the trauma setting. One study showed that major trauma team communication was audible in only 56% of cases and understandable in 44% of cases during the primary survey, and minor trauma team communication was audible in 43% and understandable in 33% of cases.12 Cooper13 and Hunziker et al14 showed that even short training had positive effects on performance. Hunziker et al also demonstrated that leadership training was superior to technical instruction in improving performance in a simulated cardiopulmonary resuscitation scenario.
Purpose The goals of this project were to (1) identify training program needs for health professionals; (2) design and develop brief training interventions intended to foster
171 desirable changes in leadership, team communication, and role-appropriate team behaviors; (3) deliver those interventions; and (4) determine the effectiveness of the training. We hypothesized that brief training would have a sustained effect on individual acquisition and retention of knowledge and team practice behaviors and on team behaviors of health care professionals in medical emergency situations.
Methods Study design The study was a prospective, observational, longitudinal study in which participants served as their own controls. The study design is illustrated in Fig. 1. The subject protocol was approved by the Springfield Committee for Research Involving Human Subjects institutional review board on July 6, 2011, and was reviewed by the US Army Medical Research and Materiel Command, Office of Research Protections, Human Research Protection Office, and found to comply with applicable Department of Defense, US Army, and US Army Medical Research and Materiel Command human subjects protection requirements.
Participants Participants included 57 medical hospital staff members who provided level 1 trauma care at Memorial Medical Center (MMC). Third-year to 5th-year general surgery residents served as team leaders. First-year, 2nd-year, and on occasion 3rd-year emergency medicine and general surgery residents served as supporting physician members of the teams. Three emergency medicine technicians and 16 nurses also participated. These individuals were randomly assigned to teams for simulated trauma resuscitations to mirror the formation of ad hoc trauma teams. An emergency medical technician who was not a study participant was trained to present the case in a standard manner. All participants had prior experience with human patient simulators. The pretest simulated trauma encounter used highfidelity inanimate computer-driven simulators and took approximately 15 minutes to complete. These preintervention data were used to estimate baseline performance. The simulations were video and audio recorded to allow analysis of targeted individual and team behaviors. After the baseline simulation, team leaders and members participated in training designed to develop those behaviors chosen to improve team communication, leadership, and other role-appropriate team characteristics. The training concluded with team practice with a second standardized patient simulation, followed by a debriefing to help cement desired behaviors and to correct undesirable behaviors. The debriefing protocol is available from the authors.
172
The American Journal of Surgery, Vol 207, No 2, February 2014 session. The simulations were randomly distributed among the preintervention, immediate postintervention, and 3week postintervention conditions to control for possible differences in case difficulty.
Instruction on team communication, leadership, and role-appropriate behavior
Figure 1
Study design.
Approximately 3 weeks later, participants reassembled for a third, different, high-fidelity simulation to measure the retention of postintervention leader and team member behaviors. These simulations were also video and audio recorded for analysis and followed by a team debriefing
Training content and methods were designed with Rogers’s15 model of the diffusion of innovations as a guide. The content of the training program was developed by a group of trauma surgeons, emergency medicine physicians, a nurse educator, and 2 Ph.D. educators. The training included both leaders and team members together. The content was based on knowledge gained from (1) indepth analysis of the video records of a previous in situ simulation16 performed at MMC and designed to expose weaknesses in trauma team behavior and functioning at MMC and (2) data collected through supplementary observation of actual trauma care at MMC in which the goals were to confirm that the in situ simulation findings were representative of individual and team behaviors that regularly occur in MMC practice and to expand our understanding of those behaviors as necessary to plan the training. Content was also informed by relevant literature17–19 and by the TeamSTEPPS curricular materials.6 The objectives of the didactic training presentation were to (1) demonstrate that good communication, leadership, and team behaviors are critical to effective trauma care; (2) describe these behaviors; (3) demonstrate those behaviors using audio and video examples of good and poor performance. This presentation was delivered as a live presentation by the trauma director. All professions were trained simultaneously in the same room. The specific behaviors targeted for training are detailed in Table 1. Of particular note is the SMARTT Stepback, a mechanism created as a part of this study for managing the typically busy trauma bay. It provides an explicit opportunity for all team members to give input into the care of the patient. This is a 6step process that can be requested at any time during the trauma resuscitation by any team member but is typically initiated by the team leader at time of disposition. At that time, the leader will announce to the room ‘‘It’s time for a SMARTT Stepback’’ and describe the situation (the presenting situation), management (what have we done so far?), action (what do we need to do next?), and rapidity (how quickly do we need to do it?). Then he or she will ask the team for troubleshooting (what problems do you foresee, and how can we prevent them?) and ‘‘talk to me’’ (what else needs to be discussed?).
Setting This study was conducted in the emergency department and simulation center at MMC, a teaching hospital affiliated with the Southern Illinois University School of
N.K. Roberts et al. Table 1
Impact of brief training
173
Team behaviors targeted for training
Team leader behaviors
Team member behaviors
Communication behaviors: SMARTT Stepback (led by team leader)
Decide what needs to be done and priorities Direct orders to specific people by name and or role (eg, nurse, respiratory technician) Issue short, clear orders Explicit transitions in team leadership Manage noise in the trauma bay Manage workload to achieve team balance Ensure that team members adhere to orders and protocol Encourage all members to volunteer key information Introduce self by name and function to the scribe and other team members Carry out orders issued by the managing physician and standing orders appropriate to role Communicate critical information to all members of the team Speak only when necessary for patient care; minimize unnecessary noise/distraction/talk Listen to information provided by other team members Verbally confirm orders and completion of tasks Seek clarification regarding who is the managing physician when ambiguity exists Situation (patient description, injury, status, circumstances) Management (treatment performed) Activity (what needs to happen next?) Rapidity (what needs to be done first and how quickly?) Troubleshoot (what may go wrong and steps to correct or prevent) Talk to me (encourage all team members to volunteer key information, ask clarifying questions, etc)
Medicine. The in situ simulation and didactic presentation occurred in the MMC emergency department, trauma center, and operating rooms.
skills learned (KH 3); and (3) other attitudes regarding the knowledge and skills learned (KH 3).
Raters Simulators Two commercially available patient simulators with remote control of vital patient physiologic variables were used (I-Stan; METI, Sarasota, FL). Half of the simulations occurred in a simulation center room specially designed to resemble as closely as possible the working environment and conditions in the trauma bay. The other half were conducted in 1 trauma bay that was equipped to allow audio and video recording of team and individual performance. All medical equipment in both settings was identical to that used in the real work environment. The cases used for simulation are presented in Appendix 1. Case development procedures are described in Appendix 2.
Outcomes and measurements We used Kirkpatrick’s hierarchy (KH) of evaluation20 to guide our choice of outcome measures. The primary outcome for the study was changes in individual and team behaviors (KH level 3). These changes were measured by comparing video records of preintervention, immediate postintervention, and delayed postintervention simulations. We investigated coded frequencies of trained behaviors and quality of those performances using the criteria specified in the training. Secondary outcomes included (1) percentage who completed training and views on the learning experience and its organization, presentation, content, and teaching methods (KH 1); (2) intention to apply knowledge and
A surgeon and a nurse who were blinded to the stage of training independently rated each audio and video record. Raters blindly rated 5 of the audio and video records twice with new video record identifiers, which allowed us to test within-rater agreement. We trained the raters by introducing and talking about the rating form, which we modified on the basis of their suggestions. We then reviewed a video and rated it together, then viewed a second video and allowed the raters to rate it and discuss their perceptions.
Data analysis We documented the impact on the trainees by determining attitudes toward the program, clarity and nature of expressed intentions to change personal work behavior, and clarity and nature of expressed intentions to advocate for changes in work practices of the department and organization. Additionally, we analyzed differences in ad hoc individual and team performance on the simulations in each study group. The unit of measurement was the performance of the team rather than that of the individuals in the group.
Statistical analysis Within-rater agreement was determined using both the relative ranking and absolute agreement models of intraclass correlation coefficients. The first determines the
174
The American Journal of Surgery, Vol 207, No 2, February 2014
Table 2
Participant views on the learning experience, broken down by medical role of respondents Medical technician (n 5 3)
Nurse (n 5 13)
Physician (n 5 26)
Total (n 5 44*)
Training characteristic
U
SA
U
SA
U
A
U
1. 2. 3. 4. 5. 6. 7. 8. 9.
0 0 0 1 0 0 0 0 0
2 2 1 2 3 1 0 0 0
1 1 2 0 0 2 3 3 3
0 0 0 1 0 0 0 0 0
6 7 9 7 6 5 4 5 4
7 6 4 5 7 8 9 8 9
0 0 0 2 1 1 0 0 0
8 8 10 14 10 4 6 6 3
18 18 16 10 15 21 20 20 23
0 0 0 4 1 1 0 0 0
18 19 22 25 21 12 12 12 8
26 25 22 15 22 31 32 32 36
0
0
3
0
5
8
0
4
22
0
10
34
0 0
2 0
1 3
0 0
5 4
8 9
1 0
6 11
19 15
1 0
15 16
28 28
1
13
22
1
67
88
5
90
217
7
190
345
Training well organized Understood training content Can perform skills that were trained Training was good use of time Skills seem easy to use Content appropriate Use could improve patient safety Use could improve care efficiency Use should improve clarity regarding team leadership 10. Use should result in better communication, situation awareness, and mutual support 11. Use should result in improved team functioning 12. I intend to apply the learned skills in my work environment Total
A
A
SA
A
SA
One nurse respondent who responded ‘‘strongly disagree’’ to all items is excluded to simplify the table. A 5 agree; SA 5 strongly agree; U 5 undecided. *Two respondents did not indicate their medical roles.
degree to which the rankings of the targeted video were similar on the first and second evaluations. The second determines the degree to which the absolute rating assigned on both rating occasions agreed. Attitudes toward the course were analyzed by determining the number and percentage of respondents choosing each response option. Comparisons between pretraining, immediate posttraining, and 3-week retention video records of team performance were compared using 1-way analysis of variance. Selected target behaviors and combinations of target behaviors (eg, SMARTT Stepback behaviors) were also singled out for analyses. In these cases, the frequencies of occurrences and percentage of occasions when the behaviors occurred were determined. All analyses were performed using SPSS version 19 (IBM SPSS, Chicago, IL).
Results The results of the within-rater agreement analyses are broken down by rater (surgeon, nurse) and by scale. The conventionally accepted standard for reliability correlations is R.80, although this standard is often not obtained when using human raters. Our findings demonstrated that the nurse rater achieved this .80 standard of consistency in 4 of the 6 measurements regardless of the model being used. The surgeon rater’s results averaged .51 and .52. For subsequent analyses, we combined the surgeon and nurse raters’ ratings. Table 2 provides a summary of participants’ responses to the posttraining questionnaire regarding their perceptions of the training. Forty-five participants (79%) completed the posttraining questionnaire. Two participants did not
identify their medical roles. Their responses are included in the ‘‘total’’ columns. The first 6 items established participants’ perceptions about the process and content of the training. The predominant response for all items was ‘‘strongly agree,’’ with the exception of the item regarding ‘‘training being a good use of the respondents’ time.’’ Within these 6 questions, there were 6 ‘‘undecided’’ responses. Four of them were responses to the questions about being a good use of the respondents’ time. The undecided responses were fairly evenly distributed among the 3 groups of professionals. Questions 7 through 11 asked the participants for their opinions regarding the potential benefits of the target training behaviors for the quality and efficiency of patient care and patient outcomes. For all 5 questions, the majority of participants strongly agreed that the training had the potential to improve patient safety, care efficiency, team functioning, clarity regarding team leadership, communication, situational awareness, and mutual support. Item 12 asked participants to indicate whether they intended to apply the skills learned in their work environments. Again, the predominant response was ‘‘strongly agree.’’ Across these 12 items, physician respondents were more positive about the benefits of the training and the potential quality and care outcomes than were nurses and medical technicians, although all but 1 nurse responded favorably. The final item asked the participants to indicate whether they had already used any or all of the trained skills in their work environment. Twenty-six participants (58%) indicated that they had used some of the trained skills already. Two indicated that they had not, and 16 participants had not participated in trauma care since the completion of their training.
N.K. Roberts et al.
Impact of brief training
175
Table 3 Mean ratings (and standard deviations) of targeted and trained ad hoc trauma team behaviors during simulated trauma resuscitations at each stage in the team training cycle
Pretraining
Immediate 3-wk posttraining retention
Immediate Pretraining posttraining Pretraining vs 3-wk vs immediate vs 3-wk retention posttraining retention
Category (best possible score)
Mean 6 SD
Mean 6 SD
P
P
P
Organized (5) Efficiency (5) Supportive (5) Volunteered important information (5) Listened to information (5) Instructions performed (5) Clarified ambiguous orders (5) Confirmed completion of tasks (5) Leader clearly identifiable (5) Leader assigned tasks to team members by name or role (5) Leader managed noise appropriately (5) Leader coordinated communication and team activity effectively (5) Leader encouraged team members to volunteer key information during SMARTT Stepback (5) Trauma nontechnical skills Leadership (5) Cooperation (5) Communication (5) Decision making (5) Situational awareness coping with stress (5)
2.67 3.00 3.00 2.61 3.06 3.17 1.89 1.89 3.78 3.11
.001* .01* .007* .00* .001* .001* NS .00* NS .02*
.001* .01* NS NS .008* .018* NS NS NS NS
NS NS NS NS NS NS NS .03† NS NS
6 6 6 6 6 6 6 6 6 6
.66 .67 .50 .55 .58 .35 .82 .33 .51 .65
3.67 3.78 3.67 3.89 3.83 3.78 3.00 3.06 4.0 3.89
6 6 6 6 6 6 6 6 6 6
.35 .36 .25 .22 .25 .26 1.20 0.68 0 .22
Mean 6 SD 3.67 3.78 3.44 3.28 3.67 3.61 2.50 2.39 3.83 3.28
6 6 6 6 6 6 6 6 6 6
.50 .51 .46 .69 .25 .33 1.04 .55 0.35 0.67
2.41 6 1.20 3.50 6 .84 2.33 6 0.75 3.44 6 .46
3.43 6 0.79 NS 2.72 6 0.67 .003*
NS NS
NS NS
1.50 6 0.71 3.58 6 .58
3.25 6 1.17 .04*
NS
NS
NS .01 .015 NS .05
NS NS NS NS NS
3.72 2.89 2.56 3.67 3.67
6 6 6 6 6
0.36 0.65 0.46 0.71 0.71
4.67 4.61 4.06 4.61 4.67
6 6 6 6 6
.43 .42 .39 .70 .35
4.22 3.94 3.50 4.17 4.39
6 6 6 6 6
0.67 0.92 0.97 0.97 0.70
.002* .000* .000* .05* .005*
*Statistically significant differences in team and leader behaviors suggesting that the training had the desired effects. † Statistically significant differences in team behavior indicating significant reductions in desired team and leader behaviors at the 3-week retention end point compared with the immediate posttraining end point.
Table 3 provides a summary of the team results. Fifteen of 17 targeted team and leader behaviors significantly improved immediately after the training. One area in which team and leader behaviors did not improve was team members’ efforts to clarify ambiguous orders. The second involved whether the team leader was clearly identifiable. As can be seen in the pretraining simulation results, the judges’ ratings indicated that the team leader was easily identified before training. The third area involved team leaders’ management of noise during the trauma resuscitation.
Table 4
Comparing the pretraining with the 3-week retention results indicated that 7 areas had lasting training effects. Moreover, 4 of these 7 areas (efficiency, listened to information, orders were carried out, and cooperation and communication) are all critical indicators of effective team performance. One area for which there was a critical relapse in team behavior was team members’ confirmation when they completed tasks. This area requires additional attention. The number of team members who announced themselves before training was virtually zero, and this was true
Team members who were announced upon arrival, broken down by profession and by reporting rater
Time
Rater
Chief resident (n 5 9)
Pretraining
1 2 1 2 1 2
1 0 7 9 5 6
Immediate posttraining 3-week posttraining
Emergency room technician (n 5 6)
Bedside nurse (n 5 18)
Junior resident (n 5 18)
Respiratory therapist (n 5 5)
Total
0 0 0 5 0 5
0 0 3 7 1 4
1 0 3 8 4 6
0 0 0 5 1 1
2 0 13 34 11 22
176
The American Journal of Surgery, Vol 207, No 2, February 2014
Table 5 Simulations in which SMARTT Stepbacks occurred as coordinated events (all components occurred at the same time), broken down by time and by rater Time
Rater 1
Rater 2
Pretraining Immediate posttraining 3-week posttraining
0 (0%) 3 (33%) 5 (56%)
0 (0%) 6 (67%) 6 (67%)
regardless of profession (Table 4). The rate of introductions improved immediately after training and was maintained at a slightly lower rate after 3 weeks. However, the rate is still not 100%. We recorded that a SMARTT Stepback occurred only if all 6 components occurred at a single time (Table 5). As can be seen, the frequency of SMARTT Stepbacks increased after training, and the rate was sustained after 3 weeks. However, the rate was still less than the desired 100%. Table 6 indicates the number of SMARTT Stepback component behaviors that occurred at some point during each simulation, a much less restrictive indication of training success. In this table, a frequency of 9 in a cell would indicate that this behavior occurred 1 time in that simulation event. As can be seen, the 2 raters appeared to use different rating strategies for the pretraining. Their frequencies for the immediate posttraining and 3-week retention after training are more similar. Raters recorded each element occurring close to once per simulation event, except in the pretraining phase of the study.
Comments Our in situ simulation showed that residents were not clear on who was leading resuscitations. This resulted in a shifting leadership focus among the residents throughout the trauma resuscitation. Communication was fragmented, incomplete, and frequently interrupted, thus requiring repetition. The trauma bay was noisy, with several people often talking at once. These findings were confirmed subsequently in observations of actual trauma resuscitations in the trauma bay. Our training intervention, the SMARTT Stepback, was developed specifically to determine whether a brief training program would lead to changes in team member behaviors that would improve team performance in these areas. Table 6
Recently, 2 other studies have addressed similar training needs for ad hoc trauma team members.21,22 Both of those training interventions involved more training time on the part of participants compared with our study. Likewise, both studies included changes in traditional trauma outcome parameters as well as changes in team behaviors, whereas our study focused exclusively on changes in targeted team behaviors manifested in simulated trauma resuscitations. Finally, both studies included attending physicians as participants, whereas our study excluded attending physicians. The primary value that our study adds to the findings of these 2 studies lies in 2 areas. First and perhaps most important, our study used expert judges who were blinded to the stage of team training for participants. Our raters reviewed audio and video records of all teams performing all trauma simulations at every stage of training. Second, our study added the 3-week posttraining simulation to measure team retention of targeted team and individual behaviors. All 3 studies demonstrated that relatively brief training episodes can lead to changes in targeted team leadership, communication, and coordination behaviors. In our study, all behaviors were observed in simulations in which participants knew what behaviors were being observed and recorded, which may raise a question about whether the trained behaviors will persist in the actual trauma environment with real patients. However, the results from the studies of Steinemann et al19 and Capella et al20 provide some evidence to support transfer of the training to real trauma cases. Our study strengthens the collective knowledge from the 3 studies by blinding the raters to stage of training and thus minimizing the possibility that raters’ judgments are influenced by their knowledge of the training stage. Our results are also conservative in that the retention outcome measure occurs before the impact of the second debriefing. Although the results of our study provide evidence that the training is producing intended results, it is clear that the effect is not robust. The targeted behaviors are not present in all teams, and the sporadic team behavioral characteristics remind us that the behaviors are likely to fade absent practice and continued hospital leadership support in the form of policies and role modeling. For example, inspection of Tables 5 and 6 indicates that, at most, two thirds of the team leaders initiated complete SMARTT Stepback events during the simulations occurring after training. However, many of the elements of a SMARTT Stepback
SMARTT Stepback components that occurred during the simulation, broken down by time and by rater
Time
Rater
Situation
Management
Activity
Rapidity
Troubleshoot
Talk to me
Pretraining
1 2 1 2 1 2
8 0 9 6 8 7
8 0 9 6 8 8
7 0 9 6 7 8
2 0 8 6 6 5
2 0 8 5 4 6
6 0 9 5 6 7
Immediate posttraining 3-week posttraining
A single performance that included 1 instance of the behavior would have a frequency of 9 in each cell.
N.K. Roberts et al.
Impact of brief training
occurred during each posttraining simulation. This suggests partial success in this portion of the training but certainly also indicates that more training and reinforcement of these coordinated behaviors will be needed if SMARTT Stepbacks are expected to occur on a regular basis. Some of our results have multiple potential interpretations. For instance, the fact that there was no improvement in clarifying ambiguous orders could be due either to a lack of ambiguous orders to begin with or to a failure before and after training to seek clarification. We suspect the latter. The fact that the team leader was easily identifiable in both the pre and post conditions was likely an artifact of the simulation. Our observations both in situ and in the trauma bay convinced us that in the natural setting, identifying the leader is much more challenging, thus warranting explicit training. The lack of difference in the team leaders’ management of noise was also likely an artifact of the simulation condition. Independent observers noted that the simulations were much more quiet than the typical trauma simulation. They were certainly quieter than any of the live trauma resuscitations or the in situ trauma resuscitation we observed. All team leaders and prospective team leaders were trained and were given the opportunity to practice these behaviors as a part of this project. A nuclear group of nursing and emergency department technicians also received the training including the practice simulated trauma resuscitations. All other trauma team members are receiving the didactic training but will not have the opportunity to practice using the simulated trauma resuscitations. The trauma leadership, hospital leadership, and emergency medicine leadership are committed to encouraging all team members to incorporate these behaviors into their individual and team practices by providing mandatory training to the remaining trauma nurses and technicians and to the new general surgery and emergency department residents. We assessed our learners immediately after the intervention and again at a 3-week interval; admittedly, this does not provide convincing proof of long-term retention of the 7 skills, but it is the first experimental evidence of retention in the trauma literature. Finally, we want to echo the views of Capella et al20 that the logistics of providing training to teams made up of health care providers from different professions was one of the most challenging aspects of this project. Coordinating the scheduling of these training exercises involved working with 5 different hospital and residency program administrative bodies. It is no surprise to us that there is a great deal of talk about the desirability of multiprofessional team training, but the number of examples of such training in hospital settings is limited.
Acknowledgments We gratefully acknowledge the contributions of the MMC project committee that provided support and oversight on a
177 monthly basis: Mitchell Johnson, Clay Bahlow, Marcia Prater, Paula Gramley, and Drew Early.
References 1. Williams RG, Silverman R, Schwind C, et al. Surgeon information transfer and communication: factors affecting quality and efficiency of inpatient care. Ann Surg 2007;245:159–69. 2. Joint Commission on Accreditation of Healthcare Organizations. National patient safety goals for 2006. Available at: http://ww.jcaho. org/accredited1organizations/patient1safety/npsg.htm. 3. Kitch BT, Cooper JB, Zapol WM, et al. Handoffs causing patient harm: a survey of medical and surgical house staff. Jt Comm J Qual Patient Saf 2008;34:563–70. 4. Bates DW, Cullen DJ, Laird N, et al, ADE Prevention Study Group. Incidence of adverse drug events and potential adverse drug events. Implications for prevention. JAMA 1995;274:29–34. 5. Greenberg CC, Regenbogen SE, Studdert DM, et al. Patterns of communication breakdowns resulting in injury to surgical patients. J Am Coll Surg 2007;204:533–40. 6. King HB, Battles JB, Baker DP, et al. TeamSTEPPS: team strategies and tools to enhance performance and patient safety. Available at: http:// www.ahrq.gov/downloads/pub/advances2/vol3/advances-king_1.pdf. 7. Meier AH, Boehler ML, McDowell CM, et al. a surgical simulation curriculum for senior medical students based on TeamSTEPPS. Arch Surg 2012;147:761–6. 8. Ginnett R. Crews as groups: their formation and their leadership. In: Helmreich R, editor. Cockpit Resource Management. San Diego, CA: Academic Press; 1993. p. 71–98. 9. Hackman J. Teams, leaders and organizations: new directions for crew-oriented training. In: Helmreich R, editor. Cockpit Resource Management. San Diego, CA: Academic Press; 1993. p. 47–69. 10. Kanki BG, Lozito S, Foushee HC. Communication indexes of crew coordination. Aviat Space Envir Md 1989;60:56–60. 11. Marsch SCU, Muller C, Marquardt K, et al. Human factors affect the quality of cardiopulmonary resuscitation in simulated cardiac arrests. Resuscitation 2004;60:51. 12. Bergs EA, Rutten FL, Tadros T, et al. Communication during trauma resuscitation: do we know what is happening? Injury 2005;36:905–11. 13. Cooper S. Developing leaders for advanced life support: evaluation of a training programme. Resuscitation 2001;49:33–8. 14. Hunziker S, Buhlman C, Tschan F. Brief leadership instructions improve cardiopulmonary resuscitation in a high-fidelity simulation: A randomized controlled trial. Crit Care Med 2010;38:1086–91 [Erratum 2010;38:1510.]. 15. Rogers EM. Diffusion Of Innovations. 4th ed. New York: Free Press; 1995. 16. Hamman WR. The complexity of team training: what we have learned from aviation and its applications to medicine. Qual Saf Health Care 2004;13(suppl):i72–9. 17. Tschan F, Semmer NK, Gautschi D, et al. Leading to recovery: group performance and coordinative activities in medical emergency driven groups. Hum Perf 2006;19:277. 18. Marsch SC, Tschan F, Semmer N, et al. Performance of first responders in simulated cardiac arrests. Crit Care Med 2005;33:963–7. 19. Rosen MA, Salas E, Wu TS, et al. Promoting teamwork: an eventbased approach to simulation-based teamwork training for emergency medicine residents. Acad Emerg Med 2008;15:1190–8. 20. Kirkpatrick DL. Evaluation of training. In: Craig RL, Bittel LR, editors. Training and Development Handbook. New York: McGraw-Hill; 1967. p. 87–112. 21. Steinemann S, Berg B, Skinner A, et al. In situ, multidisciplinary, simulation-based teamwork training improves early trauma care. J Surg Educ 2011;68:472–7. 22. Capella J, Smith S, Philp A, et al. Teamwork training improves the clinical care of trauma patients. J Surg Educ 2010;67:439–43.
178
Appendix 1 Scenarios Used for Simulated Trauma Resuscitations Scenario A was a 24-year-old man found lying face down on the ground, having sustained a gunshot wound to his left chest and upper right arm. In this case, the patient had a hemopneumothorax and needed a needle decompression or a chest tube immediately. The patient was in a semiconscious state and quickly deteriorated without appropriate management of the chest tube and fluid resuscitation. Once the patient had stabilized, the learner needed to state that the patient would go to computed tomography quickly. At that time, the scenario was terminated. Scenario B was a 50-year-old man who was involved in an explosion at his farm when the gasoline tank he was welding exploded. He was thrown 50 feet and had a 5minute loss of consciousness. He was confused, tachycardic, and hypotensive upon arrival in the trauma bay. He also had multiple abrasions and a near amputation of the right arm near the elbow. This patient had low oxygen saturation and needed to be intubated immediately. If the learners did not intubate quickly, the patient quickly deteriorated without blood pressure or pulse. If the learners did intubate, they discovered that the oxygen saturation was still low, and the patient was absent of breath sounds on the right. The patient needed a needle decompression and chest tube within 2 minutes or he deteriorated, becoming more hypotensive and tachycardic. Once the patient was stable, the learners needed to work on stabilizing the near amputation of the right arm and state that he needed to go to computed tomography quickly. Scenario C was a 25-year-old male restrained driver involved in a head-on high-speed motor vehicle crash. The patient was pinned between the dashboard and the seat and required a 30-minute extrication. He was awake but groggy upon arrival in the trauma bay. He had decreased breath sounds on the left, he had a tender rigid abdomen, and he was tachycardic, hypotensive, and tachypneic. The team was required to recognize that this patient needed to be intubated and have a chest tube placed quickly. If they failed to recognize this the patient deteriorated quickly.
The American Journal of Surgery, Vol 207, No 2, February 2014 Once the patient was intubated and a chest tube was placed, the team needed to recognize that the patient had a tender rigid abdomen and order a focused assessment with sonography for trauma scan. These results were negative, and the patient was stabilized. Finally, the team was expected to announce that the patient needed to proceed to computed tomography.
Appendix 2 Development of Simulation Scenarios After reviewing the 30 simulation cases in the METI Disaster Medical Readiness and Emergency Medical Services Learning Modules, cases developed locally, and team training cases developed for the Association of Program Directors in Surgery curriculum under the leadership of Gary Dunnington, M.D., at the Southern Illinois University School of Medicine, members of the Core Training Planning and Development Group developed 4 simulated trauma resuscitation scenarios for use in the training. This involved developing (1) the educational rationale and the learning objectives for the simulation scenarios; (2) the case stem for the learners (pertinent patient and scenario information: task, location, physician/help availability, and equipment availability; (3) background and briefing information for facilitators and coordinators (patient data background and baseline patient state, baseline simulator state); (4) supporting files (chest x-ray, echocardiography, assessment, handouts, etc); (5) a list of alterations that will occur during the simulation; (6) the trigger to move to the next state; (7) the expected learner actions; and (8) the teaching points for the debriefing. One case was used exclusively for demonstrating the target behaviors as part of the training intervention. The other 3 were alternated for use as (1) preintervention exercises designed to determine the rate of team target behaviors before training, (2) opportunities to practice using the target behaviors at the end of the training session, and (3) retention simulation exercises to document retention of the target behaviors approximately 3 weeks after training.
Association for Surgical Education
The impact of brief team communication, leadership and team behavior training on ad hoc team performance in trauma care settings Nicole K. Roberts, Ph.D.a,*, Reed G. Williams, Ph.D.b, Cathy J. Schwind, R.N., M.S.b, John A. Sutyak, M.D.b, Christopher McDowell, M.D.b, David Griffen, M.D.b, Jarrod Wall, M.D.b, Hilary Sanfey, M.B., B.Ch.b, Audra Chestnut, M.S.c, Andreas H. Meier, M.D.b, Christopher Wohltmann, M.D.b, Ted R. Clark, M.D.b, Nathan Wetter, B.S.b a
Department of Medical Education, Southern Illinois University School of Medicine, PO Box 19681, Springfield, IL 62794-9681, USA; bDepartment of Surgery, Southern Illinois University School of Medicine, Springfield, IL, USA; c Memorial Medical Center, Springfield, IL, USA KEYWORDS: Trauma care; Communication; Leadership; Team training; Interprofessional education
Abstract BACKGROUND: Communication breakdowns and care coordination problems often cause preventable adverse patient care events, which can be especially acute in the trauma setting, in which ad hoc teams have little time for advanced planning. Existing teamwork curricula do not address the particular issues associated with ad hoc emergency teams providing trauma care. METHODS: Ad hoc trauma teams completed a preinstruction simulated trauma encounter and were provided with instruction on appropriate team behaviors and team communication. Teams completed a postinstruction simulated trauma encounter immediately afterward and 3 weeks later, then completed a questionnaire. Blinded raters rated videotapes of the simulations. RESULTS: Participants expressed high levels of satisfaction and intent to change practice after the intervention. Participants changed teamwork and communication behavior on the posttest, and changes were sustained after a 3-week interval, though there was some loss of retention. CONCLUSIONS: Brief training exercises can change teamwork and communication behaviors on ad hoc trauma teams. Ó 2014 Elsevier Inc. All rights reserved.
Evidence obtained over the past decade suggests that communication breakdowns are among the most frequent
contributors to adverse events in medicine, and they waste health care providers’ time and increase costs because of
This research was supported by a contract from the Telemedicine and Advanced Technology Research Center of the US Army Medical Research and Materiel Command. The opinions expressed in this report are those of the authors and do not necessarily reflect the opinions of the sponsoring agency. The authors declare no conflicts of interest. * Corresponding author. Tel.: 11-217-545-2103; fax: 11-217-545-0210. E-mail address: [email protected] Manuscript received January 23, 2013; revised manuscript June 11, 2013 0002-9610/$ - see front matter Ó 2014 Elsevier Inc. All rights reserved. http://dx.doi.org/10.1016/j.amjsurg.2013.06.016
N.K. Roberts et al.
Impact of brief training
duplication of efforts and orders for medical procedures.1 Data collected by the Joint Commission on Accreditation of Healthcare Organizations suggest that poor communication contributed to nearly 70% of sentinel events reported during 2005.2 Kitch et al3 found that 59% of internal medicine and surgery residents reported that R1 patient had been harmed during their most recent rotations because of problematic handoffs. This rate is similar to the rate of adverse drug events each year in the same hospital.4 Greenberg et al5 found that 14% of 444 medical malpractice claims involved communication problems resulting in harm to patients. Ninety-two percent of these involved verbal communication problems. The most prevalent contributing factors were asymmetry in the status of the involved health care providers (74%) and ambiguity about the roles and responsibilities of the involved providers (58%). A number of curricula have been developed to ameliorate the impact of poor teamwork on patient care. The most extensive and systematic effort has been the joint US Department of Defense and Agency for Healthcare Research and Quality collaboration to develop and implement the Team Strategies and Tools to Enhance Performance and Patient Safety (TeamSTEPPS) curriculum.6 The TeamSTEPPS curriculum includes a number of elements that are oriented toward improving health care team communication, coordination, and adaptation.7 Emergency medical care teams have many features in common with other health care teams but differ in that they have little time for deliberate planning and elaborate communication while providing care. Second, such teams are generally ad hoc, that is, assigned to work together in ever changing compositions.8,9 Effective performance is ensured only if the roles and tasks of the team members are clearly defined and communication and leadership aspects are regulated.10,11 Unfortunately, this may not happen in all emergency situations. Marsch et al11 found that two thirds of the rapid response teams they studied failed to convert their patient to sinus rhythm, because of a lack of communication and leadership within the cardiac arrest setting. These problems also arise in the trauma setting. One study showed that major trauma team communication was audible in only 56% of cases and understandable in 44% of cases during the primary survey, and minor trauma team communication was audible in 43% and understandable in 33% of cases.12 Cooper13 and Hunziker et al14 showed that even short training had positive effects on performance. Hunziker et al also demonstrated that leadership training was superior to technical instruction in improving performance in a simulated cardiopulmonary resuscitation scenario.
Purpose The goals of this project were to (1) identify training program needs for health professionals; (2) design and develop brief training interventions intended to foster
171 desirable changes in leadership, team communication, and role-appropriate team behaviors; (3) deliver those interventions; and (4) determine the effectiveness of the training. We hypothesized that brief training would have a sustained effect on individual acquisition and retention of knowledge and team practice behaviors and on team behaviors of health care professionals in medical emergency situations.
Methods Study design The study was a prospective, observational, longitudinal study in which participants served as their own controls. The study design is illustrated in Fig. 1. The subject protocol was approved by the Springfield Committee for Research Involving Human Subjects institutional review board on July 6, 2011, and was reviewed by the US Army Medical Research and Materiel Command, Office of Research Protections, Human Research Protection Office, and found to comply with applicable Department of Defense, US Army, and US Army Medical Research and Materiel Command human subjects protection requirements.
Participants Participants included 57 medical hospital staff members who provided level 1 trauma care at Memorial Medical Center (MMC). Third-year to 5th-year general surgery residents served as team leaders. First-year, 2nd-year, and on occasion 3rd-year emergency medicine and general surgery residents served as supporting physician members of the teams. Three emergency medicine technicians and 16 nurses also participated. These individuals were randomly assigned to teams for simulated trauma resuscitations to mirror the formation of ad hoc trauma teams. An emergency medical technician who was not a study participant was trained to present the case in a standard manner. All participants had prior experience with human patient simulators. The pretest simulated trauma encounter used highfidelity inanimate computer-driven simulators and took approximately 15 minutes to complete. These preintervention data were used to estimate baseline performance. The simulations were video and audio recorded to allow analysis of targeted individual and team behaviors. After the baseline simulation, team leaders and members participated in training designed to develop those behaviors chosen to improve team communication, leadership, and other role-appropriate team characteristics. The training concluded with team practice with a second standardized patient simulation, followed by a debriefing to help cement desired behaviors and to correct undesirable behaviors. The debriefing protocol is available from the authors.
172
The American Journal of Surgery, Vol 207, No 2, February 2014 session. The simulations were randomly distributed among the preintervention, immediate postintervention, and 3week postintervention conditions to control for possible differences in case difficulty.
Instruction on team communication, leadership, and role-appropriate behavior
Figure 1
Study design.
Approximately 3 weeks later, participants reassembled for a third, different, high-fidelity simulation to measure the retention of postintervention leader and team member behaviors. These simulations were also video and audio recorded for analysis and followed by a team debriefing
Training content and methods were designed with Rogers’s15 model of the diffusion of innovations as a guide. The content of the training program was developed by a group of trauma surgeons, emergency medicine physicians, a nurse educator, and 2 Ph.D. educators. The training included both leaders and team members together. The content was based on knowledge gained from (1) indepth analysis of the video records of a previous in situ simulation16 performed at MMC and designed to expose weaknesses in trauma team behavior and functioning at MMC and (2) data collected through supplementary observation of actual trauma care at MMC in which the goals were to confirm that the in situ simulation findings were representative of individual and team behaviors that regularly occur in MMC practice and to expand our understanding of those behaviors as necessary to plan the training. Content was also informed by relevant literature17–19 and by the TeamSTEPPS curricular materials.6 The objectives of the didactic training presentation were to (1) demonstrate that good communication, leadership, and team behaviors are critical to effective trauma care; (2) describe these behaviors; (3) demonstrate those behaviors using audio and video examples of good and poor performance. This presentation was delivered as a live presentation by the trauma director. All professions were trained simultaneously in the same room. The specific behaviors targeted for training are detailed in Table 1. Of particular note is the SMARTT Stepback, a mechanism created as a part of this study for managing the typically busy trauma bay. It provides an explicit opportunity for all team members to give input into the care of the patient. This is a 6step process that can be requested at any time during the trauma resuscitation by any team member but is typically initiated by the team leader at time of disposition. At that time, the leader will announce to the room ‘‘It’s time for a SMARTT Stepback’’ and describe the situation (the presenting situation), management (what have we done so far?), action (what do we need to do next?), and rapidity (how quickly do we need to do it?). Then he or she will ask the team for troubleshooting (what problems do you foresee, and how can we prevent them?) and ‘‘talk to me’’ (what else needs to be discussed?).
Setting This study was conducted in the emergency department and simulation center at MMC, a teaching hospital affiliated with the Southern Illinois University School of
N.K. Roberts et al. Table 1
Impact of brief training
173
Team behaviors targeted for training
Team leader behaviors
Team member behaviors
Communication behaviors: SMARTT Stepback (led by team leader)
Decide what needs to be done and priorities Direct orders to specific people by name and or role (eg, nurse, respiratory technician) Issue short, clear orders Explicit transitions in team leadership Manage noise in the trauma bay Manage workload to achieve team balance Ensure that team members adhere to orders and protocol Encourage all members to volunteer key information Introduce self by name and function to the scribe and other team members Carry out orders issued by the managing physician and standing orders appropriate to role Communicate critical information to all members of the team Speak only when necessary for patient care; minimize unnecessary noise/distraction/talk Listen to information provided by other team members Verbally confirm orders and completion of tasks Seek clarification regarding who is the managing physician when ambiguity exists Situation (patient description, injury, status, circumstances) Management (treatment performed) Activity (what needs to happen next?) Rapidity (what needs to be done first and how quickly?) Troubleshoot (what may go wrong and steps to correct or prevent) Talk to me (encourage all team members to volunteer key information, ask clarifying questions, etc)
Medicine. The in situ simulation and didactic presentation occurred in the MMC emergency department, trauma center, and operating rooms.
skills learned (KH 3); and (3) other attitudes regarding the knowledge and skills learned (KH 3).
Raters Simulators Two commercially available patient simulators with remote control of vital patient physiologic variables were used (I-Stan; METI, Sarasota, FL). Half of the simulations occurred in a simulation center room specially designed to resemble as closely as possible the working environment and conditions in the trauma bay. The other half were conducted in 1 trauma bay that was equipped to allow audio and video recording of team and individual performance. All medical equipment in both settings was identical to that used in the real work environment. The cases used for simulation are presented in Appendix 1. Case development procedures are described in Appendix 2.
Outcomes and measurements We used Kirkpatrick’s hierarchy (KH) of evaluation20 to guide our choice of outcome measures. The primary outcome for the study was changes in individual and team behaviors (KH level 3). These changes were measured by comparing video records of preintervention, immediate postintervention, and delayed postintervention simulations. We investigated coded frequencies of trained behaviors and quality of those performances using the criteria specified in the training. Secondary outcomes included (1) percentage who completed training and views on the learning experience and its organization, presentation, content, and teaching methods (KH 1); (2) intention to apply knowledge and
A surgeon and a nurse who were blinded to the stage of training independently rated each audio and video record. Raters blindly rated 5 of the audio and video records twice with new video record identifiers, which allowed us to test within-rater agreement. We trained the raters by introducing and talking about the rating form, which we modified on the basis of their suggestions. We then reviewed a video and rated it together, then viewed a second video and allowed the raters to rate it and discuss their perceptions.
Data analysis We documented the impact on the trainees by determining attitudes toward the program, clarity and nature of expressed intentions to change personal work behavior, and clarity and nature of expressed intentions to advocate for changes in work practices of the department and organization. Additionally, we analyzed differences in ad hoc individual and team performance on the simulations in each study group. The unit of measurement was the performance of the team rather than that of the individuals in the group.
Statistical analysis Within-rater agreement was determined using both the relative ranking and absolute agreement models of intraclass correlation coefficients. The first determines the
174
The American Journal of Surgery, Vol 207, No 2, February 2014
Table 2
Participant views on the learning experience, broken down by medical role of respondents Medical technician (n 5 3)
Nurse (n 5 13)
Physician (n 5 26)
Total (n 5 44*)
Training characteristic
U
SA
U
SA
U
A
U
1. 2. 3. 4. 5. 6. 7. 8. 9.
0 0 0 1 0 0 0 0 0
2 2 1 2 3 1 0 0 0
1 1 2 0 0 2 3 3 3
0 0 0 1 0 0 0 0 0
6 7 9 7 6 5 4 5 4
7 6 4 5 7 8 9 8 9
0 0 0 2 1 1 0 0 0
8 8 10 14 10 4 6 6 3
18 18 16 10 15 21 20 20 23
0 0 0 4 1 1 0 0 0
18 19 22 25 21 12 12 12 8
26 25 22 15 22 31 32 32 36
0
0
3
0
5
8
0
4
22
0
10
34
0 0
2 0
1 3
0 0
5 4
8 9
1 0
6 11
19 15
1 0
15 16
28 28
1
13
22
1
67
88
5
90
217
7
190
345
Training well organized Understood training content Can perform skills that were trained Training was good use of time Skills seem easy to use Content appropriate Use could improve patient safety Use could improve care efficiency Use should improve clarity regarding team leadership 10. Use should result in better communication, situation awareness, and mutual support 11. Use should result in improved team functioning 12. I intend to apply the learned skills in my work environment Total
A
A
SA
A
SA
One nurse respondent who responded ‘‘strongly disagree’’ to all items is excluded to simplify the table. A 5 agree; SA 5 strongly agree; U 5 undecided. *Two respondents did not indicate their medical roles.
degree to which the rankings of the targeted video were similar on the first and second evaluations. The second determines the degree to which the absolute rating assigned on both rating occasions agreed. Attitudes toward the course were analyzed by determining the number and percentage of respondents choosing each response option. Comparisons between pretraining, immediate posttraining, and 3-week retention video records of team performance were compared using 1-way analysis of variance. Selected target behaviors and combinations of target behaviors (eg, SMARTT Stepback behaviors) were also singled out for analyses. In these cases, the frequencies of occurrences and percentage of occasions when the behaviors occurred were determined. All analyses were performed using SPSS version 19 (IBM SPSS, Chicago, IL).
Results The results of the within-rater agreement analyses are broken down by rater (surgeon, nurse) and by scale. The conventionally accepted standard for reliability correlations is R.80, although this standard is often not obtained when using human raters. Our findings demonstrated that the nurse rater achieved this .80 standard of consistency in 4 of the 6 measurements regardless of the model being used. The surgeon rater’s results averaged .51 and .52. For subsequent analyses, we combined the surgeon and nurse raters’ ratings. Table 2 provides a summary of participants’ responses to the posttraining questionnaire regarding their perceptions of the training. Forty-five participants (79%) completed the posttraining questionnaire. Two participants did not
identify their medical roles. Their responses are included in the ‘‘total’’ columns. The first 6 items established participants’ perceptions about the process and content of the training. The predominant response for all items was ‘‘strongly agree,’’ with the exception of the item regarding ‘‘training being a good use of the respondents’ time.’’ Within these 6 questions, there were 6 ‘‘undecided’’ responses. Four of them were responses to the questions about being a good use of the respondents’ time. The undecided responses were fairly evenly distributed among the 3 groups of professionals. Questions 7 through 11 asked the participants for their opinions regarding the potential benefits of the target training behaviors for the quality and efficiency of patient care and patient outcomes. For all 5 questions, the majority of participants strongly agreed that the training had the potential to improve patient safety, care efficiency, team functioning, clarity regarding team leadership, communication, situational awareness, and mutual support. Item 12 asked participants to indicate whether they intended to apply the skills learned in their work environments. Again, the predominant response was ‘‘strongly agree.’’ Across these 12 items, physician respondents were more positive about the benefits of the training and the potential quality and care outcomes than were nurses and medical technicians, although all but 1 nurse responded favorably. The final item asked the participants to indicate whether they had already used any or all of the trained skills in their work environment. Twenty-six participants (58%) indicated that they had used some of the trained skills already. Two indicated that they had not, and 16 participants had not participated in trauma care since the completion of their training.
N.K. Roberts et al.
Impact of brief training
175
Table 3 Mean ratings (and standard deviations) of targeted and trained ad hoc trauma team behaviors during simulated trauma resuscitations at each stage in the team training cycle
Pretraining
Immediate 3-wk posttraining retention
Immediate Pretraining posttraining Pretraining vs 3-wk vs immediate vs 3-wk retention posttraining retention
Category (best possible score)
Mean 6 SD
Mean 6 SD
P
P
P
Organized (5) Efficiency (5) Supportive (5) Volunteered important information (5) Listened to information (5) Instructions performed (5) Clarified ambiguous orders (5) Confirmed completion of tasks (5) Leader clearly identifiable (5) Leader assigned tasks to team members by name or role (5) Leader managed noise appropriately (5) Leader coordinated communication and team activity effectively (5) Leader encouraged team members to volunteer key information during SMARTT Stepback (5) Trauma nontechnical skills Leadership (5) Cooperation (5) Communication (5) Decision making (5) Situational awareness coping with stress (5)
2.67 3.00 3.00 2.61 3.06 3.17 1.89 1.89 3.78 3.11
.001* .01* .007* .00* .001* .001* NS .00* NS .02*
.001* .01* NS NS .008* .018* NS NS NS NS
NS NS NS NS NS NS NS .03† NS NS
6 6 6 6 6 6 6 6 6 6
.66 .67 .50 .55 .58 .35 .82 .33 .51 .65
3.67 3.78 3.67 3.89 3.83 3.78 3.00 3.06 4.0 3.89
6 6 6 6 6 6 6 6 6 6
.35 .36 .25 .22 .25 .26 1.20 0.68 0 .22
Mean 6 SD 3.67 3.78 3.44 3.28 3.67 3.61 2.50 2.39 3.83 3.28
6 6 6 6 6 6 6 6 6 6
.50 .51 .46 .69 .25 .33 1.04 .55 0.35 0.67
2.41 6 1.20 3.50 6 .84 2.33 6 0.75 3.44 6 .46
3.43 6 0.79 NS 2.72 6 0.67 .003*
NS NS
NS NS
1.50 6 0.71 3.58 6 .58
3.25 6 1.17 .04*
NS
NS
NS .01 .015 NS .05
NS NS NS NS NS
3.72 2.89 2.56 3.67 3.67
6 6 6 6 6
0.36 0.65 0.46 0.71 0.71
4.67 4.61 4.06 4.61 4.67
6 6 6 6 6
.43 .42 .39 .70 .35
4.22 3.94 3.50 4.17 4.39
6 6 6 6 6
0.67 0.92 0.97 0.97 0.70
.002* .000* .000* .05* .005*
*Statistically significant differences in team and leader behaviors suggesting that the training had the desired effects. † Statistically significant differences in team behavior indicating significant reductions in desired team and leader behaviors at the 3-week retention end point compared with the immediate posttraining end point.
Table 3 provides a summary of the team results. Fifteen of 17 targeted team and leader behaviors significantly improved immediately after the training. One area in which team and leader behaviors did not improve was team members’ efforts to clarify ambiguous orders. The second involved whether the team leader was clearly identifiable. As can be seen in the pretraining simulation results, the judges’ ratings indicated that the team leader was easily identified before training. The third area involved team leaders’ management of noise during the trauma resuscitation.
Table 4
Comparing the pretraining with the 3-week retention results indicated that 7 areas had lasting training effects. Moreover, 4 of these 7 areas (efficiency, listened to information, orders were carried out, and cooperation and communication) are all critical indicators of effective team performance. One area for which there was a critical relapse in team behavior was team members’ confirmation when they completed tasks. This area requires additional attention. The number of team members who announced themselves before training was virtually zero, and this was true
Team members who were announced upon arrival, broken down by profession and by reporting rater
Time
Rater
Chief resident (n 5 9)
Pretraining
1 2 1 2 1 2
1 0 7 9 5 6
Immediate posttraining 3-week posttraining
Emergency room technician (n 5 6)
Bedside nurse (n 5 18)
Junior resident (n 5 18)
Respiratory therapist (n 5 5)
Total
0 0 0 5 0 5
0 0 3 7 1 4
1 0 3 8 4 6
0 0 0 5 1 1
2 0 13 34 11 22
176
The American Journal of Surgery, Vol 207, No 2, February 2014
Table 5 Simulations in which SMARTT Stepbacks occurred as coordinated events (all components occurred at the same time), broken down by time and by rater Time
Rater 1
Rater 2
Pretraining Immediate posttraining 3-week posttraining
0 (0%) 3 (33%) 5 (56%)
0 (0%) 6 (67%) 6 (67%)
regardless of profession (Table 4). The rate of introductions improved immediately after training and was maintained at a slightly lower rate after 3 weeks. However, the rate is still not 100%. We recorded that a SMARTT Stepback occurred only if all 6 components occurred at a single time (Table 5). As can be seen, the frequency of SMARTT Stepbacks increased after training, and the rate was sustained after 3 weeks. However, the rate was still less than the desired 100%. Table 6 indicates the number of SMARTT Stepback component behaviors that occurred at some point during each simulation, a much less restrictive indication of training success. In this table, a frequency of 9 in a cell would indicate that this behavior occurred 1 time in that simulation event. As can be seen, the 2 raters appeared to use different rating strategies for the pretraining. Their frequencies for the immediate posttraining and 3-week retention after training are more similar. Raters recorded each element occurring close to once per simulation event, except in the pretraining phase of the study.
Comments Our in situ simulation showed that residents were not clear on who was leading resuscitations. This resulted in a shifting leadership focus among the residents throughout the trauma resuscitation. Communication was fragmented, incomplete, and frequently interrupted, thus requiring repetition. The trauma bay was noisy, with several people often talking at once. These findings were confirmed subsequently in observations of actual trauma resuscitations in the trauma bay. Our training intervention, the SMARTT Stepback, was developed specifically to determine whether a brief training program would lead to changes in team member behaviors that would improve team performance in these areas. Table 6
Recently, 2 other studies have addressed similar training needs for ad hoc trauma team members.21,22 Both of those training interventions involved more training time on the part of participants compared with our study. Likewise, both studies included changes in traditional trauma outcome parameters as well as changes in team behaviors, whereas our study focused exclusively on changes in targeted team behaviors manifested in simulated trauma resuscitations. Finally, both studies included attending physicians as participants, whereas our study excluded attending physicians. The primary value that our study adds to the findings of these 2 studies lies in 2 areas. First and perhaps most important, our study used expert judges who were blinded to the stage of team training for participants. Our raters reviewed audio and video records of all teams performing all trauma simulations at every stage of training. Second, our study added the 3-week posttraining simulation to measure team retention of targeted team and individual behaviors. All 3 studies demonstrated that relatively brief training episodes can lead to changes in targeted team leadership, communication, and coordination behaviors. In our study, all behaviors were observed in simulations in which participants knew what behaviors were being observed and recorded, which may raise a question about whether the trained behaviors will persist in the actual trauma environment with real patients. However, the results from the studies of Steinemann et al19 and Capella et al20 provide some evidence to support transfer of the training to real trauma cases. Our study strengthens the collective knowledge from the 3 studies by blinding the raters to stage of training and thus minimizing the possibility that raters’ judgments are influenced by their knowledge of the training stage. Our results are also conservative in that the retention outcome measure occurs before the impact of the second debriefing. Although the results of our study provide evidence that the training is producing intended results, it is clear that the effect is not robust. The targeted behaviors are not present in all teams, and the sporadic team behavioral characteristics remind us that the behaviors are likely to fade absent practice and continued hospital leadership support in the form of policies and role modeling. For example, inspection of Tables 5 and 6 indicates that, at most, two thirds of the team leaders initiated complete SMARTT Stepback events during the simulations occurring after training. However, many of the elements of a SMARTT Stepback
SMARTT Stepback components that occurred during the simulation, broken down by time and by rater
Time
Rater
Situation
Management
Activity
Rapidity
Troubleshoot
Talk to me
Pretraining
1 2 1 2 1 2
8 0 9 6 8 7
8 0 9 6 8 8
7 0 9 6 7 8
2 0 8 6 6 5
2 0 8 5 4 6
6 0 9 5 6 7
Immediate posttraining 3-week posttraining
A single performance that included 1 instance of the behavior would have a frequency of 9 in each cell.
N.K. Roberts et al.
Impact of brief training
occurred during each posttraining simulation. This suggests partial success in this portion of the training but certainly also indicates that more training and reinforcement of these coordinated behaviors will be needed if SMARTT Stepbacks are expected to occur on a regular basis. Some of our results have multiple potential interpretations. For instance, the fact that there was no improvement in clarifying ambiguous orders could be due either to a lack of ambiguous orders to begin with or to a failure before and after training to seek clarification. We suspect the latter. The fact that the team leader was easily identifiable in both the pre and post conditions was likely an artifact of the simulation. Our observations both in situ and in the trauma bay convinced us that in the natural setting, identifying the leader is much more challenging, thus warranting explicit training. The lack of difference in the team leaders’ management of noise was also likely an artifact of the simulation condition. Independent observers noted that the simulations were much more quiet than the typical trauma simulation. They were certainly quieter than any of the live trauma resuscitations or the in situ trauma resuscitation we observed. All team leaders and prospective team leaders were trained and were given the opportunity to practice these behaviors as a part of this project. A nuclear group of nursing and emergency department technicians also received the training including the practice simulated trauma resuscitations. All other trauma team members are receiving the didactic training but will not have the opportunity to practice using the simulated trauma resuscitations. The trauma leadership, hospital leadership, and emergency medicine leadership are committed to encouraging all team members to incorporate these behaviors into their individual and team practices by providing mandatory training to the remaining trauma nurses and technicians and to the new general surgery and emergency department residents. We assessed our learners immediately after the intervention and again at a 3-week interval; admittedly, this does not provide convincing proof of long-term retention of the 7 skills, but it is the first experimental evidence of retention in the trauma literature. Finally, we want to echo the views of Capella et al20 that the logistics of providing training to teams made up of health care providers from different professions was one of the most challenging aspects of this project. Coordinating the scheduling of these training exercises involved working with 5 different hospital and residency program administrative bodies. It is no surprise to us that there is a great deal of talk about the desirability of multiprofessional team training, but the number of examples of such training in hospital settings is limited.
Acknowledgments We gratefully acknowledge the contributions of the MMC project committee that provided support and oversight on a
177 monthly basis: Mitchell Johnson, Clay Bahlow, Marcia Prater, Paula Gramley, and Drew Early.
References 1. Williams RG, Silverman R, Schwind C, et al. Surgeon information transfer and communication: factors affecting quality and efficiency of inpatient care. Ann Surg 2007;245:159–69. 2. Joint Commission on Accreditation of Healthcare Organizations. National patient safety goals for 2006. Available at: http://ww.jcaho. org/accredited1organizations/patient1safety/npsg.htm. 3. Kitch BT, Cooper JB, Zapol WM, et al. Handoffs causing patient harm: a survey of medical and surgical house staff. Jt Comm J Qual Patient Saf 2008;34:563–70. 4. Bates DW, Cullen DJ, Laird N, et al, ADE Prevention Study Group. Incidence of adverse drug events and potential adverse drug events. Implications for prevention. JAMA 1995;274:29–34. 5. Greenberg CC, Regenbogen SE, Studdert DM, et al. Patterns of communication breakdowns resulting in injury to surgical patients. J Am Coll Surg 2007;204:533–40. 6. King HB, Battles JB, Baker DP, et al. TeamSTEPPS: team strategies and tools to enhance performance and patient safety. Available at: http:// www.ahrq.gov/downloads/pub/advances2/vol3/advances-king_1.pdf. 7. Meier AH, Boehler ML, McDowell CM, et al. a surgical simulation curriculum for senior medical students based on TeamSTEPPS. Arch Surg 2012;147:761–6. 8. Ginnett R. Crews as groups: their formation and their leadership. In: Helmreich R, editor. Cockpit Resource Management. San Diego, CA: Academic Press; 1993. p. 71–98. 9. Hackman J. Teams, leaders and organizations: new directions for crew-oriented training. In: Helmreich R, editor. Cockpit Resource Management. San Diego, CA: Academic Press; 1993. p. 47–69. 10. Kanki BG, Lozito S, Foushee HC. Communication indexes of crew coordination. Aviat Space Envir Md 1989;60:56–60. 11. Marsch SCU, Muller C, Marquardt K, et al. Human factors affect the quality of cardiopulmonary resuscitation in simulated cardiac arrests. Resuscitation 2004;60:51. 12. Bergs EA, Rutten FL, Tadros T, et al. Communication during trauma resuscitation: do we know what is happening? Injury 2005;36:905–11. 13. Cooper S. Developing leaders for advanced life support: evaluation of a training programme. Resuscitation 2001;49:33–8. 14. Hunziker S, Buhlman C, Tschan F. Brief leadership instructions improve cardiopulmonary resuscitation in a high-fidelity simulation: A randomized controlled trial. Crit Care Med 2010;38:1086–91 [Erratum 2010;38:1510.]. 15. Rogers EM. Diffusion Of Innovations. 4th ed. New York: Free Press; 1995. 16. Hamman WR. The complexity of team training: what we have learned from aviation and its applications to medicine. Qual Saf Health Care 2004;13(suppl):i72–9. 17. Tschan F, Semmer NK, Gautschi D, et al. Leading to recovery: group performance and coordinative activities in medical emergency driven groups. Hum Perf 2006;19:277. 18. Marsch SC, Tschan F, Semmer N, et al. Performance of first responders in simulated cardiac arrests. Crit Care Med 2005;33:963–7. 19. Rosen MA, Salas E, Wu TS, et al. Promoting teamwork: an eventbased approach to simulation-based teamwork training for emergency medicine residents. Acad Emerg Med 2008;15:1190–8. 20. Kirkpatrick DL. Evaluation of training. In: Craig RL, Bittel LR, editors. Training and Development Handbook. New York: McGraw-Hill; 1967. p. 87–112. 21. Steinemann S, Berg B, Skinner A, et al. In situ, multidisciplinary, simulation-based teamwork training improves early trauma care. J Surg Educ 2011;68:472–7. 22. Capella J, Smith S, Philp A, et al. Teamwork training improves the clinical care of trauma patients. J Surg Educ 2010;67:439–43.
178
Appendix 1 Scenarios Used for Simulated Trauma Resuscitations Scenario A was a 24-year-old man found lying face down on the ground, having sustained a gunshot wound to his left chest and upper right arm. In this case, the patient had a hemopneumothorax and needed a needle decompression or a chest tube immediately. The patient was in a semiconscious state and quickly deteriorated without appropriate management of the chest tube and fluid resuscitation. Once the patient had stabilized, the learner needed to state that the patient would go to computed tomography quickly. At that time, the scenario was terminated. Scenario B was a 50-year-old man who was involved in an explosion at his farm when the gasoline tank he was welding exploded. He was thrown 50 feet and had a 5minute loss of consciousness. He was confused, tachycardic, and hypotensive upon arrival in the trauma bay. He also had multiple abrasions and a near amputation of the right arm near the elbow. This patient had low oxygen saturation and needed to be intubated immediately. If the learners did not intubate quickly, the patient quickly deteriorated without blood pressure or pulse. If the learners did intubate, they discovered that the oxygen saturation was still low, and the patient was absent of breath sounds on the right. The patient needed a needle decompression and chest tube within 2 minutes or he deteriorated, becoming more hypotensive and tachycardic. Once the patient was stable, the learners needed to work on stabilizing the near amputation of the right arm and state that he needed to go to computed tomography quickly. Scenario C was a 25-year-old male restrained driver involved in a head-on high-speed motor vehicle crash. The patient was pinned between the dashboard and the seat and required a 30-minute extrication. He was awake but groggy upon arrival in the trauma bay. He had decreased breath sounds on the left, he had a tender rigid abdomen, and he was tachycardic, hypotensive, and tachypneic. The team was required to recognize that this patient needed to be intubated and have a chest tube placed quickly. If they failed to recognize this the patient deteriorated quickly.
The American Journal of Surgery, Vol 207, No 2, February 2014 Once the patient was intubated and a chest tube was placed, the team needed to recognize that the patient had a tender rigid abdomen and order a focused assessment with sonography for trauma scan. These results were negative, and the patient was stabilized. Finally, the team was expected to announce that the patient needed to proceed to computed tomography.
Appendix 2 Development of Simulation Scenarios After reviewing the 30 simulation cases in the METI Disaster Medical Readiness and Emergency Medical Services Learning Modules, cases developed locally, and team training cases developed for the Association of Program Directors in Surgery curriculum under the leadership of Gary Dunnington, M.D., at the Southern Illinois University School of Medicine, members of the Core Training Planning and Development Group developed 4 simulated trauma resuscitation scenarios for use in the training. This involved developing (1) the educational rationale and the learning objectives for the simulation scenarios; (2) the case stem for the learners (pertinent patient and scenario information: task, location, physician/help availability, and equipment availability; (3) background and briefing information for facilitators and coordinators (patient data background and baseline patient state, baseline simulator state); (4) supporting files (chest x-ray, echocardiography, assessment, handouts, etc); (5) a list of alterations that will occur during the simulation; (6) the trigger to move to the next state; (7) the expected learner actions; and (8) the teaching points for the debriefing. One case was used exclusively for demonstrating the target behaviors as part of the training intervention. The other 3 were alternated for use as (1) preintervention exercises designed to determine the rate of team target behaviors before training, (2) opportunities to practice using the target behaviors at the end of the training session, and (3) retention simulation exercises to document retention of the target behaviors approximately 3 weeks after training.
