EDUCATION
Getting a Head Start: High-Fidelity, Simulation-Based Operating Room Team Training of Interprofessional Students John T Paige, MD, FACS, Deborah D Garbee, PhD, Valeriy Kozmenko, MD, Qingzhao Yu, Lyubov Kozmenko, BSN, Tong Yang, MD, Laura Bonanno, DNP, William Swartz, PhD
PhD,
Effective teamwork in the operating room (OR) is often undermined by the “silo mentality” of the differing professions. Such thinking is formed early in one’s professional experience and is fostered by undergraduate medical and nursing curricula lacking interprofessional education. We investigated the immediate impact of conducting interprofessional student OR team training using high-fidelity simulation (HFS) on students’ team-related attitudes and behaviors. STUDY DESIGN: Ten HFS OR interprofessional student team training sessions were conducted involving 2 standardized HFS scenarios, each of which was followed by a structured debriefing that targeted team-based competencies. Pre- and post-session mean scores were calculated and analyzed for 15 Likert-type items measuring self-efficacy in teamwork competencies using the t-test. Additionally, mean scores of observer ratings of team performance after each scenario and participant ratings after the second scenario for an 11-item Likert-type teamwork scale were calculated and analyzed using one-way ANOVA and t-test. RESULTS: Eighteen nursing students, 20 nurse anesthetist students, and 28 medical students participated in the training. Statistically significant gains from mean pre- to post-training scores occurred on 11 of the 15 self-efficacy items. Statistically significant gains in mean observer performance scores were present on all 3 subscales of the teamwork scale from the first scenario to the second. A statistically significant difference was found in comparisons of mean observer scores with mean participant scores for the team-based behaviors subscale. CONCLUSIONS: High-fidelity simulation OR interprofessional student team training improves students’ team-based attitudes and behaviors. Students tend to overestimate their team-based behaviors. (J Am Coll Surg 2014;218:140e149. 2014 by the American College of Surgeons)
BACKGROUND:
communication,1 ineffective interpersonal skills,2 interprofessional tension,3 poor team interaction,4 and divergent interprofessional interpretations of the quality of collaboration5 combine to impact both patient care processes6,7 and outcomes.8 Consequently, the Joint Commission9 and the AHRQ10 have made teamwork a priority. A major contributor to the inadequate teamwork of ORs is a culture characterized by interprofessional friction, a “silo mentality,”11 and “tribalism” among the professions.12 Multiprofessional interaction is favored over true interprofessional collaboration,13 abetted by a rigid hierarchical structure impeding meaningful communication and collaboration.14 This toxic environment profoundly influences health care professional students as a “hidden curriculum.” One approach to counteracting negative cultural traits within the OR is to increase opportunities for positive practice and modeling of effective team-based competencies by
Although effective operating room (OR) teamwork is a critical component for the safe delivery of care, its implementation in practice is far from ideal. Frequently, failed Disclosure Information: Dr Paige is the co-editor of Simulation in Radiology; Dr Kozmenko is the co-patent holder for the software interface used to run the scenarios in this project. All other authors have nothing to disclose. This project was supported by a 2008-2009 Educational Enhancement Grant from the LSU Health New Orleans Health Sciences Center, Academy for the Advancement of Educational Scholarship. Abstract presented at the 2011 Meeting of the Consortium of American College of Surgeons Accredited Education Institutes, Chicago, IL, March 2011. Received June 17, 2013; Revised August 22, 2013; Accepted September 16, 2013. From the Department of Surgery (Paige), School of Nursing (Garbee, L Kozmenko, Bonanno), Department of Anesthesia (V Kozmenko), School of Public Health (Yu), Department of Pathology (Yang), and Department of Cell Biology and Anatomy (Swartz), LSU Health New Orleans Health Sciences Center, New Orleans, LA. Correspondence address: John T Paige, MD, Department of Surgery, LSU Health New Orleans Health Sciences Center, 1542 Tulane Ave, Room 736, New Orleans, LA 70112. email: [email protected]
ª 2014 by the American College of Surgeons Published by Elsevier Inc.
140
ISSN 1072-7515/13/$36.00 http://dx.doi.org/10.1016/j.jamcollsurg.2013.09.006
Vol. 218, No. 1, January 2014
Paige et al
Abbreviations and Acronyms
ACR HFS HPS IPE IPS OR ORTAS SBT TBB VOR
¼ ¼ ¼ ¼ ¼ ¼ ¼ ¼ ¼ ¼
adaptive communication and response high-fidelity simulation human patient simulator interprofessional education interprofessional students operating room Operating Room Teamwork Assessment Scales simulation-based training team-based behavior virtual operating room
students in an interprofessional education (IPE) setting. Interprofessional education improves students’ attitudes toward teamwork skills15 and has led to gains related to professional practice and patient care.16 Both the Lucien Leape Institute17 and the Institute of Medicine18 have emphasized IPE and teamwork. High-fidelity simulation (HFS) is an attractive methodology for IPE. It provides a realistic and safe learning environment in which students can encounter uncommon clinical situations and learn from “mistakes” without repercussions.19 Numerous IPE formats of simulation-based training (SBT) have demonstrated improvement in students’ team-based attitudes,20-23 perceptions of collaboration,22-27 and team-based performance.28-30 Simulation-based training has been used to teach senior medical students team-based competencies during a surgical elective.31 It has not focused, however, on the interprofessional OR team for training students. This article investigates the feasibility and effectiveness of HFS interprofessional student (IPS) OR team training.
METHODS Study design A quasi-experimental pre-/post-intervention comparison design was used for this study. It was chosen for several reasons. Foremost, such a design allowed each student to participate in the learning intervention. In this manner, no individual was excluded from the HFS experience to satisfy randomized controlled design requirements. The quasi-experimental structure allowed for each participant to serve as his/her own control because pre-intervention data were matched with post-intervention data for each individual. In this manner, demographic variables were the pre-intervention (ie, no training) and post-intervention (ie, training) “groups.” Second, given the time structure of the intervention (ie, occurring as one part of intensive, time-limited electives in the Schools of Medicine and Nursing), it was more feasible than performing a crossover study. Finally, the quasi-experimental design made IRB
Simulation-Based Operating Room Team Training
141
approval relatively straightforward as compared with a randomized controlled design. Training setting Training occurred at an academic urban health sciences center during spring 2009 within a previously described virtual OR (VOR)32 at the School of Medicine’s Russell C Klein MD (‘59) Center for Advanced Practice. Equipment included a full-scale computer-operated human patient simulator (HPS) mannequin (CAE Inc.) and an inanimate torso procedural training model (Simulab Corporation). The HPS mimics complex physiologic responses. The torso model has a replaceable outer skin, an upper abdomen mold, removable small bowel, and a replaceable model of an abdominal aorta capable of “bleeding” via a portable pump and fluid reservoir. Additionally, a recess in the HPS upper arm allowed placement of a hollowed-out tissue pad (Simulab Corporation) to simulate a soft-tissue mass. Simulator set up has been described previously.33 Training format Each 2-hour training session involved a dual-scenario format without any preceding didactic teaching. A brief orientation preceded the first scenario. Students then participated in a facilitator-led, focused debriefing within the VOR immediately after the scenario. They engaged in reflective practice related to the following team-based competencies: shared mental model, role clarity, situation awareness, cross monitoring, open communication, resource management, flattened hierarchy, anticipatory response, and mental rehearsal. The debriefing structure drew on Thiagi’s 6 phases of debriefing,34 Pearson and Smith’s 3 questions,35 and the plus/delta technique.36 After another scenario with debriefing, the session concluded with student reflection on strategies for translation of skills to clinical practice. At least 3 instructors facilitated a session. One (VK) operated the HPS simulator. Two (JTP, DDG) served as debriefing facilitators. Trained observers rated student team-based behavior. Expedited IRB approval was obtained before initiation of the study protocol. Training participants Twenty-eight (43% of all participants) 4th-year medical students who were enrolled in a senior elective course in surgical anatomy participated in the SBT. Eighteen (27% total) undergraduate nursing students taking a perioperative nursing course, and 20 junior-level nurse anesthesia students (30% total, 300 to 350 OR case average) also participated.
142
Paige et al
Simulation-Based Operating Room Team Training
Each OR team had 2 medical students (surgeon and first assistant roles), 2 undergraduate nursing students (circulating and scrub nurse roles), and 2 nurse anesthesia students (primary and secondary anesthetist roles). Students switched roles within their discipline (eg, an undergraduate nurse moving from scrub nurse to circulating nurse) for the second scenario. In cases where more than 2 students from a particular profession were present, 2 students would participate in the first scenario and the other 2 would observe. Students then would switch for the second scenario; all present participated in debriefings. Training scenarios Two standardized, authentic scenarios were used. The first scenario involved a life-threatening intra-abdominal hemorrhage from a stab wound. The second scenario involved local anesthetic toxicity from a regional upper arm block. The scenarios used a software algorithm designed to respond to team actions and decisions.37 Evaluation All participants completed pre- and post-training questionnaires that included a 15-item section with Likert-type questions measuring perceived self-efficacy for targeted team-based competencies in an actual OR environment (1 ¼ not confident at all, to 6 ¼ completely confident). Responses were collated; descriptive statistics were calculated for pre- and post-session responses; and paired t-test with Bonferroni adjustment was performed. Trained observers evaluated team-based performance for each scenario using the Operating Room Teamwork Assessment Scales (ORTAS).37-39 Two scales were used: a 5-item multisource evaluation individual performance scale focusing on team-based behaviors (TBB) of the individual and a 6-item overall teamwork scale that contained a 3-item shared mental model subscale and a 3-item adaptive communication and response (ACR) subscale. Each item contained behavioral anchors for a 6-point Likert-type scale (1 ¼ definitely no, to 6 ¼ definitely yes). The multisource evaluations were conducted based on the role of the participants: surgeon, nurse anesthetist, circulating nurse, and scrub nurse. Participants evaluated self-, peer-, and overall team performance after completion of the second scenario using the ORTAS. Mean item and subscale scores were calculated for observer and participant ratings. Differences between mean calculated observed performances after each scenario were evaluated using a one-way ANOVA. Differences between mean calculated observer and participant ratings were evaluated using paired samples t-test. For missing data, an ad hoc method was used in which any incomplete paired dataset was discarded before comparison.
J Am Coll Surg
Observers underwent a 2-hour training session before rating students’ performances that involved didactic instruction and video ratings of behaviors to calibrate for accuracy. Observer-based rating immediately followed each scenario. Four trained observers rated both scenarios of a session except for 1 session in which only 3 observers rated the second scenario and another session were only 3 raters were present during the entire session. A generalizability study was conducted to determine coefficients for both relative and absolute decisions made by the observers to quantify the degree of error variance and reliability of the scoring. The questionnaire contained 3 open-ended questions about the simulation experience. These responses were transcribed then analyzed using the qualitative methods of Miles and Huberman.40 Responses were first read, reread, listed, coded, and analyzed for themes. Trustworthiness was addressed through attempts at data triangulation, use of participant quotes, and looking for discrepant cases.
RESULTS Demographics Sixty-six students divided into 10 groups trained during a 1-month period: 7 groups of 6 participants (groups 1, 3, 5, 6, 7, 9, and 10) and 3 groups of 8 participants (groups 2, 4, and 8). Each 6-participant group had 2 students from each discipline, except group 6, which had 4 medical students and 2 nurse anesthesia students. For this group, 2 medical students served as circulating and scrub nurses for the first scenario. They then switched with the other 2 medical students to play surgeon and assistant in the second scenario. Each 8-participant group had 4 medical students, 2 nurse anesthesia students, and 2 undergraduate nursing students. Sex distribution was equal among the participants. Participants were predominantly Caucasian (80.3%) with some representation of African Americans (12.1%) and Asians (4.5%). Quantitative analysis Means for participant pre-test and post-test scores of the self-efficacy items for team-based competencies are displayed in Table 1. A significant increase from pre- to post-training mean item scores was found in 11 of 15 comparisons (p < 0.001). Of the significant comparisons, the mean pre-scores ranged from 3.46 to 4.10 and the mean post-scores ranged from 4.24 to 4.75. Statistically significant gains in mean observer rated performance scores were present on all 3 subscales comparing the second scenario with the first (Table 2). These significant increases were present for each role evaluated by the observers (Table 3). Only a statistically
Vol. 218, No. 1, January 2014
Table 1.
Paige et al
Simulation-Based Operating Room Team Training
143
Summary of Participant Pre- to Post-Session Self-Efficacy Mean Scores
Item statement
Pre-session score, mean
Post-session score, mean
t Value
df
p Value
4.48
4.61
0.882
63
0.381
4.44
4.67
1.591
63
0.117
3.89
4.48
3.393
63
*
3.46
4.25
5.162
62
*
3.67
4.24
3.618
62
*
3.70
4.48
5.584
63
*
3.89
4.57
4.721
62
*
3.77
4.36
4.400
63
*
4.10
4.75
4.819
60
*
4.05
4.61
3.813
63
*
4.30
4.46
1.032
62
0.306
3.95
4.48
3.578
62
*
3.92
4.52
3.750
62
*
3.94
4.52
3.878
63
*
3.79
4.25
2.853
62
0.006
Have a common understanding of patient’s condition Have a common understanding of the specific goals to be achieved in the OR Know clearly the task responsibilities of each team member Achieve sufficient familiarity with how each team member will approach his/her task responsibilities (eg, strengths, weaknesses, preferences) Have all of the anticipated human and material resources ready and “at hand” for use Use cues within the OR situation to coordinate my tasks with others Use my understanding of a patient’s situation to anticipate team members’ needs Monitor my own and other team members efforts to ensure that proper procedures and tasks occur as expected Adapt my performance to accommodate changes that might occur Use specific communication strategies to confirm that messages are received and the content is accurately understood (ie, closed-loop communication) Interact with others to maximize their strengths and manage the workload effectively Facilitate quality and continuous improvement (eg, encouraging others, speaking up when concerns arise, offering critique) Use strategies effectively to promote team cohesion and effective work interactions Use strategies to effectively provide feedback to OR team members Use strategies effectively to resolve differences/disputes among team members *p < 0.001. df, degree of freedom; OR, operating room.
significant difference was found in TBB subscale scores comparing observer with participant ratings (Table 4). Generalizability study of observer ratings revealed acceptable relative and absolute coefficients for multiple observers (Table 5). Qualitative analysis Themes that emerged for the most beneficial aspects of the SBT were enhanced communication, impact of debriefing, and realism. Examples of the enhanced communication theme included “Emphasizing that communication between team members is of the most importance made me realize where my faults as a team
leader can be improved”; “It showed lack of communication and pointed out things we would normally not be taught”; “This is an example of how some situations cannot flow smoothly when people do not communicate and are not comfortable”; and “For each discipline to recognize the importance of introducing themselves and using each other’s names.” The impact of the debriefing theme was demonstrated by the following comments: “Brainstorming our performance made us reflect on our performance and what we could have done different”; “Going over the case and explaining what went wrong and how to correct it”; and “Debriefing helped with roles and becoming a team.” Participant comments
144
Paige et al
J Am Coll Surg
Simulation-Based Operating Room Team Training
Table 2. Summary of Observer-Rated Operating Room Teamwork Assessment Scales Subscale Analysis for High-Fidelity Simulation Training: Scenario 1 vs Scenario 2 n
Scenario 1 Mean
SD
n
Scenario 2 Mean
SD
Change*
p Valuey
z
Individual performance (5 items) Team-based behaviors, 5 items Overall teamwork (6 items) Shared mental model, 3 items Adaptive communication and response, 3 items
10
2.42
0.62
10
3.98
0.44
1.56
Getting a Head Start: High-Fidelity, Simulation-Based Operating Room Team Training of Interprofessional Students John T Paige, MD, FACS, Deborah D Garbee, PhD, Valeriy Kozmenko, MD, Qingzhao Yu, Lyubov Kozmenko, BSN, Tong Yang, MD, Laura Bonanno, DNP, William Swartz, PhD
PhD,
Effective teamwork in the operating room (OR) is often undermined by the “silo mentality” of the differing professions. Such thinking is formed early in one’s professional experience and is fostered by undergraduate medical and nursing curricula lacking interprofessional education. We investigated the immediate impact of conducting interprofessional student OR team training using high-fidelity simulation (HFS) on students’ team-related attitudes and behaviors. STUDY DESIGN: Ten HFS OR interprofessional student team training sessions were conducted involving 2 standardized HFS scenarios, each of which was followed by a structured debriefing that targeted team-based competencies. Pre- and post-session mean scores were calculated and analyzed for 15 Likert-type items measuring self-efficacy in teamwork competencies using the t-test. Additionally, mean scores of observer ratings of team performance after each scenario and participant ratings after the second scenario for an 11-item Likert-type teamwork scale were calculated and analyzed using one-way ANOVA and t-test. RESULTS: Eighteen nursing students, 20 nurse anesthetist students, and 28 medical students participated in the training. Statistically significant gains from mean pre- to post-training scores occurred on 11 of the 15 self-efficacy items. Statistically significant gains in mean observer performance scores were present on all 3 subscales of the teamwork scale from the first scenario to the second. A statistically significant difference was found in comparisons of mean observer scores with mean participant scores for the team-based behaviors subscale. CONCLUSIONS: High-fidelity simulation OR interprofessional student team training improves students’ team-based attitudes and behaviors. Students tend to overestimate their team-based behaviors. (J Am Coll Surg 2014;218:140e149. 2014 by the American College of Surgeons)
BACKGROUND:
communication,1 ineffective interpersonal skills,2 interprofessional tension,3 poor team interaction,4 and divergent interprofessional interpretations of the quality of collaboration5 combine to impact both patient care processes6,7 and outcomes.8 Consequently, the Joint Commission9 and the AHRQ10 have made teamwork a priority. A major contributor to the inadequate teamwork of ORs is a culture characterized by interprofessional friction, a “silo mentality,”11 and “tribalism” among the professions.12 Multiprofessional interaction is favored over true interprofessional collaboration,13 abetted by a rigid hierarchical structure impeding meaningful communication and collaboration.14 This toxic environment profoundly influences health care professional students as a “hidden curriculum.” One approach to counteracting negative cultural traits within the OR is to increase opportunities for positive practice and modeling of effective team-based competencies by
Although effective operating room (OR) teamwork is a critical component for the safe delivery of care, its implementation in practice is far from ideal. Frequently, failed Disclosure Information: Dr Paige is the co-editor of Simulation in Radiology; Dr Kozmenko is the co-patent holder for the software interface used to run the scenarios in this project. All other authors have nothing to disclose. This project was supported by a 2008-2009 Educational Enhancement Grant from the LSU Health New Orleans Health Sciences Center, Academy for the Advancement of Educational Scholarship. Abstract presented at the 2011 Meeting of the Consortium of American College of Surgeons Accredited Education Institutes, Chicago, IL, March 2011. Received June 17, 2013; Revised August 22, 2013; Accepted September 16, 2013. From the Department of Surgery (Paige), School of Nursing (Garbee, L Kozmenko, Bonanno), Department of Anesthesia (V Kozmenko), School of Public Health (Yu), Department of Pathology (Yang), and Department of Cell Biology and Anatomy (Swartz), LSU Health New Orleans Health Sciences Center, New Orleans, LA. Correspondence address: John T Paige, MD, Department of Surgery, LSU Health New Orleans Health Sciences Center, 1542 Tulane Ave, Room 736, New Orleans, LA 70112. email: [email protected]
ª 2014 by the American College of Surgeons Published by Elsevier Inc.
140
ISSN 1072-7515/13/$36.00 http://dx.doi.org/10.1016/j.jamcollsurg.2013.09.006
Vol. 218, No. 1, January 2014
Paige et al
Abbreviations and Acronyms
ACR HFS HPS IPE IPS OR ORTAS SBT TBB VOR
¼ ¼ ¼ ¼ ¼ ¼ ¼ ¼ ¼ ¼
adaptive communication and response high-fidelity simulation human patient simulator interprofessional education interprofessional students operating room Operating Room Teamwork Assessment Scales simulation-based training team-based behavior virtual operating room
students in an interprofessional education (IPE) setting. Interprofessional education improves students’ attitudes toward teamwork skills15 and has led to gains related to professional practice and patient care.16 Both the Lucien Leape Institute17 and the Institute of Medicine18 have emphasized IPE and teamwork. High-fidelity simulation (HFS) is an attractive methodology for IPE. It provides a realistic and safe learning environment in which students can encounter uncommon clinical situations and learn from “mistakes” without repercussions.19 Numerous IPE formats of simulation-based training (SBT) have demonstrated improvement in students’ team-based attitudes,20-23 perceptions of collaboration,22-27 and team-based performance.28-30 Simulation-based training has been used to teach senior medical students team-based competencies during a surgical elective.31 It has not focused, however, on the interprofessional OR team for training students. This article investigates the feasibility and effectiveness of HFS interprofessional student (IPS) OR team training.
METHODS Study design A quasi-experimental pre-/post-intervention comparison design was used for this study. It was chosen for several reasons. Foremost, such a design allowed each student to participate in the learning intervention. In this manner, no individual was excluded from the HFS experience to satisfy randomized controlled design requirements. The quasi-experimental structure allowed for each participant to serve as his/her own control because pre-intervention data were matched with post-intervention data for each individual. In this manner, demographic variables were the pre-intervention (ie, no training) and post-intervention (ie, training) “groups.” Second, given the time structure of the intervention (ie, occurring as one part of intensive, time-limited electives in the Schools of Medicine and Nursing), it was more feasible than performing a crossover study. Finally, the quasi-experimental design made IRB
Simulation-Based Operating Room Team Training
141
approval relatively straightforward as compared with a randomized controlled design. Training setting Training occurred at an academic urban health sciences center during spring 2009 within a previously described virtual OR (VOR)32 at the School of Medicine’s Russell C Klein MD (‘59) Center for Advanced Practice. Equipment included a full-scale computer-operated human patient simulator (HPS) mannequin (CAE Inc.) and an inanimate torso procedural training model (Simulab Corporation). The HPS mimics complex physiologic responses. The torso model has a replaceable outer skin, an upper abdomen mold, removable small bowel, and a replaceable model of an abdominal aorta capable of “bleeding” via a portable pump and fluid reservoir. Additionally, a recess in the HPS upper arm allowed placement of a hollowed-out tissue pad (Simulab Corporation) to simulate a soft-tissue mass. Simulator set up has been described previously.33 Training format Each 2-hour training session involved a dual-scenario format without any preceding didactic teaching. A brief orientation preceded the first scenario. Students then participated in a facilitator-led, focused debriefing within the VOR immediately after the scenario. They engaged in reflective practice related to the following team-based competencies: shared mental model, role clarity, situation awareness, cross monitoring, open communication, resource management, flattened hierarchy, anticipatory response, and mental rehearsal. The debriefing structure drew on Thiagi’s 6 phases of debriefing,34 Pearson and Smith’s 3 questions,35 and the plus/delta technique.36 After another scenario with debriefing, the session concluded with student reflection on strategies for translation of skills to clinical practice. At least 3 instructors facilitated a session. One (VK) operated the HPS simulator. Two (JTP, DDG) served as debriefing facilitators. Trained observers rated student team-based behavior. Expedited IRB approval was obtained before initiation of the study protocol. Training participants Twenty-eight (43% of all participants) 4th-year medical students who were enrolled in a senior elective course in surgical anatomy participated in the SBT. Eighteen (27% total) undergraduate nursing students taking a perioperative nursing course, and 20 junior-level nurse anesthesia students (30% total, 300 to 350 OR case average) also participated.
142
Paige et al
Simulation-Based Operating Room Team Training
Each OR team had 2 medical students (surgeon and first assistant roles), 2 undergraduate nursing students (circulating and scrub nurse roles), and 2 nurse anesthesia students (primary and secondary anesthetist roles). Students switched roles within their discipline (eg, an undergraduate nurse moving from scrub nurse to circulating nurse) for the second scenario. In cases where more than 2 students from a particular profession were present, 2 students would participate in the first scenario and the other 2 would observe. Students then would switch for the second scenario; all present participated in debriefings. Training scenarios Two standardized, authentic scenarios were used. The first scenario involved a life-threatening intra-abdominal hemorrhage from a stab wound. The second scenario involved local anesthetic toxicity from a regional upper arm block. The scenarios used a software algorithm designed to respond to team actions and decisions.37 Evaluation All participants completed pre- and post-training questionnaires that included a 15-item section with Likert-type questions measuring perceived self-efficacy for targeted team-based competencies in an actual OR environment (1 ¼ not confident at all, to 6 ¼ completely confident). Responses were collated; descriptive statistics were calculated for pre- and post-session responses; and paired t-test with Bonferroni adjustment was performed. Trained observers evaluated team-based performance for each scenario using the Operating Room Teamwork Assessment Scales (ORTAS).37-39 Two scales were used: a 5-item multisource evaluation individual performance scale focusing on team-based behaviors (TBB) of the individual and a 6-item overall teamwork scale that contained a 3-item shared mental model subscale and a 3-item adaptive communication and response (ACR) subscale. Each item contained behavioral anchors for a 6-point Likert-type scale (1 ¼ definitely no, to 6 ¼ definitely yes). The multisource evaluations were conducted based on the role of the participants: surgeon, nurse anesthetist, circulating nurse, and scrub nurse. Participants evaluated self-, peer-, and overall team performance after completion of the second scenario using the ORTAS. Mean item and subscale scores were calculated for observer and participant ratings. Differences between mean calculated observed performances after each scenario were evaluated using a one-way ANOVA. Differences between mean calculated observer and participant ratings were evaluated using paired samples t-test. For missing data, an ad hoc method was used in which any incomplete paired dataset was discarded before comparison.
J Am Coll Surg
Observers underwent a 2-hour training session before rating students’ performances that involved didactic instruction and video ratings of behaviors to calibrate for accuracy. Observer-based rating immediately followed each scenario. Four trained observers rated both scenarios of a session except for 1 session in which only 3 observers rated the second scenario and another session were only 3 raters were present during the entire session. A generalizability study was conducted to determine coefficients for both relative and absolute decisions made by the observers to quantify the degree of error variance and reliability of the scoring. The questionnaire contained 3 open-ended questions about the simulation experience. These responses were transcribed then analyzed using the qualitative methods of Miles and Huberman.40 Responses were first read, reread, listed, coded, and analyzed for themes. Trustworthiness was addressed through attempts at data triangulation, use of participant quotes, and looking for discrepant cases.
RESULTS Demographics Sixty-six students divided into 10 groups trained during a 1-month period: 7 groups of 6 participants (groups 1, 3, 5, 6, 7, 9, and 10) and 3 groups of 8 participants (groups 2, 4, and 8). Each 6-participant group had 2 students from each discipline, except group 6, which had 4 medical students and 2 nurse anesthesia students. For this group, 2 medical students served as circulating and scrub nurses for the first scenario. They then switched with the other 2 medical students to play surgeon and assistant in the second scenario. Each 8-participant group had 4 medical students, 2 nurse anesthesia students, and 2 undergraduate nursing students. Sex distribution was equal among the participants. Participants were predominantly Caucasian (80.3%) with some representation of African Americans (12.1%) and Asians (4.5%). Quantitative analysis Means for participant pre-test and post-test scores of the self-efficacy items for team-based competencies are displayed in Table 1. A significant increase from pre- to post-training mean item scores was found in 11 of 15 comparisons (p < 0.001). Of the significant comparisons, the mean pre-scores ranged from 3.46 to 4.10 and the mean post-scores ranged from 4.24 to 4.75. Statistically significant gains in mean observer rated performance scores were present on all 3 subscales comparing the second scenario with the first (Table 2). These significant increases were present for each role evaluated by the observers (Table 3). Only a statistically
Vol. 218, No. 1, January 2014
Table 1.
Paige et al
Simulation-Based Operating Room Team Training
143
Summary of Participant Pre- to Post-Session Self-Efficacy Mean Scores
Item statement
Pre-session score, mean
Post-session score, mean
t Value
df
p Value
4.48
4.61
0.882
63
0.381
4.44
4.67
1.591
63
0.117
3.89
4.48
3.393
63
*
3.46
4.25
5.162
62
*
3.67
4.24
3.618
62
*
3.70
4.48
5.584
63
*
3.89
4.57
4.721
62
*
3.77
4.36
4.400
63
*
4.10
4.75
4.819
60
*
4.05
4.61
3.813
63
*
4.30
4.46
1.032
62
0.306
3.95
4.48
3.578
62
*
3.92
4.52
3.750
62
*
3.94
4.52
3.878
63
*
3.79
4.25
2.853
62
0.006
Have a common understanding of patient’s condition Have a common understanding of the specific goals to be achieved in the OR Know clearly the task responsibilities of each team member Achieve sufficient familiarity with how each team member will approach his/her task responsibilities (eg, strengths, weaknesses, preferences) Have all of the anticipated human and material resources ready and “at hand” for use Use cues within the OR situation to coordinate my tasks with others Use my understanding of a patient’s situation to anticipate team members’ needs Monitor my own and other team members efforts to ensure that proper procedures and tasks occur as expected Adapt my performance to accommodate changes that might occur Use specific communication strategies to confirm that messages are received and the content is accurately understood (ie, closed-loop communication) Interact with others to maximize their strengths and manage the workload effectively Facilitate quality and continuous improvement (eg, encouraging others, speaking up when concerns arise, offering critique) Use strategies effectively to promote team cohesion and effective work interactions Use strategies to effectively provide feedback to OR team members Use strategies effectively to resolve differences/disputes among team members *p < 0.001. df, degree of freedom; OR, operating room.
significant difference was found in TBB subscale scores comparing observer with participant ratings (Table 4). Generalizability study of observer ratings revealed acceptable relative and absolute coefficients for multiple observers (Table 5). Qualitative analysis Themes that emerged for the most beneficial aspects of the SBT were enhanced communication, impact of debriefing, and realism. Examples of the enhanced communication theme included “Emphasizing that communication between team members is of the most importance made me realize where my faults as a team
leader can be improved”; “It showed lack of communication and pointed out things we would normally not be taught”; “This is an example of how some situations cannot flow smoothly when people do not communicate and are not comfortable”; and “For each discipline to recognize the importance of introducing themselves and using each other’s names.” The impact of the debriefing theme was demonstrated by the following comments: “Brainstorming our performance made us reflect on our performance and what we could have done different”; “Going over the case and explaining what went wrong and how to correct it”; and “Debriefing helped with roles and becoming a team.” Participant comments
144
Paige et al
J Am Coll Surg
Simulation-Based Operating Room Team Training
Table 2. Summary of Observer-Rated Operating Room Teamwork Assessment Scales Subscale Analysis for High-Fidelity Simulation Training: Scenario 1 vs Scenario 2 n
Scenario 1 Mean
SD
n
Scenario 2 Mean
SD
Change*
p Valuey
z
Individual performance (5 items) Team-based behaviors, 5 items Overall teamwork (6 items) Shared mental model, 3 items Adaptive communication and response, 3 items
10
2.42
0.62
10
3.98
0.44
1.56
