ORIGINAL ARTICLE
Virtual-world hospital simulation for real-world disaster response: Design and validation of a virtual reality simulator for mass casualty incident management Philip H. Pucher, MRCS, Nicola Batrick, FRCS, Dave Taylor, Muzzafer Chaudery, MRCS, Daniel Cohen, MRCS, and Ara Darzi, FRCS, London, United Kingdom
BACKGROUND: Mass casualty incidents are unfortunately becoming more common. The coordination of mass casualty incident response is highly complex. Currently available options for training, however, are limited by either lack of realism or prohibitive expense and by a lack of assessment tools. Virtual worlds represent a potentially cost-effective, immersive, and easily accessible platform for training and assessment. The aim of this study was to assess feasibility of a novel virtual-worldsYbased system for assessment and training in major incident response. METHODS: Clinical areas were modeled within a virtual, online hospital. A major incident, incorporating virtual casualties, allowed multiple clinicians to simultaneously respond with appropriate in-world management and transfer plans within limits of the hospital’s available resources. Errors, delays, and completed actions were recorded, as well as Trauma-NOnTECHnical Skills (T-NOTECHS) score. Performance was compared between novice and expert clinician groups. RESULTS: Twenty-one subjects participated in three simulations: pilot (n = 7), novice (n = 8), and expert groups (n = 6). The novices committed more critical events than the experts, 11 versus 3, p = 0.006; took longer to treat patients, 560 (299) seconds versus 339 (321) seconds, p = 0.026; and achieved poorer T-NOTECHS scores, 14 (2) versus 21.5 (3.7), p = 0.003, and technical skill, 2.29 (0.34) versus 3.96 (0.69), p = 0.001. One hundred percent of the subjects thought that the simulation was realistic and superior to existing training options. CONCLUSION: A virtual-worldsYbased model for the training and assessment of major incident response has been designed and validated. The advantages of customizability, reproducibility, and recordability combined with the low cost of implementation suggest that this potentially represents a powerful adjunct to existing training methods and may be applicable to further areas of surgery as well. (J Trauma Acute Care Surg. 2014;77: 315Y321. Copyright * 2014 by Lippincott Williams & Wilkins) KEY WORDS: Simulation; online; trauma; distributed; virtual.
M
ass casualty incidents (MCIs) may be defined as events in which the demands placed upon a health system exceed that which can be met using its usual resources.1,2 This includes natural disasters (i.e., earthquakes) as well as man-made ones (terrorist events). Hospitals represent a critical link in MCI response, with close cooperation of emergency clinicians, surgeons, and allied health professionals needed to treat a rapid influx of casualties during such an event that is likely to overwhelm standard operating procedures.3 The unfortunately increasing incidence of MCIs,4 coupled with broadened public awareness in the wake of events such as the attacks of September 11, 2001, or the London bombings of July 7, 2005, highlights the need to ensure the preparedness of health systems and professionals to respond. The necessity of regular training to maintain a baseline level of hospital staff preparedness for MCI response is well recognized on both national and international levels. Semiannual
Submitted: February 19, 2014, Revised: March 11, 2014, Accepted: March 14, 2014. From the Departments of Surgery and Cancer (P.H.P., N.B., D.T., M.C., D.C., A.D.) and Emergency Medicine (N.B.), St Mary’s Hospital Major Trauma Centre, Imperial College London, London, United Kingdom. Address for reprints: Philip H. Pucher, MRCS, Department of Surgery and Cancer, 10th Floor, QEQM Building, St Mary’s Hospital, Praed Street, London W2 1NY, UK; email: [email protected]. DOI: 10.1097/TA.0000000000000308
exercises are mandated by law in both the United States5 and the United Kingdom2 as well as at a global level, where the World Health Organization has recommended training exercises at regular intervals to ensure that hospitals’ MCI response planning is up-to-date and can be correctly executed.6 The US Federal Emergency Management Agency describes four different exercise types, providing not only progressively increasing complexity and fidelity but also increasing cost and resource demand. These range from brief orientation seminars, to tabletop exercises, to the criterion standard of full-scale exercises involving deployment of personnel and equipment.7 However, a number of resident surveys have reported significant gaps in trainees’ knowledge and a perceived lack of training in MCI management.8Y10 It has been suggested that current training programs do not prepare trainees for the unique demands of an MCI.11 The current model of semiannual tabletop walk-throughs, supported by intermittent full-scale exercises, presumes experiential learning on the part of clinical staff but suffers from a lack of validated assessment to demonstrate proficiency or an improved level of performance. Post hoc analyses of both real and simulated MCI responses have consistently identified nontechnical errors, such as communication or team working failures and a lack of clinician training, as causes of suboptimal patient management.12Y17 Tabletop exercises, while cost-effective and reproducible, assess
J Trauma Acute Care Surg Volume 77, Number 2
Copyright © 2014 Lippincott Williams & Wilkins. Unauthorized reproduction of this article is prohibited.
315
J Trauma Acute Care Surg Volume 77, Number 2
Pucher et al.
knowledge of management plans but do not provide feedback on these crucial aspects of in situ performance.18 Conversely, training in a high-fidelity full-simulation exercise, providing a context in which such skills might be assessed, is prohibitively complex. Low-cost virtual-world technology offers a novel alternative, which can be run on any current-generation laptop or desktop computer without special hardware, and has the potential to provide low cost and universal accessibility.19,20 Virtual worlds are fully customizable computer-based three-dimensional environments in which multiple users, represented by individual avatars, may interact with virtual objects and each other in real time over the Internet or a private network. They have been previously used on a smaller scale to simulate the prehospital and hospital management of acute trauma21Y23 or surgical patients24 but never before to simulate an entire hospital or multipatient MCI. The aim of this study was to design and validate a novel, virtual-worldYbased system for MCI simulation and performance assessment.
MATERIALS AND METHODS Virtual Environment The Unity 4 games engine (Unity Technologies, San Francisco, CA) was used to create the simulated environment.25 The basic software development package is available to individuals free of charge. Avirtual hospital was developed, incorporating key clinical areas for mass casualty event management as follows: 1. 2. 3. 4. 5.
Twelve-bed emergency department (ED) Four-bed resuscitation room Twelve-bed acute receiving ward Eight-bed intensive care unit (ICU) Eight operating rooms (ORs) Design of the virtual hospital was based on real-world locations, a large urban tertiary unit, the designated Major Trauma Centre (equivalent to a US Level 1 trauma center) at St Mary’s Hospital, London, England (see Fig. 1). Clinical areas were spread across two floors, interconnected by accessible corridors and
stairwells. The software was designed for flexibility, with a modular design such that other hospital designs or layouts could be easily accommodated in the future.
Scenario Design A mass casualty event scenario, in which a terrorist bomb was detonated on a public transport bus, was designed. This was created in conjunction with experts in emergency medicine and trauma, with first-hand experience in both military trauma and civilian disaster management. It described a suspected terrorist detonation aboard a public transit bus resulting in multiple casualties from passengers and pedestrians, with a video brief designed to give a brief background and to allow clinicians to estimate the number and type of casualties.
Virtual Patients and User Interface Virtual patients (VPs) were scripted to reflect the expected range of injury profiles after such an attack. Scripts were matched to VP avatars appropriate to sex and gross injury, were pretriaged as either Priority 1 (critical injury) or 2 (major injury), and designated for treatment in the resuscitation room or the main ED for Priority 1 or 2, respectively. Throughout the scenario, patients continue to arrive at prescripted intervals, requiring clinician action to ensure appropriate placement, transfer, and treatment. Clinicians may access patient notes or an attached vital signs trolley to gain initial clinical information and any details of any obvious injuries, such as missing limbs, burns, or obvious fractures. A computercontrolled clinical team completes a primary assessment after a prescripted delay of 3 minutes to 10 minutes, at which point clinicians are able to access more detailed information as to VP current condition and any emergency treatment given, to decide on further management or transfer. Transfer to an appropriate area results in stabilization of the patient, whereas incorrect treatment (such as transfer of an unstable patient to the ward, rather than to intensive care or the OR) may result in further deterioration or death (Fig. 2). Clinical decision making is entirely open to the user, based upon the physiologic and injuryrelated data presented. To facilitate transfers, users must communicate with other in-world colleagues (via a virtual online telephone system) to
Figure 1. The virtual hospital: view of the ED with a casualty being assessed by a clinical team (left) and a bed on the ICU (right). 316
* 2014 Lippincott Williams & Wilkins
Copyright © 2014 Lippincott Williams & Wilkins. Unauthorized reproduction of this article is prohibited.
J Trauma Acute Care Surg Volume 77, Number 2
Pucher et al.
Use of the finalized simulation was subsequently tested in a feasibility and validity trial. Trainees and consultants (attendings) were invited to participate and were segregated by training grade. They received a lecture-based teaching session on mass casualty events and disaster response, followed by an orientation session and introduction to the simulation. The online simulation was played within a standard Internet browser (Microsoft Explorer 8, Microsoft Corp., Redmont, VA). Subjects were provided with headsets for in-world communication. Once all subjects had demonstrated their ability to use the interface, a prerecorded ‘‘newscast’’ video clip was played, setting the context and announcing a nearby disaster event. The simulation was started, and arrival of the first casualties commenced several minutes later. The simulation continued until all scripted patients had arrived, had been assessed, and had been appropriately treated.
Assessment Metrics
Figure 2. Example of patient flow within MCI simulation.
confirm bed space and appropriateness of transfer before the receiving user accepts the patient in question. In addition, the system includes an editable ‘‘whiteboard’’ function, which may be used as an ongoing patient record.
In-World Roles While users are free to walk through the entire hospital, they are expected to assume certain key strategic/operational roles, with at least one lead clinician coordinating each of the five simulated clinical areas (ED, resuscitation room, acute receiving ward, ICU, and OR), in keeping with standard MCI response policy. In addition, one user may assume the role of overall incident command role/site lead. External information such as expected number of casualties and time of arrival is related via the site lead to disseminate to other areas, as appropriate. Additional stressors can be placed upon users by the introduction of confederate (faculty) avatars as distractors, for example, in the guise of inquisitive reporters or next of kin, who must be dealt with appropriately. All users take part simultaneously in the simulation, logging in on separate computers located anywhere with Internet access.
Validation Trial The simulation was developed in an iterative process of software development between clinicians and software developers with experience in virtual-world surgical simulation.21,22,24
All in-world actions are recorded by a background software script and saved at the end of the simulation. Overall performance was assessed by analyzing patient transfer decisions and the time taken to complete patient assessment and transfer. Risk events were defined as patient transfers to an inappropriate clinical area. Individual subject technical skills performance was graded on a 1- to 5-point Likert scale across a range of critical behaviors and tasks, as defined by a regional disaster planning expert panel. Detail and reliability of this rating system have been previously described.21 Rated criteria include, for example, transfer of patients to ensure availability of beds in the designated acute receiving ward or obtaining up-to-date information on expected incoming casualties. Nontechnical skills performance was scored based on the validated T-NOTECHS score.26 All subjects were observed by three independent clinician observers with expertise in skills assessment and emergency medicine/disaster planning. Interrater reliability for all scores was calculated using Cronbach > in IBM SPSS Statistics 20 (IBM Corp., Armonk, NY), and the mean scores were used in
TABLE 1. Example of Priority 1 Scripted VPs Patient Description 38-y-old male, blast lung injury with hemopneumothorax, subsequently intubated 22-y-old female, traumatic amputation with significant hemorrhage 58-y-old male, traumatic amputation, positive FAST, hemodynamically unstable 61-y-old female, head injury with scalp laceration and reduced GCS score, not maintaining airway, subsequently intubated 37-y-old male, significant facial trauma with airway compromise, intubated at scene 51-y-old female, traumatic bilateral amputation, massive hemorrhage, cardiac arrest on arrival, deceased
Clinical Destination ICU OR OR ICU
ICU Morgue
GCS, Glasgow Coma Scale. FAST: focused abdominal ultrasonography for trauma.
* 2014 Lippincott Williams & Wilkins
Copyright © 2014 Lippincott Williams & Wilkins. Unauthorized reproduction of this article is prohibited.
317
J Trauma Acute Care Surg Volume 77, Number 2
Pucher et al.
final data. Final comparisons were performed between the junior trainee group and the expert consultant/attending group.
TABLE 2. User Demographics and Performance Score Pilot cohort n M:F ratio Years after qualification Previous training Local training Lecture based Emergo Military (Hospex) MIMMS course Not stated Risk events, n T-NOTECHS score, mean (SD) Technical score, mean (SD)
7 2:5 10 (5Y21) 3 (43%) 1 (14%) 1 (14%) 1 (14%) V V V
Junior cohort 8 4:4 1 (1Y2) 0 (0%) V V V V V V 11 14.0 (2.0)
Expert cohort 6 4:2 16.5 (10Y30) 6 (100%) V V 1 (14%) 1 (14%) 2 (28%) 2 (28%) 3 21.5 (3.7)
2.29 (0.34) 3.96 (0.69)
p
Critical Events
0.006 0.003
A total of 14 critical events were recorded, with a greater number of risk or critical events identified in the junior group, 11 versus 3, W2 test, p = 0.006. These were related to resource allocation, patient prioritization, and decision making. Failure to assess bed availability or transfer of patients from the ICU, for example, resulted in a shortage of beds, with ventilated patients blocking space in the ED. Incorrect communication of patient hospital numbers and clinical details resulted in transfer of several patients to the wrong clinical area. The mean (SD) time from patient arrival to disposition was significantly higher in the junior group, 560 (299) seconds versus 339 (321) seconds, Mann-Whitney U-test, p = 0.026 (Fig. 3).
0.001
Skills Assessment
0.467
Values are presented as median (range) or number (percentage) unless otherwise indicated. F, female; M, male; MIMMS, Major Incident Medical Management and Support course.
the final analysis. Scores were compared across groups using appropriate statistical tests, to determine the simulation’s capacity to differentiate between experts and novices (construct validity).
Face Validation Following scenario completion, the subjects filled in a feedback and validity questionnaire, with statement responses on a Likert scale of 1 (strongly disagree) to 7 (strongly agree). Responses of 5 or higher were deemed positive. Results were anonymized and summarized using Microsoft Excel (Microsoft Corp., Redmond, WA).
Interrater reliability for technical and nontechnical scores was excellent, Cronbach > = 0.807. T-NOTECHS scores for nontechnical ability were lower in the junior cohort, 14.0 (2.0) versus 21.5 (3.7), p = 0.003. Similarly, the mean technical scores assessing the completion of tasks relating to each subject’s role responsibility were also lower, 2.29 (0.34) versus 3.96 (0.69) (both Mann-Whitney U-test), p = 0.001 (see Table 2).
Validity Questionnaire The participants’ questionnaire responses were excellent, with almost all universally agreeing that the simulation was effective and realistic (Table 3). All agreed that it would be an effective and realistic training tool for mass casualty events and that it was an enjoyable addition to their training and might help improve their own practice. Specific free text responses commented on the quality of simulation of resource management required (n = 5), the
RESULTS A full mass casualty event simulation was designed (Fig. 2). Fifty-two VPs were created, 34 were preexisting inpatients and 18 MCI victims. Of these, six casualties had experienced severe injury (‘‘Priority 1’’ patients, see Table 1); the rest were categorized as ‘‘Priority 2,’’ with injuries including burns, long bone fractures, intra-abdominal injuries, or exacerbations of existing conditions such as ischemic heart disease or asthma, in reaction to the incident. Lower-priority patients were not included in this simulation.
Validation Trial A total of 21 clinicians were recruited, ranging from PGY 2Ylevel junior residents to senior attendings, with backgrounds in military trauma surgery, general surgery, and emergency medicine (see Table 2). The clinicians were segregated by training grade into three groupsVnovices/junior residents (n = 8), intermediates/residents (n = 7), and experts/consultants (attendings) (n = 6). During the first trial with the intermediate group, a technical failure led to performance metrics not being recorded, so only questionnaire responses from this group (subsequently treated as a test pilot group) were included in the 318
Figure 3. Boxplot comparison of time (y axis, half seconds) to patient disposition between junior trainee and expert groups, p = 0.026. * 2014 Lippincott Williams & Wilkins
Copyright © 2014 Lippincott Williams & Wilkins. Unauthorized reproduction of this article is prohibited.
J Trauma Acute Care Surg Volume 77, Number 2
Pucher et al.
DISCUSSION
TABLE 3. Results of User Feedback Form Question
n
Fidelity/usability I was able to navigate easily within the simulation The simulation’s user interface is easy to use The situation encountered could occur in real life I would act in a similar manner in real life Visual portrayal was realistic Effectiveness as a training tool MMCIS is more effective than existing MCI training tools MMCIS is more realistic than existing MCI training tools MMCIS addresses training needs not covered by usual curricula Dedicated MCI training is necessary for hospital staff Simulation-based MCI training/assessment should be used Personal experience The simulation is applicable to my own practice The simulation will help me improve my practice I enjoyed using the MMCIS simulation
2 1 2 1 2 1 2 0 2 1 1 8 1 8 2 0 2 1 2 0 2 1 2 1 2 1
Positive Response, % 90 95 100 86 95
86 86 95 100 95
95 95 95
MMCIS, multidisciplinary major incident simulator.
simulation of stressors present in a real incident (n = 5), the realistic nature of the scenario (n = 3), and the realism of the layout and graphics (n = 4). All trainees with previous disaster response training thought that the simulation had significant advantages over existing MCI training methods. In its present form, this study is limited to an internal validation study, with a virtual world modeled on a single center, with staff drawn from the same single trauma center. Lacking external data, there are limits to the generalizability of these findings. However, the simulation has been intentionally designed to be easily modifiable and customizable to local appearances, resources, and scenarios. The assessment metrics used are generic in nature, so as to apply to behaviors critical to any MCI response. The relatively small sample size was limited by available staff at the study center. All senior staff members of the ED took part in the study’s senior cohortVindicating staff members’ general enthusiasm to take part in, and acceptance of, such a testing method in general. Future research will seek to conduct further external validation testing.
This study describes the novel design, implementation, and validation of a virtual-worldsYbased simulation model for the planning, assessment, and training of clinicians’ and hospital systems’ responses to an MCI. This system addresses the need for additional training for all affected clinical staff, which has been well documented in retrospective analyses of past MCIs.11,27,28 Although other training opportunities exist, the most comprehensive and immersive of theseVa full-scale simulation exerciseVis highly complex, requiring a high level of human, material, and financial investiture.29 Accordingly, full-scale exercises are rare events, without opportunity to improve or refine responses through practice. Furthermore, MCI training has lacked a validated system of performance assessment until now, in part because of the difficulty of large-scale data capture during live exercises. Although some studies have inferred positive effects of incident exercising from pre-exercise and postexercise interviews30 or surveys,31 this lacks objective, validated, performance metrics. Reproducible and reliable measures of performance are needed to allow identification of deficient areas of practice and to track improvement. The identification of objective measures as performance indicators has been previously explored according to tasks such as the establishing of contact with incident officers, media representatives, and clinical staff.32 Such measures, however, do little to provide insight into the quality of care provided at a patient level. Gillett et al.,29 conversely, measured performance by observing completion of advanced trauma life supportYlike ‘‘critical actions’’ in the treatment of casualties. However, this risks excessive focus on assessment of individual casualties, neglecting the unique skill set demanded in MCI response, such as the need for effective communication and resource coordination in the face of overwhelming demand.12Y17,27 Our study demonstrates the ability of virtual worlds to record and assess all aspects of a simulated MCI response, from individual patient movements to overall nontechnical and team performance. Furthermore, the visual layout and in-world appearance can be customized to match actual hospitals to better prepare local staff for site-specific responses or scenarios. The low hardware requirementsVin this case performed on the subjects’ personal laptops as well as institutional (Hewlett Packard Corp, Palo Alto, CA, USA) 3000-series desktop computers without dedicated graphics capabilityVand Internet-based nature of the program allow easy, distributed access to health systems of all sizes and capabilities, at low cost. The total cost of development of this program was estimated at U45,000. With the underlying software engine, avatars, and hospital furnishings complete, however, a site-specific customization would cost as low as U2,000 to develop and a minimal annual maintenance fee of U500 to run. Thus, such an approach is much more cost-effective than most other MCI training programs currently available (Emergo tabletop exercise, for example, costs approximately U5,000 per use). The feasibility of virtual worlds for mass casualty event simulation and training has been previously demonstrated in smaller trials.21,31 The present study builds on this prior work
* 2014 Lippincott Williams & Wilkins
Copyright © 2014 Lippincott Williams & Wilkins. Unauthorized reproduction of this article is prohibited.
319
J Trauma Acute Care Surg Volume 77, Number 2
Pucher et al.
by, for the first time, incorporating whole-hospital simulation, multiple simultaneous users, and multiple patients, to allow whole clinician teams to participate and practice crucial skills required within the context of an MCI. This study is the first to present evidence of the construct validity of such a system. Our finding of significant differences between novice and expert clinicians (i.e., construct validation) highlights not only the feasibility of virtual worlds as a training tool for MCI response but also current training gap that exists for junior trainees. Reflecting issues previously reported in real-life MCIs, differences in patient prioritization, resource allocation, and use of communication structures in MCI response, were seen throughout the simulation. The recognized need to prioritize the availability of ‘‘downstream’’ beds with higher levels of care (i.e., ICU) in MCIs,33 for example, was more readily acted upon by the expert group, whereas junior trainees focused primarily on the ‘‘upstream’’ management of acutely presenting patients. Furthermore, once this need had been belatedly identified, overtriage and undertriage34 were seen, with inappropriate cascading transfers of patients to create necessary bed space. Such problems further demonstrate potential advantages of training in a virtual reality environment; as such, potentially life-endangering actions can, in this context, be recorded, replayed, and reflected upon, to improve disaster response and the quality of care provided to casualties in the event of an actual MCI. In summary, this study is the first to use a virtual-reality environment for validated assessment of MCI response. The use of virtual-world environments is particularly suited to clinical scenarios such as MCIs, which exhibit high levels of risk and complexity but, fortunately, are rare events. The customizable, accessible, low-cost nature of this technology has obvious benefits when compared with the current alternatives. Even with minimal resources, centers worldwide may access reproducible, customized disaster scenarios through this system, with objective and comparable outcomes, to provide clinician training and ensure institutional preparedness. MCIs are traumatic events for victims and clinicians alikeVbut while they are always unexpected, the use of such virtual-worldsYbased simulations may help ensure that we are not unprepared.
4.
5.
6.
7. 8.
9. 10.
11. 12.
13. 14.
15.
16. 17.
18.
19.
AUTHORSHIP All authors contributed to study design and conceptualization. P.H.P., N.B., and D.T. were responsible for data collection. P.H.P. and N.B. analyzed and interpreted data. P.H.P. drafted the manuscript. All authors contributed to its revision and approved the final draft.
20.
21. DISCLOSURE This study was partly funded by a grant from the London Deanery’s Simulation TechnologyYenhanced Learning Initiative (STeLI). 22.
REFERENCES 1. Civil Contingencies Secretariat. Civil Contingencies Act 2004. London, England: Cabinet Office; 2004. 2. Department of Health Emergency Preparedness Division. NHS Emergency Planning Guidance 2005. London, England: DoH; 2005. 3. California Department of Health. California department of public health standards and guidelines for healthcare surge during emergencies: foundational knowledge. Available at: http://www.bepreparedcalifornia.ca.gov/ CDPHPrograms/PublicHealthPrograms/EmergencyPreparednessOffice/
320
23.
24.
25.
EPOProgramsandServices/Surge/SurgeStandardsandGuidelines/Documents/ FoundationalKnowledge_FINAL.pdf. Accessed August 7, 2013. Blair JP, Blair JP. United States active shooter events from 2000 to 2010: training and equipment implications. 2013. ALERRT. Available at: http:// alerrt.org/files/research/ActiveShooterEvents.pdf. Accessed August 7, 2013. Joint Commission. 2009 Comprehensive Accreditation Manual for Hospitals: Emergency Management Standards. Oakbrook Terrace, IL: The Joint Commission; 2009. World Health Organization. Mass Casualty Management Systems: Strategies and Guideline for Building Health Sector Capacity. Geneva, Switzerland: WHO Press; 2007. Federal Emergency Management Agency. Exercises. Available at: http:// www.ready.gov/business/testing/exercises. Accessed August 7, 2013. Candiotti KA, Kamat A, Barach P, Nhuch F, Lubarsky D, Birnbach DJ. Emergency preparedness for biological and chemical incidents: a survey of anesthesiology residency programs in the United States. Anesth Analg. 2005;101(4):1135Y1140. Galante JM, Jacoby RC, Anderson JT. Are surgical residents prepared for mass casualty incidents? J Surg Res. 2006;132(1):85Y91. Wong K, Turner PS, Boppana A, Nugent Z, Coltman T, Cosker TD, et al. Preparation for the next major incident: are we ready? Emerg Med J. 2006;23(9):709Y712. Ryan J, Montgomery H. The London attacksVpreparedness: terrorism and the medical response. N Engl J Med. 2005;353(6):543Y545. Glow SD, Colucci VJ, Allington DR, Noonan CW, Hall EC. Managing multiple-casualty incidents: a rural medical preparedness training assessment. Prehosp Disaster Med. 2013;28(4):334Y341. Milsten A. Hospital responses to acute-onset disasters: a review. Prehosp Disaster Med. 2000;15(1):32Y45. Savoia E, Preston J, Biddinger PD. A consensus process on the use of exercises and after action reports to assess and improve public health emergency preparedness and response. Prehosp Disaster Med. 2013;28(3): 305Y308. Al-Shaqsi S, Gauld R, Lovell S, McBride D, Al-Kashmiri A, Al-Harthy A. Challenges of the New Zealand healthcare disaster preparedness prior to the Canterbury earthquakes: a qualitative analysis. N Z Med J. 2013; 126(1371):9Y18. Waage S, Poole JC, Thorgersen EB. Rural hospital mass casualty response to a terrorist shooting spree. Br J Surg. 2013;100(9):1198Y1204. Tokuda Y, Kikuchi M, Takahashi O, Stein GH. Prehospital management of sarin nerve gas terrorism in urban settings: 10 years of progress after the Tokyo subway sarin attack. Resuscitation. 2006;68(2):193Y202. Idrose AM, Adnan WA, Villa GF, Abdullah AH. The use of classroom training and simulation in the training of medical responders for airport disaster. Emerg Med J. 2007;24(1):7Y11. Hsu EB, Li Y, Bayram JD, Levinson D, Yang S, Monahan C. State of virtual reality based disaster preparedness and response training. PLoS Curr. 2013;5. Cohen D, Sevdalis N, Patel V, Taylor D, Batrick N, Darzi AW. Major incident preparation for acute hospitals: current state-of-the-art, training needs analysis, and the role of novel virtual worlds simulation technologies. J Emerg Med. 2012;43(6):1029Y1037. Cohen D, Sevdalis N, Patel V, Taylor M, Lee H, Vokes M, et al. Tactical and operational response to major incidents: feasibility and reliability of skills assessment using novel virtual environments. Resuscitation. 2013;84(7): 992Y998. Cohen D, Sevdalis N, Taylor D, Kerr K, Heys M, Willett K, et al. Emergency preparedness in the 21st century: training and preparation modules in virtual environments. Resuscitation. 2013;84(1):78Y84. LeRoy Heinrichs W, Youngblood P, Harter PM, Dev P. Simulation for team training and assessment: case studies of online training with virtual worlds. World J Surg. 2008;32(2):161Y170. Patel V, Aggarwal R, Cohen D, Taylor D, Darzi A. Implementation of an interactive virtual-world simulation for structured surgeon assessment of clinical scenarios. J Am Coll Surg. 2013;217(2):270Y279. Unity Technologies. Available at: http://www.unity3d.com. Accessed August 1, 2013.
* 2014 Lippincott Williams & Wilkins
Copyright © 2014 Lippincott Williams & Wilkins. Unauthorized reproduction of this article is prohibited.
J Trauma Acute Care Surg Volume 77, Number 2
26. Steinemann S, Berg B, Ditullio A, Skinner A, Terada K, Anzelon K, et al. Assessing teamwork in the trauma bay: introduction of a modified ‘‘NOTECHS’’ scale for trauma. Am J Surg. 2012;203(1):69Y75. 27. Schultz CH, Koenig KL, Whiteside M, Murray R. Development of national standardized all-hazard disaster core competencies for acute care physicians, nurses, and EMS professionals. Ann Emerg Med. 2012;59(3):196Y208 e1. 28. Brevard SB, Weintraub SL, Aiken JB, Halton EB, Duchesne JC, McSwain NE Jr, et al. Analysis of disaster response plans and the aftermath of Hurricane Katrina: lessons learned from a Level I trauma center. J Trauma. 2008;65(5):1126Y1132. 29. Gillett B, Peckler B, Sinert R, Onkst C, Nabors S, Issley S, et al. Simulation in a disaster drill: comparison of high-fidelity simulators versus trained actors. Acad Emerg Med. 2008;15(11):1144Y1151.
Pucher et al.
30. Einav S, Schecter WP, Matot I, Horn JK, Hersch M, Reissman P, et al. Case managers in mass casualty incidents. Ann Surg. 2009;249(3): 496Y501. 31. Heinrichs WL, Youngblood P, Harter P, Kusumoto L, Dev P. Training healthcare personnel for mass-casualty incidents in a virtual emergency department: VED II. Prehosp Disaster Med. 2010;25(5):424Y432. 32. Nilsson H, Vikstrom T, Jonson C-O. Performance indicators for initial regional medical response to major incidents: a possible quality control tool. Scand J Trauma Resusc Emerg Med. 2012;20(1):1. 33. Propper BW, Rasmussen TE, Davidson SB, Vandenberg SL, Clouse WD, Burkhardt GE, et al. Surgical response to multiple casualty incidents following single explosive events. Ann Surg. 2009;250(2):311Y315. 34. Frykberg ER. Principles of mass casualty management following terrorist disasters. Ann Surg. 2004;239(3):319Y321.
* 2014 Lippincott Williams & Wilkins
Copyright © 2014 Lippincott Williams & Wilkins. Unauthorized reproduction of this article is prohibited.
321
Virtual-world hospital simulation for real-world disaster response: Design and validation of a virtual reality simulator for mass casualty incident management Philip H. Pucher, MRCS, Nicola Batrick, FRCS, Dave Taylor, Muzzafer Chaudery, MRCS, Daniel Cohen, MRCS, and Ara Darzi, FRCS, London, United Kingdom
BACKGROUND: Mass casualty incidents are unfortunately becoming more common. The coordination of mass casualty incident response is highly complex. Currently available options for training, however, are limited by either lack of realism or prohibitive expense and by a lack of assessment tools. Virtual worlds represent a potentially cost-effective, immersive, and easily accessible platform for training and assessment. The aim of this study was to assess feasibility of a novel virtual-worldsYbased system for assessment and training in major incident response. METHODS: Clinical areas were modeled within a virtual, online hospital. A major incident, incorporating virtual casualties, allowed multiple clinicians to simultaneously respond with appropriate in-world management and transfer plans within limits of the hospital’s available resources. Errors, delays, and completed actions were recorded, as well as Trauma-NOnTECHnical Skills (T-NOTECHS) score. Performance was compared between novice and expert clinician groups. RESULTS: Twenty-one subjects participated in three simulations: pilot (n = 7), novice (n = 8), and expert groups (n = 6). The novices committed more critical events than the experts, 11 versus 3, p = 0.006; took longer to treat patients, 560 (299) seconds versus 339 (321) seconds, p = 0.026; and achieved poorer T-NOTECHS scores, 14 (2) versus 21.5 (3.7), p = 0.003, and technical skill, 2.29 (0.34) versus 3.96 (0.69), p = 0.001. One hundred percent of the subjects thought that the simulation was realistic and superior to existing training options. CONCLUSION: A virtual-worldsYbased model for the training and assessment of major incident response has been designed and validated. The advantages of customizability, reproducibility, and recordability combined with the low cost of implementation suggest that this potentially represents a powerful adjunct to existing training methods and may be applicable to further areas of surgery as well. (J Trauma Acute Care Surg. 2014;77: 315Y321. Copyright * 2014 by Lippincott Williams & Wilkins) KEY WORDS: Simulation; online; trauma; distributed; virtual.
M
ass casualty incidents (MCIs) may be defined as events in which the demands placed upon a health system exceed that which can be met using its usual resources.1,2 This includes natural disasters (i.e., earthquakes) as well as man-made ones (terrorist events). Hospitals represent a critical link in MCI response, with close cooperation of emergency clinicians, surgeons, and allied health professionals needed to treat a rapid influx of casualties during such an event that is likely to overwhelm standard operating procedures.3 The unfortunately increasing incidence of MCIs,4 coupled with broadened public awareness in the wake of events such as the attacks of September 11, 2001, or the London bombings of July 7, 2005, highlights the need to ensure the preparedness of health systems and professionals to respond. The necessity of regular training to maintain a baseline level of hospital staff preparedness for MCI response is well recognized on both national and international levels. Semiannual
Submitted: February 19, 2014, Revised: March 11, 2014, Accepted: March 14, 2014. From the Departments of Surgery and Cancer (P.H.P., N.B., D.T., M.C., D.C., A.D.) and Emergency Medicine (N.B.), St Mary’s Hospital Major Trauma Centre, Imperial College London, London, United Kingdom. Address for reprints: Philip H. Pucher, MRCS, Department of Surgery and Cancer, 10th Floor, QEQM Building, St Mary’s Hospital, Praed Street, London W2 1NY, UK; email: [email protected]. DOI: 10.1097/TA.0000000000000308
exercises are mandated by law in both the United States5 and the United Kingdom2 as well as at a global level, where the World Health Organization has recommended training exercises at regular intervals to ensure that hospitals’ MCI response planning is up-to-date and can be correctly executed.6 The US Federal Emergency Management Agency describes four different exercise types, providing not only progressively increasing complexity and fidelity but also increasing cost and resource demand. These range from brief orientation seminars, to tabletop exercises, to the criterion standard of full-scale exercises involving deployment of personnel and equipment.7 However, a number of resident surveys have reported significant gaps in trainees’ knowledge and a perceived lack of training in MCI management.8Y10 It has been suggested that current training programs do not prepare trainees for the unique demands of an MCI.11 The current model of semiannual tabletop walk-throughs, supported by intermittent full-scale exercises, presumes experiential learning on the part of clinical staff but suffers from a lack of validated assessment to demonstrate proficiency or an improved level of performance. Post hoc analyses of both real and simulated MCI responses have consistently identified nontechnical errors, such as communication or team working failures and a lack of clinician training, as causes of suboptimal patient management.12Y17 Tabletop exercises, while cost-effective and reproducible, assess
J Trauma Acute Care Surg Volume 77, Number 2
Copyright © 2014 Lippincott Williams & Wilkins. Unauthorized reproduction of this article is prohibited.
315
J Trauma Acute Care Surg Volume 77, Number 2
Pucher et al.
knowledge of management plans but do not provide feedback on these crucial aspects of in situ performance.18 Conversely, training in a high-fidelity full-simulation exercise, providing a context in which such skills might be assessed, is prohibitively complex. Low-cost virtual-world technology offers a novel alternative, which can be run on any current-generation laptop or desktop computer without special hardware, and has the potential to provide low cost and universal accessibility.19,20 Virtual worlds are fully customizable computer-based three-dimensional environments in which multiple users, represented by individual avatars, may interact with virtual objects and each other in real time over the Internet or a private network. They have been previously used on a smaller scale to simulate the prehospital and hospital management of acute trauma21Y23 or surgical patients24 but never before to simulate an entire hospital or multipatient MCI. The aim of this study was to design and validate a novel, virtual-worldYbased system for MCI simulation and performance assessment.
MATERIALS AND METHODS Virtual Environment The Unity 4 games engine (Unity Technologies, San Francisco, CA) was used to create the simulated environment.25 The basic software development package is available to individuals free of charge. Avirtual hospital was developed, incorporating key clinical areas for mass casualty event management as follows: 1. 2. 3. 4. 5.
Twelve-bed emergency department (ED) Four-bed resuscitation room Twelve-bed acute receiving ward Eight-bed intensive care unit (ICU) Eight operating rooms (ORs) Design of the virtual hospital was based on real-world locations, a large urban tertiary unit, the designated Major Trauma Centre (equivalent to a US Level 1 trauma center) at St Mary’s Hospital, London, England (see Fig. 1). Clinical areas were spread across two floors, interconnected by accessible corridors and
stairwells. The software was designed for flexibility, with a modular design such that other hospital designs or layouts could be easily accommodated in the future.
Scenario Design A mass casualty event scenario, in which a terrorist bomb was detonated on a public transport bus, was designed. This was created in conjunction with experts in emergency medicine and trauma, with first-hand experience in both military trauma and civilian disaster management. It described a suspected terrorist detonation aboard a public transit bus resulting in multiple casualties from passengers and pedestrians, with a video brief designed to give a brief background and to allow clinicians to estimate the number and type of casualties.
Virtual Patients and User Interface Virtual patients (VPs) were scripted to reflect the expected range of injury profiles after such an attack. Scripts were matched to VP avatars appropriate to sex and gross injury, were pretriaged as either Priority 1 (critical injury) or 2 (major injury), and designated for treatment in the resuscitation room or the main ED for Priority 1 or 2, respectively. Throughout the scenario, patients continue to arrive at prescripted intervals, requiring clinician action to ensure appropriate placement, transfer, and treatment. Clinicians may access patient notes or an attached vital signs trolley to gain initial clinical information and any details of any obvious injuries, such as missing limbs, burns, or obvious fractures. A computercontrolled clinical team completes a primary assessment after a prescripted delay of 3 minutes to 10 minutes, at which point clinicians are able to access more detailed information as to VP current condition and any emergency treatment given, to decide on further management or transfer. Transfer to an appropriate area results in stabilization of the patient, whereas incorrect treatment (such as transfer of an unstable patient to the ward, rather than to intensive care or the OR) may result in further deterioration or death (Fig. 2). Clinical decision making is entirely open to the user, based upon the physiologic and injuryrelated data presented. To facilitate transfers, users must communicate with other in-world colleagues (via a virtual online telephone system) to
Figure 1. The virtual hospital: view of the ED with a casualty being assessed by a clinical team (left) and a bed on the ICU (right). 316
* 2014 Lippincott Williams & Wilkins
Copyright © 2014 Lippincott Williams & Wilkins. Unauthorized reproduction of this article is prohibited.
J Trauma Acute Care Surg Volume 77, Number 2
Pucher et al.
Use of the finalized simulation was subsequently tested in a feasibility and validity trial. Trainees and consultants (attendings) were invited to participate and were segregated by training grade. They received a lecture-based teaching session on mass casualty events and disaster response, followed by an orientation session and introduction to the simulation. The online simulation was played within a standard Internet browser (Microsoft Explorer 8, Microsoft Corp., Redmont, VA). Subjects were provided with headsets for in-world communication. Once all subjects had demonstrated their ability to use the interface, a prerecorded ‘‘newscast’’ video clip was played, setting the context and announcing a nearby disaster event. The simulation was started, and arrival of the first casualties commenced several minutes later. The simulation continued until all scripted patients had arrived, had been assessed, and had been appropriately treated.
Assessment Metrics
Figure 2. Example of patient flow within MCI simulation.
confirm bed space and appropriateness of transfer before the receiving user accepts the patient in question. In addition, the system includes an editable ‘‘whiteboard’’ function, which may be used as an ongoing patient record.
In-World Roles While users are free to walk through the entire hospital, they are expected to assume certain key strategic/operational roles, with at least one lead clinician coordinating each of the five simulated clinical areas (ED, resuscitation room, acute receiving ward, ICU, and OR), in keeping with standard MCI response policy. In addition, one user may assume the role of overall incident command role/site lead. External information such as expected number of casualties and time of arrival is related via the site lead to disseminate to other areas, as appropriate. Additional stressors can be placed upon users by the introduction of confederate (faculty) avatars as distractors, for example, in the guise of inquisitive reporters or next of kin, who must be dealt with appropriately. All users take part simultaneously in the simulation, logging in on separate computers located anywhere with Internet access.
Validation Trial The simulation was developed in an iterative process of software development between clinicians and software developers with experience in virtual-world surgical simulation.21,22,24
All in-world actions are recorded by a background software script and saved at the end of the simulation. Overall performance was assessed by analyzing patient transfer decisions and the time taken to complete patient assessment and transfer. Risk events were defined as patient transfers to an inappropriate clinical area. Individual subject technical skills performance was graded on a 1- to 5-point Likert scale across a range of critical behaviors and tasks, as defined by a regional disaster planning expert panel. Detail and reliability of this rating system have been previously described.21 Rated criteria include, for example, transfer of patients to ensure availability of beds in the designated acute receiving ward or obtaining up-to-date information on expected incoming casualties. Nontechnical skills performance was scored based on the validated T-NOTECHS score.26 All subjects were observed by three independent clinician observers with expertise in skills assessment and emergency medicine/disaster planning. Interrater reliability for all scores was calculated using Cronbach > in IBM SPSS Statistics 20 (IBM Corp., Armonk, NY), and the mean scores were used in
TABLE 1. Example of Priority 1 Scripted VPs Patient Description 38-y-old male, blast lung injury with hemopneumothorax, subsequently intubated 22-y-old female, traumatic amputation with significant hemorrhage 58-y-old male, traumatic amputation, positive FAST, hemodynamically unstable 61-y-old female, head injury with scalp laceration and reduced GCS score, not maintaining airway, subsequently intubated 37-y-old male, significant facial trauma with airway compromise, intubated at scene 51-y-old female, traumatic bilateral amputation, massive hemorrhage, cardiac arrest on arrival, deceased
Clinical Destination ICU OR OR ICU
ICU Morgue
GCS, Glasgow Coma Scale. FAST: focused abdominal ultrasonography for trauma.
* 2014 Lippincott Williams & Wilkins
Copyright © 2014 Lippincott Williams & Wilkins. Unauthorized reproduction of this article is prohibited.
317
J Trauma Acute Care Surg Volume 77, Number 2
Pucher et al.
final data. Final comparisons were performed between the junior trainee group and the expert consultant/attending group.
TABLE 2. User Demographics and Performance Score Pilot cohort n M:F ratio Years after qualification Previous training Local training Lecture based Emergo Military (Hospex) MIMMS course Not stated Risk events, n T-NOTECHS score, mean (SD) Technical score, mean (SD)
7 2:5 10 (5Y21) 3 (43%) 1 (14%) 1 (14%) 1 (14%) V V V
Junior cohort 8 4:4 1 (1Y2) 0 (0%) V V V V V V 11 14.0 (2.0)
Expert cohort 6 4:2 16.5 (10Y30) 6 (100%) V V 1 (14%) 1 (14%) 2 (28%) 2 (28%) 3 21.5 (3.7)
2.29 (0.34) 3.96 (0.69)
p
Critical Events
0.006 0.003
A total of 14 critical events were recorded, with a greater number of risk or critical events identified in the junior group, 11 versus 3, W2 test, p = 0.006. These were related to resource allocation, patient prioritization, and decision making. Failure to assess bed availability or transfer of patients from the ICU, for example, resulted in a shortage of beds, with ventilated patients blocking space in the ED. Incorrect communication of patient hospital numbers and clinical details resulted in transfer of several patients to the wrong clinical area. The mean (SD) time from patient arrival to disposition was significantly higher in the junior group, 560 (299) seconds versus 339 (321) seconds, Mann-Whitney U-test, p = 0.026 (Fig. 3).
0.001
Skills Assessment
0.467
Values are presented as median (range) or number (percentage) unless otherwise indicated. F, female; M, male; MIMMS, Major Incident Medical Management and Support course.
the final analysis. Scores were compared across groups using appropriate statistical tests, to determine the simulation’s capacity to differentiate between experts and novices (construct validity).
Face Validation Following scenario completion, the subjects filled in a feedback and validity questionnaire, with statement responses on a Likert scale of 1 (strongly disagree) to 7 (strongly agree). Responses of 5 or higher were deemed positive. Results were anonymized and summarized using Microsoft Excel (Microsoft Corp., Redmond, WA).
Interrater reliability for technical and nontechnical scores was excellent, Cronbach > = 0.807. T-NOTECHS scores for nontechnical ability were lower in the junior cohort, 14.0 (2.0) versus 21.5 (3.7), p = 0.003. Similarly, the mean technical scores assessing the completion of tasks relating to each subject’s role responsibility were also lower, 2.29 (0.34) versus 3.96 (0.69) (both Mann-Whitney U-test), p = 0.001 (see Table 2).
Validity Questionnaire The participants’ questionnaire responses were excellent, with almost all universally agreeing that the simulation was effective and realistic (Table 3). All agreed that it would be an effective and realistic training tool for mass casualty events and that it was an enjoyable addition to their training and might help improve their own practice. Specific free text responses commented on the quality of simulation of resource management required (n = 5), the
RESULTS A full mass casualty event simulation was designed (Fig. 2). Fifty-two VPs were created, 34 were preexisting inpatients and 18 MCI victims. Of these, six casualties had experienced severe injury (‘‘Priority 1’’ patients, see Table 1); the rest were categorized as ‘‘Priority 2,’’ with injuries including burns, long bone fractures, intra-abdominal injuries, or exacerbations of existing conditions such as ischemic heart disease or asthma, in reaction to the incident. Lower-priority patients were not included in this simulation.
Validation Trial A total of 21 clinicians were recruited, ranging from PGY 2Ylevel junior residents to senior attendings, with backgrounds in military trauma surgery, general surgery, and emergency medicine (see Table 2). The clinicians were segregated by training grade into three groupsVnovices/junior residents (n = 8), intermediates/residents (n = 7), and experts/consultants (attendings) (n = 6). During the first trial with the intermediate group, a technical failure led to performance metrics not being recorded, so only questionnaire responses from this group (subsequently treated as a test pilot group) were included in the 318
Figure 3. Boxplot comparison of time (y axis, half seconds) to patient disposition between junior trainee and expert groups, p = 0.026. * 2014 Lippincott Williams & Wilkins
Copyright © 2014 Lippincott Williams & Wilkins. Unauthorized reproduction of this article is prohibited.
J Trauma Acute Care Surg Volume 77, Number 2
Pucher et al.
DISCUSSION
TABLE 3. Results of User Feedback Form Question
n
Fidelity/usability I was able to navigate easily within the simulation The simulation’s user interface is easy to use The situation encountered could occur in real life I would act in a similar manner in real life Visual portrayal was realistic Effectiveness as a training tool MMCIS is more effective than existing MCI training tools MMCIS is more realistic than existing MCI training tools MMCIS addresses training needs not covered by usual curricula Dedicated MCI training is necessary for hospital staff Simulation-based MCI training/assessment should be used Personal experience The simulation is applicable to my own practice The simulation will help me improve my practice I enjoyed using the MMCIS simulation
2 1 2 1 2 1 2 0 2 1 1 8 1 8 2 0 2 1 2 0 2 1 2 1 2 1
Positive Response, % 90 95 100 86 95
86 86 95 100 95
95 95 95
MMCIS, multidisciplinary major incident simulator.
simulation of stressors present in a real incident (n = 5), the realistic nature of the scenario (n = 3), and the realism of the layout and graphics (n = 4). All trainees with previous disaster response training thought that the simulation had significant advantages over existing MCI training methods. In its present form, this study is limited to an internal validation study, with a virtual world modeled on a single center, with staff drawn from the same single trauma center. Lacking external data, there are limits to the generalizability of these findings. However, the simulation has been intentionally designed to be easily modifiable and customizable to local appearances, resources, and scenarios. The assessment metrics used are generic in nature, so as to apply to behaviors critical to any MCI response. The relatively small sample size was limited by available staff at the study center. All senior staff members of the ED took part in the study’s senior cohortVindicating staff members’ general enthusiasm to take part in, and acceptance of, such a testing method in general. Future research will seek to conduct further external validation testing.
This study describes the novel design, implementation, and validation of a virtual-worldsYbased simulation model for the planning, assessment, and training of clinicians’ and hospital systems’ responses to an MCI. This system addresses the need for additional training for all affected clinical staff, which has been well documented in retrospective analyses of past MCIs.11,27,28 Although other training opportunities exist, the most comprehensive and immersive of theseVa full-scale simulation exerciseVis highly complex, requiring a high level of human, material, and financial investiture.29 Accordingly, full-scale exercises are rare events, without opportunity to improve or refine responses through practice. Furthermore, MCI training has lacked a validated system of performance assessment until now, in part because of the difficulty of large-scale data capture during live exercises. Although some studies have inferred positive effects of incident exercising from pre-exercise and postexercise interviews30 or surveys,31 this lacks objective, validated, performance metrics. Reproducible and reliable measures of performance are needed to allow identification of deficient areas of practice and to track improvement. The identification of objective measures as performance indicators has been previously explored according to tasks such as the establishing of contact with incident officers, media representatives, and clinical staff.32 Such measures, however, do little to provide insight into the quality of care provided at a patient level. Gillett et al.,29 conversely, measured performance by observing completion of advanced trauma life supportYlike ‘‘critical actions’’ in the treatment of casualties. However, this risks excessive focus on assessment of individual casualties, neglecting the unique skill set demanded in MCI response, such as the need for effective communication and resource coordination in the face of overwhelming demand.12Y17,27 Our study demonstrates the ability of virtual worlds to record and assess all aspects of a simulated MCI response, from individual patient movements to overall nontechnical and team performance. Furthermore, the visual layout and in-world appearance can be customized to match actual hospitals to better prepare local staff for site-specific responses or scenarios. The low hardware requirementsVin this case performed on the subjects’ personal laptops as well as institutional (Hewlett Packard Corp, Palo Alto, CA, USA) 3000-series desktop computers without dedicated graphics capabilityVand Internet-based nature of the program allow easy, distributed access to health systems of all sizes and capabilities, at low cost. The total cost of development of this program was estimated at U45,000. With the underlying software engine, avatars, and hospital furnishings complete, however, a site-specific customization would cost as low as U2,000 to develop and a minimal annual maintenance fee of U500 to run. Thus, such an approach is much more cost-effective than most other MCI training programs currently available (Emergo tabletop exercise, for example, costs approximately U5,000 per use). The feasibility of virtual worlds for mass casualty event simulation and training has been previously demonstrated in smaller trials.21,31 The present study builds on this prior work
* 2014 Lippincott Williams & Wilkins
Copyright © 2014 Lippincott Williams & Wilkins. Unauthorized reproduction of this article is prohibited.
319
J Trauma Acute Care Surg Volume 77, Number 2
Pucher et al.
by, for the first time, incorporating whole-hospital simulation, multiple simultaneous users, and multiple patients, to allow whole clinician teams to participate and practice crucial skills required within the context of an MCI. This study is the first to present evidence of the construct validity of such a system. Our finding of significant differences between novice and expert clinicians (i.e., construct validation) highlights not only the feasibility of virtual worlds as a training tool for MCI response but also current training gap that exists for junior trainees. Reflecting issues previously reported in real-life MCIs, differences in patient prioritization, resource allocation, and use of communication structures in MCI response, were seen throughout the simulation. The recognized need to prioritize the availability of ‘‘downstream’’ beds with higher levels of care (i.e., ICU) in MCIs,33 for example, was more readily acted upon by the expert group, whereas junior trainees focused primarily on the ‘‘upstream’’ management of acutely presenting patients. Furthermore, once this need had been belatedly identified, overtriage and undertriage34 were seen, with inappropriate cascading transfers of patients to create necessary bed space. Such problems further demonstrate potential advantages of training in a virtual reality environment; as such, potentially life-endangering actions can, in this context, be recorded, replayed, and reflected upon, to improve disaster response and the quality of care provided to casualties in the event of an actual MCI. In summary, this study is the first to use a virtual-reality environment for validated assessment of MCI response. The use of virtual-world environments is particularly suited to clinical scenarios such as MCIs, which exhibit high levels of risk and complexity but, fortunately, are rare events. The customizable, accessible, low-cost nature of this technology has obvious benefits when compared with the current alternatives. Even with minimal resources, centers worldwide may access reproducible, customized disaster scenarios through this system, with objective and comparable outcomes, to provide clinician training and ensure institutional preparedness. MCIs are traumatic events for victims and clinicians alikeVbut while they are always unexpected, the use of such virtual-worldsYbased simulations may help ensure that we are not unprepared.
4.
5.
6.
7. 8.
9. 10.
11. 12.
13. 14.
15.
16. 17.
18.
19.
AUTHORSHIP All authors contributed to study design and conceptualization. P.H.P., N.B., and D.T. were responsible for data collection. P.H.P. and N.B. analyzed and interpreted data. P.H.P. drafted the manuscript. All authors contributed to its revision and approved the final draft.
20.
21. DISCLOSURE This study was partly funded by a grant from the London Deanery’s Simulation TechnologyYenhanced Learning Initiative (STeLI). 22.
REFERENCES 1. Civil Contingencies Secretariat. Civil Contingencies Act 2004. London, England: Cabinet Office; 2004. 2. Department of Health Emergency Preparedness Division. NHS Emergency Planning Guidance 2005. London, England: DoH; 2005. 3. California Department of Health. California department of public health standards and guidelines for healthcare surge during emergencies: foundational knowledge. Available at: http://www.bepreparedcalifornia.ca.gov/ CDPHPrograms/PublicHealthPrograms/EmergencyPreparednessOffice/
320
23.
24.
25.
EPOProgramsandServices/Surge/SurgeStandardsandGuidelines/Documents/ FoundationalKnowledge_FINAL.pdf. Accessed August 7, 2013. Blair JP, Blair JP. United States active shooter events from 2000 to 2010: training and equipment implications. 2013. ALERRT. Available at: http:// alerrt.org/files/research/ActiveShooterEvents.pdf. Accessed August 7, 2013. Joint Commission. 2009 Comprehensive Accreditation Manual for Hospitals: Emergency Management Standards. Oakbrook Terrace, IL: The Joint Commission; 2009. World Health Organization. Mass Casualty Management Systems: Strategies and Guideline for Building Health Sector Capacity. Geneva, Switzerland: WHO Press; 2007. Federal Emergency Management Agency. Exercises. Available at: http:// www.ready.gov/business/testing/exercises. Accessed August 7, 2013. Candiotti KA, Kamat A, Barach P, Nhuch F, Lubarsky D, Birnbach DJ. Emergency preparedness for biological and chemical incidents: a survey of anesthesiology residency programs in the United States. Anesth Analg. 2005;101(4):1135Y1140. Galante JM, Jacoby RC, Anderson JT. Are surgical residents prepared for mass casualty incidents? J Surg Res. 2006;132(1):85Y91. Wong K, Turner PS, Boppana A, Nugent Z, Coltman T, Cosker TD, et al. Preparation for the next major incident: are we ready? Emerg Med J. 2006;23(9):709Y712. Ryan J, Montgomery H. The London attacksVpreparedness: terrorism and the medical response. N Engl J Med. 2005;353(6):543Y545. Glow SD, Colucci VJ, Allington DR, Noonan CW, Hall EC. Managing multiple-casualty incidents: a rural medical preparedness training assessment. Prehosp Disaster Med. 2013;28(4):334Y341. Milsten A. Hospital responses to acute-onset disasters: a review. Prehosp Disaster Med. 2000;15(1):32Y45. Savoia E, Preston J, Biddinger PD. A consensus process on the use of exercises and after action reports to assess and improve public health emergency preparedness and response. Prehosp Disaster Med. 2013;28(3): 305Y308. Al-Shaqsi S, Gauld R, Lovell S, McBride D, Al-Kashmiri A, Al-Harthy A. Challenges of the New Zealand healthcare disaster preparedness prior to the Canterbury earthquakes: a qualitative analysis. N Z Med J. 2013; 126(1371):9Y18. Waage S, Poole JC, Thorgersen EB. Rural hospital mass casualty response to a terrorist shooting spree. Br J Surg. 2013;100(9):1198Y1204. Tokuda Y, Kikuchi M, Takahashi O, Stein GH. Prehospital management of sarin nerve gas terrorism in urban settings: 10 years of progress after the Tokyo subway sarin attack. Resuscitation. 2006;68(2):193Y202. Idrose AM, Adnan WA, Villa GF, Abdullah AH. The use of classroom training and simulation in the training of medical responders for airport disaster. Emerg Med J. 2007;24(1):7Y11. Hsu EB, Li Y, Bayram JD, Levinson D, Yang S, Monahan C. State of virtual reality based disaster preparedness and response training. PLoS Curr. 2013;5. Cohen D, Sevdalis N, Patel V, Taylor D, Batrick N, Darzi AW. Major incident preparation for acute hospitals: current state-of-the-art, training needs analysis, and the role of novel virtual worlds simulation technologies. J Emerg Med. 2012;43(6):1029Y1037. Cohen D, Sevdalis N, Patel V, Taylor M, Lee H, Vokes M, et al. Tactical and operational response to major incidents: feasibility and reliability of skills assessment using novel virtual environments. Resuscitation. 2013;84(7): 992Y998. Cohen D, Sevdalis N, Taylor D, Kerr K, Heys M, Willett K, et al. Emergency preparedness in the 21st century: training and preparation modules in virtual environments. Resuscitation. 2013;84(1):78Y84. LeRoy Heinrichs W, Youngblood P, Harter PM, Dev P. Simulation for team training and assessment: case studies of online training with virtual worlds. World J Surg. 2008;32(2):161Y170. Patel V, Aggarwal R, Cohen D, Taylor D, Darzi A. Implementation of an interactive virtual-world simulation for structured surgeon assessment of clinical scenarios. J Am Coll Surg. 2013;217(2):270Y279. Unity Technologies. Available at: http://www.unity3d.com. Accessed August 1, 2013.
* 2014 Lippincott Williams & Wilkins
Copyright © 2014 Lippincott Williams & Wilkins. Unauthorized reproduction of this article is prohibited.
J Trauma Acute Care Surg Volume 77, Number 2
26. Steinemann S, Berg B, Ditullio A, Skinner A, Terada K, Anzelon K, et al. Assessing teamwork in the trauma bay: introduction of a modified ‘‘NOTECHS’’ scale for trauma. Am J Surg. 2012;203(1):69Y75. 27. Schultz CH, Koenig KL, Whiteside M, Murray R. Development of national standardized all-hazard disaster core competencies for acute care physicians, nurses, and EMS professionals. Ann Emerg Med. 2012;59(3):196Y208 e1. 28. Brevard SB, Weintraub SL, Aiken JB, Halton EB, Duchesne JC, McSwain NE Jr, et al. Analysis of disaster response plans and the aftermath of Hurricane Katrina: lessons learned from a Level I trauma center. J Trauma. 2008;65(5):1126Y1132. 29. Gillett B, Peckler B, Sinert R, Onkst C, Nabors S, Issley S, et al. Simulation in a disaster drill: comparison of high-fidelity simulators versus trained actors. Acad Emerg Med. 2008;15(11):1144Y1151.
Pucher et al.
30. Einav S, Schecter WP, Matot I, Horn JK, Hersch M, Reissman P, et al. Case managers in mass casualty incidents. Ann Surg. 2009;249(3): 496Y501. 31. Heinrichs WL, Youngblood P, Harter P, Kusumoto L, Dev P. Training healthcare personnel for mass-casualty incidents in a virtual emergency department: VED II. Prehosp Disaster Med. 2010;25(5):424Y432. 32. Nilsson H, Vikstrom T, Jonson C-O. Performance indicators for initial regional medical response to major incidents: a possible quality control tool. Scand J Trauma Resusc Emerg Med. 2012;20(1):1. 33. Propper BW, Rasmussen TE, Davidson SB, Vandenberg SL, Clouse WD, Burkhardt GE, et al. Surgical response to multiple casualty incidents following single explosive events. Ann Surg. 2009;250(2):311Y315. 34. Frykberg ER. Principles of mass casualty management following terrorist disasters. Ann Surg. 2004;239(3):319Y321.
* 2014 Lippincott Williams & Wilkins
Copyright © 2014 Lippincott Williams & Wilkins. Unauthorized reproduction of this article is prohibited.
321
