ORIGINAL CONTRIBUTION

Virtual Reality Simulator Training for Laparoscopic Colectomy: What Metrics Have Construct Validity? Skandan Shanmugan, M.D.1 • Fabien Leblanc, M.D.2 • Anthony J. Senagore, M.D.3 C. Neal Ellis, M.D.4 • Sharon L. Stein, M.D.1 • Sadaf Khan, M.D.1 Conor P. Delaney, M.D., Ph.D.1 • Bradley J. Champagne, M.D.1 1 Division of Colorectal Surgery, University Hospitals Case Medical Center, Cleveland, Ohio 2 Department of Digestive Surgery, University Hospitals of Bordeaux, Bordeaux, France 3 Division of Colorectal Surgery, Keck School of Medicine, University of Southern California, Los Angeles, California 4 Division of Colorectal Surgery, Veterans Administration Gulf Coast Veterans Health Care System, Biloxi, Mississippi

BACKGROUND:  Virtual reality simulation for laparoscopic colectomy has been used for training of surgical residents and has been considered as a model for technical skills assessment of board-eligible colorectal surgeons. However, construct validity (the ability to distinguish between skill levels) must be confirmed before widespread implementation. OBJECTIVE:  This study was designed to specifically determine which metrics for laparoscopic sigmoid colectomy have evidence of construct validity. DESIGN:  General surgeons that had performed fewer than 30 laparoscopic colon resections and laparoscopic colorectal experts (>200 laparoscopic colon resections) performed laparoscopic sigmoid colectomy on the LAP Mentor model. All participants received a 15-minute instructional warm-up and had never used the simulator before the study. Performance was then compared between each group for 21 metrics (procedural, 14; intraoperative errors, 7) to determine specifically which measurements demonstrate construct validity. Performance was compared with the Mann-Whitney U-test (p < 0.05 was significant). RESULTS:  Fifty-three surgeons; 29 general surgeons, and 24 colorectal surgeons enrolled in the study. The Financial Disclosures: None reported. Presented at the Research Forum at the meeting of The American Society of Colon and Rectal Surgeons, Phoenix, AZ, April 27 to May 1, 2013. Correspondence: Skandan Shanmugan, M.D., Division of Colorectal Surgery, University of Pennsylvania, 800 Walnut St, 20th Floor, Philadelphia, PA 19107-5109. E-mail: [email protected] Dis Colon Rectum 2014; 57: 210–214 DOI: 10.1097/DCR.0000000000000031 © The ASCRS 2014

210

virtual reality simulators for laparoscopic sigmoid colectomy demonstrated construct validity for 8 of 14 procedural metrics by distinguishing levels of surgical experience (p < 0.05). The most discriminatory procedural metrics (p < 0.01) favoring experts were reduced instrument path length, accuracy of the peritoneal/medial mobilization, and dissection of the inferior mesenteric artery. Intraoperative errors were not discriminatory for most metrics and favored general surgeons for colonic wall injury (general surgeons, 0.7; colorectal surgeons, 3.5; p = 0.045). LIMITATIONS:  Individual variability within the general surgeon and colorectal surgeon groups was not accounted for. CONCLUSIONS:  The virtual reality simulators for laparoscopic sigmoid colectomy demonstrated construct validity for 8 procedure-specific metrics. However, using virtual reality simulator metrics to detect intraoperative errors did not discriminate between groups. If the virtual reality simulator continues to be used for the technical assessment of trainees and board-eligible surgeons, the evaluation of performance should be limited to procedural metrics.

KEY WORDS:  Virtual reality simulation; Construct validity; Education; Laparoscopic colectomy.

B

oard-eligible colorectal surgeons are increasingly exposed to laparoscopic procedures during colorectal residency. However, it is unclear how to effectively evaluate their ability to perform a safe and appropriate laparoscopic colon resection in practice.1 The use of virtual reality simulators (VRSs) for training is well established.2 More recently, the use of VRSs for the assessment of graduates and trainees has been introduced, but Diseases of the Colon & Rectum Volume 57: 2 (2014)

211

Diseases of the Colon & Rectum Volume 57: 2 (2014)

the long-term appropriate application of this technology requires further evaluation. Virtual reality simulators are attractive in that they objectively and contemporaneously record procedural metrics providing immediate feedback of performance to trainees. This may reduce the need for faculty to perform global assessment. Furthermore, with no disposable instruments required, the trainee can do an entire procedure on the simulator without significant cost. Laparoscopic training on simulators is effective at improving technical skills.2–5 However, these previous studies have mostly reported correlation for basic tasks or deconstructed maneuvers, lacking data for specific procedures. Furthermore, the link between simulator training and performance in the operating room cannot be established without the appropriate validation of the most optimal training systems.6 Virtual reality simulators offer a model for laparoscopic sigmoid colectomy that more accurately portrays resection in the operating room than previous hybrid systems.7 Previous studies have shown evidence of face and content validity for the LAP Mentor VRS.8–11 Construct validity has not been established specifically for the laparoscopic sigmoid model with the LAP Mentor system. This must be demonstrated and appropriately evaluated before the widespread implementation of VRS for assessment.12–14 The aim of this study is to determine which metrics recorded by a VRS specifically for laparoscopic sigmoid colectomy have construct validity. If construct validity can be established, an appropriate evaluation of predictive validity is feasible. Design

This study was conducted at the Mount Sinai Skills and Simulation Center at the Case Western Reserve University School of Medicine (Cleveland, OH). Participants were board-certified general surgeons (GSs) with less than 30 laparoscopic colectomies and experienced colorectal surgeons (CSs) with greater than 200 laparoscopic colectomies. All participants were naïve to the VRS model to avoid bias, and they received a 15-minute baseline instruction on the simulator and its controls. They were then each given 5 minutes to familiarize themselves with the controls and instruments before performing the procedure. Each participant then performed a single straight laparoscopic sigmoid colectomy on a VRS (LAP Mentor I, Simbionix, Cleveland, OH). Participants were assessed based on 21 metrics that the simulator objectively recorded. This included 14 procedure-specific metrics and 7 metrics used to measure intraoperative errors. A trained technician, experienced with the simulator, was present at all times to provide guidance with simulator-related issues to minimize simulator-related variance. Laparoscopic CSs were recruited from faculty at the Colorectal Residents Annual Career Course, and GSs were recruited during

­attendance at the Laparoscopic Colorectal Surgery Training Course, both hosted by Case Medical Center, Cleveland, OH. The study was approved by the Institutional Review Board at University Hospitals Case Medical Center. Simulation

LAP Mentor I simulator comprises of a camera and 2 instrumentation channels, linked to a laptop computer and a foot pedal (http://www.simbionix.com). The peritoneal cavity is virtually replicated by the software. Surgical instruments are hybrids with real handles and virtual ends replicating graspers, scissors, staplers, and energy devices. The camera is manipulated by the user and has the ability to zoom and freeze, leaving both hands available for instrument use. This version of the simulator does not provide tactile feedback. The software divides the straight laparoscopic sigmoid colectomy procedure into 2 modules. The first module includes medial peritoneal dissection, division of the inferior mesenteric vessels (clips or stapler), medial to lateral mobilization of Toldt fascia, lateral peritoneal mobilization, and intraperitoneal colonic transection with a linear stapler. The second module requires creation of an intraperitoneal circular stapled colorectal anastomosis. Participants used either virtual scissors or ultrasonic scalpel for dissection. All participants received intraoperative assistance required to change instruments and use staplers, as part of the technical support for the simulator. Statistical Analysis

Data were prospectively collected and recorded. Results were expressed as mean ± SD. Analysis was performed with the SPSS software statistical package (IBM SPSS Statistics, IBM Corporation, Armonk, NY). Performance and scores were compared with the Mann-Whitney U-test. Operative errors were compared with the Kruskal-Wallis test. A p value < 0.05 was considered as significant.

RESULTS Participants

Fifty-three surgeons participated in the study, 29 GSs and 24 laparoscopic CSs. The group of 29 GSs had a mean experience of 13 ± 9 years in laparoscopic general surgery but less than 30 laparoscopic colorectal procedures. All 24 laparoscopic CSs had a minimum experience of 200 laparoscopic colectomies and had previous experience as invited faculty for national and international courses in laparoscopic colorectal surgery. Construct Validity

The VRS for laparoscopic sigmoid colectomy demonstrated evidence of construct validity for 8 of 14 procedural metrics by distinguishing levels of surgical experience (p 

Virtual reality simulator training for laparoscopic colectomy: what metrics have construct validity?

Virtual reality simulation for laparoscopic colectomy has been used for training of surgical residents and has been considered as a model for technica...
210KB Sizes 0 Downloads 0 Views