Minimally Invasive Surgery Force-Sensing Enhanced Simulation Environment (ForSense) for laparoscopic surgery training and assessment Thomas P. Cundy, PhD,a,b Evelyn Thangaraj, BSc,b Hedyeh Rafii-Tari, MSc,a Christopher J. Payne, MEng,a Georges Azzie, MD,c Mikael H. Sodergren, PhD,a,b Guang-Zhong Yang, PhD, FREng,a and Ara Darzi, FRS, FACS,a,b London, United Kingdom, and Toronto, Ontario, Canada
Background. Excessive or inappropriate tissue interaction force during laparoscopic surgery is a recognized contributor to surgical error, especially for robotic surgery. Measurement of force at the tool–tissue interface is, therefore, a clinically relevant skill assessment variable that may improve effectiveness of surgical simulation. Popular box trainer simulators lack the necessary technology to measure force. The aim of this study was to develop a force sensing unit that may be integrated easily with existing box trainer simulators and to (1) validate multiple force variables as objective measurements of laparoscopic skill, and (2) determine concurrent validity of a revised scoring metric. Methods. A base plate unit sensitized to a force transducer was retrofitted to a box trainer. Participants of 3 different levels of operative experience performed 5 repetitions of a peg transfer and suture task. Multiple outcome variables of force were assessed as well as a revised scoring metric that incorporated a penalty for force error. Results. Mean, maximum, and overall magnitudes of force were significantly different among the 3 levels of experience, as well as force error. Experts were found to exert the least force and fastest task completion times, and vice versa for novices. Overall magnitude of force was the variable most correlated with experience level and task completion time. The revised scoring metric had similar predictive strength for experience level compared with the standard scoring metric. Conclusion. Current box trainer simulators can be adapted for enhanced objective measurements of skill involving force sensing. These outcomes are significantly influenced by level of expertise and are relevant to operative safety in laparoscopic surgery. Conventional proficiency standards that focus predominantly on task completion time may be integrated with force-based outcomes to be more accurately reflective of skill quality. (Surgery 2015;157:723-31.) From The Hamlyn Centre, Institute of Global Health Innovation,a the Department of Surgery and Cancer,b Imperial College London, London, United Kingdom; and the Hospital for Sick Children,c University of Toronto, Toronto, Ontario, Canada
THERE IS A RELATIVE LOSS of haptic feedback and decreased manual dexterity in laparoscopic surgery. Compared with a surgeon’s gloved finger in Conflicts of interest: The authors declare no conflicts of interest. Accepted for publication October 24, 2014. Reprint requests: Thomas P. Cundy, PhD, The Hamlyn Centre, Institute of Global Health Innovation, Imperial College London, 3rd floor Paterson Building, St Mary’s Hospital, London W2 1PF, United Kingdom. E-mail: [email protected]. 0039-6060/$ - see front matter Ó 2015 Elsevier Inc. All rights reserved. http://dx.doi.org/10.1016/j.surg.2014.10.015
use during traditional open surgery, the sensitivity of palmar sensory stimuli is estimated to be decreased between 8- and 20-fold,1 and degrees-of-freedom downgraded almost 7-fold. Familiarizing and adapting to these altered sensory circumstances involves a learning process. It is not surprising, therefore, that excessive or inappropriate force is a major contributor to operative errors enacted by trainees during laparoscopic surgery.2 The teaching of safe and effective skills in tissue handling is challenging, particularly because the teaching surgeon is often ‘‘hands off,’’ as either the scrubbed camera assistant or an unscrubbed supervisor. SURGERY 723
724 Cundy et al
Simulation-based training and assessment (SBTA) comprises an increasingly established role in modern surgical curricula. Structured use of box trainer or virtual reality simulators has been well-demonstrated to improve surgical trainee performance in the actual operating room.3-9 Key performance measures for the majority of commercially available SBTA platforms concentrate almost exclusively on efficiency of time and accuracy. Although these measures are studied extensively and well-established in terms of validity and reliability,9-11 they lack sensitivity as objective indicators of the actual quality of operative skill. Force offers the potential to be a valuable discriminator of skill quality and safety in tissue interaction, with particular appeal owing to obvious clinical relevance. Direct measurement of instrument or ‘‘tissue’’ force is lacking, however, in SBTA platforms, especially in box trainers. This study therefore aims to (1) develop a force-sensing unit that is easily and unobtrusively retrofitted with existing box trainer simulators, (2) utilize this Force Sensing Enhanced Simulation Environment (ForSense) to investigate and validate multiple variables of force as objective measurements of laparoscopic skill, and (3) determine concurrent validity of a derived scoring metric of force error against the validated standard metric. MATERIALS AND METHODS Hardware development. A number of modifications were made to the existing Pediatric Laparoscopic Surgery (PLS) box trainer simulator.12 First, a replacement base was manufactured using a 3D-printer (Fig 1, a). This base incorporated a 55 3 32-mm cutout space at the site in which simulation task items would be positioned normally. A separate 53 3 30 3 6-mm rectangular plate was 3D-printed to fit within this empty space, allowing a 2-mm margin. The plate was sensitized by a 3-point screw mounting to a calibrated Nano17 6-axis force-torque (F/T) transducer (ATI Industrial Automation, Apex, NC). Four supporting columns were included in the design of the replacement base to accommodate the F/T transducer underneath the PLS box, and also to position securely both the box and F/T transducer mount to a main stabilizing board. The pegboard supplied with the PLS simulator was drilled with 2 countersunk holes for attachment to the base plate using screws (Fig 1, b). The Penrose drains supplied with the PLS simulator were affixed interchangeably to the base plate using 3M Atraumatic Command adhesive patches (3M, St. Paul, MN; Fig 1, d). When in place, the location and orientation of both the
Surgery April 2015
pegboard and Penrose drain were identical to the standard PLS simulator. Software development and system integration. The F/T transducer was tethered to the same computer used for the video input and display. Synchronization of force data and video feed was provided by custom software operated in C++, using UDP communication through LabVIEW (National Instruments Corporation, Austin, TX) for reading force data and the OpenCV (Open Source Computer Vision) library for recording video feed. The F/T transducer transmits data at a frequency of 25 Hz and is able to detect forces to a resolution of 3.5 milli-Newtons (mN). The 3 axes of measured force were aggregated to a single force modulus output. A dual monitor display was configured such that participants viewed only the video feed and were blinded to real-time force feedback that was displayed as a live trace on the other monitor (Fig 1). Participant testing. The MISTELS-based fundamentals of laparoscopic surgery (FLS) and PLS curricula consist of 5 inanimate abstract tasks. Participants were recruited to perform 5 repetitions each of 2 PLS curriculum tasks: the peg transfer and suturing with intracorporeal knot tying. These 2 tasks are regarded respectively as the easiest and hardest components of the manual skills curricula. A sample size calculation was performed a priori based on the study by Horeman et al,13 which determined maximum force for expert surgeons to complete a laparoscopic needle drive task to be 2.6 ± 0.4 N (mean ± SD).13 It was estimated that to detect a 25% difference in force between groups with a 2-sided 5% significance level and power of 80%, a sample size of 6 participants in each group was required. All participants were volunteers who were recruited from 1 university teaching hospital. Participants were stratified into novice, intermediate, and expert surgeon groups based on recent clinical experience involving 1.5 N of tractive force was applied to suture through fresh porcine intestine. Modified scoring metrics (fPLS) were calculated post hoc that imposed additional penalties based on force error. The fPLS metrics for the peg transfer and suture tasks were PLS score – (force error 3 10) and PLS score – (force error 3 2), respectively. Multiplication factors were derived from the calculation maximum force error/100 to generate an adjusted penalty based on force error that was graded from 0 to 100. Data collection and analysis. Raw force data were exported as text files for post processing. The data were preprocessed to isolate and condense outputs of force measurement to time points in which laparoscopic instruments were actively engaged with the task objects. This process was achieved by zeroing at the value of minimum recorded force, and then eliminating force data points that were
Background. Excessive or inappropriate tissue interaction force during laparoscopic surgery is a recognized contributor to surgical error, especially for robotic surgery. Measurement of force at the tool–tissue interface is, therefore, a clinically relevant skill assessment variable that may improve effectiveness of surgical simulation. Popular box trainer simulators lack the necessary technology to measure force. The aim of this study was to develop a force sensing unit that may be integrated easily with existing box trainer simulators and to (1) validate multiple force variables as objective measurements of laparoscopic skill, and (2) determine concurrent validity of a revised scoring metric. Methods. A base plate unit sensitized to a force transducer was retrofitted to a box trainer. Participants of 3 different levels of operative experience performed 5 repetitions of a peg transfer and suture task. Multiple outcome variables of force were assessed as well as a revised scoring metric that incorporated a penalty for force error. Results. Mean, maximum, and overall magnitudes of force were significantly different among the 3 levels of experience, as well as force error. Experts were found to exert the least force and fastest task completion times, and vice versa for novices. Overall magnitude of force was the variable most correlated with experience level and task completion time. The revised scoring metric had similar predictive strength for experience level compared with the standard scoring metric. Conclusion. Current box trainer simulators can be adapted for enhanced objective measurements of skill involving force sensing. These outcomes are significantly influenced by level of expertise and are relevant to operative safety in laparoscopic surgery. Conventional proficiency standards that focus predominantly on task completion time may be integrated with force-based outcomes to be more accurately reflective of skill quality. (Surgery 2015;157:723-31.) From The Hamlyn Centre, Institute of Global Health Innovation,a the Department of Surgery and Cancer,b Imperial College London, London, United Kingdom; and the Hospital for Sick Children,c University of Toronto, Toronto, Ontario, Canada
THERE IS A RELATIVE LOSS of haptic feedback and decreased manual dexterity in laparoscopic surgery. Compared with a surgeon’s gloved finger in Conflicts of interest: The authors declare no conflicts of interest. Accepted for publication October 24, 2014. Reprint requests: Thomas P. Cundy, PhD, The Hamlyn Centre, Institute of Global Health Innovation, Imperial College London, 3rd floor Paterson Building, St Mary’s Hospital, London W2 1PF, United Kingdom. E-mail: [email protected]. 0039-6060/$ - see front matter Ó 2015 Elsevier Inc. All rights reserved. http://dx.doi.org/10.1016/j.surg.2014.10.015
use during traditional open surgery, the sensitivity of palmar sensory stimuli is estimated to be decreased between 8- and 20-fold,1 and degrees-of-freedom downgraded almost 7-fold. Familiarizing and adapting to these altered sensory circumstances involves a learning process. It is not surprising, therefore, that excessive or inappropriate force is a major contributor to operative errors enacted by trainees during laparoscopic surgery.2 The teaching of safe and effective skills in tissue handling is challenging, particularly because the teaching surgeon is often ‘‘hands off,’’ as either the scrubbed camera assistant or an unscrubbed supervisor. SURGERY 723
724 Cundy et al
Simulation-based training and assessment (SBTA) comprises an increasingly established role in modern surgical curricula. Structured use of box trainer or virtual reality simulators has been well-demonstrated to improve surgical trainee performance in the actual operating room.3-9 Key performance measures for the majority of commercially available SBTA platforms concentrate almost exclusively on efficiency of time and accuracy. Although these measures are studied extensively and well-established in terms of validity and reliability,9-11 they lack sensitivity as objective indicators of the actual quality of operative skill. Force offers the potential to be a valuable discriminator of skill quality and safety in tissue interaction, with particular appeal owing to obvious clinical relevance. Direct measurement of instrument or ‘‘tissue’’ force is lacking, however, in SBTA platforms, especially in box trainers. This study therefore aims to (1) develop a force-sensing unit that is easily and unobtrusively retrofitted with existing box trainer simulators, (2) utilize this Force Sensing Enhanced Simulation Environment (ForSense) to investigate and validate multiple variables of force as objective measurements of laparoscopic skill, and (3) determine concurrent validity of a derived scoring metric of force error against the validated standard metric. MATERIALS AND METHODS Hardware development. A number of modifications were made to the existing Pediatric Laparoscopic Surgery (PLS) box trainer simulator.12 First, a replacement base was manufactured using a 3D-printer (Fig 1, a). This base incorporated a 55 3 32-mm cutout space at the site in which simulation task items would be positioned normally. A separate 53 3 30 3 6-mm rectangular plate was 3D-printed to fit within this empty space, allowing a 2-mm margin. The plate was sensitized by a 3-point screw mounting to a calibrated Nano17 6-axis force-torque (F/T) transducer (ATI Industrial Automation, Apex, NC). Four supporting columns were included in the design of the replacement base to accommodate the F/T transducer underneath the PLS box, and also to position securely both the box and F/T transducer mount to a main stabilizing board. The pegboard supplied with the PLS simulator was drilled with 2 countersunk holes for attachment to the base plate using screws (Fig 1, b). The Penrose drains supplied with the PLS simulator were affixed interchangeably to the base plate using 3M Atraumatic Command adhesive patches (3M, St. Paul, MN; Fig 1, d). When in place, the location and orientation of both the
Surgery April 2015
pegboard and Penrose drain were identical to the standard PLS simulator. Software development and system integration. The F/T transducer was tethered to the same computer used for the video input and display. Synchronization of force data and video feed was provided by custom software operated in C++, using UDP communication through LabVIEW (National Instruments Corporation, Austin, TX) for reading force data and the OpenCV (Open Source Computer Vision) library for recording video feed. The F/T transducer transmits data at a frequency of 25 Hz and is able to detect forces to a resolution of 3.5 milli-Newtons (mN). The 3 axes of measured force were aggregated to a single force modulus output. A dual monitor display was configured such that participants viewed only the video feed and were blinded to real-time force feedback that was displayed as a live trace on the other monitor (Fig 1). Participant testing. The MISTELS-based fundamentals of laparoscopic surgery (FLS) and PLS curricula consist of 5 inanimate abstract tasks. Participants were recruited to perform 5 repetitions each of 2 PLS curriculum tasks: the peg transfer and suturing with intracorporeal knot tying. These 2 tasks are regarded respectively as the easiest and hardest components of the manual skills curricula. A sample size calculation was performed a priori based on the study by Horeman et al,13 which determined maximum force for expert surgeons to complete a laparoscopic needle drive task to be 2.6 ± 0.4 N (mean ± SD).13 It was estimated that to detect a 25% difference in force between groups with a 2-sided 5% significance level and power of 80%, a sample size of 6 participants in each group was required. All participants were volunteers who were recruited from 1 university teaching hospital. Participants were stratified into novice, intermediate, and expert surgeon groups based on recent clinical experience involving 1.5 N of tractive force was applied to suture through fresh porcine intestine. Modified scoring metrics (fPLS) were calculated post hoc that imposed additional penalties based on force error. The fPLS metrics for the peg transfer and suture tasks were PLS score – (force error 3 10) and PLS score – (force error 3 2), respectively. Multiplication factors were derived from the calculation maximum force error/100 to generate an adjusted penalty based on force error that was graded from 0 to 100. Data collection and analysis. Raw force data were exported as text files for post processing. The data were preprocessed to isolate and condense outputs of force measurement to time points in which laparoscopic instruments were actively engaged with the task objects. This process was achieved by zeroing at the value of minimum recorded force, and then eliminating force data points that were
