THE INTERNATIONAL JOURNAL OF MEDICAL ROBOTICS AND COMPUTER ASSISTED SURGERY Int J Med Robotics Comput Assist Surg (2014) Published online in Wiley Online Library (wileyonlinelibrary.com) DOI: 10.1002/rcs.1582
ORIGINAL ARTICLE
Virtual reality training and assessment in laparoscopic rectum surgery
Jun J. Pan1* Jian Chang2 Xiaosong Yang2 Hui Liang5 Jian J. Zhang2 Tahseen Qureshi3 Robert Howell4 Tamas Hickish4
Abstract Background Virtual-reality (VR) based simulation techniques offer an efficient and low cost alternative to conventional surgery training. This article describes a VR training and assessment system in laparoscopic rectum surgery. Methods To give a realistic visual performance of interaction between membrane tissue and surgery tools, a generalized cylinder based collision detection and a multi-layer mass–spring model are presented. A dynamic assessment model is also designed for hierarchy training evaluation.
1
State Key Laboratory of Virtual Reality Technology and Systems, Beihang University, Beijing, China
2
National Centre for Computer Animation, Media School, Bournemouth University, Poole, UK
3
Poole Hospital, Poole, UK
4
The Royal Bournemouth and Christchurch Hospitals, Bournemouth, UK
Results With this simulator, trainees can operate on the virtual rectum with both visual and haptic sensation feedback simultaneously. The system also offers surgeons instructions in real time when improper manipulation happens. The simulator has been tested and evaluated by ten subjects. Conclusions This prototype system has been verified by colorectal surgeons through a pilot study. They believe the visual performance and the tactile feedback are realistic. It exhibits the potential to effectively improve the surgical skills of trainee surgeons and significantly shorten their learning curve. Copyright © 2014 John Wiley & Sons, Ltd.
5
School of Animation, Communication University of China, Beijing, China *Correspondence to: Jun J. Pan, State Key Laboratory of Virtual Reality Technology and Systems, Beihang University, Beijing, China. E-mail: [email protected]
Accepted: 19 January 2014
Copyright © 2014 John Wiley & Sons, Ltd.
Keywords collision detection; dissection simulation; evaluation; laparoscopic rectum surgery; dynamic assessment
Introduction In recent years there has been remarkable development in minimally invasive surgery (MIS) techniques in areas such as laparoscopic and endoscopic surgeries. Compared with traditional open surgery, due to the different hand–eye coordination and the fulcrum effect of the abdominal wall for instrument manipulation (1), the performance of MIS requires a special set of psychomotor and perceptual skills. Therefore, laparoscopic surgery is difficult to learn by observation and practice alone, using conventional apprenticeship practice. A few approaches can be used to speed up the learning of laparoscopic surgeons. One of them is called the training box (2,3). These boxes may contain inanimate objects, which allow mundane tasks to be performed. These training objects are usually made from rubber or silica gel, which understandably lack the realistic feel of handling human tissues. Animal tissue can be used as an
J. J. Pan et al.
alternative, but it is different from the human anatomy structure. Though cadavers are available, they are quite expensive and are not suitable for repetitive training. In reality, residents in many nations, including the UK, develop their laparoscopic surgical skills by directly operating on real patients under supervision from senior surgeons (4). Recently, with the rapid growth of computer power, virtual reality (VR) simulators have become a viable alternative approach, which allows the trainee to hone their skills by operating on virtual patients in a safe and repetitive manner. The aim of this paper is to present a VR-based training and assessment system for laparoscopic rectum surgery. At present, colorectal cancer is the third commonest cancer in the UK, with approximately 40 000 new cases diagnosed annually (4). It is estimated that 90% of the cases are suitable for laparoscopic resection surgery. However, owing to the complicated and technically challenging nature of laparoscopic colorectal cancer surgery, there is a relative lack of trained surgeons to perform such demanding operations. On the other hand, although there are several commercial VR based laparoscopic surgery simulators (LapSim (5), Lap Mentor (6), Mentice (7)) on the market, most of them aim to simulate procedures in cholecystectomy (gallbladder removal), ventral hernia repair, ectopic pregnancy and myomectomy. The application to rectal resection surgery for cancer is rare owing to the narrow pelvis, substantial large soft tissue deformation and complex interactions between surgery tools and soft tissue in operation. Therefore, it brings a greater challenge for the haptic response and realistic graphic computation. Owing to the advantage of MIS, researchers have applied a great deal of effort to virtual reality based surgery simulation. Pan et al. (8) described a simple simulator for virtual laparoscopic rectum surgery. They select triangle meshes rather than the tetrahedra model to represent the bowel. To meet the requirement of real-time graphic performance, a cosserat rod model had been introduced to handle the physical deformation and collision detection. Choi et al. (9) designed a VR training system for laparoscopic oesophagus surgery simulation. They used a boundary element method (BEM) to compute deformation based on the assumption of linear response. The mechanical parameters were collected from experiments on animal tissue and integrated with a haptic device. Heng et al. (10) developed a VR simulation system for knee arthroscopic surgery. In the system, they used virtual models from the database ‘Visible Human’. Finite element analysis (FEA) was also applied to simulate the deformation and incision of soft tissue in real time. Haptic rendering (11–13) is another important research field in VR based surgery simulation. Maciel et al. (14) presented a dynamic point algorithm for line based collision detection in haptics. They simplified the surgical instrument to a Copyright © 2014 John Wiley & Sons, Ltd.
section of line with two end points, and a dynamic point located between the two end points. In this approach, the dynamic point is the closest point on the line to potentially collide with the object. It can compute the interaction between a line-shaped haptic tool and the object model surface efficiently. De Novi (15) presented the Blobby modelling approach which represents organs with a set of spheres. It is able to perform fast collision detection between different organs and between virtual surgical instruments. Basdogan et al. (16) presented a survey on all the important aspects of haptic devices in MIS simulation, including haptic rendering, interface and data recording. Compared with the traditional surgical training method, besides providing repetitive tasks and variable complexity for trainees, VR based laparoscopic surgery simulators can also record the training progress, assess trainees’ surgical skill objectively and offer informative feedback. Gurusamy et al. (17) provided a total of 23 randomized clinical trials to compare virtual reality training with other forms which include no training, video training, conventional laparoscopic training for surgeons without prior laparoscopic surgery experience. 612 participants were involved in these trials. The assessment considered the following factors: (1) Time, such as calculating ‘operating time’, ‘time for one stitch and two ties’, or ‘time for driving a needle through the tissue and in tying a surgical knot’; (2) Error score, such as ‘error score by dropping the object or perforation of object’ or ‘average error incidence by tear of sheet, slack of ligature’; (3) Accuracy, such as ‘mean number of correct incisions’; and (4) Others, such as speed, blood loss, number of movements, economy of movements, distance tracing of instruments. Through experimental investigation, this feedback in training can help trainees to correct surgical mistakes efficiently and make good progress. At present, most commercial medical simulators provide the basic functions of evaluation and consultancy to trainees considering a number of surgical factors, such as time of completion and wall collisions. However, for specific laparoscopic rectum surgery training, few VR based simulators meet the above requirements. In fact, the continued need for comprehensive assessment and instructor’s feedback from a training system is often regarded as a primary reason for the reluctance of many medical schools to fully embrace the virtual training technology (3). For that purpose, we present a VR based training and assessment system for the laparoscopic rectum surgery. First, referencing to the real surgery videos and MRI data, the surface mesh model of a section of rectum is constructed and preprocessed as the training object. Generally, the cursor (proxy) from a haptic device effects to one point during interactions. It can lead to incorrect soft tissue deformation during surgical simulation. To Int J Med Robotics Comput Assist Surg (2014) DOI: 10.1002/rcs
VR training and assessment in laparoscopic rectal resection surgery
offer realistic geometric deformation in interactions, we have designed a generalized cylinder based collision detection algorithm for haptics. Second, to achieve the high deformation accuracy and real time graphic performance, we used the cosserat rod model mentioned in (18) to deal with the physical deformation of the rectum. Third, a multi-layer mass spring model is designed to cope with the deformation and surgery behaviour of membrane tissue surrounding the rectum. And a straightforward incision approach is presented based on this model. It can simulate the dissection procedure to detach the rectum from its surrounding tissue in real time. Finally, a hierarchical evaluation module coupled with a dynamic assessment model has been implemented to evalute and offer trainees constructive feedback in both an intra-operative and post-operative manner.
Materials and methods Operation environment and VR simulator interface In laparoscopic rectal resection surgery for cancer treatment, some preoperative diagnosis and preparation should be made before the operation (4). Endoscopy is used to determine the histological confirmation of the cancer and its location in the rectum. Then a CT scan will be applied to assess whether the tumour has spread to distant organs, such as lungs and liver. Sometimes, the patient will be recommended to have a MRI scan to acquire particular information on the local extent of the tumour. It can help with the formulation of a plan of treatment, in particular, whether the operation should be proceeded by chemoradiotherapy. In the operating theatre, the patient is usually placed in a modified Lloyd-Davis position (4). The number of inserted operating ports varies between surgeons, but typically four operating ports are used. An operation assistant is needed to control the laparoscope and help with the retraction of tissues during operation. The surgeon relies on a variety of tools, such as stapling devices, graspers and L-hooks, to help seal, grasp, dissect and separate tissues. During the whole procedure, the most challenging task and eventual aim of the surgery is to detach the rectum from its surrounding pelvic structure. Then the specimen will be removed from the patient through an extraction site either via a Pfannensteil incision or in the left iliac fossa (4). Figure 1(a) shows the theatre set up for laparoscopic rectal cancer surgery. Figure 1(b) is a screenshot of our simulator interface. It illustrates a section of rectum model and the surrounding soft tissue covered. The simulator hardware (Figure 1(c)) Copyright © 2014 John Wiley & Sons, Ltd.
Figure 1. Comparison of real laparoscopic rectum surgery with VR based simulation system. (a) Real operating environment for laparoscopic rectum surgery. (b) Software interface of VR based simulator. (c) Snapshot of VR based simulator hardware Int J Med Robotics Comput Assist Surg (2014) DOI: 10.1002/rcs
J. J. Pan et al.
consists of a monitor, a computer and two haptic devices (Phantom Omni). Generally, one haptic device functions as a grasper, and the other functions as an L-hook or harmonic scalpel used for dissection. The left haptic device can also be used as a navigation camera in this system. These haptic devices provide 6-DOF navigating parameters and 3-DOF force feedback. The software was developed with OpenGL, C++, Mitk (19) and Qt. The architecture of the system is described in Figure 2. From the clinical aspect, our most important contribution is to present a comprehensive framework of virtual laparoscopic rectum surgery simulation and evaluation, from system design to implementation. Compared with previous research work in VR based medical simulation, there are three innovative technical contributions: 1. A generalized cylinder based collision detection algorithm is presented to offer fast and efficient detection of the interaction between soft tissue and surgery instruments which have a complex shape. 2. A multi-layered mass–spring model is designed to cope with the deformation and surgical behaviour of membrane tissue surrounding the rectum. Correspondingly, a straightforward real-time incision approach is presented based on this model. It can simulate the dissection procedure to detach a section of rectum from its surrounding soft tissue. 3. A dynamic assessment model is developed and a hierarchical evaluation module has been implemented to collect and analyse the evaluation metrics for laparoscopic rectum surgery training.
Generalized cylinder based collision detection for haptics In a real-time VR system, the display rate of graphic rendering is about 30 frames per second. However, in a multimodal virtual environment with haptic interactions, a much higher update rate of about 1 kHz is needed to ensure smooth force feedback and continuous interactions. Theoretically, the force from the haptic device (Phantom Omni) applies to only one contact point during manipulation. The most efficient interaction method is to represent the proxy (haptic cursor) as a point. However, in many cases, the interactions of instruments in surgery simulation are using long slender tools (such as dissect, shear, L-hook and so on) whose contact surface is an area or a line. Detection problems may occur if it is simplified to a single point or a line, especially when local concavities exist in the model (Figure 3). In this simulator, we treat the surgical instrument as a generalized cylinder object (L-hook) or a combination of multiple generalized cylinder objects (dissect, shear), and an innovative generalized cylinder based collision detection algorithm is designed. It can be described briefly as follows. In Figure 4, suppose the two end points of axis (dotted red line) are p0, p1. The radius of the cylinder is variable along the axis, and in a certain area the value of the radius is r. The precondition of this collision detection algorithm is that the surgery instruments should initially be outside the organ objects when the graphic rendering starts. Then in every frame of the graphic rendering loop, each vertex (v) of the organ object will be estimated to determine whether it is inside the cylinder or not. If it is inside the instrument model, a collision will take place. And a penalty force will be placed on the vertex according to its penetration depth. This algorithm can be divided into two steps: Step 1: Calculating the angle: ∠ p0p1v and ∠ p1p0v. If ∠ p1p0v ≤ 90° and ∠ p0p1v ≤ 90°, then go to Step 2. If not, v is outside the cylinder. Step 2: Calculating the Euclidean distance (d) between v and cylinder axis line (p0p1). If d 30). Besides the high computation efficiency, another important advantage of the multi-layer mass– spring model is the simplicity in dissection simulation.
topology of model, even re-mesh the object. It is an extremely time consuming process. In this work, inspired by the mechanism of residual strain in bio-materials (31), we present a straightforward real-time incision technique which can be used to simulate the dissection procedure of detaching the rectum model from its surrounding tissue. This technique is designed based on the multilayered mass–spring model and can produce the realistic shrinking effect of the surrounding tissue when it is detached from the rectum.
The simulation of dissection The critical procedure in laparoscopic rectal cancer surgery is cutting the tissue between the rectum and its surrounding soft tissue, to enable the section of bad intestine (with tumour) to be disconnected from the pelvic structures so that it can be removed. Dissection is therefore the most important task in training. Technically, interactive realistic cutting is one of the most challenging topics in VR based surgery simulation. It has attracted a great deal of research effort. Most cutting algorithms (28–30) based on mesh processing need to change the
Figure 8. Deformation caused by the residual stress is equivalent to the results of a straight cut followed by an equivalent force applied to the cutting surface Copyright © 2014 John Wiley & Sons, Ltd.
Figure 9. Simulation of dissection to separate the rectum and its surrounding soft tissue: (a) before cutting the spring; (b) after cutting the spring Int J Med Robotics Comput Assist Surg (2014) DOI: 10.1002/rcs
VR training and assessment in laparoscopic rectal resection surgery
Residual strain exists in many kinds of bio-materials. It causes complexity for the estimation of deformation when a given bio-structure is subject to external loads. Experiments were designed to investigate the inner strain distribution of the residual stress but often is limited to tubular structures, such as arteries (31,32) and oesophagus (33,34). Analysis by Alastrué et al. (35) showed that the incorporation of residual stress in the computation affects the circumferential stress field of arterial structures as well as the deformed configuration. In our simulation, we want to produce the shrinking effect of connective tissues surrounding the rectum near the pelvis floor when they are separated from the rectum. Such deformation could have been caused by the residual strain. In real surgery, the surgeon has to estimate its effect before and during dissection to reduce the risk of unwanted damage and complication. There is a lack of experimental evidence to provide information for the residual strain distribution within the area of interest. The shrinking effect of the tissue surrounding the rectum in real surgery suggests that the connective tissue may be subject to a pre-stressed tensile force. Experiments on the oesophagus (33,34) suggested that the residual strain can vary from 0 to 0.3. Here we assume a residual tensile strain of 0.1 uniformly distributed in the connective tissue. Prime (36) suggested the contour of the exposed surface due to a cut can be related to adding a counterpart force to the cutting plane according to the superposition principle. Therefore, given the known residual tensile stress, we can identify the deformed shape by applying the equivalent force on the surface (Figure 8). We computed the residual stress σr as σ r ¼ Eεr
(5)
where E is Young’s modulus and εr is the residual strain. Here coupled with equations (3) and (5), we simplify the simulation of dissection as a ‘spring cutting’ procedure, which can be illustrated as in Figure 9. The L-hook
is usually used as the dissection instrument. If the tip of L-hook touches an incorrect place or vulnerable area which should be avoided in training, the system will give a warning signal. Otherwise, when the tip of L-hook touches the area of the second and third layer of this model, it will search the springs between these two layers in a small sphere range (the small circle in dotted line). If a spring is detected, it will be ‘cut off’ (deleted) from this mass–spring model and removed from the dynamic system. Meanwhile, a residual tensile stress is added at the nodes linked by this spring. So with 3 and (5), there is: X X F Pi ðx; v; tÞ ¼ F Pi ; Pj þ f ðPi Þ þ GðPi Þ þ σ r (6) After computation of this updated dynamic system, a shrinking effect of the rectum and its surrounding tissue is performed. Figure 14 gives some experimental results of dissection simulation. Due to the residual tensile stress and the shrinking effect, the rectum and its surrounding tissue are separated clearly after dissection (Figure 14(b)).
Evaluation of training Besides providing realistic surgical simulation in real time, our simulator can also assess the trainee’s surgical skills and offer them constructive feedback during and after the operation. There exist several commercial medical simulators (5, 6, 7) which are able to provide basic functions of evaluation and advice to the trainees. However, for laparoscopic rectum surgery training, few VR based simulators implement the assessment functions. In reality, the continued need for comprehensive assessment and instructor’s feedback from the training system is often regarded as a primary reason for the reluctance of many medical schools to fully adopt VR based training techniques (3). For that purpose,
Table 1. Metrics in hierarchy evaluation module for the simulator First Grade
Second Grade
Surgical Skill
Efficiency
Third Grade
Safety
Completion Time Redundant Fabric Used Exposure
Quality
Wall Collision Times Near Vulnerable Tissue Forces Near Vulnerable Tissue Velocities Drilling & Cutting Technique
Accurate Technique
Copyright © 2014 John Wiley & Sons, Ltd.
Fourth Grade
Directly Exposed Indirectly Exposed Overexposed
Cutting Length Cutting Depth Cutting Region Cutting Angle Jump Frequency Instrument Selection Ratio of Correct Cutting Ratio of Incorrect Cutting
Int J Med Robotics Comput Assist Surg (2014) DOI: 10.1002/rcs
J. J. Pan et al.
Figure 10. Dynamic assessment model
to implement the assessment function for laparoscopic rectum surgery in our simulator, we define a dynamic assessment model and present a hierarchical evaluation module.
simulator, which is authorized to evaluate the trainee’s surgical skills; the evaluation object is the trainee; the main purpose is providing constructive feedback to trainees for skill promotion; metrics represent the aspects of surgical skills and can be composed as a hierarchical structure (Table 1) in our system for comprehensive evaluation; for the method, we apply fuzzy logic (37) to the data recorded, and determine the levels of expertise for each trainee considering the most major evaluation metrics. By recording the training data from virtual operation on the simulator, the evaluation result is provided to trainees after analysis. Meanwhile, during the simulation, the system can also instruct the trainees when incorrect manipulation occurs. For example, the system will trigger an alarm if the trainee grasps or cuts a vulnerable area which should be avoided. Evaluation exists in both intra-operative and post-operative intervention procedures. Based on the descriptions above, we define and build a dynamic assessment model, which is shown in Figure 10. In this Figure, the evaluation results can give feedback to the trainees to improve their training behaviour in real time.
Hierarchical evaluation module Dynamic assessment model In the evaluation process, there are seven essential factors: evaluation subject, evaluation object, purpose, metrics, method, result and time. In our simulator, the evaluation subject is the VR based laparoscopic rectum surgery
The comprehensive evaluation of surgical skills is difficult since it involves the assessment of all aspects of a surgeon’s techniques, such as the ability for correct decision making in response to the discrete events in a surgical scenario, maintaining proper visibility of the surgical field,
Figure 11. Weight setting for evaluation metrics Copyright © 2014 John Wiley & Sons, Ltd.
Int J Med Robotics Comput Assist Surg (2014) DOI: 10.1002/rcs
VR training and assessment in laparoscopic rectal resection surgery
achieving sufficient exposure of critical anatomy structures, removing the optimal volume of tissue, applying appropriate velocities and forces when vulnerable anatomy
structures are approached, and so on. It is not practical to consider all possible factors to assess a trainee. And some factors belong to the domain of psychology, which
Figure 12. Evaluation results on the software interface of the simulator. (a) Recorded training data and score. (b) The distribution of evaluated surgical skill levels for all trainees Copyright © 2014 John Wiley & Sons, Ltd.
Int J Med Robotics Comput Assist Surg (2014) DOI: 10.1002/rcs
J. J. Pan et al.
is hard to measure by a quantitative approach. We thus select only the key evaluation metrics which best fit the given surgery, and assemble these metrics to a form which can reflect the trainee’s skill in a higher level. Compared with some existing simulators (5, 7), we set up a comprehensive evaluation module with a hierarchical structure rather than only providing the basic evaluation metrics. The surgical skill hierarchy evaluation module covers most major training targets with different hierarchy levels. Its structure is illustrated in Table 1. In this table, evaluation metrics are divided into four grades: the first grade indicates the surgical expertise of the trainee. The other three grades are composed of subsets of concrete evaluation metrics. On the second grade, the surgical skills are evaluated by following three categories of metrics: Efficiency (the time spent on training task, and the quantity of redundant fabrics used), Safety (the parameters that measure the safety of operations in virtual training, such as the ability to avoid unwanted collisions between instruments and vulnerable tissues), Quality (the parameters which measure the performed skills of trainees in the task). We also define adjustable weights for each evaluation metric, which are illustrated in Figure 11. After weight setting, the system can compute the total score for the trainees. The personal information and training record for each individual will be listed below the Window ‘Learning Centre’ in the system interface (Figure 12(a)). After all the trainees’ performances are evaluated and recorded, the system will classify the trainees to different levels according to the evaluation scores of their surgical expertise. The level for all the trainees can be illustrated in the system interface (Figure 12(b)).
the system gives a score to each trainee and illustrates statistical data for this trainee group. Our system runs on a workstation with Intel CoreQuad 2.83 GHz, 6 GB RAM and equipped with a NVIDIA Quadro FX 3700 graphics card. The force refreshing frequency is 1000 Hz and the graphic refreshing frequency is 45 Hz. To evaluate the effectiveness of the techniques in this simulator, we have also designed two experiments. The first tests the visual performance of the generalized
Results Implementing the techniques described above, a prototype virtual reality based simulation and assessment system was developed for laparoscopic rectum surgery. When the system starts, the simulator generates a continuous loop with force feedback. The haptic devices send position and orientation information of surgery instruments to the computer. The collisions detection between surgery instruments and organ models are processed according to these data. The soft tissue deformation and the feedback force are computed by the hybrid mechanical model of the rectum and the multi-layered mass– spring model of tissue surrounding the rectum. Then the feedback force signal is sent to the haptic devices for rendering. When the simulator is running, the assessment module records all the evaluation metrics for each trainee. Meanwhile, since the assessment is a dynamic process, it can give the trainee instructions in real time when incorrect manipulation occurs. Finally, after training is finished, Copyright © 2014 John Wiley & Sons, Ltd.
Figure 13. Comparison of visual performance of a L- hook touching the surface of liver model: (a) before collision; (b) collision detection by the generalized cylinder based algorithm; (c) collision detection by a point-based method Int J Med Robotics Comput Assist Surg (2014) DOI: 10.1002/rcs
VR training and assessment in laparoscopic rectal resection surgery
cylinder based collision detection. Figure 13 illustrates the results compared with a standard point-based method. In Figure 13(a), the whole L-hook can touch the surface of the liver and deform the model realistically. However, in Figure 13(c), the handle of the L-hook penetrates in the liver because of the ‘Local Concavity Trap’. The second experiment verifies the dissection simulation which is the most important training task in the surgery. In this test, the trainee uses three instruments (one L-hook and two graspers) to detach a section of rectum from its surrounding soft tissue. From the experimental results illustrated in Figure 14, we found that the deformation is plausible. After dissection, the rectum and its surrounding tissue are clearly separated. Moreover, our simulator also includes special visual effects, such as smoke, for training (Figure 14(f)).
We also invited ten subjects with varying laparoscopic colorectal surgery experiences to test and assess our system. The subjects were asked to perform the dissection procedure (Figure 14) on our simulator, and then to answer a feedback questionnaire consisting of nine questions: (1) realism of the anatomy; (2) realism of the appearance of the simulator interface; (3) realism of the instrument handling; (4) overall realism of the dissection task; (5) quality of force feedback in performing the tasks; (6) usefulness in learning hand–eye coordination; (7) usefulness in learning ambidexterity skills; (8) overall usefulness in learning the fundamental surgical skills; (9) accuracy of the evaluation of performance in training. Each question can be answered on a 5 point Likert scale with 5 for very good and 1 for very poor. The results from the questionnaire study are given in Table 2.
Figure 14. Screenshots of dissection to detach the rectum from its surrounding tissue. (a) Before dissection; (b) 5 s after dissection procedure starts; (c) 60 s after dissection procedure starts; (d) 80 s after dissection procedure starts; (e) 100 s after dissection procedure starts; (f) smoking effect during dissection simulation Copyright © 2014 John Wiley & Sons, Ltd.
Int J Med Robotics Comput Assist Surg (2014) DOI: 10.1002/rcs
J. J. Pan et al. Table 2. Descriptive statistics obtained from the questionnaire study
Mean SD
1
2
3
4
5
6
7
8
9
3.2 0.86
3.6 0.92
3.5 0.81
3.7 1.07
3.9 0.72
3.9 0.73
3.7 0.92
4.1 0.87
3.5 0.91
Discussion The first VR based surgery simulator for rectum cancer has been designed. Throughout the whole development process for this simulator, three senior surgeons from the Royal Bournemouth and Christchurch Hospital, Poole Hospital (NHS, UK) have been closely involved. They advise on the medical content and assist in evaluating the training results as both consultants and trainees. Satisfactory tactile feedback is achieved based on their operational experience. Through haptic rendering, users can see a realistic internal surgery environment with visual outputs and tactile feeling in real time. In the evaluation process, the result from the preliminary study (Table 2) shows the average scores for all the questions. It also indicates which aspects need to be improved for more realistic and useful simulation. In Table 2, the lowest scores (3.2) are for ‘the realism of the anatomy’. Due to the computation complexity, we are currently modifying the original rough rectum model to a clean and smooth one. In future, we can apply GPGPU programming (e.g. CUDA) to enhance the graphic computation power for our system. Then we can choose a more complex rectum model for surgery simulation.
dissection simulation. We also plan to add GPGPU programming to enhance the graphic computation speed further for our system. Then we can choose more complex rectum models for surgical simulation. Furthermore, currently we have only implemented the function of training assessment. In future, more trials and study will be made in the assessment of laparoscopic rectum surgery training through our system. With the data collected and analyzed, this VR based training system will be compared quantitatively with other approaches, such as physically based training and standard laparoscopic training.
Acknowledgements The research in this article was supported by a grant from the Royal Bournemouth and Christchurch Hospitals NHS Trust and a HEIF grant from Bournemouth University.
Conflict of interest We declare that we have no financial and personal relationships with other people or organizations that can inappropriately influence our work, there is no professional or other personal interest of any nature or kind in any product, service and/or company that could be construed as influencing the position presented in, or the review of, the manuscript entitled
Funding No specific funding.
Conclusion and future work We have developed a prototype virtual reality based training and assessment system for laparoscopic rectum surgery. Several technical innovations have been presented, including generalized cylinder based collision detection, a multi-layered mass-spring model for computing the deformation and rectum dissection, and a dynamic model for hierarchical evaluation. Real-time visual performance is achieved. The surgeon collaborators are satisfied with the realistic visual quality and tactile feedback of the simulator. Despite successful implementation of the prototype, this simulator also has limitations. Since Phantom Omni provides only a 3-DOF force feedback, torque is not considered during the surgery training process. In real laparoscopic surgery, proper cutting torque is required. So we plan to choose more sophisticated haptic devices coupled with torque feedback to enhance the tactile realism for Copyright © 2014 John Wiley & Sons, Ltd.
Appendix: Non-linear Cosserat rod model For convenience, the three directors (d1, d2, d3) are chosen as an orthogonal framework at a given location r (x1, x 2, x3) and the third director d3 is chosen to be along the tangent of the beam centreline. The Darboux vector ω is defined as ∂di ¼ ω di ; i ¼ 1; 2; 3; ∂s
(A1)
which includes the torsion (τ) and the flexures (κ1, κ2) related to the deformation: ω ¼ κ1 d1 þ κ2 d2 þ τd3
(A2)
Int J Med Robotics Comput Assist Surg (2014) DOI: 10.1002/rcs
VR training and assessment in laparoscopic rectal resection surgery
Using the Langrangian approach, we can formulate the motion equations as d ∂T ∂T ∂U ∂D þ þ ′ ¼ ∫Jk fds (A3) ′ dt ∂qk ∂qk ∂qk ∂qk T is the kinetic energy, U is the potential energy, D is the dissipation energy. Jk is the Jacobian matrix: Jk = ∂r/∂qk, and f is the external forces. T¼
1 dr dr ρA ds 2 ∫ dt dt
(A4)
The kinetic energy T is defined by the velocity of the beam’s centreline and its angular part is ignored, where ρ is the mass density and A is the cross-sectional area. U¼
1 ½B1 ðκ1 κ1 Þ2 þ B2 ðκ2 κ2 Þ2 þ GJ ðτ τ Þ2 2∫ þ EAðv3 1Þds
(A5)
The potential energy U is defined as elastic energy induced by bending (the 1st and 2nd terms in the integration), twisting (the 3rd term in the integration) and elongation (the 4th term in the integration). v3 ¼ ∂r ∂s d3 represents the length change along the tangent direction. Values with a bar above correspond to the rest configuration of the beam without any external forces, which can be viewed as residual items. B1=EI1 and B2=EI2 are the bending stiffness of the beam, where E is Young’s modulus, I1 and I2 are the respective area moments of inertia for the cross-Section. G is the shear modulus and J is the polar moment of inertia for the cross-section. " 2 # 2 1 dκ1 2 dκ2 dτ D ¼ ∫ γ1 ds (A6) þ γ2 þ γ3 2 dt dt dt The dissipation energy D is related to the non-conservative forces and γi are the damping coefficients. In Equation A7, qk is the general displacement vector and q′k is the general velocity vector of the kth node. In our discretization, qk are the angular values which describe the orientation change of local frame defined by directors in the local elongation parameter at the kth node position. Equation A7 leads to the equation of motion expressed by the general displacement vector qk: Mq″k þ Cq′k þ Kq k ¼ R
(A7)
q″k ,
where is the acceleration of qk,. M is the general mass matrix, C is the damp matrix and K is the stiffness matrix. R is the residual part which is written as R ¼ Kqk þ
∫Jk f k ds. We solve Equation A7 to update the state of the rectum in interaction. For details of collision handling, readers can refer to (8,18). Copyright © 2014 John Wiley & Sons, Ltd.
References 1. Ali MR, Mowery Y, Kaplan B, DeMaria EJ. Training the novice in laparoscopy. More challenge is better. Surg Endosc 2002; 16: 1732–1736. 2. Aggarwal R, Moorthy K, Darzi A. Laparoscopic skills training and assessment. Br J Surg 2004; 91: 1549–1558. 3. Grantcharov TP. Virtual reality simulation in training and assessment of laparoscopic skills. Eur Clin Obstet Gynaecol 2006; 2: 197–200. 4. Francone T, Weiser MR. Laparoscopic rectal resection for cancer. Seminars Colon Rectal Surg 2005; 16: 147–154. 5. http://www.surgical-science.com/portfolio/lapsim-basic-skills/ 6. http://simbionix.com/simulators/lap-mentor/ 7. http://www.mentice.com/ 8. Pan J, Chang J, Yang X, et al. Graphic and haptic simulation system for virtual laparoscopic rectum surgery. Int J Medic Robot Comput Assist Surg 2011; 7: 304–317. 9. Choi C, Kim J, Han H, et al. Graphic and haptic modeling of the oesophagus for VR based medical simulation. Int J Med Robot Comput Assist Surg 2009; 5: 257–266. 10. Heng PA, Cheng CY, Wong TT, et al. A virtual reality training system for knee arthroscopic surgery. IEEE Trans Inf Technol Biomed 2004; 8: 217–227. 11. Lin MC, Otaduy M. Haptic Rendering: Foundations, Algorithms and Applications. A.K. Peters, Ltd: Natick, Massachusetts, USA, 2008. 12. Govindaraju NK, Knott D, Jain N, et al. Interactive collision detection between deformable models using chromatic decomposition. In ACM SIGGRAPH 2005. New York, USA, 991–999. 13. Chih-Hao H, Basdogan C, Srinivasan MA. Ray-based haptic rendering: force and torque interactions between a line probe and 3D objects in virtual environments. Int J Robot Res 2000; 19: 668–683. 14. Maciel A, De S. An efficient dynamic point algorithm for linebased collision detection in real time virtual environments involving haptics. Comput Animation Virtual Worlds 2008; 19: 151–163. 15. De-Novi G. Soft tissue modeling for virtual reality surgery simulator with haptic feedback. University of Bologna, PhD thesis. 16. Basdogan C, De S. Haptics in minimally invasive surgical simulation and training. IEEE Comput Graph 2004; 24: 56–64. 17. Gurusamy KS, Aggarwal R, Palanivelu L, Davidson BR. Virtual reality training for surgical trainees in laparoscopic surgery. Cochrane Database Syst Rev 2009; 1: 13–21. 18. Chang J, Yang X, Pan J, Li W, Zhang J. A fast hybrid computation model for rectum deformation. Visual Comput 2010; 27: 97–107. 19. http://www.mitk.net/ 20. Bathe KJ. Finite Element Procedures (2nd edn). Prentice-Hall: London, 1996. 21. Wu W, Heng PA. A hybrid condensed finite element model with GPU acceleration for interactive 3D soft tissue cutting. Comput Animat Virtual Worlds 2004; 15: 219–227. 22. Mollemans W, Schutyser F, Cleynenbreugel J, Suetens P. Fast soft tissue deformation with tetrahedral mass spring model for maxillofacial surgery planning systems. In MICCAI, Vol. 3217. Springer-Verlag: Berlin/Heidelberg, 2004; 371–379. 23. Hong M, Jung S, Choi M, Welch SW. Fast volume preservation for a mass–spring system. IEEE Comput Graph Appl 2009; 26: 83–91. 24. Antman SS. Nonlinear Problems of Elasticity, Applied Mathematical Sciences, Vol. 107. Springer Verlag: Heidelberg, Berlin, Germany, 1995. 25. Chang J, Yang X, Zhang JJ. Continuous skeleton-driven skinning - a general approach for modelling skin deformation. Int J Image Graphics 2009; 9: 591–608. 26. Pan J, Yang X, Xie X, et al. Automatic rigging for animation characters with 3D silhouette. Comput Animat Virtual Worlds 2009; 20: 121–131. 27. Atkinson KA. An Introduction to Numerical Analysis (2nd edn). John Wiley & Sons: New York, 1989. 28. Ganovelli F, O’Sullivan C. Animating cuts with on-the-fly remeshing. In Eurographics. John Wiley & Sons Ltd.,: Manchester, UK, 2001; 243–247. Int J Med Robotics Comput Assist Surg (2014) DOI: 10.1002/rcs
J. J. Pan et al. 29. Picinbono G, Lombardo JC, Delingette H, Ayache N. Improving realism of a surgery simulator: linear anisotropic elasticity, complex interactions and force extrapolation. J Visual Comput Animation 2002; 13: 147–167. 30. Cotin S, Delingette H, Ayache N. A hybrid elastic model for realtime cutting, deformations, and force feedback for surgery training and simulation. Visual Comput 2000; 16: 437–452. 31. Chang J, Zhang JJ. Force mapping. Theory and Practice of Computer Graphics Proceedings, Bournemouth, UK, 2004; 204–210. 32. Chuong CJ, Fung YC. On residual stress in arteries. J Biomech Eng 1986; 108: 186–192. 33. Lu X, Gregersen H. Regional distribution of axial strain and circumferential residual strain in the layered rabbit oesophagus. J Biomech 2001; 34: 225–233. 34. Zhao J, Chen X, Yang J, et al. Opening angle and residual strain in a three-layered model of pig oesophagus. J Biomech 2007; 40: 3187–3192.
Copyright © 2014 John Wiley & Sons, Ltd.
35. Alastrué V, Garía A, Peña E, et al. Numerical framework for patient-specific computational modelling of vascular tissue. Int J Numerical Methods Biomed Eng 2010; 26: 35–51. 36. Prime MB. Cross-sectional mapping of residual stresses by measuring the surface contour after a cut. J Eng Mater Technol 2001; 123: 162–168. 37. Hájek P. Fuzzy logic and arithmetical hierarchy. Fuzzy Sets Syst 1995; 3: 359–363.
Supporting information Additional supporting information may be found in the online version if this article at the publisher’s website.
Int J Med Robotics Comput Assist Surg (2014) DOI: 10.1002/rcs
ORIGINAL ARTICLE
Virtual reality training and assessment in laparoscopic rectum surgery
Jun J. Pan1* Jian Chang2 Xiaosong Yang2 Hui Liang5 Jian J. Zhang2 Tahseen Qureshi3 Robert Howell4 Tamas Hickish4
Abstract Background Virtual-reality (VR) based simulation techniques offer an efficient and low cost alternative to conventional surgery training. This article describes a VR training and assessment system in laparoscopic rectum surgery. Methods To give a realistic visual performance of interaction between membrane tissue and surgery tools, a generalized cylinder based collision detection and a multi-layer mass–spring model are presented. A dynamic assessment model is also designed for hierarchy training evaluation.
1
State Key Laboratory of Virtual Reality Technology and Systems, Beihang University, Beijing, China
2
National Centre for Computer Animation, Media School, Bournemouth University, Poole, UK
3
Poole Hospital, Poole, UK
4
The Royal Bournemouth and Christchurch Hospitals, Bournemouth, UK
Results With this simulator, trainees can operate on the virtual rectum with both visual and haptic sensation feedback simultaneously. The system also offers surgeons instructions in real time when improper manipulation happens. The simulator has been tested and evaluated by ten subjects. Conclusions This prototype system has been verified by colorectal surgeons through a pilot study. They believe the visual performance and the tactile feedback are realistic. It exhibits the potential to effectively improve the surgical skills of trainee surgeons and significantly shorten their learning curve. Copyright © 2014 John Wiley & Sons, Ltd.
5
School of Animation, Communication University of China, Beijing, China *Correspondence to: Jun J. Pan, State Key Laboratory of Virtual Reality Technology and Systems, Beihang University, Beijing, China. E-mail: [email protected]
Accepted: 19 January 2014
Copyright © 2014 John Wiley & Sons, Ltd.
Keywords collision detection; dissection simulation; evaluation; laparoscopic rectum surgery; dynamic assessment
Introduction In recent years there has been remarkable development in minimally invasive surgery (MIS) techniques in areas such as laparoscopic and endoscopic surgeries. Compared with traditional open surgery, due to the different hand–eye coordination and the fulcrum effect of the abdominal wall for instrument manipulation (1), the performance of MIS requires a special set of psychomotor and perceptual skills. Therefore, laparoscopic surgery is difficult to learn by observation and practice alone, using conventional apprenticeship practice. A few approaches can be used to speed up the learning of laparoscopic surgeons. One of them is called the training box (2,3). These boxes may contain inanimate objects, which allow mundane tasks to be performed. These training objects are usually made from rubber or silica gel, which understandably lack the realistic feel of handling human tissues. Animal tissue can be used as an
J. J. Pan et al.
alternative, but it is different from the human anatomy structure. Though cadavers are available, they are quite expensive and are not suitable for repetitive training. In reality, residents in many nations, including the UK, develop their laparoscopic surgical skills by directly operating on real patients under supervision from senior surgeons (4). Recently, with the rapid growth of computer power, virtual reality (VR) simulators have become a viable alternative approach, which allows the trainee to hone their skills by operating on virtual patients in a safe and repetitive manner. The aim of this paper is to present a VR-based training and assessment system for laparoscopic rectum surgery. At present, colorectal cancer is the third commonest cancer in the UK, with approximately 40 000 new cases diagnosed annually (4). It is estimated that 90% of the cases are suitable for laparoscopic resection surgery. However, owing to the complicated and technically challenging nature of laparoscopic colorectal cancer surgery, there is a relative lack of trained surgeons to perform such demanding operations. On the other hand, although there are several commercial VR based laparoscopic surgery simulators (LapSim (5), Lap Mentor (6), Mentice (7)) on the market, most of them aim to simulate procedures in cholecystectomy (gallbladder removal), ventral hernia repair, ectopic pregnancy and myomectomy. The application to rectal resection surgery for cancer is rare owing to the narrow pelvis, substantial large soft tissue deformation and complex interactions between surgery tools and soft tissue in operation. Therefore, it brings a greater challenge for the haptic response and realistic graphic computation. Owing to the advantage of MIS, researchers have applied a great deal of effort to virtual reality based surgery simulation. Pan et al. (8) described a simple simulator for virtual laparoscopic rectum surgery. They select triangle meshes rather than the tetrahedra model to represent the bowel. To meet the requirement of real-time graphic performance, a cosserat rod model had been introduced to handle the physical deformation and collision detection. Choi et al. (9) designed a VR training system for laparoscopic oesophagus surgery simulation. They used a boundary element method (BEM) to compute deformation based on the assumption of linear response. The mechanical parameters were collected from experiments on animal tissue and integrated with a haptic device. Heng et al. (10) developed a VR simulation system for knee arthroscopic surgery. In the system, they used virtual models from the database ‘Visible Human’. Finite element analysis (FEA) was also applied to simulate the deformation and incision of soft tissue in real time. Haptic rendering (11–13) is another important research field in VR based surgery simulation. Maciel et al. (14) presented a dynamic point algorithm for line based collision detection in haptics. They simplified the surgical instrument to a Copyright © 2014 John Wiley & Sons, Ltd.
section of line with two end points, and a dynamic point located between the two end points. In this approach, the dynamic point is the closest point on the line to potentially collide with the object. It can compute the interaction between a line-shaped haptic tool and the object model surface efficiently. De Novi (15) presented the Blobby modelling approach which represents organs with a set of spheres. It is able to perform fast collision detection between different organs and between virtual surgical instruments. Basdogan et al. (16) presented a survey on all the important aspects of haptic devices in MIS simulation, including haptic rendering, interface and data recording. Compared with the traditional surgical training method, besides providing repetitive tasks and variable complexity for trainees, VR based laparoscopic surgery simulators can also record the training progress, assess trainees’ surgical skill objectively and offer informative feedback. Gurusamy et al. (17) provided a total of 23 randomized clinical trials to compare virtual reality training with other forms which include no training, video training, conventional laparoscopic training for surgeons without prior laparoscopic surgery experience. 612 participants were involved in these trials. The assessment considered the following factors: (1) Time, such as calculating ‘operating time’, ‘time for one stitch and two ties’, or ‘time for driving a needle through the tissue and in tying a surgical knot’; (2) Error score, such as ‘error score by dropping the object or perforation of object’ or ‘average error incidence by tear of sheet, slack of ligature’; (3) Accuracy, such as ‘mean number of correct incisions’; and (4) Others, such as speed, blood loss, number of movements, economy of movements, distance tracing of instruments. Through experimental investigation, this feedback in training can help trainees to correct surgical mistakes efficiently and make good progress. At present, most commercial medical simulators provide the basic functions of evaluation and consultancy to trainees considering a number of surgical factors, such as time of completion and wall collisions. However, for specific laparoscopic rectum surgery training, few VR based simulators meet the above requirements. In fact, the continued need for comprehensive assessment and instructor’s feedback from a training system is often regarded as a primary reason for the reluctance of many medical schools to fully embrace the virtual training technology (3). For that purpose, we present a VR based training and assessment system for the laparoscopic rectum surgery. First, referencing to the real surgery videos and MRI data, the surface mesh model of a section of rectum is constructed and preprocessed as the training object. Generally, the cursor (proxy) from a haptic device effects to one point during interactions. It can lead to incorrect soft tissue deformation during surgical simulation. To Int J Med Robotics Comput Assist Surg (2014) DOI: 10.1002/rcs
VR training and assessment in laparoscopic rectal resection surgery
offer realistic geometric deformation in interactions, we have designed a generalized cylinder based collision detection algorithm for haptics. Second, to achieve the high deformation accuracy and real time graphic performance, we used the cosserat rod model mentioned in (18) to deal with the physical deformation of the rectum. Third, a multi-layer mass spring model is designed to cope with the deformation and surgery behaviour of membrane tissue surrounding the rectum. And a straightforward incision approach is presented based on this model. It can simulate the dissection procedure to detach the rectum from its surrounding tissue in real time. Finally, a hierarchical evaluation module coupled with a dynamic assessment model has been implemented to evalute and offer trainees constructive feedback in both an intra-operative and post-operative manner.
Materials and methods Operation environment and VR simulator interface In laparoscopic rectal resection surgery for cancer treatment, some preoperative diagnosis and preparation should be made before the operation (4). Endoscopy is used to determine the histological confirmation of the cancer and its location in the rectum. Then a CT scan will be applied to assess whether the tumour has spread to distant organs, such as lungs and liver. Sometimes, the patient will be recommended to have a MRI scan to acquire particular information on the local extent of the tumour. It can help with the formulation of a plan of treatment, in particular, whether the operation should be proceeded by chemoradiotherapy. In the operating theatre, the patient is usually placed in a modified Lloyd-Davis position (4). The number of inserted operating ports varies between surgeons, but typically four operating ports are used. An operation assistant is needed to control the laparoscope and help with the retraction of tissues during operation. The surgeon relies on a variety of tools, such as stapling devices, graspers and L-hooks, to help seal, grasp, dissect and separate tissues. During the whole procedure, the most challenging task and eventual aim of the surgery is to detach the rectum from its surrounding pelvic structure. Then the specimen will be removed from the patient through an extraction site either via a Pfannensteil incision or in the left iliac fossa (4). Figure 1(a) shows the theatre set up for laparoscopic rectal cancer surgery. Figure 1(b) is a screenshot of our simulator interface. It illustrates a section of rectum model and the surrounding soft tissue covered. The simulator hardware (Figure 1(c)) Copyright © 2014 John Wiley & Sons, Ltd.
Figure 1. Comparison of real laparoscopic rectum surgery with VR based simulation system. (a) Real operating environment for laparoscopic rectum surgery. (b) Software interface of VR based simulator. (c) Snapshot of VR based simulator hardware Int J Med Robotics Comput Assist Surg (2014) DOI: 10.1002/rcs
J. J. Pan et al.
consists of a monitor, a computer and two haptic devices (Phantom Omni). Generally, one haptic device functions as a grasper, and the other functions as an L-hook or harmonic scalpel used for dissection. The left haptic device can also be used as a navigation camera in this system. These haptic devices provide 6-DOF navigating parameters and 3-DOF force feedback. The software was developed with OpenGL, C++, Mitk (19) and Qt. The architecture of the system is described in Figure 2. From the clinical aspect, our most important contribution is to present a comprehensive framework of virtual laparoscopic rectum surgery simulation and evaluation, from system design to implementation. Compared with previous research work in VR based medical simulation, there are three innovative technical contributions: 1. A generalized cylinder based collision detection algorithm is presented to offer fast and efficient detection of the interaction between soft tissue and surgery instruments which have a complex shape. 2. A multi-layered mass–spring model is designed to cope with the deformation and surgical behaviour of membrane tissue surrounding the rectum. Correspondingly, a straightforward real-time incision approach is presented based on this model. It can simulate the dissection procedure to detach a section of rectum from its surrounding soft tissue. 3. A dynamic assessment model is developed and a hierarchical evaluation module has been implemented to collect and analyse the evaluation metrics for laparoscopic rectum surgery training.
Generalized cylinder based collision detection for haptics In a real-time VR system, the display rate of graphic rendering is about 30 frames per second. However, in a multimodal virtual environment with haptic interactions, a much higher update rate of about 1 kHz is needed to ensure smooth force feedback and continuous interactions. Theoretically, the force from the haptic device (Phantom Omni) applies to only one contact point during manipulation. The most efficient interaction method is to represent the proxy (haptic cursor) as a point. However, in many cases, the interactions of instruments in surgery simulation are using long slender tools (such as dissect, shear, L-hook and so on) whose contact surface is an area or a line. Detection problems may occur if it is simplified to a single point or a line, especially when local concavities exist in the model (Figure 3). In this simulator, we treat the surgical instrument as a generalized cylinder object (L-hook) or a combination of multiple generalized cylinder objects (dissect, shear), and an innovative generalized cylinder based collision detection algorithm is designed. It can be described briefly as follows. In Figure 4, suppose the two end points of axis (dotted red line) are p0, p1. The radius of the cylinder is variable along the axis, and in a certain area the value of the radius is r. The precondition of this collision detection algorithm is that the surgery instruments should initially be outside the organ objects when the graphic rendering starts. Then in every frame of the graphic rendering loop, each vertex (v) of the organ object will be estimated to determine whether it is inside the cylinder or not. If it is inside the instrument model, a collision will take place. And a penalty force will be placed on the vertex according to its penetration depth. This algorithm can be divided into two steps: Step 1: Calculating the angle: ∠ p0p1v and ∠ p1p0v. If ∠ p1p0v ≤ 90° and ∠ p0p1v ≤ 90°, then go to Step 2. If not, v is outside the cylinder. Step 2: Calculating the Euclidean distance (d) between v and cylinder axis line (p0p1). If d 30). Besides the high computation efficiency, another important advantage of the multi-layer mass– spring model is the simplicity in dissection simulation.
topology of model, even re-mesh the object. It is an extremely time consuming process. In this work, inspired by the mechanism of residual strain in bio-materials (31), we present a straightforward real-time incision technique which can be used to simulate the dissection procedure of detaching the rectum model from its surrounding tissue. This technique is designed based on the multilayered mass–spring model and can produce the realistic shrinking effect of the surrounding tissue when it is detached from the rectum.
The simulation of dissection The critical procedure in laparoscopic rectal cancer surgery is cutting the tissue between the rectum and its surrounding soft tissue, to enable the section of bad intestine (with tumour) to be disconnected from the pelvic structures so that it can be removed. Dissection is therefore the most important task in training. Technically, interactive realistic cutting is one of the most challenging topics in VR based surgery simulation. It has attracted a great deal of research effort. Most cutting algorithms (28–30) based on mesh processing need to change the
Figure 8. Deformation caused by the residual stress is equivalent to the results of a straight cut followed by an equivalent force applied to the cutting surface Copyright © 2014 John Wiley & Sons, Ltd.
Figure 9. Simulation of dissection to separate the rectum and its surrounding soft tissue: (a) before cutting the spring; (b) after cutting the spring Int J Med Robotics Comput Assist Surg (2014) DOI: 10.1002/rcs
VR training and assessment in laparoscopic rectal resection surgery
Residual strain exists in many kinds of bio-materials. It causes complexity for the estimation of deformation when a given bio-structure is subject to external loads. Experiments were designed to investigate the inner strain distribution of the residual stress but often is limited to tubular structures, such as arteries (31,32) and oesophagus (33,34). Analysis by Alastrué et al. (35) showed that the incorporation of residual stress in the computation affects the circumferential stress field of arterial structures as well as the deformed configuration. In our simulation, we want to produce the shrinking effect of connective tissues surrounding the rectum near the pelvis floor when they are separated from the rectum. Such deformation could have been caused by the residual strain. In real surgery, the surgeon has to estimate its effect before and during dissection to reduce the risk of unwanted damage and complication. There is a lack of experimental evidence to provide information for the residual strain distribution within the area of interest. The shrinking effect of the tissue surrounding the rectum in real surgery suggests that the connective tissue may be subject to a pre-stressed tensile force. Experiments on the oesophagus (33,34) suggested that the residual strain can vary from 0 to 0.3. Here we assume a residual tensile strain of 0.1 uniformly distributed in the connective tissue. Prime (36) suggested the contour of the exposed surface due to a cut can be related to adding a counterpart force to the cutting plane according to the superposition principle. Therefore, given the known residual tensile stress, we can identify the deformed shape by applying the equivalent force on the surface (Figure 8). We computed the residual stress σr as σ r ¼ Eεr
(5)
where E is Young’s modulus and εr is the residual strain. Here coupled with equations (3) and (5), we simplify the simulation of dissection as a ‘spring cutting’ procedure, which can be illustrated as in Figure 9. The L-hook
is usually used as the dissection instrument. If the tip of L-hook touches an incorrect place or vulnerable area which should be avoided in training, the system will give a warning signal. Otherwise, when the tip of L-hook touches the area of the second and third layer of this model, it will search the springs between these two layers in a small sphere range (the small circle in dotted line). If a spring is detected, it will be ‘cut off’ (deleted) from this mass–spring model and removed from the dynamic system. Meanwhile, a residual tensile stress is added at the nodes linked by this spring. So with 3 and (5), there is: X X F Pi ðx; v; tÞ ¼ F Pi ; Pj þ f ðPi Þ þ GðPi Þ þ σ r (6) After computation of this updated dynamic system, a shrinking effect of the rectum and its surrounding tissue is performed. Figure 14 gives some experimental results of dissection simulation. Due to the residual tensile stress and the shrinking effect, the rectum and its surrounding tissue are separated clearly after dissection (Figure 14(b)).
Evaluation of training Besides providing realistic surgical simulation in real time, our simulator can also assess the trainee’s surgical skills and offer them constructive feedback during and after the operation. There exist several commercial medical simulators (5, 6, 7) which are able to provide basic functions of evaluation and advice to the trainees. However, for laparoscopic rectum surgery training, few VR based simulators implement the assessment functions. In reality, the continued need for comprehensive assessment and instructor’s feedback from the training system is often regarded as a primary reason for the reluctance of many medical schools to fully adopt VR based training techniques (3). For that purpose,
Table 1. Metrics in hierarchy evaluation module for the simulator First Grade
Second Grade
Surgical Skill
Efficiency
Third Grade
Safety
Completion Time Redundant Fabric Used Exposure
Quality
Wall Collision Times Near Vulnerable Tissue Forces Near Vulnerable Tissue Velocities Drilling & Cutting Technique
Accurate Technique
Copyright © 2014 John Wiley & Sons, Ltd.
Fourth Grade
Directly Exposed Indirectly Exposed Overexposed
Cutting Length Cutting Depth Cutting Region Cutting Angle Jump Frequency Instrument Selection Ratio of Correct Cutting Ratio of Incorrect Cutting
Int J Med Robotics Comput Assist Surg (2014) DOI: 10.1002/rcs
J. J. Pan et al.
Figure 10. Dynamic assessment model
to implement the assessment function for laparoscopic rectum surgery in our simulator, we define a dynamic assessment model and present a hierarchical evaluation module.
simulator, which is authorized to evaluate the trainee’s surgical skills; the evaluation object is the trainee; the main purpose is providing constructive feedback to trainees for skill promotion; metrics represent the aspects of surgical skills and can be composed as a hierarchical structure (Table 1) in our system for comprehensive evaluation; for the method, we apply fuzzy logic (37) to the data recorded, and determine the levels of expertise for each trainee considering the most major evaluation metrics. By recording the training data from virtual operation on the simulator, the evaluation result is provided to trainees after analysis. Meanwhile, during the simulation, the system can also instruct the trainees when incorrect manipulation occurs. For example, the system will trigger an alarm if the trainee grasps or cuts a vulnerable area which should be avoided. Evaluation exists in both intra-operative and post-operative intervention procedures. Based on the descriptions above, we define and build a dynamic assessment model, which is shown in Figure 10. In this Figure, the evaluation results can give feedback to the trainees to improve their training behaviour in real time.
Hierarchical evaluation module Dynamic assessment model In the evaluation process, there are seven essential factors: evaluation subject, evaluation object, purpose, metrics, method, result and time. In our simulator, the evaluation subject is the VR based laparoscopic rectum surgery
The comprehensive evaluation of surgical skills is difficult since it involves the assessment of all aspects of a surgeon’s techniques, such as the ability for correct decision making in response to the discrete events in a surgical scenario, maintaining proper visibility of the surgical field,
Figure 11. Weight setting for evaluation metrics Copyright © 2014 John Wiley & Sons, Ltd.
Int J Med Robotics Comput Assist Surg (2014) DOI: 10.1002/rcs
VR training and assessment in laparoscopic rectal resection surgery
achieving sufficient exposure of critical anatomy structures, removing the optimal volume of tissue, applying appropriate velocities and forces when vulnerable anatomy
structures are approached, and so on. It is not practical to consider all possible factors to assess a trainee. And some factors belong to the domain of psychology, which
Figure 12. Evaluation results on the software interface of the simulator. (a) Recorded training data and score. (b) The distribution of evaluated surgical skill levels for all trainees Copyright © 2014 John Wiley & Sons, Ltd.
Int J Med Robotics Comput Assist Surg (2014) DOI: 10.1002/rcs
J. J. Pan et al.
is hard to measure by a quantitative approach. We thus select only the key evaluation metrics which best fit the given surgery, and assemble these metrics to a form which can reflect the trainee’s skill in a higher level. Compared with some existing simulators (5, 7), we set up a comprehensive evaluation module with a hierarchical structure rather than only providing the basic evaluation metrics. The surgical skill hierarchy evaluation module covers most major training targets with different hierarchy levels. Its structure is illustrated in Table 1. In this table, evaluation metrics are divided into four grades: the first grade indicates the surgical expertise of the trainee. The other three grades are composed of subsets of concrete evaluation metrics. On the second grade, the surgical skills are evaluated by following three categories of metrics: Efficiency (the time spent on training task, and the quantity of redundant fabrics used), Safety (the parameters that measure the safety of operations in virtual training, such as the ability to avoid unwanted collisions between instruments and vulnerable tissues), Quality (the parameters which measure the performed skills of trainees in the task). We also define adjustable weights for each evaluation metric, which are illustrated in Figure 11. After weight setting, the system can compute the total score for the trainees. The personal information and training record for each individual will be listed below the Window ‘Learning Centre’ in the system interface (Figure 12(a)). After all the trainees’ performances are evaluated and recorded, the system will classify the trainees to different levels according to the evaluation scores of their surgical expertise. The level for all the trainees can be illustrated in the system interface (Figure 12(b)).
the system gives a score to each trainee and illustrates statistical data for this trainee group. Our system runs on a workstation with Intel CoreQuad 2.83 GHz, 6 GB RAM and equipped with a NVIDIA Quadro FX 3700 graphics card. The force refreshing frequency is 1000 Hz and the graphic refreshing frequency is 45 Hz. To evaluate the effectiveness of the techniques in this simulator, we have also designed two experiments. The first tests the visual performance of the generalized
Results Implementing the techniques described above, a prototype virtual reality based simulation and assessment system was developed for laparoscopic rectum surgery. When the system starts, the simulator generates a continuous loop with force feedback. The haptic devices send position and orientation information of surgery instruments to the computer. The collisions detection between surgery instruments and organ models are processed according to these data. The soft tissue deformation and the feedback force are computed by the hybrid mechanical model of the rectum and the multi-layered mass– spring model of tissue surrounding the rectum. Then the feedback force signal is sent to the haptic devices for rendering. When the simulator is running, the assessment module records all the evaluation metrics for each trainee. Meanwhile, since the assessment is a dynamic process, it can give the trainee instructions in real time when incorrect manipulation occurs. Finally, after training is finished, Copyright © 2014 John Wiley & Sons, Ltd.
Figure 13. Comparison of visual performance of a L- hook touching the surface of liver model: (a) before collision; (b) collision detection by the generalized cylinder based algorithm; (c) collision detection by a point-based method Int J Med Robotics Comput Assist Surg (2014) DOI: 10.1002/rcs
VR training and assessment in laparoscopic rectal resection surgery
cylinder based collision detection. Figure 13 illustrates the results compared with a standard point-based method. In Figure 13(a), the whole L-hook can touch the surface of the liver and deform the model realistically. However, in Figure 13(c), the handle of the L-hook penetrates in the liver because of the ‘Local Concavity Trap’. The second experiment verifies the dissection simulation which is the most important training task in the surgery. In this test, the trainee uses three instruments (one L-hook and two graspers) to detach a section of rectum from its surrounding soft tissue. From the experimental results illustrated in Figure 14, we found that the deformation is plausible. After dissection, the rectum and its surrounding tissue are clearly separated. Moreover, our simulator also includes special visual effects, such as smoke, for training (Figure 14(f)).
We also invited ten subjects with varying laparoscopic colorectal surgery experiences to test and assess our system. The subjects were asked to perform the dissection procedure (Figure 14) on our simulator, and then to answer a feedback questionnaire consisting of nine questions: (1) realism of the anatomy; (2) realism of the appearance of the simulator interface; (3) realism of the instrument handling; (4) overall realism of the dissection task; (5) quality of force feedback in performing the tasks; (6) usefulness in learning hand–eye coordination; (7) usefulness in learning ambidexterity skills; (8) overall usefulness in learning the fundamental surgical skills; (9) accuracy of the evaluation of performance in training. Each question can be answered on a 5 point Likert scale with 5 for very good and 1 for very poor. The results from the questionnaire study are given in Table 2.
Figure 14. Screenshots of dissection to detach the rectum from its surrounding tissue. (a) Before dissection; (b) 5 s after dissection procedure starts; (c) 60 s after dissection procedure starts; (d) 80 s after dissection procedure starts; (e) 100 s after dissection procedure starts; (f) smoking effect during dissection simulation Copyright © 2014 John Wiley & Sons, Ltd.
Int J Med Robotics Comput Assist Surg (2014) DOI: 10.1002/rcs
J. J. Pan et al. Table 2. Descriptive statistics obtained from the questionnaire study
Mean SD
1
2
3
4
5
6
7
8
9
3.2 0.86
3.6 0.92
3.5 0.81
3.7 1.07
3.9 0.72
3.9 0.73
3.7 0.92
4.1 0.87
3.5 0.91
Discussion The first VR based surgery simulator for rectum cancer has been designed. Throughout the whole development process for this simulator, three senior surgeons from the Royal Bournemouth and Christchurch Hospital, Poole Hospital (NHS, UK) have been closely involved. They advise on the medical content and assist in evaluating the training results as both consultants and trainees. Satisfactory tactile feedback is achieved based on their operational experience. Through haptic rendering, users can see a realistic internal surgery environment with visual outputs and tactile feeling in real time. In the evaluation process, the result from the preliminary study (Table 2) shows the average scores for all the questions. It also indicates which aspects need to be improved for more realistic and useful simulation. In Table 2, the lowest scores (3.2) are for ‘the realism of the anatomy’. Due to the computation complexity, we are currently modifying the original rough rectum model to a clean and smooth one. In future, we can apply GPGPU programming (e.g. CUDA) to enhance the graphic computation power for our system. Then we can choose a more complex rectum model for surgery simulation.
dissection simulation. We also plan to add GPGPU programming to enhance the graphic computation speed further for our system. Then we can choose more complex rectum models for surgical simulation. Furthermore, currently we have only implemented the function of training assessment. In future, more trials and study will be made in the assessment of laparoscopic rectum surgery training through our system. With the data collected and analyzed, this VR based training system will be compared quantitatively with other approaches, such as physically based training and standard laparoscopic training.
Acknowledgements The research in this article was supported by a grant from the Royal Bournemouth and Christchurch Hospitals NHS Trust and a HEIF grant from Bournemouth University.
Conflict of interest We declare that we have no financial and personal relationships with other people or organizations that can inappropriately influence our work, there is no professional or other personal interest of any nature or kind in any product, service and/or company that could be construed as influencing the position presented in, or the review of, the manuscript entitled
Funding No specific funding.
Conclusion and future work We have developed a prototype virtual reality based training and assessment system for laparoscopic rectum surgery. Several technical innovations have been presented, including generalized cylinder based collision detection, a multi-layered mass-spring model for computing the deformation and rectum dissection, and a dynamic model for hierarchical evaluation. Real-time visual performance is achieved. The surgeon collaborators are satisfied with the realistic visual quality and tactile feedback of the simulator. Despite successful implementation of the prototype, this simulator also has limitations. Since Phantom Omni provides only a 3-DOF force feedback, torque is not considered during the surgery training process. In real laparoscopic surgery, proper cutting torque is required. So we plan to choose more sophisticated haptic devices coupled with torque feedback to enhance the tactile realism for Copyright © 2014 John Wiley & Sons, Ltd.
Appendix: Non-linear Cosserat rod model For convenience, the three directors (d1, d2, d3) are chosen as an orthogonal framework at a given location r (x1, x 2, x3) and the third director d3 is chosen to be along the tangent of the beam centreline. The Darboux vector ω is defined as ∂di ¼ ω di ; i ¼ 1; 2; 3; ∂s
(A1)
which includes the torsion (τ) and the flexures (κ1, κ2) related to the deformation: ω ¼ κ1 d1 þ κ2 d2 þ τd3
(A2)
Int J Med Robotics Comput Assist Surg (2014) DOI: 10.1002/rcs
VR training and assessment in laparoscopic rectal resection surgery
Using the Langrangian approach, we can formulate the motion equations as d ∂T ∂T ∂U ∂D þ þ ′ ¼ ∫Jk fds (A3) ′ dt ∂qk ∂qk ∂qk ∂qk T is the kinetic energy, U is the potential energy, D is the dissipation energy. Jk is the Jacobian matrix: Jk = ∂r/∂qk, and f is the external forces. T¼
1 dr dr ρA ds 2 ∫ dt dt
(A4)
The kinetic energy T is defined by the velocity of the beam’s centreline and its angular part is ignored, where ρ is the mass density and A is the cross-sectional area. U¼
1 ½B1 ðκ1 κ1 Þ2 þ B2 ðκ2 κ2 Þ2 þ GJ ðτ τ Þ2 2∫ þ EAðv3 1Þds
(A5)
The potential energy U is defined as elastic energy induced by bending (the 1st and 2nd terms in the integration), twisting (the 3rd term in the integration) and elongation (the 4th term in the integration). v3 ¼ ∂r ∂s d3 represents the length change along the tangent direction. Values with a bar above correspond to the rest configuration of the beam without any external forces, which can be viewed as residual items. B1=EI1 and B2=EI2 are the bending stiffness of the beam, where E is Young’s modulus, I1 and I2 are the respective area moments of inertia for the cross-Section. G is the shear modulus and J is the polar moment of inertia for the cross-section. " 2 # 2 1 dκ1 2 dκ2 dτ D ¼ ∫ γ1 ds (A6) þ γ2 þ γ3 2 dt dt dt The dissipation energy D is related to the non-conservative forces and γi are the damping coefficients. In Equation A7, qk is the general displacement vector and q′k is the general velocity vector of the kth node. In our discretization, qk are the angular values which describe the orientation change of local frame defined by directors in the local elongation parameter at the kth node position. Equation A7 leads to the equation of motion expressed by the general displacement vector qk: Mq″k þ Cq′k þ Kq k ¼ R
(A7)
q″k ,
where is the acceleration of qk,. M is the general mass matrix, C is the damp matrix and K is the stiffness matrix. R is the residual part which is written as R ¼ Kqk þ
∫Jk f k ds. We solve Equation A7 to update the state of the rectum in interaction. For details of collision handling, readers can refer to (8,18). Copyright © 2014 John Wiley & Sons, Ltd.
References 1. Ali MR, Mowery Y, Kaplan B, DeMaria EJ. Training the novice in laparoscopy. More challenge is better. Surg Endosc 2002; 16: 1732–1736. 2. Aggarwal R, Moorthy K, Darzi A. Laparoscopic skills training and assessment. Br J Surg 2004; 91: 1549–1558. 3. Grantcharov TP. Virtual reality simulation in training and assessment of laparoscopic skills. Eur Clin Obstet Gynaecol 2006; 2: 197–200. 4. Francone T, Weiser MR. Laparoscopic rectal resection for cancer. Seminars Colon Rectal Surg 2005; 16: 147–154. 5. http://www.surgical-science.com/portfolio/lapsim-basic-skills/ 6. http://simbionix.com/simulators/lap-mentor/ 7. http://www.mentice.com/ 8. Pan J, Chang J, Yang X, et al. Graphic and haptic simulation system for virtual laparoscopic rectum surgery. Int J Medic Robot Comput Assist Surg 2011; 7: 304–317. 9. Choi C, Kim J, Han H, et al. Graphic and haptic modeling of the oesophagus for VR based medical simulation. Int J Med Robot Comput Assist Surg 2009; 5: 257–266. 10. Heng PA, Cheng CY, Wong TT, et al. A virtual reality training system for knee arthroscopic surgery. IEEE Trans Inf Technol Biomed 2004; 8: 217–227. 11. Lin MC, Otaduy M. Haptic Rendering: Foundations, Algorithms and Applications. A.K. Peters, Ltd: Natick, Massachusetts, USA, 2008. 12. Govindaraju NK, Knott D, Jain N, et al. Interactive collision detection between deformable models using chromatic decomposition. In ACM SIGGRAPH 2005. New York, USA, 991–999. 13. Chih-Hao H, Basdogan C, Srinivasan MA. Ray-based haptic rendering: force and torque interactions between a line probe and 3D objects in virtual environments. Int J Robot Res 2000; 19: 668–683. 14. Maciel A, De S. An efficient dynamic point algorithm for linebased collision detection in real time virtual environments involving haptics. Comput Animation Virtual Worlds 2008; 19: 151–163. 15. De-Novi G. Soft tissue modeling for virtual reality surgery simulator with haptic feedback. University of Bologna, PhD thesis. 16. Basdogan C, De S. Haptics in minimally invasive surgical simulation and training. IEEE Comput Graph 2004; 24: 56–64. 17. Gurusamy KS, Aggarwal R, Palanivelu L, Davidson BR. Virtual reality training for surgical trainees in laparoscopic surgery. Cochrane Database Syst Rev 2009; 1: 13–21. 18. Chang J, Yang X, Pan J, Li W, Zhang J. A fast hybrid computation model for rectum deformation. Visual Comput 2010; 27: 97–107. 19. http://www.mitk.net/ 20. Bathe KJ. Finite Element Procedures (2nd edn). Prentice-Hall: London, 1996. 21. Wu W, Heng PA. A hybrid condensed finite element model with GPU acceleration for interactive 3D soft tissue cutting. Comput Animat Virtual Worlds 2004; 15: 219–227. 22. Mollemans W, Schutyser F, Cleynenbreugel J, Suetens P. Fast soft tissue deformation with tetrahedral mass spring model for maxillofacial surgery planning systems. In MICCAI, Vol. 3217. Springer-Verlag: Berlin/Heidelberg, 2004; 371–379. 23. Hong M, Jung S, Choi M, Welch SW. Fast volume preservation for a mass–spring system. IEEE Comput Graph Appl 2009; 26: 83–91. 24. Antman SS. Nonlinear Problems of Elasticity, Applied Mathematical Sciences, Vol. 107. Springer Verlag: Heidelberg, Berlin, Germany, 1995. 25. Chang J, Yang X, Zhang JJ. Continuous skeleton-driven skinning - a general approach for modelling skin deformation. Int J Image Graphics 2009; 9: 591–608. 26. Pan J, Yang X, Xie X, et al. Automatic rigging for animation characters with 3D silhouette. Comput Animat Virtual Worlds 2009; 20: 121–131. 27. Atkinson KA. An Introduction to Numerical Analysis (2nd edn). John Wiley & Sons: New York, 1989. 28. Ganovelli F, O’Sullivan C. Animating cuts with on-the-fly remeshing. In Eurographics. John Wiley & Sons Ltd.,: Manchester, UK, 2001; 243–247. Int J Med Robotics Comput Assist Surg (2014) DOI: 10.1002/rcs
J. J. Pan et al. 29. Picinbono G, Lombardo JC, Delingette H, Ayache N. Improving realism of a surgery simulator: linear anisotropic elasticity, complex interactions and force extrapolation. J Visual Comput Animation 2002; 13: 147–167. 30. Cotin S, Delingette H, Ayache N. A hybrid elastic model for realtime cutting, deformations, and force feedback for surgery training and simulation. Visual Comput 2000; 16: 437–452. 31. Chang J, Zhang JJ. Force mapping. Theory and Practice of Computer Graphics Proceedings, Bournemouth, UK, 2004; 204–210. 32. Chuong CJ, Fung YC. On residual stress in arteries. J Biomech Eng 1986; 108: 186–192. 33. Lu X, Gregersen H. Regional distribution of axial strain and circumferential residual strain in the layered rabbit oesophagus. J Biomech 2001; 34: 225–233. 34. Zhao J, Chen X, Yang J, et al. Opening angle and residual strain in a three-layered model of pig oesophagus. J Biomech 2007; 40: 3187–3192.
Copyright © 2014 John Wiley & Sons, Ltd.
35. Alastrué V, Garía A, Peña E, et al. Numerical framework for patient-specific computational modelling of vascular tissue. Int J Numerical Methods Biomed Eng 2010; 26: 35–51. 36. Prime MB. Cross-sectional mapping of residual stresses by measuring the surface contour after a cut. J Eng Mater Technol 2001; 123: 162–168. 37. Hájek P. Fuzzy logic and arithmetical hierarchy. Fuzzy Sets Syst 1995; 3: 359–363.
Supporting information Additional supporting information may be found in the online version if this article at the publisher’s website.
Int J Med Robotics Comput Assist Surg (2014) DOI: 10.1002/rcs
