Surgical Education
Current status and effectiveness of mentorship programmes in urology: a systematic review Daniel Hay*†, Mohammed Shamim Khan*†, Hendrik Van Poppel‡, Ben Van Cleynenbreugel‡, James Peabody§, Khurshid Guru¶, Ben Challacombe*†, Prokar Dasgupta*† and Kamran Ahmed*† *MRC Centre for Transplantation, King’s College London, †King’s Health Partners,Department of Urology, Guy’s Hospital, London, UK, ‡Department of Urology, University Hospital Gasthuisberg, Katholieke Universiteit Leuven, Leuven, Belgium, §Vattikuti Urology Institute, Henry Ford Hospital, Detroit, MI, and ¶Department of Urology, Roswell Park Cancer Institute, Buffalo, NY, USA
The objectives of this review were to identify and evaluate the efficacy of mentorship programmes for minimally invasive procedures in urology and give recommendations on how to improve mentorship. A systematic literature search of the PubMed/Medline databases was carried out according to the Preferred Reporting Items for Systematic Reviews and Meta-Analyses (PRISMA) guidelines. In all, 21 articles were included in the review and divided into four categories: fellowships, mini-fellowships, mentored skills courses and novel mentorship programmes. Various structures of mentorship programme were identified and in general, mentorship programmes were found to be feasible, having content validity and educational impact.
Introduction A mentorship programme is an educational system that teaches a set of practical skills or allows an individual to learn how to perform a specific procedure. There is considerable variation in the exact structures of current programmes, but mentorship as a standalone concept is the process of an expert individual advising and guiding a junior colleague and it is commonly accepted to be a crucial component in the education of a surgeon. Toledo-Pereyra [1] surmised that mentorship created the foundation for all surgeons and offered benefits other than just technical skills, such as providing a model for professional attitudes, standards of patient care and exemplary practice in research. Presently, there are three broad groups of mentorship programme. Firstly, there are fellowships, which generally last for a period ranging from 6 to 24 months and usually occur after the completion of specialty training. In a fellowship, surgeons focus on one aspect of their chosen specialty to gain a better understanding of the various techniques and procedures that are a part of that discipline. Fellowships are then sub-categorised into modular and non-modular; © 2014 The Authors BJU International © 2014 BJU International | doi:10.1111/bju.12713 Published by John Wiley & Sons Ltd. www.bjui.org
Perioperative data showed equally good outcomes when comparing trainees and specialists. Mentorship programmes are effective and represent one of the best current methods of training in urology. However, participation in such programmes is not widespread. The structure of mentorship programmes is highly variable, with no clearly defined ‘best approach’ for postgraduate training. This review offers recommendations as to how this ‘best approach’ can be established.
Keywords education, mentors, outcome assessment (health care), patient safety, surgery
modular training involves dividing a procedure into a series of steps according to technical difficulty, with the trainee only being allowed to proceed to the next step when the previous one is performed successfully, whereas non-modular fellowships take a less regimented approach. Similarly to fellowships, mini-fellowships (MFs) are designed to provide surgeons with sufficient confidence to undertake a newly learned procedure, but they aim to do so without the extended time commitments of a full fellowship. Typically lasting 1 week, they offer an insight into a technique or specific approach, but are not as comprehensive as full fellowships. Lastly, there are mentored skills courses that are shorter still and tend to focus on an introduction to a specific procedure, rather than a broad range of clinical exposure. This review aims to explore the different types of mentorship programmes available for surgeons in training and has the following aims: 1. To identify mentorship programmes for minimally invasive urological surgery encompassing laparoscopy, robotics and endourology.
BJU Int 2015; 116: 487–494 wileyonlinelibrary.com
Hay et al.
2. To evaluate the relative strengths and weaknesses of each programme identified. This involves assessing the content validity, feasibility, educational impact and cost effectiveness of each programme. 3. To make recommendations on how best to implement and improve mentorship programmes.
Materials and Methods This review has been carried out following the Preferred Reporting Items for Systematic Reviews and Meta-Analyses (PRISMA) guidelines [2]. The eligibility criteria for a paper to be included in this review were that the full paper had to be available in English, be published in the last 10 years (from 2002 onwards) and consider training specifically in urology. Information Sources, Search and Data Items To find relevant papers we carried out a literature review by means of a free text search of the PubMed and Cochrane Library databases. The search terms used were ‘mentorship programmes/programs’, ‘mentorship surgery’, ‘mentorship urology’, ‘urology fellowship’, ‘urology mentoring’, ‘urology training’, ‘urology proctoring’ and ‘urological mentored surgery’. The reference lists of papers that had been selected for inclusion in this review were also used to identify other relevant studies. Study Selection We included studies for review if they made reference to minimally invasive surgical (MIS) training (both laparoscopic and robotic), specific mentorship programmes and the
teaching of MIS skills. We excluded papers that were bulletins, reviews and editorial letters. The key information extracted included format, characteristics and results of each programme. Outcome data recorded included blood loss, operation time, rate of complications, and ‘take’ rate (a measure of how many surgeons are still practicing surgery). Our initial search yielded 256 results. Studies were then excluded if they did not meet the above criteria, leaving a total of 64 abstracts, which were reviewed, and of these, 28 were read in full with 17 included in this review and an additional four studies were identified by examining the reference lists of included papers (Fig. 1). Selected papers were then divided into the common forms of mentorship programme (as discussed above) with the addition of a ‘novel mentorship programme’ category. This was used for papers that did not fit into the other three and used an unconventional approach, e.g. not having a mentor physically present, or tele-mentoring. Evaluation of Study Quality Each of the studies included in this review were evaluated for their effectiveness. Our evaluation centred on four general components that are essential to the success of any teaching programme as outlined by Ahmed et al. [3]: 1. Content validity; the extent to which the content domain is being taught by the programme; does the programme ensure that the skills required to operate independently are being taught? Fig. 1 Process of article selection.
256 papers identified from literature search of Medline and the Cochrane Library
64 abstracts reviewed
192 papers excluded based on title, usually due to a lack of relevance
28 papers evaluated in full
36 papers excluded, most often for describing mentorship in a specialty other than urology
17 papers selected
21 papers included in the final analysis for evaluation
488
© 2014 The Authors BJU International © 2014 BJU International
11 papers excluded for describing mentorship, but without quantitative or qualitative results
4 papers identified through reference lists and included in the final review
Status and effectiveness of mentorship programmes in urology
2. Feasibility/acceptability; is the programme compatible with the demands of current training? 3. Educational impact; is there any improvement following the programme? 4. Cost effectiveness; do the benefits of the programme justify the cost? The most common method of analysis used in studies was comparing operative data from the mentee to those of their mentor or general targets, such as those set by the BAUS. Some studies also included take rates, which assessed the percentage of mentees that were still performing the skills taught at various times after the programme. Where available, quantitative data from each study has been included in this review.
Results Fellowships The most common structure seen in this review was that of a typical fellowship, with 10 papers analysing such a programme. Fellowships are currently available that teach laparoscopic nephrectomy, laparoscopic prostatectomy, robotassisted laparoscopic prostatectomy (RALP), and laser enucleation of the prostate. In half of the programmes a modular training system was used [4–8], whilst in the others, the procedure was not explicitly split up [9–13]. Results from all the fellowship programmes were positive and comparable (Table 2), suggesting that the distinction between step-by-step learning and a less regimented approach may be of little
importance. Trainees may benefit more from one training system than another, but this may possibly be based on personal learning style and preference. MF Kolla et al. [14] describe a 5-day programme, where questionnaires at 3 years follow-up revealed that the overall take rate was 70% (Table 2). The success of MFs is further shown by Shalhav et al. [15], who conclude that MFs allow trainees to progress along their learning curve quickly and be able to operate independently and safely. Gamboa et al. [16] describe a robotic MF with questionnaires from 1, 2 and 3 years after the programme showing that a large proportion of participants are still performing RALP (Tables 1 and 2). Interestingly, data also shows that participants who undertook the course with a partner (another surgeon from the same institution) had a non-significantly higher take rate than those who did not. The same questionnaire had been previously sent out and analysed by McDougall et al. [17] showing that the take rate at a mean follow-up of 7.2 months was 95%. Mentored Skills Courses Data from the hand-assisted laparoscopic nephrectomy course at Duke University Medical Centre [18] shows that 93% of trainees with a mentor were still performing laparoscopic procedures at 6 months follow-up compared with 44% of trainees without a mentor. Rather than simply looking at take rates for laparoscopy, Patel et al. [19] question whether
Table 1 Evaluation of the quality of the mentorship programmes included in this review. Reference
Stewart et al. [4] Stolzenburg et al. [5] Rashid et al. [6] Badani et al. [7] Mirheydar et al. [8] Rane [9] Zorn et al. [10] El-Hakim and Elhilali [11] Netsch et al. [12] Fabrizio et al. [13] Kolla et al. [14] Shalhav et al. [15] Gamboa et al. [16] and McDougall et al. [17] Marguet et al. [18] Patel et al. [19] Nakada et al. [20] Jones et al. [21] Kwon et al. [22] Challacombe et al. [23] Bruschi et al. [24]
Structure
Feasibility (a)
Educational impact (b)
Fellowship Fellowship Fellowship Fellowship Fellowship Fellowship Fellowship Fellowship Fellowship Fellowship MF MF MF Mentored skills course Mentored skills course Mentored skills course Novel mentorship programme Novel mentorship programme Novel mentorship programme Novel mentorship programme
Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes
Yes – OD Yes – OD Yes – OD Yes – OD Yes – IP Yes – IP Yes – OD Yes – OD Yes – OD Yes – OD Yes – EP Yes – IP Yes – EP Yes – IP Yes – EP Yes – OD Yes – OD Yes – OD Yes – OD Yes – OD
Cost effective (c)
Yes
No No No
Yes
Content validity (d) Yes Yes Yes Yes Yes Yes No Yes Yes Yes Yes Yes Yes Yes Yes No No Yes Yes Yes
(a) Whether both trainees and mentors feel that the programme is compatible with the demands of current training and clinical practice. (b) Whether there is any identifiable improvement in skills after the programme. (c) Whether the benefits of the programme justify its cost. (d) Whether the programme ensures that the skills required to operate independently are being taught. OD, operative data; IP, independent practice; EP, expanded practice.
© 2014 The Authors BJU International © 2014 BJU International
489
Hay et al.
Table 2 Results of the mentorship programmes included in this review. A, operative data for trainee and mentor from the laparoscopic fellowships included in this review Reference
Stewart et al. [4] Stolzenburg et al. [5] Fabrizio et al. [13]
Procedure performed
Laparoscopic nephrectomy Endoscopic extraperitoneal radical prostatectomy Laparoscopic radical prostatectomy
Mean operative time, min
Mean estimated blood loss, mL
Mean postoperative stay, days
Transfusion rate, %
Trainees
Mentor
Trainees
Mentor
Trainees
Mentor
Trainees
Mentor
140 174.5
108 172
30 –
20 –
4 –
5 –
3.8 2
4.2 0
313
248
250
150
3
3
–
–
B, take rates of trainees at specific points after the mentorship programme has finished Reference
Kolla et al. [14] Gamboa et al. [16] Marguet et al. [18]
Take rate 6 months without mentor, %
6 months with mentor, %
1 years, %
2 years, %
3 years, %
– – 44
– – 93
72 78 –
71 78 –
71 86 –
C, operative data comparing first and second independent year of practice in mutual mentoring Reference
Jones et al. [21]
Procedure performed
Laparoscopic radical nephrectomy Laparoscopic simple nephrectomy Laparoscopic pyeloplasty
trainees have expanded their laparoscopy practice in terms of case volume and complexity after the Urological Association Mentored Laparoscopy Course. At a mean follow-up of 34.5 months, 77% of participants had done so. Nakada et al. [20] note the success of a standard laparoscopic skills course that involved mentoring via videotape analysis of participants. They found that the expert mentor’s analysis of each trainee’s footage led to statistically significant improvements in various laparoscopic skills.
Mean operative time, min
Mean estimated blood loss, mL
Year 1
Year 2
Year 1
Year 2
246 208 238
231 200 198
160 220 39
314 153 42
independent cases show that the overall operative and warm ischaemia times are highly comparable to standard reported times [23]. Bruschi et al. report similar results, with operative time and blood loss being highly satisfactory and comparing well with averages seen in large laparoscopic adrenalectomy reports.
Discussion Comparison of Mentorship Programmes
Novel Mentorship Programmes Jones et al. [21] discuss their ‘mutual mentoring’ experience following a laparoscopic fellowship, which involved two recently qualified consultants acting as the mentor for one another. In this study, the surgeons operated as the assistant for the other instead of having a junior as the assistant, which allowed them to offer advice and support and bypass the need for a conventional senior mentor figure. Audit data shows that there was a non-significant improvement in results between their first and second independent years (Table 2). Kwon et al. [22] introduced a RALP programme with a similar mutual mentoring system and operative data shows that the programme has succeeded thus far. An interesting potential field for future mentorship programmes is that of tele-mentoring, which Challacombe et al. [23] and Bruschi et al. [24] both evaluated. Operative data from the
490
© 2014 The Authors BJU International © 2014 BJU International
The most obvious difference between the different programmes is the time each takes, but rather than simply being an advantage or disadvantage, this is more a reflection of the level of trainee skill at the start, and intended outcome at the end of each programme. The most common mentorship programme seen in this review was the fellowship, and whilst it is hard to compare the different types of programme, the fellowship is believed to be the ‘gold standard’ of mentorship in terms of achieving proficiency and maintaining patient safety. One of the advantages of a fellowship is that it lasts longer than other programmes, so mentees can immerse themselves in a large number of cases. Consequently, mentees will have more experience and should be able to respond well to complications and unforeseen circumstances. More importantly, the extended training period allows surgeons to appreciate the specific nuances of
Status and effectiveness of mentorship programmes in urology
advanced techniques, such as coordinating the actions of the surgical team with robotics, something that is rarely possible in shorter programmes. Another positive is that due to the 1:1 mentor to mentee ratio, there is often personalised and flexible teaching that can focus on specific difficulties, a benefit that is known to be of upmost importance in surgical training [25]. The obvious disadvantage of a fellowship is the time and effort associated with such training. Fellowships taken before CCT (Certificates of Completion of Training) add to what is already a long training pathway and it is difficult for a consultant to arrange the necessary leave to undertake a full fellowship. Moreover, unless fellowships are undertaken in a centre of excellence or a hospital with a large volume of cases there may be prolonged intervals between cases, further extending the length of the fellowship. All MFs in this review reported a high percentage of mentees continuing to practice the skills taught to them, as well as increasing the number and complexity of cases. The advantages of MFs revolve mainly around the high intensity of the course, which allows substantial progress to be made in a short space of time. A second virtue of MF is that mentees can potentially be paired during and after the course with another surgeon from a nearby hospital, which has been shown to improve take rates [16]. One considerable disadvantage is the total cost of the programme. Kolla et al. [14] noted that the cost of a single course is around $3500 and that attendance fell significantly when the cost was no longer fully subsidised. There is also the question of skill advancement. Bollens et al. [26] theorised that after a certain number of cases, it is important to confer with an expert or visit a centre of excellence to further improve one’s technique. Because of the short nature of the MF, there is no real facility for this and mentees who operate in an institution without experienced surgeons may struggle to progress beyond the initial slope of the learning curve. The clearest advantages of mentored skills courses is their duration and cost; it is much easier to commit to a 2-day course than the months required for a full fellowship and there is not quite such a considerable cost as seen in the MFs. Patel et al. [19] documented another interesting value of skills courses, where a laparoscopic skills course helped participants progress to robotic surgery. This opens the door to the possibility of crossmodality training, where trainees can become proficient in multiple surgical modalities (i.e. laparoscopic and robotic) from a single course. Colegrove et al. [27] found that following laparoscopic skills courses, only 53% of participants were still performing laparoscopy. One possible reason for this is that participants gain confidence in the basic principles whilst on the course, but are unable to carry this over into clinical practice in the weeks and months that follow. This is a particularly noticeable disadvantage for surgeons from a lowvolume centre, where there is little opportunity to make use of learned skills, further increasing the likelihood of stopping.
The importance of training with a partner was discussed in the context of the MF programmes, but its use as a primary method of instruction was clearly shown in the implementation of a robotic surgery programme at Kaiser Permanente Los Angeles Medical Centre [22]. Such triumphs suggest that urology departments could introduce new technologies, e.g. robotics, via mutual mentoring, but it is important to remember the importance of an expert mentor, as noted in the BAUS mentorship guidelines [28]. Regular consultation with an expert at a centre of excellence could further improve the usefulness of such programmes, as could the application of tele-mentoring to aid in such consultations. Tele-mentoring allows for the dissemination of an expert’s knowledge across countries and continents, without the expert having to leave their office. If put to optimal use, tele-mentoring could revolutionise the teaching of robotic surgery and allow urology departments to safely offer a whole new range of procedures. Evaluating the Quality of Programmes Whilst all programmes included in this review show clear educational impact (Table 1), it is not clear however, the extent and specific way in which this benefit is seen. Most commonly, the authors have collected operative data to analyse either the learning curve of their mentees or to directly compare mentee and mentor performance. Comparisons made between the mentee and the available guidelines for a procedure, such as those outlined by the BAUS, can give an absolute indicator of whether they have reached a satisfactory and safe standard. One tool used to evaluate each programme was an analysis of its content validity. Beck and Gable [29] surmised that this could be done in an a priori fashion, where development involves careful consideration of existing literature or in an a posteriori fashion, with a panel of experts determining validity. As nearly all of the programme developers were experts, each programme was designed with validity in mind. The exceptions to this were the mutual mentoring programme established by Jones et al. [21] and the skills course described by Nakada et al. [20], which did not have validity for the following reasons. Firstly, mutual mentoring does not involve an expert mentor and thus there is no expert evaluation of the programme design and structure. Secondly, the programme described by Nakada et al. involves only laparoscopic simulation, which does not equate to a surgeon performing actual laparoscopy independently. Although not included in the evaluation of each programme, patient safety is arguably one of the most important assessment tools. In part, its exclusion was due to the assumption that a programme would only exist if it did not compromise the quality of care that patients received and this was generally the case. Across all programmes there was a low rate of complications and an even smaller number of © 2014 The Authors BJU International © 2014 BJU International
491
Hay et al.
cases that required conversion to open surgery. Fabrizio et al. [13] and Kwon et al. [22] noted rates of just 5%, with one of 20 and five of 100 conversions respectively, for laparoscopic and robot-assisted prostatectomy. Shalhav et al. [15] reported two conversions and three major complications in 30 laparoscopic procedures and Stewart et al. [4] claimed three conversions and two major complications in 105 laparoscopic nephrectomies, which equated to better figures than for the mentor (although this was in part attributed to selection bias with the mentor taking on more complex cases). Stolzenburg et al. [5] described four complications in the mentee’s first 50 laparoscopic prostatectomies and seven complications in the following 100. Numerous programmes reported zero complications or conversions and in nearly all programmes, complication rates were comparable to those of the mentor with two exceptions. The Jones et al. [21] unusual mutual mentoring programme had a conversion rate of 17% (11/66) in the first year of independent laparoscopic practice, although this fell to 4% (three of 84) in the second year, suggesting that the mentees were still moving along the learning curve to proficiency when they began operating independently. One must question whether it is acceptable to have a higher initial rate of conversions if it leads to a considerable improvement the following year. Furthermore, Zorn et al. [10] recorded four conversions and 10 complications in the first 30 RALP cases. There were no conversions in the following 60, but there were another 13 complications. Again, one has to ask whether these figures for adverse events are acceptable, despite having good positive surgical margin rates. Their marginally poorer results are reflected in the fact that both the Zorn et al. and Jones et al. programmes do not have content validity. As discussed earlier, this is due to the lack of an expert robotic/ laparoscopic mentor figure, with the programmes relying on an expert open surgeon and mutual proficiency respectively. Gamboa et al. [16] utilise inanimate and cadaveric simulation in their programme to teach the initial surgical skills needed. Similarly, Rane [9] encourages all mentees to make extensive use of simulators before attending their fellowship to gain the most from it. Rashid et al. [6] goes as far as suggesting that simulation will supersede mentorship, although this would require unequivocal evidence showing how skills learned in simulation equate to ability in theatre. Until this happens, simulation can only act as an adjunct and not a replacement for mentorship, but it is clear that simulation will form an increasingly important part of surgical training in the future.
Barriers to Mentorship Programmes It is important to remember that there are numerous barriers to mentorship programmes. Firstly, there is the issue of operating privileges and malpractice insurance for the visiting surgeon. There is also a question about which surgeon (mentor or
492
© 2014 The Authors BJU International © 2014 BJU International
mentee) should be responsible for any complications that arise during the procedure. Even though the mentor is expected to be the ‘lead surgeon’, it is likely that the mentee will be performing a large proportion of the operation and as such, should legal culpability reside with them? Secondly, special considerations must be made about the process of consenting patients, who should be informed that their procedure is being carried out by a trainee surgeon under guidance. Thirdly, many surgeons would be unable or unwilling to take off the necessary time for a full fellowship reducing their uptake. Birch et al. [30] and Hedican and Nakada [31] conclude that financial compensation, time constraints and acquiring temporary operating privileges were all major obstacles, with Birch et al. also noting that a lack of simple initial cases in laparoscopic and robotic urology was a significant obstacle. Abboudi et al. [32] suggested that the easiest way to overcome many of these issues lies in tele-mentoring and this has been shown to be a viable option for mentorship [23,24]. Alternatively, the path to surmounting these problems may involve the action of organisations, such as the BAUS and AUA. These professional bodies could take on a leadership role in the creation of mentorship programmes, as well as highlighting the importance of such programmes to improve uptake and allay concerns over ‘taking time out’. Hopefully, this would make programmes more easily available and improve participation.
Limitations The main limitations of the present review are due to deficiencies in the available data, particularly for comparing mentorship programmes. As most programmes had different aims and endpoints in terms of data collection, it was impossible to perform a meaningful quantitative analysis. Moreover, operative data alone cannot give a true picture of the calibre of a surgeon, as it is how a patient is affected by the surgery that is truly important. Functional outcomes (e.g. continence and potency after prostatectomy) could give a very clear picture of a surgeon’s level of ability, particularly as these figures have been seen to continue to improve for many years after the completion of a surgeon’s training. To overcome the incompleteness of data, we chose general criteria with which we could evaluate each study. However, this also faced the problem of variability between studies as few papers made any reference to validity or cost. One particular challenge was our assessment of content validity. As no programme specifically discussed their development with regards to validity of any form, we performed our analysis based on currently accepted practices of evaluating validity as outlined by Beck and Gable [29] and described above. For cost effectiveness, the lack of data on programme costs/fees prevented an objective assessment being made, which limits the usefulness of such an evaluation. One alternative to assessing surgeons based on specific intraand postoperative parameters would be to use a validated
Status and effectiveness of mentorship programmes in urology
assessment tool such as the Objective Structured Assessment of Technical Skills (OSATS). In transoral robotic surgery this has been shown to be highly effective [33] and there has also been development of an assessment tool that is based on the OSATS model, but specific for endoscopic sinus surgery [34]. If such a tool were to be developed for urology, it could greatly improve the assessment process of trainees and also provide a level of standardisation in the reporting of results for mentorship programmes. A further difficulty was the lack of published studies on mentorship in urology. To our knowledge this is the first review of its kind and as such, there was no basis of knowledge to work from. Further research is required to confirm the present findings and undertake a quantitative analysis, but this would require more detailed assessment of mentorship programmes with mentors/programme leaders collecting detailed operative data. Additionally, as there are only a few published studies, it is very possible that there is an element of publication bias at work with only mentorship programmes showing positive improvements in trainee skill levels being published, which would have distorted the present analysis.
Recommendations
analysis of different systems. Ideally, all mentorship programmes would regularly audit operative data and functional patient outcomes to ensure that quality of care is not being compromised. As outlined in the present review, there is a wide range of mentorship styles and programmes that are successful and we have given recommendations on how mentorship programmes can be improved through the use of more robust assessment tools and the potential usefulness of developing one specifically for urology.
Conflict of Interest None declared.
References 1 2
3
4
5
The present review makes the following recommendations for improving mentorship in urology: 6
1. To establish the most effective form of mentorship programme, it is vital that as many programmes as possible use functional patient outcomes as an endpoint or assessment criterion, rather than just ‘raw’ operative data. This would make it easier to compare between programmes in a quantitative fashion and allow for more detailed analysis. 2. Further research should be done into using validated assessment tools to measure trainee proficiency in urology. This could involve the use of OSATS or the development and use of an OSATS-based tool, with the support of professional bodies such as the BAUS and European Association of Urology.
7
8
9 10
11 12
Conclusions In an age of increasing patient demands’ and with the advent of ever more advanced technology in urology, it is paramount that surgeons embrace new surgical techniques and look to mentorship programmes in order to do so safely and effectively. Unfortunately, there is a noticeable lack of designated mentorship programmes available to surgeons and until professional bodies act to encourage and implement such programmes, this is unlikely to change. A further problem surrounding mentorship programmes is the lack of reporting of results, as well as little standardisation or definitive assessment criteria, which prevents quantitative
13
14
15
16
Toledo-Pereyra LH. Mentoring surgeons. J Invest Surg 2009; 22: 77–81 Moher D, Liberati A, Tetzlaff J, Altman DG. Preferred reporting items for systematic reviews and meta-analyses: the PRISMA Statement. Open Med 2009; 3: e123–30 Ahmed K, Miskovic D, Darzi A, Athanasiou T, Hanna GB. Observational tools for assessment of procedural skills: a systematic review. Am J Surg 2011; 202: 469–80 Stewart GD, Phipps S, Little B et al. Description and validation of a modular training system for laparoscopic nephrectomy. J Endourol 2012; 26: 1512–7 Stolzenburg JU, Rabenalt R, Do M, Horn LC, Liatsikos EN. Modular training for residents with no prior experience with open pelvic surgery in endoscopic extraperitoneal radical prostatectomy. Eur Urol 2006; 49: 491–500 Rashid HH, Leung YY, Rashid MJ, Oleyourryk G, Valvo JR, Eichel L. Robotic surgical education: a systematic approach to training urology residents to perform robotic-assisted laparoscopic radical prostatectomy. Urology 2006; 68: 75–9 Badani KK, Hemal AK, Peabody JO, Menon M. Robotic radical prostatectomy: the Vattikuti Urology Institute training experience. World J Urol 2006; 24: 148–51 Mirheydar H, Jones M, Koeneman KS, Sweet RM. Robotic surgical education: a collaborative approach to training postgraduate urologists and endourology fellows. JSLS 2009; 13: 287–92 Rane A. A training module for laparoscopic urology. JSLS 2005; 9: 460–2 Zorn KC, Orvieto MA, Gong EM et al. Robotic radical prostatectomy learning curve of a fellowship-trained laparoscopic surgeon. J Endourol 2007; 21: 441–7 El-Hakim A, Elhilali MM. Holmium laser enucleation of the prostate can be taught: the first learning experience. BJU Int 2002; 90: 863–9 Netsch C, Bach T, Herrmann TR, Neubauer O, Gross AJ. Evaluation of the learning curve for Thulium VapoEnucleation of the prostate (ThuVEP) using a mentor-based approach. World J Urol 2013; 31: 1231–8 Fabrizio MD, Tuerk I, Schellhammer PF. Laparoscopic radical prostatectomy: decreasing the learning curve using a mentor initiated approach. J Urol 2003; 169: 2063–5 Kolla SB, Gamboa AJ, Li R et al. Impact of a laparoscopic renal surgery mini-fellowship program on postgraduate urologist practice patterns at 3year followup. J Urol 2010; 184: 2089–93 Shalhav AL, Dabagia MD, Wagner TT, Koch MO, Lingeman JE. Training postgraduate urologists in laparoscopic surgery: the current challenge. J Urol 2002; 167: 2135–7 Gamboa AJ, Santos RT, Sargent ER et al. Long-term impact of a robot assisted laparoscopic prostatectomy mini fellowship training program on postgraduate urological practice patterns. J Urol 2009; 181: 778–82
© 2014 The Authors BJU International © 2014 BJU International
493
Hay et al.
17 McDougall EM, Corica FA, Chou DS et al. Short-term impact of a robot-assisted laparoscopic prostatectomy ‘mini-residency’ experience on postgraduate urologists’ practice patterns. Int J Med Robot 2006; 2: 70–4 18 Marguet CG, Young MD, L’Esperance JO et al. Hand assisted laparoscopic training for postgraduate urologists: the role of mentoring. J Urol 2004; 172: 286–9 19 Patel SR, Hedican SP, Bishoff JT et al. Skill based mentored laparoscopy course participation leads to laparoscopic practice expansion and assists in transition to robotic surgery. J Urol 2011; 186: 1997–2000 20 Nakada SY, Hedican SP, Bishoff JT, Shichman SJ, Wolf JS Jr. Expert videotape analysis and critiquing benefit laparoscopic skills training of urologists. JSLS 2004; 8: 183–6 21 Jones A, Eden C, Sullivan ME. Mutual mentoring in laparoscopic urology – a natural progression from laparoscopic fellowship. Ann R Coll Surg Engl 2007; 89: 422–5 22 Kwon EO, Bautista TC, Blumberg JM et al. Rapid implementation of a robot-assisted prostatectomy program in a large health maintenance organization setting. J Endourol 2010; 24: 461–5 23 Challacombe B, Kandaswamy R, Dasgupta P, Mamode N. Telementoring facilitates independent hand-assisted laparoscopic living donor nephrectomy. Transplant Proc 2005; 37: 613–6 24 Bruschi M, Micali S, Porpiglia F et al. Laparoscopic telementored adrenalectomy: the Italian experience. Surg Endosc 2005; 19: 836–40 25 Tang B, Hanna GB, Cuschieri A. Analysis of errors enacted by surgical trainees during skills training courses. Surgery 2005; 138: 14–20 26 Bollens R, Sandhu S, Roumeguere T, Quackels T, Schulman C. Laparoscopic radical prostatectomy: the learning curve. Curr Opin Urol 2005; 15: 79–82 27 Colegrove PM, Winfield HN, Donovan JF Jr, See WA. Laparoscopic practice patterns among North American urologists 5 years after formal training. J Urol 1999; 161: 881–6
494
© 2014 The Authors BJU International © 2014 BJU International
28 Keeley FX Jr, Rimington PD, Timoney AG, McClinton S. British association of urological surgeons laparoscopic mentorship guidelines. BJU Int 2007; 100: 247–8 29 Beck CT, Gable RK. Ensuring content validity: an illustration of the process. J Nurs Meas 2001; 9: 201–15 30 Birch DW, Asiri AH, de Gara CJ. The impact of a formal mentoring program for minimally invasive surgery on surgeon practice and patient outcomes. Am J Surg 2007; 193: 589–92 31 Hedican SP, Nakada SY. Videotape mentoring and surgical simulation in laparoscopic courses. J Endourol 2007; 21: 288–93 32 Abboudi M, Ahmed K, Kirby R, Khan MS, Dasgupta P, Challacombe B. Mentorship programmes for laparoscopic and robotic urology. BJU Int 2011; 107: 1869–71 33 Curry M, Malpani A, Li R et al. Objective assessment in residency-based training for transoral robotic surgery. Laryngoscope 2012; 122: 2184–92 34 Lin SY, Laeeq K, Ishii M et al. Development and pilot-testing of a feasible, reliable, and valid operative competency assessment tool for endoscopic sinus surgery. Am J Rhinol Allergy 2009; 23: 354–9
Correspondence: Kamran Ahmed, MRC Centre for Transplantation, 5th Floor Southwark Wing, Guy’s Hospital, London SE1 9RT, UK. e-mail: [email protected] Abbreviations: MF, mini-fellowship; MIS, minimally invasive surgery; OSATS, Objective Structured Assessment of Technical Skills; RALP, robot-assisted laparoscopic prostatectomy.
Current status and effectiveness of mentorship programmes in urology: a systematic review Daniel Hay*†, Mohammed Shamim Khan*†, Hendrik Van Poppel‡, Ben Van Cleynenbreugel‡, James Peabody§, Khurshid Guru¶, Ben Challacombe*†, Prokar Dasgupta*† and Kamran Ahmed*† *MRC Centre for Transplantation, King’s College London, †King’s Health Partners,Department of Urology, Guy’s Hospital, London, UK, ‡Department of Urology, University Hospital Gasthuisberg, Katholieke Universiteit Leuven, Leuven, Belgium, §Vattikuti Urology Institute, Henry Ford Hospital, Detroit, MI, and ¶Department of Urology, Roswell Park Cancer Institute, Buffalo, NY, USA
The objectives of this review were to identify and evaluate the efficacy of mentorship programmes for minimally invasive procedures in urology and give recommendations on how to improve mentorship. A systematic literature search of the PubMed/Medline databases was carried out according to the Preferred Reporting Items for Systematic Reviews and Meta-Analyses (PRISMA) guidelines. In all, 21 articles were included in the review and divided into four categories: fellowships, mini-fellowships, mentored skills courses and novel mentorship programmes. Various structures of mentorship programme were identified and in general, mentorship programmes were found to be feasible, having content validity and educational impact.
Introduction A mentorship programme is an educational system that teaches a set of practical skills or allows an individual to learn how to perform a specific procedure. There is considerable variation in the exact structures of current programmes, but mentorship as a standalone concept is the process of an expert individual advising and guiding a junior colleague and it is commonly accepted to be a crucial component in the education of a surgeon. Toledo-Pereyra [1] surmised that mentorship created the foundation for all surgeons and offered benefits other than just technical skills, such as providing a model for professional attitudes, standards of patient care and exemplary practice in research. Presently, there are three broad groups of mentorship programme. Firstly, there are fellowships, which generally last for a period ranging from 6 to 24 months and usually occur after the completion of specialty training. In a fellowship, surgeons focus on one aspect of their chosen specialty to gain a better understanding of the various techniques and procedures that are a part of that discipline. Fellowships are then sub-categorised into modular and non-modular; © 2014 The Authors BJU International © 2014 BJU International | doi:10.1111/bju.12713 Published by John Wiley & Sons Ltd. www.bjui.org
Perioperative data showed equally good outcomes when comparing trainees and specialists. Mentorship programmes are effective and represent one of the best current methods of training in urology. However, participation in such programmes is not widespread. The structure of mentorship programmes is highly variable, with no clearly defined ‘best approach’ for postgraduate training. This review offers recommendations as to how this ‘best approach’ can be established.
Keywords education, mentors, outcome assessment (health care), patient safety, surgery
modular training involves dividing a procedure into a series of steps according to technical difficulty, with the trainee only being allowed to proceed to the next step when the previous one is performed successfully, whereas non-modular fellowships take a less regimented approach. Similarly to fellowships, mini-fellowships (MFs) are designed to provide surgeons with sufficient confidence to undertake a newly learned procedure, but they aim to do so without the extended time commitments of a full fellowship. Typically lasting 1 week, they offer an insight into a technique or specific approach, but are not as comprehensive as full fellowships. Lastly, there are mentored skills courses that are shorter still and tend to focus on an introduction to a specific procedure, rather than a broad range of clinical exposure. This review aims to explore the different types of mentorship programmes available for surgeons in training and has the following aims: 1. To identify mentorship programmes for minimally invasive urological surgery encompassing laparoscopy, robotics and endourology.
BJU Int 2015; 116: 487–494 wileyonlinelibrary.com
Hay et al.
2. To evaluate the relative strengths and weaknesses of each programme identified. This involves assessing the content validity, feasibility, educational impact and cost effectiveness of each programme. 3. To make recommendations on how best to implement and improve mentorship programmes.
Materials and Methods This review has been carried out following the Preferred Reporting Items for Systematic Reviews and Meta-Analyses (PRISMA) guidelines [2]. The eligibility criteria for a paper to be included in this review were that the full paper had to be available in English, be published in the last 10 years (from 2002 onwards) and consider training specifically in urology. Information Sources, Search and Data Items To find relevant papers we carried out a literature review by means of a free text search of the PubMed and Cochrane Library databases. The search terms used were ‘mentorship programmes/programs’, ‘mentorship surgery’, ‘mentorship urology’, ‘urology fellowship’, ‘urology mentoring’, ‘urology training’, ‘urology proctoring’ and ‘urological mentored surgery’. The reference lists of papers that had been selected for inclusion in this review were also used to identify other relevant studies. Study Selection We included studies for review if they made reference to minimally invasive surgical (MIS) training (both laparoscopic and robotic), specific mentorship programmes and the
teaching of MIS skills. We excluded papers that were bulletins, reviews and editorial letters. The key information extracted included format, characteristics and results of each programme. Outcome data recorded included blood loss, operation time, rate of complications, and ‘take’ rate (a measure of how many surgeons are still practicing surgery). Our initial search yielded 256 results. Studies were then excluded if they did not meet the above criteria, leaving a total of 64 abstracts, which were reviewed, and of these, 28 were read in full with 17 included in this review and an additional four studies were identified by examining the reference lists of included papers (Fig. 1). Selected papers were then divided into the common forms of mentorship programme (as discussed above) with the addition of a ‘novel mentorship programme’ category. This was used for papers that did not fit into the other three and used an unconventional approach, e.g. not having a mentor physically present, or tele-mentoring. Evaluation of Study Quality Each of the studies included in this review were evaluated for their effectiveness. Our evaluation centred on four general components that are essential to the success of any teaching programme as outlined by Ahmed et al. [3]: 1. Content validity; the extent to which the content domain is being taught by the programme; does the programme ensure that the skills required to operate independently are being taught? Fig. 1 Process of article selection.
256 papers identified from literature search of Medline and the Cochrane Library
64 abstracts reviewed
192 papers excluded based on title, usually due to a lack of relevance
28 papers evaluated in full
36 papers excluded, most often for describing mentorship in a specialty other than urology
17 papers selected
21 papers included in the final analysis for evaluation
488
© 2014 The Authors BJU International © 2014 BJU International
11 papers excluded for describing mentorship, but without quantitative or qualitative results
4 papers identified through reference lists and included in the final review
Status and effectiveness of mentorship programmes in urology
2. Feasibility/acceptability; is the programme compatible with the demands of current training? 3. Educational impact; is there any improvement following the programme? 4. Cost effectiveness; do the benefits of the programme justify the cost? The most common method of analysis used in studies was comparing operative data from the mentee to those of their mentor or general targets, such as those set by the BAUS. Some studies also included take rates, which assessed the percentage of mentees that were still performing the skills taught at various times after the programme. Where available, quantitative data from each study has been included in this review.
Results Fellowships The most common structure seen in this review was that of a typical fellowship, with 10 papers analysing such a programme. Fellowships are currently available that teach laparoscopic nephrectomy, laparoscopic prostatectomy, robotassisted laparoscopic prostatectomy (RALP), and laser enucleation of the prostate. In half of the programmes a modular training system was used [4–8], whilst in the others, the procedure was not explicitly split up [9–13]. Results from all the fellowship programmes were positive and comparable (Table 2), suggesting that the distinction between step-by-step learning and a less regimented approach may be of little
importance. Trainees may benefit more from one training system than another, but this may possibly be based on personal learning style and preference. MF Kolla et al. [14] describe a 5-day programme, where questionnaires at 3 years follow-up revealed that the overall take rate was 70% (Table 2). The success of MFs is further shown by Shalhav et al. [15], who conclude that MFs allow trainees to progress along their learning curve quickly and be able to operate independently and safely. Gamboa et al. [16] describe a robotic MF with questionnaires from 1, 2 and 3 years after the programme showing that a large proportion of participants are still performing RALP (Tables 1 and 2). Interestingly, data also shows that participants who undertook the course with a partner (another surgeon from the same institution) had a non-significantly higher take rate than those who did not. The same questionnaire had been previously sent out and analysed by McDougall et al. [17] showing that the take rate at a mean follow-up of 7.2 months was 95%. Mentored Skills Courses Data from the hand-assisted laparoscopic nephrectomy course at Duke University Medical Centre [18] shows that 93% of trainees with a mentor were still performing laparoscopic procedures at 6 months follow-up compared with 44% of trainees without a mentor. Rather than simply looking at take rates for laparoscopy, Patel et al. [19] question whether
Table 1 Evaluation of the quality of the mentorship programmes included in this review. Reference
Stewart et al. [4] Stolzenburg et al. [5] Rashid et al. [6] Badani et al. [7] Mirheydar et al. [8] Rane [9] Zorn et al. [10] El-Hakim and Elhilali [11] Netsch et al. [12] Fabrizio et al. [13] Kolla et al. [14] Shalhav et al. [15] Gamboa et al. [16] and McDougall et al. [17] Marguet et al. [18] Patel et al. [19] Nakada et al. [20] Jones et al. [21] Kwon et al. [22] Challacombe et al. [23] Bruschi et al. [24]
Structure
Feasibility (a)
Educational impact (b)
Fellowship Fellowship Fellowship Fellowship Fellowship Fellowship Fellowship Fellowship Fellowship Fellowship MF MF MF Mentored skills course Mentored skills course Mentored skills course Novel mentorship programme Novel mentorship programme Novel mentorship programme Novel mentorship programme
Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes
Yes – OD Yes – OD Yes – OD Yes – OD Yes – IP Yes – IP Yes – OD Yes – OD Yes – OD Yes – OD Yes – EP Yes – IP Yes – EP Yes – IP Yes – EP Yes – OD Yes – OD Yes – OD Yes – OD Yes – OD
Cost effective (c)
Yes
No No No
Yes
Content validity (d) Yes Yes Yes Yes Yes Yes No Yes Yes Yes Yes Yes Yes Yes Yes No No Yes Yes Yes
(a) Whether both trainees and mentors feel that the programme is compatible with the demands of current training and clinical practice. (b) Whether there is any identifiable improvement in skills after the programme. (c) Whether the benefits of the programme justify its cost. (d) Whether the programme ensures that the skills required to operate independently are being taught. OD, operative data; IP, independent practice; EP, expanded practice.
© 2014 The Authors BJU International © 2014 BJU International
489
Hay et al.
Table 2 Results of the mentorship programmes included in this review. A, operative data for trainee and mentor from the laparoscopic fellowships included in this review Reference
Stewart et al. [4] Stolzenburg et al. [5] Fabrizio et al. [13]
Procedure performed
Laparoscopic nephrectomy Endoscopic extraperitoneal radical prostatectomy Laparoscopic radical prostatectomy
Mean operative time, min
Mean estimated blood loss, mL
Mean postoperative stay, days
Transfusion rate, %
Trainees
Mentor
Trainees
Mentor
Trainees
Mentor
Trainees
Mentor
140 174.5
108 172
30 –
20 –
4 –
5 –
3.8 2
4.2 0
313
248
250
150
3
3
–
–
B, take rates of trainees at specific points after the mentorship programme has finished Reference
Kolla et al. [14] Gamboa et al. [16] Marguet et al. [18]
Take rate 6 months without mentor, %
6 months with mentor, %
1 years, %
2 years, %
3 years, %
– – 44
– – 93
72 78 –
71 78 –
71 86 –
C, operative data comparing first and second independent year of practice in mutual mentoring Reference
Jones et al. [21]
Procedure performed
Laparoscopic radical nephrectomy Laparoscopic simple nephrectomy Laparoscopic pyeloplasty
trainees have expanded their laparoscopy practice in terms of case volume and complexity after the Urological Association Mentored Laparoscopy Course. At a mean follow-up of 34.5 months, 77% of participants had done so. Nakada et al. [20] note the success of a standard laparoscopic skills course that involved mentoring via videotape analysis of participants. They found that the expert mentor’s analysis of each trainee’s footage led to statistically significant improvements in various laparoscopic skills.
Mean operative time, min
Mean estimated blood loss, mL
Year 1
Year 2
Year 1
Year 2
246 208 238
231 200 198
160 220 39
314 153 42
independent cases show that the overall operative and warm ischaemia times are highly comparable to standard reported times [23]. Bruschi et al. report similar results, with operative time and blood loss being highly satisfactory and comparing well with averages seen in large laparoscopic adrenalectomy reports.
Discussion Comparison of Mentorship Programmes
Novel Mentorship Programmes Jones et al. [21] discuss their ‘mutual mentoring’ experience following a laparoscopic fellowship, which involved two recently qualified consultants acting as the mentor for one another. In this study, the surgeons operated as the assistant for the other instead of having a junior as the assistant, which allowed them to offer advice and support and bypass the need for a conventional senior mentor figure. Audit data shows that there was a non-significant improvement in results between their first and second independent years (Table 2). Kwon et al. [22] introduced a RALP programme with a similar mutual mentoring system and operative data shows that the programme has succeeded thus far. An interesting potential field for future mentorship programmes is that of tele-mentoring, which Challacombe et al. [23] and Bruschi et al. [24] both evaluated. Operative data from the
490
© 2014 The Authors BJU International © 2014 BJU International
The most obvious difference between the different programmes is the time each takes, but rather than simply being an advantage or disadvantage, this is more a reflection of the level of trainee skill at the start, and intended outcome at the end of each programme. The most common mentorship programme seen in this review was the fellowship, and whilst it is hard to compare the different types of programme, the fellowship is believed to be the ‘gold standard’ of mentorship in terms of achieving proficiency and maintaining patient safety. One of the advantages of a fellowship is that it lasts longer than other programmes, so mentees can immerse themselves in a large number of cases. Consequently, mentees will have more experience and should be able to respond well to complications and unforeseen circumstances. More importantly, the extended training period allows surgeons to appreciate the specific nuances of
Status and effectiveness of mentorship programmes in urology
advanced techniques, such as coordinating the actions of the surgical team with robotics, something that is rarely possible in shorter programmes. Another positive is that due to the 1:1 mentor to mentee ratio, there is often personalised and flexible teaching that can focus on specific difficulties, a benefit that is known to be of upmost importance in surgical training [25]. The obvious disadvantage of a fellowship is the time and effort associated with such training. Fellowships taken before CCT (Certificates of Completion of Training) add to what is already a long training pathway and it is difficult for a consultant to arrange the necessary leave to undertake a full fellowship. Moreover, unless fellowships are undertaken in a centre of excellence or a hospital with a large volume of cases there may be prolonged intervals between cases, further extending the length of the fellowship. All MFs in this review reported a high percentage of mentees continuing to practice the skills taught to them, as well as increasing the number and complexity of cases. The advantages of MFs revolve mainly around the high intensity of the course, which allows substantial progress to be made in a short space of time. A second virtue of MF is that mentees can potentially be paired during and after the course with another surgeon from a nearby hospital, which has been shown to improve take rates [16]. One considerable disadvantage is the total cost of the programme. Kolla et al. [14] noted that the cost of a single course is around $3500 and that attendance fell significantly when the cost was no longer fully subsidised. There is also the question of skill advancement. Bollens et al. [26] theorised that after a certain number of cases, it is important to confer with an expert or visit a centre of excellence to further improve one’s technique. Because of the short nature of the MF, there is no real facility for this and mentees who operate in an institution without experienced surgeons may struggle to progress beyond the initial slope of the learning curve. The clearest advantages of mentored skills courses is their duration and cost; it is much easier to commit to a 2-day course than the months required for a full fellowship and there is not quite such a considerable cost as seen in the MFs. Patel et al. [19] documented another interesting value of skills courses, where a laparoscopic skills course helped participants progress to robotic surgery. This opens the door to the possibility of crossmodality training, where trainees can become proficient in multiple surgical modalities (i.e. laparoscopic and robotic) from a single course. Colegrove et al. [27] found that following laparoscopic skills courses, only 53% of participants were still performing laparoscopy. One possible reason for this is that participants gain confidence in the basic principles whilst on the course, but are unable to carry this over into clinical practice in the weeks and months that follow. This is a particularly noticeable disadvantage for surgeons from a lowvolume centre, where there is little opportunity to make use of learned skills, further increasing the likelihood of stopping.
The importance of training with a partner was discussed in the context of the MF programmes, but its use as a primary method of instruction was clearly shown in the implementation of a robotic surgery programme at Kaiser Permanente Los Angeles Medical Centre [22]. Such triumphs suggest that urology departments could introduce new technologies, e.g. robotics, via mutual mentoring, but it is important to remember the importance of an expert mentor, as noted in the BAUS mentorship guidelines [28]. Regular consultation with an expert at a centre of excellence could further improve the usefulness of such programmes, as could the application of tele-mentoring to aid in such consultations. Tele-mentoring allows for the dissemination of an expert’s knowledge across countries and continents, without the expert having to leave their office. If put to optimal use, tele-mentoring could revolutionise the teaching of robotic surgery and allow urology departments to safely offer a whole new range of procedures. Evaluating the Quality of Programmes Whilst all programmes included in this review show clear educational impact (Table 1), it is not clear however, the extent and specific way in which this benefit is seen. Most commonly, the authors have collected operative data to analyse either the learning curve of their mentees or to directly compare mentee and mentor performance. Comparisons made between the mentee and the available guidelines for a procedure, such as those outlined by the BAUS, can give an absolute indicator of whether they have reached a satisfactory and safe standard. One tool used to evaluate each programme was an analysis of its content validity. Beck and Gable [29] surmised that this could be done in an a priori fashion, where development involves careful consideration of existing literature or in an a posteriori fashion, with a panel of experts determining validity. As nearly all of the programme developers were experts, each programme was designed with validity in mind. The exceptions to this were the mutual mentoring programme established by Jones et al. [21] and the skills course described by Nakada et al. [20], which did not have validity for the following reasons. Firstly, mutual mentoring does not involve an expert mentor and thus there is no expert evaluation of the programme design and structure. Secondly, the programme described by Nakada et al. involves only laparoscopic simulation, which does not equate to a surgeon performing actual laparoscopy independently. Although not included in the evaluation of each programme, patient safety is arguably one of the most important assessment tools. In part, its exclusion was due to the assumption that a programme would only exist if it did not compromise the quality of care that patients received and this was generally the case. Across all programmes there was a low rate of complications and an even smaller number of © 2014 The Authors BJU International © 2014 BJU International
491
Hay et al.
cases that required conversion to open surgery. Fabrizio et al. [13] and Kwon et al. [22] noted rates of just 5%, with one of 20 and five of 100 conversions respectively, for laparoscopic and robot-assisted prostatectomy. Shalhav et al. [15] reported two conversions and three major complications in 30 laparoscopic procedures and Stewart et al. [4] claimed three conversions and two major complications in 105 laparoscopic nephrectomies, which equated to better figures than for the mentor (although this was in part attributed to selection bias with the mentor taking on more complex cases). Stolzenburg et al. [5] described four complications in the mentee’s first 50 laparoscopic prostatectomies and seven complications in the following 100. Numerous programmes reported zero complications or conversions and in nearly all programmes, complication rates were comparable to those of the mentor with two exceptions. The Jones et al. [21] unusual mutual mentoring programme had a conversion rate of 17% (11/66) in the first year of independent laparoscopic practice, although this fell to 4% (three of 84) in the second year, suggesting that the mentees were still moving along the learning curve to proficiency when they began operating independently. One must question whether it is acceptable to have a higher initial rate of conversions if it leads to a considerable improvement the following year. Furthermore, Zorn et al. [10] recorded four conversions and 10 complications in the first 30 RALP cases. There were no conversions in the following 60, but there were another 13 complications. Again, one has to ask whether these figures for adverse events are acceptable, despite having good positive surgical margin rates. Their marginally poorer results are reflected in the fact that both the Zorn et al. and Jones et al. programmes do not have content validity. As discussed earlier, this is due to the lack of an expert robotic/ laparoscopic mentor figure, with the programmes relying on an expert open surgeon and mutual proficiency respectively. Gamboa et al. [16] utilise inanimate and cadaveric simulation in their programme to teach the initial surgical skills needed. Similarly, Rane [9] encourages all mentees to make extensive use of simulators before attending their fellowship to gain the most from it. Rashid et al. [6] goes as far as suggesting that simulation will supersede mentorship, although this would require unequivocal evidence showing how skills learned in simulation equate to ability in theatre. Until this happens, simulation can only act as an adjunct and not a replacement for mentorship, but it is clear that simulation will form an increasingly important part of surgical training in the future.
Barriers to Mentorship Programmes It is important to remember that there are numerous barriers to mentorship programmes. Firstly, there is the issue of operating privileges and malpractice insurance for the visiting surgeon. There is also a question about which surgeon (mentor or
492
© 2014 The Authors BJU International © 2014 BJU International
mentee) should be responsible for any complications that arise during the procedure. Even though the mentor is expected to be the ‘lead surgeon’, it is likely that the mentee will be performing a large proportion of the operation and as such, should legal culpability reside with them? Secondly, special considerations must be made about the process of consenting patients, who should be informed that their procedure is being carried out by a trainee surgeon under guidance. Thirdly, many surgeons would be unable or unwilling to take off the necessary time for a full fellowship reducing their uptake. Birch et al. [30] and Hedican and Nakada [31] conclude that financial compensation, time constraints and acquiring temporary operating privileges were all major obstacles, with Birch et al. also noting that a lack of simple initial cases in laparoscopic and robotic urology was a significant obstacle. Abboudi et al. [32] suggested that the easiest way to overcome many of these issues lies in tele-mentoring and this has been shown to be a viable option for mentorship [23,24]. Alternatively, the path to surmounting these problems may involve the action of organisations, such as the BAUS and AUA. These professional bodies could take on a leadership role in the creation of mentorship programmes, as well as highlighting the importance of such programmes to improve uptake and allay concerns over ‘taking time out’. Hopefully, this would make programmes more easily available and improve participation.
Limitations The main limitations of the present review are due to deficiencies in the available data, particularly for comparing mentorship programmes. As most programmes had different aims and endpoints in terms of data collection, it was impossible to perform a meaningful quantitative analysis. Moreover, operative data alone cannot give a true picture of the calibre of a surgeon, as it is how a patient is affected by the surgery that is truly important. Functional outcomes (e.g. continence and potency after prostatectomy) could give a very clear picture of a surgeon’s level of ability, particularly as these figures have been seen to continue to improve for many years after the completion of a surgeon’s training. To overcome the incompleteness of data, we chose general criteria with which we could evaluate each study. However, this also faced the problem of variability between studies as few papers made any reference to validity or cost. One particular challenge was our assessment of content validity. As no programme specifically discussed their development with regards to validity of any form, we performed our analysis based on currently accepted practices of evaluating validity as outlined by Beck and Gable [29] and described above. For cost effectiveness, the lack of data on programme costs/fees prevented an objective assessment being made, which limits the usefulness of such an evaluation. One alternative to assessing surgeons based on specific intraand postoperative parameters would be to use a validated
Status and effectiveness of mentorship programmes in urology
assessment tool such as the Objective Structured Assessment of Technical Skills (OSATS). In transoral robotic surgery this has been shown to be highly effective [33] and there has also been development of an assessment tool that is based on the OSATS model, but specific for endoscopic sinus surgery [34]. If such a tool were to be developed for urology, it could greatly improve the assessment process of trainees and also provide a level of standardisation in the reporting of results for mentorship programmes. A further difficulty was the lack of published studies on mentorship in urology. To our knowledge this is the first review of its kind and as such, there was no basis of knowledge to work from. Further research is required to confirm the present findings and undertake a quantitative analysis, but this would require more detailed assessment of mentorship programmes with mentors/programme leaders collecting detailed operative data. Additionally, as there are only a few published studies, it is very possible that there is an element of publication bias at work with only mentorship programmes showing positive improvements in trainee skill levels being published, which would have distorted the present analysis.
Recommendations
analysis of different systems. Ideally, all mentorship programmes would regularly audit operative data and functional patient outcomes to ensure that quality of care is not being compromised. As outlined in the present review, there is a wide range of mentorship styles and programmes that are successful and we have given recommendations on how mentorship programmes can be improved through the use of more robust assessment tools and the potential usefulness of developing one specifically for urology.
Conflict of Interest None declared.
References 1 2
3
4
5
The present review makes the following recommendations for improving mentorship in urology: 6
1. To establish the most effective form of mentorship programme, it is vital that as many programmes as possible use functional patient outcomes as an endpoint or assessment criterion, rather than just ‘raw’ operative data. This would make it easier to compare between programmes in a quantitative fashion and allow for more detailed analysis. 2. Further research should be done into using validated assessment tools to measure trainee proficiency in urology. This could involve the use of OSATS or the development and use of an OSATS-based tool, with the support of professional bodies such as the BAUS and European Association of Urology.
7
8
9 10
11 12
Conclusions In an age of increasing patient demands’ and with the advent of ever more advanced technology in urology, it is paramount that surgeons embrace new surgical techniques and look to mentorship programmes in order to do so safely and effectively. Unfortunately, there is a noticeable lack of designated mentorship programmes available to surgeons and until professional bodies act to encourage and implement such programmes, this is unlikely to change. A further problem surrounding mentorship programmes is the lack of reporting of results, as well as little standardisation or definitive assessment criteria, which prevents quantitative
13
14
15
16
Toledo-Pereyra LH. Mentoring surgeons. J Invest Surg 2009; 22: 77–81 Moher D, Liberati A, Tetzlaff J, Altman DG. Preferred reporting items for systematic reviews and meta-analyses: the PRISMA Statement. Open Med 2009; 3: e123–30 Ahmed K, Miskovic D, Darzi A, Athanasiou T, Hanna GB. Observational tools for assessment of procedural skills: a systematic review. Am J Surg 2011; 202: 469–80 Stewart GD, Phipps S, Little B et al. Description and validation of a modular training system for laparoscopic nephrectomy. J Endourol 2012; 26: 1512–7 Stolzenburg JU, Rabenalt R, Do M, Horn LC, Liatsikos EN. Modular training for residents with no prior experience with open pelvic surgery in endoscopic extraperitoneal radical prostatectomy. Eur Urol 2006; 49: 491–500 Rashid HH, Leung YY, Rashid MJ, Oleyourryk G, Valvo JR, Eichel L. Robotic surgical education: a systematic approach to training urology residents to perform robotic-assisted laparoscopic radical prostatectomy. Urology 2006; 68: 75–9 Badani KK, Hemal AK, Peabody JO, Menon M. Robotic radical prostatectomy: the Vattikuti Urology Institute training experience. World J Urol 2006; 24: 148–51 Mirheydar H, Jones M, Koeneman KS, Sweet RM. Robotic surgical education: a collaborative approach to training postgraduate urologists and endourology fellows. JSLS 2009; 13: 287–92 Rane A. A training module for laparoscopic urology. JSLS 2005; 9: 460–2 Zorn KC, Orvieto MA, Gong EM et al. Robotic radical prostatectomy learning curve of a fellowship-trained laparoscopic surgeon. J Endourol 2007; 21: 441–7 El-Hakim A, Elhilali MM. Holmium laser enucleation of the prostate can be taught: the first learning experience. BJU Int 2002; 90: 863–9 Netsch C, Bach T, Herrmann TR, Neubauer O, Gross AJ. Evaluation of the learning curve for Thulium VapoEnucleation of the prostate (ThuVEP) using a mentor-based approach. World J Urol 2013; 31: 1231–8 Fabrizio MD, Tuerk I, Schellhammer PF. Laparoscopic radical prostatectomy: decreasing the learning curve using a mentor initiated approach. J Urol 2003; 169: 2063–5 Kolla SB, Gamboa AJ, Li R et al. Impact of a laparoscopic renal surgery mini-fellowship program on postgraduate urologist practice patterns at 3year followup. J Urol 2010; 184: 2089–93 Shalhav AL, Dabagia MD, Wagner TT, Koch MO, Lingeman JE. Training postgraduate urologists in laparoscopic surgery: the current challenge. J Urol 2002; 167: 2135–7 Gamboa AJ, Santos RT, Sargent ER et al. Long-term impact of a robot assisted laparoscopic prostatectomy mini fellowship training program on postgraduate urological practice patterns. J Urol 2009; 181: 778–82
© 2014 The Authors BJU International © 2014 BJU International
493
Hay et al.
17 McDougall EM, Corica FA, Chou DS et al. Short-term impact of a robot-assisted laparoscopic prostatectomy ‘mini-residency’ experience on postgraduate urologists’ practice patterns. Int J Med Robot 2006; 2: 70–4 18 Marguet CG, Young MD, L’Esperance JO et al. Hand assisted laparoscopic training for postgraduate urologists: the role of mentoring. J Urol 2004; 172: 286–9 19 Patel SR, Hedican SP, Bishoff JT et al. Skill based mentored laparoscopy course participation leads to laparoscopic practice expansion and assists in transition to robotic surgery. J Urol 2011; 186: 1997–2000 20 Nakada SY, Hedican SP, Bishoff JT, Shichman SJ, Wolf JS Jr. Expert videotape analysis and critiquing benefit laparoscopic skills training of urologists. JSLS 2004; 8: 183–6 21 Jones A, Eden C, Sullivan ME. Mutual mentoring in laparoscopic urology – a natural progression from laparoscopic fellowship. Ann R Coll Surg Engl 2007; 89: 422–5 22 Kwon EO, Bautista TC, Blumberg JM et al. Rapid implementation of a robot-assisted prostatectomy program in a large health maintenance organization setting. J Endourol 2010; 24: 461–5 23 Challacombe B, Kandaswamy R, Dasgupta P, Mamode N. Telementoring facilitates independent hand-assisted laparoscopic living donor nephrectomy. Transplant Proc 2005; 37: 613–6 24 Bruschi M, Micali S, Porpiglia F et al. Laparoscopic telementored adrenalectomy: the Italian experience. Surg Endosc 2005; 19: 836–40 25 Tang B, Hanna GB, Cuschieri A. Analysis of errors enacted by surgical trainees during skills training courses. Surgery 2005; 138: 14–20 26 Bollens R, Sandhu S, Roumeguere T, Quackels T, Schulman C. Laparoscopic radical prostatectomy: the learning curve. Curr Opin Urol 2005; 15: 79–82 27 Colegrove PM, Winfield HN, Donovan JF Jr, See WA. Laparoscopic practice patterns among North American urologists 5 years after formal training. J Urol 1999; 161: 881–6
494
© 2014 The Authors BJU International © 2014 BJU International
28 Keeley FX Jr, Rimington PD, Timoney AG, McClinton S. British association of urological surgeons laparoscopic mentorship guidelines. BJU Int 2007; 100: 247–8 29 Beck CT, Gable RK. Ensuring content validity: an illustration of the process. J Nurs Meas 2001; 9: 201–15 30 Birch DW, Asiri AH, de Gara CJ. The impact of a formal mentoring program for minimally invasive surgery on surgeon practice and patient outcomes. Am J Surg 2007; 193: 589–92 31 Hedican SP, Nakada SY. Videotape mentoring and surgical simulation in laparoscopic courses. J Endourol 2007; 21: 288–93 32 Abboudi M, Ahmed K, Kirby R, Khan MS, Dasgupta P, Challacombe B. Mentorship programmes for laparoscopic and robotic urology. BJU Int 2011; 107: 1869–71 33 Curry M, Malpani A, Li R et al. Objective assessment in residency-based training for transoral robotic surgery. Laryngoscope 2012; 122: 2184–92 34 Lin SY, Laeeq K, Ishii M et al. Development and pilot-testing of a feasible, reliable, and valid operative competency assessment tool for endoscopic sinus surgery. Am J Rhinol Allergy 2009; 23: 354–9
Correspondence: Kamran Ahmed, MRC Centre for Transplantation, 5th Floor Southwark Wing, Guy’s Hospital, London SE1 9RT, UK. e-mail: [email protected] Abbreviations: MF, mini-fellowship; MIS, minimally invasive surgery; OSATS, Objective Structured Assessment of Technical Skills; RALP, robot-assisted laparoscopic prostatectomy.
