Original article
Surgeons’ and trainees’ perceived self-efficacy in operating theatre non-technical skills G. Pena1,2 , M. Altree1 , J. Field3 , M. J. W. Thomas4 , P. Hewett2 , W. Babidge1,2 and G. J. Maddern1,2 1
Australian Safety and Efficacy Register of New Interventional Procedures – Surgical (ASERNIP-S), Royal Australasian College of Surgeons, University of Adelaide Discipline of Surgery, Queen Elizabeth Hospital, 3 John Field Consulting and 4 Westwood-Thomas Associates, Adelaide, South Australia, Australia Correspondence to: Professor G. J. Maddern, ASERNIP-S, Royal Australasian College of Surgeons, 199 Ward Street, North Adelaide, South Australia 5006, Australia (e-mail: [email protected]) 2
Background: An important factor that may influence an individual’s performance is self-efficacy, a
personal judgement of capability to perform a particular task successfully. This prospective study explored newly qualified surgeons’ and surgical trainees’ self-efficacy in non-technical skills compared with their non-technical skills performance in simulated scenarios. Methods: Participants undertook surgical scenarios challenging non-technical skills in two simulation sessions 6 weeks apart. Some participants attended a non-technical skills workshop between sessions. Participants completed pretraining and post-training surveys about their perceived self-efficacy in non-technical skills, which were analysed and compared with their performance in surgical scenarios in two simulation sessions. Change in performance between sessions was compared with any change in participants’ perceived self-efficacy. Results: There were 40 participants in all, 17 of whom attended the non-technical skills workshop. There was no significant difference in participants’ self-efficacy regarding non-technical skills from the pretraining to the post-training survey. However, there was a tendency for participants with the highest reported self-efficacy to adjust their score downwards after training and for participants with the lowest self-efficacy to adjust their score upwards. Although there was significant improvement in non-technical skills performance from the first to second simulation sessions, a correlation between participants’ self-efficacy and performance in scenarios in any of the comparisons was not found. Conclusion: The results suggest that new surgeons and surgical trainees have poor insight into their non-technical skills. Although it was not possible to correlate participants’ self-belief in their abilities directly with their performance in a simulation, in general they became more critical in appraisal of their abilities as a result of the intervention. Paper accepted 16 January 2015 Published online 19 March 2015 in Wiley Online Library (www.bjs.co.uk). DOI: 10.1002/bjs.9787
Introduction
Surgical competence refers to a surgeon’s knowledge and skills in both technical and non-technical domains, and encompasses what surgeons have learned and can do. Performance is what surgeons actually do in day-to-day practice. Performance depends on competence but is also influenced by individual and system-related factors1,2 . One of the individual-related factors that may influence surgical performance is the surgeon’s self-efficacy, which refers to personal judgement of how well the individual can execute a course of action to deal with prospective situations3 . Studies have found that self-efficacy influences a person’s decisions about what behaviours they choose to undertake, © 2015 BJS Society Ltd Published by John Wiley & Sons Ltd
the effort expended and sustained in attempting those behaviours, the emotional responses and ultimately the actual performance4,5 . Task execution requires not only skills but also an individual’s self-belief in their own ability to undertake the task well. Therefore, individuals with the same knowledge and skills may perform poorly, adequately or extraordinarily depending on fluctuations in self-efficacy thinking6 . Cognitive, social and personal resources that complement technical skills in surgery are referred to as non-technical skills7 . Self-efficacy in non-technical skills may influence performance and affect patient safety. A surgical trainee insecure about their leadership and team-related skills may avoid situations in which they BJS 2015; 102: 708–715
Perceived self-efficacy in operating theatre non-technical skills
are required to take control, as these situations cause significant stress and anxiety. Conversely, surgeons with inflated self-efficacy may make mistakes, as they fail to consider and discuss options owing to overconfidence in their decision-making processes, or may avoid calling for help in emergency situations. Either low or overinflated self-efficacy regarding non-technical skills may have deleterious consequences. An ideal training programme builds participants’ skills and proportionally their self-efficacy, improving performance by training new skills, and by enhancing the trainees’ belief in what they can do safely, corresponding with their skill level. Conversely, an exercise that expands self-efficacy but not the requisite skills can result in an individual with lack of insight and confidence greater than their abilities. The Royal Australasian College of Surgeons (RACS) developed a training programme to teach surgical trainees and newly qualified surgeons about non-technical skills using simulation and a non-technical skills workshop. This article explores the correlation between participants’ perceived self-efficacy regarding non-technical skills, both before and after training, and their performance in a simulated environment. It was hypothesized that after formal training there would be a significant positive correlation between participants’ self-efficacy and their performance in the simulated environment. It was considered that by making participants more knowledgeable about non-technical skills they would gain insight into their strengths and shortcomings, and judge more accurately their ability to perform non-technical skill-related tasks. Methods
The study was part of a project investigating the use of simulation for non-technical skills training in the operating room, which was conducted in two public hospitals in Adelaide, Australia, between April and July 2013. The study was approved by each institutional review board and written consent was obtained from all participants. Participants consisted of surgical trainees and surgeons who had been awarded Fellowship of the RACS in the previous 3 years (2009–2012) with no previous experience in operating room simulation for non-technical skill training. The study involved simulation of surgical scenarios requiring non-technical skills over two sessions, 6 weeks apart, incorporating a cross-over and a repeat scenario design. In the first session, each participant underwent simulation of either scenario 1 (respiratory distress during laparoscopy) or scenario 2 (possibility of wrong-site surgery) by selecting © 2015 BJS Society Ltd Published by John Wiley & Sons Ltd
709
a sealed opaque envelope at random, followed by scenario 3 (haemodynamic instability in a patient who is possibly a Jehovah Witness). According to participants’ availability, they attended a RACS non-technical skills workshop (NTS workshop) between the two simulation sessions. At the second simulation session each participant underwent simulation of the alternative scenario, 1 or 2, to that of their first session, and all participants repeated scenario 3. All of the simulated scenarios occurred within real operating rooms and lasted for between 7 and 10 min. Colleagues, playing roles similar to the real occupation, were used to compose a standard operating room team. The scenarios were created by experts in surgery, anaesthesia and human factors. For each scenario, a series of stressors (such as equipment problems, distractions, interruptions, team factors related to interpersonal issues and clinical factors) were introduced to challenge trainees’ non-technical skills. All simulated scenarios were video-recorded and assessed after the data collection phase was complete by two independent assessors using the Non-Technical Skills for Surgeons (NOTSS) scoring system8,9 . The independent assessors, one consultant surgeon and one psychologist crew resource management expert, had received training in the use of the scoring tool and evaluated the videos blinded to each cohort and simulation session. All scenarios were followed by a one-on-one debriefing for 20–30 min, facilitated by a medical doctor with expertise in healthcare communication and the field of human factors. Participants were asked to complete a pretraining and post-training questionnaire about their perceived self-efficacy regarding non-technical skills. The pretraining questionnaire was delivered online in conjunction with a demographic questionnaire before simulation session 1. The post-training questionnaire was delivered immediately after simulation session 2. The questionnaire was modelled on the General Self Efficacy scale developed by Schwarzer and Jerusalem10,11 , and included 12 statements based on elements of the NOTSS taxonomy7,12 to be answered on a five-point Likert scale. The statements reflected participants’ optimistic self-belief about their non-technical skills concerning situation awareness, decision-making, communication and teamwork, and leadership (Table 1). The order of the statements presented to participants was altered from the first to the second questionnaire. Participants’ pretraining and post-training self-efficacy scores were analysed and compared with their nontechnical skills performances in the scenarios during simulation session 1 and simulation session 2 respectively. Furthermore, any change in performance in scenario 3 was correlated with the change in participants’ perceived www.bjs.co.uk
BJS 2015; 102: 708–715
710
Table 1
G. Pena, M. Altree, J. Field, M. J. W. Thomas, P. Hewett, W. Babidge and G. J. Maddern
Participants’ perceived confidence questionnaire
NOTSS category Situation awareness
Decision-making
Communication and teamwork
Leadership
Statements I always seek information in the operating room concerning the operative findings and from the operating room environment, equipment and people I am able to interpret the information gathered and use it to understand the match or mismatch between the situation and the expected state I can predict what may happen in the near future as a result of possible actions, interventions or non-intervention I can assess the threats and benefits of potential courses of action and to solve a problem When I choose a solution to a problem, I always let all relevant personnel know the chosen option I am flexible and I can change plans if required to cope with changing circumstances During surgery, I am able to exchange information in a timely manner to aid establishment of a shared understanding among team members I always ensure that the team not only has the relevant information to carry out the operation, but that they understand it I am good at working as part of a team I always follow codes of good clinical practice, and operating room protocols I can provide cognitive and emotional help to team members according to their needs When I am under pressure I can always retain a calm demeanour
Only the 12 statements were presented to participants, in a different order in the pretraining and post-training questionnaire. Participants were asked to respond to the statements on a five-point Likert scale (1, strongly disagree; 3, neutral; 5, strongly agree). NOTSS, Non-Technical Skills for Surgeons.
self-efficacy, as this scenario was used by all participants in both simulation sessions.
Statistical analysis Videos were rated independently by two assessors using the NOTSS scoring system. Scores were converted to numerical values to facilitate detailed analysis, and were averaged across assessors to simplify the analyses. Self-efficacy ratings were converted to numbers, with the lowest possible total score being 12 and the highest 60. Demographic variables were compared using t tests or Fisher’s exact tests. Significance of self-efficacy scores was tested by permutation tests, using the coin package13 in R statistical software14 . Pearson correlation was used to assess the relationship between video scores and self-assessed efficacy in non-technical skills. Statistical significance was defined as α < 0⋅050. Results
Forty participants were enrolled and all completed the study. Seventeen attended the non-technical skills workshop between simulation sessions, and 23 participants had no workshop during the study. There was no statistically significant difference between these two groups regarding participants’ age, surgical experience and surgical specialty (Table 2). Correlation coefficients between the assessors’ NOTSS scores across all scenarios ranged between 0⋅3 and 0⋅6 (all significantly greater than zero at P < 0⋅001), with the surgeon assessor consistently scoring higher than the psychologist assessor. © 2015 BJS Society Ltd Published by John Wiley & Sons Ltd
Table 2
Characteristics of participants
Age (years)* Sex ratio (M : F) Training level SET 1 SET 2 SET 3 SET 4 SET 5 SET 6 Fellow Surgical specialty Cardiothoracic ENT General Neurosurgery Orthopaedic Plastic Urology Vascular Order of scenarios 1 then 2 2 then 1
Simulation (n = 23)
Simulation plus workshop (n = 17)
32⋅9(3⋅9) 19 : 4
34⋅2(4⋅5) 15 : 2
5 4 3 0 7 0 4
3 6 1 1 1 1 4
1 2 12 1 1 3 1 2
0 0 10 1 1 0 3 2
11 12
9 8
P† 0⋅337‡ 1⋅000 0⋅251
0⋅581
1⋅000
*Values are mean(s.d.). SET, Surgical Education and Training; ENT, ear, nose and throat. †Fisher’s exact test, except ‡t test.
Self-efficacy in non-technical skills domains The initial mean overall self-efficacy score in non-technical skills reported by participants was 47⋅55 (maximum 60) (Table 3). After the training programme, the mean overall self-efficacy reported was 48⋅35. For overall and for all non-technical skills categories the post-training score was higher, but the difference did not reach statistical www.bjs.co.uk
BJS 2015; 102: 708–715
Perceived self-efficacy in operating theatre non-technical skills
Table 3
711
Pretraining and post-training perceived self-efficacy
15
scores Post-training self-efficacy score
P*
47⋅55(0⋅76) 11⋅90(0⋅23) 12⋅20(0⋅21) 11⋅95(0⋅21) 11⋅50(0⋅23)
48⋅35(0⋅65) 12⋅23(0⋅18) 12⋅33(0⋅17) 12⋅15(0⋅21) 11⋅65(0⋅20)
0⋅122 0⋅197 0⋅429 0⋅394 0⋅425
10
Values are mean(s.e.m.). *Permutation tests (refers to the mean of the individual changes).
Change in self-efficacy and non-technical skills performance for scenario 3 from simulation session 1 to simulation session 2
Table 4
Self-efficacy score
Junior trainees Senior trainees Fellows All
5
0
–5
–10
Scenario 3 performance score
n
Increase
P*
Increase
P*
18 14 8 40
0⋅06 1⋅71 0⋅88 0⋅80
0⋅753 0⋅031 0⋅350 0⋅122
5⋅06 4⋅07 3⋅38 4⋅44
0⋅003 0⋅023 0⋅279 < 0⋅001
*Permution tests.
significance. Both before and after training, participants reported lower self-efficacy in leadership skills than in decision-making (P = 0⋅025 and P = 0⋅023 respectively). The two statements for which participants scored lowest in both pretraining and post-training surveys were: ‘I can provide cognitive and emotional help to team members according to their needs’ and ‘When I am under pressure I can always retain a calm demeanour’. Participants’ rating of self-efficacy concerning non-technical skills was similar across training grades before training (junior 48⋅57, senior 45⋅72, fellow 48⋅52; P = 0⋅676) and did not change after training (junior 48⋅62, senior 47⋅42, fellow 49⋅42; P = 0⋅857) (Table 4). Participants with an initial self-efficacy score below average tended to have a positive change and those with higher self-efficacy tended to have a negative change in their perception of skills after the intervention (Fig. 1). The addition of a non-technical skills workshop to simulation-based training with feedback had no impact on participants’ confidence in their skills. There was no significant difference in the initial self-efficacy scores for the simulation and simulation plus workshop groups (46⋅94 and 48⋅41 respectively; P = 0⋅291). Nor was there a significant difference between their change in pretraining and post-training scores (P = 0⋅466). For all NOTSS categories combined, the changes in self-efficacy were not © 2015 BJS Society Ltd Published by John Wiley & Sons Ltd
Change in self-efficacy score
Overall Situation awareness Decision-making Communication and teamwork Leadership
Pretraining self-efficacy score
–15 35
40
45
50
55
60
Pretraining self-efficacy score
Change in self-efficacy against initial self-efficacy scores. The slope of the regression line is −0⋅510 (P < 0⋅001), demonstrating that the difference between the two scores tends to be positive for those with lower pretraining scores and negative for those with higher pretraining scores
Fig. 1
significant for either group; the change was 0⋅22 in the simulation group (P = 0⋅478) and 1⋅59 in the simulation plus workshop group (P = 0⋅091).
Relationship between participants’ self-efficacy rating of non-technical skills and skills demonstrated during simulation Participants’ non-technical skills performance scores were significantly higher in the second simulation session compared with the first session for two scenarios (scenario 1, P = 0⋅001; scenario 3, P < 0⋅001). For the repeated scenario, the less experienced the participant, the greater the improvement in score from simulation session 1 to simulation session 2 (Table 4). There was no significant correlation between participants’ initial self-efficacy rating and their performance in the simulated scenarios in the first simulation session (Fig. 2). Likewise, there was no significant correlation between participants’ final self-efficacy concerning non-technical skills and their performance in the simulated scenarios in the second simulation session, even though there was a significant improvement in their non-technical skills scores (Fig. S1, supporting information). Scenario 3 was completed in both the first and second simulation sessions. The correlation between the differences in participants’ score in scenario 3 was compared www.bjs.co.uk
BJS 2015; 102: 708–715
G. Pena, M. Altree, J. Field, M. J. W. Thomas, P. Hewett, W. Babidge and G. J. Maddern
60
60
55
55 Pretraining self-efficacy score
Pretraining self-efficacy score
712
50
45
40
35 20
50
45
40
25
30
35
35 30
40
35
Scenario 1 score
a
45
40 Scenario 2 score
b
Scenario 1 then scenario 2
Scenario 2 then scenario 1
60
Pretraining self-efficacy score
55
50
45
40
35 20
25
30
35
40
45
50
Pretraining scenario 3 score
c
Scenario 3 repeated
Correlation between pretraining self-efficacy and performance in the first simulation session. In the first simulation session, scenarios 1 and 2 were each attempted by 20 participants, whereas scenario 3 was attempted by all 40 participants. The correlation between pretraining participants’ self-efficacy and simulation performance was a r = −0⋅060 (P = 0⋅790) for scenario 1, b r = 0⋅051 (P = 0⋅840) for scenario 2 and c r = 0⋅086 (P = 0⋅596) for scenario 3
Fig. 2
© 2015 BJS Society Ltd Published by John Wiley & Sons Ltd
www.bjs.co.uk
BJS 2015; 102: 708–715
Perceived self-efficacy in operating theatre non-technical skills
against the change in participants’ perceived self-efficacy regarding non-technical skills. No correlation was found overall (P = 0⋅485) or for any group of participants by experience level (P = 0⋅721 for junior trainees, P = 0⋅600 for senior trainees and P = 0⋅557 for fellows) (Fig. S2, supporting information). Discussion
This study investigated surgical trainees’ and newly qualified surgeons’ perceptions of self-efficacy concerning their non-technical skills, and the relationship between this and their non-technical skills performance in simulated scenarios, before and after a formal training programme. In a survey of 4136 surgical trainees in North America assessing overall confidence in surgical skills (not restricted to technical or non-technical skills), those more advanced in training reported a greater confidence in their surgical skills15 . It was expected that there would be similar results in the present study. However, there was no significant difference in participants’ perceived self-efficacy for any of the non-technical skill categories according to training level. Junior trainees reported feeling as confident as newly qualified surgeons about their non-technical abilities, despite the larger operative experience of the latter group. This finding suggests that current surgical training in Australia does not modify non-technical skills confidence. This may be because confidence in non-technical abilities requires considerable practice and experience to develop, or due to a lack of formal training in this area. Only fellows at the beginning of their surgical careers, with up to 3 years of experience after completion of surgical training, were included in this study. Excellence in surgery is a result of lifelong learning16,17 . Surgeons continue to gain experience after the completion of formal training and it may be during this time that many consolidate self-efficacy regarding non-technical skills. This may be particularly pertinent for leadership skills, the non-technical skill category participants reported being less confident about, in both the pretraining and post-training surveys. Trainees may not see themselves in leadership roles, because in critical situations there is usually a supervisor or more experienced colleague present. Non-technical skills are usually not taught explicitly, or assessed during surgical training. Trainees commonly develop these skills by simply observing senior colleagues and rarely receive feedback on their own non-technical skills performance18,19 . As pointed out by Bandura20 : ‘If one does not know what demands must be fulfilled in a given endeavour, one cannot accurately judge whether one © 2015 BJS Society Ltd Published by John Wiley & Sons Ltd
713
has the requisite abilities to perform the task’. Therefore, it is perhaps not surprising that there was no significant correlation between participants’ pretraining self-efficacy and initial non-technical skills performance. The study intervention intended to enable participants to gain insight into their skills by improving competence, thus providing the metacognitive skills necessary to be able to critique performance. All participants engaged in simulation undertook one-on-one debriefing sessions facilitated by an expert, with the opportunity to watch recordings of themselves. In addition, 17 participants attended a 1-day non-technical skills (RACS NTS) workshop. This course explores the NOTSS behaviour rating system, developed by the University of Aberdeen and the Royal College of Surgeons of Edinburgh, which aims to give attendees a better understanding of how non-technical skills underpin safer operative performance. It was hypothesized that there would be a significant positive correlation between participants’ post-training self-efficacy and their performance in the second simulation session (final performance). Although it was not possible to demonstrate this positive correlation, the results nevertheless suggest that participants generally became more critical of their abilities as a result of the simulation intervention. Although a significant improvement in performance from the first to the second simulation session was noted for two of the scenarios, there was no significant improvement in perceived self-efficacy overall, or by non-technical skill category. This was particularly noticeable for the less experienced group of participants. Junior trainees gained the greatest improvement in score from simulation session 1 to simulation session 2 for the repeated scenario, but had the smallest absolute growth in self-efficacy. There is evidence that the training programme had a favourable effect, modifying both inflated and low self-efficacy. The participants with initially depressed self-efficacy tended to gain confidence in their skills, whereas those with extreme confidence tended to become more critical about their skills after training. Some participants with low perceived self-efficacy performed very well, whereas others with high perceived self-efficacy performed poorly. A mismatch between perceived self-efficacy in non-technical skills performance was identified across the three clinical scenarios with variable difficulty levels, reinforcing the finding that surgical trainees and new surgeons lack insight into their non-technical skills. A major problem with lack of insight is that it usually leads to inflated self-assessment21,22 . This situation is well illustrated by the sociologist Bosk23 , who interviewed neurosurgical trainees who had resigned or been fired; these trainees believed they hardly ever www.bjs.co.uk
BJS 2015; 102: 708–715
714
G. Pena, M. Altree, J. Field, M. J. W. Thomas, P. Hewett, W. Babidge and G. J. Maddern
made mistakes. Noteworthy, in the present study, the participant who consistently received the lowest score for non-technical skill performance in the simulated scenarios reported one of the highest perceived self-efficacy scores. The ability to recognize one’s limits and to be aware when making mistakes (in both technical and non-technical domains) is essential for surgeons, and it is desirable for professional development. The lack of correlation between perceived self-efficacy and performance in this research study concurs with data in the literature. In a meta-analysis examining the relationship between self-efficacy and work-related performance, Stajkovic24 reported that the magnitude of the correlation decreased with increasing task complexity. Few studies have evaluated self-efficacy (personal judgement of how well one can execute a course of action required to deal with prospective situations) and performance in surgery. The studies that tested the relationship between surgical trainees’ and surgeons’ self-assessment regarding non-technical skills performance in a retrospective situation, both inside and outside the operating room, and external evaluation reported poor correlations25 – 27 . This study has a number of limitations, including those inherent to a self-report survey28 . Participants assessed their self-efficacy in the light of performance in an actual situation, whereas their performance in the study was measured in simulated scenarios. Furthermore, the surgical scenarios, although valuable as educational tools, did not represent routine situations in the operating room. They contained a variety of crises and stressors that are possible, but do not commonly occur together during surgery. The results may have been influenced by social practices, notably related to the hierarchy present in the operating room. In societies with a high level of hierarchy, people are expected to accept inequalities in power and are less likely to initiate communication and speak up29 . This is especially true for the junior members of the culture, such as trainees. Therefore, it is expected that trainees/new surgeons would appraise their efficacy differently depending on the hierarchy of the system they are familiar with. Specifically, participants from more hierarchical systems than Australia are expected to report a lower sense of personal efficacy. Finally, there is still no evidence that non-technical skill performance in a simulation represents participants’ performance in an operating theatre. Further investigation of whether trainees’ and surgeon’s self-efficacy regarding non-technical skills correlates with their real performance in the operating room would be valuable. The research design would have been stronger if participants had been allocated randomly to groups, rather than allocation being
based on participant availability. The unavailability of several surgeons and surgical trainees to attend the RACS NTS workshop on the appointed days made randomization impossible; however, there was no significant difference in the demographic characteristics and initial self-efficacy scores of the two groups. This study suggests that new surgeons and surgical trainees may lack insight into their non-technical skill behaviours and abilities. This must be addressed in training programmes. A formal educational experience using simulation and performance feedback appears to be an appropriate way to equip surgical trainees and new surgeons with the knowledge and skills to improve performance and heighten awareness in the non-technical area.
© 2015 BJS Society Ltd Published by John Wiley & Sons Ltd
www.bjs.co.uk
Acknowledgments
The authors acknowledge the contributions made by P. Lee, R. McGuire, D. Sainsbury, S. Nair, A. Wallace and L. Waring. This research was made possible by funding from the Australian Government Department of Health. Disclosure: The authors declare no conflict of interest. References 1 Rethans J, Norcini J, Barón-Maldonado M, Blackmore D, Jolly B, LaDuca T et al. The relationship between competence and performance: implications for assessing practice performance. Med Educ 2002; 36: 901–909. 2 Royal Australasian College of Surgeons. Becoming a Competent and Proficient Surgeon: Training Standards for the Nine RACS Competencies; 2012. http://www.surgeons.org/ media/18726523/mnl_2012-02-24_training_standards_ final_1.pdf [accessed 1 November 2013]. 3 Bandura A. Self-efficacy mechanism in human agency. American Psychologist 1982; 37: 122–147. 4 Bandura A. Self-efficacy: toward a unifing theory of behavioural change. Psychol Rev 1977; 84: 191–215. 5 Bandura A, Adams N, Hardy A, Howells G. Test of the generality of self-efficacy theory. Cognit Ther Res 1980; 4: 39–66. 6 Bandura A. Perceived self-efficacy in cognitive development and functioning. Educational Psychologist 1993; 28: 117–148. 7 Flin R, Yule S, Paterson-Brown S, Maran N, Rowley D. Teaching surgeons about non-technical skills. Surgeon 2007; 5: 1098–1104. 8 Yule S, Flin R, Paterson-Brown S, Maran N, Rowley D. Development of a rating system for surgeons’ non-technical skills. Med Educ 2006; 40: 1098–1104. 9 Yule S, Flin R, Maran N, Rowley D, Youngson G, Paterson-Brown S. Surgeons’ non-technical skills in the operating room: reliability testing of the NOTSS behavior rating system. World J Surg 2008; 32: 548–556.
BJS 2015; 102: 708–715
Perceived self-efficacy in operating theatre non-technical skills
10 Schwarzer R, Jerusalem M. Generalized Self-Efficacy scale. In Measures in Health Psychology: a User’s Portfolio Casual and Control Beliefs, Weinman J, Wright S, Johnston M (eds). NFER-NELSON: Windsor, 1995; 35–37. 11 Schwarzer R. Measurement of Perceived Self-Efficacy: Psychometric for Cross-Cultural Research. Freien Universitat: Berlin, 1993. 12 Yule S, Paterson-Brown S. Surgeons’ non-technical skills. Surg Clin North Am 2012; 92: 37–50. 13 Hothorn TH, Hornik K, van de Wiel MA, Zeileis A. Implementing a class of permutation tests: the coin package. J Stat Softw 2008; 28: 1–23. 14 Team RDC. R: a Language and Environment for Statistical Computing. R Foundation for Statistical Computing: Vienna, 2008. 15 Bucholz E, Sue G, Yeo H, Roman S, Bell R Jr, Sosa J. Our trainees’ confidence: results from a national survey of 4136 US general surgery residents. Arch Surg 2011; 146: 907–914. 16 Gruen R, Walters D, Hollands M. Surgical wisdom. Br J Surg 2012; 99: 3–5. 17 Gawande A. Creating the educated surgeon in the 21st century. Am J Surg 2001; 181: 551–556. 18 Yule S, Flin R, Paterson-Brown S, Maran N. Non-technical skills for surgeons in the operating room: a review of the literature. Surgery 2006; 139: 140–149. 19 Ahmed M, Sevdalis N, Paige J, Paragi-Gururaja R, Nestel D, Arora S. Identifying best practice guidelines for debriefing in surgery: a tri-continental study. Am J Surg 2012; 203: 523–529. 20 Bandura A. Self-Efficacy: the Exercise of Control. Freeman: New York, 1997.
715
21 Kruger J, Dunning D. Unskilled and unaware of it: how difficulties in recognizing one’s own incompetence lead to inflated self-assessment. J Pers Soc Psychol 1999; 77: 1121–1134. 22 Hall J, Ellis C, Hamdorf J. Surgeons and cognitive processes. Br J Surg 2003; 90: 10–16. 23 Bosk C. What are the determinants of a competent neurosurgeon? Clin Neurosurg 1989; 1989: 474–486. 24 Stajkovic A. Self-efficacy and work-related performance: a meta-analysis. Psychol Bull 1998; 124: 240–261. 25 Moorthy K, Munz Y, Adams S, Pandey V, Darzi A; Imperial College–St Mary’s Hospital Simulation Group. Self-assessment of performance among surgical trainees during simulated procedures in a simulated operating theater. Am J Surg 2006; 192: 114–118. 26 Arora S, Miskovic D, Hull L, Moorthy K, Aggarwal R, Johannsson H et al. Self vs expert assessment of technical and non-technical skills in high fidelity simulation. Am J Surg 2011; 202: 500–506. 27 Hassett J, Zinnerstrom K, Nawotniak R, Schimpfhauser F, Dayton M. Utilization of standardized patients to evaluate clinical and interpersonal skills of surgical residents. Surgery 2006; 140: 633–639. 28 Fowler FJ Jr (ed.). Types of error in surveys. In Survey Research Methods (4th edn). Sage: Thousands Oaks, 2009; 11–18. 29 Oettingen G. Cross-cultural perspectives on self-efficacy. In Self-Efficacy in Changing Societies (1st edn), Bandura A (ed.). Cambridge University Press: New York, 1995; 149–176.
Supporting information
Additional supporting information may be found in the online version of this article: Fig. S1 Correlation between post-training self-efficacy and performance in the second simulation session (Word document) Fig. S2 Correlation between change in participants’ perceived self-efficacy and change in performance for scenario 3 by experience group (Word document)
If you wish to comment on this, or any other article published in the BJS, please visit the online Your Views section of the website (www.bjs.co.uk).
© 2015 BJS Society Ltd Published by John Wiley & Sons Ltd
www.bjs.co.uk
BJS 2015; 102: 708–715
Surgeons’ and trainees’ perceived self-efficacy in operating theatre non-technical skills G. Pena1,2 , M. Altree1 , J. Field3 , M. J. W. Thomas4 , P. Hewett2 , W. Babidge1,2 and G. J. Maddern1,2 1
Australian Safety and Efficacy Register of New Interventional Procedures – Surgical (ASERNIP-S), Royal Australasian College of Surgeons, University of Adelaide Discipline of Surgery, Queen Elizabeth Hospital, 3 John Field Consulting and 4 Westwood-Thomas Associates, Adelaide, South Australia, Australia Correspondence to: Professor G. J. Maddern, ASERNIP-S, Royal Australasian College of Surgeons, 199 Ward Street, North Adelaide, South Australia 5006, Australia (e-mail: [email protected]) 2
Background: An important factor that may influence an individual’s performance is self-efficacy, a
personal judgement of capability to perform a particular task successfully. This prospective study explored newly qualified surgeons’ and surgical trainees’ self-efficacy in non-technical skills compared with their non-technical skills performance in simulated scenarios. Methods: Participants undertook surgical scenarios challenging non-technical skills in two simulation sessions 6 weeks apart. Some participants attended a non-technical skills workshop between sessions. Participants completed pretraining and post-training surveys about their perceived self-efficacy in non-technical skills, which were analysed and compared with their performance in surgical scenarios in two simulation sessions. Change in performance between sessions was compared with any change in participants’ perceived self-efficacy. Results: There were 40 participants in all, 17 of whom attended the non-technical skills workshop. There was no significant difference in participants’ self-efficacy regarding non-technical skills from the pretraining to the post-training survey. However, there was a tendency for participants with the highest reported self-efficacy to adjust their score downwards after training and for participants with the lowest self-efficacy to adjust their score upwards. Although there was significant improvement in non-technical skills performance from the first to second simulation sessions, a correlation between participants’ self-efficacy and performance in scenarios in any of the comparisons was not found. Conclusion: The results suggest that new surgeons and surgical trainees have poor insight into their non-technical skills. Although it was not possible to correlate participants’ self-belief in their abilities directly with their performance in a simulation, in general they became more critical in appraisal of their abilities as a result of the intervention. Paper accepted 16 January 2015 Published online 19 March 2015 in Wiley Online Library (www.bjs.co.uk). DOI: 10.1002/bjs.9787
Introduction
Surgical competence refers to a surgeon’s knowledge and skills in both technical and non-technical domains, and encompasses what surgeons have learned and can do. Performance is what surgeons actually do in day-to-day practice. Performance depends on competence but is also influenced by individual and system-related factors1,2 . One of the individual-related factors that may influence surgical performance is the surgeon’s self-efficacy, which refers to personal judgement of how well the individual can execute a course of action to deal with prospective situations3 . Studies have found that self-efficacy influences a person’s decisions about what behaviours they choose to undertake, © 2015 BJS Society Ltd Published by John Wiley & Sons Ltd
the effort expended and sustained in attempting those behaviours, the emotional responses and ultimately the actual performance4,5 . Task execution requires not only skills but also an individual’s self-belief in their own ability to undertake the task well. Therefore, individuals with the same knowledge and skills may perform poorly, adequately or extraordinarily depending on fluctuations in self-efficacy thinking6 . Cognitive, social and personal resources that complement technical skills in surgery are referred to as non-technical skills7 . Self-efficacy in non-technical skills may influence performance and affect patient safety. A surgical trainee insecure about their leadership and team-related skills may avoid situations in which they BJS 2015; 102: 708–715
Perceived self-efficacy in operating theatre non-technical skills
are required to take control, as these situations cause significant stress and anxiety. Conversely, surgeons with inflated self-efficacy may make mistakes, as they fail to consider and discuss options owing to overconfidence in their decision-making processes, or may avoid calling for help in emergency situations. Either low or overinflated self-efficacy regarding non-technical skills may have deleterious consequences. An ideal training programme builds participants’ skills and proportionally their self-efficacy, improving performance by training new skills, and by enhancing the trainees’ belief in what they can do safely, corresponding with their skill level. Conversely, an exercise that expands self-efficacy but not the requisite skills can result in an individual with lack of insight and confidence greater than their abilities. The Royal Australasian College of Surgeons (RACS) developed a training programme to teach surgical trainees and newly qualified surgeons about non-technical skills using simulation and a non-technical skills workshop. This article explores the correlation between participants’ perceived self-efficacy regarding non-technical skills, both before and after training, and their performance in a simulated environment. It was hypothesized that after formal training there would be a significant positive correlation between participants’ self-efficacy and their performance in the simulated environment. It was considered that by making participants more knowledgeable about non-technical skills they would gain insight into their strengths and shortcomings, and judge more accurately their ability to perform non-technical skill-related tasks. Methods
The study was part of a project investigating the use of simulation for non-technical skills training in the operating room, which was conducted in two public hospitals in Adelaide, Australia, between April and July 2013. The study was approved by each institutional review board and written consent was obtained from all participants. Participants consisted of surgical trainees and surgeons who had been awarded Fellowship of the RACS in the previous 3 years (2009–2012) with no previous experience in operating room simulation for non-technical skill training. The study involved simulation of surgical scenarios requiring non-technical skills over two sessions, 6 weeks apart, incorporating a cross-over and a repeat scenario design. In the first session, each participant underwent simulation of either scenario 1 (respiratory distress during laparoscopy) or scenario 2 (possibility of wrong-site surgery) by selecting © 2015 BJS Society Ltd Published by John Wiley & Sons Ltd
709
a sealed opaque envelope at random, followed by scenario 3 (haemodynamic instability in a patient who is possibly a Jehovah Witness). According to participants’ availability, they attended a RACS non-technical skills workshop (NTS workshop) between the two simulation sessions. At the second simulation session each participant underwent simulation of the alternative scenario, 1 or 2, to that of their first session, and all participants repeated scenario 3. All of the simulated scenarios occurred within real operating rooms and lasted for between 7 and 10 min. Colleagues, playing roles similar to the real occupation, were used to compose a standard operating room team. The scenarios were created by experts in surgery, anaesthesia and human factors. For each scenario, a series of stressors (such as equipment problems, distractions, interruptions, team factors related to interpersonal issues and clinical factors) were introduced to challenge trainees’ non-technical skills. All simulated scenarios were video-recorded and assessed after the data collection phase was complete by two independent assessors using the Non-Technical Skills for Surgeons (NOTSS) scoring system8,9 . The independent assessors, one consultant surgeon and one psychologist crew resource management expert, had received training in the use of the scoring tool and evaluated the videos blinded to each cohort and simulation session. All scenarios were followed by a one-on-one debriefing for 20–30 min, facilitated by a medical doctor with expertise in healthcare communication and the field of human factors. Participants were asked to complete a pretraining and post-training questionnaire about their perceived self-efficacy regarding non-technical skills. The pretraining questionnaire was delivered online in conjunction with a demographic questionnaire before simulation session 1. The post-training questionnaire was delivered immediately after simulation session 2. The questionnaire was modelled on the General Self Efficacy scale developed by Schwarzer and Jerusalem10,11 , and included 12 statements based on elements of the NOTSS taxonomy7,12 to be answered on a five-point Likert scale. The statements reflected participants’ optimistic self-belief about their non-technical skills concerning situation awareness, decision-making, communication and teamwork, and leadership (Table 1). The order of the statements presented to participants was altered from the first to the second questionnaire. Participants’ pretraining and post-training self-efficacy scores were analysed and compared with their nontechnical skills performances in the scenarios during simulation session 1 and simulation session 2 respectively. Furthermore, any change in performance in scenario 3 was correlated with the change in participants’ perceived www.bjs.co.uk
BJS 2015; 102: 708–715
710
Table 1
G. Pena, M. Altree, J. Field, M. J. W. Thomas, P. Hewett, W. Babidge and G. J. Maddern
Participants’ perceived confidence questionnaire
NOTSS category Situation awareness
Decision-making
Communication and teamwork
Leadership
Statements I always seek information in the operating room concerning the operative findings and from the operating room environment, equipment and people I am able to interpret the information gathered and use it to understand the match or mismatch between the situation and the expected state I can predict what may happen in the near future as a result of possible actions, interventions or non-intervention I can assess the threats and benefits of potential courses of action and to solve a problem When I choose a solution to a problem, I always let all relevant personnel know the chosen option I am flexible and I can change plans if required to cope with changing circumstances During surgery, I am able to exchange information in a timely manner to aid establishment of a shared understanding among team members I always ensure that the team not only has the relevant information to carry out the operation, but that they understand it I am good at working as part of a team I always follow codes of good clinical practice, and operating room protocols I can provide cognitive and emotional help to team members according to their needs When I am under pressure I can always retain a calm demeanour
Only the 12 statements were presented to participants, in a different order in the pretraining and post-training questionnaire. Participants were asked to respond to the statements on a five-point Likert scale (1, strongly disagree; 3, neutral; 5, strongly agree). NOTSS, Non-Technical Skills for Surgeons.
self-efficacy, as this scenario was used by all participants in both simulation sessions.
Statistical analysis Videos were rated independently by two assessors using the NOTSS scoring system. Scores were converted to numerical values to facilitate detailed analysis, and were averaged across assessors to simplify the analyses. Self-efficacy ratings were converted to numbers, with the lowest possible total score being 12 and the highest 60. Demographic variables were compared using t tests or Fisher’s exact tests. Significance of self-efficacy scores was tested by permutation tests, using the coin package13 in R statistical software14 . Pearson correlation was used to assess the relationship between video scores and self-assessed efficacy in non-technical skills. Statistical significance was defined as α < 0⋅050. Results
Forty participants were enrolled and all completed the study. Seventeen attended the non-technical skills workshop between simulation sessions, and 23 participants had no workshop during the study. There was no statistically significant difference between these two groups regarding participants’ age, surgical experience and surgical specialty (Table 2). Correlation coefficients between the assessors’ NOTSS scores across all scenarios ranged between 0⋅3 and 0⋅6 (all significantly greater than zero at P < 0⋅001), with the surgeon assessor consistently scoring higher than the psychologist assessor. © 2015 BJS Society Ltd Published by John Wiley & Sons Ltd
Table 2
Characteristics of participants
Age (years)* Sex ratio (M : F) Training level SET 1 SET 2 SET 3 SET 4 SET 5 SET 6 Fellow Surgical specialty Cardiothoracic ENT General Neurosurgery Orthopaedic Plastic Urology Vascular Order of scenarios 1 then 2 2 then 1
Simulation (n = 23)
Simulation plus workshop (n = 17)
32⋅9(3⋅9) 19 : 4
34⋅2(4⋅5) 15 : 2
5 4 3 0 7 0 4
3 6 1 1 1 1 4
1 2 12 1 1 3 1 2
0 0 10 1 1 0 3 2
11 12
9 8
P† 0⋅337‡ 1⋅000 0⋅251
0⋅581
1⋅000
*Values are mean(s.d.). SET, Surgical Education and Training; ENT, ear, nose and throat. †Fisher’s exact test, except ‡t test.
Self-efficacy in non-technical skills domains The initial mean overall self-efficacy score in non-technical skills reported by participants was 47⋅55 (maximum 60) (Table 3). After the training programme, the mean overall self-efficacy reported was 48⋅35. For overall and for all non-technical skills categories the post-training score was higher, but the difference did not reach statistical www.bjs.co.uk
BJS 2015; 102: 708–715
Perceived self-efficacy in operating theatre non-technical skills
Table 3
711
Pretraining and post-training perceived self-efficacy
15
scores Post-training self-efficacy score
P*
47⋅55(0⋅76) 11⋅90(0⋅23) 12⋅20(0⋅21) 11⋅95(0⋅21) 11⋅50(0⋅23)
48⋅35(0⋅65) 12⋅23(0⋅18) 12⋅33(0⋅17) 12⋅15(0⋅21) 11⋅65(0⋅20)
0⋅122 0⋅197 0⋅429 0⋅394 0⋅425
10
Values are mean(s.e.m.). *Permutation tests (refers to the mean of the individual changes).
Change in self-efficacy and non-technical skills performance for scenario 3 from simulation session 1 to simulation session 2
Table 4
Self-efficacy score
Junior trainees Senior trainees Fellows All
5
0
–5
–10
Scenario 3 performance score
n
Increase
P*
Increase
P*
18 14 8 40
0⋅06 1⋅71 0⋅88 0⋅80
0⋅753 0⋅031 0⋅350 0⋅122
5⋅06 4⋅07 3⋅38 4⋅44
0⋅003 0⋅023 0⋅279 < 0⋅001
*Permution tests.
significance. Both before and after training, participants reported lower self-efficacy in leadership skills than in decision-making (P = 0⋅025 and P = 0⋅023 respectively). The two statements for which participants scored lowest in both pretraining and post-training surveys were: ‘I can provide cognitive and emotional help to team members according to their needs’ and ‘When I am under pressure I can always retain a calm demeanour’. Participants’ rating of self-efficacy concerning non-technical skills was similar across training grades before training (junior 48⋅57, senior 45⋅72, fellow 48⋅52; P = 0⋅676) and did not change after training (junior 48⋅62, senior 47⋅42, fellow 49⋅42; P = 0⋅857) (Table 4). Participants with an initial self-efficacy score below average tended to have a positive change and those with higher self-efficacy tended to have a negative change in their perception of skills after the intervention (Fig. 1). The addition of a non-technical skills workshop to simulation-based training with feedback had no impact on participants’ confidence in their skills. There was no significant difference in the initial self-efficacy scores for the simulation and simulation plus workshop groups (46⋅94 and 48⋅41 respectively; P = 0⋅291). Nor was there a significant difference between their change in pretraining and post-training scores (P = 0⋅466). For all NOTSS categories combined, the changes in self-efficacy were not © 2015 BJS Society Ltd Published by John Wiley & Sons Ltd
Change in self-efficacy score
Overall Situation awareness Decision-making Communication and teamwork Leadership
Pretraining self-efficacy score
–15 35
40
45
50
55
60
Pretraining self-efficacy score
Change in self-efficacy against initial self-efficacy scores. The slope of the regression line is −0⋅510 (P < 0⋅001), demonstrating that the difference between the two scores tends to be positive for those with lower pretraining scores and negative for those with higher pretraining scores
Fig. 1
significant for either group; the change was 0⋅22 in the simulation group (P = 0⋅478) and 1⋅59 in the simulation plus workshop group (P = 0⋅091).
Relationship between participants’ self-efficacy rating of non-technical skills and skills demonstrated during simulation Participants’ non-technical skills performance scores were significantly higher in the second simulation session compared with the first session for two scenarios (scenario 1, P = 0⋅001; scenario 3, P < 0⋅001). For the repeated scenario, the less experienced the participant, the greater the improvement in score from simulation session 1 to simulation session 2 (Table 4). There was no significant correlation between participants’ initial self-efficacy rating and their performance in the simulated scenarios in the first simulation session (Fig. 2). Likewise, there was no significant correlation between participants’ final self-efficacy concerning non-technical skills and their performance in the simulated scenarios in the second simulation session, even though there was a significant improvement in their non-technical skills scores (Fig. S1, supporting information). Scenario 3 was completed in both the first and second simulation sessions. The correlation between the differences in participants’ score in scenario 3 was compared www.bjs.co.uk
BJS 2015; 102: 708–715
G. Pena, M. Altree, J. Field, M. J. W. Thomas, P. Hewett, W. Babidge and G. J. Maddern
60
60
55
55 Pretraining self-efficacy score
Pretraining self-efficacy score
712
50
45
40
35 20
50
45
40
25
30
35
35 30
40
35
Scenario 1 score
a
45
40 Scenario 2 score
b
Scenario 1 then scenario 2
Scenario 2 then scenario 1
60
Pretraining self-efficacy score
55
50
45
40
35 20
25
30
35
40
45
50
Pretraining scenario 3 score
c
Scenario 3 repeated
Correlation between pretraining self-efficacy and performance in the first simulation session. In the first simulation session, scenarios 1 and 2 were each attempted by 20 participants, whereas scenario 3 was attempted by all 40 participants. The correlation between pretraining participants’ self-efficacy and simulation performance was a r = −0⋅060 (P = 0⋅790) for scenario 1, b r = 0⋅051 (P = 0⋅840) for scenario 2 and c r = 0⋅086 (P = 0⋅596) for scenario 3
Fig. 2
© 2015 BJS Society Ltd Published by John Wiley & Sons Ltd
www.bjs.co.uk
BJS 2015; 102: 708–715
Perceived self-efficacy in operating theatre non-technical skills
against the change in participants’ perceived self-efficacy regarding non-technical skills. No correlation was found overall (P = 0⋅485) or for any group of participants by experience level (P = 0⋅721 for junior trainees, P = 0⋅600 for senior trainees and P = 0⋅557 for fellows) (Fig. S2, supporting information). Discussion
This study investigated surgical trainees’ and newly qualified surgeons’ perceptions of self-efficacy concerning their non-technical skills, and the relationship between this and their non-technical skills performance in simulated scenarios, before and after a formal training programme. In a survey of 4136 surgical trainees in North America assessing overall confidence in surgical skills (not restricted to technical or non-technical skills), those more advanced in training reported a greater confidence in their surgical skills15 . It was expected that there would be similar results in the present study. However, there was no significant difference in participants’ perceived self-efficacy for any of the non-technical skill categories according to training level. Junior trainees reported feeling as confident as newly qualified surgeons about their non-technical abilities, despite the larger operative experience of the latter group. This finding suggests that current surgical training in Australia does not modify non-technical skills confidence. This may be because confidence in non-technical abilities requires considerable practice and experience to develop, or due to a lack of formal training in this area. Only fellows at the beginning of their surgical careers, with up to 3 years of experience after completion of surgical training, were included in this study. Excellence in surgery is a result of lifelong learning16,17 . Surgeons continue to gain experience after the completion of formal training and it may be during this time that many consolidate self-efficacy regarding non-technical skills. This may be particularly pertinent for leadership skills, the non-technical skill category participants reported being less confident about, in both the pretraining and post-training surveys. Trainees may not see themselves in leadership roles, because in critical situations there is usually a supervisor or more experienced colleague present. Non-technical skills are usually not taught explicitly, or assessed during surgical training. Trainees commonly develop these skills by simply observing senior colleagues and rarely receive feedback on their own non-technical skills performance18,19 . As pointed out by Bandura20 : ‘If one does not know what demands must be fulfilled in a given endeavour, one cannot accurately judge whether one © 2015 BJS Society Ltd Published by John Wiley & Sons Ltd
713
has the requisite abilities to perform the task’. Therefore, it is perhaps not surprising that there was no significant correlation between participants’ pretraining self-efficacy and initial non-technical skills performance. The study intervention intended to enable participants to gain insight into their skills by improving competence, thus providing the metacognitive skills necessary to be able to critique performance. All participants engaged in simulation undertook one-on-one debriefing sessions facilitated by an expert, with the opportunity to watch recordings of themselves. In addition, 17 participants attended a 1-day non-technical skills (RACS NTS) workshop. This course explores the NOTSS behaviour rating system, developed by the University of Aberdeen and the Royal College of Surgeons of Edinburgh, which aims to give attendees a better understanding of how non-technical skills underpin safer operative performance. It was hypothesized that there would be a significant positive correlation between participants’ post-training self-efficacy and their performance in the second simulation session (final performance). Although it was not possible to demonstrate this positive correlation, the results nevertheless suggest that participants generally became more critical of their abilities as a result of the simulation intervention. Although a significant improvement in performance from the first to the second simulation session was noted for two of the scenarios, there was no significant improvement in perceived self-efficacy overall, or by non-technical skill category. This was particularly noticeable for the less experienced group of participants. Junior trainees gained the greatest improvement in score from simulation session 1 to simulation session 2 for the repeated scenario, but had the smallest absolute growth in self-efficacy. There is evidence that the training programme had a favourable effect, modifying both inflated and low self-efficacy. The participants with initially depressed self-efficacy tended to gain confidence in their skills, whereas those with extreme confidence tended to become more critical about their skills after training. Some participants with low perceived self-efficacy performed very well, whereas others with high perceived self-efficacy performed poorly. A mismatch between perceived self-efficacy in non-technical skills performance was identified across the three clinical scenarios with variable difficulty levels, reinforcing the finding that surgical trainees and new surgeons lack insight into their non-technical skills. A major problem with lack of insight is that it usually leads to inflated self-assessment21,22 . This situation is well illustrated by the sociologist Bosk23 , who interviewed neurosurgical trainees who had resigned or been fired; these trainees believed they hardly ever www.bjs.co.uk
BJS 2015; 102: 708–715
714
G. Pena, M. Altree, J. Field, M. J. W. Thomas, P. Hewett, W. Babidge and G. J. Maddern
made mistakes. Noteworthy, in the present study, the participant who consistently received the lowest score for non-technical skill performance in the simulated scenarios reported one of the highest perceived self-efficacy scores. The ability to recognize one’s limits and to be aware when making mistakes (in both technical and non-technical domains) is essential for surgeons, and it is desirable for professional development. The lack of correlation between perceived self-efficacy and performance in this research study concurs with data in the literature. In a meta-analysis examining the relationship between self-efficacy and work-related performance, Stajkovic24 reported that the magnitude of the correlation decreased with increasing task complexity. Few studies have evaluated self-efficacy (personal judgement of how well one can execute a course of action required to deal with prospective situations) and performance in surgery. The studies that tested the relationship between surgical trainees’ and surgeons’ self-assessment regarding non-technical skills performance in a retrospective situation, both inside and outside the operating room, and external evaluation reported poor correlations25 – 27 . This study has a number of limitations, including those inherent to a self-report survey28 . Participants assessed their self-efficacy in the light of performance in an actual situation, whereas their performance in the study was measured in simulated scenarios. Furthermore, the surgical scenarios, although valuable as educational tools, did not represent routine situations in the operating room. They contained a variety of crises and stressors that are possible, but do not commonly occur together during surgery. The results may have been influenced by social practices, notably related to the hierarchy present in the operating room. In societies with a high level of hierarchy, people are expected to accept inequalities in power and are less likely to initiate communication and speak up29 . This is especially true for the junior members of the culture, such as trainees. Therefore, it is expected that trainees/new surgeons would appraise their efficacy differently depending on the hierarchy of the system they are familiar with. Specifically, participants from more hierarchical systems than Australia are expected to report a lower sense of personal efficacy. Finally, there is still no evidence that non-technical skill performance in a simulation represents participants’ performance in an operating theatre. Further investigation of whether trainees’ and surgeon’s self-efficacy regarding non-technical skills correlates with their real performance in the operating room would be valuable. The research design would have been stronger if participants had been allocated randomly to groups, rather than allocation being
based on participant availability. The unavailability of several surgeons and surgical trainees to attend the RACS NTS workshop on the appointed days made randomization impossible; however, there was no significant difference in the demographic characteristics and initial self-efficacy scores of the two groups. This study suggests that new surgeons and surgical trainees may lack insight into their non-technical skill behaviours and abilities. This must be addressed in training programmes. A formal educational experience using simulation and performance feedback appears to be an appropriate way to equip surgical trainees and new surgeons with the knowledge and skills to improve performance and heighten awareness in the non-technical area.
© 2015 BJS Society Ltd Published by John Wiley & Sons Ltd
www.bjs.co.uk
Acknowledgments
The authors acknowledge the contributions made by P. Lee, R. McGuire, D. Sainsbury, S. Nair, A. Wallace and L. Waring. This research was made possible by funding from the Australian Government Department of Health. Disclosure: The authors declare no conflict of interest. References 1 Rethans J, Norcini J, Barón-Maldonado M, Blackmore D, Jolly B, LaDuca T et al. The relationship between competence and performance: implications for assessing practice performance. Med Educ 2002; 36: 901–909. 2 Royal Australasian College of Surgeons. Becoming a Competent and Proficient Surgeon: Training Standards for the Nine RACS Competencies; 2012. http://www.surgeons.org/ media/18726523/mnl_2012-02-24_training_standards_ final_1.pdf [accessed 1 November 2013]. 3 Bandura A. Self-efficacy mechanism in human agency. American Psychologist 1982; 37: 122–147. 4 Bandura A. Self-efficacy: toward a unifing theory of behavioural change. Psychol Rev 1977; 84: 191–215. 5 Bandura A, Adams N, Hardy A, Howells G. Test of the generality of self-efficacy theory. Cognit Ther Res 1980; 4: 39–66. 6 Bandura A. Perceived self-efficacy in cognitive development and functioning. Educational Psychologist 1993; 28: 117–148. 7 Flin R, Yule S, Paterson-Brown S, Maran N, Rowley D. Teaching surgeons about non-technical skills. Surgeon 2007; 5: 1098–1104. 8 Yule S, Flin R, Paterson-Brown S, Maran N, Rowley D. Development of a rating system for surgeons’ non-technical skills. Med Educ 2006; 40: 1098–1104. 9 Yule S, Flin R, Maran N, Rowley D, Youngson G, Paterson-Brown S. Surgeons’ non-technical skills in the operating room: reliability testing of the NOTSS behavior rating system. World J Surg 2008; 32: 548–556.
BJS 2015; 102: 708–715
Perceived self-efficacy in operating theatre non-technical skills
10 Schwarzer R, Jerusalem M. Generalized Self-Efficacy scale. In Measures in Health Psychology: a User’s Portfolio Casual and Control Beliefs, Weinman J, Wright S, Johnston M (eds). NFER-NELSON: Windsor, 1995; 35–37. 11 Schwarzer R. Measurement of Perceived Self-Efficacy: Psychometric for Cross-Cultural Research. Freien Universitat: Berlin, 1993. 12 Yule S, Paterson-Brown S. Surgeons’ non-technical skills. Surg Clin North Am 2012; 92: 37–50. 13 Hothorn TH, Hornik K, van de Wiel MA, Zeileis A. Implementing a class of permutation tests: the coin package. J Stat Softw 2008; 28: 1–23. 14 Team RDC. R: a Language and Environment for Statistical Computing. R Foundation for Statistical Computing: Vienna, 2008. 15 Bucholz E, Sue G, Yeo H, Roman S, Bell R Jr, Sosa J. Our trainees’ confidence: results from a national survey of 4136 US general surgery residents. Arch Surg 2011; 146: 907–914. 16 Gruen R, Walters D, Hollands M. Surgical wisdom. Br J Surg 2012; 99: 3–5. 17 Gawande A. Creating the educated surgeon in the 21st century. Am J Surg 2001; 181: 551–556. 18 Yule S, Flin R, Paterson-Brown S, Maran N. Non-technical skills for surgeons in the operating room: a review of the literature. Surgery 2006; 139: 140–149. 19 Ahmed M, Sevdalis N, Paige J, Paragi-Gururaja R, Nestel D, Arora S. Identifying best practice guidelines for debriefing in surgery: a tri-continental study. Am J Surg 2012; 203: 523–529. 20 Bandura A. Self-Efficacy: the Exercise of Control. Freeman: New York, 1997.
715
21 Kruger J, Dunning D. Unskilled and unaware of it: how difficulties in recognizing one’s own incompetence lead to inflated self-assessment. J Pers Soc Psychol 1999; 77: 1121–1134. 22 Hall J, Ellis C, Hamdorf J. Surgeons and cognitive processes. Br J Surg 2003; 90: 10–16. 23 Bosk C. What are the determinants of a competent neurosurgeon? Clin Neurosurg 1989; 1989: 474–486. 24 Stajkovic A. Self-efficacy and work-related performance: a meta-analysis. Psychol Bull 1998; 124: 240–261. 25 Moorthy K, Munz Y, Adams S, Pandey V, Darzi A; Imperial College–St Mary’s Hospital Simulation Group. Self-assessment of performance among surgical trainees during simulated procedures in a simulated operating theater. Am J Surg 2006; 192: 114–118. 26 Arora S, Miskovic D, Hull L, Moorthy K, Aggarwal R, Johannsson H et al. Self vs expert assessment of technical and non-technical skills in high fidelity simulation. Am J Surg 2011; 202: 500–506. 27 Hassett J, Zinnerstrom K, Nawotniak R, Schimpfhauser F, Dayton M. Utilization of standardized patients to evaluate clinical and interpersonal skills of surgical residents. Surgery 2006; 140: 633–639. 28 Fowler FJ Jr (ed.). Types of error in surveys. In Survey Research Methods (4th edn). Sage: Thousands Oaks, 2009; 11–18. 29 Oettingen G. Cross-cultural perspectives on self-efficacy. In Self-Efficacy in Changing Societies (1st edn), Bandura A (ed.). Cambridge University Press: New York, 1995; 149–176.
Supporting information
Additional supporting information may be found in the online version of this article: Fig. S1 Correlation between post-training self-efficacy and performance in the second simulation session (Word document) Fig. S2 Correlation between change in participants’ perceived self-efficacy and change in performance for scenario 3 by experience group (Word document)
If you wish to comment on this, or any other article published in the BJS, please visit the online Your Views section of the website (www.bjs.co.uk).
© 2015 BJS Society Ltd Published by John Wiley & Sons Ltd
www.bjs.co.uk
BJS 2015; 102: 708–715
