ORIGINAL REPORTS
Evaluation of Urology Residents’ Perception of Surgical Theater Educational Environment Saleh Binsaleh, MD, FRCS(C),* Abdulrahman Babaeer, MD,† Danny Rabah, MD, FRCS(C),* and Khaled Madbouly, MD‡ Department of Surgery, Faculty of Medicine, King Saud University, Riyadh, Saudi Arabia; †Department of Urology, King Abdulaziz Medical City, Riyadh, Saudi Arabia; and ‡Department of Urology, Prince Mohammed bin Abdulaziz Hospital, Riyadh, Saudi Arabia
*
PURPOSE: To evaluate surgical theater learning environ-
ment perception in urology residents in Saudi Arabia and to investigate association of learning environment perception and stages of residency program, sectors of health care system, and regions of Saudi Arabia. DESIGN: A cross-sectional survey using the surgical theater educational environment measure (STEEM) inventory. SETTINGS AND PARTICIPANTS: The STEEM inventory was used to measure theater learning environment perception of urology residents in Saudi Arabia. Respondents’ perception was compared regarding different residency stages, sectors of the health care system, and regions of Saudi Arabia. Internal reliability of the inventory was assessed using the Cronbach α coefficient. Correlation analysis was done using the Spearman ρ coefficient. RESULTS: Of 72 registered residents, 33 (45.8%) completed the questionnaire. The residents perceived their environment less than acceptable (135.9 ⫾ 16.7, 67.95%). No significant differences in perception were found among residents of different program stages, different sectors of health care system, or different regions in Saudi Arabia. Residents from the eastern region perceived the training and teaching domain better (p ¼ 0.025). The inventory showed a high internal consistency with a Cronbach α of 0.862. CONCLUSIONS: STEEM survey is an applicable and
reliable instrument for assessing the learning environment and training skills of urology residency program in Saudi Arabia. Urology residents in Saudi Arabia perceived the theater learning environment as less than ideal. The perceptions of theater learning environment did not change significantly among different stages of the program, different
Correspondence: Inquiries to Saleh Binsaleh, MD, FRCS(C), Department of Surgery, Faculty of Medicine, King Saud University, P.O. Box 36175, Riyadh, 1149, Saudi Arabia; fax: (96611) 467 9493; e-mail: [email protected]
sectors of health care system, or different training regions of Saudi Arabia assuring the uniformity of urology training all over Saudi Arabia. The training programs should address significant concerns and pay close attention to areas in surgical theater educational environment, which need development and enhancement, mainly planned fashion of training, supportive supervision and hospital environment, and proper coverage and management of workloads. ( J Surg C ]:]]]-]]]. J 2014 Association of Program Directors in Surgery. Published by Elsevier Inc. All rights reserved.) KEY WORDS: surgical theatre educational environment
measure (STEEM), educational environment, residency program, urology residents, Saudi Arabia COMPETENCIES: Medical Knowledge, Practice Based
Learning and Improvement, Systems Based Practice, Interpersonal Skills and Communication
INTRODUCTION Urology residency training in Saudi Arabia is a hospitalbased 5 years structured program, with central supervision of Saudi Commission for Health Specialties, in which the first year is for general surgery and surgical intensive care rotation. Residents who complete their residency training ultimately receive certification as a specialist. The educational environment is an important measure, which has a large effect on the satisfaction, achievement, and success of medical education.1 Positive learning climates have been associated with improved learner’s performance.2 To develop an environment that is conducive to learning, the major elements that contribute to the particular learning environment need to be identified and an instrument to measure that learning environment needs to be available to allow accurate assessment of the learning environment and to identify areas that require attention.3
Journal of Surgical Education & 2014 Association of Program Directors in Surgery. Published by 1931-7204/$30.00 Elsevier Inc. All rights reserved. http://dx.doi.org/10.1016/j.jsurg.2014.08.002
1
Surgical career is perceived by medical students to be one of high pressure and stress, with no control over one’s time and an income that is not adequate for the level of commitment and the amount of work.4 Furthermore, the operating theater is reported as a complex place with a hostile environment that is inconducive to learning. It is hard to plan and prepare teaching within the operating theater, especially during stressful emergency procedures.5 Urology continues to evolve and the skill set required of urologists has become multidimensional,6 adding to the complexity of urology theater. A number of studies have been conducted to explore students’ perceptions of their learning environment. Cassar3 developed surgical theatre educational environment measure (STEEM), an instrument that measures the learning environment in the surgical operating theater. The objective of the present study was to evaluate the surgical theater learning environment of urology residents in Saudi Arabia and to identify factors influencing their perception of this environment. The study also evaluated associations of surgical theater learning environment perception with stages of the residency program, different health care sectors, and different regions of Saudi Arabia. To our knowledge, this is the first study to evaluate urology residency surgical theater learning environment since its implementation in Saudi Arabia more than a decade ago. It is also the first to use STEEM inventory to evaluate surgical theater learning environment perception of urology residents in general.
METHODS The STEEM questionnaire was used to assess the perception of urology residents in Saudi Arabia toward the surgical theater learning environment. It consists of 40 statements with which the respondents were asked to indicate their agreement using a 5-point Likert scale ranging from strongly agree (5), agree (4), uncertain (3), disagree (2), to strongly disagree (1). The scoring was reversed for negative statements (statement numbers: 8, 11, 14, 16, 19, 22, 23, 26, 27, 28, 30, 31, 33, 34, 35, 36, 37, 38, and 40) so that for all items, the higher the score the more positive the perception. The overall possible maximum score of STEEM is 200 and the minimum score is 40. An overall score of 120 of 200 would indicate a neutral perception, any value 4120 would indicate a more satisfactory environment, whereas any value of o120 would indicate a less than satisfactory learning environment.3 STEEM is divided into 4 subscales: statements 1 to 13 cover trainees’ perceptions of their teaching and training with a maximum score of 65, statements 14 to 24 cover trainees’ perceptions of learning opportunities with a maximum score of 55, statements 25 to 32 cover trainees’ perceptions of the atmosphere in the 2
operating theater with a maximum score of 40, and statements 33 to 40 trainees’ cover perceptions on supervision, workload, and support with a maximum score of 40. The questionnaire was e-mailed by a third independent party using SurveyMonkey to all residents registered in the urology training program and the identity of the collected data was kept anonymous to the researchers. Trainees provided information regarding their age, sex, stage of training, in what region of the country they were doing the training, and in which health care sector. Responses of residents in the first residency year were excluded being involved in nonurology rotation year. Residents were further classified into juniors including residents in second and third residency years and seniors including residents of fourth and fifth residency years. Scores for each item and subscale were calculated and entered into a spreadsheet. Descriptive statistics were reported as mean ⫾ standard deviation. For comparative statistics, nonparametric method of Kruskal-Wallis was used to compare any significant perception differences among different residency stages, different regions of Saudi Arabia, and sectors of health care system.7 Only 1 resident responded from the northern region, so his response was excluded from the comparison of different regions in Saudi Arabia. Likewise, 3 respondents did not mention their sector of health care system and only 1 resident responded from the security forces health care sector and their responses were excluded from the comparisons of different health care systems. The Mann-Whitney test was used to assess any significant differences between junior and senior residents. The correlation analysis was done using the Spearman ρ coefficient to assess the association among the overall STEEM score and the 4 subscales and the association between each other.7 The reliability analysis was performed using the Cronbach α coefficient to measure the internal consistency of the whole questionnaire and each of the 4 subscales.8 Using the “α if item deleted,” Cronbach α was used to identify questions whose exclusion would improve the reliability. Data were analyzed using SPSS, version 18.
RESULTS Of 72 registered residents, 33 (45.8%) completed the questionnaire. One responder was in the first residency year and his response was excluded from the analysis. Responses of 32 residents; 1 (3.1%) woman and 31 (96.9%) men, with a mean age of 30 ⫾ 2.6 years were analyzed. Table 1 illustrates demographic details of the included respondents, their region of training in Saudi Arabia, and their main health care sector. Mean (⫾ standard deviation) scores for each item of the questionnaire, for the 4 subscales and for the overall inventory were summarized in Table 2. The 2 highest Journal of Surgical Education Volume ]/Number ] ] 2014
TABLE 1. Characteristics of Study Respondents Sex Male Female Residency stage R2 R3 R4 R5 Region in Saudi Arabia Central Western Eastern Southern Northern Sector of health service Ministry of health National Guard Hospital Armed Forces Hospital Security Forces Hospital University Hospital Missing
Number
Percentage
31 1
96.9 3.1
4 11 5 12
12.5 34.4 15.6 37.5
15 6 6 4 1
46.9 18.8 18.8 12.5 3
12 3 6 1 7 3
37.5 9.4 18.8 3.1 21.9 9.3
scoring items were no. 31 “I (do not) feel discriminated in the operating room because of my race,” and no. 30 “I (do not) feel discriminated in the operating room because of my sex.” The lowest ranked item was statement no. 38 “I get bleeped (paged-called through my pager) during operations.” There were no statistically significant differences in perception of the total score as well as the 4 subscale scores of theater learning environment among different stages of the residency program (Kruskal-Wallis, p 4 0.05). Even, when the residents were classified to juniors and seniors, the Mann-Whitney test showed no statistically significant difference in the total questionnaire score or in any of its subscales scores. However, junior residents responded positively more significantly to statement 5; “I understand what my trainer is trying to teach me” and statement 13 “My trainer’s criticism is constructive.” There was no statistically significant difference (KruskalWallis, p 4 0.05) in perception of theater learning environment among residents in different main sectors of the health care system regarding the total and the 4 subscales’ score. However, residents from the Ministry of Health care system were more significantly satisfied with the sufficiency of emergency procedures they are exposed to (statement 20, p ¼ 0.041). We identified significant differences among residents from different training regions of Saudi Arabia, excluding the northern region, regarding the training and teaching subscale score where residents from the eastern region perceived it better (p ¼ 0.025). They responded more significantly positive to statements 3, “My trainer is enthusiastic about teaching” (p ¼ 0.026); 12, “My trainer gives me feedback on my performance” (p ¼ 0.017); and Journal of Surgical Education Volume ]/Number ] ] 2014
14, “On this unit the type of operations performed are too complex for my level” (p ¼ 0.033). Western region residents responded significantly positive to statement 20, “The number of emergency procedures is sufficient for me to gain the right operative experience” (p ¼ 0.046) and they disagreed significantly to statement 19 “More senior trainees take my opportunities to operate” (p ¼ 0.004). All 4 subscales of STEEM correlated significantly and positively with the overall STEEM score ranging from 0.602 to 0.813. The subscale that correlated most strongly with overall satisfaction was teaching and training (Spearman ρ ¼ 0.813), followed by learning atmosphere (Spearman ρ ¼ 0.796). Learning opportunity subscale had the lowest correlation with the overall scale (Spearman ρ ¼ 0.602). Intercorrelation of the subscales with each other was given in Table 3. The Cronbach α coefficient was computed to measure the internal consistency of the overall questionnaire and each of the 4 subscales. For the whole questionnaire, the Cronbach α was 0.862. For the “teaching-and-training” subscale, the Cronbach α was 0.848, for learning opportunity it was 0.461, for atmosphere perception it was 0.761, and for supervision/workload/support subscale it was 0.666. Statements 8, 10, 11, and 12 (the teaching-and-learning domain); 15, 16, 19, 21, and 22 (the learning opportunity domain); 27, 28, 30, and 31(the atmosphere domain); and 34, 36, and 40 (supervision/workload/support domain) were not consistent with the rest of the scale and could be deleted (Table 4). On deletion of these factors from the reliability analysis, the Cronbach α coefficient increased for the entire inventory as well as the teaching and learning, the learning opportunity, and the supervision/workload/support subscales to 0.902, 0.869, 0.715, and 0.764, respectively.
DISCUSSION Surgical trainees’ operating theater experiences significantly influence their ability to attain key professional competencies.9 A measure of trainees’ satisfaction with their learning environment would allow accurate assessment of this environment and recognition of strengths and weaknesses of residency programs.3 STEEM has been validated and used to assess the learning environment in the surgical operating theater.3,9-13 We used STEEM to evaluate the surgical theater learning environment of urology residents in Saudi Arabia and factors influencing their perception of this environment. The total inventory score (67.95%) and the 4 subscales’ scores in this study were similar to the scores reported by Nagraj et al.12 and higher than the approximately 55% perception scores of medical interns in a local Saudi report.11 Although our total score is slightly lower than the scores of 73.6% and 74.4% reported by Cassar3 and Kanashiro et al.,10 respectively, our learning opportunity 3
TABLE 2. Mean and Standard Deviation of Each Statement, Overall and Subscales’ Scores 1. My trainer has a pleasant personality 2. I get on well with my trainer 3. My trainer is enthusiastic about teaching 4. My trainer has a genuine interest in my progress 5. I understand what my trainer is trying to teach me 6. My trainer’s surgical skills are very good 7. My trainer gives me time to practice surgical skills in theatre 8. My trainer immediately takes the instruments away when I do not perform well 9. Before the operation my trainer discusses the surgical technique planned 10. Before the operation my trainer discusses what part of the procedure I will perform 11. My trainer expects my surgical skills to be as good as his/hers 12. My trainer gives me feedback on my performance 13. My trainer’s criticism is constructive 14. On this unit the type of operations performed are too complex for my level 15. The elective operating list has the right case mix to suit my training 16. There are far too many cases on the elective list to give me the opportunity to operate 17. I get enough opportunity to assist 18. There are enough theatre sessions per week for me to gain the appropriate experience 19. More senior trainees take my opportunities to operate 20. The number of emergency procedures is sufficient for me to gain the right operative experience 21. The variety of emergency cases gives me the appropriate exposure 22. My trainer is in too much of a rush during emergency cases to let me operate 23. I miss out on operative experience because of restrictions on working hours 24. I have the opportunity to develop the skills required at my stage 25. The atmosphere in theatre is pleasant 26. In theatre I don’t like being corrected in front of medical students, nurses and residents 27. The nursing staff dislike it when I operate as the operation takes longer 28. The anesthetists put pressure on my trainer to operate himself to reduceanesthetic time 29. The theatre staff are friendly 30. I feel discriminated against in theatre because of my sex 31. I feel discriminated against in theatre because of my race 32. I feel part of a team in theatre 33. I am too busy doing other work to go to theatre 34. I am often too tired to get the most out of theatre teaching 35 I am so stressed in theatre that I do not learn as much as I could 36 I am asked to perform operations alone that I do not feel competent at 37. When I am in theatre, there is nobody to cover the ward 38. I get bleeped during operations 39. The level of supervision in theatre is adequate for my level 40. Theatre sessions are too long Total score Teaching and training subscale Learning opportunity subscale Atmosphere subscale Supervision, workload and support subscale
Mean
SD
4 3.9 3.3 3.5 3.6 3.9 3.4 2.5 2.9 3 2.8 3.4 3.3 3.7 3.3 2.8 3.9 3.5 3.4 2.9 3.1 3.1 3.4 3.3 3.7 3 2.9 2.9 4.3 4.3 4.5 4.1 3.3 3.7 3.1 3.9 2.8 1.9 3.8 3.7 135.9 43.6 36.5 29.7 26.1
0.9 0.8 0.9 0.9 0.9 0.9 1.2 1.1 1.1 1.1 0.9 0.9 0.9 0.9 1.1 1 0.9 1 1.2 1.2 1.2 1.1 1 1 1.3 1.4 1.3 1.3 0.7 0.9 0.7 0.8 1.1 1 1.3 1 1.4 1 1.1 0.7 16.7 7.5 4.7 5.3 4.8
SD, standard deviation.
(66.4%) and atmosphere subscale (74.3%) scores were comparable to Scottish trainees.3 Although scores of 73.6% and 69.5% were seen as satisfactory by Cassar3 and Nagraj et al.,12 respectively, others and we10 believe that scores less than 80% represent a learning environment that is less than agreeable, as it corresponds to a score that lies between that of uncertain (60%) and agree (80%) on a Likert’s nonparametric scale. The highest scoring statements were “I (do not) feel discriminated in the operating room because of my sex or race” (items 19 and 20), which represent the rarity of women and non-Saudis in Saudi urology residency program.
Most residents (475%) were satisfied with their trainers’ personality (statements 1 and 2) and their surgical skills (statement 6). They were also satisfied with the opportunity to assist (statement 17), the theater staff (statement 29), the feeling of being a part of the theater team (statement 32), and the level of supervision in the theater (statement 39). However, most residents also reported that they are bleeped during operations (statement 38), a condition that makes them stressed and distracted. More than half of the trainees (62%) reported that their trainer does not discuss their role in surgical procedures or the steps involved in operations, which is a
4
Journal of Surgical Education Volume ]/Number ] ] 2014
TABLE 3. Correlation and Intercorrelation of Total STEEM Score and Subscales
Total score and teaching and training subscale Total score and learning opportunity subscale Total score and atmosphere subscale Total score and supervision/ workload/support subscale Teaching and training and learning opportunity subscales Teaching and training and atmosphere subscales Teaching and training and supervision/workload/ support subscales Learning opportunity and atmosphere subscales Learning opportunity and supervision/workload/ support Atmosphere and supervision/ workload/support subscales
Spearman ρ
Significance
0.813
0.000
0.602
0.000
0.796
0.000
0.769
0.000
0.385
0.030
0.414
0.018
0.470
0.007
0.372
0.036
0.415
0.018
0.626
0.000
significant concern given the importance of teaching procedural skills in a planned fashion.14 Eleven statements were poorly rated with a mean value of 3 or less, which should be examined more closely as they indicate problem areas. They included statements 8, 9, 10, and 11 of the “teaching-and-learning” domain, which entail the trainers’ attitude if trainees did not perform well and the lack of planned fashion of training. They also included statement 20 of the learning opportunity domain, which indicates the insufficiency of emergency procedures. Other poorly rated items were statement 26, which speaks about humiliation and correction of the trainee in front of other staff; statements 27 and 28, which discuss the attitude of the nursing staff and anesthetists toward the trainee performance; and statements 37 and 38, which represent the absence of coverage of the trainee when he is operating. Preplanned teaching, supportive hospital environment, supportive supervisors together with proper coverage, and management of workloads are areas that need development and enhancement. We found no significant perception differences among residents of different program stages regarding the total inventory and the 4 subscales’ scores. Similar results were also shown in Scottish and Canadian surgical residents.3,10 When the residents were classified into juniors who usually observe and assist more in the operating theater and seniors who usually perform most of the surgical procedures with assistance and have a more supervisory role in the ward and perform administrative tasks such as organizing rounds and on-call schedules, no statistically significant differences were Journal of Surgical Education Volume ]/Number ] ] 2014
shown between them regarding the total questionnaire score or any of its subscales’ scores. However, junior residents were listening more to their trainers’ instructions (statement 5, p ¼ 0.034) and were more acceptable to their criticism (statement 13, p ¼ 0.042), which is expected from residents willing to be trained at the start of their career. Our results showed no statistically significant perception differences (p 4 0.05) among residents in different main sectors of the health care system. This is greatly important in assessing the educational environment at individual teaching centers within a postgraduate training program to determine uniformity of the educational environment across all the training centers. TABLE 4. Reliability Analysis of the Overall Survey Scale Mean if Item Item Deleted 1 2 3 4 5 6 7 8* 9 10* 11* 12* 13 14 15* 16* 17 18 19* 20 21* 22* 23 24 25 26 27* 28* 29 30* 31* 32 33 34* 35 36* 37 38 39 40*
131.84 131.97 132.59 132.34 132.25 131.94 132.47 133.38 132.94 132.91 133.09 132.47 132.63 132.16 132.63 133.09 131.94 132.34 132.47 132.94 132.72 132.75 132.50 132.56 132.22 132.84 132.94 133.03 131.56 131.56 131.38 131.81 132.53 132.22 132.81 131.97 133.09 134.00 132.09 132.16
Scale Variance if Item Deleted 265.749 263.644 259.668 261.781 264.194 265.673 252.644 274.694 259.867 267.959 272.217 268.193 267.274 265.168 268.629 282.475 265.028 261.717 292.838 263.996 271.757 274.774 263.677 256.899 244.886 254.588 266.964 263.709 265.544 268.964 271.339 257.770 259.805 271.467 246.286 274.225 259.959 260.581 260.604 275.362
Corrected Cronbach Item—Total α if Item Correlation Deleted 0.414 0.520 0.591 0.523 0.465 0.390 0.622 0.060 0.464 0.251 0.161 0.320 0.350 0.420 0.228 0.156 0.395 0.452 0.397 0.320 0.121 0.058 0.396 0.610 0.753 0.464 0.230 0.302 0.535 0.299 0.286 0.766 0.470 0.163 0.759 0.093 0.356 0.490 0.459 0.092
0.858 0.857 0.855 0.856 0.857 0.858 0.852 0.865 0.857 0.861 0.863 0.860 0.859 0.858 0.862 0.869 0.858 0.857 0.876 0.860 0.865 0.865 0.858 0.854 0.848 0.856 0.862 0.861 0.857 0.860 0.861 0.853 0.856 0.863 0.849 0.864 0.859 0.856 0.857 0.863
*Items not consistent with the rest of the scale. 5
Residents from the Eastern region perceived the teachingand-learning domain more significantly positive (p ¼ 0.025) than residents of other regions in Saudi Arabia as they appreciated the enthusiasm of their trainers more (item 3) and the feedback given to them on their performance (item 12). However, no significant differences were detected regarding the total inventory score or the other 3 domains assuring the uniformity of urology training across Saudi Arabia. It is expected that 2 items are unlikely to contribute vast differences in the learning environment among the different regions. All 4 subscales of STEEM correlated significantly and positively with the overall STEEM score. The subscale that correlated most strongly with overall score was teaching and training (Spearman ρ ¼ 0.813), followed by learning atmosphere (Spearman ρ ¼ 0.796). Different correlations were shown in other reports.3,11 The correlation analysis between the subscale scores revealed significant acceptable correlations between the subscale scores of “supervision/ workload/support” and both “atmosphere” and “teachingand-training” subscales. Significant correlations, albeit low, were also found between the subscale scores of “atmosphere” and “learning opportunities” as well as between “teaching and training” and “learning opportunity.” The low correlation between the various subscales (with the exception of “atmosphere” with “supervision/workload/support”) suggests that each subscale assesses a distinct component of the theater learning environment. The Cronbach α was high at 0.862 for the 40 statements. A high reliability of STEEM questionnaire was also shown in different studies.3,10-12 The Cronbach α also was high at 0.848 for the teaching-and-training domain, and it was good at 0.761 for atmosphere perception and at 0.666 for supervision/workload/support domains, which were higher than those reported by others.3,11,12 Our results showed that the STEEM’s content internal consistency is acceptable and valid to distinguish between various influences that contribute to the educational environment in the operating room. However, when our data were analyzed to exclude each question in turn, using the “α if item deleted,” 16 statements were found inconsistent with the rest of the scale and could be deleted (Table 4). On deletion of these statements from the reliability analysis, the Cronbach α coefficient increased for the entire inventory as well as for the “teaching and learning,” the “learning opportunity,” and the “supervision/workload/support” subscales to 0.902, 0.869, 0.715, and 0.764, respectively, suggesting reduction of the questionnaire items provided that the same results could be reproduced in a larger sample size. Reduction of the inventory to 13 items was previously proposed.13 Although our sample size is more than in the studies conducted by Cassar3 and Kanashiro et al.,10 our study is still limited by the small sample size as well as the rarity of women in Saudi urology residency program, which precluded comparison of gender perception. Another
important limitation is the low response rate (45.8%), which limits generalization of conclusions among the residents in the urology training program. However, we believe that our study, being the first of its nature, represents a good chance to evaluate the current urology training programs and help improving them.
6
Journal of Surgical Education Volume ]/Number ] ] 2014
CONCLUSIONS STEEM survey is an applicable and reliable instrument for assessing the learning environment of urology residency program in Saudi Arabia. Urology residents perceived the theater learning environment of urology residency program as less than ideal. Perceptions of theater learning environment did not change significantly among different stages of the program, different training regions of Saudi Arabia, or different sectors of health care system assuring the uniformity of urology training across Saudi Arabia. The training programs should address significant concerns to poorly rated statements, as they indicate problem areas. Teaching procedural skills in a planned fashion, supportive hospital environment and supervisors together with proper coverage and management of workloads are areas that need development and enhancement. Finally, we hope that the results provided by this study could be used to improve the theater learning environment of urology residency program in Saudi Arabia.
ACKNOWLEDGMENT This study was supported by a grant from the College of Medicine Research Center, Deanship of Scientific Research, King Saud University, Riyadh, Saudi Arabia.
REFERENCES 1. Genn JM. AMEE Medical Education Guide No. 23
(Part 1): curriculum, environment, climate, quality and change in medical education—a unifying perspective. Med Teach. 2001;23(4):337-344. http://dx.doi. org/10.1080/01421590120063330. 2. Haertel GD, Walberg HJ, Haertel EH. Socio-
psychological environments and learning: a quantitative synthesis. Br Educ Res J. 1981;7(1):27-36. http: //dx.doi.org/10.1080/0141192810070103. 3. Cassar K. Development of an instrument to measure
the surgical operating theatre learning environment as perceived by basic surgical trainees. Med Teach. 2004;26(3):260-264. http://dx.doi.org/10.1080/0142 159042000191975.
4. Gelfand DV, Podnos YD, Wilson SE, Cooke J,
Williams RA. Choosing general surgery: insights into career choices of current medical students. Arch Surg. 2002;137(8):941-947. http://dx.doi.org/10.1001/archsurg. 137.8.941. 5. Silen W. Surgical education: in need of a shift in
paradigm. Surgery. 2003;134(2):399-402. http://dx. doi.org/10.1067/msy.2003.232. 6. Ahmed K, Jawad M, Dasgupta P, Darzi A, Athanasiou T,
(12):884-889. http://dx.doi.org/10.1111/j.1445-21 97.2010.05430.x. 10. Kanashiro J, McAleer S, Roff S. Assessing the educa-
tional environment in the operating room — a measure of resident perception at one Canadian institution. Surgery. 2006;139(2):150-158. http://dx. doi.org/10.101W6/j.surg.2005.07.005.
11. Al-Qahtani MF, Al-Sheikh M. Assessment of educational
Khan MS. Assessment and maintenance of competence in urology. Nat Rev Urol. 2010;7(7):403-413. http://dx.doi. org/10.1038/nrurol.2010.81.
environment of surgical theatre at a teaching hospital of a saudi university: using surgical theatre educational environment measures. Oman Med J. 2012;27(3):217-223. http://dx.doi.org/10.5001/omj.2012.49.
7. Tomkins CC, Hall C. An introduction to non-parametric
12. Nagraj S, Wall D, Jones E. Can STEEM be used to
statistics for health scientists. Sci York. 2006;3:20-26 Available at: 〈http://scholar.google.com/scholar?hl=en& btnG=Search&q=intitle:AnþIntroductionþtoþNonparametricþStatisticsþforþHealthþScientists#4〉.
measure the educational environment within the operating theatre for undergraduate medical students? Med Teach. 2006;28(7):642-647. http://dx.doi.org/ 10.1080/01421590600922875.
8. Tavakol M, Dennick R. Making sense of Cronbach’s
13. Nagraj S, Wall D, Jones E. The development and
alpha. Int J Med Educ. 2011;2:53-55. http://dx.doi. org/10.5116/ijme.4dfb.8dfd.
validation of the mini-surgical theatre educational environment measure. Med Teach. 2007;29(6):e192-e197. http://dx.doi.org/10.1080/01421590701299249.
9. Mahoney A, Crowe PJ, Harris P. Exploring Austral-
asian surgical trainees’ satisfaction with operating theatre learning using the “surgical theatre educational environment measure”. ANZ J Surg. 2010;80
14. Grantcharov TP, Reznick RK. Teaching procedural
Journal of Surgical Education Volume ]/Number ] ] 2014
7
skills. Br Med J. 2008;336(7653):1129-1131. http: //dx.doi.org/10.1136/bmj.39517.686956.47.
Evaluation of Urology Residents’ Perception of Surgical Theater Educational Environment Saleh Binsaleh, MD, FRCS(C),* Abdulrahman Babaeer, MD,† Danny Rabah, MD, FRCS(C),* and Khaled Madbouly, MD‡ Department of Surgery, Faculty of Medicine, King Saud University, Riyadh, Saudi Arabia; †Department of Urology, King Abdulaziz Medical City, Riyadh, Saudi Arabia; and ‡Department of Urology, Prince Mohammed bin Abdulaziz Hospital, Riyadh, Saudi Arabia
*
PURPOSE: To evaluate surgical theater learning environ-
ment perception in urology residents in Saudi Arabia and to investigate association of learning environment perception and stages of residency program, sectors of health care system, and regions of Saudi Arabia. DESIGN: A cross-sectional survey using the surgical theater educational environment measure (STEEM) inventory. SETTINGS AND PARTICIPANTS: The STEEM inventory was used to measure theater learning environment perception of urology residents in Saudi Arabia. Respondents’ perception was compared regarding different residency stages, sectors of the health care system, and regions of Saudi Arabia. Internal reliability of the inventory was assessed using the Cronbach α coefficient. Correlation analysis was done using the Spearman ρ coefficient. RESULTS: Of 72 registered residents, 33 (45.8%) completed the questionnaire. The residents perceived their environment less than acceptable (135.9 ⫾ 16.7, 67.95%). No significant differences in perception were found among residents of different program stages, different sectors of health care system, or different regions in Saudi Arabia. Residents from the eastern region perceived the training and teaching domain better (p ¼ 0.025). The inventory showed a high internal consistency with a Cronbach α of 0.862. CONCLUSIONS: STEEM survey is an applicable and
reliable instrument for assessing the learning environment and training skills of urology residency program in Saudi Arabia. Urology residents in Saudi Arabia perceived the theater learning environment as less than ideal. The perceptions of theater learning environment did not change significantly among different stages of the program, different
Correspondence: Inquiries to Saleh Binsaleh, MD, FRCS(C), Department of Surgery, Faculty of Medicine, King Saud University, P.O. Box 36175, Riyadh, 1149, Saudi Arabia; fax: (96611) 467 9493; e-mail: [email protected]
sectors of health care system, or different training regions of Saudi Arabia assuring the uniformity of urology training all over Saudi Arabia. The training programs should address significant concerns and pay close attention to areas in surgical theater educational environment, which need development and enhancement, mainly planned fashion of training, supportive supervision and hospital environment, and proper coverage and management of workloads. ( J Surg C ]:]]]-]]]. J 2014 Association of Program Directors in Surgery. Published by Elsevier Inc. All rights reserved.) KEY WORDS: surgical theatre educational environment
measure (STEEM), educational environment, residency program, urology residents, Saudi Arabia COMPETENCIES: Medical Knowledge, Practice Based
Learning and Improvement, Systems Based Practice, Interpersonal Skills and Communication
INTRODUCTION Urology residency training in Saudi Arabia is a hospitalbased 5 years structured program, with central supervision of Saudi Commission for Health Specialties, in which the first year is for general surgery and surgical intensive care rotation. Residents who complete their residency training ultimately receive certification as a specialist. The educational environment is an important measure, which has a large effect on the satisfaction, achievement, and success of medical education.1 Positive learning climates have been associated with improved learner’s performance.2 To develop an environment that is conducive to learning, the major elements that contribute to the particular learning environment need to be identified and an instrument to measure that learning environment needs to be available to allow accurate assessment of the learning environment and to identify areas that require attention.3
Journal of Surgical Education & 2014 Association of Program Directors in Surgery. Published by 1931-7204/$30.00 Elsevier Inc. All rights reserved. http://dx.doi.org/10.1016/j.jsurg.2014.08.002
1
Surgical career is perceived by medical students to be one of high pressure and stress, with no control over one’s time and an income that is not adequate for the level of commitment and the amount of work.4 Furthermore, the operating theater is reported as a complex place with a hostile environment that is inconducive to learning. It is hard to plan and prepare teaching within the operating theater, especially during stressful emergency procedures.5 Urology continues to evolve and the skill set required of urologists has become multidimensional,6 adding to the complexity of urology theater. A number of studies have been conducted to explore students’ perceptions of their learning environment. Cassar3 developed surgical theatre educational environment measure (STEEM), an instrument that measures the learning environment in the surgical operating theater. The objective of the present study was to evaluate the surgical theater learning environment of urology residents in Saudi Arabia and to identify factors influencing their perception of this environment. The study also evaluated associations of surgical theater learning environment perception with stages of the residency program, different health care sectors, and different regions of Saudi Arabia. To our knowledge, this is the first study to evaluate urology residency surgical theater learning environment since its implementation in Saudi Arabia more than a decade ago. It is also the first to use STEEM inventory to evaluate surgical theater learning environment perception of urology residents in general.
METHODS The STEEM questionnaire was used to assess the perception of urology residents in Saudi Arabia toward the surgical theater learning environment. It consists of 40 statements with which the respondents were asked to indicate their agreement using a 5-point Likert scale ranging from strongly agree (5), agree (4), uncertain (3), disagree (2), to strongly disagree (1). The scoring was reversed for negative statements (statement numbers: 8, 11, 14, 16, 19, 22, 23, 26, 27, 28, 30, 31, 33, 34, 35, 36, 37, 38, and 40) so that for all items, the higher the score the more positive the perception. The overall possible maximum score of STEEM is 200 and the minimum score is 40. An overall score of 120 of 200 would indicate a neutral perception, any value 4120 would indicate a more satisfactory environment, whereas any value of o120 would indicate a less than satisfactory learning environment.3 STEEM is divided into 4 subscales: statements 1 to 13 cover trainees’ perceptions of their teaching and training with a maximum score of 65, statements 14 to 24 cover trainees’ perceptions of learning opportunities with a maximum score of 55, statements 25 to 32 cover trainees’ perceptions of the atmosphere in the 2
operating theater with a maximum score of 40, and statements 33 to 40 trainees’ cover perceptions on supervision, workload, and support with a maximum score of 40. The questionnaire was e-mailed by a third independent party using SurveyMonkey to all residents registered in the urology training program and the identity of the collected data was kept anonymous to the researchers. Trainees provided information regarding their age, sex, stage of training, in what region of the country they were doing the training, and in which health care sector. Responses of residents in the first residency year were excluded being involved in nonurology rotation year. Residents were further classified into juniors including residents in second and third residency years and seniors including residents of fourth and fifth residency years. Scores for each item and subscale were calculated and entered into a spreadsheet. Descriptive statistics were reported as mean ⫾ standard deviation. For comparative statistics, nonparametric method of Kruskal-Wallis was used to compare any significant perception differences among different residency stages, different regions of Saudi Arabia, and sectors of health care system.7 Only 1 resident responded from the northern region, so his response was excluded from the comparison of different regions in Saudi Arabia. Likewise, 3 respondents did not mention their sector of health care system and only 1 resident responded from the security forces health care sector and their responses were excluded from the comparisons of different health care systems. The Mann-Whitney test was used to assess any significant differences between junior and senior residents. The correlation analysis was done using the Spearman ρ coefficient to assess the association among the overall STEEM score and the 4 subscales and the association between each other.7 The reliability analysis was performed using the Cronbach α coefficient to measure the internal consistency of the whole questionnaire and each of the 4 subscales.8 Using the “α if item deleted,” Cronbach α was used to identify questions whose exclusion would improve the reliability. Data were analyzed using SPSS, version 18.
RESULTS Of 72 registered residents, 33 (45.8%) completed the questionnaire. One responder was in the first residency year and his response was excluded from the analysis. Responses of 32 residents; 1 (3.1%) woman and 31 (96.9%) men, with a mean age of 30 ⫾ 2.6 years were analyzed. Table 1 illustrates demographic details of the included respondents, their region of training in Saudi Arabia, and their main health care sector. Mean (⫾ standard deviation) scores for each item of the questionnaire, for the 4 subscales and for the overall inventory were summarized in Table 2. The 2 highest Journal of Surgical Education Volume ]/Number ] ] 2014
TABLE 1. Characteristics of Study Respondents Sex Male Female Residency stage R2 R3 R4 R5 Region in Saudi Arabia Central Western Eastern Southern Northern Sector of health service Ministry of health National Guard Hospital Armed Forces Hospital Security Forces Hospital University Hospital Missing
Number
Percentage
31 1
96.9 3.1
4 11 5 12
12.5 34.4 15.6 37.5
15 6 6 4 1
46.9 18.8 18.8 12.5 3
12 3 6 1 7 3
37.5 9.4 18.8 3.1 21.9 9.3
scoring items were no. 31 “I (do not) feel discriminated in the operating room because of my race,” and no. 30 “I (do not) feel discriminated in the operating room because of my sex.” The lowest ranked item was statement no. 38 “I get bleeped (paged-called through my pager) during operations.” There were no statistically significant differences in perception of the total score as well as the 4 subscale scores of theater learning environment among different stages of the residency program (Kruskal-Wallis, p 4 0.05). Even, when the residents were classified to juniors and seniors, the Mann-Whitney test showed no statistically significant difference in the total questionnaire score or in any of its subscales scores. However, junior residents responded positively more significantly to statement 5; “I understand what my trainer is trying to teach me” and statement 13 “My trainer’s criticism is constructive.” There was no statistically significant difference (KruskalWallis, p 4 0.05) in perception of theater learning environment among residents in different main sectors of the health care system regarding the total and the 4 subscales’ score. However, residents from the Ministry of Health care system were more significantly satisfied with the sufficiency of emergency procedures they are exposed to (statement 20, p ¼ 0.041). We identified significant differences among residents from different training regions of Saudi Arabia, excluding the northern region, regarding the training and teaching subscale score where residents from the eastern region perceived it better (p ¼ 0.025). They responded more significantly positive to statements 3, “My trainer is enthusiastic about teaching” (p ¼ 0.026); 12, “My trainer gives me feedback on my performance” (p ¼ 0.017); and Journal of Surgical Education Volume ]/Number ] ] 2014
14, “On this unit the type of operations performed are too complex for my level” (p ¼ 0.033). Western region residents responded significantly positive to statement 20, “The number of emergency procedures is sufficient for me to gain the right operative experience” (p ¼ 0.046) and they disagreed significantly to statement 19 “More senior trainees take my opportunities to operate” (p ¼ 0.004). All 4 subscales of STEEM correlated significantly and positively with the overall STEEM score ranging from 0.602 to 0.813. The subscale that correlated most strongly with overall satisfaction was teaching and training (Spearman ρ ¼ 0.813), followed by learning atmosphere (Spearman ρ ¼ 0.796). Learning opportunity subscale had the lowest correlation with the overall scale (Spearman ρ ¼ 0.602). Intercorrelation of the subscales with each other was given in Table 3. The Cronbach α coefficient was computed to measure the internal consistency of the overall questionnaire and each of the 4 subscales. For the whole questionnaire, the Cronbach α was 0.862. For the “teaching-and-training” subscale, the Cronbach α was 0.848, for learning opportunity it was 0.461, for atmosphere perception it was 0.761, and for supervision/workload/support subscale it was 0.666. Statements 8, 10, 11, and 12 (the teaching-and-learning domain); 15, 16, 19, 21, and 22 (the learning opportunity domain); 27, 28, 30, and 31(the atmosphere domain); and 34, 36, and 40 (supervision/workload/support domain) were not consistent with the rest of the scale and could be deleted (Table 4). On deletion of these factors from the reliability analysis, the Cronbach α coefficient increased for the entire inventory as well as the teaching and learning, the learning opportunity, and the supervision/workload/support subscales to 0.902, 0.869, 0.715, and 0.764, respectively.
DISCUSSION Surgical trainees’ operating theater experiences significantly influence their ability to attain key professional competencies.9 A measure of trainees’ satisfaction with their learning environment would allow accurate assessment of this environment and recognition of strengths and weaknesses of residency programs.3 STEEM has been validated and used to assess the learning environment in the surgical operating theater.3,9-13 We used STEEM to evaluate the surgical theater learning environment of urology residents in Saudi Arabia and factors influencing their perception of this environment. The total inventory score (67.95%) and the 4 subscales’ scores in this study were similar to the scores reported by Nagraj et al.12 and higher than the approximately 55% perception scores of medical interns in a local Saudi report.11 Although our total score is slightly lower than the scores of 73.6% and 74.4% reported by Cassar3 and Kanashiro et al.,10 respectively, our learning opportunity 3
TABLE 2. Mean and Standard Deviation of Each Statement, Overall and Subscales’ Scores 1. My trainer has a pleasant personality 2. I get on well with my trainer 3. My trainer is enthusiastic about teaching 4. My trainer has a genuine interest in my progress 5. I understand what my trainer is trying to teach me 6. My trainer’s surgical skills are very good 7. My trainer gives me time to practice surgical skills in theatre 8. My trainer immediately takes the instruments away when I do not perform well 9. Before the operation my trainer discusses the surgical technique planned 10. Before the operation my trainer discusses what part of the procedure I will perform 11. My trainer expects my surgical skills to be as good as his/hers 12. My trainer gives me feedback on my performance 13. My trainer’s criticism is constructive 14. On this unit the type of operations performed are too complex for my level 15. The elective operating list has the right case mix to suit my training 16. There are far too many cases on the elective list to give me the opportunity to operate 17. I get enough opportunity to assist 18. There are enough theatre sessions per week for me to gain the appropriate experience 19. More senior trainees take my opportunities to operate 20. The number of emergency procedures is sufficient for me to gain the right operative experience 21. The variety of emergency cases gives me the appropriate exposure 22. My trainer is in too much of a rush during emergency cases to let me operate 23. I miss out on operative experience because of restrictions on working hours 24. I have the opportunity to develop the skills required at my stage 25. The atmosphere in theatre is pleasant 26. In theatre I don’t like being corrected in front of medical students, nurses and residents 27. The nursing staff dislike it when I operate as the operation takes longer 28. The anesthetists put pressure on my trainer to operate himself to reduceanesthetic time 29. The theatre staff are friendly 30. I feel discriminated against in theatre because of my sex 31. I feel discriminated against in theatre because of my race 32. I feel part of a team in theatre 33. I am too busy doing other work to go to theatre 34. I am often too tired to get the most out of theatre teaching 35 I am so stressed in theatre that I do not learn as much as I could 36 I am asked to perform operations alone that I do not feel competent at 37. When I am in theatre, there is nobody to cover the ward 38. I get bleeped during operations 39. The level of supervision in theatre is adequate for my level 40. Theatre sessions are too long Total score Teaching and training subscale Learning opportunity subscale Atmosphere subscale Supervision, workload and support subscale
Mean
SD
4 3.9 3.3 3.5 3.6 3.9 3.4 2.5 2.9 3 2.8 3.4 3.3 3.7 3.3 2.8 3.9 3.5 3.4 2.9 3.1 3.1 3.4 3.3 3.7 3 2.9 2.9 4.3 4.3 4.5 4.1 3.3 3.7 3.1 3.9 2.8 1.9 3.8 3.7 135.9 43.6 36.5 29.7 26.1
0.9 0.8 0.9 0.9 0.9 0.9 1.2 1.1 1.1 1.1 0.9 0.9 0.9 0.9 1.1 1 0.9 1 1.2 1.2 1.2 1.1 1 1 1.3 1.4 1.3 1.3 0.7 0.9 0.7 0.8 1.1 1 1.3 1 1.4 1 1.1 0.7 16.7 7.5 4.7 5.3 4.8
SD, standard deviation.
(66.4%) and atmosphere subscale (74.3%) scores were comparable to Scottish trainees.3 Although scores of 73.6% and 69.5% were seen as satisfactory by Cassar3 and Nagraj et al.,12 respectively, others and we10 believe that scores less than 80% represent a learning environment that is less than agreeable, as it corresponds to a score that lies between that of uncertain (60%) and agree (80%) on a Likert’s nonparametric scale. The highest scoring statements were “I (do not) feel discriminated in the operating room because of my sex or race” (items 19 and 20), which represent the rarity of women and non-Saudis in Saudi urology residency program.
Most residents (475%) were satisfied with their trainers’ personality (statements 1 and 2) and their surgical skills (statement 6). They were also satisfied with the opportunity to assist (statement 17), the theater staff (statement 29), the feeling of being a part of the theater team (statement 32), and the level of supervision in the theater (statement 39). However, most residents also reported that they are bleeped during operations (statement 38), a condition that makes them stressed and distracted. More than half of the trainees (62%) reported that their trainer does not discuss their role in surgical procedures or the steps involved in operations, which is a
4
Journal of Surgical Education Volume ]/Number ] ] 2014
TABLE 3. Correlation and Intercorrelation of Total STEEM Score and Subscales
Total score and teaching and training subscale Total score and learning opportunity subscale Total score and atmosphere subscale Total score and supervision/ workload/support subscale Teaching and training and learning opportunity subscales Teaching and training and atmosphere subscales Teaching and training and supervision/workload/ support subscales Learning opportunity and atmosphere subscales Learning opportunity and supervision/workload/ support Atmosphere and supervision/ workload/support subscales
Spearman ρ
Significance
0.813
0.000
0.602
0.000
0.796
0.000
0.769
0.000
0.385
0.030
0.414
0.018
0.470
0.007
0.372
0.036
0.415
0.018
0.626
0.000
significant concern given the importance of teaching procedural skills in a planned fashion.14 Eleven statements were poorly rated with a mean value of 3 or less, which should be examined more closely as they indicate problem areas. They included statements 8, 9, 10, and 11 of the “teaching-and-learning” domain, which entail the trainers’ attitude if trainees did not perform well and the lack of planned fashion of training. They also included statement 20 of the learning opportunity domain, which indicates the insufficiency of emergency procedures. Other poorly rated items were statement 26, which speaks about humiliation and correction of the trainee in front of other staff; statements 27 and 28, which discuss the attitude of the nursing staff and anesthetists toward the trainee performance; and statements 37 and 38, which represent the absence of coverage of the trainee when he is operating. Preplanned teaching, supportive hospital environment, supportive supervisors together with proper coverage, and management of workloads are areas that need development and enhancement. We found no significant perception differences among residents of different program stages regarding the total inventory and the 4 subscales’ scores. Similar results were also shown in Scottish and Canadian surgical residents.3,10 When the residents were classified into juniors who usually observe and assist more in the operating theater and seniors who usually perform most of the surgical procedures with assistance and have a more supervisory role in the ward and perform administrative tasks such as organizing rounds and on-call schedules, no statistically significant differences were Journal of Surgical Education Volume ]/Number ] ] 2014
shown between them regarding the total questionnaire score or any of its subscales’ scores. However, junior residents were listening more to their trainers’ instructions (statement 5, p ¼ 0.034) and were more acceptable to their criticism (statement 13, p ¼ 0.042), which is expected from residents willing to be trained at the start of their career. Our results showed no statistically significant perception differences (p 4 0.05) among residents in different main sectors of the health care system. This is greatly important in assessing the educational environment at individual teaching centers within a postgraduate training program to determine uniformity of the educational environment across all the training centers. TABLE 4. Reliability Analysis of the Overall Survey Scale Mean if Item Item Deleted 1 2 3 4 5 6 7 8* 9 10* 11* 12* 13 14 15* 16* 17 18 19* 20 21* 22* 23 24 25 26 27* 28* 29 30* 31* 32 33 34* 35 36* 37 38 39 40*
131.84 131.97 132.59 132.34 132.25 131.94 132.47 133.38 132.94 132.91 133.09 132.47 132.63 132.16 132.63 133.09 131.94 132.34 132.47 132.94 132.72 132.75 132.50 132.56 132.22 132.84 132.94 133.03 131.56 131.56 131.38 131.81 132.53 132.22 132.81 131.97 133.09 134.00 132.09 132.16
Scale Variance if Item Deleted 265.749 263.644 259.668 261.781 264.194 265.673 252.644 274.694 259.867 267.959 272.217 268.193 267.274 265.168 268.629 282.475 265.028 261.717 292.838 263.996 271.757 274.774 263.677 256.899 244.886 254.588 266.964 263.709 265.544 268.964 271.339 257.770 259.805 271.467 246.286 274.225 259.959 260.581 260.604 275.362
Corrected Cronbach Item—Total α if Item Correlation Deleted 0.414 0.520 0.591 0.523 0.465 0.390 0.622 0.060 0.464 0.251 0.161 0.320 0.350 0.420 0.228 0.156 0.395 0.452 0.397 0.320 0.121 0.058 0.396 0.610 0.753 0.464 0.230 0.302 0.535 0.299 0.286 0.766 0.470 0.163 0.759 0.093 0.356 0.490 0.459 0.092
0.858 0.857 0.855 0.856 0.857 0.858 0.852 0.865 0.857 0.861 0.863 0.860 0.859 0.858 0.862 0.869 0.858 0.857 0.876 0.860 0.865 0.865 0.858 0.854 0.848 0.856 0.862 0.861 0.857 0.860 0.861 0.853 0.856 0.863 0.849 0.864 0.859 0.856 0.857 0.863
*Items not consistent with the rest of the scale. 5
Residents from the Eastern region perceived the teachingand-learning domain more significantly positive (p ¼ 0.025) than residents of other regions in Saudi Arabia as they appreciated the enthusiasm of their trainers more (item 3) and the feedback given to them on their performance (item 12). However, no significant differences were detected regarding the total inventory score or the other 3 domains assuring the uniformity of urology training across Saudi Arabia. It is expected that 2 items are unlikely to contribute vast differences in the learning environment among the different regions. All 4 subscales of STEEM correlated significantly and positively with the overall STEEM score. The subscale that correlated most strongly with overall score was teaching and training (Spearman ρ ¼ 0.813), followed by learning atmosphere (Spearman ρ ¼ 0.796). Different correlations were shown in other reports.3,11 The correlation analysis between the subscale scores revealed significant acceptable correlations between the subscale scores of “supervision/ workload/support” and both “atmosphere” and “teachingand-training” subscales. Significant correlations, albeit low, were also found between the subscale scores of “atmosphere” and “learning opportunities” as well as between “teaching and training” and “learning opportunity.” The low correlation between the various subscales (with the exception of “atmosphere” with “supervision/workload/support”) suggests that each subscale assesses a distinct component of the theater learning environment. The Cronbach α was high at 0.862 for the 40 statements. A high reliability of STEEM questionnaire was also shown in different studies.3,10-12 The Cronbach α also was high at 0.848 for the teaching-and-training domain, and it was good at 0.761 for atmosphere perception and at 0.666 for supervision/workload/support domains, which were higher than those reported by others.3,11,12 Our results showed that the STEEM’s content internal consistency is acceptable and valid to distinguish between various influences that contribute to the educational environment in the operating room. However, when our data were analyzed to exclude each question in turn, using the “α if item deleted,” 16 statements were found inconsistent with the rest of the scale and could be deleted (Table 4). On deletion of these statements from the reliability analysis, the Cronbach α coefficient increased for the entire inventory as well as for the “teaching and learning,” the “learning opportunity,” and the “supervision/workload/support” subscales to 0.902, 0.869, 0.715, and 0.764, respectively, suggesting reduction of the questionnaire items provided that the same results could be reproduced in a larger sample size. Reduction of the inventory to 13 items was previously proposed.13 Although our sample size is more than in the studies conducted by Cassar3 and Kanashiro et al.,10 our study is still limited by the small sample size as well as the rarity of women in Saudi urology residency program, which precluded comparison of gender perception. Another
important limitation is the low response rate (45.8%), which limits generalization of conclusions among the residents in the urology training program. However, we believe that our study, being the first of its nature, represents a good chance to evaluate the current urology training programs and help improving them.
6
Journal of Surgical Education Volume ]/Number ] ] 2014
CONCLUSIONS STEEM survey is an applicable and reliable instrument for assessing the learning environment of urology residency program in Saudi Arabia. Urology residents perceived the theater learning environment of urology residency program as less than ideal. Perceptions of theater learning environment did not change significantly among different stages of the program, different training regions of Saudi Arabia, or different sectors of health care system assuring the uniformity of urology training across Saudi Arabia. The training programs should address significant concerns to poorly rated statements, as they indicate problem areas. Teaching procedural skills in a planned fashion, supportive hospital environment and supervisors together with proper coverage and management of workloads are areas that need development and enhancement. Finally, we hope that the results provided by this study could be used to improve the theater learning environment of urology residency program in Saudi Arabia.
ACKNOWLEDGMENT This study was supported by a grant from the College of Medicine Research Center, Deanship of Scientific Research, King Saud University, Riyadh, Saudi Arabia.
REFERENCES 1. Genn JM. AMEE Medical Education Guide No. 23
(Part 1): curriculum, environment, climate, quality and change in medical education—a unifying perspective. Med Teach. 2001;23(4):337-344. http://dx.doi. org/10.1080/01421590120063330. 2. Haertel GD, Walberg HJ, Haertel EH. Socio-
psychological environments and learning: a quantitative synthesis. Br Educ Res J. 1981;7(1):27-36. http: //dx.doi.org/10.1080/0141192810070103. 3. Cassar K. Development of an instrument to measure
the surgical operating theatre learning environment as perceived by basic surgical trainees. Med Teach. 2004;26(3):260-264. http://dx.doi.org/10.1080/0142 159042000191975.
4. Gelfand DV, Podnos YD, Wilson SE, Cooke J,
Williams RA. Choosing general surgery: insights into career choices of current medical students. Arch Surg. 2002;137(8):941-947. http://dx.doi.org/10.1001/archsurg. 137.8.941. 5. Silen W. Surgical education: in need of a shift in
paradigm. Surgery. 2003;134(2):399-402. http://dx. doi.org/10.1067/msy.2003.232. 6. Ahmed K, Jawad M, Dasgupta P, Darzi A, Athanasiou T,
(12):884-889. http://dx.doi.org/10.1111/j.1445-21 97.2010.05430.x. 10. Kanashiro J, McAleer S, Roff S. Assessing the educa-
tional environment in the operating room — a measure of resident perception at one Canadian institution. Surgery. 2006;139(2):150-158. http://dx. doi.org/10.101W6/j.surg.2005.07.005.
11. Al-Qahtani MF, Al-Sheikh M. Assessment of educational
Khan MS. Assessment and maintenance of competence in urology. Nat Rev Urol. 2010;7(7):403-413. http://dx.doi. org/10.1038/nrurol.2010.81.
environment of surgical theatre at a teaching hospital of a saudi university: using surgical theatre educational environment measures. Oman Med J. 2012;27(3):217-223. http://dx.doi.org/10.5001/omj.2012.49.
7. Tomkins CC, Hall C. An introduction to non-parametric
12. Nagraj S, Wall D, Jones E. Can STEEM be used to
statistics for health scientists. Sci York. 2006;3:20-26 Available at: 〈http://scholar.google.com/scholar?hl=en& btnG=Search&q=intitle:AnþIntroductionþtoþNonparametricþStatisticsþforþHealthþScientists#4〉.
measure the educational environment within the operating theatre for undergraduate medical students? Med Teach. 2006;28(7):642-647. http://dx.doi.org/ 10.1080/01421590600922875.
8. Tavakol M, Dennick R. Making sense of Cronbach’s
13. Nagraj S, Wall D, Jones E. The development and
alpha. Int J Med Educ. 2011;2:53-55. http://dx.doi. org/10.5116/ijme.4dfb.8dfd.
validation of the mini-surgical theatre educational environment measure. Med Teach. 2007;29(6):e192-e197. http://dx.doi.org/10.1080/01421590701299249.
9. Mahoney A, Crowe PJ, Harris P. Exploring Austral-
asian surgical trainees’ satisfaction with operating theatre learning using the “surgical theatre educational environment measure”. ANZ J Surg. 2010;80
14. Grantcharov TP, Reznick RK. Teaching procedural
Journal of Surgical Education Volume ]/Number ] ] 2014
7
skills. Br Med J. 2008;336(7653):1129-1131. http: //dx.doi.org/10.1136/bmj.39517.686956.47.
