Correspondence
Can We More Wisely Choose Residents? LETTER: is an enviable position. There is significant N eurosurgery competition for training appointments with a large pool of
highly qualified applicants. This situation allows programs to be very selective and, consequently, our residencies are filled with some of the brightest students in the world. Most do well, as might be expected given the abilities of the group; however 15% of those who start a residency don’t finish and a similar number fail the oral boards (24). Given that some attrition is expected, and probably advantageous, it is notable that 30% of those who start a neurological residency fail to meet the most basic goals of training. The rate of attrition is significantly greater than those reported in other specialties: overall, only 2.4% of residents in accredited programs fail to complete training (1, 16, 17, 27). There are likely several reasons that so many residents are unsuccessful. Occasionally, a trainee is able to complete the program but chooses not to; perhaps the field isn’t what he or she expected it to be or some other issue intervenes. More commonly, either because of problems with performance or behavior, a trainee is encouraged to leave or, ultimately, is unable to become board certified. Effective screening of applicants would be expected to minimize these outcomes. The fact that they are commonplace suggests a problem with the process of resident selection. It shouldn’t be surprising that such a problem might exist, given that essentially all of the criteria used to select residents have been shown to be unreliable. Any number of investigators has studied the relationships of medical school grades and comments, board scores, letters of recommendation, and Dean’s letters to residency performance (3, 4, 6-8, 10, 11, 18, 22, 34, 38, 40). Nearly all have found them unrelated. There is also little evidence that interviews, as commonly conducted, are of much value (5, 9, 15, 32). It may be that more extensive exposure, for instance a 2- to 4-week rotation, could be useful in choosing a resident. Unfortunately, as a practical issue, most students cannot do rotations at most of the programs to which they apply. Recognizing the deficiencies of our current methods, what can be done to improve the selection of residents? Two changes would be relatively easy to make: 1) Adopt the use of structured interviews. In a structured interview, each applicant is asked the same questions; the questions reflect the skills needed to succeed in the job. The answers are then graded using a predefined scale. Structured interviews have been shown, in a variety of settings, to be markedly more effective than free-form conversations in evaluating applicants (2, 23, 25, 26, 33, 37). However, structured interviews are rarely used to help choose a neurosurgery resident. I suspect that many of the objections to changing the interview format reflect the interviewer’s confidence that they possess a unique ability to judge the character and potential of the applicants. Little data support this contention. 2) Be a bit more straightforward in writing letters of recommendation. Only in Lake Wobegon can all of the children be above average.
More significant changes in the evaluation of applicants would likely be more burdensome. For instance, there is a suggestion that personality testing might predict residents’ performance (12-14, 19-21, 28-30, 35, 36). Unfortunately, formal neuropsychometric testing is thought to be intrusive and can be expensive. If only a handful of programs included testing in the interview process, the suspicion is that they would be at a disadvantage in attracting residents. Similarly, there is information that suggests that performance in clinical simulations predicts clinical performance (12, 31, 39). The worry, however, is that if only a few programs required objective structure clinical examinations, students would be less likely to attend. Perhaps the easiest way to address these concerns would be to have more time or resource-intensive testing performed at an applicant’s home institution. For example, if most program directors agreed that personality testing of applicants was of interest, the department at the applicant’s medical school would arrange it. For applicants from institutions without neurosurgery departments, testing could be offered on away rotations or at the first interview site. This scheme offers several advantages: 1) Each school would have relatively few students to test and, therefore, the costs of testing should be small. 2) It would eliminate duplicate testing. 3) It would eliminate the perceived disadvantage of testing in just a few centers. 4) Most importantly, even if it were found that personality testing or OSCEs were of no value, organized neurosurgery would have created a mechanism to test the utility of other longer or more involved activities in selecting residents. Changes in Accreditation Council for Graduate Medical Education resident evaluations make this a particularly good time to change the selection process. The new evaluation forms are common to all of the programs and, potentially for the first time, allow a uniform evaluation of performance. Common selection criteria can be compared with common outcome measures. This offers as least the hope of useful data. It will be difficult to change the way neurosurgeons select residents. Many academic neurosurgeons likely have little problem with the current process. The loss of a resident is inconvenient, but it is often more keenly felt by the junior attendings and the other trainees. Resident attrition reinforces the exclusivity of the specialty and is in keeping with the experience of senior surgeons; however, the loss of a resident is more costly than it is often admitted to be. Time and resources are wasted by both the faculty and trainee while other applicants are denied a position. More troubling, the situation can become extraordinarily unpleasant for all involved and life-changing for the trainee. If, with relatively little work, the process of resident selection can be improved, it should be. Thomas Pittman Department of Neurosurgery, University of Kentucky, Lexington, Kentucky, USA To whom correspondence should be addressed: Thomas Pittman, M.D. [E-mail: [email protected]] Published online 18 June, 2014; http://dx.doi.org/10.1016/j.wneu.2014.06.028.
WORLD NEUROSURGERY 82 [3/4]: e553-e577, SEPTEMBER/OCTOBER 2014
www.WORLDNEUROSURGERY.org
e553
CORRESPONDENCE
REFERENCES 1. Accreditation Council for Graduate Academic Year 2010-2011. Available at: http://www.acgme.org/acgmeweb/Portals/0/PFAssets/PublicationsBooks/20102011_ACGME_DATA_RESOURCE_BOOK.pdf. Accessed June 27, 2014.
22. Metro DG, Talarico JF, Patel RM, Wetmore AL: The resident application process and its correlation to future performance as a resident. Anesth Analg 100: 502-505, 2005. 23. Neitzschman HR, Neitzschman LH, Dowling A: Key component of resident selection: the semi-structured conversation. Acad Radiol 9:1423-1429, 2002.
2. Bandiera G, Regehr G: Reliability of a structured interview scoring instrument for a Canadian postgraduate emergency medicine training program. Acad Emerg Med 11:27-32, 2004.
24. Pevehouse BC, Colenbrander A: The United States Neurological Surgery Residency Matching Program. Neurosurgery 35:1172-1175, 1994: discussion 1175-1182.
3. Bell JG, Kanellitsas I, Shaffer L: Selection of obstetrics and gynecology residents on the basis of medical school performance. Am J Obstet Gynecol 186:1091-1094, 2002.
25. Poole A, Catano VM, Cunningham DP: Prediction performance in Canadian dental schools: the new CDA structured interview, a new personality assessment, and the DAT. J Dent Educ 71(5):664-676, 2007.
4. Berner ES, Brooks CM, Erdmann JV: Use of the USMLE to select residents. Acad Med 68:753-755, 1993.
26. Powis DA, Neame RLB, Bristow T, Murphy LB: The objective structured interview for medical student selection. Br Med J 296:765-768, 1988.
5. Borowitz SM, Saulsbury FR, Wilson WG: Information collected during the residency match process does not predict clinical performance. Arch Pediatr Adolesc Med 154:256-260, 2000. 6. Boyse TD, Patterson SK, Cohan RH, Korobkin M, Fitzgerald JT, Oh MS, Gross BH, Quint DJ: Does medical school performance predict radiology resident performance? Acad Radiol 9:437-445, 2002. 7. Dirschl DR, Campion ER, Gilliam K: Resident selection and predictors of performance. Clin Orthop Relat Res 449:44-49, 2006. 8. Dirschl DR, Dahners LE, Adams GL, Crouch JH, Wilson FC: Correlating selection criteria with subsequent performance as residents. Clin Orthop Relat Res 399:265-271, 2002. 9. Dubovsky SL, Gendel MH, Dubovsky AN, Levin R, Rosse J, House R: Can admissions interviews predict performance in residency? Acad Pysch 32(6):498-503, 2008.
27. Prager JD, Myer CM 4th, Myer CM 3rd: Attrition in otolaryngology residency. Otolaryngol Head Neck Surg 145:753-754, 2001. 28. Reeve PE, Vickers MD, Hortorn JN: Selecting anaesthetists: the use of psychological tests and structured interviews. J R Soc Med 86:400-403, 1993. 29. Rhoton MF, Barnes A, Flashburg M, Ronai A, Springman S: Influence of anesthesiology residents’ noncognitive skills on the occurrence of critical incidents and the residents’ overall clinical performance. Acad Med 66:359-361, 1991. 30. Schell RM, Dilorenzo AN, Li HF, Fragneto RY, Bowe EA, Hessel EA: Anesthesiology resident personality type correlates with faculty assessment of resident performance. J Clin Anesth 2012:566-572, 2012. 31. Sloan DA, Donnelly MB, Schwartz RW, Felts JK, Blue AV, Strodel WE: The use of objective structured clinical examination (OSCE) for evaluation and instruction in graduate medical education. J Surg Res 63:225-230, 1996. 32. Smilen SW, Funai EF, Bianco AT: Residency selection: should interviewers be given applicants’ board scores. Am J Obstet Gynecol 184:508-513, 2001.
10. Harfmann K, Zirwas J: Can performance in medical school predict performance in residency? A compilation and review of correlative studies. J Am Acad Dermatol 65:1010-1022.e2, 2011.
33. Strand EA, Moore E, Laube DW: Can a structured, behavior-based interview predict future resident success? Am J Obstet Gynecol 204:446.e1-446.e13, 2011.
11. Hunt DD, Maclaren CF, Scott CS, Chu J, Leiden LL: Characteristics of dean’s letters in 1981 and 1992. Acad Med 68:905-911, 1993.
34. Takayama H, Grinsell R, Brock D, Foy H, Pellegrini C, Horvath K: Is it appropriate to use core clerkship grades in the selection of residents? Curr Surg 63: 391-396, 2006.
12. Kahn MJ, Merrill WW, Anderson DS, Szerlip HM: Residency program director evaluations do not correlate with performance on a required 4th-year objective structured clinical examination. Teach Learn Med 13:9-12, 2001.
35. Talarico JF, Metro DG, Patel RM, Carney P, Wetmore AL: Emotional intelligence and its correlation to performance as a resident: a preliminary study. J Clin Anesth 20:84-89, 2008.
13. Keck JW, Arnold L, Willoughby L, Calkins V: Efficacy of cognitive/noncognitive measures in predicting resident physician performance. J Med Educ 54:759-765, 1979.
36. Talarico JF, Varon AJ, Banks SE, Berger JS, Pivalizza EG, Medina-Rivera G: Emotional intelligence and the relationship to resident performance: a multiinstitutional study. J Clin Anesth 25:181-187, 2013.
14. Kilpatrick CC, Doyle PD, Reichman EF, Chohan L, Uthman MO, Orejuela FJ: Emotional intelligence and selection to administrative chief residency. Acad Psychiatry 36:388-390, 2012.
37. Tarico VS, Altmaier EM, Smith WL, Franken EA Jr, Berbaum KS: Development and validation of an accomplishment interview for radiology residents. J Med Educ 61:845-847, 1986.
15. Komives E, St Weiss, Rosa RM: The applicant interview as a predictor of resident performance. J Med Educ 59:425-426, 1984.
38. Volle RL: Using national board of medical examiners scores in selection of residents. JAMA 68:753-755, 1988.
16. Kwakwa F, Jonasson O: Attrition in graduate surgical education: an analysis of the 1993 entering cohort of surgical. J Am Coll Surg 189:602-610, 1999.
39. Wallenstein J, Heron S, Santen S, Shayne P, Ander D: A core competency-based objective structure clinical examination (OSCE) can predict future resident performance. Acad Emerg Med 2:S67-S71, 2010.
17. Manriquez Gilpin M: Residency attrition rate in obstetrics and gynecology: are we losing more postgraduates today? Am J Obstet Gynecol 193:1804-1806, 2005.
40. Yindra KJ, Rosenfeld PS, Donnelly MB: Medical school achievement as predictors of residency performance. J Med Educ 63:356-363, 1998.
18. Market RJ: The relationship of academic measures in medical school to performance after graduation. Acad Med 68:S31-S34, 1993. 19. Matveevskii AS, Loyden JJ, Merlo LJ: Testing program to improve anesthesia resident selection. Anesthesiology 107:A995, 2007. 20. McDonald JS, Lingam RP, Gupta B, Jacoby J, Gough HG, Bradley P: Psychologic testing as an aid to selection of residents in anesthesiology. Anesth Analg 78: 542-547, 1994. 21. Merlo LJ, Matveevkii AS: Personality testing may improve resident selection in anesthesiology programs. Med Teach 31:e551-e554, 2009.
e554
www.SCIENCEDIRECT.com
IN REPLY: When talking about Europe but still including only the United Kingdom, one is talking about only less than a tenth of the population of more than 700 million people in Europe. There are many historical differences in training, health care systems, and
82 [3/4]: e553-e577, SEPTEMBER/OCTOBER 2014 WORLD NEUROSURGERY
Can We More Wisely Choose Residents? LETTER: is an enviable position. There is significant N eurosurgery competition for training appointments with a large pool of
highly qualified applicants. This situation allows programs to be very selective and, consequently, our residencies are filled with some of the brightest students in the world. Most do well, as might be expected given the abilities of the group; however 15% of those who start a residency don’t finish and a similar number fail the oral boards (24). Given that some attrition is expected, and probably advantageous, it is notable that 30% of those who start a neurological residency fail to meet the most basic goals of training. The rate of attrition is significantly greater than those reported in other specialties: overall, only 2.4% of residents in accredited programs fail to complete training (1, 16, 17, 27). There are likely several reasons that so many residents are unsuccessful. Occasionally, a trainee is able to complete the program but chooses not to; perhaps the field isn’t what he or she expected it to be or some other issue intervenes. More commonly, either because of problems with performance or behavior, a trainee is encouraged to leave or, ultimately, is unable to become board certified. Effective screening of applicants would be expected to minimize these outcomes. The fact that they are commonplace suggests a problem with the process of resident selection. It shouldn’t be surprising that such a problem might exist, given that essentially all of the criteria used to select residents have been shown to be unreliable. Any number of investigators has studied the relationships of medical school grades and comments, board scores, letters of recommendation, and Dean’s letters to residency performance (3, 4, 6-8, 10, 11, 18, 22, 34, 38, 40). Nearly all have found them unrelated. There is also little evidence that interviews, as commonly conducted, are of much value (5, 9, 15, 32). It may be that more extensive exposure, for instance a 2- to 4-week rotation, could be useful in choosing a resident. Unfortunately, as a practical issue, most students cannot do rotations at most of the programs to which they apply. Recognizing the deficiencies of our current methods, what can be done to improve the selection of residents? Two changes would be relatively easy to make: 1) Adopt the use of structured interviews. In a structured interview, each applicant is asked the same questions; the questions reflect the skills needed to succeed in the job. The answers are then graded using a predefined scale. Structured interviews have been shown, in a variety of settings, to be markedly more effective than free-form conversations in evaluating applicants (2, 23, 25, 26, 33, 37). However, structured interviews are rarely used to help choose a neurosurgery resident. I suspect that many of the objections to changing the interview format reflect the interviewer’s confidence that they possess a unique ability to judge the character and potential of the applicants. Little data support this contention. 2) Be a bit more straightforward in writing letters of recommendation. Only in Lake Wobegon can all of the children be above average.
More significant changes in the evaluation of applicants would likely be more burdensome. For instance, there is a suggestion that personality testing might predict residents’ performance (12-14, 19-21, 28-30, 35, 36). Unfortunately, formal neuropsychometric testing is thought to be intrusive and can be expensive. If only a handful of programs included testing in the interview process, the suspicion is that they would be at a disadvantage in attracting residents. Similarly, there is information that suggests that performance in clinical simulations predicts clinical performance (12, 31, 39). The worry, however, is that if only a few programs required objective structure clinical examinations, students would be less likely to attend. Perhaps the easiest way to address these concerns would be to have more time or resource-intensive testing performed at an applicant’s home institution. For example, if most program directors agreed that personality testing of applicants was of interest, the department at the applicant’s medical school would arrange it. For applicants from institutions without neurosurgery departments, testing could be offered on away rotations or at the first interview site. This scheme offers several advantages: 1) Each school would have relatively few students to test and, therefore, the costs of testing should be small. 2) It would eliminate duplicate testing. 3) It would eliminate the perceived disadvantage of testing in just a few centers. 4) Most importantly, even if it were found that personality testing or OSCEs were of no value, organized neurosurgery would have created a mechanism to test the utility of other longer or more involved activities in selecting residents. Changes in Accreditation Council for Graduate Medical Education resident evaluations make this a particularly good time to change the selection process. The new evaluation forms are common to all of the programs and, potentially for the first time, allow a uniform evaluation of performance. Common selection criteria can be compared with common outcome measures. This offers as least the hope of useful data. It will be difficult to change the way neurosurgeons select residents. Many academic neurosurgeons likely have little problem with the current process. The loss of a resident is inconvenient, but it is often more keenly felt by the junior attendings and the other trainees. Resident attrition reinforces the exclusivity of the specialty and is in keeping with the experience of senior surgeons; however, the loss of a resident is more costly than it is often admitted to be. Time and resources are wasted by both the faculty and trainee while other applicants are denied a position. More troubling, the situation can become extraordinarily unpleasant for all involved and life-changing for the trainee. If, with relatively little work, the process of resident selection can be improved, it should be. Thomas Pittman Department of Neurosurgery, University of Kentucky, Lexington, Kentucky, USA To whom correspondence should be addressed: Thomas Pittman, M.D. [E-mail: [email protected]] Published online 18 June, 2014; http://dx.doi.org/10.1016/j.wneu.2014.06.028.
WORLD NEUROSURGERY 82 [3/4]: e553-e577, SEPTEMBER/OCTOBER 2014
www.WORLDNEUROSURGERY.org
e553
CORRESPONDENCE
REFERENCES 1. Accreditation Council for Graduate Academic Year 2010-2011. Available at: http://www.acgme.org/acgmeweb/Portals/0/PFAssets/PublicationsBooks/20102011_ACGME_DATA_RESOURCE_BOOK.pdf. Accessed June 27, 2014.
22. Metro DG, Talarico JF, Patel RM, Wetmore AL: The resident application process and its correlation to future performance as a resident. Anesth Analg 100: 502-505, 2005. 23. Neitzschman HR, Neitzschman LH, Dowling A: Key component of resident selection: the semi-structured conversation. Acad Radiol 9:1423-1429, 2002.
2. Bandiera G, Regehr G: Reliability of a structured interview scoring instrument for a Canadian postgraduate emergency medicine training program. Acad Emerg Med 11:27-32, 2004.
24. Pevehouse BC, Colenbrander A: The United States Neurological Surgery Residency Matching Program. Neurosurgery 35:1172-1175, 1994: discussion 1175-1182.
3. Bell JG, Kanellitsas I, Shaffer L: Selection of obstetrics and gynecology residents on the basis of medical school performance. Am J Obstet Gynecol 186:1091-1094, 2002.
25. Poole A, Catano VM, Cunningham DP: Prediction performance in Canadian dental schools: the new CDA structured interview, a new personality assessment, and the DAT. J Dent Educ 71(5):664-676, 2007.
4. Berner ES, Brooks CM, Erdmann JV: Use of the USMLE to select residents. Acad Med 68:753-755, 1993.
26. Powis DA, Neame RLB, Bristow T, Murphy LB: The objective structured interview for medical student selection. Br Med J 296:765-768, 1988.
5. Borowitz SM, Saulsbury FR, Wilson WG: Information collected during the residency match process does not predict clinical performance. Arch Pediatr Adolesc Med 154:256-260, 2000. 6. Boyse TD, Patterson SK, Cohan RH, Korobkin M, Fitzgerald JT, Oh MS, Gross BH, Quint DJ: Does medical school performance predict radiology resident performance? Acad Radiol 9:437-445, 2002. 7. Dirschl DR, Campion ER, Gilliam K: Resident selection and predictors of performance. Clin Orthop Relat Res 449:44-49, 2006. 8. Dirschl DR, Dahners LE, Adams GL, Crouch JH, Wilson FC: Correlating selection criteria with subsequent performance as residents. Clin Orthop Relat Res 399:265-271, 2002. 9. Dubovsky SL, Gendel MH, Dubovsky AN, Levin R, Rosse J, House R: Can admissions interviews predict performance in residency? Acad Pysch 32(6):498-503, 2008.
27. Prager JD, Myer CM 4th, Myer CM 3rd: Attrition in otolaryngology residency. Otolaryngol Head Neck Surg 145:753-754, 2001. 28. Reeve PE, Vickers MD, Hortorn JN: Selecting anaesthetists: the use of psychological tests and structured interviews. J R Soc Med 86:400-403, 1993. 29. Rhoton MF, Barnes A, Flashburg M, Ronai A, Springman S: Influence of anesthesiology residents’ noncognitive skills on the occurrence of critical incidents and the residents’ overall clinical performance. Acad Med 66:359-361, 1991. 30. Schell RM, Dilorenzo AN, Li HF, Fragneto RY, Bowe EA, Hessel EA: Anesthesiology resident personality type correlates with faculty assessment of resident performance. J Clin Anesth 2012:566-572, 2012. 31. Sloan DA, Donnelly MB, Schwartz RW, Felts JK, Blue AV, Strodel WE: The use of objective structured clinical examination (OSCE) for evaluation and instruction in graduate medical education. J Surg Res 63:225-230, 1996. 32. Smilen SW, Funai EF, Bianco AT: Residency selection: should interviewers be given applicants’ board scores. Am J Obstet Gynecol 184:508-513, 2001.
10. Harfmann K, Zirwas J: Can performance in medical school predict performance in residency? A compilation and review of correlative studies. J Am Acad Dermatol 65:1010-1022.e2, 2011.
33. Strand EA, Moore E, Laube DW: Can a structured, behavior-based interview predict future resident success? Am J Obstet Gynecol 204:446.e1-446.e13, 2011.
11. Hunt DD, Maclaren CF, Scott CS, Chu J, Leiden LL: Characteristics of dean’s letters in 1981 and 1992. Acad Med 68:905-911, 1993.
34. Takayama H, Grinsell R, Brock D, Foy H, Pellegrini C, Horvath K: Is it appropriate to use core clerkship grades in the selection of residents? Curr Surg 63: 391-396, 2006.
12. Kahn MJ, Merrill WW, Anderson DS, Szerlip HM: Residency program director evaluations do not correlate with performance on a required 4th-year objective structured clinical examination. Teach Learn Med 13:9-12, 2001.
35. Talarico JF, Metro DG, Patel RM, Carney P, Wetmore AL: Emotional intelligence and its correlation to performance as a resident: a preliminary study. J Clin Anesth 20:84-89, 2008.
13. Keck JW, Arnold L, Willoughby L, Calkins V: Efficacy of cognitive/noncognitive measures in predicting resident physician performance. J Med Educ 54:759-765, 1979.
36. Talarico JF, Varon AJ, Banks SE, Berger JS, Pivalizza EG, Medina-Rivera G: Emotional intelligence and the relationship to resident performance: a multiinstitutional study. J Clin Anesth 25:181-187, 2013.
14. Kilpatrick CC, Doyle PD, Reichman EF, Chohan L, Uthman MO, Orejuela FJ: Emotional intelligence and selection to administrative chief residency. Acad Psychiatry 36:388-390, 2012.
37. Tarico VS, Altmaier EM, Smith WL, Franken EA Jr, Berbaum KS: Development and validation of an accomplishment interview for radiology residents. J Med Educ 61:845-847, 1986.
15. Komives E, St Weiss, Rosa RM: The applicant interview as a predictor of resident performance. J Med Educ 59:425-426, 1984.
38. Volle RL: Using national board of medical examiners scores in selection of residents. JAMA 68:753-755, 1988.
16. Kwakwa F, Jonasson O: Attrition in graduate surgical education: an analysis of the 1993 entering cohort of surgical. J Am Coll Surg 189:602-610, 1999.
39. Wallenstein J, Heron S, Santen S, Shayne P, Ander D: A core competency-based objective structure clinical examination (OSCE) can predict future resident performance. Acad Emerg Med 2:S67-S71, 2010.
17. Manriquez Gilpin M: Residency attrition rate in obstetrics and gynecology: are we losing more postgraduates today? Am J Obstet Gynecol 193:1804-1806, 2005.
40. Yindra KJ, Rosenfeld PS, Donnelly MB: Medical school achievement as predictors of residency performance. J Med Educ 63:356-363, 1998.
18. Market RJ: The relationship of academic measures in medical school to performance after graduation. Acad Med 68:S31-S34, 1993. 19. Matveevskii AS, Loyden JJ, Merlo LJ: Testing program to improve anesthesia resident selection. Anesthesiology 107:A995, 2007. 20. McDonald JS, Lingam RP, Gupta B, Jacoby J, Gough HG, Bradley P: Psychologic testing as an aid to selection of residents in anesthesiology. Anesth Analg 78: 542-547, 1994. 21. Merlo LJ, Matveevkii AS: Personality testing may improve resident selection in anesthesiology programs. Med Teach 31:e551-e554, 2009.
e554
www.SCIENCEDIRECT.com
IN REPLY: When talking about Europe but still including only the United Kingdom, one is talking about only less than a tenth of the population of more than 700 million people in Europe. There are many historical differences in training, health care systems, and
82 [3/4]: e553-e577, SEPTEMBER/OCTOBER 2014 WORLD NEUROSURGERY
