currently being conducted to investigate the amount of time program directors spend reviewing each application in the setting of such a large application burden. The strengths of this study include its large sample size and assessment of factors that negatively affected an applicant, as well as earned an applicant special consideration. A limitation of the study included the use of a nonvalidated questionnaire, although to our knowledge, no validated questionnaire exists which assesses program directors’ selection criteria. Additionally, aside from program directors, we did not survey other faculty members or house staff who may have had input in the selection process. Across the country, program directors are the only individuals who reliably have input into resident selection. Therefore, extending the survey to others may have captured individuals who genuinely did not have input during the selection process.
CONCLUSION Applicants to urology are reminded of the importance of USMLE performance and the need to obtain strong reference letters from urologists. Performing an away elective clearly earns applicants special consideration, and personal factors, such as gender, are considered in the application process. Based on the identified criteria used by program directors in this report, future studies should focus on each criterion’s utility for predicting applicants’ success as residents and its impact on medical students’ pursuits and training during medical school. Acknowledgment. The authors thank Heather Swanson for her assistance in gathering the data. References 1. Andriole DA, Schechtman KB, Ryan K, et al. How competitive is my surgical specialty? Am J Surg. 2002;184:1-5. 2. American Urological Association: urology residency match statistics. Available at: https://www.auanet.org/education/urology-and -specialty-matches.cfm. Accessed September 25,2014. 3. Kerfoot BP, Asher KP, McCullough DL. Financial and educational costs of the residency interview process for urology applicants. Urology. 2008;71:990-994. 4. National Resident Matching Program, Data Release and Research Committee: results of the 2014 NRMP Program Director Survey. Available at: http://www.nrmp.org/wp-content/uploads/2014/09/ PD-Survey-Report-2014.pdf. Accessed September 29,2014. 5. Green M, Jones P, Thomas JX Jr. Selection criteria for residency: results of a national program directors survey. Acad Med. 2009;84: 362-367. 6. Rifkin WD, Rifkin A. Correlation between housestaff performance on the United States Medical Licensing Examination and standardized patient encounters. Mt Sinai J Med. 2005;72: 47-49. 7. Andriole DA, Jeffe DB, Whelan AJ. What predicts surgical internship performance? Am J Surg. 2004;188:161-164. 8. Grewal SG, Yeung LS, Brandes SB. Predictors of success in a urology residency program. J Surg Educ. 2013;70:138-143. 9. Alpha Omega Alpha: chapters. Available at: http://www. alphaomegaalpha.org/chapters.html. Accessed September 29,2014. 10. Kaffenberger BH, Kaffenberger JA, Zirwas MJ. Academic dermatologists’ views on the value of residency letters of recommendation. J Am Acad Dermatol. 2014;71:395-396.
UROLOGY 85 (4), 2015
11. Dirschl DR, Adams GL. Reliability in evaluating letters of recommendation. Acad Med. 2000;75:1029. 12. Keim SM, Rein JA, Chisholm C, et al. A standardized letter of recommendation for residency application. Acad Emerg Med. 1999; 6:1141-1146. 13. McGaghie WC, Cohen ER, Wayne DB. Are United States Medical Licensing Exam Step 1 and 2 scores valid measures for postgraduate medical residency selection decisions? Acad Med. 2011;86: 48-52.
APPENDIX SUPPLEMENTARY DATA
Supplementary data associated with this article can be found, in the online version, at http://dx.doi.org/10.1016/j.urology. 2014.12.041.
EDITORIAL COMMENT With an increasingly competitive pool of applicants, Program Directors (PDs) look for unique aspects in a candidate’s application that may translate to good patient care, proficiency with surgical techniques, strong communication and professionalism, and academic potential. This is an interesting article1 that surveys urology PDs for their perspectives about the process of selecting residency candidates including positive and negative factors and parameters that draw special consideration. Similar to previous studies, robust United States Medical Licensing Examination (USMLE) scores stood out as the most important consideration, followed closely by urology reference letters and Alpha Omega Alpha status.2 Interestingly, research publications, clinical grades, and Dean’s letters were significantly less impactful. When examining negative factors, a unique aspect of this article,1 USMLE Step 1 and 2 scores 220 were the top 2 detrimental variables, followed by previous match failure. Special consideration is given to gender, minority status, being from the same medical school as the PD, completion of an away rotation at the PD’s institution, and being the child of an academic urologist.
APPLICANT’S PERSPECTIVE The match process is a time of great uncertainty and anxiety for candidates, given the diminishing match rate (64% in 2014) and increasing competition.3 This article1 is very informative by demystifying factors that PDs consider although it is disappointing to see the low priority of research and special consideration for some factors that appear to be somewhat arbitrary.4 There are also other troubling realities with this process, including fiscal burdens and opportunity costs. Candidates are now applying to an average of 59 programs, and associated costs can be exorbitant with many applicants reporting a total outlay of $7000-$8000 related to the interview process, often leading to additional debt.5 Also, the current process is extremely time consuming, with many applicants spending approximately 5-6 weeks on the road, representing >10% of the fourth-year medical school experience. A possible solution to these challenges may be to organize several regional “conventions” that allow applicants to travel to central locations and interview with selected faculty from several programs on a single weekend, similar to what was performed in Canada in the past. Shah et al6 also propose videoconference interviewing as a possible adjunct, which may supplement limited interview times in a regional convention model. 735
PROGRAM DIRECTORS’ PERSPECTIVE In general, most urology applicants are high-achieving medical students who have strong USMLE scores and clinical grades, and it is often very difficult to stratify the candidates in a meaningful manner. Reference letters tend to be uniformly positive, rendering them difficult to interpret. Our bias is that interest in research is critically important for the future of the field, but sorting this out within the electronic residency application service system is very challenging. Many candidates enter long lists of projects, and it is difficult to determine which ones are genuine, and the extent of true involvement is difficult to assess. The current system is in great need of improvement, and the authors1 should be congratulated for initiating further discussion about how this can be accomplished. Steven C. Campbell, M.D., Ph.D., Center for Urologic Oncology, Glickman Urologic and Kidney Institute, Cleveland Clinic, Cleveland, OH Kirtishri Mishra, Case Western Reserve University School of Medicine, Cleveland, OH
References 1. Weissbart SJ, Stock JA, Wein AJ. Program directors’ criteria for selection into urology residency. Urology. 2015;85:731-736. 2. Green M, Jones P, Thomas JX Jr. Selection criteria for residency: results of a national program directors survey. Acad Med. 2009;84: 362-367. 3. American Urological Association (AUA), Urology match statistics. Available at: https://www.auanet.org/education/urology-and-specialty -matches.cfm. Accessed on December 23, 2014. 4. Andriole DA, Jeffe DB, Whelan AJ. What predicts surgical internship performance? Am J Surg. 2004;188:161-164. 5. Kerfoot BP, Asher KP, McCullough DL. Financial and educational costs of the residency interview process for urology applicants. Urology. 2008;71:990-994. 6. Shah SK, Arora S, Skipper B, et al. Randomized evaluation of a web based interview process for urology resident selection. J Urol. 2012; 187:1380-1384.
urologist. Making this determination is no easy task for program directors; deans’ letters that omit student rank and medical schools with nongrading policies do not help. Additionally, program directors who also have clinical responsibilities are being flooded with more applications than ever before, hindering their ability to determine who is sincerely interested in their program. Amidst this congested system, like it or not, USMLE scores remain the only objective metric that program directors can easily use to compare applicants. The matching process should become more transparent for applicants and program directors. A reasonably set application limit would clarify applicants’ levels of interest in programs and provide program directors with more time to review each application. Grading students during medical school and providing class rank in deans’ letters would allow program directors to more thoroughly evaluate applicants and, perhaps, rely less on United States Medical Licensing Examination score. Although Web-based interviewing would engender a financial savings for applicants, urology applicants have found traditional on-site interviewing to be more effective,2 and changing the interviewing modality would not solve the problems with the current system that we delineate. The field of urology should embrace the competitive nature of the match as the field should seek out only the best and the brightest for its future. The matching system should not be reformed to make the process less competitive. However, as there are only a limited number of residency positions available, medical schools should not obscure those students who truly are the best candidates. Steven J. Weissbart, M.D., Division of Urogynecology and Reconstructive Pelvic Surgery, Department of Obstetrics and Gynecology, University of Pennsylvania, Philadelphia, PA Jeffrey A. Stock, M.D., Department of Urology, Icahn School of Medicine at Mount Sinai, New York, NY Alan J. Wein, M.D., Ph.D. (hon), Division of Urology, Department of Surgery, University of Pennsylvania, Philadelphia, PA
http://dx.doi.org/10.1016/j.urology.2014.12.045 UROLOGY 85: 735e736, 2015. Ó 2015 Elsevier Inc.
REPLY The number of applicants to the urology match has consistently exceeded the number of residency positions available.1 As there are not enough positions available to accept everyone who wishes to train in urology, a competitive system must exist to determine who will be given the opportunity to become a
736
References 1. American Urological Association (AUA), Urology match statistics. Available at: https://www.auanet.org/education/urology-and-specialty -matches.cfm. Accessed on December 30, 2014. 2. Shah SK, Arora S, Skipper B, et al. Randomized evaluation of a web based interview process for urology resident selection. J Urol. 2012; 187:1380-1384.
http://dx.doi.org/10.1016/j.urology.2014.12.046 UROLOGY 85: 736, 2015. Ó 2015 Elsevier Inc.
UROLOGY 85 (4), 2015
CONCLUSION Applicants to urology are reminded of the importance of USMLE performance and the need to obtain strong reference letters from urologists. Performing an away elective clearly earns applicants special consideration, and personal factors, such as gender, are considered in the application process. Based on the identified criteria used by program directors in this report, future studies should focus on each criterion’s utility for predicting applicants’ success as residents and its impact on medical students’ pursuits and training during medical school. Acknowledgment. The authors thank Heather Swanson for her assistance in gathering the data. References 1. Andriole DA, Schechtman KB, Ryan K, et al. How competitive is my surgical specialty? Am J Surg. 2002;184:1-5. 2. American Urological Association: urology residency match statistics. Available at: https://www.auanet.org/education/urology-and -specialty-matches.cfm. Accessed September 25,2014. 3. Kerfoot BP, Asher KP, McCullough DL. Financial and educational costs of the residency interview process for urology applicants. Urology. 2008;71:990-994. 4. National Resident Matching Program, Data Release and Research Committee: results of the 2014 NRMP Program Director Survey. Available at: http://www.nrmp.org/wp-content/uploads/2014/09/ PD-Survey-Report-2014.pdf. Accessed September 29,2014. 5. Green M, Jones P, Thomas JX Jr. Selection criteria for residency: results of a national program directors survey. Acad Med. 2009;84: 362-367. 6. Rifkin WD, Rifkin A. Correlation between housestaff performance on the United States Medical Licensing Examination and standardized patient encounters. Mt Sinai J Med. 2005;72: 47-49. 7. Andriole DA, Jeffe DB, Whelan AJ. What predicts surgical internship performance? Am J Surg. 2004;188:161-164. 8. Grewal SG, Yeung LS, Brandes SB. Predictors of success in a urology residency program. J Surg Educ. 2013;70:138-143. 9. Alpha Omega Alpha: chapters. Available at: http://www. alphaomegaalpha.org/chapters.html. Accessed September 29,2014. 10. Kaffenberger BH, Kaffenberger JA, Zirwas MJ. Academic dermatologists’ views on the value of residency letters of recommendation. J Am Acad Dermatol. 2014;71:395-396.
UROLOGY 85 (4), 2015
11. Dirschl DR, Adams GL. Reliability in evaluating letters of recommendation. Acad Med. 2000;75:1029. 12. Keim SM, Rein JA, Chisholm C, et al. A standardized letter of recommendation for residency application. Acad Emerg Med. 1999; 6:1141-1146. 13. McGaghie WC, Cohen ER, Wayne DB. Are United States Medical Licensing Exam Step 1 and 2 scores valid measures for postgraduate medical residency selection decisions? Acad Med. 2011;86: 48-52.
APPENDIX SUPPLEMENTARY DATA
Supplementary data associated with this article can be found, in the online version, at http://dx.doi.org/10.1016/j.urology. 2014.12.041.
EDITORIAL COMMENT With an increasingly competitive pool of applicants, Program Directors (PDs) look for unique aspects in a candidate’s application that may translate to good patient care, proficiency with surgical techniques, strong communication and professionalism, and academic potential. This is an interesting article1 that surveys urology PDs for their perspectives about the process of selecting residency candidates including positive and negative factors and parameters that draw special consideration. Similar to previous studies, robust United States Medical Licensing Examination (USMLE) scores stood out as the most important consideration, followed closely by urology reference letters and Alpha Omega Alpha status.2 Interestingly, research publications, clinical grades, and Dean’s letters were significantly less impactful. When examining negative factors, a unique aspect of this article,1 USMLE Step 1 and 2 scores 220 were the top 2 detrimental variables, followed by previous match failure. Special consideration is given to gender, minority status, being from the same medical school as the PD, completion of an away rotation at the PD’s institution, and being the child of an academic urologist.
APPLICANT’S PERSPECTIVE The match process is a time of great uncertainty and anxiety for candidates, given the diminishing match rate (64% in 2014) and increasing competition.3 This article1 is very informative by demystifying factors that PDs consider although it is disappointing to see the low priority of research and special consideration for some factors that appear to be somewhat arbitrary.4 There are also other troubling realities with this process, including fiscal burdens and opportunity costs. Candidates are now applying to an average of 59 programs, and associated costs can be exorbitant with many applicants reporting a total outlay of $7000-$8000 related to the interview process, often leading to additional debt.5 Also, the current process is extremely time consuming, with many applicants spending approximately 5-6 weeks on the road, representing >10% of the fourth-year medical school experience. A possible solution to these challenges may be to organize several regional “conventions” that allow applicants to travel to central locations and interview with selected faculty from several programs on a single weekend, similar to what was performed in Canada in the past. Shah et al6 also propose videoconference interviewing as a possible adjunct, which may supplement limited interview times in a regional convention model. 735
PROGRAM DIRECTORS’ PERSPECTIVE In general, most urology applicants are high-achieving medical students who have strong USMLE scores and clinical grades, and it is often very difficult to stratify the candidates in a meaningful manner. Reference letters tend to be uniformly positive, rendering them difficult to interpret. Our bias is that interest in research is critically important for the future of the field, but sorting this out within the electronic residency application service system is very challenging. Many candidates enter long lists of projects, and it is difficult to determine which ones are genuine, and the extent of true involvement is difficult to assess. The current system is in great need of improvement, and the authors1 should be congratulated for initiating further discussion about how this can be accomplished. Steven C. Campbell, M.D., Ph.D., Center for Urologic Oncology, Glickman Urologic and Kidney Institute, Cleveland Clinic, Cleveland, OH Kirtishri Mishra, Case Western Reserve University School of Medicine, Cleveland, OH
References 1. Weissbart SJ, Stock JA, Wein AJ. Program directors’ criteria for selection into urology residency. Urology. 2015;85:731-736. 2. Green M, Jones P, Thomas JX Jr. Selection criteria for residency: results of a national program directors survey. Acad Med. 2009;84: 362-367. 3. American Urological Association (AUA), Urology match statistics. Available at: https://www.auanet.org/education/urology-and-specialty -matches.cfm. Accessed on December 23, 2014. 4. Andriole DA, Jeffe DB, Whelan AJ. What predicts surgical internship performance? Am J Surg. 2004;188:161-164. 5. Kerfoot BP, Asher KP, McCullough DL. Financial and educational costs of the residency interview process for urology applicants. Urology. 2008;71:990-994. 6. Shah SK, Arora S, Skipper B, et al. Randomized evaluation of a web based interview process for urology resident selection. J Urol. 2012; 187:1380-1384.
urologist. Making this determination is no easy task for program directors; deans’ letters that omit student rank and medical schools with nongrading policies do not help. Additionally, program directors who also have clinical responsibilities are being flooded with more applications than ever before, hindering their ability to determine who is sincerely interested in their program. Amidst this congested system, like it or not, USMLE scores remain the only objective metric that program directors can easily use to compare applicants. The matching process should become more transparent for applicants and program directors. A reasonably set application limit would clarify applicants’ levels of interest in programs and provide program directors with more time to review each application. Grading students during medical school and providing class rank in deans’ letters would allow program directors to more thoroughly evaluate applicants and, perhaps, rely less on United States Medical Licensing Examination score. Although Web-based interviewing would engender a financial savings for applicants, urology applicants have found traditional on-site interviewing to be more effective,2 and changing the interviewing modality would not solve the problems with the current system that we delineate. The field of urology should embrace the competitive nature of the match as the field should seek out only the best and the brightest for its future. The matching system should not be reformed to make the process less competitive. However, as there are only a limited number of residency positions available, medical schools should not obscure those students who truly are the best candidates. Steven J. Weissbart, M.D., Division of Urogynecology and Reconstructive Pelvic Surgery, Department of Obstetrics and Gynecology, University of Pennsylvania, Philadelphia, PA Jeffrey A. Stock, M.D., Department of Urology, Icahn School of Medicine at Mount Sinai, New York, NY Alan J. Wein, M.D., Ph.D. (hon), Division of Urology, Department of Surgery, University of Pennsylvania, Philadelphia, PA
http://dx.doi.org/10.1016/j.urology.2014.12.045 UROLOGY 85: 735e736, 2015. Ó 2015 Elsevier Inc.
REPLY The number of applicants to the urology match has consistently exceeded the number of residency positions available.1 As there are not enough positions available to accept everyone who wishes to train in urology, a competitive system must exist to determine who will be given the opportunity to become a
736
References 1. American Urological Association (AUA), Urology match statistics. Available at: https://www.auanet.org/education/urology-and-specialty -matches.cfm. Accessed on December 30, 2014. 2. Shah SK, Arora S, Skipper B, et al. Randomized evaluation of a web based interview process for urology resident selection. J Urol. 2012; 187:1380-1384.
http://dx.doi.org/10.1016/j.urology.2014.12.046 UROLOGY 85: 736, 2015. Ó 2015 Elsevier Inc.
UROLOGY 85 (4), 2015
