ORIGINAL REPORTS
Impact of Fellowship Training on Research Productivity in Academic Ophthalmology Grace Huang, BS,* Christina H. Fang, BS,* Santiago A. Lopez, BS,* Neelakshi Bhagat, MD,* Paul D. Langer, MD,* and Jean Anderson Eloy, MD†,‡,§ *
The Institute of Ophthalmology and Visual Science, Rutgers New Jersey Medical School, Newark, New Jersey; †Department of Otolaryngology—Head and Neck Surgery, Rutgers New Jersey Medical School, Newark, New Jersey; ‡Center for Skull Base and Pituitary Surgery, Neurological Institute of New Jersey, Rutgers New Jersey Medical School, Newark, New Jersey; and §Department of Neurological Surgery, Rutgers New Jersey Medical School, Newark, New Jersey OBJECTIVES: To assess whether scholarly impact of
academic ophthalmologists, as measured using the h-index, is affected by fellowship training status and to further characterize differences in productivity among the various subspecialties and by departmental rank. DESIGN: A descriptive and correlational design was used. In total, 1440 academic ophthalmologists from 99 ophthalmology training programs were analyzed. The h-index data were obtained from the Scopus database. Faculty members were classified by academic rank and grouped into 10 categories based on fellowship training: anterior segment, corneal and external disease, glaucoma, uveitis and ocular immunology, vitreoretinal disease, ophthalmic plastic surgery, pediatric ophthalmology, neuro-ophthalmology, ophthalmic pathology, and “other.” A one-way analysis of variance or Student t test using Microsoft Excel and “R” statistical software were used for comparison of continuous variables, with significance set at p o 0.05. SETTINGS: Faculty working in academic ophthalmology
residency training programs in the United States whose information is stored in the American Medical Association’s Fellowship and Residency Electronic Interactive Database. RESULTS: Fellowship-trained ophthalmologists had signifi-
cantly higher research productivity, as measured using the h-index, than non–fellowship-trained ophthalmologists in this study (p o 0.0005). Academic ophthalmologists trained in vitreoretinal disease or ophthalmic pathology had the highest scholarly productivity compared with those in other ophthalmology subspecialties (p o 0.05). There was a significant increase in scholarly productivity with Correspondence: Inquiries to Jean Anderson Eloy, MD, FACS, Department of Otolaryngology—Head and Neck Surgery, Rutgers New Jersey Medical School, 90 Bergen St., Suite 8100, Newark, NJ 07103; fax: (973) 972-3767; e-mail: jean. [email protected]
410
increasing academic rank from Assistant Professor to Professor (p o 0.05). A significant difference in productivity between fellowship-trained and non–fellowship-trained ophthalmologists existed individually only at the level of Assistant Professor (p o 0.0005). CONCLUSION: Academic ophthalmologists with fellow-
ship training have significantly higher scholarly output than non–fellowship-trained ophthalmologists do, as measured using the h-index. Research productivity increases with departmental academic rank from Assistant Professor to C 2015 Association of Professor. ( J Surg 72:410-417. J Program Directors in Surgery. Published by Elsevier Inc. All rights reserved.) KEY WORDS: fellowship training, academic promotion,
research productivity, scholarly productivity, ophthalmology, h-index COMPETENCIES: Medical Knowledge, Professionalism,
Practice-Based Learning and Improvement
INTRODUCTION Subspecialty training in ophthalmology has become increasingly common over the past 2 decades. Data from the San Francisco Ophthalmology Fellowship Match and the American Society of Ophthalmic and Plastic Reconstructive Surgery fellowship match reveal that approximately 64% of US ophthalmology residents graduating in 2014 entered a fellowship training program. Currently, 3 subspecialty areas—vitreoretinal disease, cornea and external disease, and glaucoma—capture more than 75% of all residents who match into fellowships.1 An anonymous survey sent to 222 graduating ophthalmology residents in 2012 revealed several factors that
Journal of Surgical Education & 2015 Association of Program Directors in Surgery. Published by 1931-7204/$30.00 Elsevier Inc. All rights reserved. http://dx.doi.org/10.1016/j.jsurg.2014.10.010
influenced career choices among trainees. Residents who sought subspecialty training had on average a greater desire to acquire special skills, to enter a (perceived) prestigious field, and to be concerned with a favorable job market. By contrast, lifestyle considerations such as anticipated work hours and geographic preferences were more important to residents choosing a comprehensive ophthalmology career. Residents who chose comprehensive ophthalmology careers were more likely to plan to practice in a private group practice, whereas those seeking subspecialty training were more likely to intend to practice in an academic setting or were undecided in their future practice type.2 Scholarly productivity in the form of research contributions is exceedingly important in determining appointment and promotion within academic medicine.3,4 Greater research productivity has been associated with better clinical care and enhanced research opportunities, which may be factors in the decision to pursue subspecialization.4,5 Increased research output may lead to greater prestige in the field, another factor influencing many graduates who pursue fellowship training.2 Despite its obvious importance, quantifying research contribution is nevertheless complicated and should in theory reflect the cumulative significance of an individual’s publication history.6 Several metrics related to research productivity are commonly used in this evaluation process, such as the total number of publications, funding history, and frequency of citation; however, none of these measures reliably convey the overall quality of scholarly output.7 The h-index was first described in 2005 by Dr. J. E. Hirsch as a tool to assess scholarly productivity. It is an easily computable index and gives an estimate of the broad impact of a scholar’s cumulative research contributions, taking into account both the quantity and quality of publications.8 Briefly, an individual with an h-index of n has published n number of articles, each of which has been cited in the peer-reviewed literature n times. Thus, for example, an individual with an h-index of 15 has published 15 articles that have each been cited at least 15 times in the literature. If this individual with an h-index of 15 has published 50 articles overall, this means that his other 35 articles have been cited less than h (15) times in peerreviewed journals. Therefore, this measurement gauges the general influence of an author’s scholarly contributions and is not disproportionately affected by one or a handful or heavily cited publications.3 Studies using the h-index have repeatedly shown that there is a strong association between the h-index and academic advancement, procurement of funding from the National Institutes of Health (NIH), and other measures of scholarly impact in many fields including otolaryngology, radiology, urology, neurosurgery, and anesthesiology.3,6,9-12 In otolaryngology, it has been found that fellowship-trained academic practitioners have greater research productivity, as measured using the h-index, than their non–fellowship-
trained colleagues.3 There have been no previous studies comparing whether postresidency fellowship training has a similar impact on scholarly influence in ophthalmology. The purpose of this study was to determine whether the scholarly impact of faculty members of academic ophthalmology departments, as measured using the h-index, is correlated to fellowship training status. In addition, our study aimed to further characterize the differences in research productivity among academic ophthalmologists by subspecialty and by academic rank.
MATERIALS AND METHODS A list of academic ophthalmology residency training programs in the United States was generated using the American Medical Association’s Fellowship and Residency Electronic Interactive Database. Faculty information such as academic rank and fellowship training was obtained from online listings taken from the website of each academic department. Faculty members whose academic rank was undeterminable from their online profiles or whose program’s website did not present pertinent information regarding fellowship attainment were excluded from this analysis. Additionally, nonacademic, nonphysician, and part-time faculty were not included in this study. The subspecialties included in this study were obtained from the San Francisco Ophthalmology Fellowship Match1 and comprise anterior segment, cornea and external disease, glaucoma, uveitis and ocular immunology, vitreoretinal diseases, ophthalmic plastic surgery, pediatric ophthalmology, neuro-ophthalmology, ophthalmic pathology, and “other” fellowships. “Other” fellowships were those fellowships that were not listed specifically from the San Francisco Ophthalmology Fellowship Match including ocular oncology, molecular ophthalmology, ophthalmic genetics, ocular pharmacology, and ocular immunology. The initial Fellowship and Residency Electronic Interactive Database search yielded 117 ophthalmology training programs from which 18 institutions were excluded. After application of the exclusion criteria, 1440 academic ophthalmologists from 99 departments were examined. To obtain data concerning an individual’s h-index, the Scopus database (http://www. scopus.com) was used to search for each author as this database covers more than 40 million publication records from 18,500 peer-reviewed venues. The Scopus database has been previously used in this manner in a plethora of medical and basic sciences.3,6,9-36 Although other h-index database calculators such as Google Scholar or ISI Web of Knowledge exist, studies have published similar results when comparing them, suggesting a strong correlation between h-indices obtained from either Google Scholar or Scopus.11 In addition, geographic location of the ophthalmologists’ department was obtained and grouped into one of the following regions designated by the US Census
Journal of Surgical Education Volume 72/Number 3 May/June 2015
411
Bureau. These included the following: (1) Northeast: New England (CT, ME, MA, NH, RI, and VT) and MidAtlantic (NJ, NY, and PA); (2) Midwest: East North Central (IN, IL, MI, OH, and WI) and West North Central (IA, KS, MN, MO, NE, ND, and SD); (3) South: South Atlantic (DE, DC, FL, GA, MD, NC, SC, VA, and WV), East South Central (AL, KY, MS, and TN), and West South Central (AR, LA, OK, and TX); and (4) West: Mountain (AZ, CO, ID, NM, MT, UT, NV, and WY) and Pacific (AK, CA, HI, OR, and WA). To ensure accuracy in our results, initial search results for each individual author were cross-matched by departmental affiliations, previous positions in other departments, and the presence or absence of publications in ophthalmology or ophthalmology-related journals. This methodology ensured any results obtained pertained to the appropriate individual. Data collection was completed in April 2014. Statistical Analysis The Kruskal-Wallis one-way analysis of variance or Student t test was used for comparison of continuous variables. Threshold for statistical significance was set at p o 0.05, effectively rejecting the null hypothesis. Microsoft Excel (Redmond, WA) and “R” statistical software (http://www. r-project.org) were used for statistical calculation.
RESULTS Of the 1440 academic ophthalmologists included in this analysis, 83% were trained in 1 of the 9 fellowships examined (Fig. 1). None of the included ophthalmologists
FIGURE 1. Distribution of the academic ophthalmologists according to subspecialty. 412
FIGURE 2. The mean h-index of fellowship-trained and non–fellowship-trained academic ophthalmologists from 99 departments. n, sample size; error bars, standard error of the mean.
were trained in an “anterior segment” fellowship. When measured using the h-index, fellowship-trained academic ophthalmologists had significantly higher research productivity than the non–fellowship-trained academic ophthalmologists included in this study (t test, p o 0.0005) (Fig. 2). There were significant statistical differences among the 9 fellowships examined (Kruskal-Wallis analysis of variance, p o 0.05). Academic ophthalmologists trained in vitreoretinal disease or ophthalmic pathology had higher scholarly productivity than those trained in ophthalmic plastic surgery, pediatric ophthalmology, cornea and external disease, or uveitis and ocular immunology (p o 0.0005) (Fig. 3). There was no statistical difference in research productivity, as measured using the h-index, between ophthalmologists who trained in vitreoretinal disease, ophthalmic pathology, neuroophthalmology, glaucoma, and other fellowships (p 4 0.05). Mean h-index increased with the number of fellowships obtained. Ophthalmologists with zero fellowships had a mean h-index of 7.03, whereas those with 3 or more fellowships had a mean h-index of 10.86. Mean h-index of ophthalmologists with 1 or 2 fellowships were similar at 9.57 and 9.42, respectively. When stratifying ophthalmologists by geographic location of practice, the highest mean h-index of 14.04 was revealed in the West, and the lowest mean h-index of 8.71 was revealed in the South. The Table summarizes h-indices by region and division. There was a consistent and significant increase in scholarly productivity, as measured using the h-index, with increases in academic rank from Assistant Professor to Professor (Fig. 4). There was no statistical difference in scholarly productivity between Professors and Chairpersons. A significant difference in research productivity between fellowship-trained and non– fellowship-trained academic ophthalmologists was only seen at the level of Assistant Professors (p o 0.0005). At the level of Associate Professor and Professor, fellowship-trained academic ophthalmologists had higher h-indices, but this did not reach statistical significant (p ¼ 0.22 and 0.11, respectively).
Journal of Surgical Education Volume 72/Number 3 May/June 2015
FIGURE 3. Mean h-indices of the academic ophthalmologists included in this study distributed by fellowship. n, sample size; error bars, standard error of the mean.
DISCUSSION The proportion of ophthalmology residents who apply for fellowships has been steadily rising for the past decade according to data from the National Residency Matching Program and Ophthalmology Fellowship Match Program.37 Fellowship training provides graduating residents with an in-depth exposure to a subspecialty area of ophthalmology, allowing them to develop focused clinical and surgical skills related to that subspecialty area. In our study, we found that fellowship-trained ophthalmologists had significantly higher research productivity, as measured using the h-index, compared with non–fellowship-trained ophthalmologists (h-index ¼ 9.60 and 7.05, respectively; p ¼ 0.0004). Ophthalmologists that pursue advanced TABLE. Mean h-Index by US Region and Division Region
Average h-Index
Midwest
9.95
Northeast
8.04
West
14.04
South
8.71
Division East North Central West North Central New England Mid-Atlantic Pacific Mountain South Atlantic East South Central West South Central
Average h-Index 8.89 11.00 8.27 7.81 11.46 16.63 9.97 7.72 8.44
fellowship training may have higher scholarly productivity than their non–fellowship-trained colleagues, because they may be inclined to participate in research or because more research opportunities are available to fellowship-trained individuals. Greater research productivity has far-reaching implications for academic physicians. In addition to its effect on appointment decisions, research output is an important aspect of determining an academic physician’s tenure, promotion, and ranking within an academic department.18 A study found that when strategies were introduced to assess the productivity of faculty members at academic medical centers, the faculty appeared to have improved research productivity and possibly improved clinical productivity.7 Having highly productive faculty has potential advantages for an academic department, in terms of not only notoriety and prestige, but also as an enhanced ability to recruit other high-quality faculty and residents.10 The h-index attempts to incorporate both the quantity and the quality of an individual’s scholarly accomplishments by measuring how often an author is producing influential work. It is increasingly being used throughout a number of scientific disciplines to assess scholarly productivity among faculty members.6,9,18,29 In ophthalmology, it has been used to assess gender differences in scholarly impact,21 to examine the association between scholarly impact and NIH award procurement,10 and to examine whether gender differences in NIH awards and funding totals exist.22 This study is the first investigation that aimed to determine the relationship between fellowship training and scholarly impact among academic ophthalmologists.
Journal of Surgical Education Volume 72/Number 3 May/June 2015
413
FIGURE 4. The mean h-index of fellowship-trained and non–fellowship-trained ophthalmologists stratified by academic rank. Errors bars, standard error of the mean.
When comparing h-indices among the various ophthalmology subspecialties, significant differences emerged. Academic ophthalmologists trained in vitreoretinal disease and ophthalmic pathology had the highest average h-indices (h-index ¼ 11.64 and 11.55, respectively). These h-indices were significantly different from those of academic ophthalmologists that had trained in uveitis and ocular immunology, cornea and external disease, pediatric ophthalmology, and ophthalmic plastic surgery fellowships. These findings suggest that emphasis on research may be stronger in vitreoretinal disease and ophthalmic pathology fellowships. Retina fellows appear to be very productive in research and publications, with 40% having dedicated research time during their fellowship training.38 Another factor that might be considered when evaluating academic productivity among specialties is the number of practicing physicians in each particular subspecialty. For example, a physician shortage apparently exists in both neuroophthalmology and pediatric ophthalmology.39 Academic pediatric ophthalmologists had the second to lowest h-index (8.29) of all fellowship-trained academic ophthalmologists. A shortage of physicians in a subspecialty may theoretically lead to greater clinical responsibilities compared with other subspecialties and a corresponding reduction in protected research time, leading to fewer publications and fewer citations. When stratifying h-indices by academic rank, the h-index increased with academic rank from Assistant Professor to Professor, with no significant difference between Professor and Chairperson. The positive correlation between h-index and increasing academic rank is consistent with previous studies.3,6,11,12,29,40 These data confirm that research is strongly considered in the process of academic promotion. When the cohorts of fellowship-trained and non– 414
fellowship-trained academic ophthalmologists were stratified by academic rank, significant differences in h-indices were observed only at the level of Assistant Professor. At the level of Associate Professor and Professor, the trend continued revealing higher h-indices in the fellowship-trained ophthalmologists compared with their non–fellowship-trained colleagues of equal rank; however, this comparison did not achieve statistical significance. An explanation of this result is possibly due to the smaller n value in the non–fellowshiptrained ophthalmologists at the level of Associate Professor and Professor, making it more difficult to detect significance (Fig. 4). Upon further analysis, the p value was lower (0.09) when combining the ranks of Associate Professor and Professor to increase n when comparing fellowship-trained vs non–fellowship-trained ophthalmologists. This suggests that increasing n may reveal statistical significance. These data also provide a statistical benchmark of the mean h-index scores of academic ophthalmologists at different points in their careers. There are several limitations to this study. A criticism of the h-index is its insensitivity to the type of research an academic physician conducts.8 For example, an ophthalmologist with an interest in basic science who participates in laboratory-oriented projects may be involved in more timeintensive work than a colleague producing influential publications on clinical research that requires access only to medical records. Each of these types of scholarship is valuable, but the number of articles published and the number of citations may not accurately reflect the relative effect of a physician performing these different types of research.18 Another weakness of the h-index is the potential of self-citation, which may artificially inflate this statistic.41 However, to significantly raise the h-index, repeated and
Journal of Surgical Education Volume 72/Number 3 May/June 2015
sustained citation of an individual’s work would be required, making this an unlikely and rare phenomenon. In reality, if selective self-citations were to occur, it is unlikely to have more than a negligible effect on an author’s h-index.42 Another potential limitation of the h-index is that it does not necessarily account for the level of input from each author of a given published article. Individuals who work with frequently published authors may have high h-index scores, effectively piggy-backing off the work of their colleagues.29 However, it is difficult to quantify an author’s actual level of contribution to any article, rendering this phenomenon quite difficult to accurately assess by any metric.6 Lastly, this study groups together all physicians with a Doctor of Medicine degree pursuing fellowship training. It is important to note that many of these physicians may have additional postgraduate degrees, including Master of Business Administration degrees, Doctor of Philosophy degrees, or degrees in Public Health or Statistics. These physicians may lead careers that are more research oriented; therefore, obtaining higher h-indices. It would be interesting for a future study to stratify physicians by postgraduate degrees and analyze the effect on their h-index. The importance of having high-quality non–fellowshiptrained comprehensive ophthalmology faculty in an academic department cannot be overstated. These physicians are crucial in providing and maintaining the broad educational experience necessary for competence in office-based and operative management of a wide variety of ophthalmic conditions.3 The responsibilities of comprehensive ophthalmologists in academic departments are often shifted away from research output toward clinical service and the education of residents and medical students. It is therefore not surprising that, on the whole, especially at the more junior academic ranks, research productivity of comprehensive ophthalmologists in academia suffers by comparison with that of their fellowship-trained colleagues. Scholarly productivity, as measured using the h-index, will continue to be a major factor in the evaluation process in academic medicine. It is increasing in popularity, and job applicants have even started to include an h-index in their curriculum vitae.43 There are understandable reservations toward reducing research output to a single variable, and caution should be exercised when using this measurement without being aware of its limitations. Other factors such as clinical performance, educational contributions, and administrative duties also are important when evaluating performance in academia.3 Nevertheless, the h-index will continue to be one tool that can objectively, reliably, and efficiently assess scholarly relevance in academic medicine.
promotion within academic medicine. As subspecialty training becomes increasingly common in ophthalmology, it is important to determine whether there is an association between fellowship training and level of scholarly productivity. Academic ophthalmologists with fellowship training have significantly higher scholarly output than non–fellowship-trained ophthalmologists, as measured using the h-index. Research productivity increases with departmental academic rank from Assistant Professor to Professor.
CONCLUSION
10. Svider PF, Lopez SA, Husain Q, Bhagat N, Eloy JA,
Scholarly productivity in the form of research contributions is a major factor in determining appointment and
REFERENCES 1. Residency and Fellowship Match. Available at: 〈http://
www.sfmatch.org/SpecialtyInsideAll.aspx?id=2&typ=1 &name=Ophthalmology〉; [Accessed 23.4.14]. 2. Gedde SJ, Budenz DL, Haft P, Tielsch JM, Lee Y,
Quigley HA. Factors influencing career choices among graduating ophthalmology residents. Ophthalmology. 2005;112(7):1247-1254. 3. Eloy JA, Svider PF, Mauro KM, Setzen M, Baredes S.
Impact of fellowship training on research productivity in academic otolaryngology. Laryngoscope. 2012;122(12): 2690-2694. 4. Dixon AK. Publishing and academic promotion.
Singapore Med J. 2009;50(9):847-850. 5. Beasley BW, Wright SM, Cofrancesco J Jr., Babbott
SF, Thomas PA, Bass EB. Promotion criteria for clinician-educators in the United States and Canada. A survey of promotion committee chairpersons. J Am Med Assoc. 1997;278(9):723-728. 6. Kasabwala K, Morton CM, Svider PF, Nahass TA,
Eloy JA, Jackson-Rosario I. Factors influencing scholarly impact: does urology fellowship training affect research output? J Surg Educ. 2014;71(3):345-352. 7. Akl EA, Meerpohl JJ, Raad D, et al. Effects of assessing
the productivity of faculty in academic medical centres: a systematic review. CMAJ. 2012;184(11): E602-E612. 8. Hirsch JE. An index to quantify an individual’s
scientific research output. Proc Natl Acad Sci U S A. 2005;102(46):16569-16572. 9. Svider PF, Choudhry ZA, Choudhry OJ, Baredes S,
Liu JK, Eloy JA. The use of the h-index in academic otolaryngology. Laryngoscope. 2013;123(1):103-106. Langer PD. The association between scholarly impact and national institutes of health funding in ophthalmology. Ophthalmology. 2014;121(1):423-428.
Journal of Surgical Education Volume 72/Number 3 May/June 2015
415
11. Lee J, Kraus KL, Couldwell WT. Use of the h index in
neurosurgery. Clinical article. J Neurosurg. 2009;111 (2):387-392. 12. Rad AE, Brinjikji W, Cloft HJ, Kallmes DF. The H-
funding in ophthalmology. J Surg Educ. 2014;71(5): 680-688. 23. Eloy JA, Svider PF, Kovalerchik O, Baredes S,
index in academic radiology. Acad Radiol. 2010;17(7): 817-821.
Kalyoussef E, Chandrasekhar SS. Gender differences in successful NIH grant funding in otolaryngology. Otolaryngol Head Neck Surg. 2013;149(1):77-83.
13. Eloy JA, Svider PF, Folbe AJ, Setzen M, Baredes S.
24. Eloy JA, Svider PF, Cherla DV, et al. Gender disparities
AAO-HNSF CORE grant acquisition is associated with greater scholarly impact. Otolaryngol Head Neck Surg. 2014;150(1):53-60.
in research productivity among 9952 academic physicians. Laryngoscope. 2013;123(8):1865-1875.
14. Svider PF, Husain Q, Folbe AJ, Couldwell WT, Liu
JK, Eloy JA. Assessing National Institutes of Health funding and scholarly impact in neurological surgery. J Neurosurg. 2014;120(1):191-196. 15. Eloy JA, Svider PF, Folbe AJ, Couldwell WT, Liu JK.
25. Eloy JA, Svider P, Chandrasekhar SS, et al. Gender
disparities in scholarly productivity within academic otolaryngology departments. Otolaryngol Head Neck Surg. 2013;148(2):215-222. 26. Pashkova AA, Svider PF, Chang CY, Diaz L, Eloy JA,
Comparison of plaintiff and defendant expert witness qualification in malpractice litigation in neurological surgery. J Neurosurg. 2014;120(1):185-190.
Eloy JD. Gender disparity among US anaesthesiologists: are women underrepresented in academic ranks and scholarly productivity? Acta Anaesthesiol Scand. 2013;57(8):1058-1064.
16. Eloy JA, Svider PF, Patel D, Setzen M, Baredes S.
27. Tomei KL, Nahass MM, Husain Q, et al. A gender-
Comparison of plaintiff and defendant expert witness qualification in malpractice litigation in otolaryngology. Otolaryngol Head Neck Surg. 2013;148(5):764-769.
based comparison of academic rank and scholarly productivity in academic neurological surgery. J Clin Neurosci. 2014;21(7):1102-1105.
17. Eloy JA, Svider PF, Patel D, Setzen M, Baredes S. In
28. Svider PF, Mady LJ, Husain Q, et al. Geographic
response to “comparison of plaintiff and defendant expert witness qualification in malpractice litigation in otolaryngology”. Otolaryngol Head Neck Surg. 2013;149(4):649-650.
18. Svider PF, Pashkova AA, Choudhry Z, et al. Compar-
ison of scholarly impact among surgical specialties: an examination of 2429 academic surgeons. Laryngoscope. 2013;123(4):884-889. 19. Eloy JA, Svider PF, Kanumuri VV, Folbe AJ, Setzen
M, Baredes S. Do AAO-HNSF CORE grants predict future NIH funding success? Otolaryngol Head Neck Surg. 2014;151(2):246-252. 20. Eloy JA, Svider PF, Setzen M, Baredes S, Folbe AJ.
differences in academic promotion practices, fellowship training, and scholarly impact. Am J Otolaryngol. 2013;34(5):464-470. 29. Agarwal N, Clark S, Svider PF, Couldwell WT, Eloy
JA, Liu JK. Impact of fellowship training on research productivity in academic neurological surgery. World Neurosurg. 2013;80(6):738-744. 30. Svider PF, Husain Q, Mauro KM, Folbe AJ, Baredes
S, Eloy JA. Impact of mentoring medical students on scholarly productivity. Int Forum Allergy Rhinol. 2014;4(2):138-142. 31. Svider PF, Mauro KM, Sanghvi S, Setzen M, Baredes
S, Eloy JA. Is NIH funding predictive of greater research productivity and impact among academic otolaryngologists? Laryngoscope. 2013;123(1):118-122.
Does receiving an American Academy of Otolaryngology-Head and Neck Surgery Foundation Centralized Otolaryngology Research Efforts grant influence career path and scholarly impact among fellowship-trained rhinologists? Int Forum Allergy Rhinol. 2014;4(1):85-90.
32. Colaco M, Svider PF, Mauro KM, Eloy JA, Jackson-
21. Lopez SA, Svider PF, Misra P, Bhagat N, Langer PD,
33. Eloy JA, Mady LJ, Svider PF, et al. Regional differ-
Eloy JA. Gender differences in promotion and scholarly impact: an analysis of 1460 academic ophthalmologists. J Surg Educ. 2014. http://dx.doi.org/10. 1016/j.jsurg.2014.03.015 [Epub ahead of print]. 22. Svider PF, D’Aguillo CM, White PE, et al. Gender
differences in successful national institutes of health 416
Rosario I. Is there a relationship between National Institutes of Health funding and research impact on academic urology? J Urol. 2013;190(3):999-1003. ences in gender promotion and scholarly productivity in otolaryngology. Otolaryngol Head Neck Surg. 2014;150(3):371-377. 34. Paik AM, Mady LJ, Villanueva NL, et al. Research
productivity and gender disparities: a look at academic plastic surgery. J Surg Educ. 2014;71(4):593-600.
Journal of Surgical Education Volume 72/Number 3 May/June 2015
35. Svider PF, Blake DM, Setzen M, Folbe AJ, Baredes S,
Eloy JA. Rhinology fellowship training and its scholarly impact. Am J Rhinol Allergy. 2013;27(5): e131-e134. 36. Folbe AJ, Svider PF, Setzen M, Zuliani G, Lin HS,
Eloy JA. Scientific inquiry into rhinosinusitis: who is receiving funding from the National Institutes of Health? Laryngoscope. 2014;124(6):1301-1307. 37. Gedde SJ, Budenz DL, Haft P, Lee Y, Quigley HA.
Factors affecting the decision to pursue glaucoma fellowship training. J Glaucoma. 2007;16(1):81-87. 38. Bakri SJ, Alniemi ST, Chan RV. Experiences of
vitreoretinal surgery fellows in the United States. Retina. 2013;33(2):392-396.
39. Frohman LP. The human resource crisis in neuro-
ophthalmology. 231-234.
J
Neuroophthalmol.
2008;28(3):
40. Pagel PS, Hudetz JA. H-index is a sensitive indicator
of academic activity in highly productive anaesthesiologists: results of a bibliometric analysis. Acta Anaesthesiol Scand. 2011;55(9):1085-1089. 41. Purvis A. The h index: playing the numbers game.
Trends Ecol Evol. 2006;21(8):422. 42. Engqvist L, Frommen JG. The h-index and self-
citations. Trends Ecol Evol. 2008;23(5):250-252. 43. Ball P. Achievement index climbs the ranks. Nature.
2007;448(7155):737.
Journal of Surgical Education Volume 72/Number 3 May/June 2015
417
Impact of Fellowship Training on Research Productivity in Academic Ophthalmology Grace Huang, BS,* Christina H. Fang, BS,* Santiago A. Lopez, BS,* Neelakshi Bhagat, MD,* Paul D. Langer, MD,* and Jean Anderson Eloy, MD†,‡,§ *
The Institute of Ophthalmology and Visual Science, Rutgers New Jersey Medical School, Newark, New Jersey; †Department of Otolaryngology—Head and Neck Surgery, Rutgers New Jersey Medical School, Newark, New Jersey; ‡Center for Skull Base and Pituitary Surgery, Neurological Institute of New Jersey, Rutgers New Jersey Medical School, Newark, New Jersey; and §Department of Neurological Surgery, Rutgers New Jersey Medical School, Newark, New Jersey OBJECTIVES: To assess whether scholarly impact of
academic ophthalmologists, as measured using the h-index, is affected by fellowship training status and to further characterize differences in productivity among the various subspecialties and by departmental rank. DESIGN: A descriptive and correlational design was used. In total, 1440 academic ophthalmologists from 99 ophthalmology training programs were analyzed. The h-index data were obtained from the Scopus database. Faculty members were classified by academic rank and grouped into 10 categories based on fellowship training: anterior segment, corneal and external disease, glaucoma, uveitis and ocular immunology, vitreoretinal disease, ophthalmic plastic surgery, pediatric ophthalmology, neuro-ophthalmology, ophthalmic pathology, and “other.” A one-way analysis of variance or Student t test using Microsoft Excel and “R” statistical software were used for comparison of continuous variables, with significance set at p o 0.05. SETTINGS: Faculty working in academic ophthalmology
residency training programs in the United States whose information is stored in the American Medical Association’s Fellowship and Residency Electronic Interactive Database. RESULTS: Fellowship-trained ophthalmologists had signifi-
cantly higher research productivity, as measured using the h-index, than non–fellowship-trained ophthalmologists in this study (p o 0.0005). Academic ophthalmologists trained in vitreoretinal disease or ophthalmic pathology had the highest scholarly productivity compared with those in other ophthalmology subspecialties (p o 0.05). There was a significant increase in scholarly productivity with Correspondence: Inquiries to Jean Anderson Eloy, MD, FACS, Department of Otolaryngology—Head and Neck Surgery, Rutgers New Jersey Medical School, 90 Bergen St., Suite 8100, Newark, NJ 07103; fax: (973) 972-3767; e-mail: jean. [email protected]
410
increasing academic rank from Assistant Professor to Professor (p o 0.05). A significant difference in productivity between fellowship-trained and non–fellowship-trained ophthalmologists existed individually only at the level of Assistant Professor (p o 0.0005). CONCLUSION: Academic ophthalmologists with fellow-
ship training have significantly higher scholarly output than non–fellowship-trained ophthalmologists do, as measured using the h-index. Research productivity increases with departmental academic rank from Assistant Professor to C 2015 Association of Professor. ( J Surg 72:410-417. J Program Directors in Surgery. Published by Elsevier Inc. All rights reserved.) KEY WORDS: fellowship training, academic promotion,
research productivity, scholarly productivity, ophthalmology, h-index COMPETENCIES: Medical Knowledge, Professionalism,
Practice-Based Learning and Improvement
INTRODUCTION Subspecialty training in ophthalmology has become increasingly common over the past 2 decades. Data from the San Francisco Ophthalmology Fellowship Match and the American Society of Ophthalmic and Plastic Reconstructive Surgery fellowship match reveal that approximately 64% of US ophthalmology residents graduating in 2014 entered a fellowship training program. Currently, 3 subspecialty areas—vitreoretinal disease, cornea and external disease, and glaucoma—capture more than 75% of all residents who match into fellowships.1 An anonymous survey sent to 222 graduating ophthalmology residents in 2012 revealed several factors that
Journal of Surgical Education & 2015 Association of Program Directors in Surgery. Published by 1931-7204/$30.00 Elsevier Inc. All rights reserved. http://dx.doi.org/10.1016/j.jsurg.2014.10.010
influenced career choices among trainees. Residents who sought subspecialty training had on average a greater desire to acquire special skills, to enter a (perceived) prestigious field, and to be concerned with a favorable job market. By contrast, lifestyle considerations such as anticipated work hours and geographic preferences were more important to residents choosing a comprehensive ophthalmology career. Residents who chose comprehensive ophthalmology careers were more likely to plan to practice in a private group practice, whereas those seeking subspecialty training were more likely to intend to practice in an academic setting or were undecided in their future practice type.2 Scholarly productivity in the form of research contributions is exceedingly important in determining appointment and promotion within academic medicine.3,4 Greater research productivity has been associated with better clinical care and enhanced research opportunities, which may be factors in the decision to pursue subspecialization.4,5 Increased research output may lead to greater prestige in the field, another factor influencing many graduates who pursue fellowship training.2 Despite its obvious importance, quantifying research contribution is nevertheless complicated and should in theory reflect the cumulative significance of an individual’s publication history.6 Several metrics related to research productivity are commonly used in this evaluation process, such as the total number of publications, funding history, and frequency of citation; however, none of these measures reliably convey the overall quality of scholarly output.7 The h-index was first described in 2005 by Dr. J. E. Hirsch as a tool to assess scholarly productivity. It is an easily computable index and gives an estimate of the broad impact of a scholar’s cumulative research contributions, taking into account both the quantity and quality of publications.8 Briefly, an individual with an h-index of n has published n number of articles, each of which has been cited in the peer-reviewed literature n times. Thus, for example, an individual with an h-index of 15 has published 15 articles that have each been cited at least 15 times in the literature. If this individual with an h-index of 15 has published 50 articles overall, this means that his other 35 articles have been cited less than h (15) times in peerreviewed journals. Therefore, this measurement gauges the general influence of an author’s scholarly contributions and is not disproportionately affected by one or a handful or heavily cited publications.3 Studies using the h-index have repeatedly shown that there is a strong association between the h-index and academic advancement, procurement of funding from the National Institutes of Health (NIH), and other measures of scholarly impact in many fields including otolaryngology, radiology, urology, neurosurgery, and anesthesiology.3,6,9-12 In otolaryngology, it has been found that fellowship-trained academic practitioners have greater research productivity, as measured using the h-index, than their non–fellowship-
trained colleagues.3 There have been no previous studies comparing whether postresidency fellowship training has a similar impact on scholarly influence in ophthalmology. The purpose of this study was to determine whether the scholarly impact of faculty members of academic ophthalmology departments, as measured using the h-index, is correlated to fellowship training status. In addition, our study aimed to further characterize the differences in research productivity among academic ophthalmologists by subspecialty and by academic rank.
MATERIALS AND METHODS A list of academic ophthalmology residency training programs in the United States was generated using the American Medical Association’s Fellowship and Residency Electronic Interactive Database. Faculty information such as academic rank and fellowship training was obtained from online listings taken from the website of each academic department. Faculty members whose academic rank was undeterminable from their online profiles or whose program’s website did not present pertinent information regarding fellowship attainment were excluded from this analysis. Additionally, nonacademic, nonphysician, and part-time faculty were not included in this study. The subspecialties included in this study were obtained from the San Francisco Ophthalmology Fellowship Match1 and comprise anterior segment, cornea and external disease, glaucoma, uveitis and ocular immunology, vitreoretinal diseases, ophthalmic plastic surgery, pediatric ophthalmology, neuro-ophthalmology, ophthalmic pathology, and “other” fellowships. “Other” fellowships were those fellowships that were not listed specifically from the San Francisco Ophthalmology Fellowship Match including ocular oncology, molecular ophthalmology, ophthalmic genetics, ocular pharmacology, and ocular immunology. The initial Fellowship and Residency Electronic Interactive Database search yielded 117 ophthalmology training programs from which 18 institutions were excluded. After application of the exclusion criteria, 1440 academic ophthalmologists from 99 departments were examined. To obtain data concerning an individual’s h-index, the Scopus database (http://www. scopus.com) was used to search for each author as this database covers more than 40 million publication records from 18,500 peer-reviewed venues. The Scopus database has been previously used in this manner in a plethora of medical and basic sciences.3,6,9-36 Although other h-index database calculators such as Google Scholar or ISI Web of Knowledge exist, studies have published similar results when comparing them, suggesting a strong correlation between h-indices obtained from either Google Scholar or Scopus.11 In addition, geographic location of the ophthalmologists’ department was obtained and grouped into one of the following regions designated by the US Census
Journal of Surgical Education Volume 72/Number 3 May/June 2015
411
Bureau. These included the following: (1) Northeast: New England (CT, ME, MA, NH, RI, and VT) and MidAtlantic (NJ, NY, and PA); (2) Midwest: East North Central (IN, IL, MI, OH, and WI) and West North Central (IA, KS, MN, MO, NE, ND, and SD); (3) South: South Atlantic (DE, DC, FL, GA, MD, NC, SC, VA, and WV), East South Central (AL, KY, MS, and TN), and West South Central (AR, LA, OK, and TX); and (4) West: Mountain (AZ, CO, ID, NM, MT, UT, NV, and WY) and Pacific (AK, CA, HI, OR, and WA). To ensure accuracy in our results, initial search results for each individual author were cross-matched by departmental affiliations, previous positions in other departments, and the presence or absence of publications in ophthalmology or ophthalmology-related journals. This methodology ensured any results obtained pertained to the appropriate individual. Data collection was completed in April 2014. Statistical Analysis The Kruskal-Wallis one-way analysis of variance or Student t test was used for comparison of continuous variables. Threshold for statistical significance was set at p o 0.05, effectively rejecting the null hypothesis. Microsoft Excel (Redmond, WA) and “R” statistical software (http://www. r-project.org) were used for statistical calculation.
RESULTS Of the 1440 academic ophthalmologists included in this analysis, 83% were trained in 1 of the 9 fellowships examined (Fig. 1). None of the included ophthalmologists
FIGURE 1. Distribution of the academic ophthalmologists according to subspecialty. 412
FIGURE 2. The mean h-index of fellowship-trained and non–fellowship-trained academic ophthalmologists from 99 departments. n, sample size; error bars, standard error of the mean.
were trained in an “anterior segment” fellowship. When measured using the h-index, fellowship-trained academic ophthalmologists had significantly higher research productivity than the non–fellowship-trained academic ophthalmologists included in this study (t test, p o 0.0005) (Fig. 2). There were significant statistical differences among the 9 fellowships examined (Kruskal-Wallis analysis of variance, p o 0.05). Academic ophthalmologists trained in vitreoretinal disease or ophthalmic pathology had higher scholarly productivity than those trained in ophthalmic plastic surgery, pediatric ophthalmology, cornea and external disease, or uveitis and ocular immunology (p o 0.0005) (Fig. 3). There was no statistical difference in research productivity, as measured using the h-index, between ophthalmologists who trained in vitreoretinal disease, ophthalmic pathology, neuroophthalmology, glaucoma, and other fellowships (p 4 0.05). Mean h-index increased with the number of fellowships obtained. Ophthalmologists with zero fellowships had a mean h-index of 7.03, whereas those with 3 or more fellowships had a mean h-index of 10.86. Mean h-index of ophthalmologists with 1 or 2 fellowships were similar at 9.57 and 9.42, respectively. When stratifying ophthalmologists by geographic location of practice, the highest mean h-index of 14.04 was revealed in the West, and the lowest mean h-index of 8.71 was revealed in the South. The Table summarizes h-indices by region and division. There was a consistent and significant increase in scholarly productivity, as measured using the h-index, with increases in academic rank from Assistant Professor to Professor (Fig. 4). There was no statistical difference in scholarly productivity between Professors and Chairpersons. A significant difference in research productivity between fellowship-trained and non– fellowship-trained academic ophthalmologists was only seen at the level of Assistant Professors (p o 0.0005). At the level of Associate Professor and Professor, fellowship-trained academic ophthalmologists had higher h-indices, but this did not reach statistical significant (p ¼ 0.22 and 0.11, respectively).
Journal of Surgical Education Volume 72/Number 3 May/June 2015
FIGURE 3. Mean h-indices of the academic ophthalmologists included in this study distributed by fellowship. n, sample size; error bars, standard error of the mean.
DISCUSSION The proportion of ophthalmology residents who apply for fellowships has been steadily rising for the past decade according to data from the National Residency Matching Program and Ophthalmology Fellowship Match Program.37 Fellowship training provides graduating residents with an in-depth exposure to a subspecialty area of ophthalmology, allowing them to develop focused clinical and surgical skills related to that subspecialty area. In our study, we found that fellowship-trained ophthalmologists had significantly higher research productivity, as measured using the h-index, compared with non–fellowship-trained ophthalmologists (h-index ¼ 9.60 and 7.05, respectively; p ¼ 0.0004). Ophthalmologists that pursue advanced TABLE. Mean h-Index by US Region and Division Region
Average h-Index
Midwest
9.95
Northeast
8.04
West
14.04
South
8.71
Division East North Central West North Central New England Mid-Atlantic Pacific Mountain South Atlantic East South Central West South Central
Average h-Index 8.89 11.00 8.27 7.81 11.46 16.63 9.97 7.72 8.44
fellowship training may have higher scholarly productivity than their non–fellowship-trained colleagues, because they may be inclined to participate in research or because more research opportunities are available to fellowship-trained individuals. Greater research productivity has far-reaching implications for academic physicians. In addition to its effect on appointment decisions, research output is an important aspect of determining an academic physician’s tenure, promotion, and ranking within an academic department.18 A study found that when strategies were introduced to assess the productivity of faculty members at academic medical centers, the faculty appeared to have improved research productivity and possibly improved clinical productivity.7 Having highly productive faculty has potential advantages for an academic department, in terms of not only notoriety and prestige, but also as an enhanced ability to recruit other high-quality faculty and residents.10 The h-index attempts to incorporate both the quantity and the quality of an individual’s scholarly accomplishments by measuring how often an author is producing influential work. It is increasingly being used throughout a number of scientific disciplines to assess scholarly productivity among faculty members.6,9,18,29 In ophthalmology, it has been used to assess gender differences in scholarly impact,21 to examine the association between scholarly impact and NIH award procurement,10 and to examine whether gender differences in NIH awards and funding totals exist.22 This study is the first investigation that aimed to determine the relationship between fellowship training and scholarly impact among academic ophthalmologists.
Journal of Surgical Education Volume 72/Number 3 May/June 2015
413
FIGURE 4. The mean h-index of fellowship-trained and non–fellowship-trained ophthalmologists stratified by academic rank. Errors bars, standard error of the mean.
When comparing h-indices among the various ophthalmology subspecialties, significant differences emerged. Academic ophthalmologists trained in vitreoretinal disease and ophthalmic pathology had the highest average h-indices (h-index ¼ 11.64 and 11.55, respectively). These h-indices were significantly different from those of academic ophthalmologists that had trained in uveitis and ocular immunology, cornea and external disease, pediatric ophthalmology, and ophthalmic plastic surgery fellowships. These findings suggest that emphasis on research may be stronger in vitreoretinal disease and ophthalmic pathology fellowships. Retina fellows appear to be very productive in research and publications, with 40% having dedicated research time during their fellowship training.38 Another factor that might be considered when evaluating academic productivity among specialties is the number of practicing physicians in each particular subspecialty. For example, a physician shortage apparently exists in both neuroophthalmology and pediatric ophthalmology.39 Academic pediatric ophthalmologists had the second to lowest h-index (8.29) of all fellowship-trained academic ophthalmologists. A shortage of physicians in a subspecialty may theoretically lead to greater clinical responsibilities compared with other subspecialties and a corresponding reduction in protected research time, leading to fewer publications and fewer citations. When stratifying h-indices by academic rank, the h-index increased with academic rank from Assistant Professor to Professor, with no significant difference between Professor and Chairperson. The positive correlation between h-index and increasing academic rank is consistent with previous studies.3,6,11,12,29,40 These data confirm that research is strongly considered in the process of academic promotion. When the cohorts of fellowship-trained and non– 414
fellowship-trained academic ophthalmologists were stratified by academic rank, significant differences in h-indices were observed only at the level of Assistant Professor. At the level of Associate Professor and Professor, the trend continued revealing higher h-indices in the fellowship-trained ophthalmologists compared with their non–fellowship-trained colleagues of equal rank; however, this comparison did not achieve statistical significance. An explanation of this result is possibly due to the smaller n value in the non–fellowshiptrained ophthalmologists at the level of Associate Professor and Professor, making it more difficult to detect significance (Fig. 4). Upon further analysis, the p value was lower (0.09) when combining the ranks of Associate Professor and Professor to increase n when comparing fellowship-trained vs non–fellowship-trained ophthalmologists. This suggests that increasing n may reveal statistical significance. These data also provide a statistical benchmark of the mean h-index scores of academic ophthalmologists at different points in their careers. There are several limitations to this study. A criticism of the h-index is its insensitivity to the type of research an academic physician conducts.8 For example, an ophthalmologist with an interest in basic science who participates in laboratory-oriented projects may be involved in more timeintensive work than a colleague producing influential publications on clinical research that requires access only to medical records. Each of these types of scholarship is valuable, but the number of articles published and the number of citations may not accurately reflect the relative effect of a physician performing these different types of research.18 Another weakness of the h-index is the potential of self-citation, which may artificially inflate this statistic.41 However, to significantly raise the h-index, repeated and
Journal of Surgical Education Volume 72/Number 3 May/June 2015
sustained citation of an individual’s work would be required, making this an unlikely and rare phenomenon. In reality, if selective self-citations were to occur, it is unlikely to have more than a negligible effect on an author’s h-index.42 Another potential limitation of the h-index is that it does not necessarily account for the level of input from each author of a given published article. Individuals who work with frequently published authors may have high h-index scores, effectively piggy-backing off the work of their colleagues.29 However, it is difficult to quantify an author’s actual level of contribution to any article, rendering this phenomenon quite difficult to accurately assess by any metric.6 Lastly, this study groups together all physicians with a Doctor of Medicine degree pursuing fellowship training. It is important to note that many of these physicians may have additional postgraduate degrees, including Master of Business Administration degrees, Doctor of Philosophy degrees, or degrees in Public Health or Statistics. These physicians may lead careers that are more research oriented; therefore, obtaining higher h-indices. It would be interesting for a future study to stratify physicians by postgraduate degrees and analyze the effect on their h-index. The importance of having high-quality non–fellowshiptrained comprehensive ophthalmology faculty in an academic department cannot be overstated. These physicians are crucial in providing and maintaining the broad educational experience necessary for competence in office-based and operative management of a wide variety of ophthalmic conditions.3 The responsibilities of comprehensive ophthalmologists in academic departments are often shifted away from research output toward clinical service and the education of residents and medical students. It is therefore not surprising that, on the whole, especially at the more junior academic ranks, research productivity of comprehensive ophthalmologists in academia suffers by comparison with that of their fellowship-trained colleagues. Scholarly productivity, as measured using the h-index, will continue to be a major factor in the evaluation process in academic medicine. It is increasing in popularity, and job applicants have even started to include an h-index in their curriculum vitae.43 There are understandable reservations toward reducing research output to a single variable, and caution should be exercised when using this measurement without being aware of its limitations. Other factors such as clinical performance, educational contributions, and administrative duties also are important when evaluating performance in academia.3 Nevertheless, the h-index will continue to be one tool that can objectively, reliably, and efficiently assess scholarly relevance in academic medicine.
promotion within academic medicine. As subspecialty training becomes increasingly common in ophthalmology, it is important to determine whether there is an association between fellowship training and level of scholarly productivity. Academic ophthalmologists with fellowship training have significantly higher scholarly output than non–fellowship-trained ophthalmologists, as measured using the h-index. Research productivity increases with departmental academic rank from Assistant Professor to Professor.
CONCLUSION
10. Svider PF, Lopez SA, Husain Q, Bhagat N, Eloy JA,
Scholarly productivity in the form of research contributions is a major factor in determining appointment and
REFERENCES 1. Residency and Fellowship Match. Available at: 〈http://
www.sfmatch.org/SpecialtyInsideAll.aspx?id=2&typ=1 &name=Ophthalmology〉; [Accessed 23.4.14]. 2. Gedde SJ, Budenz DL, Haft P, Tielsch JM, Lee Y,
Quigley HA. Factors influencing career choices among graduating ophthalmology residents. Ophthalmology. 2005;112(7):1247-1254. 3. Eloy JA, Svider PF, Mauro KM, Setzen M, Baredes S.
Impact of fellowship training on research productivity in academic otolaryngology. Laryngoscope. 2012;122(12): 2690-2694. 4. Dixon AK. Publishing and academic promotion.
Singapore Med J. 2009;50(9):847-850. 5. Beasley BW, Wright SM, Cofrancesco J Jr., Babbott
SF, Thomas PA, Bass EB. Promotion criteria for clinician-educators in the United States and Canada. A survey of promotion committee chairpersons. J Am Med Assoc. 1997;278(9):723-728. 6. Kasabwala K, Morton CM, Svider PF, Nahass TA,
Eloy JA, Jackson-Rosario I. Factors influencing scholarly impact: does urology fellowship training affect research output? J Surg Educ. 2014;71(3):345-352. 7. Akl EA, Meerpohl JJ, Raad D, et al. Effects of assessing
the productivity of faculty in academic medical centres: a systematic review. CMAJ. 2012;184(11): E602-E612. 8. Hirsch JE. An index to quantify an individual’s
scientific research output. Proc Natl Acad Sci U S A. 2005;102(46):16569-16572. 9. Svider PF, Choudhry ZA, Choudhry OJ, Baredes S,
Liu JK, Eloy JA. The use of the h-index in academic otolaryngology. Laryngoscope. 2013;123(1):103-106. Langer PD. The association between scholarly impact and national institutes of health funding in ophthalmology. Ophthalmology. 2014;121(1):423-428.
Journal of Surgical Education Volume 72/Number 3 May/June 2015
415
11. Lee J, Kraus KL, Couldwell WT. Use of the h index in
neurosurgery. Clinical article. J Neurosurg. 2009;111 (2):387-392. 12. Rad AE, Brinjikji W, Cloft HJ, Kallmes DF. The H-
funding in ophthalmology. J Surg Educ. 2014;71(5): 680-688. 23. Eloy JA, Svider PF, Kovalerchik O, Baredes S,
index in academic radiology. Acad Radiol. 2010;17(7): 817-821.
Kalyoussef E, Chandrasekhar SS. Gender differences in successful NIH grant funding in otolaryngology. Otolaryngol Head Neck Surg. 2013;149(1):77-83.
13. Eloy JA, Svider PF, Folbe AJ, Setzen M, Baredes S.
24. Eloy JA, Svider PF, Cherla DV, et al. Gender disparities
AAO-HNSF CORE grant acquisition is associated with greater scholarly impact. Otolaryngol Head Neck Surg. 2014;150(1):53-60.
in research productivity among 9952 academic physicians. Laryngoscope. 2013;123(8):1865-1875.
14. Svider PF, Husain Q, Folbe AJ, Couldwell WT, Liu
JK, Eloy JA. Assessing National Institutes of Health funding and scholarly impact in neurological surgery. J Neurosurg. 2014;120(1):191-196. 15. Eloy JA, Svider PF, Folbe AJ, Couldwell WT, Liu JK.
25. Eloy JA, Svider P, Chandrasekhar SS, et al. Gender
disparities in scholarly productivity within academic otolaryngology departments. Otolaryngol Head Neck Surg. 2013;148(2):215-222. 26. Pashkova AA, Svider PF, Chang CY, Diaz L, Eloy JA,
Comparison of plaintiff and defendant expert witness qualification in malpractice litigation in neurological surgery. J Neurosurg. 2014;120(1):185-190.
Eloy JD. Gender disparity among US anaesthesiologists: are women underrepresented in academic ranks and scholarly productivity? Acta Anaesthesiol Scand. 2013;57(8):1058-1064.
16. Eloy JA, Svider PF, Patel D, Setzen M, Baredes S.
27. Tomei KL, Nahass MM, Husain Q, et al. A gender-
Comparison of plaintiff and defendant expert witness qualification in malpractice litigation in otolaryngology. Otolaryngol Head Neck Surg. 2013;148(5):764-769.
based comparison of academic rank and scholarly productivity in academic neurological surgery. J Clin Neurosci. 2014;21(7):1102-1105.
17. Eloy JA, Svider PF, Patel D, Setzen M, Baredes S. In
28. Svider PF, Mady LJ, Husain Q, et al. Geographic
response to “comparison of plaintiff and defendant expert witness qualification in malpractice litigation in otolaryngology”. Otolaryngol Head Neck Surg. 2013;149(4):649-650.
18. Svider PF, Pashkova AA, Choudhry Z, et al. Compar-
ison of scholarly impact among surgical specialties: an examination of 2429 academic surgeons. Laryngoscope. 2013;123(4):884-889. 19. Eloy JA, Svider PF, Kanumuri VV, Folbe AJ, Setzen
M, Baredes S. Do AAO-HNSF CORE grants predict future NIH funding success? Otolaryngol Head Neck Surg. 2014;151(2):246-252. 20. Eloy JA, Svider PF, Setzen M, Baredes S, Folbe AJ.
differences in academic promotion practices, fellowship training, and scholarly impact. Am J Otolaryngol. 2013;34(5):464-470. 29. Agarwal N, Clark S, Svider PF, Couldwell WT, Eloy
JA, Liu JK. Impact of fellowship training on research productivity in academic neurological surgery. World Neurosurg. 2013;80(6):738-744. 30. Svider PF, Husain Q, Mauro KM, Folbe AJ, Baredes
S, Eloy JA. Impact of mentoring medical students on scholarly productivity. Int Forum Allergy Rhinol. 2014;4(2):138-142. 31. Svider PF, Mauro KM, Sanghvi S, Setzen M, Baredes
S, Eloy JA. Is NIH funding predictive of greater research productivity and impact among academic otolaryngologists? Laryngoscope. 2013;123(1):118-122.
Does receiving an American Academy of Otolaryngology-Head and Neck Surgery Foundation Centralized Otolaryngology Research Efforts grant influence career path and scholarly impact among fellowship-trained rhinologists? Int Forum Allergy Rhinol. 2014;4(1):85-90.
32. Colaco M, Svider PF, Mauro KM, Eloy JA, Jackson-
21. Lopez SA, Svider PF, Misra P, Bhagat N, Langer PD,
33. Eloy JA, Mady LJ, Svider PF, et al. Regional differ-
Eloy JA. Gender differences in promotion and scholarly impact: an analysis of 1460 academic ophthalmologists. J Surg Educ. 2014. http://dx.doi.org/10. 1016/j.jsurg.2014.03.015 [Epub ahead of print]. 22. Svider PF, D’Aguillo CM, White PE, et al. Gender
differences in successful national institutes of health 416
Rosario I. Is there a relationship between National Institutes of Health funding and research impact on academic urology? J Urol. 2013;190(3):999-1003. ences in gender promotion and scholarly productivity in otolaryngology. Otolaryngol Head Neck Surg. 2014;150(3):371-377. 34. Paik AM, Mady LJ, Villanueva NL, et al. Research
productivity and gender disparities: a look at academic plastic surgery. J Surg Educ. 2014;71(4):593-600.
Journal of Surgical Education Volume 72/Number 3 May/June 2015
35. Svider PF, Blake DM, Setzen M, Folbe AJ, Baredes S,
Eloy JA. Rhinology fellowship training and its scholarly impact. Am J Rhinol Allergy. 2013;27(5): e131-e134. 36. Folbe AJ, Svider PF, Setzen M, Zuliani G, Lin HS,
Eloy JA. Scientific inquiry into rhinosinusitis: who is receiving funding from the National Institutes of Health? Laryngoscope. 2014;124(6):1301-1307. 37. Gedde SJ, Budenz DL, Haft P, Lee Y, Quigley HA.
Factors affecting the decision to pursue glaucoma fellowship training. J Glaucoma. 2007;16(1):81-87. 38. Bakri SJ, Alniemi ST, Chan RV. Experiences of
vitreoretinal surgery fellows in the United States. Retina. 2013;33(2):392-396.
39. Frohman LP. The human resource crisis in neuro-
ophthalmology. 231-234.
J
Neuroophthalmol.
2008;28(3):
40. Pagel PS, Hudetz JA. H-index is a sensitive indicator
of academic activity in highly productive anaesthesiologists: results of a bibliometric analysis. Acta Anaesthesiol Scand. 2011;55(9):1085-1089. 41. Purvis A. The h index: playing the numbers game.
Trends Ecol Evol. 2006;21(8):422. 42. Engqvist L, Frommen JG. The h-index and self-
citations. Trends Ecol Evol. 2008;23(5):250-252. 43. Ball P. Achievement index climbs the ranks. Nature.
2007;448(7155):737.
Journal of Surgical Education Volume 72/Number 3 May/June 2015
417
