ORIGINAL REPORTS
Does Intentional Support of Degree Programs in General Surgery Residency Affect Research Productivity or Pursuit of Academic Surgery? Jesse Joshua Smith, MD, PhD*,1 Ravi K. Patel, MD*,1 Xi Chen, PhD† Margaret J. Tarpley, MLS* and Kyla P. Terhune, MD* Department of Surgery, Vanderbilt University Medical Center, Nashville, Tennessee; and †Division of Cancer Biostatistics, Department of Biostatistics, Vanderbilt University Medical Center, Nashville, Tennessee *
OBJECTIVE: Many residents supplement general surgery training with years of dedicated research, and an increasing number at our institution pursue additional degrees. We sought to determine whether it was worth the financial cost for residency programs to support degrees. DESIGN: We reviewed graduating chief residents (n ¼ 69)
in general surgery at Vanderbilt University from 2001 to 2010 and collected the data including research time and additional degrees obtained. We then compared this information with the following parameters: (1) total papers, (2) first-author papers, (3) Journal Citation Reports impact factors of journals in which papers were published, and (4) first job after residency or fellowship training. SETTING: The general surgery resident training program at Vanderbilt University is an academic program, approved to finish training 7 chief residents yearly during the time period studied.
significantly correlated with a first job in academia if compared to those with dedicated research time only (p ¼ 0.046). CONCLUSIONS: Our data support the utility of degree
completion when economically feasible and use of dedicated research time as an effective way to significantly increase research productivity and retain graduates in academic surgery. Aggregating data from other academic surgery programs would allow us to further determine association of funding of additional degrees as a means to encourage academic producC 2014 Association of tivity and retention. ( J Surg ]:]]]-]]]. J Program Directors in Surgery. Published by Elsevier Inc. All rights reserved.) KEYWORDS: resident education, graduate degree, research
fellowship, general surgery/education COMPETENCY: Practice-Based Learning and Improvement
PARTICIPANTS: Chief residents in general surgery at
Vanderbilt who finished their training 2001 through 2010.
RESULTS: We found that completion of a degree during
residency was significantly associated with more total and first-author publications as compared with those by residents with only dedicated research time (p ¼ 0.001 and p ¼ 0.017). Residents completing a degree also produced publications of a higher caliber and level of authorship as determined by an adjusted resident impact factor score as compared with those by residents with laboratory research time only (p ¼ 0.005). Degree completion also was
Correspondence: Inquiries to Kyla P. Terhune, MD, Department of Surgery, Vanderbilt University, D-4309 MCN, 1161 21st Ave S., Nashville, TN 37232-2730; fax: (615) 322-0689.; e-mail: [email protected], [email protected] 1
These authors contributed equally to this work.
INTRODUCTION As many as 36% of residents at academic institutions across the nation supplement their clinical training for a mean of 1.7 years with research fellowships.1 During these years, residents are expected to be productive, present nationally, publish papers, write chapters, and prepare for and augment the beginnings of an academic career by increasing their chances of entering competitive fellowships and academic positions. However, these years require funding of salary, benefits, and insurance through grants or direct institutional support. From a survey sent in 2006 to all National Resident Matching Program participating program directors, Robertson et al.1 estimated that it costs $41.5 million to fund the 634 trainees involved in research fellowships
Journal of Surgical Education & 2014 Association of Program Directors in Surgery. Published by 1931-7204/$30.00 Elsevier Inc. All rights reserved. http://dx.doi.org/10.1016/j.jsurg.2014.01.018
1
Growth of Interest in Degree Programs 6
Residents
5 4 Lab Only
3
All Degrees
2
Linear (Lab Only)
1
Linear (All Degrees)
0 2001 2002 2003 2004 2005 2006 2007 2008 2009 2010 Years
FIGURE. Increasing trend amongst general surgery residents at this institution to pursue a graduate degree during dedicated research time. Positive trend shows the growth of residents pursuing a graduate degree during their dedicated laboratory research time as compared with completing laboratory research without concurrent degree pursuits.
each year, primarily paid through departmental funds and institutional training grants. At our tertiary care institution, there appears to be a trend toward the pursuit of additional degrees during the resident research years (Fig.). This trend may be influenced by a national interest in and focus on outcomes research.2 However, when determining whether to support such degree programs, a surgical program must take into account the weighty expense and determine the ultimate outcome desired providing this opportunity. Potential outcomes could be individual, institutional, or global. Individually, by augmenting a resident’s publication credentials, chances of obtaining a fellowship or faculty position would be increased. Institutionally, increased productivity and national exposure would reflect well on the respective program and academic medical center. Globally, retention and productivity of the academic surgery community as a whole could be augmented. At this institution, there are 2 popular pathways for additional degrees that are very feasible during dedicated research time. The Master of Public Health (MPH) is an interdisciplinary program founded in 1996 to train public health scientists and professionals dedicated to improving the public health and preventing disease and disability. The Master of Science in Clinical Investigation (MSCI) Program was approved in 1999 and trains investigators in the techniques and processes used in patient-oriented research. Tuition for Vanderbilt University’s MPH and MSCI in 2012 to 2013 was $46,425 and $47,000, respectively.3,4 Given this expense, we sought to determine from an administrative perspective whether the end result justified the expenses, using the individual, institutional, and global goals outlined earlier as a guide. We examined graduating chief residents at Vanderbilt for the years 2001 through 2010 comparing those completing degree programs vs those who did not. The outcomes measured included whether residents completed degree programs, whether they completed dedicated research time, their publication record, and their first job after completion of residency and fellowship training (e.g., community, military, academic affiliated, or academic). 2
METHODS A request for exemption was approved by the Vanderbilt University’s institutional review board for this study. We reviewed all graduating chief residents (n ¼ 69) from general surgery at Vanderbilt University Medical Center from 2001 through 2010, determined whether they had spent any years in dedicated research time, and recorded whether they had pursued an additional degree, including MSCI, MPH, Doctor of Philosophy, and Master of Business Administration (Table 1). One resident who completed only 2 years of residency within the United States was excluded, as he was not afforded the opportunity to complete dedicated research time. Outcome measures were dependent on the following: (1) total papers published; (2) paper contribution level (designated as first author, second author, last author, or collaborative papers); (3) Journal Citation Reports (JCR) impact factors of journals5; and (4) classification of first job after residency or fellowship training (community, academic, military, or academicaffiliated) (Table 2). PubMed was used as a search engine for a time frame of 7 years before the completion of residency through 1 year
TABLE 1. Degree program completion. Laboratory research experience and degree completion for surgical residents under study at Vanderbilt University are depicted (n ¼ 68). No significant difference (p ¼ 0.294) for average laboratory research years existed between residents completing a degree (2.1 y) and those who completed only structured laboratory research time (2.0 y) Research Years
Residents
Mean Laboratory Years
MPH þ laboratory MBA þ laboratory MS þ laboratory MSCI þ laboratory Laboratory only No laboratory Total residents
10 1 1 5 27 24 68
2.1 2.0 3.0 2.0 2.0 0.0 1.3
MBA, Master of Business Administration; MS, Master of Science. Journal of Surgical Education Volume ]/Number ] ] 2014
TABLE 2. First job. Description of first jobs for general surgery residents after completion of residency or fellowship training or both First Job Academics Academic affiliated Community Military Total residents
Degree Program 14 (82%)
Laboratory No Only Laboratory 14 (52%)
7 (29%)
0 (0%)
4 (15%)
1 (4%)
1 (6%) 2 (12%) 17
6 (22%) 3 (11%) 27
15 (63%) 1 (4%) 24
after the completion of chief residency to account for the length of residency and publishing delays. Results were filtered by resident names and institutional affiliations for both during residency and the year after residency to include all publications. The JCR of 2010 and level of contribution were the components used to establish the productivity levels of all papers published by residents. By convention, we assumed the sequence of authors in publications was in reverse order of contribution, excepting the last author position, which was assumed to be the principal investigator.6 Two scores then were developed to summarize and quantify a resident’s productivity. The first was the unweighted resident impact factor (RIF), defined as the sum of JCR impact factors for each of the journals within which the resident had a publication, yielding a single score for the productivity of the resident. The second score was the adjusted RIF (aRIF), which was weighted to account for levels of authorship and thus contribution. Multiplying factors were assigned for levels of authorship as follows: first or last author ¼ 3, second author ¼ 2, and collaborating author ¼ 1. These were then multiplied by the impact factor of the journal of the publication, the sum of these products yielding the aRIF. This aRIF thus considers not only number of publications and contribution level, but also impact and potential significance of the work. A total of 31 papers analyzed were published within journals without an assigned JCR impact factor and were excluded. These excluded papers were distributed as follows: 26 from residents with dedicated research time and 5 from residents without dedicated research time. Of those with dedicated research time, 16 were from residents who completed a degree and 10 from residents who did not complete a degree. Information regarding first job was obtained from the surgical education office and verified by the general surgery program director at this institution (present since 1993). Academic institutions were defined as those with residency or fellowship training programs, while academic-affiliated institutions were defined as those where residents rotate but are without an independently accredited general surgery training program. Wilcoxon rank sum tests with continuity correction were used to determine differences in total publications, firstJournal of Surgical Education Volume ]/Number ] ] 2014
author publications, RIFs, and aRIFs between residents who completed a degree during their residency and those who did not. The analysis compared all residents within both the groups. The analysis also compared residents who completed degrees against those who pursued only dedicated research time. Additionally, both of these groups were compared with those who did not spend any time in dedicated research years but instead completed the residency program without any gaps in training. Individual degree groups (MPH and MSCI) were also examined against dedicated research time only and absence of dedicated research time. Differences between mean years spent in dedicated research time were determined by the MannWhitney U test. Pearson chi-square test with Yates continuity correction was used to determine if degree completion as compared with dedicated research time only correlated with pursuing a first job in academic general surgery. Additional baseline comparisons of both groups were made with those who did not have any dedicated research time during residency. All residents pursuing their first job in the military were excluded from this analysis owing to length of military commitment and delay of fellowship training.
RESULTS We hypothesized that the pursuit of a degree during residency increased productivity and would also correlate with transition into a posttraining academic position. For residents with dedicated research time, we compared residents who completed a degree and those who did not. Completion of a degree during residency was significantly associated with more publications and first-author papers when compared with those who did not complete degrees (Table 3; n ¼ 10.3 vs 5.30, p o 0.001 and n ¼ 4.06 vs 2.30, p ¼ 0.017). Residents completing a degree were also more productive in terms of the caliber of papers published and levels of authorship when comparing the calculated RIF and the aRIF scores (32.3 vs 17.8 score, p ¼ 0.001 and 65.4 vs 36.1 score, p ¼ 0.005). The mean number of years in the laboratory for both groups, degree completed (2.1 y) and laboratory only (2.0 y), were not significantly different (Table 1; 2.1 vs 2.0 y, p ¼ 0.294). In subgroup analysis, residents completing an MPH were significantly more productive than laboratory research only residents in all previously mentioned accounts (p r 0.004). Those completing an MSCI had a significantly higher RIF compared with that of residents in dedicated research only (p ¼ 0.046), but this significance was not seen in total publications, first-author publications, or in the aRIF (Table 3). They still were significantly more productive than residents who had no dedicated research time on all accounts (p r 0.001; Table 3). Although residents completing dedicated research time only were not as productive as their 3
TABLE 3. Productivity of residents completing degree programs. Residents completing a degree program were significantly more productive compared with residents with only dedicated laboratory research time in terms of total publications (p ¼ 0.001) and firstauthor publications (p ¼ 0.0017) with an average of 10.3 publications and 4.06 first-author publications. Those completing a degree were also more productive in terms of producing higher-caliber publications with overall higher levels of authorship as shown by the resident impact factor (p ¼ 0.001) and adjusted resident impact factor (p ¼ 0.005), respectively Total Publications Comparisons (Mean) p Values
Laboratory only (5.30) No laboratory time (1.29)
All Degrees (10.3)
MPH (12.8)
MSCI (6.20)
No Laboratory Time (1.29)
0.001 o0.001
o0.001 o0.001
0.132 o0.001
0.002 —
Total First-Author Publications Comparisons (Mean) p Values
Laboratory only (2.30) No laboratory time (0.458)
All Degrees (4.06)
MPH (5.70)
MSCI (2.20)
No Laboratory Time (0.458)
0.017 o0.001
0.001 o0.001
0.311 0.001
0.003 —
Comparisons of Resident Impact Factors (Mean) p Values
Laboratory only (17.8) No laboratory time (2.69)
All Degrees (32.3)
MPH (35.7)
MSCI (26.2)
No Laboratory Time (2.69)
0.001 o0.001
0.004 o0.001
0.046 o0.001
o0.001 —
Comparisons of Adjusted Resident Impact Factors (Mean) p Values
Laboratory only (36.1) No laboratory time (5.17)
All Degrees (65.4)
MPH (79.5)
MSCI (46.8)
No Laboratory Time (5.17)
0.005 o0.001
0.003 o0.001
0.168 o0.001
o0.001 —
counterparts who completed a degree, they were more productive than residents without dedicated research time in all of the aforementioned factors (p r 0.003; Table 3). Residents completing a degree were more productive and more likely to pursue a first job in academics when compared with residents who completed dedicated research time without a degree (Table 4; 93% vs 58%, p ¼ 0.046). When an academic job definition was expanded to include both academic and academic-affiliated positions, the association was no longer significant (p ¼ 0.306). Residents with dedicated research time only were no more likely to pursue academics compared with those without any dedicated research time (Table 4; 58% vs 30%, p ¼ 0.103).
However, when expanded to consider both academic and academic-affiliated positions, there was a significant difference (Table 4; 75% vs 35%, p ¼ 0.013).
DISCUSSION Conventional wisdom and tradition have promoted dedicated research time during residency as an important part of general surgery training for those intending to pursue an academic career. This trend has been perpetuated by both experienced surgeons looking retrospectively as well as those currently in training.7-9 General surgery is one of the few
TABLE 4. Proportion of academic appointments for residents completing a degree. Residents completing a degree program were significantly more likely to pursue a first job in academics as compared with their counterparts who completed only dedicated laboratory research First Job Degree program Laboratory only p Value Degree program No laboratory time p Value Laboratory only No laboratory time p Value 4
Academics
Academics þ Academic Affiliated
93.3% (14/15) 58.3% (14/24) 0.046 93.3% (14/15) 30.4% (7/23) o0.001 58.3% (14/24) 30.4% (7/23) 0.103
93.3% (14/15) 75.0% (18/24) 0.306 93.3% (14/15) 34.8% (8/23) 0.001 75.0% (18/24) 34.8% (8/23) 0.013 Journal of Surgical Education Volume ]/Number ] ] 2014
specialties in which formal research training (e.g., postdoctoral research fellowship) occurs during the residency training period for a significant proportion of physicians.9 It has been shown that residents are more likely to pursue fellowship training in the field related to their dedicated research time.10 In a review of the monies spent on residents at our institution in 1 year, almost $450,000 was spent to support residents not covered on T32 grants (6 of 12 residents in research) and $70,000 was used to support those on T32 support to bridge the gap between stipend funding and the need for mandatory fringe-benefit coverage. The total support required for this research enterprise is approximately $520,000 per year and does not take into account the monies spent on travel expenses and other miscellaneous items that support “resident in research” training ventures. As the trend to pursue graduate degrees during dedicated laboratory research years continues to grow, the goals of supporting such programs must be determined to provide justification for the significant financial cost associated with such programs. The tuition of the graduate degree programs reaches an amount equivalent to a starting resident salary (without benefits) in many regions. The financial ramifications of these costs must be carefully considered in future analyses. If the motivation for supporting graduate degree programs is to help individual residents bolster their productivity in terms of publications and presentations, then the aforementioned data indicate that formal support of degrees can help programs attain this goal. Residents at this institution who pursued additional degrees produced significantly more papers, both collaborative and first author, than their peers who did not pursue degrees. In many instances of collaboration, this may have been secondary to the fact that they were able to provide critical statistical support and clinical understanding to fellow residents as they worked on separate projects. Additionally, support of degrees or a structured laboratory environment or both could lead to more collaborative papers, but additional data need to be gathered to support this notion and whether the use of tools obtained during a degree program lead to more collaborative project involvement. In instances of first authorship, we also theorize that many of these projects and papers are produced comparatively quickly as the degree program curricula provided the motivation, structure, feedback, and expertise to help residents complete projects in a timely fashion (often requiring them as “homework” for courses). Of note, completion of a degree program often requires a first-author paper. In addition, by enrolling in degree programs, it seems that residents are able to readily collaborate with other residents, likely with some secondary gain to their instructors when publications include them as well. Looking at a more global outcome, if the goal of supporting additional degree programs is to produce more academic surgeons or to retain those interested in academics within the field of academic surgery, this study provides some support to the notion that degree programs influence
the pursuit of academic surgery after training as compared with dedicated research time alone. On consideration of a pure academic appointment, there was also a significant correlation with those who pursued a degree during dedicated research time vs those completing dedicated research time only (93% vs 58%, p ¼ 0.046). When academic-affiliated appointments were included within this, it was no longer significant (93% vs 75%, p ¼ 0.306). This suggests that graduates of this program who have had dedicated research time may still gravitate toward an environment involved in resident education whether or not they intend to pursue research. Therefore, spending additional money to support a degree-based program may be a fiscally sound approach for retaining graduates in academic general surgery. Simply providing dedicated research time may not be as effective. We recognize that a first job is not an equivalent for longterm career choices. To determine this, a long-term prospective analysis across multiple institutions would be required. Another limitation of this study is the sample size of the study, given the number of graduates in the past 10 years. It may be that if a larger study of similarly sized institutions nationally were to be performed, there would be an even stronger association between those pursuing degree programs in a structured laboratory environment and those accepting their first job in academic surgery. The varied educational environments in terms of different options for graduate degrees and tracks within each degree program may also provide a more holistic picture of the qualitative and quantitative benefits of graduate degree programs across different residency programs. Another potential confounder could be selection bias during recent years. The same program director has been at this institution during the study period, and he has indicated that his motivation and inclination toward ranking applicants has not changed drastically during that time period (J. Tarpley, personal communication). However, it is possible that other factors such as gender proportion, the morale of the program, and the support of degree programs institutionwide has influenced applicants toward a group of more academically minded individuals at the start of the training pipeline, leading to production of academicians upon completion of training. The Chair of the Section of Surgical Sciences did change in 2001, the first year of this study, and a Vice-Chair of Research was recruited to the faculty in 2002. This study does not examine the effect of these factors, although a questionnaire of past applicants at this institution or other institutions could elicit this trend along with the National Institute of Health research dollars available in the department over the 10-year period studied. In determining the cost-effectiveness of supporting degree programs, general surgery residency programs must also determine financial institutional benefits associated with such training programs. With a higher productivity for residents completing a degree (Table 3), there may be
Journal of Surgical Education Volume ]/Number ] ] 2014
5
increased access to research funding for the institution, as many mentors also have a higher productivity in conjunction with their residents, which further supports institutional research growth. A future study may be able to associate institutional research funding growth and increased resident research productivity. This growth in funding may outweigh the costs associated with providing opportunities for residents to complete graduate degrees, thereby meeting not only an individual resident’s goal for increased productivity but also an institutional goal for increased research productivity without significant increases in cost. Review of the historical context of organized degree programs at this institution indicates that the impetus for these programs was a desire for a more structured training environment, to provide better mentoring opportunities, and to provide “quality control” by pushing research mentors to focus on research training. The hope was that this would augment individual trainee productivity during costly research time, as experiences outside structured programs seemed to provide less accountability for training. Our data support this notion of an organized degree program to augment productivity. A structured training program during the research phase likely develops skills useful for application in current and future career scenarios. A future study could provide deeper insight regarding long-term productivity of the residents completing degree programs and its effect on their academic careers.
Margaret J. Tarpley performed data collection and was active in manuscript revision. Kyla P. Terhune performed data collection, data analysis, wrote significant portions of the manuscript, and was active in manuscript revision. The authors would like to acknowledge John L. Tarpley, MD, program director for general surgery at Vanderbilt University, for providing information through personal communication regarding graduates and motivation toward resident selection over the time period studied.
REFERENCES 1. Robertson CM, Klingensmith ME, Coopersmith CM.
Prevalence and cost of full-time research fellowships during general surgery residency: a national survey. Ann Surg. 2009;249(1):155-161. 2. Manchikanti L, Helm S, Hirsch JA. The evolution of
patient-centered outcomes research institute. J Neurointervent Surg. 2012;4(3):157-162. 3. Available at: 〈https://medschool.vanderbilt.edu/mph/〉.
March 6, 2011. 4. Available at: 〈http://www.mc.vanderbilt.edu/msci/〉.
CONCLUSION The support of graduate degrees during dedicated research time does appear to significantly bolster the number of publications, total and first author, the caliber of publications produced by residents during their training, and the propensity of graduates to enter academic surgery. Depending on a program’s primary goal (individual, institutional, or global) in support of degree programs, the financial support required for degree programs may or may not be cost-effective. These data represent a pilot study of an institution’s experience; further investigation into the experience of multiple national programs should be used to validate these findings.
ACKNOWLEDGMENTS J. Joshua Smith performed data collection, data analysis, wrote significant portions of the manuscript, and was active in manuscript revision. Ravi Patel performed data analysis, wrote significant portions of the manuscript, and was active in manuscript revision. Xi Chen performed data collection, data analysis, and wrote portions of the Methods section.
6
March 6, 2011. 5. Journal Citation Reportss Science Edition. Thomson
Reuters. 2010. 6. Wager E. Recognition, reward and responsibility: why
the authorship of scientific papers matters. Maturitas. 2009;62(2):109-112. 7. Ko CY, Whang EE, Longmire WP Jr, McFadden
DW. Improving the surgeon’s participation in research: is it a problem of training or priority? J Surg Res. 2000;91(1):5-8. 8. Stewart RD, Doyle J, Lollis SS, Stone MD. Surgical
resident research in New England. Arch Surg. 2000;135(4):439-444. 9. Robertson CM, Klingensmith ME, Coopersmith CM.
Long-term outcomes of performing a postdoctoral research fellowship during general surgery residency. Ann Surg. 2007;245(4):516-523. 10. Thakur A, Thakur V, Fonkalsrud EW, Singh S,
Buchmiller TL. The outcome of research training during surgical residency. J Surg Res. 2000;90 (1):10-12.
Journal of Surgical Education Volume ]/Number ] ] 2014
Does Intentional Support of Degree Programs in General Surgery Residency Affect Research Productivity or Pursuit of Academic Surgery? Jesse Joshua Smith, MD, PhD*,1 Ravi K. Patel, MD*,1 Xi Chen, PhD† Margaret J. Tarpley, MLS* and Kyla P. Terhune, MD* Department of Surgery, Vanderbilt University Medical Center, Nashville, Tennessee; and †Division of Cancer Biostatistics, Department of Biostatistics, Vanderbilt University Medical Center, Nashville, Tennessee *
OBJECTIVE: Many residents supplement general surgery training with years of dedicated research, and an increasing number at our institution pursue additional degrees. We sought to determine whether it was worth the financial cost for residency programs to support degrees. DESIGN: We reviewed graduating chief residents (n ¼ 69)
in general surgery at Vanderbilt University from 2001 to 2010 and collected the data including research time and additional degrees obtained. We then compared this information with the following parameters: (1) total papers, (2) first-author papers, (3) Journal Citation Reports impact factors of journals in which papers were published, and (4) first job after residency or fellowship training. SETTING: The general surgery resident training program at Vanderbilt University is an academic program, approved to finish training 7 chief residents yearly during the time period studied.
significantly correlated with a first job in academia if compared to those with dedicated research time only (p ¼ 0.046). CONCLUSIONS: Our data support the utility of degree
completion when economically feasible and use of dedicated research time as an effective way to significantly increase research productivity and retain graduates in academic surgery. Aggregating data from other academic surgery programs would allow us to further determine association of funding of additional degrees as a means to encourage academic producC 2014 Association of tivity and retention. ( J Surg ]:]]]-]]]. J Program Directors in Surgery. Published by Elsevier Inc. All rights reserved.) KEYWORDS: resident education, graduate degree, research
fellowship, general surgery/education COMPETENCY: Practice-Based Learning and Improvement
PARTICIPANTS: Chief residents in general surgery at
Vanderbilt who finished their training 2001 through 2010.
RESULTS: We found that completion of a degree during
residency was significantly associated with more total and first-author publications as compared with those by residents with only dedicated research time (p ¼ 0.001 and p ¼ 0.017). Residents completing a degree also produced publications of a higher caliber and level of authorship as determined by an adjusted resident impact factor score as compared with those by residents with laboratory research time only (p ¼ 0.005). Degree completion also was
Correspondence: Inquiries to Kyla P. Terhune, MD, Department of Surgery, Vanderbilt University, D-4309 MCN, 1161 21st Ave S., Nashville, TN 37232-2730; fax: (615) 322-0689.; e-mail: [email protected], [email protected] 1
These authors contributed equally to this work.
INTRODUCTION As many as 36% of residents at academic institutions across the nation supplement their clinical training for a mean of 1.7 years with research fellowships.1 During these years, residents are expected to be productive, present nationally, publish papers, write chapters, and prepare for and augment the beginnings of an academic career by increasing their chances of entering competitive fellowships and academic positions. However, these years require funding of salary, benefits, and insurance through grants or direct institutional support. From a survey sent in 2006 to all National Resident Matching Program participating program directors, Robertson et al.1 estimated that it costs $41.5 million to fund the 634 trainees involved in research fellowships
Journal of Surgical Education & 2014 Association of Program Directors in Surgery. Published by 1931-7204/$30.00 Elsevier Inc. All rights reserved. http://dx.doi.org/10.1016/j.jsurg.2014.01.018
1
Growth of Interest in Degree Programs 6
Residents
5 4 Lab Only
3
All Degrees
2
Linear (Lab Only)
1
Linear (All Degrees)
0 2001 2002 2003 2004 2005 2006 2007 2008 2009 2010 Years
FIGURE. Increasing trend amongst general surgery residents at this institution to pursue a graduate degree during dedicated research time. Positive trend shows the growth of residents pursuing a graduate degree during their dedicated laboratory research time as compared with completing laboratory research without concurrent degree pursuits.
each year, primarily paid through departmental funds and institutional training grants. At our tertiary care institution, there appears to be a trend toward the pursuit of additional degrees during the resident research years (Fig.). This trend may be influenced by a national interest in and focus on outcomes research.2 However, when determining whether to support such degree programs, a surgical program must take into account the weighty expense and determine the ultimate outcome desired providing this opportunity. Potential outcomes could be individual, institutional, or global. Individually, by augmenting a resident’s publication credentials, chances of obtaining a fellowship or faculty position would be increased. Institutionally, increased productivity and national exposure would reflect well on the respective program and academic medical center. Globally, retention and productivity of the academic surgery community as a whole could be augmented. At this institution, there are 2 popular pathways for additional degrees that are very feasible during dedicated research time. The Master of Public Health (MPH) is an interdisciplinary program founded in 1996 to train public health scientists and professionals dedicated to improving the public health and preventing disease and disability. The Master of Science in Clinical Investigation (MSCI) Program was approved in 1999 and trains investigators in the techniques and processes used in patient-oriented research. Tuition for Vanderbilt University’s MPH and MSCI in 2012 to 2013 was $46,425 and $47,000, respectively.3,4 Given this expense, we sought to determine from an administrative perspective whether the end result justified the expenses, using the individual, institutional, and global goals outlined earlier as a guide. We examined graduating chief residents at Vanderbilt for the years 2001 through 2010 comparing those completing degree programs vs those who did not. The outcomes measured included whether residents completed degree programs, whether they completed dedicated research time, their publication record, and their first job after completion of residency and fellowship training (e.g., community, military, academic affiliated, or academic). 2
METHODS A request for exemption was approved by the Vanderbilt University’s institutional review board for this study. We reviewed all graduating chief residents (n ¼ 69) from general surgery at Vanderbilt University Medical Center from 2001 through 2010, determined whether they had spent any years in dedicated research time, and recorded whether they had pursued an additional degree, including MSCI, MPH, Doctor of Philosophy, and Master of Business Administration (Table 1). One resident who completed only 2 years of residency within the United States was excluded, as he was not afforded the opportunity to complete dedicated research time. Outcome measures were dependent on the following: (1) total papers published; (2) paper contribution level (designated as first author, second author, last author, or collaborative papers); (3) Journal Citation Reports (JCR) impact factors of journals5; and (4) classification of first job after residency or fellowship training (community, academic, military, or academicaffiliated) (Table 2). PubMed was used as a search engine for a time frame of 7 years before the completion of residency through 1 year
TABLE 1. Degree program completion. Laboratory research experience and degree completion for surgical residents under study at Vanderbilt University are depicted (n ¼ 68). No significant difference (p ¼ 0.294) for average laboratory research years existed between residents completing a degree (2.1 y) and those who completed only structured laboratory research time (2.0 y) Research Years
Residents
Mean Laboratory Years
MPH þ laboratory MBA þ laboratory MS þ laboratory MSCI þ laboratory Laboratory only No laboratory Total residents
10 1 1 5 27 24 68
2.1 2.0 3.0 2.0 2.0 0.0 1.3
MBA, Master of Business Administration; MS, Master of Science. Journal of Surgical Education Volume ]/Number ] ] 2014
TABLE 2. First job. Description of first jobs for general surgery residents after completion of residency or fellowship training or both First Job Academics Academic affiliated Community Military Total residents
Degree Program 14 (82%)
Laboratory No Only Laboratory 14 (52%)
7 (29%)
0 (0%)
4 (15%)
1 (4%)
1 (6%) 2 (12%) 17
6 (22%) 3 (11%) 27
15 (63%) 1 (4%) 24
after the completion of chief residency to account for the length of residency and publishing delays. Results were filtered by resident names and institutional affiliations for both during residency and the year after residency to include all publications. The JCR of 2010 and level of contribution were the components used to establish the productivity levels of all papers published by residents. By convention, we assumed the sequence of authors in publications was in reverse order of contribution, excepting the last author position, which was assumed to be the principal investigator.6 Two scores then were developed to summarize and quantify a resident’s productivity. The first was the unweighted resident impact factor (RIF), defined as the sum of JCR impact factors for each of the journals within which the resident had a publication, yielding a single score for the productivity of the resident. The second score was the adjusted RIF (aRIF), which was weighted to account for levels of authorship and thus contribution. Multiplying factors were assigned for levels of authorship as follows: first or last author ¼ 3, second author ¼ 2, and collaborating author ¼ 1. These were then multiplied by the impact factor of the journal of the publication, the sum of these products yielding the aRIF. This aRIF thus considers not only number of publications and contribution level, but also impact and potential significance of the work. A total of 31 papers analyzed were published within journals without an assigned JCR impact factor and were excluded. These excluded papers were distributed as follows: 26 from residents with dedicated research time and 5 from residents without dedicated research time. Of those with dedicated research time, 16 were from residents who completed a degree and 10 from residents who did not complete a degree. Information regarding first job was obtained from the surgical education office and verified by the general surgery program director at this institution (present since 1993). Academic institutions were defined as those with residency or fellowship training programs, while academic-affiliated institutions were defined as those where residents rotate but are without an independently accredited general surgery training program. Wilcoxon rank sum tests with continuity correction were used to determine differences in total publications, firstJournal of Surgical Education Volume ]/Number ] ] 2014
author publications, RIFs, and aRIFs between residents who completed a degree during their residency and those who did not. The analysis compared all residents within both the groups. The analysis also compared residents who completed degrees against those who pursued only dedicated research time. Additionally, both of these groups were compared with those who did not spend any time in dedicated research years but instead completed the residency program without any gaps in training. Individual degree groups (MPH and MSCI) were also examined against dedicated research time only and absence of dedicated research time. Differences between mean years spent in dedicated research time were determined by the MannWhitney U test. Pearson chi-square test with Yates continuity correction was used to determine if degree completion as compared with dedicated research time only correlated with pursuing a first job in academic general surgery. Additional baseline comparisons of both groups were made with those who did not have any dedicated research time during residency. All residents pursuing their first job in the military were excluded from this analysis owing to length of military commitment and delay of fellowship training.
RESULTS We hypothesized that the pursuit of a degree during residency increased productivity and would also correlate with transition into a posttraining academic position. For residents with dedicated research time, we compared residents who completed a degree and those who did not. Completion of a degree during residency was significantly associated with more publications and first-author papers when compared with those who did not complete degrees (Table 3; n ¼ 10.3 vs 5.30, p o 0.001 and n ¼ 4.06 vs 2.30, p ¼ 0.017). Residents completing a degree were also more productive in terms of the caliber of papers published and levels of authorship when comparing the calculated RIF and the aRIF scores (32.3 vs 17.8 score, p ¼ 0.001 and 65.4 vs 36.1 score, p ¼ 0.005). The mean number of years in the laboratory for both groups, degree completed (2.1 y) and laboratory only (2.0 y), were not significantly different (Table 1; 2.1 vs 2.0 y, p ¼ 0.294). In subgroup analysis, residents completing an MPH were significantly more productive than laboratory research only residents in all previously mentioned accounts (p r 0.004). Those completing an MSCI had a significantly higher RIF compared with that of residents in dedicated research only (p ¼ 0.046), but this significance was not seen in total publications, first-author publications, or in the aRIF (Table 3). They still were significantly more productive than residents who had no dedicated research time on all accounts (p r 0.001; Table 3). Although residents completing dedicated research time only were not as productive as their 3
TABLE 3. Productivity of residents completing degree programs. Residents completing a degree program were significantly more productive compared with residents with only dedicated laboratory research time in terms of total publications (p ¼ 0.001) and firstauthor publications (p ¼ 0.0017) with an average of 10.3 publications and 4.06 first-author publications. Those completing a degree were also more productive in terms of producing higher-caliber publications with overall higher levels of authorship as shown by the resident impact factor (p ¼ 0.001) and adjusted resident impact factor (p ¼ 0.005), respectively Total Publications Comparisons (Mean) p Values
Laboratory only (5.30) No laboratory time (1.29)
All Degrees (10.3)
MPH (12.8)
MSCI (6.20)
No Laboratory Time (1.29)
0.001 o0.001
o0.001 o0.001
0.132 o0.001
0.002 —
Total First-Author Publications Comparisons (Mean) p Values
Laboratory only (2.30) No laboratory time (0.458)
All Degrees (4.06)
MPH (5.70)
MSCI (2.20)
No Laboratory Time (0.458)
0.017 o0.001
0.001 o0.001
0.311 0.001
0.003 —
Comparisons of Resident Impact Factors (Mean) p Values
Laboratory only (17.8) No laboratory time (2.69)
All Degrees (32.3)
MPH (35.7)
MSCI (26.2)
No Laboratory Time (2.69)
0.001 o0.001
0.004 o0.001
0.046 o0.001
o0.001 —
Comparisons of Adjusted Resident Impact Factors (Mean) p Values
Laboratory only (36.1) No laboratory time (5.17)
All Degrees (65.4)
MPH (79.5)
MSCI (46.8)
No Laboratory Time (5.17)
0.005 o0.001
0.003 o0.001
0.168 o0.001
o0.001 —
counterparts who completed a degree, they were more productive than residents without dedicated research time in all of the aforementioned factors (p r 0.003; Table 3). Residents completing a degree were more productive and more likely to pursue a first job in academics when compared with residents who completed dedicated research time without a degree (Table 4; 93% vs 58%, p ¼ 0.046). When an academic job definition was expanded to include both academic and academic-affiliated positions, the association was no longer significant (p ¼ 0.306). Residents with dedicated research time only were no more likely to pursue academics compared with those without any dedicated research time (Table 4; 58% vs 30%, p ¼ 0.103).
However, when expanded to consider both academic and academic-affiliated positions, there was a significant difference (Table 4; 75% vs 35%, p ¼ 0.013).
DISCUSSION Conventional wisdom and tradition have promoted dedicated research time during residency as an important part of general surgery training for those intending to pursue an academic career. This trend has been perpetuated by both experienced surgeons looking retrospectively as well as those currently in training.7-9 General surgery is one of the few
TABLE 4. Proportion of academic appointments for residents completing a degree. Residents completing a degree program were significantly more likely to pursue a first job in academics as compared with their counterparts who completed only dedicated laboratory research First Job Degree program Laboratory only p Value Degree program No laboratory time p Value Laboratory only No laboratory time p Value 4
Academics
Academics þ Academic Affiliated
93.3% (14/15) 58.3% (14/24) 0.046 93.3% (14/15) 30.4% (7/23) o0.001 58.3% (14/24) 30.4% (7/23) 0.103
93.3% (14/15) 75.0% (18/24) 0.306 93.3% (14/15) 34.8% (8/23) 0.001 75.0% (18/24) 34.8% (8/23) 0.013 Journal of Surgical Education Volume ]/Number ] ] 2014
specialties in which formal research training (e.g., postdoctoral research fellowship) occurs during the residency training period for a significant proportion of physicians.9 It has been shown that residents are more likely to pursue fellowship training in the field related to their dedicated research time.10 In a review of the monies spent on residents at our institution in 1 year, almost $450,000 was spent to support residents not covered on T32 grants (6 of 12 residents in research) and $70,000 was used to support those on T32 support to bridge the gap between stipend funding and the need for mandatory fringe-benefit coverage. The total support required for this research enterprise is approximately $520,000 per year and does not take into account the monies spent on travel expenses and other miscellaneous items that support “resident in research” training ventures. As the trend to pursue graduate degrees during dedicated laboratory research years continues to grow, the goals of supporting such programs must be determined to provide justification for the significant financial cost associated with such programs. The tuition of the graduate degree programs reaches an amount equivalent to a starting resident salary (without benefits) in many regions. The financial ramifications of these costs must be carefully considered in future analyses. If the motivation for supporting graduate degree programs is to help individual residents bolster their productivity in terms of publications and presentations, then the aforementioned data indicate that formal support of degrees can help programs attain this goal. Residents at this institution who pursued additional degrees produced significantly more papers, both collaborative and first author, than their peers who did not pursue degrees. In many instances of collaboration, this may have been secondary to the fact that they were able to provide critical statistical support and clinical understanding to fellow residents as they worked on separate projects. Additionally, support of degrees or a structured laboratory environment or both could lead to more collaborative papers, but additional data need to be gathered to support this notion and whether the use of tools obtained during a degree program lead to more collaborative project involvement. In instances of first authorship, we also theorize that many of these projects and papers are produced comparatively quickly as the degree program curricula provided the motivation, structure, feedback, and expertise to help residents complete projects in a timely fashion (often requiring them as “homework” for courses). Of note, completion of a degree program often requires a first-author paper. In addition, by enrolling in degree programs, it seems that residents are able to readily collaborate with other residents, likely with some secondary gain to their instructors when publications include them as well. Looking at a more global outcome, if the goal of supporting additional degree programs is to produce more academic surgeons or to retain those interested in academics within the field of academic surgery, this study provides some support to the notion that degree programs influence
the pursuit of academic surgery after training as compared with dedicated research time alone. On consideration of a pure academic appointment, there was also a significant correlation with those who pursued a degree during dedicated research time vs those completing dedicated research time only (93% vs 58%, p ¼ 0.046). When academic-affiliated appointments were included within this, it was no longer significant (93% vs 75%, p ¼ 0.306). This suggests that graduates of this program who have had dedicated research time may still gravitate toward an environment involved in resident education whether or not they intend to pursue research. Therefore, spending additional money to support a degree-based program may be a fiscally sound approach for retaining graduates in academic general surgery. Simply providing dedicated research time may not be as effective. We recognize that a first job is not an equivalent for longterm career choices. To determine this, a long-term prospective analysis across multiple institutions would be required. Another limitation of this study is the sample size of the study, given the number of graduates in the past 10 years. It may be that if a larger study of similarly sized institutions nationally were to be performed, there would be an even stronger association between those pursuing degree programs in a structured laboratory environment and those accepting their first job in academic surgery. The varied educational environments in terms of different options for graduate degrees and tracks within each degree program may also provide a more holistic picture of the qualitative and quantitative benefits of graduate degree programs across different residency programs. Another potential confounder could be selection bias during recent years. The same program director has been at this institution during the study period, and he has indicated that his motivation and inclination toward ranking applicants has not changed drastically during that time period (J. Tarpley, personal communication). However, it is possible that other factors such as gender proportion, the morale of the program, and the support of degree programs institutionwide has influenced applicants toward a group of more academically minded individuals at the start of the training pipeline, leading to production of academicians upon completion of training. The Chair of the Section of Surgical Sciences did change in 2001, the first year of this study, and a Vice-Chair of Research was recruited to the faculty in 2002. This study does not examine the effect of these factors, although a questionnaire of past applicants at this institution or other institutions could elicit this trend along with the National Institute of Health research dollars available in the department over the 10-year period studied. In determining the cost-effectiveness of supporting degree programs, general surgery residency programs must also determine financial institutional benefits associated with such training programs. With a higher productivity for residents completing a degree (Table 3), there may be
Journal of Surgical Education Volume ]/Number ] ] 2014
5
increased access to research funding for the institution, as many mentors also have a higher productivity in conjunction with their residents, which further supports institutional research growth. A future study may be able to associate institutional research funding growth and increased resident research productivity. This growth in funding may outweigh the costs associated with providing opportunities for residents to complete graduate degrees, thereby meeting not only an individual resident’s goal for increased productivity but also an institutional goal for increased research productivity without significant increases in cost. Review of the historical context of organized degree programs at this institution indicates that the impetus for these programs was a desire for a more structured training environment, to provide better mentoring opportunities, and to provide “quality control” by pushing research mentors to focus on research training. The hope was that this would augment individual trainee productivity during costly research time, as experiences outside structured programs seemed to provide less accountability for training. Our data support this notion of an organized degree program to augment productivity. A structured training program during the research phase likely develops skills useful for application in current and future career scenarios. A future study could provide deeper insight regarding long-term productivity of the residents completing degree programs and its effect on their academic careers.
Margaret J. Tarpley performed data collection and was active in manuscript revision. Kyla P. Terhune performed data collection, data analysis, wrote significant portions of the manuscript, and was active in manuscript revision. The authors would like to acknowledge John L. Tarpley, MD, program director for general surgery at Vanderbilt University, for providing information through personal communication regarding graduates and motivation toward resident selection over the time period studied.
REFERENCES 1. Robertson CM, Klingensmith ME, Coopersmith CM.
Prevalence and cost of full-time research fellowships during general surgery residency: a national survey. Ann Surg. 2009;249(1):155-161. 2. Manchikanti L, Helm S, Hirsch JA. The evolution of
patient-centered outcomes research institute. J Neurointervent Surg. 2012;4(3):157-162. 3. Available at: 〈https://medschool.vanderbilt.edu/mph/〉.
March 6, 2011. 4. Available at: 〈http://www.mc.vanderbilt.edu/msci/〉.
CONCLUSION The support of graduate degrees during dedicated research time does appear to significantly bolster the number of publications, total and first author, the caliber of publications produced by residents during their training, and the propensity of graduates to enter academic surgery. Depending on a program’s primary goal (individual, institutional, or global) in support of degree programs, the financial support required for degree programs may or may not be cost-effective. These data represent a pilot study of an institution’s experience; further investigation into the experience of multiple national programs should be used to validate these findings.
ACKNOWLEDGMENTS J. Joshua Smith performed data collection, data analysis, wrote significant portions of the manuscript, and was active in manuscript revision. Ravi Patel performed data analysis, wrote significant portions of the manuscript, and was active in manuscript revision. Xi Chen performed data collection, data analysis, and wrote portions of the Methods section.
6
March 6, 2011. 5. Journal Citation Reportss Science Edition. Thomson
Reuters. 2010. 6. Wager E. Recognition, reward and responsibility: why
the authorship of scientific papers matters. Maturitas. 2009;62(2):109-112. 7. Ko CY, Whang EE, Longmire WP Jr, McFadden
DW. Improving the surgeon’s participation in research: is it a problem of training or priority? J Surg Res. 2000;91(1):5-8. 8. Stewart RD, Doyle J, Lollis SS, Stone MD. Surgical
resident research in New England. Arch Surg. 2000;135(4):439-444. 9. Robertson CM, Klingensmith ME, Coopersmith CM.
Long-term outcomes of performing a postdoctoral research fellowship during general surgery residency. Ann Surg. 2007;245(4):516-523. 10. Thakur A, Thakur V, Fonkalsrud EW, Singh S,
Buchmiller TL. The outcome of research training during surgical residency. J Surg Res. 2000;90 (1):10-12.
Journal of Surgical Education Volume ]/Number ] ] 2014
