Special Report Designing and Implementing INTREPID, an Intensive Program in Translational Research Methodologies for New Investigators Claudia S. Plottel, M.D., M.S.1,2, Yindalon Aphinyanaphongs, M.D., Ph.D.1,2, Yongzhao Shao, Ph.D.1,3, Keith J. Micoli, Ph.D.1,4, Yixin Fang, Ph.D.3, Judith D. Goldberg, Ph.D.1,3, Claudia R. Galeano, B.A.1, Jessica H. Stangel, M.P.A.1, Deborah Chavis-Keeling, M.S.1, Judith S. Hochman, M.D.1,2, Bruce N. Cronstein, M.D.1,2, and Michael H. Pillinger, M.D.1,2 Abstract Senior housestaff and junior faculty are often expected to perform clinical research, yet may not always have the requisite knowledge and skills to do so successfully. Formal degree programs provide such knowledge, but require a significant commitment of time and money. Short-term training programs (days to weeks) provide alternative ways to accrue essential information and acquire fundamental methodological skills. Unfortunately, published information about short-term programs is sparse. To encourage discussion and exchange of ideas regarding such programs, we here share our experience developing and implementing INtensive Training in Research Statistics, Ethics, and Protocol Informatics and Design (INTREPID), a 24-day immersion training program in clinical research methodologies. Designing, planning, and offering INTREPID was feasible, and required significant faculty commitment, support personnel and infrastructure, as well as committed trainees. Clin Trans Sci 2014; Volume 7: 493–499
Keywords: immersion training, INTREPID course, research certificate program, research intensive, short-term training program, summer intensive, study design, teaching clinical research methods Introduction
Inadequate mastery of clinical/translational research skills, time constraints, and competing tasks are major impediments to junior faculty and senior housestaff interested in performing patientcentered research.1–11 Of 73 internal medicine residents presenting their work at an American College of Physicians-American Society of Internal Medicine annual session, nearly half cited lack of research skills as a “major obstacle” to research.12 Earlystage academic physicians often plan clinical research projects without formal training in relevant methodologies.3,13–15 Reliance on experiential learning may result in delayed or less than optimal study design, recruitment, and implementation, as well as reduced likelihood of project funding. 16 In response to these challenges, the Clinical and Translational Science Awards (CTSA) initiative of the National Institutes of Health’s National Center for Advancing Translational Science (NCATS) has promoted the development of Masters-level programs in translational research methods.17–20 Such programs commonly require a 2- to 3-year, full- or parttime engagement, which not all junior trainees may be willing or able to commit to. In response to these concerns, many institutions have developed short-term programs in clinical research methodologies, often with a focus on predoctoral students or residents.21–25 A review of the 62 CTSA recipient Websites in February of 2014 reveals that at least 20 of the institutions offer some form of short-term educational programs. These range from brief offerings (one to several days) that are convenient but limited in scope, to more extensive programs (6–8 weeks) that may require more time away from clinical duties than many trainees can afford. Still other programs may be part-time but longitudinal over one or more semesters.26–28 An appreciation of best practices for short-term program design would help educators determine what programs would be best for their own institutions and trainee populations. Few reports however, address the design, implementation, successes, and limitations of the varying models of short-term training programs.29,30
The Clinical and Translational Science Institute (CTSI) of New York University and the New York City Health and Hospitals Corporation (NYU-HHC CTSI) recently designed and implemented INtensive Training in Research Statistics, Ethics, and Protocol Informatics and Design (INTREPID), a focused, intensive, cohesive immersion program in clinical research methodologies, providing in-depth knowledge of clinical research skills in less than 1 month. We hypothesized that such a structured didactic educational program would address the need for rapid learning and early acquisition of core translational research skills, and improve the probability of successfully conducting research. To promote community discussion of the best approaches for short-term program design, we here share our experience developing and implementing INTREPID, and the initial feedback we received from trainees and faculty. Methods
Program structure and overview INTREPID consists of three core courses and an optional course in Ethical Conduct of Research (Figure 1). The core courses, Introduction to Clinical Research Methods, Introductory Biostatistics, and Introduction to Biomedical Informatics and Computing, were derived and adapted from our preexisting Masters of Science in Clinical Investigation (MSCI) program. As the course content was already approved, these topics allowed us to meet requirements of both New York University and the New York State Department of Education, earning graduates a New York State–approved Certificate in Translational Research. Each course met for 36 face-to-face instruction hours, complemented by a carefully designed homework and self-learning component, with extensive preprogram, weeknight, and weekend assignments. The noncredit Ethical Conduct of Research course allowed participants to meet the NIH mandate for ethics training; participants were permitted to opt out if they had already met this requirement.
The NYU-HHC Clinical and Translational Science Institute, New York University School of Medicine, New York, New York, USA; 2Department of Medicine, New York University School of Medicine, New York, New York, USA; 3Department of Population Health, New York University School of Medicine, New York, New York, USA; 4Office of Science and Research, New York University Langone Medical Center, New York, New York, USA. 1
Correspondence : Claudia S. Plottel ([email protected]) DOI: 10.1111/cts.12198
WWW.CTSJOURNAL.COM
VOLUME 7 • ISSUE 6
493
Plottel et al. Designing/Implementing Clinical Research Training n
his/her area of expertise. To maintain the highest level of instruction, the Introductory Biostatistics and Introduction to Biomedical Informatics and Computing course directors invited a limited number of supplementary lecturers to teach specific topics in their specialty. The Program Director was responsible for overall organization, coordination, and program launch and in addition, directed the Introduction to Clinical Research Methods. In the months prior to the program start date, all course directors met weekly, facilitating a high degree of cooperation and collegiality. The excellent interpersonal rapport that ensued proved essential to the program, and enhanced the logistics of developing and integrating the required specialized and complementary course syllabi and curricula. Support staff A dedicated Program Coordinator supported INTREPID’s development and operations. The Program Coordinator, working closely with the Program Director, developed INTREPID-related announcements, and an online application and postprogram surveys using Research Electronic Data Capture (REDCap), a secure, Web-based application designed to support data capture for research studies hosted at NYU31 (http://www. project-redcap.org). The coordinator’s additional responsibilities included setting up INTREPID’s Web page, coordinating and attending faculty meetings, taking minutes, responding to phone and online inquiries, purchasing textbooks, arranging catered lunches, developing curricular information for the NYU School of Medicine’s learning portal (Advanced Learning Exchange [ALEX]), and drafting, distributing, and compiling online surveys at the program’s completion.
Figure 1. INTREPID program overview: courses, format, deliverables, faculty commitment. INTREPID took place over 24 consecutive calendar days and consisted of three 3-credit courses and an optional course in the ethical conduct of research to fulfill the NIH-mandated ethics training (courses listed in white boxes). Both the biostatistics course and the biomedical informatics course included a lab session, in addition to didactic lectures and seminar-style teaching (course format represented in black boxes). One hundred percent attendance was required, and specific deliverables varied according to each course (course deliverables specified in dark grey boxes). The overall faculty-to-student ratio exceeded 1 in the first year the program was offered (faculty cohorts indicated in lighter grey boxes).
Participants took all three core courses and the Ethical Conduct of Research course concurrently over 24 consecutive days (18 classroom instruction days), with 7.0–8.5 hours of classroom time daily. The program was intentionally scheduled for July to coincide with the hiring, onboarding, and/or advancement of housestaff and junior faculty. Successful completion of all courses required 100% attendance, which along with the submission of specific deliverables earned the student a total of 9 credit hours. Graduates are invited to apply their credits toward future enrollment in the NYU-HHC CTSI MSCI program, should they choose to seek additional training. Program faculty Faculty were selected from NYU Langone Medical Center and NYU School of Medicine, as well as from the Sackler Institute of Graduate Biomedical Sciences, a division of NYU’s Graduate School of the Arts and Sciences (Table 1). Each course director was responsible for developing course content and teaching in
Curriculum design The central pedagogical approach of INTREPID was to integrate topics across the courses (Figure 2). Weekly meetings (described above) leading up to the start of the program provided opportunity to draft syllabi, determine a weekly program focus, integrate coursework, and design interlocking exercises and approaches. In contrast to our MSCI program, whose longitudinal curriculum
Programmatic Role
Academic rank
Additional faculty
INTREPID Program Director
M.D., M.S. Clinical Associate Professor Medicine
No
Course Director: Intro to Clinical Research Methods
Same individual as above
No
Course Director: Introductory Biostatistics
Ph.D. Professor / Biostatistics
Yes: 2 additional tenured faculty
Course Director: Intro to Biomedical Informatics and Computing
M.D., Ph.D. Assistant Professor Biomedical Informatics
Yes: visiting lecturers
Course Director: Ethical Conduct of Research
Ph.D. Adjunct Assistant Professor Sackler Institute
Yes: 1 graduate assistant
INTREPID Program Coordinator
B.A. NYU-HHC CTSI
N/A
Table 1. INTREPID faculty characteristics for the inaugural year of the INTREPID program in 2013.
494
VOLUME 7 • ISSUE 6
WWW.CTSJOURNAL.COM
Plottel et al. Designing/Implementing Clinical Research Training n
Figure 2. Syllabus development for each 3-credit required course. Each Course Director created a syllabus de novo, taking advantage of redundancies and overlap between subject matter across the courses. The amount of classroom instruction remained constant each week, but the relative amount of homework varied as the students worked to complete their projects.
emphasizes career development and mentoring, we designed INTREPID to focus on skills acquisition. Educational strategies employed included: seminar-style classroom interactions, learnercentered experiential assignments (research protocol and code writing, close reading and written critiques of published research papers), hands-on real-time computing, quiz and homework sets, and regular access to faculty both during the day and off-hours. Directors sought to exploit overlap and synergies while balancing self-study and homework. For example, when programming in “R” was introduced in Introduction to Biomedical Informatics and Computing, relevant lab exercises in Introductory Biostatistics were assigned concurrently to reinforce the use of “R.” Similarly, when a conceptual overview of hypothesis testing was presented in the Introduction to Clinical Research Methods in the morning during week 1, the topic was complemented and reinforced in the Introductory Biostatistics afternoon class sessions, and in the days following when the mathematical and statistical methodologies were taught in depth. Each syllabus detailed the overall course goals, the daily lecture title and learning objectives, and daily reading and assignments. All syllabi were posted on ALEX and made available to student participants prior to the start of classes. ALEX also housed other course-specific materials, such as primary reference sources and relevant papers from the literature. Recruitment In the spring of 2013, information regarding INTREPID was emailed to department chairs, faculty, and housestaff at NYU Schools of Medicine, Dentistry, and Nursing, as well to NYUHHC CTSI-member hospitals. We limited the request for applications to all of our CTSI-affiliated institutions for our “pilot” year. To advertise INTREPID, the Program Director announced the program at venues and events within our medical center (e.g., Department of Medicine and Department of Pediatrics research days, and the NYU-HHC CTSI’s Translational Researchin-Progress seminars). Interested healthcare professionals were invited to contact the Program Director or Coordinator with WWW.CTSJOURNAL.COM
questions. A hyperlink was provided to access and submit the online application, including a statement of interest and a letter of recommendation. Each applicant’s department chair had to agree to cover the cost of tuition, and guarantee fully protected time, free of any clinical, on-call, or research duties, for the program duration. Tuition was set at a discounted rate (75%) relative to NYU’s usual per-credit charge. Applications were due 2 months prior to the course start date and were reviewed by a committee consisting of the INTREPID faculty, the CTSI Director, and the Director of Translational Research Education and Careers (TREC), the CTSI’s educational unit. Participants were notified once accepted, allowing sufficient time for their departments to make arrangements for the requisite protected time.
Teaching materials Enduring materials were selected and/or created for each course. Textbooks were provided to participants at no charge one week before classes began, along with required reading assignments. Textbook readings were augmented by relevant e-book chapters and curated papers from the literature, available via the NYU Health Sciences Library subscription collection. Learning environment A conference room in NYU’s Translational Research Building was reserved for INTREPID daily from 8:00 a.m. to 6:00 p.m. throughout the program. The conference room included a large table, state-of-the-art multimedia audiovisual capability, and Internet access. Classes were taught seminar-style. A computer lab equipped with PC-based computers and a SmartBoard® was reserved for the hands-on lab sessions of Introductory Biostatistics and Introduction to Biomedical Informatics and Computing, as well as for individual practice with “R Studio” and Power Analysis and Sample Size (PASS) statistical software. Each student was assigned his or her own lab PC for the duration of INTREPID. Schedule A sample schedule from week 2 of INTREPID is shown in Table 2. Each course met 4 out of 5 days; Mondays had the greatest number of class hours, and Fridays had the least (decreasing intensity of face-to-face time across the week). Both Introductory Biostatistics and Introduction to Biomedical Informatics and Computing were allocated one lab session weekly for hands on programming in “R” and instruction and practice in PASS. Homework assignments increased as the week progressed (increasing intensity of selfstudy across the week). Lunch was provided in the classroom daily, and at least one (more often 3 or 4) faculty member(s) attended lunch, allowing for ongoing interactions and feedback. In addition, during four lunch sessions spread throughout the program, invited guests gave informal “Learning Lunch” talks. Lunch speakers included: (i) a junior faculty member speaking about her “real-life” interventional clinical research project, describing pitfalls and successes; (ii) the CTSI librarian discussing literature searches, describing databases, and other available tools; (iii) the VOLUME 7 • ISSUE 6
495
Plottel et al. Designing/Implementing Clinical Research Training n
Week 2
Monday
Tuesday
Wednesday
Thursday
Friday
09:00–11:00
Research Methods
Research Methods
Informatics and Computing
Research Methods
Research Methods
11:00–11:30
Research Methods
Research Methods
Biostatistics
Research Methods
Research Methods
11:30–13:00
Informatics and Computing
Informatics and Computing
Biostatistics
Informatics and Computing
Ethics
13:00–14:00
LUNCH
LEARNING LUNCH
LUNCH
LUNCH
LUNCH
14:00–16:00
Informatics and omputing LAB C
Biostatistics
Biostatistics LAB
Biostatistics
Biostatistics
16:00–17:00
Informatics and Computing
Ethics
Ethics
Ethics
17:00–17:30
Informatics and Computing
Table 2. Sample schedule from Week 2 of INTREPID. Classroom hours decreased, and the amount of homework increased as the week progressed. Lunch was provided each weekday for participants and faculty, allowing for ongoing informal and formal interactions. See text for details.
Students
Terminal degree
Academic rank
Specialization
1
Sex M
M.D.
Fellow, Year 1
Maternal-Fetal Medicine
2
F
M.D.
Fellow, Year 2
Gynecological Oncology
3
M
M.D., M.P.H.
Instructor
Geriatrics
4
F
M.D.
Assistant Professor
Emergency Medicine
5
F
Ph.D.
Director of Research
Medicine; HHC Affiliate
6
M
M.D.
Visiting Physician
Medicine; International
Table 3. INTREPID participant characteristics of the inaugural INTREPID class in July 2013.
CTSI librarian speaking on systematic and narrative reviews; and (iv) an Institutional Review Board (IRB) representative addressing the role of the IRB and providing practical information about the submission and review processes.
second week, the students reported increased understanding of “R”. By the third week, students were independently writing code for their required individual informatics projects, and interrogating large datasets to answer specific clinical questions.
Results
Assessments To obtain formal assessments of INTREPID, we asked each student to complete two surveys. All students completed both surveys, which revealed a high degree of satisfaction with the program in general and with specific course elements. Survey 1 was e-mailed to each student at the completion of the program (Figure 3). All students (n = 6) indicated that INTREPID had increased their knowledge of biomedical informatics, programming, biostatistics, and research methods, and that the Clinical Research Methods class had met their educational expectations. Four of six students rated the Clinical Research Methods course as “very relevant” and two rated it “relevant” to their professional development and career goals, while five of six would “definitely” recommend the course to a colleague interested in research. Five students reported that the Ethical Conduct of Research course “definitely increased” their knowledge of research ethics. Half of the students reported that the Introduction to Biomedical Informatics and Computing and the Introductory Biostatistics courses “definitely” met their educational expectations and the remaining students stated that those classes had “likely” met expectations. All six students responded that “R” taught in Introduction to Biomedical Informatics and Computing was helpful in the Introductory Biostatistics course. Four students
Inaugural class INTREPID’s first cohort included six participants: four from NYU School of Medicine, one from an HHC facility, and one visiting international trainee supported by a Fogarty grant via the NIH (Table 3). All participants had previously completed an MD or PhD degree; one additionally held an MPH. Participants came from a variety of clinical specialties. Academic rank ranged from fellow to Assistant Professor. No participant had prior knowledge of programming or programming software. All enrolled in the ethics class. All participants successfully completed the program, meeting all requirements. Challenges The biggest challenge that our students identified during the program was learning “R” in the first week of INTREPID. Students reported that the workload and the need to rapidly develop new programming skills were daunting. We addressed this challenge by adding focused, supplemental, one-on-one instruction between each student and the Introduction to Biomedical Informatics and Computing professor, in person, by email, and by phone. By the 496
VOLUME 7 • ISSUE 6
WWW.CTSJOURNAL.COM
Plottel et al. Designing/Implementing Clinical Research Training n
Figure 3. Survey A results. All participants (n = 6) completed Survey A, which assessed each course separately on the last day of INTREPID. There was a high degree of satisfaction with course content.
would “definitely” recommend Introduction to Biomedical Informatics and Computing to a colleague and the remainder reported that they might. Half the students would “definitely” recommend the Introductory Biostatistics and Ethical Conduct of Research courses to a colleague, and the remainder would possibly (“maybe yes”) recommend them. Survey 2 was emailed to the participants six weeks after program completion, and focused on the program as a whole (Figure 4). All students reiterated that they would recommend INTREPID to a colleague. Four rated the program as “effective,” and two “very effective” in improving their overall knowledge of clinical research methodologies. Survey respondents were encouraged to enter free-text comments elaborating on their thoughts about the program. Most of the remarks focused on the individualized attention, intensity of the immersion experience, and rapid knowledge acquisition that the participants experienced. In addition, the integration of information between the courses was mentioned as effective and useful. Discussion
INTREPID’s development was facilitated by the availability of physical space and technological resources, and by motivated faculty members who had previously taught similar course content in NYU-HHC CTSI’s MSCI Program. The MSCI courses provided WWW.CTSJOURNAL.COM
guidance as to what materials should be presented but were significantly “re-fitted” specifically for an intense immersion. The integration of material common to research methods, biostatistics, and biomedical informatics in an intensive, side-by-side structure presented both challenges and opportunities. We quickly learned to avoid redundancy while exploiting pedagogically useful overlap between the courses. To do so required coordination between the faculty during curricular development, necessitating that the faculty relationship be close and collegial. In lieu of longitudinal career mentoring, we substituted an enriched emphasis on skills building, making INTREPID particularly well suited to physicians planning to “jump-start” their research careers in fellowship or early in their positions as junior faculty. We anticipate a limited, but steady demand from this group for INTREPID training. Our decision to include a brief but cognitively intensive training in “R” programming was challenging, particularly in the first week. In this, and several other aspects of the program, the constant availability of the faculty proved essential to ultimate student success. Although designed with postdoctoral learners in mind, we did open INTREPID applications to medical students and residents. NYU medical students are offered a 12-week block of research time in their fourth year, yet none chose to use a portion of that time to take INTREPID, despite support for the program from their Associate Dean. Instead, our medical students interested VOLUME 7 • ISSUE 6
497
Plottel et al. Designing/Implementing Clinical Research Training n
hours might find it salutary to consider a schedule between 14 and 21 days, rather than our full 24. Our decision to charge discounted tuition relative to NYU’s per-credit fee was controversial among both the faculty and consumers. Some considered that as a credit-granting program, we should charge the same full per-credit tuition amount as the university. Others affirmed that as a service to our research community we ought to offer INTREPID gratis. We decided that the program should at minimum cover its own costs, and that requiring a per-credit discounted tuition “copayment” from a sponsoring department would insure that highly motivated students would participate. Although several departments expressed reservations about the tuition, in only one case—a clinical fellow who would be leaving at the end of the year—did a department decline support, explicitly stating to us that it was not in the departmental interest to invest in the professional development of a trainee who would not be remaining at our institution to apply the skills obtained. Figure 4. Survey B results. All participants (n = 6) completed a second survey 6 weeks after the course completion, when they had returned to their professional responsibilities and duties.
in research chose to enroll in our five-year, combined MD-MS in Clinical Investigation program, which is longer, and more comprehensive than INTREPID, offers more mentoring, and results in publications, all of which are attractive to predoctoral students planning their residencies. As for residents, we believe that the requirement that INTREPID participants be free of all clinical duties for 24 consecutive days in July makes participation unlikely as even a resident on elective is required to take call as part of clinical duties. For both medical students and residents who need a more basic grounding in clinical research methodologies, a less intense, more flexible, possibly asynchronous program might be desirable. The fact that we received applications only from physicians postresidency and beyond reinforces our belief that senior housestaff (fellowship level) and junior faculty have a largely unrecognized and unmet need for didactic instruction in translational research. Our decision to offer course credit and a university-sponsored certificate shaped INTREPID’s structure and content. Our 24-day schedule was the absolute minimum in which all three courses could be delivered with an appropriate number of credit hours. Trainees, all of whom were postdoctoral, commented on both the duration and intensity of the program; some found it particularly intense, especially during the first week. Others commented that it had been a challenge to secure 24 full days of protected time, an observation seconded by several of the department chairs called upon to protect that time. By the final week of the program however, all students reported a genuine sense of accomplishment, and had managed to give their full attention to program activities. Although we plan to continue INTREPID on its current schedule, we could easily envision that a program not designed to offer credit 498
VOLUME 7 • ISSUE 6
Conclusion
Going forward, we plan to offer INTREPID for a second cycle in July of 2014. Based on our inaugural experience, we expect that the program as designed can accommodate up to 20 students. We plan however, to first increase our student body from 6 to no more than 12 participants in year 2 of the program. While a staggered increase in class size will preserve a low student-faculty ratio, we anticipate a future need for additional faculty as class size grows to maintain ready faculty access for our students. In the meantime, we are tracking our current program graduates, and plan to survey them annually, both to observe their career development and to collect their reflections on the benefits and/ or deficiencies of the program. We are particularly interested in the extent to which these students continue to participate in our CTSI community and avail themselves of our other training and research resources. Our experience in developing and instituting INTREPID offers insights for others considering offering short-term training programs in clinical research. Overall, we observed that developing and implementing an intensive didactic immersion program in clinical research methodologies is feasible but requires significant faculty commitment, personnel, and infrastructure, as well as hard-working and committed trainees. The INTREPID program model met with excellent acceptance from faculty and participants, but surely represents only one of many possible alternatives. Acknowledgments
The authors acknowledge the NYU Advanced Learning EXchange (ALEX) for educational, Web, and informatics support. We thank Kevin Chan for compiling online information on educational programs offered by CTSA institutions, and Sondra R. Zabar, WWW.CTSJOURNAL.COM
Plottel et al. Designing/Implementing Clinical Research Training n
MD, for her review of an early draft version of the manuscript. We are grateful to the student-participants of the 2013 inaugural class of INTREPID for their enthusiasm, interest, and dedication to learning and thank their program directors for supporting our efforts and their trainees’ career development. This publication was supported in part by the National Center for Advancing Translational Sciences (NCATS) of the National Institutes of Health, through NYU-HHC CTSI grant number UL1 TR000038. Its contents are solely the responsibility of the authors and do not necessarily represent the official views of the National Institutes of Health. References
1. Cull WL, Yudkowsky BK, Schonfeld DJ, Berkowitz CD, Pan RJ. Research exposure during pediatric residency: influence on career expectations. J Pediatr. 2003; 143(5): 564–569. 2. Melnick A. Transitioning from fellowship to a physician-scientist career track. Hematology Am Soc Hematol Educ Program. 2008: 16–22. 3. Murillo H, Reece EA, Snyderman R, Sung NS. Meeting the challenges facing clinical research: solutions proposed by leaders of medical specialty and clinical research societies. Acad Med. 2006; 81(2): 107–112. 4. Neacy K, Stern SA, Kim HM, Dronen SC. Resident perception of academic skills training and impact on academic career choice. Acad Emerg Med. 2000; 7(12): 1408–1415.
14. Meyers FJ, Begg MD, Fleming M, Merchant C. Strengthening the career development of clinical translational scientist trainees: a consensus statement of the Clinical Translational Science Award (CTSA) Research Education and Career Development Committees. Clin Transl Sci. 2012; 5(2): 132–137. 15. Meyers FJ, Pomeroy C. Creating the future biomedical research workforce. Sci Transl Med. 2011; 3(102): 102fs5. 16. Chi KR. Conducting a trial. Nature. 2013; 493: 565–567. 17. Kon AA. The Clinical and Translational Science Award (CTSA) Consortium and the translational research model. Am J Bioeth. 2008; 8(3): 58–60; discussion W51–53. 18. Speicher LA, Fromell G, Avery S, Brassil D, Carlson L, Stevens E, Toms M. The critical need for academic health centers to assess the training, support, and career development requirements of clinical research coordinators: recommendations from the Clinical and Translational Science Award Research Coordinator Taskforce. Clin Transl Sci. 2012; 5(6): 470–475. 19. Zerhouni E. Medicine. The NIH Roadmap. Science. 2003; 302(5642): 63–72. 20. Zerhouni EA, Alving B. Clinical and translational science awards: a framework for a national research agenda. Transl Res 2006; 148(1): 4–5. 21. Halpain MC, Jeste DV, Trinidad GI, Wetherell JL, Lebowitz BD. Intensive short-term research training for undergraduate, graduate, and medical students: early experience with a new nationallevel approach in geriatric mental health. Acad Psychiatry. 2005; 29(1): 58–65. 22. Jeste DV, Halpain MC, Trinidad GI, Reichstadt JL, Lebowitz BD. UCSD’s short-term research training programs for trainees at different levels of career development. Acad Psychiatry. 2007; 31(2): 160–167. 23. Carek PJ, Dickerson LM, Diaz VA, Steyer TE. Addressing the scholarly activity requirements for residents: one program’s solution. J Grad Med Educ 2011; 3(3): 379–382.
5. Teo AR. The development of clinical research training: past history and current trends in the United States. Acad Med. 2009; 84(4): 433–438.
24. Vinci RJ, Bauchner H, Finkelstein J, Newby PK, Muret-Wagstaff S, Lovejoy FH Jr. Research during pediatric residency training: outcome of a senior resident block rotation. Pediatrics. 2009; 124(4): 1126–1134.
6. Bartels SJ, Lebowitz BD, Reynolds CF 3rd, Bruce ML, Helpain M, Faison WE, Kirwin PD. P rograms for developing the pipeline of early-career geriatric mental health researchers: o utcomes and implications for other fields. Acad Med. 2010; 85(1): 26–35.
25. Wagner RF Jr, Lewis SA. Teaching medical students dermatology research skills: six years of experience with the University of Texas Medical Branch dermatology non-degree research honors program, 2001–2006. Dermatol Online J. 2006; 12(4): 20.
7. Levine RB, Hebert RS, Wright SM. Resident research and scholarly activity in internal medicine residency training programs. J Gen Intern Med. 2005; 20(2): 155–159.
26. Lowe B, Hartmann M, Wild B, Nikendei C, Kroenke K, Niehoff D, Henningsen P, Zipfel S, Herzog W. Effectiveness of a 1-year resident training program in clinical research: a controlled before-and-after study. J Gen Intern Med 2008; 23(2): 122–128.
8. Basu Ray I, Henry TL, Davis W, Alam J, Amedee RG, Pinsky WW. Consolidated academic and research exposition: a pilot study of an innovative education method to increase residents’ research involvement. Ochsner J. 2012; 12(4): 367–372. 9. Oakley SH, Crisp CC, Estanol MV, Fellner AN, Kleeman SD, Pauls RN. Attitudes and compliance with research requirements in OB/GYN residencies: a national survey. Gynecol Obstet Invest. 2013; 75(4): 275–280.
27. Merwin SL, Fornari A, Lane LB. A preliminary report on the initiation of a clinical research program in an orthopaedic surgery department: roadmaps and tool kits. J Surg Educ. 2014; 71(1): 43–51. 28. Wagner RF Jr, Raimer SS, Kelly BC. Incorporating resident research into the dermatology residency program. Adv Med Educ Pract. 2013; 4: 77–81.
10. Rahman S, Majumder MA, Shaban SF, Rahman N, Ahmed M, Abdulrahman KB, D’Souza UJ. Physician participation in clinical research and trials: issues and approaches. Adv Med Educ Pract. 2011; 2: 85–93.
29. Armstrong AY, Decherney A, Leppert P, Rebar R, Maddox YT. Keeping clinicians in clinical research: the Clinical Research/Reproductive Scientist Training Program. Fertil Steril. 2009; 91(3): 664–666.
11. Gill S, Levin A, Djurdjev O, Yoshida EM. Obstacles to residents’ conducting research and predictors of publication. Acad Med. 2001; 76(5): 477.
30. Shaw KB, Mist S, Dixon MW, Goldby M, Weih J, Bauer VK, Ritenbaugh C. The Oregon Center for Complementary and Alternative Medicine career development program: innovation in research training for complementary and alternative medicine. Teach Learn Med. 2003; 15(1): 45–51.
12. Rivera JA, Levine RB, Wright SM. Completing a scholarly project during residency training. Perspectives of residents who have been successful. J Gen Intern Med. 2005; 20(4): 366–369. 13. Kurpinski K, Johnson T, Kumar S, Desai T, Li S. Mastering translational medicine: interdisciplinary education for a new generation. Sci Transl Med. 2014; 6(218): 218fs2.
WWW.CTSJOURNAL.COM
31. Harris PA, Taylor R, Thielke R, Payne J, Gonzalez N, Conde JG. Research electronic data capture (REDCap)—a metadata-driven methodology and workflow process for providing translational research informatics support. J Biomed Inform. 2009; 42(2): 377–381.
VOLUME 7 • ISSUE 6
499
Copyright of CTS: Clinical & Translational Science is the property of Wiley-Blackwell and its content may not be copied or emailed to multiple sites or posted to a listserv without the copyright holder's express written permission. However, users may print, download, or email articles for individual use.
Keywords: immersion training, INTREPID course, research certificate program, research intensive, short-term training program, summer intensive, study design, teaching clinical research methods Introduction
Inadequate mastery of clinical/translational research skills, time constraints, and competing tasks are major impediments to junior faculty and senior housestaff interested in performing patientcentered research.1–11 Of 73 internal medicine residents presenting their work at an American College of Physicians-American Society of Internal Medicine annual session, nearly half cited lack of research skills as a “major obstacle” to research.12 Earlystage academic physicians often plan clinical research projects without formal training in relevant methodologies.3,13–15 Reliance on experiential learning may result in delayed or less than optimal study design, recruitment, and implementation, as well as reduced likelihood of project funding. 16 In response to these challenges, the Clinical and Translational Science Awards (CTSA) initiative of the National Institutes of Health’s National Center for Advancing Translational Science (NCATS) has promoted the development of Masters-level programs in translational research methods.17–20 Such programs commonly require a 2- to 3-year, full- or parttime engagement, which not all junior trainees may be willing or able to commit to. In response to these concerns, many institutions have developed short-term programs in clinical research methodologies, often with a focus on predoctoral students or residents.21–25 A review of the 62 CTSA recipient Websites in February of 2014 reveals that at least 20 of the institutions offer some form of short-term educational programs. These range from brief offerings (one to several days) that are convenient but limited in scope, to more extensive programs (6–8 weeks) that may require more time away from clinical duties than many trainees can afford. Still other programs may be part-time but longitudinal over one or more semesters.26–28 An appreciation of best practices for short-term program design would help educators determine what programs would be best for their own institutions and trainee populations. Few reports however, address the design, implementation, successes, and limitations of the varying models of short-term training programs.29,30
The Clinical and Translational Science Institute (CTSI) of New York University and the New York City Health and Hospitals Corporation (NYU-HHC CTSI) recently designed and implemented INtensive Training in Research Statistics, Ethics, and Protocol Informatics and Design (INTREPID), a focused, intensive, cohesive immersion program in clinical research methodologies, providing in-depth knowledge of clinical research skills in less than 1 month. We hypothesized that such a structured didactic educational program would address the need for rapid learning and early acquisition of core translational research skills, and improve the probability of successfully conducting research. To promote community discussion of the best approaches for short-term program design, we here share our experience developing and implementing INTREPID, and the initial feedback we received from trainees and faculty. Methods
Program structure and overview INTREPID consists of three core courses and an optional course in Ethical Conduct of Research (Figure 1). The core courses, Introduction to Clinical Research Methods, Introductory Biostatistics, and Introduction to Biomedical Informatics and Computing, were derived and adapted from our preexisting Masters of Science in Clinical Investigation (MSCI) program. As the course content was already approved, these topics allowed us to meet requirements of both New York University and the New York State Department of Education, earning graduates a New York State–approved Certificate in Translational Research. Each course met for 36 face-to-face instruction hours, complemented by a carefully designed homework and self-learning component, with extensive preprogram, weeknight, and weekend assignments. The noncredit Ethical Conduct of Research course allowed participants to meet the NIH mandate for ethics training; participants were permitted to opt out if they had already met this requirement.
The NYU-HHC Clinical and Translational Science Institute, New York University School of Medicine, New York, New York, USA; 2Department of Medicine, New York University School of Medicine, New York, New York, USA; 3Department of Population Health, New York University School of Medicine, New York, New York, USA; 4Office of Science and Research, New York University Langone Medical Center, New York, New York, USA. 1
Correspondence : Claudia S. Plottel ([email protected]) DOI: 10.1111/cts.12198
WWW.CTSJOURNAL.COM
VOLUME 7 • ISSUE 6
493
Plottel et al. Designing/Implementing Clinical Research Training n
his/her area of expertise. To maintain the highest level of instruction, the Introductory Biostatistics and Introduction to Biomedical Informatics and Computing course directors invited a limited number of supplementary lecturers to teach specific topics in their specialty. The Program Director was responsible for overall organization, coordination, and program launch and in addition, directed the Introduction to Clinical Research Methods. In the months prior to the program start date, all course directors met weekly, facilitating a high degree of cooperation and collegiality. The excellent interpersonal rapport that ensued proved essential to the program, and enhanced the logistics of developing and integrating the required specialized and complementary course syllabi and curricula. Support staff A dedicated Program Coordinator supported INTREPID’s development and operations. The Program Coordinator, working closely with the Program Director, developed INTREPID-related announcements, and an online application and postprogram surveys using Research Electronic Data Capture (REDCap), a secure, Web-based application designed to support data capture for research studies hosted at NYU31 (http://www. project-redcap.org). The coordinator’s additional responsibilities included setting up INTREPID’s Web page, coordinating and attending faculty meetings, taking minutes, responding to phone and online inquiries, purchasing textbooks, arranging catered lunches, developing curricular information for the NYU School of Medicine’s learning portal (Advanced Learning Exchange [ALEX]), and drafting, distributing, and compiling online surveys at the program’s completion.
Figure 1. INTREPID program overview: courses, format, deliverables, faculty commitment. INTREPID took place over 24 consecutive calendar days and consisted of three 3-credit courses and an optional course in the ethical conduct of research to fulfill the NIH-mandated ethics training (courses listed in white boxes). Both the biostatistics course and the biomedical informatics course included a lab session, in addition to didactic lectures and seminar-style teaching (course format represented in black boxes). One hundred percent attendance was required, and specific deliverables varied according to each course (course deliverables specified in dark grey boxes). The overall faculty-to-student ratio exceeded 1 in the first year the program was offered (faculty cohorts indicated in lighter grey boxes).
Participants took all three core courses and the Ethical Conduct of Research course concurrently over 24 consecutive days (18 classroom instruction days), with 7.0–8.5 hours of classroom time daily. The program was intentionally scheduled for July to coincide with the hiring, onboarding, and/or advancement of housestaff and junior faculty. Successful completion of all courses required 100% attendance, which along with the submission of specific deliverables earned the student a total of 9 credit hours. Graduates are invited to apply their credits toward future enrollment in the NYU-HHC CTSI MSCI program, should they choose to seek additional training. Program faculty Faculty were selected from NYU Langone Medical Center and NYU School of Medicine, as well as from the Sackler Institute of Graduate Biomedical Sciences, a division of NYU’s Graduate School of the Arts and Sciences (Table 1). Each course director was responsible for developing course content and teaching in
Curriculum design The central pedagogical approach of INTREPID was to integrate topics across the courses (Figure 2). Weekly meetings (described above) leading up to the start of the program provided opportunity to draft syllabi, determine a weekly program focus, integrate coursework, and design interlocking exercises and approaches. In contrast to our MSCI program, whose longitudinal curriculum
Programmatic Role
Academic rank
Additional faculty
INTREPID Program Director
M.D., M.S. Clinical Associate Professor Medicine
No
Course Director: Intro to Clinical Research Methods
Same individual as above
No
Course Director: Introductory Biostatistics
Ph.D. Professor / Biostatistics
Yes: 2 additional tenured faculty
Course Director: Intro to Biomedical Informatics and Computing
M.D., Ph.D. Assistant Professor Biomedical Informatics
Yes: visiting lecturers
Course Director: Ethical Conduct of Research
Ph.D. Adjunct Assistant Professor Sackler Institute
Yes: 1 graduate assistant
INTREPID Program Coordinator
B.A. NYU-HHC CTSI
N/A
Table 1. INTREPID faculty characteristics for the inaugural year of the INTREPID program in 2013.
494
VOLUME 7 • ISSUE 6
WWW.CTSJOURNAL.COM
Plottel et al. Designing/Implementing Clinical Research Training n
Figure 2. Syllabus development for each 3-credit required course. Each Course Director created a syllabus de novo, taking advantage of redundancies and overlap between subject matter across the courses. The amount of classroom instruction remained constant each week, but the relative amount of homework varied as the students worked to complete their projects.
emphasizes career development and mentoring, we designed INTREPID to focus on skills acquisition. Educational strategies employed included: seminar-style classroom interactions, learnercentered experiential assignments (research protocol and code writing, close reading and written critiques of published research papers), hands-on real-time computing, quiz and homework sets, and regular access to faculty both during the day and off-hours. Directors sought to exploit overlap and synergies while balancing self-study and homework. For example, when programming in “R” was introduced in Introduction to Biomedical Informatics and Computing, relevant lab exercises in Introductory Biostatistics were assigned concurrently to reinforce the use of “R.” Similarly, when a conceptual overview of hypothesis testing was presented in the Introduction to Clinical Research Methods in the morning during week 1, the topic was complemented and reinforced in the Introductory Biostatistics afternoon class sessions, and in the days following when the mathematical and statistical methodologies were taught in depth. Each syllabus detailed the overall course goals, the daily lecture title and learning objectives, and daily reading and assignments. All syllabi were posted on ALEX and made available to student participants prior to the start of classes. ALEX also housed other course-specific materials, such as primary reference sources and relevant papers from the literature. Recruitment In the spring of 2013, information regarding INTREPID was emailed to department chairs, faculty, and housestaff at NYU Schools of Medicine, Dentistry, and Nursing, as well to NYUHHC CTSI-member hospitals. We limited the request for applications to all of our CTSI-affiliated institutions for our “pilot” year. To advertise INTREPID, the Program Director announced the program at venues and events within our medical center (e.g., Department of Medicine and Department of Pediatrics research days, and the NYU-HHC CTSI’s Translational Researchin-Progress seminars). Interested healthcare professionals were invited to contact the Program Director or Coordinator with WWW.CTSJOURNAL.COM
questions. A hyperlink was provided to access and submit the online application, including a statement of interest and a letter of recommendation. Each applicant’s department chair had to agree to cover the cost of tuition, and guarantee fully protected time, free of any clinical, on-call, or research duties, for the program duration. Tuition was set at a discounted rate (75%) relative to NYU’s usual per-credit charge. Applications were due 2 months prior to the course start date and were reviewed by a committee consisting of the INTREPID faculty, the CTSI Director, and the Director of Translational Research Education and Careers (TREC), the CTSI’s educational unit. Participants were notified once accepted, allowing sufficient time for their departments to make arrangements for the requisite protected time.
Teaching materials Enduring materials were selected and/or created for each course. Textbooks were provided to participants at no charge one week before classes began, along with required reading assignments. Textbook readings were augmented by relevant e-book chapters and curated papers from the literature, available via the NYU Health Sciences Library subscription collection. Learning environment A conference room in NYU’s Translational Research Building was reserved for INTREPID daily from 8:00 a.m. to 6:00 p.m. throughout the program. The conference room included a large table, state-of-the-art multimedia audiovisual capability, and Internet access. Classes were taught seminar-style. A computer lab equipped with PC-based computers and a SmartBoard® was reserved for the hands-on lab sessions of Introductory Biostatistics and Introduction to Biomedical Informatics and Computing, as well as for individual practice with “R Studio” and Power Analysis and Sample Size (PASS) statistical software. Each student was assigned his or her own lab PC for the duration of INTREPID. Schedule A sample schedule from week 2 of INTREPID is shown in Table 2. Each course met 4 out of 5 days; Mondays had the greatest number of class hours, and Fridays had the least (decreasing intensity of face-to-face time across the week). Both Introductory Biostatistics and Introduction to Biomedical Informatics and Computing were allocated one lab session weekly for hands on programming in “R” and instruction and practice in PASS. Homework assignments increased as the week progressed (increasing intensity of selfstudy across the week). Lunch was provided in the classroom daily, and at least one (more often 3 or 4) faculty member(s) attended lunch, allowing for ongoing interactions and feedback. In addition, during four lunch sessions spread throughout the program, invited guests gave informal “Learning Lunch” talks. Lunch speakers included: (i) a junior faculty member speaking about her “real-life” interventional clinical research project, describing pitfalls and successes; (ii) the CTSI librarian discussing literature searches, describing databases, and other available tools; (iii) the VOLUME 7 • ISSUE 6
495
Plottel et al. Designing/Implementing Clinical Research Training n
Week 2
Monday
Tuesday
Wednesday
Thursday
Friday
09:00–11:00
Research Methods
Research Methods
Informatics and Computing
Research Methods
Research Methods
11:00–11:30
Research Methods
Research Methods
Biostatistics
Research Methods
Research Methods
11:30–13:00
Informatics and Computing
Informatics and Computing
Biostatistics
Informatics and Computing
Ethics
13:00–14:00
LUNCH
LEARNING LUNCH
LUNCH
LUNCH
LUNCH
14:00–16:00
Informatics and omputing LAB C
Biostatistics
Biostatistics LAB
Biostatistics
Biostatistics
16:00–17:00
Informatics and Computing
Ethics
Ethics
Ethics
17:00–17:30
Informatics and Computing
Table 2. Sample schedule from Week 2 of INTREPID. Classroom hours decreased, and the amount of homework increased as the week progressed. Lunch was provided each weekday for participants and faculty, allowing for ongoing informal and formal interactions. See text for details.
Students
Terminal degree
Academic rank
Specialization
1
Sex M
M.D.
Fellow, Year 1
Maternal-Fetal Medicine
2
F
M.D.
Fellow, Year 2
Gynecological Oncology
3
M
M.D., M.P.H.
Instructor
Geriatrics
4
F
M.D.
Assistant Professor
Emergency Medicine
5
F
Ph.D.
Director of Research
Medicine; HHC Affiliate
6
M
M.D.
Visiting Physician
Medicine; International
Table 3. INTREPID participant characteristics of the inaugural INTREPID class in July 2013.
CTSI librarian speaking on systematic and narrative reviews; and (iv) an Institutional Review Board (IRB) representative addressing the role of the IRB and providing practical information about the submission and review processes.
second week, the students reported increased understanding of “R”. By the third week, students were independently writing code for their required individual informatics projects, and interrogating large datasets to answer specific clinical questions.
Results
Assessments To obtain formal assessments of INTREPID, we asked each student to complete two surveys. All students completed both surveys, which revealed a high degree of satisfaction with the program in general and with specific course elements. Survey 1 was e-mailed to each student at the completion of the program (Figure 3). All students (n = 6) indicated that INTREPID had increased their knowledge of biomedical informatics, programming, biostatistics, and research methods, and that the Clinical Research Methods class had met their educational expectations. Four of six students rated the Clinical Research Methods course as “very relevant” and two rated it “relevant” to their professional development and career goals, while five of six would “definitely” recommend the course to a colleague interested in research. Five students reported that the Ethical Conduct of Research course “definitely increased” their knowledge of research ethics. Half of the students reported that the Introduction to Biomedical Informatics and Computing and the Introductory Biostatistics courses “definitely” met their educational expectations and the remaining students stated that those classes had “likely” met expectations. All six students responded that “R” taught in Introduction to Biomedical Informatics and Computing was helpful in the Introductory Biostatistics course. Four students
Inaugural class INTREPID’s first cohort included six participants: four from NYU School of Medicine, one from an HHC facility, and one visiting international trainee supported by a Fogarty grant via the NIH (Table 3). All participants had previously completed an MD or PhD degree; one additionally held an MPH. Participants came from a variety of clinical specialties. Academic rank ranged from fellow to Assistant Professor. No participant had prior knowledge of programming or programming software. All enrolled in the ethics class. All participants successfully completed the program, meeting all requirements. Challenges The biggest challenge that our students identified during the program was learning “R” in the first week of INTREPID. Students reported that the workload and the need to rapidly develop new programming skills were daunting. We addressed this challenge by adding focused, supplemental, one-on-one instruction between each student and the Introduction to Biomedical Informatics and Computing professor, in person, by email, and by phone. By the 496
VOLUME 7 • ISSUE 6
WWW.CTSJOURNAL.COM
Plottel et al. Designing/Implementing Clinical Research Training n
Figure 3. Survey A results. All participants (n = 6) completed Survey A, which assessed each course separately on the last day of INTREPID. There was a high degree of satisfaction with course content.
would “definitely” recommend Introduction to Biomedical Informatics and Computing to a colleague and the remainder reported that they might. Half the students would “definitely” recommend the Introductory Biostatistics and Ethical Conduct of Research courses to a colleague, and the remainder would possibly (“maybe yes”) recommend them. Survey 2 was emailed to the participants six weeks after program completion, and focused on the program as a whole (Figure 4). All students reiterated that they would recommend INTREPID to a colleague. Four rated the program as “effective,” and two “very effective” in improving their overall knowledge of clinical research methodologies. Survey respondents were encouraged to enter free-text comments elaborating on their thoughts about the program. Most of the remarks focused on the individualized attention, intensity of the immersion experience, and rapid knowledge acquisition that the participants experienced. In addition, the integration of information between the courses was mentioned as effective and useful. Discussion
INTREPID’s development was facilitated by the availability of physical space and technological resources, and by motivated faculty members who had previously taught similar course content in NYU-HHC CTSI’s MSCI Program. The MSCI courses provided WWW.CTSJOURNAL.COM
guidance as to what materials should be presented but were significantly “re-fitted” specifically for an intense immersion. The integration of material common to research methods, biostatistics, and biomedical informatics in an intensive, side-by-side structure presented both challenges and opportunities. We quickly learned to avoid redundancy while exploiting pedagogically useful overlap between the courses. To do so required coordination between the faculty during curricular development, necessitating that the faculty relationship be close and collegial. In lieu of longitudinal career mentoring, we substituted an enriched emphasis on skills building, making INTREPID particularly well suited to physicians planning to “jump-start” their research careers in fellowship or early in their positions as junior faculty. We anticipate a limited, but steady demand from this group for INTREPID training. Our decision to include a brief but cognitively intensive training in “R” programming was challenging, particularly in the first week. In this, and several other aspects of the program, the constant availability of the faculty proved essential to ultimate student success. Although designed with postdoctoral learners in mind, we did open INTREPID applications to medical students and residents. NYU medical students are offered a 12-week block of research time in their fourth year, yet none chose to use a portion of that time to take INTREPID, despite support for the program from their Associate Dean. Instead, our medical students interested VOLUME 7 • ISSUE 6
497
Plottel et al. Designing/Implementing Clinical Research Training n
hours might find it salutary to consider a schedule between 14 and 21 days, rather than our full 24. Our decision to charge discounted tuition relative to NYU’s per-credit fee was controversial among both the faculty and consumers. Some considered that as a credit-granting program, we should charge the same full per-credit tuition amount as the university. Others affirmed that as a service to our research community we ought to offer INTREPID gratis. We decided that the program should at minimum cover its own costs, and that requiring a per-credit discounted tuition “copayment” from a sponsoring department would insure that highly motivated students would participate. Although several departments expressed reservations about the tuition, in only one case—a clinical fellow who would be leaving at the end of the year—did a department decline support, explicitly stating to us that it was not in the departmental interest to invest in the professional development of a trainee who would not be remaining at our institution to apply the skills obtained. Figure 4. Survey B results. All participants (n = 6) completed a second survey 6 weeks after the course completion, when they had returned to their professional responsibilities and duties.
in research chose to enroll in our five-year, combined MD-MS in Clinical Investigation program, which is longer, and more comprehensive than INTREPID, offers more mentoring, and results in publications, all of which are attractive to predoctoral students planning their residencies. As for residents, we believe that the requirement that INTREPID participants be free of all clinical duties for 24 consecutive days in July makes participation unlikely as even a resident on elective is required to take call as part of clinical duties. For both medical students and residents who need a more basic grounding in clinical research methodologies, a less intense, more flexible, possibly asynchronous program might be desirable. The fact that we received applications only from physicians postresidency and beyond reinforces our belief that senior housestaff (fellowship level) and junior faculty have a largely unrecognized and unmet need for didactic instruction in translational research. Our decision to offer course credit and a university-sponsored certificate shaped INTREPID’s structure and content. Our 24-day schedule was the absolute minimum in which all three courses could be delivered with an appropriate number of credit hours. Trainees, all of whom were postdoctoral, commented on both the duration and intensity of the program; some found it particularly intense, especially during the first week. Others commented that it had been a challenge to secure 24 full days of protected time, an observation seconded by several of the department chairs called upon to protect that time. By the final week of the program however, all students reported a genuine sense of accomplishment, and had managed to give their full attention to program activities. Although we plan to continue INTREPID on its current schedule, we could easily envision that a program not designed to offer credit 498
VOLUME 7 • ISSUE 6
Conclusion
Going forward, we plan to offer INTREPID for a second cycle in July of 2014. Based on our inaugural experience, we expect that the program as designed can accommodate up to 20 students. We plan however, to first increase our student body from 6 to no more than 12 participants in year 2 of the program. While a staggered increase in class size will preserve a low student-faculty ratio, we anticipate a future need for additional faculty as class size grows to maintain ready faculty access for our students. In the meantime, we are tracking our current program graduates, and plan to survey them annually, both to observe their career development and to collect their reflections on the benefits and/ or deficiencies of the program. We are particularly interested in the extent to which these students continue to participate in our CTSI community and avail themselves of our other training and research resources. Our experience in developing and instituting INTREPID offers insights for others considering offering short-term training programs in clinical research. Overall, we observed that developing and implementing an intensive didactic immersion program in clinical research methodologies is feasible but requires significant faculty commitment, personnel, and infrastructure, as well as hard-working and committed trainees. The INTREPID program model met with excellent acceptance from faculty and participants, but surely represents only one of many possible alternatives. Acknowledgments
The authors acknowledge the NYU Advanced Learning EXchange (ALEX) for educational, Web, and informatics support. We thank Kevin Chan for compiling online information on educational programs offered by CTSA institutions, and Sondra R. Zabar, WWW.CTSJOURNAL.COM
Plottel et al. Designing/Implementing Clinical Research Training n
MD, for her review of an early draft version of the manuscript. We are grateful to the student-participants of the 2013 inaugural class of INTREPID for their enthusiasm, interest, and dedication to learning and thank their program directors for supporting our efforts and their trainees’ career development. This publication was supported in part by the National Center for Advancing Translational Sciences (NCATS) of the National Institutes of Health, through NYU-HHC CTSI grant number UL1 TR000038. Its contents are solely the responsibility of the authors and do not necessarily represent the official views of the National Institutes of Health. References
1. Cull WL, Yudkowsky BK, Schonfeld DJ, Berkowitz CD, Pan RJ. Research exposure during pediatric residency: influence on career expectations. J Pediatr. 2003; 143(5): 564–569. 2. Melnick A. Transitioning from fellowship to a physician-scientist career track. Hematology Am Soc Hematol Educ Program. 2008: 16–22. 3. Murillo H, Reece EA, Snyderman R, Sung NS. Meeting the challenges facing clinical research: solutions proposed by leaders of medical specialty and clinical research societies. Acad Med. 2006; 81(2): 107–112. 4. Neacy K, Stern SA, Kim HM, Dronen SC. Resident perception of academic skills training and impact on academic career choice. Acad Emerg Med. 2000; 7(12): 1408–1415.
14. Meyers FJ, Begg MD, Fleming M, Merchant C. Strengthening the career development of clinical translational scientist trainees: a consensus statement of the Clinical Translational Science Award (CTSA) Research Education and Career Development Committees. Clin Transl Sci. 2012; 5(2): 132–137. 15. Meyers FJ, Pomeroy C. Creating the future biomedical research workforce. Sci Transl Med. 2011; 3(102): 102fs5. 16. Chi KR. Conducting a trial. Nature. 2013; 493: 565–567. 17. Kon AA. The Clinical and Translational Science Award (CTSA) Consortium and the translational research model. Am J Bioeth. 2008; 8(3): 58–60; discussion W51–53. 18. Speicher LA, Fromell G, Avery S, Brassil D, Carlson L, Stevens E, Toms M. The critical need for academic health centers to assess the training, support, and career development requirements of clinical research coordinators: recommendations from the Clinical and Translational Science Award Research Coordinator Taskforce. Clin Transl Sci. 2012; 5(6): 470–475. 19. Zerhouni E. Medicine. The NIH Roadmap. Science. 2003; 302(5642): 63–72. 20. Zerhouni EA, Alving B. Clinical and translational science awards: a framework for a national research agenda. Transl Res 2006; 148(1): 4–5. 21. Halpain MC, Jeste DV, Trinidad GI, Wetherell JL, Lebowitz BD. Intensive short-term research training for undergraduate, graduate, and medical students: early experience with a new nationallevel approach in geriatric mental health. Acad Psychiatry. 2005; 29(1): 58–65. 22. Jeste DV, Halpain MC, Trinidad GI, Reichstadt JL, Lebowitz BD. UCSD’s short-term research training programs for trainees at different levels of career development. Acad Psychiatry. 2007; 31(2): 160–167. 23. Carek PJ, Dickerson LM, Diaz VA, Steyer TE. Addressing the scholarly activity requirements for residents: one program’s solution. J Grad Med Educ 2011; 3(3): 379–382.
5. Teo AR. The development of clinical research training: past history and current trends in the United States. Acad Med. 2009; 84(4): 433–438.
24. Vinci RJ, Bauchner H, Finkelstein J, Newby PK, Muret-Wagstaff S, Lovejoy FH Jr. Research during pediatric residency training: outcome of a senior resident block rotation. Pediatrics. 2009; 124(4): 1126–1134.
6. Bartels SJ, Lebowitz BD, Reynolds CF 3rd, Bruce ML, Helpain M, Faison WE, Kirwin PD. P rograms for developing the pipeline of early-career geriatric mental health researchers: o utcomes and implications for other fields. Acad Med. 2010; 85(1): 26–35.
25. Wagner RF Jr, Lewis SA. Teaching medical students dermatology research skills: six years of experience with the University of Texas Medical Branch dermatology non-degree research honors program, 2001–2006. Dermatol Online J. 2006; 12(4): 20.
7. Levine RB, Hebert RS, Wright SM. Resident research and scholarly activity in internal medicine residency training programs. J Gen Intern Med. 2005; 20(2): 155–159.
26. Lowe B, Hartmann M, Wild B, Nikendei C, Kroenke K, Niehoff D, Henningsen P, Zipfel S, Herzog W. Effectiveness of a 1-year resident training program in clinical research: a controlled before-and-after study. J Gen Intern Med 2008; 23(2): 122–128.
8. Basu Ray I, Henry TL, Davis W, Alam J, Amedee RG, Pinsky WW. Consolidated academic and research exposition: a pilot study of an innovative education method to increase residents’ research involvement. Ochsner J. 2012; 12(4): 367–372. 9. Oakley SH, Crisp CC, Estanol MV, Fellner AN, Kleeman SD, Pauls RN. Attitudes and compliance with research requirements in OB/GYN residencies: a national survey. Gynecol Obstet Invest. 2013; 75(4): 275–280.
27. Merwin SL, Fornari A, Lane LB. A preliminary report on the initiation of a clinical research program in an orthopaedic surgery department: roadmaps and tool kits. J Surg Educ. 2014; 71(1): 43–51. 28. Wagner RF Jr, Raimer SS, Kelly BC. Incorporating resident research into the dermatology residency program. Adv Med Educ Pract. 2013; 4: 77–81.
10. Rahman S, Majumder MA, Shaban SF, Rahman N, Ahmed M, Abdulrahman KB, D’Souza UJ. Physician participation in clinical research and trials: issues and approaches. Adv Med Educ Pract. 2011; 2: 85–93.
29. Armstrong AY, Decherney A, Leppert P, Rebar R, Maddox YT. Keeping clinicians in clinical research: the Clinical Research/Reproductive Scientist Training Program. Fertil Steril. 2009; 91(3): 664–666.
11. Gill S, Levin A, Djurdjev O, Yoshida EM. Obstacles to residents’ conducting research and predictors of publication. Acad Med. 2001; 76(5): 477.
30. Shaw KB, Mist S, Dixon MW, Goldby M, Weih J, Bauer VK, Ritenbaugh C. The Oregon Center for Complementary and Alternative Medicine career development program: innovation in research training for complementary and alternative medicine. Teach Learn Med. 2003; 15(1): 45–51.
12. Rivera JA, Levine RB, Wright SM. Completing a scholarly project during residency training. Perspectives of residents who have been successful. J Gen Intern Med. 2005; 20(4): 366–369. 13. Kurpinski K, Johnson T, Kumar S, Desai T, Li S. Mastering translational medicine: interdisciplinary education for a new generation. Sci Transl Med. 2014; 6(218): 218fs2.
WWW.CTSJOURNAL.COM
31. Harris PA, Taylor R, Thielke R, Payne J, Gonzalez N, Conde JG. Research electronic data capture (REDCap)—a metadata-driven methodology and workflow process for providing translational research informatics support. J Biomed Inform. 2009; 42(2): 377–381.
VOLUME 7 • ISSUE 6
499
Copyright of CTS: Clinical & Translational Science is the property of Wiley-Blackwell and its content may not be copied or emailed to multiple sites or posted to a listserv without the copyright holder's express written permission. However, users may print, download, or email articles for individual use.
