CTSA Profile
The Research Launchpad: Measuring the Efficiency and Effectiveness of CTSA Resource and Service Provision Arianne E. Rhea, M.P.H., Wei Xu, M.S., and Jingyuan Gu, Ph.D. Realizing a Need
In June of 2013, the Institutes of Medicine (IOM) issued an independent review of the Clinical and Translational Science Award (CTSA) program, currently administered through the National Center for Advancing Translational Sciences (NCATS) at the National Institutes of Health (NIH). In this review, heavy emphasis was placed on tracking and evaluation of individual and program-wide CTSA resources, return on investment (ROI), and clinical and translational research outcomes. Specifically, the IOM called for “a formalize[d] and standardize[d] evaluation process for individual CTSAs and the CTSA Program [that should] use clear, consistent, and innovative metrics that align with the program’s mission and goals and that go beyond standard academic benchmarks of publications and number of grant awards….”1 Additionally, the IOM stated “…the CTSA program can be a leader in developing evaluation methodologies and metrics that could provide more real-time assessments of progress in advancing clinical and translational research, overcoming research barriers, fulfilling the program's mission and strategic goals, and, whenever possible, changing clinical care and improving public health.”1 In order to support these calls to action, it is helpful to link process and outcome assessments through standard definitions, real-time data collection, and concrete metrics that demonstrate the efficiency and effectiveness of CTSA program activities. At UT Southwestern Medical Center, efficiency and effectiveness are ingrained into the Tracking and Evaluation component of the Center for Translational Medicine (CTM), home of our CTSA program. We measure efficiency, or “the ability to do something or produce something without wasting materials, time, or energy….”2 by gathering baselines of quantitative indicators for all of our programs (including process time, resource and funding allocations, etc.); we measure effectiveness, or “the degree to which something is successful in producing a desired result….”3 by documenting process and outcome metrics for all program components and measuring progress toward them at regular intervals. In fall of 2012, we began work on an internal resource and service request system, called the Research Launchpad, which contributes both efficiency and effectiveness indicators. Like many of our peers, the CTM supports an array of clinical and translational research resources and services across campus; in order to capture the full breadth and impact of this support, we designed the Research Launchpad to serve as a one-stop shop for clinical and translational investigators seeking our services. The system has been used internally since summer of 2013 and
was released to the campus in January 2014; since that time, the Research Launchpad has processed requests for services across a wide range of CTSA specialties, including bioinformatics, biostatistical consultations, research participant registries, use of our clinical research unit, and translational pilot program funds. To summarize, the Research Launchpad provides investigators with direct access to CTSA resources and services, allows the CTM to continuously measure efficiency and effectiveness, and facilitates transparency of service utilization to our leaders and the research community. Designing a User-Friendly Interface
To facilitate automated resource tracking through real-time data collection and ensure ease of use, the CTM created a Web-based system our investigators and study teams can access using institutional login credentials. The Research Launchpad is a JAVA EE Web application built on a Spring MVC Framework; users can access the application through any java-compatible Web browser. An SQL Server database is used to log and track all service requests; currently, it is deployed on aWebLogic application server for transaction management. To make full use of preexisting institutional data, we designed interfaces between our electronic IRB system (eIRB) and internal human resources database to prepopulate active research protocol information and demographic data for each user. When investigators or study team members log in to the Research Launchpad, they are immediately brought to a dashboard display of their activity within the system: details for all previous requests and the status of each are listed. Entering a new request is easy; users simply click the “new request” button and they are directed through a series of screens to submit their information. Users receive automated email updates each time the status of their request changes and can communicate with their service provider directly through the Website. The advantages of using this Web-based system linked to institutional databases are numerous: accurate institutional data can be easily retrieved, users do not need to enter duplicate information, and data pulled for reporting and analyses supports standard metrics and an efficient workflow. Figure 1 shows a process map for the life cycle of a request entered in Research Launchpad; milestones for each service offered in the system are measured and compared to this template. If a resource or service is completed outside the upper or lower request processing time boundaries, it is further
Center for Translational Medicine (CTM), UT Southwestern Medical Center, Dallas, Texas, USA . Research reported in this publication was supported by the National Center for Advancing Translational Sciences (NCATS) of the National Institutes of Health (NIH) under award Number UL1TR001105. The content is solely the responsibility of the authors and does not necessarily represent the official views of the NIH. Correspondence: Arianne E. Rhea ([email protected]) DOI: 10.1111/cts.12200
WWW.CTSJOURNAL.COM
VOLUME 7 • ISSUE 5
355
Figure 1. Research Launchpad request process.
analyzed to determine whether this deviation can be replicated or avoided for future requests. Documenting the Research Launchpad's standard request processing time provides a benchmark to determine the efficiency and effectiveness of individual requests. Access and Availability—Whom Do We Serve?
The Research Launchpad is available to anyone with a UT Southwestern username and password. Members of partnering institutions can request login credentials from the institution; external investigators and study teams have full access to CTSA services once this request is granted. In order to support full disclosure of CTSA activities across campus, the CTM instituted a policy requiring all resource and service requests to originate through the Research Launchpad; service providers now require a submitted request before beginning work on any CTSA project. These official requests allow us to analyze the population of investigators utilizing CTSA resources at UT Southwestern; to date, this population is comprised of 48% Assistant Professors, 62% postgraduate degrees, (38% MD, 14% PhD, 10% dual degree), 29% Asian and 24% white, non-Hispanic users. Departments represented include Internal Medicine, Dermatology, Ophthalmology, Pediatrics, Clinical Sciences, and Surgery; examples of research specialties include cutaneous lupus erythematosus, corneal cell biology, health disparities, cancer, psychiatry, HPV vaccination, cystic fibrosis, geographic information systems, and patient-centered outcomes research. These statistics provide baseline measurements to compare Research Launchpad user demographics to 356
VOLUME 7 • ISSUE 5
institutional data and to design future resource marketing strategies. The End Product—Our Impact
In order to demonstrate our impact on clinical and translational science across the UT Southwestern campus and North Texas, the CTM follows CTSA-supported research from inception to completion. Service requests in the Research Launchpad represent the initial stage of this process: providing investigators the means to translate their research ideas into concrete products through automated request submission and tracking. Subsequent research outcomes from service requests initiated in the Research Launchpad represent CTSA success stories; our CTM can demonstrate significant involvement in the initial stages of these research projects. The Research Launchpad not only helps the CTM measure success at UT Southwestern; it also helps demonstrate our impact on clinical and translational science to the NIH and other CTSA institutions. The CTM is eager to collaborate with fellow CTSA institutions to achieve IOM recommendations and will work with interested groups to implement this tracking solution free-of-charge. As more institutions track the efficiency and effectiveness of CTSA resource and service provision through the Research Launchpad, the system can be expanded to fulfill requirements of individual programs and NCATS/NIH. The Research Launchpad allows the CTM to define populations of investigators and research studies utilizing CTSA expertise, identify service gaps, direct future marketing strategies, and quantify our efforts to catalyze research theories into tangible outcomes. The IOM summarized its 2013 WWW.CTSJOURNAL.COM
recommendations by stating “Evaluation can be an incentive or catalyst for positive change and improved outcomes and is necessary for accountability, transparency, informed decision making, and communication about outcomes and the value of an investment.”1 The Research Launchpad emulates this statement by increasing transparency of and improving accountability for CTM resources: supporting data-driven decision making and informed allocation of CTSA funds to research services at UT Southwestern Medical Center.
WWW.CTSJOURNAL.COM
References 1. IOM (Institute of Medicine). The CTSA program at NIH: Opportunities for advancing clinical and translational research. Washington, DC: The National Academies Press; 2013. Vol. 8: 81, 89.
2. Efficiency ( 2014). Available at: http://www.merriam-webster.com/dictionary/efficiency. Accessed June 2, 2014.
3. Definition of effectiveness in English ( 2014). Available at: http://www.oxforddictionaries. com/us/definition/american_english/effectiveness. Accessed June 2, 2014.
VOLUME 7 • ISSUE 5
357
Copyright of CTS: Clinical & Translational Science is the property of Wiley-Blackwell and its content may not be copied or emailed to multiple sites or posted to a listserv without the copyright holder's express written permission. However, users may print, download, or email articles for individual use.
The Research Launchpad: Measuring the Efficiency and Effectiveness of CTSA Resource and Service Provision Arianne E. Rhea, M.P.H., Wei Xu, M.S., and Jingyuan Gu, Ph.D. Realizing a Need
In June of 2013, the Institutes of Medicine (IOM) issued an independent review of the Clinical and Translational Science Award (CTSA) program, currently administered through the National Center for Advancing Translational Sciences (NCATS) at the National Institutes of Health (NIH). In this review, heavy emphasis was placed on tracking and evaluation of individual and program-wide CTSA resources, return on investment (ROI), and clinical and translational research outcomes. Specifically, the IOM called for “a formalize[d] and standardize[d] evaluation process for individual CTSAs and the CTSA Program [that should] use clear, consistent, and innovative metrics that align with the program’s mission and goals and that go beyond standard academic benchmarks of publications and number of grant awards….”1 Additionally, the IOM stated “…the CTSA program can be a leader in developing evaluation methodologies and metrics that could provide more real-time assessments of progress in advancing clinical and translational research, overcoming research barriers, fulfilling the program's mission and strategic goals, and, whenever possible, changing clinical care and improving public health.”1 In order to support these calls to action, it is helpful to link process and outcome assessments through standard definitions, real-time data collection, and concrete metrics that demonstrate the efficiency and effectiveness of CTSA program activities. At UT Southwestern Medical Center, efficiency and effectiveness are ingrained into the Tracking and Evaluation component of the Center for Translational Medicine (CTM), home of our CTSA program. We measure efficiency, or “the ability to do something or produce something without wasting materials, time, or energy….”2 by gathering baselines of quantitative indicators for all of our programs (including process time, resource and funding allocations, etc.); we measure effectiveness, or “the degree to which something is successful in producing a desired result….”3 by documenting process and outcome metrics for all program components and measuring progress toward them at regular intervals. In fall of 2012, we began work on an internal resource and service request system, called the Research Launchpad, which contributes both efficiency and effectiveness indicators. Like many of our peers, the CTM supports an array of clinical and translational research resources and services across campus; in order to capture the full breadth and impact of this support, we designed the Research Launchpad to serve as a one-stop shop for clinical and translational investigators seeking our services. The system has been used internally since summer of 2013 and
was released to the campus in January 2014; since that time, the Research Launchpad has processed requests for services across a wide range of CTSA specialties, including bioinformatics, biostatistical consultations, research participant registries, use of our clinical research unit, and translational pilot program funds. To summarize, the Research Launchpad provides investigators with direct access to CTSA resources and services, allows the CTM to continuously measure efficiency and effectiveness, and facilitates transparency of service utilization to our leaders and the research community. Designing a User-Friendly Interface
To facilitate automated resource tracking through real-time data collection and ensure ease of use, the CTM created a Web-based system our investigators and study teams can access using institutional login credentials. The Research Launchpad is a JAVA EE Web application built on a Spring MVC Framework; users can access the application through any java-compatible Web browser. An SQL Server database is used to log and track all service requests; currently, it is deployed on aWebLogic application server for transaction management. To make full use of preexisting institutional data, we designed interfaces between our electronic IRB system (eIRB) and internal human resources database to prepopulate active research protocol information and demographic data for each user. When investigators or study team members log in to the Research Launchpad, they are immediately brought to a dashboard display of their activity within the system: details for all previous requests and the status of each are listed. Entering a new request is easy; users simply click the “new request” button and they are directed through a series of screens to submit their information. Users receive automated email updates each time the status of their request changes and can communicate with their service provider directly through the Website. The advantages of using this Web-based system linked to institutional databases are numerous: accurate institutional data can be easily retrieved, users do not need to enter duplicate information, and data pulled for reporting and analyses supports standard metrics and an efficient workflow. Figure 1 shows a process map for the life cycle of a request entered in Research Launchpad; milestones for each service offered in the system are measured and compared to this template. If a resource or service is completed outside the upper or lower request processing time boundaries, it is further
Center for Translational Medicine (CTM), UT Southwestern Medical Center, Dallas, Texas, USA . Research reported in this publication was supported by the National Center for Advancing Translational Sciences (NCATS) of the National Institutes of Health (NIH) under award Number UL1TR001105. The content is solely the responsibility of the authors and does not necessarily represent the official views of the NIH. Correspondence: Arianne E. Rhea ([email protected]) DOI: 10.1111/cts.12200
WWW.CTSJOURNAL.COM
VOLUME 7 • ISSUE 5
355
Figure 1. Research Launchpad request process.
analyzed to determine whether this deviation can be replicated or avoided for future requests. Documenting the Research Launchpad's standard request processing time provides a benchmark to determine the efficiency and effectiveness of individual requests. Access and Availability—Whom Do We Serve?
The Research Launchpad is available to anyone with a UT Southwestern username and password. Members of partnering institutions can request login credentials from the institution; external investigators and study teams have full access to CTSA services once this request is granted. In order to support full disclosure of CTSA activities across campus, the CTM instituted a policy requiring all resource and service requests to originate through the Research Launchpad; service providers now require a submitted request before beginning work on any CTSA project. These official requests allow us to analyze the population of investigators utilizing CTSA resources at UT Southwestern; to date, this population is comprised of 48% Assistant Professors, 62% postgraduate degrees, (38% MD, 14% PhD, 10% dual degree), 29% Asian and 24% white, non-Hispanic users. Departments represented include Internal Medicine, Dermatology, Ophthalmology, Pediatrics, Clinical Sciences, and Surgery; examples of research specialties include cutaneous lupus erythematosus, corneal cell biology, health disparities, cancer, psychiatry, HPV vaccination, cystic fibrosis, geographic information systems, and patient-centered outcomes research. These statistics provide baseline measurements to compare Research Launchpad user demographics to 356
VOLUME 7 • ISSUE 5
institutional data and to design future resource marketing strategies. The End Product—Our Impact
In order to demonstrate our impact on clinical and translational science across the UT Southwestern campus and North Texas, the CTM follows CTSA-supported research from inception to completion. Service requests in the Research Launchpad represent the initial stage of this process: providing investigators the means to translate their research ideas into concrete products through automated request submission and tracking. Subsequent research outcomes from service requests initiated in the Research Launchpad represent CTSA success stories; our CTM can demonstrate significant involvement in the initial stages of these research projects. The Research Launchpad not only helps the CTM measure success at UT Southwestern; it also helps demonstrate our impact on clinical and translational science to the NIH and other CTSA institutions. The CTM is eager to collaborate with fellow CTSA institutions to achieve IOM recommendations and will work with interested groups to implement this tracking solution free-of-charge. As more institutions track the efficiency and effectiveness of CTSA resource and service provision through the Research Launchpad, the system can be expanded to fulfill requirements of individual programs and NCATS/NIH. The Research Launchpad allows the CTM to define populations of investigators and research studies utilizing CTSA expertise, identify service gaps, direct future marketing strategies, and quantify our efforts to catalyze research theories into tangible outcomes. The IOM summarized its 2013 WWW.CTSJOURNAL.COM
recommendations by stating “Evaluation can be an incentive or catalyst for positive change and improved outcomes and is necessary for accountability, transparency, informed decision making, and communication about outcomes and the value of an investment.”1 The Research Launchpad emulates this statement by increasing transparency of and improving accountability for CTM resources: supporting data-driven decision making and informed allocation of CTSA funds to research services at UT Southwestern Medical Center.
WWW.CTSJOURNAL.COM
References 1. IOM (Institute of Medicine). The CTSA program at NIH: Opportunities for advancing clinical and translational research. Washington, DC: The National Academies Press; 2013. Vol. 8: 81, 89.
2. Efficiency ( 2014). Available at: http://www.merriam-webster.com/dictionary/efficiency. Accessed June 2, 2014.
3. Definition of effectiveness in English ( 2014). Available at: http://www.oxforddictionaries. com/us/definition/american_english/effectiveness. Accessed June 2, 2014.
VOLUME 7 • ISSUE 5
357
Copyright of CTS: Clinical & Translational Science is the property of Wiley-Blackwell and its content may not be copied or emailed to multiple sites or posted to a listserv without the copyright holder's express written permission. However, users may print, download, or email articles for individual use.
