SPECIAL FOCUS y The ARRA investment in CER Research Article For reprint orders, please contact:
[email protected] American Recovery and Reinvestment Act-comparative effectiveness research infrastructure investments: emerging data resources, tools and publications The Recovery Act provided a substantial, one-time investment in data infrastructure for comparative effectiveness research (CER). A review of the publications, data, and tools developed as a result of this support has informed understanding of the level of effort undertaken by these projects. Structured search queries, as well as outreach efforts, were conducted to identify and review resources from American Recovery and Reinvestment Act of 2009 CER projects building electronic clinical data infrastructure. The findings from this study provide a spectrum of productivity across a range of topics and settings. A total of 451 manuscripts published in 192 journals, and 141 data resources and tools were identified and address gaps in evidence on priority populations, conditions, and the infrastructure needed to support CER.
Courtney Segal*,1 & Erin Holve1 Research & Education in Health Services Research, AcademyHealth, 1150 17th Street NW, Suite 600, Washington, DC 20036, USA *Author for correspondence: Tel.: +1 202 292 6737 Fax: +1 202 292 6837
[email protected] 1
Keywords: clinical informatics • comparative effectiveness research • data infrastructure • health information technology • learning health system • patient-centered outcomes research
Given the substantial, one-time investment in infrastructure for comparative effectiveness research (CER) provided by the American Recovery and Reinvestment Act of 2009 (ARRA), it is critical to understand the level of productivity and dissemination efforts undertaken by these unique projects so that the contributions and impact of this significant federal investment are transparent and accessible. A review of the publications, data and informatics tools developed as a result of this support provides a unique opportunity, and first important step, to understand the contributions as well as the timeline for dissemination and translation of ARRA-funded work. The ARRA funding provided a unique opportunity to improve the data infrastructure, capacity and methods for conducting CER. These resources opened the door to innovative approaches to cultivate new evidence and support CER and patient-centered outcomes research (PCOR). Furthermore, it was explicitly hoped that the ARRA data infrastructure awards would expand capacity
10.2217/CER.14.71 © 2014 Future Medicine Ltd
not only in the research domain, but would build and implement tools using real-world data and contribute to building learning systems that will improve the practice of healthcare [1] . A learning health system aligns the evidence generated through research, quality assessment and clinical outcomes to support improvement and innovation for healthcare [2,3] . A robust infrastructure (i.e., governance, data, methods and training) is key to sustaining evidence generation in a learning health system [4,5] . To achieve these goals, it was viewed that investments in infrastructure were needed to develop new approaches to securely exchange data, and develop new methods to answer questions that span diverse care settings [1] . The majority of the ARRA-CER data infrastructure awards concluded in the fall of 2013 [6] . As a result, it is timely to review the available resources and products that have been disseminated in order to present an initial understanding of the productivity of the awards, to lead to understandings of how new and enhanced infrastructure has
J. Comp. Eff. Res. (2014) 3(6), 647–655
part of
ISSN 2042-6305
647
Research Article Segal & Holve improved the capacity to conduct rigorous and timely CER. This insight can inform where progress has been made to generate evidence on priority populations and conditions, as well as address identified gaps in the literature on the infrastructure needed to use clinical informatics for CER [7] . This paper focuses on a subset of grants and contracts funded through the ARRA-CER awards building electronic clinical data infrastructure, and provides early insight into the productivity of the awards, including publications, gray literature and data resources and tools. Methods This study was conducted in the spring of 2014, based on a prior portfolio analysis of the ARRA-CER awards conducted in 2012 [8] , and subsequently in 2013 [9] . All efforts focused on identifying the products resulting from grants and contracts funded through the ARRA-CER awards building electronic clinical data infrastructure. This paper reviews information from 65 projects funded through 13 grant programs. Table 1 provides a breakdown of the programs and awards identified in the previous funding analysis that led to the identification of the awards reviewed in this paper. Further details and methods for the previous funding analysis are published and available online [10] . To obtain individual project information, several federal databases and websites were cross-referenced and used to extract information about the individual awards, because spending is not tracked and reported
in any central, publicly accessible location. The subsequent search strategy relied on the unique identifier for the grant program (the funding opportunity announcement [FOA] number) or contract (the task order number) to obtain information about the individual awards in NIH RePORTER [11] . Although the number of awards within each grant program was known (86 awards within 130 programs), the method to use the FOA number in RePORTER to find the individual ARRA awards that fit within the study’s cohort was not feasible for all of the projects. For example, programs like the Clinical and Translational Science Awards, which repeatedly use the name FOA number for successive award cycles, produced thousands of results. Due to this limitation, the unique number of ARRA projects that met the electronic data infrastructure criteria and were included in this analysis was 65 awards, from 13 grant programs. Information about the projects, including abstract and contact information for the principal investigator, was obtained for the 65 projects from NIH RePORTER using the unique identifier. Appendix A provides the list of 65 projects reviewed in this study, and includes the funding organization, FOA number (which represents the grant program), grant ID, title and website (if available) for each of the projects. Search strategies to identify publications & resources
To identify peer-reviewed literature that was supported by these awards, a structured query using
Table 1. Identification of Electronic Clinical Data Infrastructure Awards from the total American Reinvestment Act-comparative effectiveness research funding. Funding category
Grant programs Individual Amount of and contract awards within funding (US$) task orders (n) each program (n)
1. Overall ARRA-CER investment
160
520
1,100,000,000
2. ARRA-CER infrastructure and methods development awards. Identified subset of programs focused on methods, data, governance and training
130
173
417,200,000
3. ARRA-CER infrastructure and Methods Development Awards, specifically building electronic clinical data infrastructure. Grant programs and contract task orders were reviewed to identify those building or enhancing clinical and claims databases, EHRs, data warehouses, patient registries and distributed and federated data networks, as well as the informatics platforms, systems and models to collect, link and exchange data within and across networks and data resources
30
86
275,965,521
4. ARRA-CER Electronic Clinical Data Infrastructure Awards Included in 13 2014 Analysis. Individual project information was obtained for analysis in this study
65
163,804,242
ARRA: American Recovery and Reinvestment Act; CER: Comparative effectiveness research; EHR: Electronic health record.
648
J. Comp. Eff. Res. (2014) 3(6)
future science group
American Recovery & Reinvestment Act-comparative effectiveness research infrastructure investments
the Medical Subject Headings (MeSH) tag for grant numbers, ‘gr,’ was performed in PubMed to identify publications tagged with the grant ID numbers. The full abstract and summary information was extracted for the identified articles indexed in PubMed. In addition, an Internet search of the project titles was conducted, and 41 websites or webpages for relevant infrastructure projects were identified and reviewed. Outreach effort
A targeted outreach effort was conducted in both 2013 and 2014 to each of the principal investigators identified for the 65 projects. An e-mail message was sent and included the list of identified publications and data resources and tools for verification. Responses to verify and provide supplementary information were provided by the majority of the projects, a 61.5% response rate (n = 41). Analysis of publications
The primary goals of the publication review were to: identify the number articles produced by each project; and identify the articles that present CER study findings versus papers that discuss infrastructure development. The following four primary categories were used to code the purpose or focus of manuscripts based on a review of the article title and abstract: • Research findings – papers that present findings from research studies; • Methods – papers that present methodological work for CER and PCOR (e.g., development, validation and/or proof-of-concept of measures, standards and statistical methods); • Infrastructure – papers that present an overview of data infrastructure, networks or informatics tools and platforms; and, • Other – papers that present work that focus on challenges and issues for CER, and commentaries about CER as a topic. Papers in this category include topics focused on stakeholder engagement, governance and privacy issues and priority setting for CER. The journal, date, keyword and notes fields from the extracted records were used to count the frequency of articles published in each journal, and the keyword MeSH tags were used to understand the key characteristics of the publications. Analysis of data resources & tools
The primary goal of the review of the project abstracts, websites and other resources provided in response to
future science group
Research Article
the outreach was to identify the number and types of data resources and tools that have been developed or enhanced by the projects. Categorization of the data resources and informatics tools was based on a framework developed by the EDM Forum to represent informatics platforms and tools as they apply to the cycle of evidence generation in a learning health system [12] , including: • Person-level data collection tools; • Data access, exchange and aggregation tools; • Population-level analytic tools; • Provider, researcher or patient-facing decision support tools; • Databases and datasets. Results Findings are categorized by number and types of publications, and number and types of data resources and tools. Publications
Across all sources, a total of 451 manuscripts published between January 2010 and May 2014 were identified (see Appendix B for the full list of citations, and the number of times the paper has been cited according to Google Scholar). The structured search string in PubMed resulted in 394 publications of the total set. The outreach effort and project website review resulted in 57 papers of the total set. Out of the 65 projects reviewed in the study, 48 projects published at least one paper, and the highest number of publications produced by a project is 88. The rate of publication increased over each year, but varied by month which may be related to the journals’ publication schedules (Figure 1) . Rates of publication peaked in the months of July 2012, January 2013, July 2013 and March 2014. The number of publications increased each year; 46 papers in 2010, 67 in 2011, 141 in 2012, 149 in 2013 and 48 between January and May in 2014. The identified set of manuscripts (n = 451) were published in a range of journals (n = 192), with 17 journals publishing at least six articles between January 2010 through May 2014 (Figure 2) . Table 2 provides the categorization of publications based on a review of the title and abstract for each paper. Publications categorized as ‘research findings’ represent nearly half of the set, followed by ‘methods’ articles and then ‘infrastructure’ and ‘other’ (Table 2) . The review of the keyword MeSH terms tagged in the publications highlights the most frequent topics,
www.futuremedicine.com
649
Research Article Segal & Holve
60
50
Publications
40
2014 30
2013 2012 2011
20
2010
December
November
October
September
Month
August
July
June
May
April
March
February
0
January
10
Figure 1. Number of published manuscripts, by month and year (January 2010–May 2014), n = 451.
and priority populations and conditions (as defined by Agency for Healthcare Research and Quality [AHRQ]). Table 3 provides the list of keyword terms that were tagged in at least 11 articles or more. Data resources & tools
A total of 141 data resources and tools were identified based on a text review of the project abstracts, websites, publications and information provided by project investigators. The numbers of the data resources and tools are provided in Table 4, and are organized by the major categories of health IT in a learning health system. The data resources and tools include data collection tools, distributed network platforms and distributed querying, patient registries, security and privacy modules, clinical decision support and QI tools, natural language processing software and algorithms, instruments and metrics for research and QI, and patient and consumer engagement tools. Verification of the results
Outreach to the principal investigators of each project was used to verify and identify additional publications and resources not found in the PubMed search, which resulted in the 57 additional papers added to the final
650
J. Comp. Eff. Res. (2014) 3(6)
set. Out of the 394 publications identified in PubMed, 211 papers were verified directly by project investigators (from 41 of the 65 projects) or by reviewing the project or grant portfolio websites. Seven papers from the initial PubMed results were flagged by project investigators as not directly connected to the respective ARRA-CER data infrastructure project and were removed from the final set. Discussion As an early view into ARRA-CER data infrastructure awards, the findings from this study demonstrate a high level of productivity across the awards not only in the number of publications but also in the amount of data resources and tools that can support further research efforts. These resources contribute to the advancement of data infrastructure to support learning health systems which has potential to serve future projects generating evidence to improve outcomes for priority populations and conditions. The publication types are diverse and respond to several objectives of the ARRA funding, as well as priorities identified for CER infrastructure [13] . Nearly half of the publications present research findings, demonstrating a response to a primary aim of
future science group
American Recovery & Reinvestment Act-comparative effectiveness research infrastructure investments
the ARRA funds to support CER and evidence generation. Out of these publications, approximately 80% of the papers were published between January 2010 and September 2013 – before the end of the grant period for the majority of the awards – which demonstrates a rapid response to produce and disseminate research results mid-stream. Based on the review of the most frequently tagged keyword MeSH terms in the publications, much of the research conducted by the ARRA-CER data infrastructure projects are responding to priority populations and conditions as defined by AHRQ, including cancer, diabetes, women’s and maternal health and substance abuse. In addition to publications, the projects have produced a number of data resources and tools that can improve the capacity to conduct research and generate evidence. Given that the rate of publication steadily increased each year (Figure 1) , we anticipate that the number of publications presenting research findings from these awards will continue to increase. Other categories of publications demonstrate responsiveness to priority topics for building data infrastructure as well as identified gaps in the literature. A quarter of the publications relate to analytic methods, and a tenth relate to CER as a topic or the conduct of CER, including the social and legal aspects of data infrastructure (e.g., ‘governance’ and ‘confidentiality’). The publications that reference or provide an overview of the data resources, tools and infrastructure for CER using electronic health data (17%) contribute to the limited pool of publications at the intersection of clinical informatics and CER [14] , including cohort
Research Article
identification, cloud computing and single-point access to research data. Dissemination vehicles for this work reflect a variety of content and clinical specialty journals, which demonstrate the multidisciplinary nature of data infrastructure and of CER to a variety of populations and conditions. The journals producing the majority of these papers (Figure 2) capture the trending content areas and disciplines that interact with CER, as well as special efforts to collectively disseminate the work of the ARRA awards. The journals Medical Care, Health Services Research and the Journal of Comparative Effectiveness Research have or plan to publish special issues focused on the ARRA-CER awards. One of these special efforts to disseminate publications from ARRA-CER data infrastructure awards include two supplements published in Medical Care in 2012 and 2013, commissioned by the Electronic Data Methods (EDM) Forum, which include 27 papers. The EDM Forum publishes papers focused on research and quality improvement (QI) activities using electronic health data in its open-access, online journal, eGEMs (Generating Evidence & Methods to improve patient outcomes), where 15 manuscripts produced by the ARRA-CER Data Infrastructure projects were published. In addition, a special June 2015 issue of Health Services Research will be dedicated to the work of eight projects funded by AHRQ, within the Enhanced State Data for Analysis and Tracking of Comparative Effectiveness Impact: Improved Clinical Content and Race-Ethnicity Data (R01) grant portfolio. A toolkit to disseminate the resources emerging from projects 31
Med. Care Egems
15 12
J. Natl Cancer Inst. 11 11
Cancer Epidemiol. Biomarkers Prev. J. Am. Med. Inform. Assoc. 9
Amia Annu. Symp. Proc. Breast Cancer Res. Treat.
9 9 9
JAMA Pharmacoepidemiol. Drug Saf. Ann. Intern. Med.
8 8
Fertil. Steril. 7 7
Acad. Radiol. Drug Alcohol Depend J. Comp. Eff. Res.
7 7
Radiology 6 6
Clin. Pharmacol. Ther. Psychol. Addict. Behav. 0
5
10
15
20
25
30
Figure 2. Journals publishing at least six papers between January 2010 and May 2014 (n = 17).
future science group
www.futuremedicine.com
651
Research Article Segal & Holve
Table 2. Distribution of papers, by paper type (n = 451). Paper type
Publications (n)
Total (%)
Infrastructure
76
16.9
Methods
111
24.6
Research findings
218
48.3
Other
46
10.2
Total
451
100.0
within this grant portfolio will be available in late 2014 [1] . Measuring the level of precision and recall of the structured search queries and information gleaned through the outreach effort indicates that the level of productivity may be higher than what we were able to capture in this paper. Through the outreach effort, only seven papers were removed from the initial set; 46% of the total set of publications found in PubMed Table 3. Most frequently tagged keywords: by topic, populations or condition of focus. Keyword
Articles (n)
Mammography†
73
Mass screening/methods/standards/ economics
35
Early detection of cancer†
28
Breast neoplasms/diagnosis Pregnancy
21
†
19
†
Neoplasms†
18
Substance-related disorders
18
†
Clinical trials as topic
16
Randomized controlled trials as topic
16
Biomedical research
15
Breast neoplasms/radiography
†
14
Neoplasm staging†
14
Confidentiality
13
Socioeconomic factors
13
BMI
12
Comorbidity
12
Gambling
12
Metformin
12
†
Prostatic neoplasms
†
12
Reinforcement (psychology)
12
Polycystic ovary syndrome
11
Agency for Healthcare Research and Quality priority population or condition, or relates to a treatment or intervention for the priority condition. †
652
J. Comp. Eff. Res. (2014) 3(6)
were verified by 63% of the projects, and 57 additional papers were added by investigators or through the review of the project or grant portfolio website. This level of precision and recall indicates a low level of false positives (3%) and that the results presented in this paper may be an undercount of the true denominator of publications emerging from the ARRA-CER data infrastructure awards. Despite the amount of information found in this literature search – and the collaborative efforts to disseminate information about the new data resources and tools, as well new research findings from studies – there were limitations that created challenges to identify and track the information. The varying journal publication frequency and indexing lags limited the ability to develop a structured search strategy to query publication databases based on the grant information in an automated fashion. A multi-method approach was required to identify relevant publications and data resources for this cohort of awards. With the rate of innovation in health technology rapidly growing, it is apparent that current methods to monitor and identify publications, data sources and tools may not be as rapid as would be ideal to monitor trends and assess impact overtime. The best source of information about a project’s resources came directly from the project teams; however, that is not a feasible or sustainable method for continued monitoring of emerging publications and resources. Further efforts to catalog and disseminate information about the work of the ARRA-CER data infrastructure awards will be important to continue to share tools, evidence and lessons that have been produced. Further analysis of project start and end dates in relation to the publication dates will inform an understanding of the time to disseminate and translate the work of the ARRACER data infrastructure awards, and where there has been success in the projects that have demonstrated a higher level of productivity. Conclusion The ARRA-CER data infrastructure projects represent a unique, substantial federal investment in building infrastructure to support CER, PCOR and QI using electronic health data. This paper demonstrates an approach to establish a defined cohort and conduct structured queries across numerous resources to identify early productivity emerging from this investment, including publications, gray literature and data resources and tools. More than 450 publications and 100 data resources were identified, however – given the rate that these products have emerged, and that the majority of the project end dates were within a year ago – it is likely that the total number of products will con-
future science group
American Recovery & Reinvestment Act-comparative effectiveness research infrastructure investments
tinue to increase as the infrastructure, tools and techniques are tested and propagated. In addition, a key aim of the data infrastructure investments was to improve the efficiency, rigor and timeliness of research, specifically for CER [15] . With this information, we can begin to understand how the new or enhanced infrastructure enables more rapid, rigorous and efficient CER. However, a barrier to achieving this aim is that current methods for monitoring and identifying publications, data sources and tools may not be efficient enough to capture trends and address this lag in dissemination and translation. Translation and dissemination are increasingly important as the landscape continues to expand as new funding opportunities that support data infrastructure emerge, including PCORnet, NIH Big Data to Knowledge (BD2K), and the funds allocated to data infrastructure from the Patient-Centered Outcomes Research Trust Fund. Although there are requirements in place to stimulate rapid dissemination of research findings and data, such as the requirement for PCORI awards to release research findings within 90 days [16] , there is still a lack of a centralized resource or method to identify and access research results in a systemized (e.g., by funding mechanism, related to a priority population or condition, or by the type of data resource of tool used in the research effort) and transparent manner (e.g., open access) [17] . To truly understand the ‘return on investment’ and impact of new and enhanced data infrastructure on the capacity to conduct research, new resources and dissemination mechanisms are needed to provide multifaceted information about the research and the infrastructure that supports CER. The EDM Forum will continue to explore and engage the community building electronic health data infrastructure, to help support further dissemination and collaboration. One activity includes an updated literature review to identify potentially new articles indexed in MEDLINE. The review will leverage a novel technique to semi-automate the process of searching relevant MeSH terms and identify complex concepts related to informatics and CER. This study demonstrates that there is already a high level of productivity emerging from the ARRA-CER data infrastructure awards, and provides insight about how the efforts have contributed to address known gaps in evidence about priority populations, conditions and clinical informatics. These results are a first step to support further analysis of citations and products to assess the time to publication from midstream and post-award dissemination efforts. Future efforts, including a co-citation analysis, can generate a dditional understanding about dissemination and translation of this work and its impact on the conduct of science as well as clinical care.
future science group
Research Article
Table 4. Number of data resources and/or tools (n = 141). Data resource or tool
Count (n)
Person-level data collection tools
27
Data access, exchange and aggregation 34 tools Population-level analytic tools
37
Provider, researcher or patient-facing decision support tools
11
Databases or datasets
32
Total
141
Future perspective This study offers an early view into the productivity of the data resources, tools, and research findings that have emerged from the significant federal investment in electronic health data infrastructure to support CER. While the results do not fully capture the depth of the resources and publications that will eventually be released in the public domain, the study provides a ‘cohort’ of infrastructure projects that can be reviewed over time to gain a clearer understanding of the ARRA investment contributions to CER. Of particular interest will be the ways in which the infrastructure cohort will mature and extend to new sites, populations, and research topics. Improving the capacity to integrate and enrich data resources and methods that enhance the efficiency, timeliness, and representativeness of CER was a key goal of the ARRA investment in data infrastructure and the current study provides a baseline metric of productivity to enrich future understanding of foundational investments to achieve this aim. Financial & competing interests disclosure Funding was provided by AHRQ Grant U18 HS022789-01 (Electronic Data Methods Forum). The authors have no other relevant affiliations or financial involvement with any organization or entity with a financial interest in or financial conflict with the subject matter or materials discussed in the manuscript apart from those disclosed. No writing assistance was utilized in the production of this manuscript.
Ethical conduct of research The authors state that they have obtained appropriate institutional review board approval or have followed the principles outlined in the Declaration of Helsinki for all human or animal experimental investigations. In addition, for investigations involving human subjects, informed consent has been obtained from the participants involved.
www.futuremedicine.com
653
Research Article Segal & Holve
Executive summary • A review of funding and resulting publications, data and informatics tools developed as a result of support from The American Recovery and Reinvestment Act of 2009 (ARRA) provides a unique opportunity, and first important step, to understand the timeline for dissemination and translation of ARRA-funded work. • This paper focuses on a subset of grants and contracts funded through the ARRA-comparative effectiveness research (CER) awards building electronic clinical data infrastructure, and provides early insight into productivity of the awards, including publications, gray literature and data resources and tools. • Project information was retrieved from federal databases, and structured search queries in PubMed and Internet searches were conducted. An outreach effort was conducted to verify and identify further publications and resources. • Publications and resources were reviewed and categorized to understand the type and amount of productivity that has emerged from the ARRA-CER data infrastructure awards. • A total of 451 manuscripts and 141 data resources and tools were identified and categorized. • The awards are addressing gaps in evidence on priority populations, conditions and the intersection of clinical informatics and CER. • The data resources and tools developed or enhanced by the projects will contribute to the advancement of data infrastructure to support learning health systems and generate evidence to improve outcomes for priority populations and conditions. • This effort provides an important first step to support further analysis regarding time to publication from midstream and postaward dissemination efforts of major infrastructure awards. The data from this and subsequent studies can generate a better understanding about dissemination and translation of this work and its impact on the conduct of science as well as clinical care.
References
5
Selby JV, Beal AC, Frank L. The Patient-Centered Outcomes Research Institute (PCORI) national priorities for research and initial research agenda. JAMA 307(15), 1583–1584 (2012).
6
Use of Recovery Act and Patient Protection and Affordable Care Act Funds for Comparative Effectiveness Research. HHS Research Awards, June 14, 2011 (GAO-11-712R). Government Printing Office, Washington, DC, USA (2011). www.gao.gov/new.items/d11712r.pdf
•
Provides a comprehensive overview of the funding amounts allocated to the ARRA-CER awards.
7
Hamilton Lopez M, Holve E, Sarkar IN, Segal C. Building the informatics infrastructure for comparative effectiveness research (CER): a review of the literature. Med. Care 50(Suppl.), S38–S48 (2012).
•
This literature review, conducted at the early stages of the ARRA-CER awards, characterizes literature at the intersection of clinical informatics and CER, and identifies cross-cutting themes and gaps in the literature.
8
Segal C, Holve E. CER infrastructure investments to evidence generation in a learning health system, EDM forum, academy health (2012). http://repository.academyhealth.org/edm_briefs/3/
9
Segal C. Emerging Data Resources, Tools, and Publications from the ARRA-CER Infrastructure Awards. AcademyHealth Annual Research Meeting (ARM), Baltimore, MD, USA, 24 June 2013.
10
Segal E, Holve E. CER infrastructure investments to evidence generation in a learning health system. AcademyHealth EDM Forum, August 2012.
11
National Institutes of Health. NIH Research Portfolio Online reporting Tools (RePORT). NIH, Bethesda, MD, USA (2012). http://projectreporter.nih.gov/reporter.cfm
Papers of special note have been highlighted as: • of interest 1
•
2
Provides a comprehensive definition and criteria for the conduct of comparative effectiveness research (CER); priority population, conditions and methods; and, a vision for funding mechanisms and priorities that informed the distribution of American Recovery and Reinvestment Act of 2009 (ARRA) funds for CER. Roundtable on Evidence-Based Medicine. Olsen LA, Aisner D, McGinnis JM (Eds). National Academies Press, Washington, DC, USA (2007).
3
IOM (Institute of Medicine). Engineering a Learning Health System: a Look at the Future – Workshop Summary. The National Academies Press, Washington, DC, USA (2011).
•
Summarizes a workshop jointly held between the IOM’s Roundtable on Value and Science-Driven Health Care and the National Academy of Engineering. The workshop identifies priorities for building a learning health system, including data infrastructure and explores lessons learned from systems and operations engineering, especially pertaining to continuous improvement principles, that could be applied to healthcare delivery.
4
654
US Department of Health and Human Services, Federal Coordinating Council for Comparative Effectiveness Research. Report to the president and the Congress. Washington, DC, USA , 30 June 2009. www.tuftsctsi.org
Agency for Healthcare Research and Quality. American Recovery and Reinvestment Act Investments in Comparative Effectiveness Research for Data Infrastructure, Rockville, MD, USA (April 2010). www.ahrq.gov/fund/cerfactsheets/osfsinfra.htm
J. Comp. Eff. Res. (2014) 3(6)
future science group
American Recovery & Reinvestment Act-comparative effectiveness research infrastructure investments
12
•
AcademyHealth EDM Forum. “Informatics Tools and Approaches To Facilitate the Use of Electronic Data for CER, PCOR, and QI: Resources Developed by the PROSPECT, DRN, and Enhanced Registry Projects” Issue Briefs and Reports, Paper 11 (2013). http://repository.academyhealth.org/edm_briefs/11 Describes 31 informatics platforms and tools that have been developed, adopted or adapted by projects from three ARRA-CER grant portfolios managed by Agency for Healthcare Research and Quality, as they apply to various dimensions in the development and execution of research and QI.
13
Institutes of Medicine (IOM). Engineering a learning health system: a look at the future – workshop summary. (2012).
14
Hamilton Lopez M, Holve E, Sarkar IN, Segal C.Building the informatics infrastructure for comparative effectiveness research: a review of the literature. Med. Care 50(Suppl.), S38–S48 (2012).
15
EDM Forum, “Building the Electronic Clinical Data Infrastructure to Improve Patient Outcomes: CER Project
future science group
Research Article
Profiles” (2012). Issue Briefs and Reports. Paper 7. http://repository.academyhealth.org/edm_briefs/7 16
Subtitle D-Patient-Centered Outcomes Research. Public Law 111–148, Statute Number: 124 STAT. 733, Section: 6301. (2010).
17
Benner JS, Morrison MR, Karnes EK, Kocot SL, McClellan M. An evaluation of recent federal spending on comparative effectiveness research: priorities, gaps, and next steps. Health Aff. (Millwood). 29(10), 1768–1776 (2010).
•
Presents findings from an analysis of ARRA-CER funds, and found that nearly 90% of the US$1.1 billion will eventually be spent on two main types of activities: developing and synthesizing comparative effectiveness evidence, and improving the capacity to conduct CER. The authors recommend that priorities for the new funding should include greater emphasis on experimental research; evaluation of reforms at the health system level; identification of effects on subgroups of patients; inclusion of understudied groups of patients; and dissemination of results.
www.futuremedicine.com
655