Research Article
chneeeffecations
Stakeholder assessment of comparative effectiveness research needs for Medicaid populations
Patients, providers and policy-makers rely heavily on comparative effectiveness research (CER) when making complex, real-world medical decisions. In particular, Medicaid providers and policy-makers face unique challenges in decision-making because their program cares for traditionally underserved populations, especially children, pregnant women and people with mental illness. Because these patient populations have generally been underrepresented in research discussions, CER questions for these groups may be understudied. To address this problem, the Agency for Healthcare Research and Quality commissioned our team to work with Medicaid Medical Directors and other stakeholders to identify relevant CER questions. Through an iterative process of topic identification and refinement, we developed relevant, feasible and actionable questions based on issues affecting Medicaid programs nationwide. We describe challenges and limitations and provide recommendations for future stakeholder engagement.
Michael A Fischer*,1, Cora Allen-Coleman1, Stephen F Farrell1 & Sebastian Schneeweiss1 1 Division of Pharmacoepidemiology & Pharmacoeconomics, Department of Medicine, Brigham & Women’s Hospital & Harvard Medical School, 1620 Tremont Street, Suite 3030, Boston, MA 02120, USA *Author for correspondence: Tel.: +1 617 278 0930 Fax: +1 617 232 8602 mfischer@ partners.org
Keywords: CER • comparative effectiveness research • Medicaid • stakeholder • underserved populations
By providing information on the relative benefits of real-world therapeutic options, comparative effectiveness research (CER) can help patients, clinicians and policy-makers make more informed decisions about medical care. Medicaid provides health coverage for low-income Americans, with combined federal and state funding. While some elements of the program are guided by federal mandates, state Medicaid officials have control of a wide range of coverage decisions, but also often face budgetary and other challenges as they work to make evidence-based choices to care for the populations they serve. New CER that addresses the real-world choices most relevant to Medicaid populations represents an important priority. Several prior initiatives have developed broad-based lists of potential CER topics. In 2005, the Agency for Healthcare Research and Quality (AHRQ) published a list of priority conditions for CER, incorporating input from a broad range of stakeholders
10.2217/cer.15.26 © 2015 Future Medicine Ltd
and ultimately prioritized by a steering committee from the Department of Health and Human Services [1] . Over the subsequent years these priorities became the focus for the AHRQ-supported Effective Health Care (EHC) program, as well as AHRQ’s DEcIDE (Developing Evidence to Inform Decisions about Effectiveness) and DEcIDE-2 Networks. In 2009, a committee convened by the Institute of Medicine (IOM) created a list of 100 priority topics for CER focused on improving the quality of healthcare at the request of Congress to help guide research efforts support by the American Recovery and Reinvestment Act of 2009. The IOM described this work as reflecting: “the insights of health professionals, consumer advocates, policy analysts and others. The committee received 1268 unique topic suggestions, which it narrowed to 100. The list reflects a range of clinical categories, populations to be studied, categories of interventions and research methodologies” [2] .
J. Comp. Eff. Res. (2015) 4(5), 465–471
part of
ISSN 2042-6305
465
Research Article Fischer, Allen-Coleman, Farrell & Schneeweiss Although this work applies to healthcare in the USA broadly, there is still a pressing need for research targeting the unique needs of the Medicaid population. Medicaid provides coverage for groups that may be underrepresented in other patient populations, including large numbers of children, pregnant women and people with mental illness. CER that addresses the specific choices confronting Medicaid programs can inform the decisions of Medicaid officials. Recognizing this need, AHRQ commissioned our team to seek input from Medicaid Medical Directors (MMDs) and to engage with a wide range of additional stakeholders to identify relevant, feasible and actionable CER topics that would address the needs of the Medicaid population. Stakeholder engagement The work described here was supported by an AHRQ task order, which specified the stages in which the process was to be conducted. In order to identify topics most relevant to the Medicaid population, the process began with areas of interest and concern identified by state MMDs and then incorporated patient, provider, policy-maker and other stakeholder input [3] . The process included six stages (Table 1) , beginning with the MMDs identifying general topic areas for potential research. Through group and individual discussions these initial insights were developed into a large group of possible research avenues. The potential topics were further refined during interviews and discussions with governmental, clinical and patient advocacy stakeholders who provided perspectives on the clinical areas identified by the MMDs. A webbased meeting was used to review the research ideas developed and determine the final list of CER topics. Topic identification with MMDs
The engagement process began in November 2012 at the fall meeting of the MMD Learning Network (MMDLN) in Washington DC with a 90-min workshop-style session moderated by the principal
investigator from Brigham and Women’s Hospital (BWH). MMDs from approximately 25 states participated in the workshop in person, with several more joining by teleconference. The workshop introduced the topic identification and stakeholder engagement process, reviewed the history of the DEcIDE program and consisted mainly of a guided discussion to elicit broad topic areas for CER that the directors felt would be most helpful to their programs. The MMDs participated enthusiastically during the session and generated an extensive list of potential CER topics relevant to the needs of the Medicaid population. Following the MMDLN workshop, the BWH team reviewed the list of topics and categorized them into six major clusters of needs for CER from the perspective of Medicaid programs. Table 2 shows the six topic clusters, with details on some of the specific elements within each cluster mentioned most often by MMDs. Individual interviews with MMDs
To explore these six-topic clusters more deeply, the BWH team conducted individual telephone interviews with MMDs from November 2012 to February 2013. Then the team divided the topics list into two general groupings: clinical topics and systems topics (Box 1) . Stakeholder group engagement
During the following 5 months, the BWH team solicited input on these priority Medicaid topics from stakeholder groups in order to explore in-depth the specific potential research questions contained in each of the clusters. Before project initiation, the BWH team proposed a list of 33 stakeholder groups that represented patients (eight), providers (17) and policy-makers (eight). Detailed interviews were conducted with 23 of these representatives from stakeholder groups (five patients, 13 providers, five policymakers) to discuss the topic areas and begin to identify more specific and actionable research questions (see Supplementary Table 1). Seven additional interviews
Table 1. Process for identification of priority topics for comparative effectiveness research. Activity
Product
1. Topic identification at Medicaid Medical Director Learning Network meeting
Six clusters of topic areas
2. Individual Medicaid Medical Directors interviews
14 possible topic areas
3. Stakeholder interviews
13 possible topic areas
4. Medicaid Medical Director web conference
25 specific topics, narrowed to ten areas
5. Stakeholder web conference
Ten research questions
6. Agency for Healthcare Research and Quality input Ten research questions, PICOT formulation and finalization PICOT: Population, intervention, comparator, outcome, timing.
466
J. Comp. Eff. Res. (2015) 4(5)
future science group
Stakeholder assessment of comparative effectiveness research needs for Medicaid populations
Research Article
Table 2. Six major clusters of need for comparative effectiveness research. Topic
Main questions
Secondary and specific concerns
Diagnostics
• What is the effectiveness of newer diagnostic tests?
• Cancer genotyping • Prenatal testing for trisomy 21 and other conditions • Pharmacogenetic testing • Genomic sequencing • Microarray testing for pathogens • Comparisons of invasive vs noninvasive prenatal screening tests
Systems of care
• What makes a primary care practice effective? • What systems provide the best care? • How can we measure quality of care delivery?
• Comparative effectiveness of Accountable Care Organizations, Patient-Centered Medical Homes and Managed Care Organizations • Effectiveness of home healthcare • Identification of benchmarks and measurement approaches for assessing quality of care
Site of care
• What care needs to be delivered in medical offices vs home or other settings? • What is the quality of care at nursing homes and long-term care facilities?
• Effectiveness of speech and physical therapy and related treatments in office vs home settings • Impact of the Patient-Centered Medical Homes on emergency room demand, clinical outcomes • What interventions can reduce long-term care admissions? • Effectiveness of Federally Qualified Health Centers compared with other primary care sites
Patientcenteredness
• How can we engage patients to improve • Relative effectiveness of patient and participation and treatment adherence family-centered treatment approaches, as for all conditions? opposed to solely physician-directed ones
Payment models
• Are changes in payment systems rewarding best approaches to care?
Priority diseases
• What are the most effective treatments • Substance abuse for these conditions that are common in • Autism Medicaid populations? • Hepatitis C • Care of developmentally disabled adults • Chronic back pain • Pediatric obesity • Fertility treatment
were conducted with MMD and provider stakeholders specifically suggested by AHRQ. Stakeholder input clarified the topics with greatest gaps in knowledge and greatest suitability for CER. These discussions were intended to narrow the list to the areas of greatest opportunity for Medicaid programs, where reducing gaps in knowledge about comparative effectiveness would have the greatest positive impact. The BWH team used these contributions to create a tentative list of 13 research areas.
future science group
• Efficacy of Federally Qualified Health Centers • Do changes in Medicare payment increase access to care? • Are specialty drugs adequately integrated into systems?
Second meeting with MMDs to prioritize topics
Using these 13 research areas as a base, the team conducted a web-based conference in July 2013 with approximately 20 members of the MMDLN to discuss and prioritize. Participants were polled on the various topics within each grouping and asked to select the most important. The participating MMDs ranked the most highly (with polling %) for the clinical topics: substance abuse (69%), autism (50%), specialty drugs (44%), imaging studies (38%); and for the sys-
www.futuremedicine.com
467
Research Article Fischer, Allen-Coleman, Farrell & Schneeweiss
Box 1. Priority topic areas identified by Medicaid Medical Directors. Clinical topics Prenatal testing Autism Substance abuse Specialty drugs† Use of diagnostic imaging Pediatric medicine Long term care Systems topics Re-admission rates Birth outcomes Patient adherence Behavioral health Medical homes Managed care † Specialty drugs included treatments for rheumatoid arthritis, multiple sclerosis and hepatitis C, as well as intravenous immunoglobulin and long-acting antipsychotic medications.
tems topics: re-admission rates (75%), medical homes (69%) and behavioral health (56%). A total of 25 possible research questions were generated during these sessions. Through subsequent internal review as well as consultation with AHRQ, ten questions were selected for more in-depth development. Decisions about which topics to select were driven by the feasibility of performing CER for that question (e.g., whether relevant data sources existed) and priorities within AHRQ. These topics areas became the focus of the stakeholder web conference during which CER topics were selected.
The BWH team then reviewed the literature to identify existing research and ongoing studies relevant to these clinical areas. This information was applied in a PICOTS (population, intervention, comparator, outcome, timing, setting) framework so that each CER topic identified could be outlined clearly to guide future research (see Supplementary Data) [4,5] . Limitations & challenges Stakeholder specialization vs topic breadth
This process successfully incorporated input from a broad range of stakeholders to produce a list of relevant CER questions for Medicaid populations. However, the team encountered challenges during the process with resulting limitations that must be kept in mind when assessing the list of topics. While input from MMDs on broad healthcare topics was generally straightforward and consistent, incorporating clinically specialized stakeholders into broad discussions was more challenging. While MMDs are concerned with a diverse range of areas – including prenatal care and birth outcomes, hepatitis C treatments, antipsychotic medication use and many more – clinically specialized stakeholder groups have a much narrower focus of interest. This type of stakeholder group is likely to have strong experience and knowledge regarding one specific area, but less investment in or knowledge of others. Therefore, bringing stakeholders with specific interests together for broad group discussion did not reliably lead to consensus on research priorities. Challenges for objective ranking
Stakeholder input
In September 2013 we conducted a day-long web conference based on these ten key questions to elicit further input and feedback from stakeholders. Stakeholder participants were asked to consider the relevance, feasibility and actionability of each research question and topic. For a topic to be considered relevant it had to address MMD and stakeholder priorities, fill gaps in current research and fit within the goals of CER/patient-centered outcomes research (PCOR). For a topic to be feasible, the data and methods needs and timeline must be reasonable. Last, a topic should be actionable; it must provide data that MMDs and other stakeholders can use in decision-making. A total of 23 people participated in the web-based workshop and discussed the specific CER questions.
Many advocacy groups focus on one healthcare issue and advocate for the issue passionately, and accordingly may not value research that does not directly address that issue of interest. Given this focus on a single segment of care or therapeutic issue, clinically specific stakeholders provided valuable insights in specific topics, but without an unbiased scoring system, were sometimes unable to compare the importance of different issues objectively. As other stakeholder engagement research groups have found, the lack of an objective rating or polling system can create difficulties in reaching consensus in group discussions [6,7] . It is important to note that this project was not designed as a pure epidemiologic study. Our process was guided by a predetermined set of steps specified in the initial task order from AHRQ, which also had input at multiple stages to narrow down the list of topics based on their priorities.
Priority finalization with AHRQ
After the stakeholder meeting, the BWH team reviewed and organized the MMD and stakeholder feedback and worked with AHRQ to develop the final list of ten potential CER topics and questions (Table 3) .
468
J. Comp. Eff. Res. (2015) 4(5)
Uneven knowledge of stakeholders in any given area
Even when bringing specialized groups together around a given healthcare topic, we found challenges
future science group
Stakeholder assessment of comparative effectiveness research needs for Medicaid populations
in bridging knowledge gaps between stakeholders. Patients, providers and researchers may all focus on one disease area, but their experiences, priorities and knowledge will vary. For instance, providers may have a strong grasp of the clinical issues in a disease area, but not understand the research challenges involved. This grouping of varied perspectives is one of the main strengths of stakeholder engagement, but in this project the range of stakeholder experience was so broad that it complicated the process of reaching consensus. Limitations of web-based meetings
The original project plan envisioned a one-day inperson stakeholder meeting conducted at the AHRQ conference center. Due to travel restrictions which affected many agencies and organizations, the decision was made to convert the 1-day meeting to a web conference format. We found the format itself to be effective: many participants had experience with webbased meetings, and there did not seem to be decreased interaction or discussion among attendees due to the format. However, retaining the single-day format may not be the optimal structure for presentation and discussion-generating sessions. Attendance varied for individual sessions, reflecting participants joining
Research Article
from different time zones, and attendance decreased later in the day, probably due to participant fatigue from many hours in web conference session. Once the in-person format was discarded, it is possible that consistent participant energy through all sessions would have been easier to maintain if the meeting was divided into shorter sessions over a series of different days. Strengths of approach Despite some challenges, we found this iterative model of input, consolidation and discussion with our stakeholder groups effective. First, the project benefited from focused and consistent input from the MMD Learning Network: the MMDs worked well together as a team during the two MMD meetings and many generously contributed time for additional individual interviews. Eager participation and input from a broad range of stakeholder groups allowed the team to gather a broad representation of experiences from researchers, providers, payers, patients and caregivers. During the stakeholder interviews and web conferences, 38 stakeholders worked together to refine research questions. Despite differences in clinical expertise, research knowledge and disease experience, the stakeholders collaborated to generate ideas, refine topics and develop research questions.
Table 3. Final ten comparative effectiveness research topics for Medicaid populations. Priority topic
Suggested research question
Multiple sclerosis treatments
What are the comparative effectiveness and tolerability of newer and older agents for the treatment of relapsing-remitting multiple sclerosis?
Hepatitis C
What is the comparative effectiveness of new hepatitis C antiviral therapies and regimens compared with each other and compared with interferon-based treatment?
Substance abuse treatment
What is the comparative effectiveness of treatment for substance abuse, in both the short and long term?
Substance abuse screening
Which screening protocol works best for identifying adolescents and adults with possible substance abuse problems, and in which settings?
Long-acting antipsychotic medications
Which long-acting antipsychotic medications are most effective for the treatment of schizophrenia and other severe mental illnesses, and how do these regimens compare with oral treatment regimens?
Use of antipsychotics in elderly
What approaches to managing the use of antipsychotics in elderly patients are the most effective?
Prenatal screening tests
What are the comparative diagnostic values of currently available prenatal screening tests?
Hospital re-admissions
What condition-specific approaches are the most effective in reducing hospital re-admissions?
Autism spectrum diagnostic approaches
What approaches for initial diagnosis and assessment for autism spectrum disorders are the most effective?
Autism therapy
Which treatment approaches provide the greatest comparative improvement in symptoms, short and long term, in children and young adults with autism spectrum disorders?
future science group
www.futuremedicine.com
469
Research Article Fischer, Allen-Coleman, Farrell & Schneeweiss Although some challenges of web conferences were noted above, the format was generally effective and efficient. Using virtual meetings reduced travel time and cost, allowing the BWH team to consult with MMDs and stakeholders more often and gather more input. It also offered participants multiple ways to participate actively through traditional voice discussion, typed question and comment submission and online polling. This allowed all participants to voice their ideas and concerns, even if they were hesitant to voice them to the whole group. In particular, the MMDs found online polling useful for collaboration as the team worked to prioritize topics. Recommendations for future stakeholder engagement in CER Timing of engagement
Throughout this process, we found that thoughtful engagement timing was crucial. To address our concerns with the efficacy of including clinically specialized stakeholder groups in broad discussions, we recommend that future groups involve MMDs and generalist stakeholder groups more heavily in the initial and final phases of topic generation. Because MMDs and generalists deal with a broad range of diseases and healthcare system issues, they are well equipped to identify crosscutting issues and are especially helpful during general brainstorming discussions. Clinically specialized groups bring expertise in their particular topic areas and may be most effectively included in interviews or small group discussion to refine specific research questions, rather than in the more general discussions.
Linking to funding
Lastly, future stakeholder engagement in research priority development could benefit from a linkage of CER priority lists to research funding announcements. This would have a threefold positive effect. First, it would underline the long-term importance of the project and encourage active engagement from all stakeholder groups. Second, the inclusion of the priority list by funding organizations would ensure that this list is disseminated broadly within the research community. Third, funding opportunities linked to the priority list would guarantee that researchers have the financial tools to pursue the stakeholder-identified topics. Conclusion By combining state MMD and stakeholder group input through a six-step engagement process, we created a list of CER priorities based on the concerns most relevant to Medicaid programs nationwide. Despite differences between the state programs, there was general agreement on priorities for Medicaid populations. The MMD’s priorities are strongly influenced by the new and challenging decisions and choices which Medicaid programs need to make. Areas with high cost, increasing incidence and/or proliferating treatments tended to have the greatest need for improved information on comparative effectiveness. Potentially owing to the growing body of information compiled over the last several years regarding pharmacologic CER, nonpharmacological interventions and health services issues were identified as areas of particular importance for new CER across the states.
Objective ranking of topics
To combat the challenges in balancing priorities of advocacy groups, we recommend that future research teams develop methods for stakeholders to rate topics objectively. This would allow groups to develop consensus more effectively and move discussions forward more efficiently. It would also bolster the credibility of the topic selection process and ensure that all stakeholder perspectives are incorporated in decision-making. Web conference tools
We highly recommend online commenting, survey and polling tools, which are often built into web conference software. We found that these tools encourage participation, especially from those stakeholders who may be uncomfortable sharing their opinions in large groups. Additionally, we recommend that future group organize meetings using a multiday format and spreading sessions over several days. Fatigue for moderators, organizers and participants in multiple sessions could be lessened by avoiding day-long meetings.
470
J. Comp. Eff. Res. (2015) 4(5)
Future perspective In addition to the identification of ten areas for which new CER could meet the needs of MMDs and Medicaid populations, this project highlighted the evolution from earlier lists of CER topics to our list, suggesting trends in the priorities for Medicaid that are likely to affect future research needs. Specifically, Medicaid officials are increasingly interested in research that evaluates delivery system approaches to care, such as community-based screening or treatment programs or managed care interventions that may cut across a range of conditions. The MMDs are especially interested in understanding whether these systems-based interventions actually have a meaningful impact on the quality of medical care. While the classical CER framework comparing two specific treatments or testing options will remain important for patients, clinicians and other decision-makers, future CER will need to assess broader-based interventions that impact many elements of the delivery system simultaneously.
future science group
Stakeholder assessment of comparative effectiveness research needs for Medicaid populations
Leveraging the perspectives and expertise of these important stakeholder groups allowed the team to develop a list of relevant, feasible and actionable research questions. Besides yielding this list, the six-step process of topic identification and refinement with stakeholders generated insights into best practices for stakeholder engagement and recommendations for future work. We believe that these tools could be used to identify and refine topics in fields beyond CER research. Supplementary data To view the supplementary data that accompany this paper please visit the journal website at: www.futuremedicine.com/ doi/full/10.2217/cer.15.26.
Financial & competing interests disclosure S Schneeweiss is principal investigator of the Harvard–Brigham Drug Safety and Risk Management Research Center funded by
Research Article
the US FDA. His work is partially funded by grants/contracts from the Patient-Centered Outcomes Research Institute, the FDA, and the National Heart, Lung, and Blood Institute. He is also a consultant to WHISCON, LLC and Aetion, Inc., a software manufacturer of which he also owns shares, and is principal investigator of investigator-initiated grants to the Brigham and Women’s Hospital from Novartis and Boehringer-Ingelheim, unrelated to the topic of this review. MA Fischer is a clinical consultant for the Alosa Foundation, an educational non-profit and has research support through his institution from CVSCaremark and Otsuka America. HHSA290200500161 TO3 was the source of funding. The authors have no other relevant affiliations or financial involvement with any organization or entity with a financial interest in or financial conflict with the subject matter or materials discussed in the manuscript apart from those disclosed. No writing assistance was utilized in the production of this manuscript.
Executive summary Background • Increased stakeholder engagement in research prioritization is critical to guide relevant, feasible and actionable comparative effectiveness research. However, there remain many unanswered questions about optimal models of stakeholder engagement. • Medicaid providers and policy-makers face unique challenges in clinical decision-making because the program cares for traditionally underserved populations.
Stakeholder engagement • In order to develop a list of priority comparative effectiveness research topics for Medicaid populations, our team gathered input from Medicaid Medical Directors and stakeholder groups through one initial in-person conference, 38 individual stakeholder interviews and three web conferences.
Limitations & challenges • For future research teams looking to develop research priority lists, we recommend thoughtful timing of engagement to include stakeholder expertise most effectively, use of web conference tools to encourage participation and consensus, objective rating systems to balance diverse perspectives and promote consensus, and linkage of a final list to funding announcements to promote dissemination and future research.
References
5
Gliklich RE, Leavy MB, Velentgas P et al. Incorporating stakeholder perspectives in developing a translation table framework for comparative effectiveness research. J. Comp. Eff. Res. 1(3), 281–292 (2012).
6
Concannon TW, Meissner P, Grunbaum JA et al. A new taxonomy for stakeholder engagement in patient-centered outcomes research. J. Gen. Intern. Med. 27(8), 985–991 (2012).
•
Provides thorough recommendations for stakeholder engagement that can be adapted by those seeking to implement similar processes.
7
Greenberg CC, Wind JK, Chang GJ, Chen RC, Schrag D. Stakeholder engagement for comparative effectiveness research in cancer care: experience of the DEcIDE Cancer Consortium. J. Comp. Eff. Res. 2(2), 117–125 (2013).
•
Methods for stakeholder engagement in research are still in development, so this thoughtful and careful use of stakeholder input to develop a list of cancer research priorities is a useful guide.
Papers of special note have been highlighted as: • of interest 1
Chalkidou K, Whicher D, Kary W, Tunis S. Comparative effectiveness research priorities: identifying critical gaps in evidence for clinical and health policy decision making. Int. J. Technol. Assess. Health Care 25(3), 241–248 (2009).
2
IOM Report Recommends 100 Initial Priorities for Research to Determine Which Health Care Approaches Work Best (2009). http://psqh.com/iom-report-recommends
3
Deverka PA, Lavallee DC, Desai PJ et al. Stakeholder participation in comparative effectiveness research: defining a framework for effective engagement. J. Comp. Eff. Res. 1(2), 181–194 (2012).
4
Riva JJ, Malik KM, Burnie SJ, Endicott AR, Busse JW. What is your research question? An introduction to the PICOT format for clinicians. J. Can. Chiropr. Assoc. 56(3), 167–171 (2012).
future science group
www.futuremedicine.com
471
chneeeffecations
Stakeholder assessment of comparative effectiveness research needs for Medicaid populations
Patients, providers and policy-makers rely heavily on comparative effectiveness research (CER) when making complex, real-world medical decisions. In particular, Medicaid providers and policy-makers face unique challenges in decision-making because their program cares for traditionally underserved populations, especially children, pregnant women and people with mental illness. Because these patient populations have generally been underrepresented in research discussions, CER questions for these groups may be understudied. To address this problem, the Agency for Healthcare Research and Quality commissioned our team to work with Medicaid Medical Directors and other stakeholders to identify relevant CER questions. Through an iterative process of topic identification and refinement, we developed relevant, feasible and actionable questions based on issues affecting Medicaid programs nationwide. We describe challenges and limitations and provide recommendations for future stakeholder engagement.
Michael A Fischer*,1, Cora Allen-Coleman1, Stephen F Farrell1 & Sebastian Schneeweiss1 1 Division of Pharmacoepidemiology & Pharmacoeconomics, Department of Medicine, Brigham & Women’s Hospital & Harvard Medical School, 1620 Tremont Street, Suite 3030, Boston, MA 02120, USA *Author for correspondence: Tel.: +1 617 278 0930 Fax: +1 617 232 8602 mfischer@ partners.org
Keywords: CER • comparative effectiveness research • Medicaid • stakeholder • underserved populations
By providing information on the relative benefits of real-world therapeutic options, comparative effectiveness research (CER) can help patients, clinicians and policy-makers make more informed decisions about medical care. Medicaid provides health coverage for low-income Americans, with combined federal and state funding. While some elements of the program are guided by federal mandates, state Medicaid officials have control of a wide range of coverage decisions, but also often face budgetary and other challenges as they work to make evidence-based choices to care for the populations they serve. New CER that addresses the real-world choices most relevant to Medicaid populations represents an important priority. Several prior initiatives have developed broad-based lists of potential CER topics. In 2005, the Agency for Healthcare Research and Quality (AHRQ) published a list of priority conditions for CER, incorporating input from a broad range of stakeholders
10.2217/cer.15.26 © 2015 Future Medicine Ltd
and ultimately prioritized by a steering committee from the Department of Health and Human Services [1] . Over the subsequent years these priorities became the focus for the AHRQ-supported Effective Health Care (EHC) program, as well as AHRQ’s DEcIDE (Developing Evidence to Inform Decisions about Effectiveness) and DEcIDE-2 Networks. In 2009, a committee convened by the Institute of Medicine (IOM) created a list of 100 priority topics for CER focused on improving the quality of healthcare at the request of Congress to help guide research efforts support by the American Recovery and Reinvestment Act of 2009. The IOM described this work as reflecting: “the insights of health professionals, consumer advocates, policy analysts and others. The committee received 1268 unique topic suggestions, which it narrowed to 100. The list reflects a range of clinical categories, populations to be studied, categories of interventions and research methodologies” [2] .
J. Comp. Eff. Res. (2015) 4(5), 465–471
part of
ISSN 2042-6305
465
Research Article Fischer, Allen-Coleman, Farrell & Schneeweiss Although this work applies to healthcare in the USA broadly, there is still a pressing need for research targeting the unique needs of the Medicaid population. Medicaid provides coverage for groups that may be underrepresented in other patient populations, including large numbers of children, pregnant women and people with mental illness. CER that addresses the specific choices confronting Medicaid programs can inform the decisions of Medicaid officials. Recognizing this need, AHRQ commissioned our team to seek input from Medicaid Medical Directors (MMDs) and to engage with a wide range of additional stakeholders to identify relevant, feasible and actionable CER topics that would address the needs of the Medicaid population. Stakeholder engagement The work described here was supported by an AHRQ task order, which specified the stages in which the process was to be conducted. In order to identify topics most relevant to the Medicaid population, the process began with areas of interest and concern identified by state MMDs and then incorporated patient, provider, policy-maker and other stakeholder input [3] . The process included six stages (Table 1) , beginning with the MMDs identifying general topic areas for potential research. Through group and individual discussions these initial insights were developed into a large group of possible research avenues. The potential topics were further refined during interviews and discussions with governmental, clinical and patient advocacy stakeholders who provided perspectives on the clinical areas identified by the MMDs. A webbased meeting was used to review the research ideas developed and determine the final list of CER topics. Topic identification with MMDs
The engagement process began in November 2012 at the fall meeting of the MMD Learning Network (MMDLN) in Washington DC with a 90-min workshop-style session moderated by the principal
investigator from Brigham and Women’s Hospital (BWH). MMDs from approximately 25 states participated in the workshop in person, with several more joining by teleconference. The workshop introduced the topic identification and stakeholder engagement process, reviewed the history of the DEcIDE program and consisted mainly of a guided discussion to elicit broad topic areas for CER that the directors felt would be most helpful to their programs. The MMDs participated enthusiastically during the session and generated an extensive list of potential CER topics relevant to the needs of the Medicaid population. Following the MMDLN workshop, the BWH team reviewed the list of topics and categorized them into six major clusters of needs for CER from the perspective of Medicaid programs. Table 2 shows the six topic clusters, with details on some of the specific elements within each cluster mentioned most often by MMDs. Individual interviews with MMDs
To explore these six-topic clusters more deeply, the BWH team conducted individual telephone interviews with MMDs from November 2012 to February 2013. Then the team divided the topics list into two general groupings: clinical topics and systems topics (Box 1) . Stakeholder group engagement
During the following 5 months, the BWH team solicited input on these priority Medicaid topics from stakeholder groups in order to explore in-depth the specific potential research questions contained in each of the clusters. Before project initiation, the BWH team proposed a list of 33 stakeholder groups that represented patients (eight), providers (17) and policy-makers (eight). Detailed interviews were conducted with 23 of these representatives from stakeholder groups (five patients, 13 providers, five policymakers) to discuss the topic areas and begin to identify more specific and actionable research questions (see Supplementary Table 1). Seven additional interviews
Table 1. Process for identification of priority topics for comparative effectiveness research. Activity
Product
1. Topic identification at Medicaid Medical Director Learning Network meeting
Six clusters of topic areas
2. Individual Medicaid Medical Directors interviews
14 possible topic areas
3. Stakeholder interviews
13 possible topic areas
4. Medicaid Medical Director web conference
25 specific topics, narrowed to ten areas
5. Stakeholder web conference
Ten research questions
6. Agency for Healthcare Research and Quality input Ten research questions, PICOT formulation and finalization PICOT: Population, intervention, comparator, outcome, timing.
466
J. Comp. Eff. Res. (2015) 4(5)
future science group
Stakeholder assessment of comparative effectiveness research needs for Medicaid populations
Research Article
Table 2. Six major clusters of need for comparative effectiveness research. Topic
Main questions
Secondary and specific concerns
Diagnostics
• What is the effectiveness of newer diagnostic tests?
• Cancer genotyping • Prenatal testing for trisomy 21 and other conditions • Pharmacogenetic testing • Genomic sequencing • Microarray testing for pathogens • Comparisons of invasive vs noninvasive prenatal screening tests
Systems of care
• What makes a primary care practice effective? • What systems provide the best care? • How can we measure quality of care delivery?
• Comparative effectiveness of Accountable Care Organizations, Patient-Centered Medical Homes and Managed Care Organizations • Effectiveness of home healthcare • Identification of benchmarks and measurement approaches for assessing quality of care
Site of care
• What care needs to be delivered in medical offices vs home or other settings? • What is the quality of care at nursing homes and long-term care facilities?
• Effectiveness of speech and physical therapy and related treatments in office vs home settings • Impact of the Patient-Centered Medical Homes on emergency room demand, clinical outcomes • What interventions can reduce long-term care admissions? • Effectiveness of Federally Qualified Health Centers compared with other primary care sites
Patientcenteredness
• How can we engage patients to improve • Relative effectiveness of patient and participation and treatment adherence family-centered treatment approaches, as for all conditions? opposed to solely physician-directed ones
Payment models
• Are changes in payment systems rewarding best approaches to care?
Priority diseases
• What are the most effective treatments • Substance abuse for these conditions that are common in • Autism Medicaid populations? • Hepatitis C • Care of developmentally disabled adults • Chronic back pain • Pediatric obesity • Fertility treatment
were conducted with MMD and provider stakeholders specifically suggested by AHRQ. Stakeholder input clarified the topics with greatest gaps in knowledge and greatest suitability for CER. These discussions were intended to narrow the list to the areas of greatest opportunity for Medicaid programs, where reducing gaps in knowledge about comparative effectiveness would have the greatest positive impact. The BWH team used these contributions to create a tentative list of 13 research areas.
future science group
• Efficacy of Federally Qualified Health Centers • Do changes in Medicare payment increase access to care? • Are specialty drugs adequately integrated into systems?
Second meeting with MMDs to prioritize topics
Using these 13 research areas as a base, the team conducted a web-based conference in July 2013 with approximately 20 members of the MMDLN to discuss and prioritize. Participants were polled on the various topics within each grouping and asked to select the most important. The participating MMDs ranked the most highly (with polling %) for the clinical topics: substance abuse (69%), autism (50%), specialty drugs (44%), imaging studies (38%); and for the sys-
www.futuremedicine.com
467
Research Article Fischer, Allen-Coleman, Farrell & Schneeweiss
Box 1. Priority topic areas identified by Medicaid Medical Directors. Clinical topics Prenatal testing Autism Substance abuse Specialty drugs† Use of diagnostic imaging Pediatric medicine Long term care Systems topics Re-admission rates Birth outcomes Patient adherence Behavioral health Medical homes Managed care † Specialty drugs included treatments for rheumatoid arthritis, multiple sclerosis and hepatitis C, as well as intravenous immunoglobulin and long-acting antipsychotic medications.
tems topics: re-admission rates (75%), medical homes (69%) and behavioral health (56%). A total of 25 possible research questions were generated during these sessions. Through subsequent internal review as well as consultation with AHRQ, ten questions were selected for more in-depth development. Decisions about which topics to select were driven by the feasibility of performing CER for that question (e.g., whether relevant data sources existed) and priorities within AHRQ. These topics areas became the focus of the stakeholder web conference during which CER topics were selected.
The BWH team then reviewed the literature to identify existing research and ongoing studies relevant to these clinical areas. This information was applied in a PICOTS (population, intervention, comparator, outcome, timing, setting) framework so that each CER topic identified could be outlined clearly to guide future research (see Supplementary Data) [4,5] . Limitations & challenges Stakeholder specialization vs topic breadth
This process successfully incorporated input from a broad range of stakeholders to produce a list of relevant CER questions for Medicaid populations. However, the team encountered challenges during the process with resulting limitations that must be kept in mind when assessing the list of topics. While input from MMDs on broad healthcare topics was generally straightforward and consistent, incorporating clinically specialized stakeholders into broad discussions was more challenging. While MMDs are concerned with a diverse range of areas – including prenatal care and birth outcomes, hepatitis C treatments, antipsychotic medication use and many more – clinically specialized stakeholder groups have a much narrower focus of interest. This type of stakeholder group is likely to have strong experience and knowledge regarding one specific area, but less investment in or knowledge of others. Therefore, bringing stakeholders with specific interests together for broad group discussion did not reliably lead to consensus on research priorities. Challenges for objective ranking
Stakeholder input
In September 2013 we conducted a day-long web conference based on these ten key questions to elicit further input and feedback from stakeholders. Stakeholder participants were asked to consider the relevance, feasibility and actionability of each research question and topic. For a topic to be considered relevant it had to address MMD and stakeholder priorities, fill gaps in current research and fit within the goals of CER/patient-centered outcomes research (PCOR). For a topic to be feasible, the data and methods needs and timeline must be reasonable. Last, a topic should be actionable; it must provide data that MMDs and other stakeholders can use in decision-making. A total of 23 people participated in the web-based workshop and discussed the specific CER questions.
Many advocacy groups focus on one healthcare issue and advocate for the issue passionately, and accordingly may not value research that does not directly address that issue of interest. Given this focus on a single segment of care or therapeutic issue, clinically specific stakeholders provided valuable insights in specific topics, but without an unbiased scoring system, were sometimes unable to compare the importance of different issues objectively. As other stakeholder engagement research groups have found, the lack of an objective rating or polling system can create difficulties in reaching consensus in group discussions [6,7] . It is important to note that this project was not designed as a pure epidemiologic study. Our process was guided by a predetermined set of steps specified in the initial task order from AHRQ, which also had input at multiple stages to narrow down the list of topics based on their priorities.
Priority finalization with AHRQ
After the stakeholder meeting, the BWH team reviewed and organized the MMD and stakeholder feedback and worked with AHRQ to develop the final list of ten potential CER topics and questions (Table 3) .
468
J. Comp. Eff. Res. (2015) 4(5)
Uneven knowledge of stakeholders in any given area
Even when bringing specialized groups together around a given healthcare topic, we found challenges
future science group
Stakeholder assessment of comparative effectiveness research needs for Medicaid populations
in bridging knowledge gaps between stakeholders. Patients, providers and researchers may all focus on one disease area, but their experiences, priorities and knowledge will vary. For instance, providers may have a strong grasp of the clinical issues in a disease area, but not understand the research challenges involved. This grouping of varied perspectives is one of the main strengths of stakeholder engagement, but in this project the range of stakeholder experience was so broad that it complicated the process of reaching consensus. Limitations of web-based meetings
The original project plan envisioned a one-day inperson stakeholder meeting conducted at the AHRQ conference center. Due to travel restrictions which affected many agencies and organizations, the decision was made to convert the 1-day meeting to a web conference format. We found the format itself to be effective: many participants had experience with webbased meetings, and there did not seem to be decreased interaction or discussion among attendees due to the format. However, retaining the single-day format may not be the optimal structure for presentation and discussion-generating sessions. Attendance varied for individual sessions, reflecting participants joining
Research Article
from different time zones, and attendance decreased later in the day, probably due to participant fatigue from many hours in web conference session. Once the in-person format was discarded, it is possible that consistent participant energy through all sessions would have been easier to maintain if the meeting was divided into shorter sessions over a series of different days. Strengths of approach Despite some challenges, we found this iterative model of input, consolidation and discussion with our stakeholder groups effective. First, the project benefited from focused and consistent input from the MMD Learning Network: the MMDs worked well together as a team during the two MMD meetings and many generously contributed time for additional individual interviews. Eager participation and input from a broad range of stakeholder groups allowed the team to gather a broad representation of experiences from researchers, providers, payers, patients and caregivers. During the stakeholder interviews and web conferences, 38 stakeholders worked together to refine research questions. Despite differences in clinical expertise, research knowledge and disease experience, the stakeholders collaborated to generate ideas, refine topics and develop research questions.
Table 3. Final ten comparative effectiveness research topics for Medicaid populations. Priority topic
Suggested research question
Multiple sclerosis treatments
What are the comparative effectiveness and tolerability of newer and older agents for the treatment of relapsing-remitting multiple sclerosis?
Hepatitis C
What is the comparative effectiveness of new hepatitis C antiviral therapies and regimens compared with each other and compared with interferon-based treatment?
Substance abuse treatment
What is the comparative effectiveness of treatment for substance abuse, in both the short and long term?
Substance abuse screening
Which screening protocol works best for identifying adolescents and adults with possible substance abuse problems, and in which settings?
Long-acting antipsychotic medications
Which long-acting antipsychotic medications are most effective for the treatment of schizophrenia and other severe mental illnesses, and how do these regimens compare with oral treatment regimens?
Use of antipsychotics in elderly
What approaches to managing the use of antipsychotics in elderly patients are the most effective?
Prenatal screening tests
What are the comparative diagnostic values of currently available prenatal screening tests?
Hospital re-admissions
What condition-specific approaches are the most effective in reducing hospital re-admissions?
Autism spectrum diagnostic approaches
What approaches for initial diagnosis and assessment for autism spectrum disorders are the most effective?
Autism therapy
Which treatment approaches provide the greatest comparative improvement in symptoms, short and long term, in children and young adults with autism spectrum disorders?
future science group
www.futuremedicine.com
469
Research Article Fischer, Allen-Coleman, Farrell & Schneeweiss Although some challenges of web conferences were noted above, the format was generally effective and efficient. Using virtual meetings reduced travel time and cost, allowing the BWH team to consult with MMDs and stakeholders more often and gather more input. It also offered participants multiple ways to participate actively through traditional voice discussion, typed question and comment submission and online polling. This allowed all participants to voice their ideas and concerns, even if they were hesitant to voice them to the whole group. In particular, the MMDs found online polling useful for collaboration as the team worked to prioritize topics. Recommendations for future stakeholder engagement in CER Timing of engagement
Throughout this process, we found that thoughtful engagement timing was crucial. To address our concerns with the efficacy of including clinically specialized stakeholder groups in broad discussions, we recommend that future groups involve MMDs and generalist stakeholder groups more heavily in the initial and final phases of topic generation. Because MMDs and generalists deal with a broad range of diseases and healthcare system issues, they are well equipped to identify crosscutting issues and are especially helpful during general brainstorming discussions. Clinically specialized groups bring expertise in their particular topic areas and may be most effectively included in interviews or small group discussion to refine specific research questions, rather than in the more general discussions.
Linking to funding
Lastly, future stakeholder engagement in research priority development could benefit from a linkage of CER priority lists to research funding announcements. This would have a threefold positive effect. First, it would underline the long-term importance of the project and encourage active engagement from all stakeholder groups. Second, the inclusion of the priority list by funding organizations would ensure that this list is disseminated broadly within the research community. Third, funding opportunities linked to the priority list would guarantee that researchers have the financial tools to pursue the stakeholder-identified topics. Conclusion By combining state MMD and stakeholder group input through a six-step engagement process, we created a list of CER priorities based on the concerns most relevant to Medicaid programs nationwide. Despite differences between the state programs, there was general agreement on priorities for Medicaid populations. The MMD’s priorities are strongly influenced by the new and challenging decisions and choices which Medicaid programs need to make. Areas with high cost, increasing incidence and/or proliferating treatments tended to have the greatest need for improved information on comparative effectiveness. Potentially owing to the growing body of information compiled over the last several years regarding pharmacologic CER, nonpharmacological interventions and health services issues were identified as areas of particular importance for new CER across the states.
Objective ranking of topics
To combat the challenges in balancing priorities of advocacy groups, we recommend that future research teams develop methods for stakeholders to rate topics objectively. This would allow groups to develop consensus more effectively and move discussions forward more efficiently. It would also bolster the credibility of the topic selection process and ensure that all stakeholder perspectives are incorporated in decision-making. Web conference tools
We highly recommend online commenting, survey and polling tools, which are often built into web conference software. We found that these tools encourage participation, especially from those stakeholders who may be uncomfortable sharing their opinions in large groups. Additionally, we recommend that future group organize meetings using a multiday format and spreading sessions over several days. Fatigue for moderators, organizers and participants in multiple sessions could be lessened by avoiding day-long meetings.
470
J. Comp. Eff. Res. (2015) 4(5)
Future perspective In addition to the identification of ten areas for which new CER could meet the needs of MMDs and Medicaid populations, this project highlighted the evolution from earlier lists of CER topics to our list, suggesting trends in the priorities for Medicaid that are likely to affect future research needs. Specifically, Medicaid officials are increasingly interested in research that evaluates delivery system approaches to care, such as community-based screening or treatment programs or managed care interventions that may cut across a range of conditions. The MMDs are especially interested in understanding whether these systems-based interventions actually have a meaningful impact on the quality of medical care. While the classical CER framework comparing two specific treatments or testing options will remain important for patients, clinicians and other decision-makers, future CER will need to assess broader-based interventions that impact many elements of the delivery system simultaneously.
future science group
Stakeholder assessment of comparative effectiveness research needs for Medicaid populations
Leveraging the perspectives and expertise of these important stakeholder groups allowed the team to develop a list of relevant, feasible and actionable research questions. Besides yielding this list, the six-step process of topic identification and refinement with stakeholders generated insights into best practices for stakeholder engagement and recommendations for future work. We believe that these tools could be used to identify and refine topics in fields beyond CER research. Supplementary data To view the supplementary data that accompany this paper please visit the journal website at: www.futuremedicine.com/ doi/full/10.2217/cer.15.26.
Financial & competing interests disclosure S Schneeweiss is principal investigator of the Harvard–Brigham Drug Safety and Risk Management Research Center funded by
Research Article
the US FDA. His work is partially funded by grants/contracts from the Patient-Centered Outcomes Research Institute, the FDA, and the National Heart, Lung, and Blood Institute. He is also a consultant to WHISCON, LLC and Aetion, Inc., a software manufacturer of which he also owns shares, and is principal investigator of investigator-initiated grants to the Brigham and Women’s Hospital from Novartis and Boehringer-Ingelheim, unrelated to the topic of this review. MA Fischer is a clinical consultant for the Alosa Foundation, an educational non-profit and has research support through his institution from CVSCaremark and Otsuka America. HHSA290200500161 TO3 was the source of funding. The authors have no other relevant affiliations or financial involvement with any organization or entity with a financial interest in or financial conflict with the subject matter or materials discussed in the manuscript apart from those disclosed. No writing assistance was utilized in the production of this manuscript.
Executive summary Background • Increased stakeholder engagement in research prioritization is critical to guide relevant, feasible and actionable comparative effectiveness research. However, there remain many unanswered questions about optimal models of stakeholder engagement. • Medicaid providers and policy-makers face unique challenges in clinical decision-making because the program cares for traditionally underserved populations.
Stakeholder engagement • In order to develop a list of priority comparative effectiveness research topics for Medicaid populations, our team gathered input from Medicaid Medical Directors and stakeholder groups through one initial in-person conference, 38 individual stakeholder interviews and three web conferences.
Limitations & challenges • For future research teams looking to develop research priority lists, we recommend thoughtful timing of engagement to include stakeholder expertise most effectively, use of web conference tools to encourage participation and consensus, objective rating systems to balance diverse perspectives and promote consensus, and linkage of a final list to funding announcements to promote dissemination and future research.
References
5
Gliklich RE, Leavy MB, Velentgas P et al. Incorporating stakeholder perspectives in developing a translation table framework for comparative effectiveness research. J. Comp. Eff. Res. 1(3), 281–292 (2012).
6
Concannon TW, Meissner P, Grunbaum JA et al. A new taxonomy for stakeholder engagement in patient-centered outcomes research. J. Gen. Intern. Med. 27(8), 985–991 (2012).
•
Provides thorough recommendations for stakeholder engagement that can be adapted by those seeking to implement similar processes.
7
Greenberg CC, Wind JK, Chang GJ, Chen RC, Schrag D. Stakeholder engagement for comparative effectiveness research in cancer care: experience of the DEcIDE Cancer Consortium. J. Comp. Eff. Res. 2(2), 117–125 (2013).
•
Methods for stakeholder engagement in research are still in development, so this thoughtful and careful use of stakeholder input to develop a list of cancer research priorities is a useful guide.
Papers of special note have been highlighted as: • of interest 1
Chalkidou K, Whicher D, Kary W, Tunis S. Comparative effectiveness research priorities: identifying critical gaps in evidence for clinical and health policy decision making. Int. J. Technol. Assess. Health Care 25(3), 241–248 (2009).
2
IOM Report Recommends 100 Initial Priorities for Research to Determine Which Health Care Approaches Work Best (2009). http://psqh.com/iom-report-recommends
3
Deverka PA, Lavallee DC, Desai PJ et al. Stakeholder participation in comparative effectiveness research: defining a framework for effective engagement. J. Comp. Eff. Res. 1(2), 181–194 (2012).
4
Riva JJ, Malik KM, Burnie SJ, Endicott AR, Busse JW. What is your research question? An introduction to the PICOT format for clinicians. J. Can. Chiropr. Assoc. 56(3), 167–171 (2012).
future science group
www.futuremedicine.com
471
