Heart & Lung 43 (2014) 453e461

Contents lists available at ScienceDirect

Heart & Lung journal homepage: www.heartandlung.org

Research Methods

Methods for knowledge synthesis: An overview Robin Whittemore, PhD, APRN, FAAN, Ariana Chao, MSN, RN, FNP-BC, Myoungock Jang, MSN, RN *, Karl E. Minges, MPH, Chorong Park, MSN, RN Yale University School of Nursing, 400 West Campus Drive, Orange, CT 06477, USA

a r t i c l e i n f o

a b s t r a c t

Article history: Received 13 March 2014 Received in revised form 2 May 2014 Accepted 2 May 2014 Available online 8 July 2014

Objectives: To highlight recent advances in knowledge synthesis methods and reporting guidelines. Background: Knowledge synthesis is critical to advancing practice, research, and policy, but synthesizing knowledge from an often-heterogenous body of literature is challenging. Methods: A review of knowledge synthesis methods and reporting guidelines for health sciences research was completed using Google Scholar, Medline, CINAHL, and PsycInfo. Relevant information was critiqued and summarized for applicability to health science and practice. Results: Recent advances and guidelines pertaining to systematic reviews, meta-analysis, qualitative synthesis, mixed studies reviews, integrative reviews, scoping reviews, RE-AIM reviews, and umbrella reviews are discussed and examples of the application of each method to cardiopulmonary research are provided. Methods of quality appraisal are also presented. Conclusions: Advancements in knowledge synthesis and reporting guidelines enhance the quality, scope, and applicability of results; thus improving health science and clinical practice, and advancing health policy. Ó 2014 Elsevier Inc. All rights reserved.

Keywords: Systematic review Meta-analysis Metasynthesis Mixed studies reviews Integrative review

Introduction While knowledge synthesis e a summary of all pertinent studies on a specific question or topic1 e is critical to advancing practice, research, and policy, synthesizing knowledge from a heterogenous body of literature in a clear and accurate manner can be challenging. Searching the literature for all relevant studies, assessing quality of studies, and synthesizing heterogenous data are some of the methodological challenges researchers experience when undertaking a systematic review of the literature. Established knowledge synthesis methods include the systematic review, meta-analysis, qualitative synthesis, and integrative review. More recently, methods on mixed-studies reviews, scoping reviews, reviews that examine efficacy and generalizability (i.e., RE-AIM review), and reviews of systematic reviews (i.e., umbrella reviews) have been proposed. These methods and guidelines for reporting results from reviews, as well as individual studies, continue to be revised to enhance the rigor of knowledge synthesis methods and the applicability and transparency of findings to practice and policy. Clinicians and scientists benefit from information about the different types of synthesis methods and current resources on reporting guidelines. This knowledge helps to

* Corresponding author. Tel.: þ1 203 785 2389. E-mail address: [email protected] (M. Jang). 0147-9563/$ e see front matter Ó 2014 Elsevier Inc. All rights reserved. http://dx.doi.org/10.1016/j.hrtlng.2014.05.014

enhance appropriate and accurate reporting of individual studies, to determine an appropriate review method when undertaking a review of the literature, to consider quality and bias of studies, and to interpret reviews.2 Thus, the purpose of this manuscript is to highlight recent advances in knowledge synthesis methods and reporting guidelines for reviews and individual studies. Searches in Google Scholar, Medline, CINAHL, and PsycInfo were undertaken to identify publications and websites on knowledge synthesis methodology and reporting guidelines for reviews and individual studies. The reference lists of relevant publications and websites were also reviewed for additional information. An overview of each knowledge synthesis method, guidelines for knowledge synthesis methods, and examples from cardiopulmonary research are presented. In addition, guidelines on the reporting of individual studies and systematic reviews are discussed, as well as opportunities for further methodological development of knowledge synthesis techniques.

Overview of knowledge synthesis methodology All knowledge synthesis methods must include a systematic and auditable approach to assure that individual studies have trustworthy findings and that the synthesized findings accurately represent the aggregation or synthesis of individual studies.3 Common to all knowledge synthesis methods are: a) an explicit

454

R. Whittemore et al. / Heart & Lung 43 (2014) 453e461

aim; b) development of a methodological protocol; c) comprehensive search strategies to find relevant research articles; d) a method or tradition of evaluating quality and potential risk of bias in individual studies; and e) explicit data collection and synthesis procedures.3e5 Presentation of the results varies depending on the specific method and the underlying assumptions of the method. Gough and colleagues2 have articulated a differentiation in review methods that either use aggregative logic or configuring logic, which subsequently affects the presentation of results. Reviews that include homogenous empirical data in order to determine average effect sizes or numerical counting are based in aggregative logic (e.g., some systematic reviews, meta-analyses), and results are typically presented as numerical data and figures. In contrast, reviews that are more exploratory and seek variation and complexity of phenomenon are based on configuring logic (e.g., qualitative synthesis, mixed-studies review) with results presented as a conceptual model or narrative text.

statement,22 and it provides an explicit and systematic approach that researchers may use to conduct a review and summarize findings.19 The PRISMA Statement comprises a 27-item checklist and a four-phase flow diagram.21 The report emphasizes the need to assess risk of bias in individual studies and risk of bias across studies that may affect the cumulative evidence.19 An example of a systematic review in heart failure is included in Table 3. The purpose of this review was to evaluate different types of treatment for depression in adults with heart failure.6 Six different types of treatment were evaluated with some demonstrating moderate to strong evidence of efficacy and others with inconclusive or inconsistent results. This review begins to highlight the types of treatment that may be beneficial to prescribe in clinical practice as well as the types of treatment that need more research.

Guidelines for knowledge synthesis methods

Meta-analysis is a type of systematic review that combines evidence using statistical methods to determine overall effects and magnitude of effect size.23 This technique can be used to answer questions about the convergence, divergence, and robustness of study results, statistical significance of an effect with increased power, development of a more precise estimate of effects, and examination of moderators of effects (i.e., whether they are consistent across subgroups and conditions).24,25 The objective of these studies is to present a balanced and impartial summary of existing research, and numerous strategies have been designed to increase the quality of findings. There are a variety of unique features and considerations to think about when conducting a meta-analysis to help ensure the quality of findings. First, it is suggested that researchers assign quality scores to each individual study included in the analysis. Though numerous tools have been developed,26,27 the most well-established quality appraisal tool is by the Cochrane Collaboration, which recommends assessing the risk of bias of each individual study.24 Second, to assess the impact of low quality studies and sample size, it is recommended to include a sensitivity analysis.28,29 Techniques have also been developed for calculating effect sizes including odds ratios and risk ratios, and assessing publication bias using funnel plots29,30 and Egger’s regression tests.25,31 Researchers can use forest plots to provide a pictorial presentation of the results and can use Cochrane’s Q to test the heterogeneity of the studies and to guide analysis decisions (e.g., using weighted averages or a fixed or random effects model).32,33 Metaregression can be utilized to further explore these heterogenous factors.34 Despite the numerous choices researchers have when conducting meta-analyses, guidelines are essential to help researchers and clinicians conduct a high quality meta-analysis and assess the comprehensiveness and completeness of the review. Guidelines have been established for meta-analyses of both randomized controlled trials and observational studies. Researchers, reviewers, and editors often use these guidelines as standards for publication. The guideline to improve the reporting of meta-analyses of randomized controlled trials is the same guideline used for systematic reviews, PRISMA, as previously discussed. The Meta-Analysis of Observational Studies in Epidemiology (MOOSE) checklist has been created for those conducting a metaanalysis of observational studies.35 An example of a meta-analysis in the heart failure literature is provided in Table 3. The purpose of this review was to evaluate the benefits and harms of cardiac resynchronization therapy (CRT) in patients with advanced heart failure and those with less symptomatic disease.8 As can be seen by this example, the study design, the interventions, and the outcomes were very homogenous. A more focused research question allows for the determination of an

Methodological guidelines for systematic reviews, metaanalysis, qualitative synthesis, and mixed-studies reviews are available. Guidelines specify the type of study design required for that type of review, search strategies and data collection methods, quality appraisal tools, data extraction and analysis procedures, and reporting recommendations. Guidelines for systematic reviews and meta-analyses are well developed and provide detailed information and checklists to conduct a rigorous review and the necessary information to include in the publication of a review method. In contrast, guidelines for qualitative syntheses, mixed studies reviews, and umbrella reviews are emerging and provide direction to conduct the review and synthesize text, however there is no consensus on the methodological approach at this time. One of the major challenges for all review methods is the appraisal of the quality of studies included in the review. Assessment of the quality of individual studies is difficult and potentially biased, and with the exception of the meta-analysis, it is unclear how to factor in the quality of an individual study into the review analysis. Nonetheless, quality standards for all review methods have been proposed. Resources for reporting guidelines and review methods are provided in Tables 1 and 2 and examples of different types of reviews conducted on heart failure are included in Table 3. Each of the review methods will be briefly discussed. Systematic review A systematic review is a method used to combine evidence of multiple studies by identifying relevant research, appraising study quality, and summarizing findings.17,18 It utilizes explicit, systematic, and transparent methods to minimize bias and provide reliable findings and evidence.5 Systematic reviews can be conducted using a statistical analysis (if study designs and outcomes are homogenous) or a narrative analysis. Regardless of analytic approach, synthesizing the results of individual studies that includes interventions with multiple components or different measures poses additional challenges. Poor quality and inconsistent reporting of the systematic review may diminish their usefulness to make scientific inferences and conclusions.19,20 To enhance the standardization of reporting in systematic reviews, the Preferred Reporting Items for Systematic Reviews and Meta-Analyses (PRISMA) has been recommended to improve the transparency of reporting systematic reviews and enhance consistency across systematic reviews.21 The PRISMA statement evolved from the Quality of Reporting Meta-analyses (QUOROM)

Meta-analysis

R. Whittemore et al. / Heart & Lung 43 (2014) 453e461

455

Table 1 Guidelines for knowledge synthesis methods and reporting guidelines. Review type

Purpose

Systematic review

To synthesize the results of homogenous experimental OR non-experimental research using a narrative analysis

Meta-analysis of RCT

Meta-analysis of observational study Qualitative synthesis Mixed studies review

Examples of reporting guideline

4-Phase flow diagram and 27-item checklist. Sections include: Title, abstract, introduction, methods, results, & discussion To synthesize the effect of Cochrane Handbook for Handbook with sections that include: homogenous Systematic Background, method, results, conclusion, interventions evaluated by RCT using Reviews of Interventions and discussion4-Phase flow diagram established statistical procedures PRISMA and 27-item checklist. Sections include: Abstract, introduction, methods, results, & discussion Meta-analysis of observational 35-Item checklist sections include: To synthesize the results of background, search strategy, methods, studies in epidemiology homogenous observational results, discussion & conclusion (MOOSE) studies using established statistical procedures 21-Item checklist that includes: Enhancing transparency To synthesize the results of Introduction, methods, literature in reporting the synthesis of qualitative qualitative research (ENTREQ) search, appraisal, and synthesis) research using a narrative analysis None N/A To synthesize the results of experimental and non-experimental research using a statistical approach and/or a narrative analysis with integration of results Preferred Reporting Items for Systematic Reviews and Meta-analyses (PRISMA)

Integrative review To synthesize the results of research None or theory using a narrative analysis None Scoping review To map the key concepts and evidence of a particular phenomenon RE-AIM review To evaluate and synthesize the reach, RE-AIM framework efficacy, adoption, implementation, and maintenance of interventions Umbrella review

Content of the guideline

To summarize results from systematic reviews on a topic

None

effect size of a specific intervention on a select outcome when applied to a specific patient population. Guidelines for the use of CRT in patients with different grades of heart failure could be specified.

Qualitative synthesis The purposes of conducting a synthesis of primary qualitative studies include to: develop a deeper understanding of the phenomenon under investigation and uncover previously hidden patterns, generate theory, explore the barriers, facilitators, perceptions, and attitudes from the participants’ perspective, identify emerging areas of practice and research, and reveal reasons for variation in complex interventions and clinical trials.36e38 The number of published syntheses of qualitative studies continues to increase,39,40 and there is diversity and disagreement as to the number of approaches one may take to conduct a qualitative synthesis, ranging from four to nine synthesis methods that span both interpretative and integrative paradigms.41e43 Ultimately, the choice of which method to employ varies on several factors, including the question and purpose of the synthesis, number of published studies, and knowledge and expertise of the team conducting the synthesis,42 as well as the researcher’s epistemological stance, methodology of primary qualitative studies, and whether one seeks to describe or synthesize results.41 While no method to synthesizing qualitative research has become well-established, the

Quality appraisal Risk of bias in individual studies. Risk of bias across studies that may affect the cumulative evidence

N/A

Risk of bias in individual studies (sequence generation, allocation concealment, blinding participants, personnel and outcome assessors, incomplete outcome data, selective outcome reporting, other sources of bias) Assessment of study quality (blinding of quality assessors; stratification or regression on possible predictors of study results) No well-established criteria although numerous guidelines are available No well-established criteria although scoring system for mixed studies reviews proposed (qualitative, quantitative experimental, quantitative observational, and mixed methods criteria) No well-established criteria

N/A

No well-established criteria

21-Item validated extraction tool. Sections include: Reach, efficacy, adoption, implementation, and maintenance N/A

No well-established criteria

No well-established criteria

more popular methods of meta-ethnography, metasynthesis, and meta-grounded theory will be briefly discussed. The meta-ethnography has been the most widely cited qualitative synthesis method and comprises an inductive and interpretive approach to qualitative data analysis.12,44 It involves theoretical sampling and reciprocal translations of findings from each primary study into the overall synthesis, which can then be presented textually and graphically via summary tables, models, or diagrams.45 The meta-ethnography often results in new insights to a phenomenon, including the generation of new research questions and theories, or indicates a point where saturation has been reached. The metasynthesis is another frequently used method to synthesize qualitative studies. While there is a lack of consensus about the technique’s methods,46 and its somewhat ambiguous and interchangeable use,38,47 the umbrella term we are referring to here is the qualitative metasynthesis method as provided by Sandelowski and Barroso.48 Qualitative metasynthesis is a systematic approach that integrates results from a number of different, but inter-related qualitative studies to synthesize research findings and contribute to a new higher conceptual and contextual understanding of the phenomenon of interest.46,48 Meta-grounded theory methodology, also known as “grounded formal theory,” involves an inductive approach that uses constant comparative and theoretical sampling to analyze primary grounded theory studies. It often informs higher-level insight into the processes of a phenomenon and the development of new theories.47,49,50

456

R. Whittemore et al. / Heart & Lung 43 (2014) 453e461

Table 2 Resources for synthesis methodology. Review type

Resources for method

Systematic review

Handbook.cochrane.org84 PRISMA21 (http://www.prisma-statement.org) handbook.cochrane.org84 handbook.cochrane.org84

Meta-analysis of RCT Meta-analysis of observational study Qualitative synthesis

Mixed studies review

Integrative review Scoping review

RE-AIM review

Umbrella review

Barnett-Page and Thomas,41 2009 Centre for Reviews and Dissemination,95 2009 Dixon-Woods et al,44 2007 Noblit and Hare,12 1988 Sandelowski and Barroso,48 2006 Kearney,50 1998 Heyvaert et al,96 2013 Pluye et al,67 2009 Sandelowski et al,97 2006 Whittemore and Knafl,15 2005 Arksey and O’Malley,74 2005 Armstrong et al,98 2011 Levac et al,72 2010 http://hlwiki.slais.ubc.ca/index.php /Scoping_reviews99 http://www.re-aim.org/index.html100 http://www.re-aim.org/resources_and_tools /measures/literaturecoding.pdf101 http://joannabriggs.org/assets/docs/sumari/ ReviewersManual-Methodology-JBI_Umbrella %20Reviews-2014.pdf69 handbook.cochrane.org84

Although searching for primary qualitative studies can be a difficult task, several search strategies exist to locate and retrieve qualitative studies. These approaches use database filters, and maximizing sensitivity and specificity algorithms to develop optimal search strategies for key databases.51e54 Identification of inclusion and exclusion criteria is contingent upon the qualitative synthesis method that is selected. Exclusion on the basis of quality is also a consideration with some qualitative techniques. Indeed, the decision to apply quality appraisal guidelines to primary qualitative studies is one of contention.40,55e57 While many guidelines and criteria exist for assessing quality,10,44,58e60 none have reached consensus as the preferred approach. Further, it has not yet been determined how to incorporate quality scores into reporting of qualitative synthesis techniques. While the debate to include quality appraisal guidelines applied to primary qualitative studies continues, some strategies may be suggested to enhance the quality of qualitative syntheses. Much in the same way that researchers conducting primary qualitative studies achieve methodological rigor through trustworthiness (credibility, transferability, dependability, and confirmability),61 researchers conducting a qualitative synthesis should report transparency in the methods used to synthesize primary studies. One example of a reporting tool that incorporates transparency is entitled “Enhancing Transparency in Reporting the Synthesis of Qualitative Research (ENTREQ).”62 This guideline uses a 21-item checklist that entails the appraisal of key aspects of the synthesis, including introduction, methods, and literature search. While several other guidelines exist for the reporting of qualitative syntheses reviews,62e65 key considerations encompassed in most quality appraisal guides include an audit trail, member checking, and transparency of methodology. Other comprehensive guides to manage, appraise, analyze, and synthesize qualitative research are available.41,63,66 An example of a metasynthesis in the heart failure literature is provided in Table 3. The purpose of this study was to synthesize results of qualitative studies on how older people live with chronic

heart failure.9 By combining the results of several qualitative studies, an understanding of the physical and psychosocial challenges unique to the older adult are identified which has important clinical and research implications. Improved clinical assessment and/or intervention development could result from this type of review. Mixed studies review Mixed studies review is a literature review that concurrently examines qualitative, quantitative and/or mixed methods primary studies.67 The term mixed studies review is often used interchangeably with integrative review, mixed methods review, mixed methods (research) synthesis, or synthesis of results of quantitative and qualitative studies.67 Pluye and colleagues67 have suggested the term ‘mixed studies review’ be used for reviews that include quantitative, qualitative, and/or mixed methods study designs. A mixed studies review aims to address the same overlapping or complementary review questions3 by combining the strengths of both qualitative and quantitative studies. Thus, a mixed studies review can provide robust insight to a complex phenomenon67 as well as development of an intervention by answering “what is it about this kind of intervention that works, for whom, in what circumstances, in what respects, and why.”68 However, there is a lack of consensus for guidelines to plan, design, and assess mixed studies reviews. Depending on the review question(s), a mixed studies review can be conducted with a diverse combination of different types of primary studies (qualitative, quantitative-observational, quantitative-experimental, and mixed method studies) and with varied ways of integrating those findings. Due to the nature of mixed studies review, it is challenging to develop a well-specified guideline or criteria for appraising the methodological quality of a mixed studies review. The Joanna Briggs Institute (JBI), a non-profit organization focused on evidence based practice, has developed a manual that contains guidelines about methods to conduct mixed methods reviews.69 The manual also contains guidelines of essential components to include when reporting results from a mixed studies review. To our knowledge, only one guideline for evaluating quality of primary research, which is unique to mixed studies reviews has been developed thus far.67 The scoring system is straightforward and simple (having a total of 15 quality criteria within four types of primary studies), however this quality appraisal tool is limited because it mainly focuses on randomized controlled studies and its criteria were not derived from usual quality appraisal standards for systematic reviews such as QUOROM and MOOSE. More work is required in developing a systematic guideline and quality appraisal criteria for mixed studies reviews. An example of a mixed studies review in the heart failure literature is provided in Table 3. The purpose of this study was to describe caregivers’ activities contributing to self-care in adults with heart failure.11 The quantitative data were used to highlight the operationalization of the caregiver activities and the relationship between the caregiver activities and patient outcomes. The qualitative data highlighted the caregiver activities and varying responses to the role. The combination of quantitative and qualitative data provides complementary and in-depth perceptions of caregivers of adults with heart failure. Because this mixed studies review focused on descriptive data, results have the potential to inform clinical assessment and/or intervention development. Integrative review Integrative reviews are a category of knowledge synthesis methods that include a broad approach and sampling frame that can include empirical or theoretical literature, or both, depending

Table 3 Application of knowledge synthesis methods to heart failure (HF). Study aim

Method/reporting guidelinesNumber of publications inclusion Criteria

Main findings

Quality appraisal

Clinical implications

Systematic review Woltz et al,6 2012

To evaluate the effects of interventions on depression in adults with HF.

PRISMA N ¼ 23 Experimental or quasiexperimental designs with HF patients (NYHA class II and III HF) and outcome of depression

Six types of interventions: (1) selective serotonin reuptake inhibitors; (2) an erythropoiesis-stimulating agent; (3) exercise; (4) disease management programs; (5) complementary and alternative medicine (CAM); and (6) a multimodal intervention of cognitive behavioral therapy and exercise.

Qualitative assessmentTool for Quantitative Studies (QATQS)7

Evidence supports that pharmacology, CAM, and exercise may improve depression. Insufficient/inconsistent evidence to support the development of clinical guidelines for the treatment of depression in persons with HF.

Meta-analysis Al-Majed et al,8 2011

To assess the benefits and harms of cardiac resynchronization therapy (CRT) in patients with advanced HF and those with less symptomatic disease.

PRISMA (not explicitly stated in manuscript but required by journal) N ¼ 25 RCTs with >25 HF patients and left ventricular ejection fraction (LVEF) .40; compared CRT with inactive pacing, right ventricular pacing alone, left ventricular pacing alone, implantable cardioverter-defibrillator alone, or usual care; reported all-cause mortality, heart failure rehospitalization, change in LVEF, or functional outcomes.

In patients with milder symptoms of heart failure (class I and II), CRT improved LVEF and reduced all-cause mortality and HF hospitalization without improving functional outcomes or quality of life; in NYHA class III or IV, CRT improved functional outcomes and reduced both all-cause mortality and HF hospitalizations

Cochrane Risk of Bias Tool7

CRT is beneficial for patients with reduced LVEF, symptoms of HF, and prolonged QRS, regardless of NYHA class Evidence to support expansion of indications for CRT to less symptomatic patients with HF who have LVEF 120 ms and are in sinus rhythm but results for NYHA class I symptoms and asymptomatic left ventricular dysfunction is inconclusive

Qualitative synthesis Yu et al,9 2008

To conduct a systematic review of qualitative studies of how older people live with chronic HF.

Method not reported. Metasynthesis inferred. N ¼ 14 Studies were included if they focused on older patients’ experiences of living with chronic HF.

Participants conceptualized HF as a debilitating and distressing condition that had many detrimental physical and psychosocial sequelae. Most viewed effective self-care as the key factor that ensured successful management of HF. Yet, some older patients who were unable to relate to the acute and chronic symptoms of HF described having distressing symptoms, compromised physical functioning, feelings of powerlessness and hopelessness, and social and role dysfunction behaviors. Males interpreted the disease as a devitalizing condition whereas female patients focused more on psychological adjustment.

Standards for assessing quality (Popay, Rogers, and Williams,10 1998)

Patients need to make sense of the experience, an essential step to initiate successful adjustment. Interventions to address feelings of powerlessness and hopelessness may be important to promote disease acceptance and adjustment. Consider gender differences in the way older patients with HF conceive the illness experience and design interventions accordingly.

Mixed studies review Buck et al,11 2014

To describe the caregivers’ activities contributing to self-care in patients with HF

PRISMA and meta-ethnography (Noblit and Hare,12 1988) N ¼ 40 (qualitative N ¼ 17, quantitative N ¼ 23) Quantitative or qualitative studies of informal caregivers of adults with HF

Caregivers’ activities included: 1) measurement/quantitative skills (weighing the patient or blood pressure monitoring), 2) system navigators (scheduling appointments, equipment, services), and 3) interpersonal skills (encouraging/facilitating exercise). Few studies (N ¼ 2) linked caregivers’ contribution with patient outcomes. Caregivers’ experience was described as burdensome in 7 studies and positive in 5 studies.

Critical Appraisal Skills Program checklist (CASP)13

HF patients and caregivers should be educated and supported by clinicians and researchers as a unit of care.

R. Whittemore et al. / Heart & Lung 43 (2014) 453e461

Method Author, year

(continued on next page)

457

Clinical implications

Nurse-led transitional care programs have the potential to improve hospital readmission rates and quality of life in adults with HF. Multidisciplinary programs with inpatient and homecare follow-up within 72 h were most successful. Dose and length of follow-up may need to be based on individual characteristics, such as acuity.

Quality appraisal

Smith and Stullenbarger,16 1991

Main findings

Interventions were multi-faceted and diverse with respect to strategies, dose, and length of follow-up. Strategies included: Education, counseling, medication titration, progressive activities, and dietary, physical therapy or occupational therapy consult. Improvements in hospital readmission rates (n ¼ 7), quality of life (n ¼ 5), and cost savings (n ¼ 3) were reported. Integrative review (Whittemore and Knafl,15 2005) N ¼ 20 Transitional care intervention and a control group of usual care

R. Whittemore et al. / Heart & Lung 43 (2014) 453e461

Method/reporting guidelinesNumber of publications inclusion Criteria

458

on the purpose of the review.15 Integrative reviews can focus on methodology (e.g., evaluating the conceptual and operational definitions of a concept), theory (e.g., evaluating different behavior change theories), and/or research.70 Due to the methodological development of knowledge synthesis methods, the integrative review method is best suited for synthesizing knowledge on primary research combined with methodological and/or theoretical manuscripts. If all of the articles in a review are primary research, then a systematic review, meta-analysis, or mixed studies review method should be used. Methodological guidelines for the integrative review have been proposed15; however quality appraisal is challenging, particularly with the inclusion of theoretical and empirical sources. Evaluating the quality of primary sources needs to be considered and one possible method would be to use the approach of historical research, examining authenticity, methodological quality, informational value, and representativeness of primary sources.71 An example of an integrative review in the heart failure literature is provided in Table 3. The purpose of this study was to examine the effects of nurse-led transitional care interventions for adults with heart failure on various outcomes.14 Some interventions improved hospital readmission rates and patient quality of life; however, all interventions were multi-faceted and complex and it was difficult for the authors to ascertain the optimal combination of strategies to improve outcomes. As can be seen, this integrative review could also be classified as a systematic review as all manuscripts included in the review were quantitative research evaluating the efficacy of an intervention. As greater attention is paid to the developing methodology of different types of reviews, the distinguishing features of each method and the use of terminology should become more consistent. Scoping review Scoping reviews have recently been proposed as a knowledge synthesis method to summarize a range of evidence in order to understand broadly what is known about a phenomenon.72 Scoping reviews “aim to rapidly identify the key concepts underpinning a research area and the main sources and types of evidence available, and can be undertaken as stand alone projects in their own right, especially where an area is complex or has not been reviewed comprehensively before.”73 Scoping reviews are differentiated from systematic reviews in their broad approach to a topic, purposive sampling frame, and identification of gaps in the literature.74 Methodological guidelines have been proposed72 and quality is generally not appraised due to the variety of primary sources that can be used in the scoping review. A scoping review about heart failure was unable to be located.

Study aim

To critically examine the effects of nurse-led transitional care interventions for adults with HF on hospital readmissions, quality of life and cost-effectiveness

Method Author, year

Integrative review Stamp et al,14 2014

Table 3 (continued )

RE-AIM review In response to concerns regarding the need to assess the public health impact of interventions and translation of research to practice, Glasgow and colleagues75e77 have undertaken extensive work to establish guidelines that promote and evaluate the reach, efficacy or effectiveness, adoption, implementation, and maintenance (RE-AIM Framework) of interventions. This framework has recently been extended to knowledge synthesis methods78e80 through the use of a 23-item data extraction tool.81 This tool can be used to code and synthesize data from individual studies across the five RE-AIM dimensions.81 The result of this type of review provides evidence not only on the efficacy of an intervention, but also the generalizability of the intervention. An RE-AIM review about heart failure was unable to be located; however the method has been applied to the evaluation of physical activity and health

R. Whittemore et al. / Heart & Lung 43 (2014) 453e461

literacy interventions.78,80 These reviews report not only on the efficacy of the interventions but also the processes that could influence implementation.

459

peer reviewers, and reporting guideline developers. In addition, the network also organizes training courses, workshops, and meetings (http://www.equator-network.org/home/).92

Umbrella review Discussion Driven by the rapid increase in the number of systematic reviews and need for timely translation of evidence to practice and policy, the need for and utility of umbrella reviews has recently become apparent. Umbrella reviews are also known as metareviews and overviews of reviews, and are reviews of existing systemic reviews. Umbrella reviews focus on breadth with the aim of summarizing available evidence. They can be used to assess similarities and differences in published reviews to summarize what is known about a topic69,82,83 and typically involve a number of the different types of syntheses. Guidelines for conducting umbrella reviews and reporting the results are available from JBI69 and Cochrane (referred to as overviews of reviews).84 Within these types of reviews, researchers often use the assessment of multiple systematic reviews (AMSTAR), a validated 11-item measurement tool, to assess the methodological quality of the systematic reviews85; however, there are no clearly established guidelines. An umbrella review about heart failure was unable to be located; however the method has been applied to summarize the results from 22 studies on a diverse array of factors related to overweight and obesity.86 Guidelines for reporting of individual studies Accurate reporting of individual studies is essential to improve the quality of any knowledge synthesis method. A lack of standardization on reporting of data in individual studies can make quality appraisal difficult when conducting a review and has the potential to contribute to missing data. Thus, guidelines for reporting of individual studies continue to be developed and revised. Many journals require the adherence to specific reporting guidelines in order for a research manuscript to be considered for publication. A guideline for reporting results of a randomized control trial (RCT) was one of the first to be developed with the Consolidated Standards of Reporting Trials (CONSORT). The CONSORT guidelines specify the minimum criteria for reporting randomized trials in a 25-item checklist, which includes recommendations to diagram participant flow through the trial.87 The CONSORT standards have gained extensive support by editorial groups and are required by most journals for publication of a research report on an RCT. The improvement in standardization of reporting trails that resulted from the use of the CONSORT guidelines has resulted in additional initiatives to improve the reporting of specific types of trials. For example, there are guidelines for the reporting of cluster randomized trials and eHealth interventions.88,89 Guidelines have also been developed for different types of research. For example, guidelines for reporting of observational studies (STROBE),90 reporting for non-randomized behavioral interventions (TREND),91 and reporting qualitative research have been developed (COREQ).65 Since the development of the CONSORT guidelines in 1993, over 200 different reporting guidelines according to the type of study/research design have been proposed. As a result of this burgeoning information that is critical to scientists worldwide, the Catalogue of Reporting Guidelines for Health Research e EQUATOR Network e was established. This repository was specifically established to improve the use of guidelines in the reporting of health research. The network provides easily accessible resources related to health research reporting for authors, journal editors,

As evidence continues to burgeon and accumulate, knowledge synthesis has become an invaluable component of research, practice, and policy. Similar to the research process that scientists use when conducting and reporting results from an individual study, it is necessary to apply the same scientific integrity and principles when conducting a knowledge synthesis: the trustworthiness, accuracy, and validity of conclusions rely on this. The importance of knowledge synthesis and standards is reflected in the numerous methods and guidelines for knowledge synthesis and reporting that have been created. Guidelines for methods and reporting continue to develop to keep pace with increasingly sophisticated and diverse primary study designs. While guidelines for systematic reviews and metaanalyses are well-established, more recent developments have occurred for qualitative, mixed methods and scoping reviews. There are also numerous methods for qualitative synthesis and the similarities and differences of these approaches have not been well specified. Mixed methods have recently been established and there is less consensus on standards for conducting and reporting these types of reviews. In this regard, this paper has identified several areas for future methodological development of knowledge synthesis techniques. While researchers should first consider the underlying assumptions of each type of review, opportunities exist to address methodological approaches and reporting guidelines, as well as quality appraisal for qualitative synthesis, mixed studies review, integrative review, and scoping reviews. As several of these methods are new (e.g., mixed studies review, scoping review, umbrella reviews), the advanced development of each type of review will ultimately improve the quality, scope, and applicability of the method to synthesis future health care research. Of note, many of these standards and guidelines are primarily focused on internal validity and only recently has there been discussion of external validity in regards to knowledge synthesis with the development of the RE-AIM guidelines for systematic reviews.81 Given the research to practice gap,93 explicit attention is needed to assess and synthesize the external validity of interventions. Without such work, the ability for researchers, clinicians, and policymakers to transfer and generalize results from intervention studies is diminished. Ultimately, the validity and quality of knowledge synthesis studies relies on the quality of the individual studies included: good reporting of individual studies and knowledge synthesis methods are inextricably linked. There has been an implosion of guidelines for reporting the results of individual studies (e.g., CONSORT Statement, RE-AIM). While these reporting guidelines are often onerous and present challenges to authors and editors managing journal page constraints, they have increased the standardization of reporting study results and often help to ensure that crucial information is available for knowledge synthesis reviews, as well as critiquing and interpreting the primary study results. To increase the transparency and quality of different types of reviews, as well as avoid the duplication of effort, an international initiative has been recently undertaken to promote the registration of systematic reviews. The International Prospective Register of Systematic Reviews was developed in February 2011, and as of February 2014 a total of 1704 systematic review protocols were registered.94 This organization uses the terminology of ‘systematic

460

R. Whittemore et al. / Heart & Lung 43 (2014) 453e461

review’ to mean any type of review, with the exclusion of integrative and scoping reviews. Discussion is also occurring in funding agencies about the need for some type of knowledge synthesis review to be completed before any new research is conducted. For example, some research funding agencies in the United Kingdom have made this a requirement for funding.3 Thus, knowledge synthesis reports will not only be important to inform practice and policy, but also new research. Conclusion Advancements in methodology of knowledge synthesis and reporting guidelines have the potential to improve science and the translation of evidence into practice and policy. This manuscript has provided an overview of these methods. Knowledge synthesis methods and guidelines are evolving and becoming more specific both in content and design; however, their usefulness for scientists performing, assessing, and interpreting results from a knowledge synthesis cannot be understated. Rigorous knowledge synthesis methods enhance the quality, scope, and applicability of results; thus improving clinical care and advancing health research and policy. References

21.

22.

23. 24.

25. 26. 27.

28.

29.

30. 31. 32. 33.

1. Kastner M, Tricco AC, Soobiah C, et al. What is the most appropriate knowledge synthesis method to conduct a review? Protocol for a scoping review. BMC Med Res Methodol. 2012;12:1e10. 2. Gough D, Thomas J, Oliver S. Clarifying differences between review designs and methods. Syst Rev. 2012;1(1):1e9. 3. Harden A, Thomas J. Methodological issues in combining diverse study types in systematic reviews. Int J Soc Res Methodol. 2005;8(3):257e271. 4. Khan KS, Kunz R, Kleijnen J, Antes G. Five steps to conducting a systematic review. J R Soc Med. 2003;96(3):118e121. 5. Cook DJ, Mulrow CD, Haynes RB. Systematic reviews: synthesis of best evidence for clinical decisions. Ann Intern Med. 1997;126(5):376e380. 6. Woltz PC, Chapa DW, Friedmann E, Son H, Akintade B, Thomas SA. Effects of interventions on depression in heart failure: a systematic review. Heart Lung J Acute Crit Care. 2012;41(5):469e483. 7. Effective Public Health Practice Project (EPHPP). Quality Assessment Tool for Quantitative Studies. http://www.ephpp.ca/tools.html; Updated 2009. Accessed 20.04.14. 8. Al-Majed NS, McAlister FA, Bakal JA, Ezekowitz JA. Meta-analysis: cardiac resynchronization therapy for patients with less symptomatic heart failure. Ann Intern Med. 2011;154(6):401e412. 9. Yu DSF, Lee DTF, Kwong ANT, Thompson DR, Woo J. Living with chronic heart failure: a review of qualitative studies of older people. J Adv Nurs. 2008;61(5): 474e483. 10. Popay J, Rogers A, Williams G. Rationale and standards for the systematic review of qualitative literature in health services research. Qual Health Res. 1998;8(3):341e351. 11. Buck HG, Harkness K, Wion R, et al. Caregivers’ contributions to heart failure self-care: a systematic review. Eur J Cardiovasc Nurs; 2014:1e11. 12. Noblit GW, Hare RD. Meta-ethnography: Synthesizing Qualitative Studies, vol. 11. SAGE; 1988. 13. Critical Appraisal Skills Programme (CASP). Making Sense of Evidence. http:// www.casp-uk.net/find-appraise-act/appraising-the-evidence; Updated 2013. Accessed 20.04.14. 14. Stamp KD, Machado MA, Allen NA. Transitional care programs improve outcomes for heart failure patients: an integrative review. J Cardiovasc Nurs. 2014;29(2):140e154. 15. Whittemore R, Knafl K. The integrative review: updated methodology. J Adv Nurs. 2005;52(5):546e553. 16. Smith MC, Stullenbarger E. A prototype for integrative review and metaanalysis of nursing research. J Adv Nurs. 1991;16(11):1272e1283. 17. Crowther M, Lim W, Crowther MA. Systematic review and meta-analysis methodology. Blood. 2010;116(17):3140e3146. 18. Smith V, Devane D, Begley CM, Clarke M. Methodology in conducting a systematic review of systematic reviews of healthcare interventions. BMC Med Res Methodol. 2011;11:1e6. 19. Liberati A, Altman DG, Tetzlaff J, et al. The PRISMA statement for reporting systematic reviews and meta-analyses of studies that evaluate health care interventions: explanation and elaboration. Ann Intern Med. 2009;151(4):W65eW-94. 20. Moja LP, Telaro E, D’Amico R, Moschetti I, Coe L, Liberati A. Assessment of methodological quality of primary studies by systematic reviews: results of

34.

35.

36. 37.

38.

39. 40. 41. 42. 43. 44. 45.

46. 47. 48. 49. 50. 51.

52.

53.

the metaquality study cross sectional study. BMJ. 2005;330(7499): 1053e1055. Moher D, Liberati A, Tetzlaff J, et al. Preferred reporting items for systematic reviews and meta-analyses: the PRISMA statement. PLoS Med. 2009;6(7): e1000097. Moher D, Cook DJ, Eastwood S, Olkin I, Rennie D, Stroup DF. Improving the quality of reports of meta-analyses of randomised controlled trials: the QUOROM statement. Lancet. 1999;354(9193):1896e1900. Glass GV. Primary, secondary, and meta-analysis of research. Educ Res. 1976;5(10):3e8. The Cochrane Collaboration. Module 9: Assessing the Quality of Studies. Collecting data from relevant studies Web site. http://www.cochrane-net.org/ openlearning/html/mod7-6.htm; Updated 2012. Accessed 03.03.14. Egger M, Smith GD. Meta-analysis. Potentials and promise. BMJ. 1997;315(7119):1371e1374. Chalmers TC, Smith Jr H, Blackburn B, et al. A method for assessing the quality of a randomized control trial. Control Clin Trials. 1981;2(1):31e49. Jadad AR, Moore RA, Carroll D, et al. Assessing the quality of reports of randomized clinical trials: is blinding necessary? Control Clin Trials. 1996; 17(1):1e12. Kjaergard LL, Villumsen J, Gluud C. Correction: reported methodologic quality and discrepancies between large and small randomized trials in metaanalyses (annals of internal medicine (2001) 135 (982-989)). Ann Intern Med. 2008;149(3):219. Sterne JAC, Egger M, Smith GD. Systematic reviews in health care: investigating and dealing with publication and other biases in meta-analysis. BMJ. 2001;323(7304):101e105. Sterne JAC, Egger M. Funnel plots for detecting bias in meta-analysis: guidelines on choice of axis. J Clin Epidemiol. 2001;54(10):1046e1055. Thornton A, Lee P. Publication bias in meta-analysis: Its causes and consequences. J Clin Epidemiol. 2000;53(2):207e216. Sutton AJ, Higgins JPT. Recent developments in meta-analysis. Stat Med. 2008;27(5):625e650. Higgins JPT, Thompson SG, Deeks JJ, Altman DG. Measuring inconsistency in meta-analyses. BMJ. 2003;327(7414):557e560. Deeks JJ, Altman DG, Bradburn MJ. Statistical methods for examining heterogeneity and combining results from several studies in meta-analysis. In: Egger M, Davey Smith G, Altman DG, eds. Systematic Reviews in Healthcare: Meta-analysis in Context. 2nd ed. London, England: BMJ Books; 2001: 285e312. Stroup DF, Berlin JA, Morton SC, et al. Meta-analysis of observational studies in epidemiology: a proposal for reporting. J Am Med Assoc. 2000;283(15): 2008e2012. Candy B, King M, Jones L, Oliver S. Using qualitative synthesis to explore heterogeneity of complex interventions. BMC Med Res Methodol. 2011;11. Popay J. Moving Beyond Effectiveness in Evidence Synthesis: Methodological Issues in the Synthesis of Diverse Sources of Evidence. London, UK: National Institute for Health and Clinical Excellence; 2006. Ring N, Ritchie K, Mandava L, Jepson R. A guide to synthesising qualitative research for researchers undertaking health technology assessments and systematic reviews. http://www.nhshealthquality.org/nhsqis/8837.html; Updated 2010. Accessed 07.02.14. Jones ML. Application of systematic review methods to qualitative research: practical issues. J Adv Nurs. 2004;48(3):271e278. Mays N, Pope C. Qualitative research in health care: assessing quality in qualitative research. BMJ. 2000;320(7226):50e52. Barnett-Page E, Thomas J. Methods for the synthesis of qualitative research: a critical review. BMC Med Res Methodol. 2009;9(1):1e11. Centre for Reviews and Dissemination. Systematic Reviews: CRD’s Guidance for Undertaking Reviews in Health Care. York, UK: University of York; 2009. Finlayson KW, Dixon A. Qualitative meta-synthesis: a guide for the novice. Nurse Res. 2008;15(2):59e71. Dixon-Woods M, Booth A, Sutton AJ. Synthesizing qualitative research: a review of published reports. Qual Res. 2007;7(3):375e421. Campbell R, Pound P, Morgan M, et al. Evaluating meta-ethnography: systematic analysis and synthesis of qualitative research. Health Technol Assess. 2011;15(43):1e164. Walsh D, Downe S. Meta-synthesis method for qualitative research: a literature review. J Adv Nurs. 2005;50(2):204e211. Paterson BL, Canam C. Meta-study of Qualitative Health Research: A Practical Guide to Meta-analysis and Meta-synthesis, vol. 3. SAGE; 2001. Sandelowski M, Barroso J. Handbook for Synthesizing Qualitative Research. Springer Publishing Company; 2006. Finfgeld DL. Metasynthesis: the state of the art e so far. Qual Health Res. 2003;13(7):893e904. Kearney MH. Focus on qualitative methods: ready-to-wear: discovering grounded formal theory. Res Nurs Health. 1998;21(2):179e186. McKibbon KA, Wilczynski NL, Haynes RB. Developing optimal search strategies for retrieving qualitative studies in PsycINFO. Eval Health Prof. 2006;29(4):440e454. Walters LA, Wilczynski NL, Haynes RB. Developing optimal search strategies for retrieving clinically relevant qualitative studies in EMBASE. Qual Health Res. 2006;16(1):162e168. Wilczynski NL, Marks S, Haynes RB. Search strategies for identifying qualitative studies in CINAHL. Qual Health Res. 2007;17(5):705e710.

R. Whittemore et al. / Heart & Lung 43 (2014) 453e461 54. Wong SS, Wilczynski NL, Haynes RB. Developing optimal search strategies for detecting clinically relevant qualitative studies in MEDLINE. Stud Health Technol Inform. 2004;107:311e314. 55. Atkins S, Lewin S, Smith H, Engel M, Fretheim A, Volmink J. Conducting a meta-ethnography of qualitative literature: lessons learnt. BMC Med Res Methodol. 2008;8. 56. Barbour RS. Checklists for improving rigour in qualitative research: a case of the tail wagging the dog? BMJ. 2001;322(7294):1115e1117. 57. Sandelowski M. Rigor or rigor mortis: the problem of rigor in qualitative research revisited. ANS Adv Nurs Sci. 1993;16(2):1e8. 58. Cochrane Qualitative Research Methods Group. Tools to assist qualitative reviewers. http://www.joannabriggs.edu.au/cqrmg/tools_3.html; Updated 2014. Accessed 07.02.14. 59. Noyes J, Popay J, Pearson A, Hannes K. 20 qualitative research and cochrane reviews. In: Cochrane Handbook for Systematic Reviews of Interventions; 2008:571. 60. Whittemore R, Chase SK, Mandle CL. Validity in qualitative research. Qual Health Res. 2001;11(4):522e537. 61. Lincoln YS, Guba EG. Naturalistic Inquiry, vol. 75. SAGE Publications, Incorporated; 1985. 62. Tong A, Flemming K, McInnes E, Oliver S, Craig J. Enhancing transparency in reporting the synthesis of qualitative research: ENTREQ. BMC Med Res Methodol. 2012;12. 63. Hannes K, Lockwood C, Pearson A. A comparative analysis of three online appraisal instruments’ ability to assess validity in qualitative research. Qual Health Res. 2010;20(12):1736e1743. 64. Higgins JP, Green S. Cochrane Handbook for Systematic Reviews of Interventions: Cochrane Book Series; 2008:1e649. 65. Tong A, Sainsbury P, Craig J. Consolidated criteria for reporting qualitative research (COREQ): a 32-item checklist for interviews and focus groups. Int J Qual Health Care. 2007;19(6):349e357. 66. Joanna Briggs Institute. SUMARI: The Joanna Briggs Institute System for the Unified Management, Assessment and Review of Information. http://www .joannabriggs.edu.au/services/sumari.php; Updated 2014. Accessed 06.02.14. 67. Pluye P, Gagnon M, Griffiths F, Johnson-Lafleur J. A scoring system for appraising mixed methods research, and concomitantly appraising qualitative, quantitative and mixed methods primary studies in mixed studies reviews. Int J Nurs Stud. 2009;46(4):529e546. 68. Pawson R, Greenhalgh T, Harvey G, Walshe K. Realist review e a new method of systematic review designed for complex policy interventions. J Health Serv Res Policy. 2005;10(suppl 1):21e34. 69. The Joanna Briggs Institute. Joanna Briggs Institute Reviewers’ Manual: 2014 Edition/Supplement. Australia: The Joanna Briggs Institute; 2014. 70. Whittemore R. Combining evidence in nursing research: methods and implications. Nurs Res. 2005;54(1):56e62. 71. Kirkevold M. Integrative nursing research e an important strategy to further the development of nursing science and nursing practice. J Adv Nurs. 1997;25(5):977e984. 72. Levac D, Colquhoun H, O’Brien KK. Scoping studies: advancing the methodology. Implement Sci. 2010;5(1). 73. Mays N, Roberts E, Popay J. Synthesising research evidence. In: Fulop N, Allen P, Clarke A, Black N, eds. Studying the Organisation and Delivery of Health Services: Research Methods. London: Routledge; 2001:188e220. 74. Arksey H, O’Malley L. Scoping studies: towards a methodological framework. Int J Soc Res Methodol. 2005;8(1):19e32. 75. Glasgow RE, Vogt TM, Boles SM. Evaluating the public health impact of health promotion interventions: the RE-AIM framework. Am J Public Health. 1999;89(9):1322e1327. 76. Glasgow RE, McKay HG, Piette JD, Reynolds KD. The RE-AIM framework for evaluating interventions: what can it tell us about approaches to chronic illness management? Patient Educ Couns. 2001;44(2):119e127. 77. Glasgow RE, Lichtenstein E, Marcus AC. Why don’t we see more translation of health promotion research to practice? Rethinking the efficacy-toeffectiveness transition. Am J Public Health. 2003;93(8):1261e1267. 78. Allen K, Zoellner J, Motley M, Estabrooks PA. Understanding the internal and external validity of health literacy interventions: a systematic literature review using the RE-AIM framework. J Health Commun. 2011;16(suppl 3): 55e72.

461

79. Glasgow RE, Bull SS, Gillette C, Klesges LM, Dzewaltowski DA. Behavior change intervention research in healthcare settings: a review of recent reports with emphasis on external validity. Am J Prev Med. 2002;23(1):62e69. 80. Blackman KCA, Zoellner J, Berrey LM, et al. Assessing the internal and external validity of mobile health physical activity promotion interventions: a systematic literature review using the RE-AIM framework. J Med Internet Res. 2013;15(10). 81. Virginia Tech College of Agriculture and Life Science. Resources and tools. Coding sheet for publications reporting on RE-AIM elements Web site. http:// www.re-aim.org/resources_and_tools/measures/literaturecoding.pdf; Updated 2014. Accessed 03.03.14. 82. Thomson D, Russell K, Becker L, Klassen T, Hartling L. The evolution of a new publication type: steps and challenges of producing overviews of reviews. Res Synth Methods. 2011;1(3e4):198e211. 83. Ioannidis JPA. Integration of evidence from multiple meta-analyses: a primer on umbrella reviews, treatment networks and multiple treatments metaanalyses. CMAJ. 2009;181(8):488e493. 84. Higgins JPT, Green S, eds. Cochrane Handbook for Systematic Reviews of Interventions Version 5.1.0, http://www.cochrane-handbook.org; Updated 2011. Accessed 05.03.14. 85. Shea BJ, Grimshaw JM, Wells GA, et al. Development of AMSTAR: a measurement tool to assess the methodological quality of systematic reviews. BMC Med Res Methodol. 2007;7. 86. Monasta L, Batty GD, Cattaneo A, et al. Early-life determinants of overweight and obesity: a review of systematic reviews. Obes Rev. 2010;11(10):695e708. 87. Schulz KF, Altman DG, Moher D. CONSORT 2010 statement: updated guidelines for reporting parallel group randomised trials. BMJ. 2010;340(7748): 698e702. 88. Eysenbach G, CONSORT-EHEALTH Group. CONSORT-EHEALTH: improving and standardizing evaluation reports of web-based and mobile health interventions. J Med Internet Res. 2011;13(4). 89. Campbell MK, Elbourne DR, Altman DG. CONSORT statement: extension to cluster randomised trials. BMJ. 2004;328(7441):702e708. 90. Von Elm E, Egger M, Altman DG, Pocock SJ, Gøtzsche PC, Vandenbroucke JP. Strengthening the reporting of observational studies in epidemiology (STROBE) statement: guidelines for reporting observational studies. BMJ. 2007;335(7624):806e808. 91. Des Jarlais DC, Lyles C, Crepaz N. Improving the reporting quality of nonrandomized evaluations of behavioral and public health interventions: the TREND statement. Am J Public Health. 2004;94(3):361e366. 92. Enhancing the Quality and Transparency of Health Research (EQUATER network). Resource centre. http://www.equator-network.org/ resource-centre/; Updated 2012. Accessed 27.03.13. 93. McGinnis JM, Foege W. Guide to community preventive services: harnessing the science. Am J Prev Med. 2000;18(1 suppl 1):1e2. 94. University of York Centre for Reviews and Dissemination. International prospective register of systematic reviews. Welcome to PROSPERO: International Prospective Register of Systematic Reviews Web site. http://www.crd.york.ac. uk/PROSPERO/; Updated 2013. Accessed 03.03.14. 95. Centre for Reviews and Dissemination. Chapter 6: Incorporating qualitative evidence in or alongside effectiveness reviews. In: Systematic Reviews: CRD’s Guidance for Undertaking Reviews in Health Care. York, UK: University of York; 2009:219e237. 96. Heyvaert M, Maes B, Onghena P. Mixed methods research synthesis: definition, framework, and potential. Qual Quant. 2013;47(2):659e676. 97. Sandelowski M, Voils CI, Barroso J. Defining and designing mixed research synthesis studies. Res Sch. 2006;13(1):29. 98. Armstrong R, Hall BJ, Doyle J, Waters E. ‘Scoping the scope’ of a cochrane review. J Public Health (Oxf). 2011;33(1):147e150. 99. HLWIKI International. Scoping reviews. http://hlwiki.slais.ubc.ca/index.php/ Scoping_reviews; Updated 2014. Accessed 28.02.14. 100. Virginia Tech College of Agriculture and Life Science. Reach Effectiveness Adoption Implementation Maintenance (RE-AIM). http://www.re-aim.org/ index.html; Updated 2014. Accessed 20.02.14. 101. Virginia Tech College of Agriculture and Life Science. Measures and checklists. Resources and tools Web site. http://www.re-aim.org/resources_and_tools/ measures/index.html; Updated 2014. Accessed 21.04.14.