RESEARCH
Relation of completeness of reporting of health research to journals’ endorsement of reporting guidelines: systematic review Adrienne Stevens,1 Larissa Shamseer,1 2 Erica Weinstein,3 Fatemeh Yazdi,1 Lucy Turner,1 Justin Thielman,1 Douglas G Altman,4 Allison Hirst,5 John Hoey,6 Anita Palepu,7 8 Kenneth F Schulz,9 David Moher1 2 1
Centre for Practice-Changing Research, Ottawa Hospital Research Institute, Ottawa, ON, Canada, K1H 8L6 2 Department of Epidemiology and Community Medicine, University of Ottawa, K1H 8M5 Ottawa, Canada 3 Albert Einstein College of Medicine, Yeshiva University, Bronx, NY 10461, USA 4 Centre for Statistics in Medicine, University of Oxford, Oxford OX3 7LD, UK 5 Nuffield Department of Surgical Sciences, University of Oxford, Oxford OX3 9DU, UK 6 Population and Public Health Initiative, Queen’s University, Kingston, ON, Canada, K7L 3N6 7 Centre for Health Evaluation and Outcome Sciences, St Paul’s Hospital, Vancouver, BC, Canada, V6Z 1Y9 8 Department of Medicine, University of British Columbia, Vancouver, BC, Canada, V5Z 1M9 9 International Clinical Sciences Support Center, FHI 360, Durham, NC 27713, USA Correspondence to: D Moher [email protected] Cite this as: BMJ 2014;348:g3804 doi: 10.1136/bmj.g3804
This is a summary of a paper that was published on bmj.com as BMJ 2014;348:g3804
bmj.com/archive ЖЖResearch methods & reporting: Better reporting of interventions: template for intervention description and replication (TIDieR) checklist and guide (BMJ 2014;348:g1687) ЖЖVideo abstract: TIDieR: Better reporting of interventions (http://bit. ly/1hcsAZG)
STUDY QUESTION Is the completeness of reporting of health research related to journals’ endorsement of reporting guidelines?
section (or similar) specifically listed a guideline, we considered it an “endorser.”
SUMMARY ANSWER Insufficient evidence exists to determine the relation between journals’ endorsement of reporting guidelines and the completeness of reporting of published health research reports.
Primary outcome The main outcome was the completeness of reporting, defined as complete reporting of all elements within a guidance checklist item.
WHAT IS KNOWN AND WHAT THIS PAPER ADDS Endorsement of CONSORT guidelines by journals is associated with more complete reporting for several checklist items, but no systematic review has comprehensively reviewed evaluations of other reporting guidelines. Apart from CONSORT, 101 rigorously developed reporting guidelines exist for reporting health research, only nine of which could be evaluated regarding their journal endorsement status, with data from only a few evaluations.
Main results and role of chance We included 101 reporting guidelines. Of 15 249 records retrieved from the search for evaluations, we identified 26 evaluations that assessed the completeness of reporting in relation to endorsement for nine reporting guidelines. Of those, 13 evaluations assessing seven reporting guidelines (BMJ economic checklist, CONSORT for harms, PRISMA, QUOROM, STARD, STRICTA, and STROBE) could be analysed. Items of reporting guidelines were assessed by few evaluations, and some guidelines were assessed on a subset of items.
Selection criteria for studies We identified reporting guidelines from a published systematic review and the EQUATOR Network (October 2011; addendum searches in January 2012). We included English language reporting guidelines (excluding CONSORT) that provided explicit guidance for reporting, described the guidance development process, and indicated use of a consensus development process. We searched for evaluations of reporting guidelines in Medline, Embase, and the Cochrane Methodology Register, or Scopus, depending on guidelines’ names. English or French language evaluations of included reporting guidelines were eligible if they assessed the completeness of reporting of studies as a primary objective and the included studies enabled the comparisons of interest (that is, after versus before journal endorsement, endorsing versus non-endorsing journals, or both). We first assessed each published study within an included evaluation according to the journal it was published in. We collected information on endorsement information from evaluations or from journals’ websites. If the journal’s “Instructions to authors”
Bias, confounding, and other reasons for caution Validity assessments were done, but no methods exist for synthesising those assessments in methods reviews. With few included evaluations, we were unable to do our pre-defined subgroup and sensitivity analyses, such as the extent of journals’ endorsement (for example, suggesting use of guideline or submitting a completed guideline checklist). More research is needed to supplement the evidence base. Future evaluations of reporting guidelines could take the form of comparisons based on journal endorsement status, but researchers should consider prospectively designed, controlled studies conducted in the context of the journal’s editorial process. Study funding/potential competing interests This study was funded by the Canadian Institutes of Health Research. DGA is supported by Cancer Research UK, DM by a University of Ottawa Research Chair, and KFS by FHI 360. DGA, DM, and KFS are executive members of the EQUATOR network; AH served as an EQUATOR staff member during this project.
Schematic representation of relation between evaluation, included studies, and journal endorsement status Evaluating completeness of reporting of studies (such as obstetric interventions)
Study 1 Study 4 Study 6 Studies published in endorsing journals (for example, check for guideline in author instructions on journal website)
Study 2 Study 7 Studies not eligible (for example, could not locate journal website to check endorsement)
Study 3 Study 5 Study 8 Studies published in non-endorsing journals (or in same endorsing journal but published before time of endorsement)
Do these studies completely report according to reporting guideline checklist items?
BMJ | 28 JUNE 2014 | VOLUME 348
11
Copyright of BMJ: British Medical Journal is the property of BMJ Publishing Group and its content may not be copied or emailed to multiple sites or posted to a listserv without the copyright holder's express written permission. However, users may print, download, or email articles for individual use.
Relation of completeness of reporting of health research to journals’ endorsement of reporting guidelines: systematic review Adrienne Stevens,1 Larissa Shamseer,1 2 Erica Weinstein,3 Fatemeh Yazdi,1 Lucy Turner,1 Justin Thielman,1 Douglas G Altman,4 Allison Hirst,5 John Hoey,6 Anita Palepu,7 8 Kenneth F Schulz,9 David Moher1 2 1
Centre for Practice-Changing Research, Ottawa Hospital Research Institute, Ottawa, ON, Canada, K1H 8L6 2 Department of Epidemiology and Community Medicine, University of Ottawa, K1H 8M5 Ottawa, Canada 3 Albert Einstein College of Medicine, Yeshiva University, Bronx, NY 10461, USA 4 Centre for Statistics in Medicine, University of Oxford, Oxford OX3 7LD, UK 5 Nuffield Department of Surgical Sciences, University of Oxford, Oxford OX3 9DU, UK 6 Population and Public Health Initiative, Queen’s University, Kingston, ON, Canada, K7L 3N6 7 Centre for Health Evaluation and Outcome Sciences, St Paul’s Hospital, Vancouver, BC, Canada, V6Z 1Y9 8 Department of Medicine, University of British Columbia, Vancouver, BC, Canada, V5Z 1M9 9 International Clinical Sciences Support Center, FHI 360, Durham, NC 27713, USA Correspondence to: D Moher [email protected] Cite this as: BMJ 2014;348:g3804 doi: 10.1136/bmj.g3804
This is a summary of a paper that was published on bmj.com as BMJ 2014;348:g3804
bmj.com/archive ЖЖResearch methods & reporting: Better reporting of interventions: template for intervention description and replication (TIDieR) checklist and guide (BMJ 2014;348:g1687) ЖЖVideo abstract: TIDieR: Better reporting of interventions (http://bit. ly/1hcsAZG)
STUDY QUESTION Is the completeness of reporting of health research related to journals’ endorsement of reporting guidelines?
section (or similar) specifically listed a guideline, we considered it an “endorser.”
SUMMARY ANSWER Insufficient evidence exists to determine the relation between journals’ endorsement of reporting guidelines and the completeness of reporting of published health research reports.
Primary outcome The main outcome was the completeness of reporting, defined as complete reporting of all elements within a guidance checklist item.
WHAT IS KNOWN AND WHAT THIS PAPER ADDS Endorsement of CONSORT guidelines by journals is associated with more complete reporting for several checklist items, but no systematic review has comprehensively reviewed evaluations of other reporting guidelines. Apart from CONSORT, 101 rigorously developed reporting guidelines exist for reporting health research, only nine of which could be evaluated regarding their journal endorsement status, with data from only a few evaluations.
Main results and role of chance We included 101 reporting guidelines. Of 15 249 records retrieved from the search for evaluations, we identified 26 evaluations that assessed the completeness of reporting in relation to endorsement for nine reporting guidelines. Of those, 13 evaluations assessing seven reporting guidelines (BMJ economic checklist, CONSORT for harms, PRISMA, QUOROM, STARD, STRICTA, and STROBE) could be analysed. Items of reporting guidelines were assessed by few evaluations, and some guidelines were assessed on a subset of items.
Selection criteria for studies We identified reporting guidelines from a published systematic review and the EQUATOR Network (October 2011; addendum searches in January 2012). We included English language reporting guidelines (excluding CONSORT) that provided explicit guidance for reporting, described the guidance development process, and indicated use of a consensus development process. We searched for evaluations of reporting guidelines in Medline, Embase, and the Cochrane Methodology Register, or Scopus, depending on guidelines’ names. English or French language evaluations of included reporting guidelines were eligible if they assessed the completeness of reporting of studies as a primary objective and the included studies enabled the comparisons of interest (that is, after versus before journal endorsement, endorsing versus non-endorsing journals, or both). We first assessed each published study within an included evaluation according to the journal it was published in. We collected information on endorsement information from evaluations or from journals’ websites. If the journal’s “Instructions to authors”
Bias, confounding, and other reasons for caution Validity assessments were done, but no methods exist for synthesising those assessments in methods reviews. With few included evaluations, we were unable to do our pre-defined subgroup and sensitivity analyses, such as the extent of journals’ endorsement (for example, suggesting use of guideline or submitting a completed guideline checklist). More research is needed to supplement the evidence base. Future evaluations of reporting guidelines could take the form of comparisons based on journal endorsement status, but researchers should consider prospectively designed, controlled studies conducted in the context of the journal’s editorial process. Study funding/potential competing interests This study was funded by the Canadian Institutes of Health Research. DGA is supported by Cancer Research UK, DM by a University of Ottawa Research Chair, and KFS by FHI 360. DGA, DM, and KFS are executive members of the EQUATOR network; AH served as an EQUATOR staff member during this project.
Schematic representation of relation between evaluation, included studies, and journal endorsement status Evaluating completeness of reporting of studies (such as obstetric interventions)
Study 1 Study 4 Study 6 Studies published in endorsing journals (for example, check for guideline in author instructions on journal website)
Study 2 Study 7 Studies not eligible (for example, could not locate journal website to check endorsement)
Study 3 Study 5 Study 8 Studies published in non-endorsing journals (or in same endorsing journal but published before time of endorsement)
Do these studies completely report according to reporting guideline checklist items?
BMJ | 28 JUNE 2014 | VOLUME 348
11
Copyright of BMJ: British Medical Journal is the property of BMJ Publishing Group and its content may not be copied or emailed to multiple sites or posted to a listserv without the copyright holder's express written permission. However, users may print, download, or email articles for individual use.
