DOI: 10.1111/eci.12484
PERSPECTIVE Journal editors impasse with outcome reporting bias * and Arthur L. Caplan† Rafael Dal-Re noma *Clinical Research, BUC (Biosciences UAM+CSIC) Programme, International Campus of Excellence, Universidad Auto de Madrid, Madrid, Spain, †Division of Medical Ethics, Langone Medical Center, New York University, New York, NY, USA
KEY POINTS Several requirements and regulations have been issued to promote clinical trial transparency through prospective registration of trials, disclosure of results, access to trial reports submitted to regulatory agencies and access to anonymized patient-level data. Clinical trial results are disseminated through articles. Yet, many present outcome reporting bias. Open access to trial data will help to deter outcome reporting bias. However, this is not enough to clinicians. Access to trial protocols by journal staff has proven rather inefficacious in preventing outcome reporting bias. Two proposals have been suggested or implemented to tackle outcome reporting bias that are discussed in this article. Editors should implement quality-control processes aiming at preventing outcome reporting bias. Readers should be informed as to the efficiency of the implemented process. Eur J Clin Invest 2015; 45 (9): 895–898 In 2015, hardly anyone could seriously argue against open access to clinical trial results, the type of research design that could most impact people’s health. Yet, publication bias and outcomes reporting bias prompted different stakeholders, for example the International Committee of Medical Journal Editors (ICMJE) [1], the World Health Organization (WHO) [2], the U.S. Food and Drug Administration [3] and the European Medicines Agency (EMA) [4,5], to issue a number of requirements and regulations asking for both prospective registration of trials and disclosure of trial results aimed at strengthening trial transparency. Investigators willing to work in a completely transparent environment called for access to research protocols [6] and to individual participant data (raw data) [7]. Having access to trial protocols (or information included on registries) will help to deter publication bias and outcome reporting bias; having access to raw data will facilitate the conduct of meta-analyses, re-analysis of already published endpoints and analyses of those that were not originally performed and avoid wasting resources for redundant studies. These activities are of critical importance. 35% of published re-analyses led to conclusions different from those of the original article [8].
Open access to participant-level data or extensive clinical trial reports All critical stakeholders involved in the clinical trial enterprise have moved towards trying to fulfil the demands for
trial transparency. Trialists publishing in high impact journals strongly support data sharing [9]. Twenty private and public funders belonging to the Public Health Research Data Forum [10] have agreed to promote greater access to data. Eleven pharmaceutical companies have a mechanism to provide access to anonymized patient-level data of trials upon request [11]. Beginning 1 January 2015, the EMA is disclosing Clinical Study Reports (a 1000 + pages long document) of each trial on medicines conducted in the European Union [12]. This report –which includes the trial protocol, the statistical analysis, trial results and a summary in lay language – is accessible upon request. In the near future, the EMA will address how to provide participant-level data [12]. Finally, PLOS journals [13] and The BMJ [14] require authors to make all data available.
Selective reporting of outcomes in clinical trials Readers expect published trial results to be accurate and reliable. However, outcome reporting bias is common [15–17], is present in subtle ways [18] and is seriously impacting the credibility of clinical trials [19]. Furthermore, selective reporting of outcomes contributes to the waste of scarce resources available for clinical research [20]. It is one of the factors that could contribute to the lack of clinical utility of medicines once they are commercially available [21].
European Journal of Clinical Investigation Vol 45
895
AND A. L. CAPLAN R. DAL-RE
Trial registration has not deterred the selective reporting of outcomes: is still happening even in the most prestigious journals [22]. Open access to participant-level data even if fully achieved is not sufficient to prevent outcome reporting bias. Moreover, it is almost useless for busy clinicians that rarely have time to check an article’s information against that provided on a database.
Two key journal editor’s responsibilities Following the Committee on Publication Ethics’ Code of conduct [23], journal editors are accountable for every article published in their journals. This means that they have to strive to meet readers’ expectations about the reliability of the trial data they publish. They must ensure the quality and integrity of all articles involving clinical trials [23]. In addition, the publication of deceptive selective trial outcomes is a form of data falsification – a recognized form of misconduct [24]. Journal editors should support initiatives that aim to reduce publication misconduct and to educate clinical investigators about publication ethics [23]. To the best of our knowledge with the exception of The BMJ and the BMJ Open, no journal has implemented a specific process to tackle selective reporting of trial outcomes. A number of top medical journals ask authors to submit the trial protocol when submitting a clinical trial results manuscript. However, access to protocols by journal staff has proven to be inadequate in preventing outcome reporting bias [17,22]. Journals unfortunately are failing in their obligation to minimize selective reporting of trial outcomes.
Focus on outcome reporting bias prevention Two proposals have been suggested that journals can institute to minimize bias in clinical trials. One, currently utilized by The BMJ and the BMJ Open, requires a declaration of transparency by the lead author, stating the manuscript shows an ‘honest, accurate and transparent account of the study being reported’ [25]. Lead authors of research articles are currently assuring readers that ‘no important aspects of the research have been omitted from the published article and any discrepancies from the study as planned (and, if relevant, registered) have been explained’ [25]. This attestation, however, is not enough. It will not have much sway with investigators intent on duplicate publication or plagiarism. Both could be present even when authors are required to formally state that the information provided in the submitted manuscript is original and has not been already published. And, the usefulness of this attestation to prevent outcome reporting bias is almost impossible to assess. The second proposed method tries to fulfil this requirement. The author has to submit part of the information provided on
896
www.ejci-online.com
the public trials registry – where it was initially registered – and an explanation of any discrepancy between the information provided in the manuscript with that included on the registry [26]. Without providing this information, the manuscript should not be processed for publication. The registry information proposed to be provided includes 8 of the 20 World Health Organization items of The Trial Registration Data Set [26]. As most reviewers are not willing to cross-check the information in the manuscript with that on the registry [27], journal editorial staff should conduct this task prior to any decisions about suitability for peer-review. Granted, the usefulness of a crosschecking requirement depends on the quality of the registered information, which in many trials is deficient [28]. Lack of timely registration is important with regard to preventing (or documenting) outcome reporting bias. Recent data showed that although there is a notable trend of increasing timely registration, in 19% of all 24,769 trials started in 20082011 registered on ClinicalTrials.gov [29] it cannot be assured that trial outcomes were not registered to favour the results obtained in the trial through, for instance, conducting interim analyses. Editors should consider the publication on a regular basis of a report on the number of manuscripts that were rejected because of unacceptable discrepancies between submitted manuscripts and registry data. Thus, for instance, omission, modification or introduction of a primary end-point in a manuscript not reflected in the registry – present in 4% to 50% of articles [17] – could suggest, unless clearly explained and strongly supported [30], a lack of integrity: there is evidence of manipulation of outcomes analysis and reporting [19]. Such a report could also comment on the number of those trial results published with acceptable discrepancies and on the number having no discrepancies from the time of manuscript submission. Submission of the trial protocol (and relevant amendments) at the time of manuscript submission and of an explanation for any discrepancy between the information provided in the manuscript with that included in the protocol could be an appropriate alternative. However, registry information, if appropriately provided, seems more adequate as it is public from the time of trial registration and registries like ClinicalTrials.gov provide for the tracking of all changes introduced throughout the life of a trial. The regular publication of the number of manuscripts that were rejected by the journal and reasons supporting it will inform readers about the extent of the outcome reporting bias problem in clinical trials and thereby will probably help deterring this type of misconduct. In addition, journals should state in their instructions to authors that if important discrepancies are observed between a published article and the information provided by third parties (e.g. research based on data from Regulatory Agencies or open-access databases), an
ª 2015 Stichting European Society for Clinical Investigation Journal Foundation
EDITORS AND OUTCOME REPORTING BIAS
Editorial ‘expression of concern’ could be published or, the article might be retracted. Implementing any quality-control process generates additional costs for journals. The key question is the price they and their subscribers and publishers are willing to invest to strive to make their journal free of selective outcome reporting.
Clinical trial transparency: looking forward Recent developments are reshaping clinical trials transparency landscape. Thus, on the one hand, the Institute of Medicine recommendations on trial data sharing will probably have a strong influence in many arenas, even outside the US, with the result that hopefully data sharing will soon be the expected norm [31]. The published U.S. National Institutes of Health (NIH) draft policy for public comment requiring prospective registration and results disclosure of all NIH-funded trials [32] is an another important step towards the transparency of all trials – including those conducted with nonregulated interventions such as behaviour, surgery or physiotherapy. This requirement should be followed by many other funders and other stakeholders [33] to be really meaningful. Finally, the WHO has recently called for open disclosure of results of all clinical trials, including past trials [34]. On the other hand, the EMA has disappointed the expectations of many concerning securing full access to all clinical trial data. Although a major step forward, the new EMA policy [12] is only applicable to marketing authorizations submitted after January 1st 2015. In addition, as the trial sponsor is primarily responsible for redacting Clinical Study Reports, some information could remain hidden [35]. Journal editors should work together with other stakeholders and consider new approaches aiming to increase transparency and deter outcome reporting bias. Thus, funding agencies could audit (randomly or following prespecified criteria) the reliability and completeness of trial results published and/or posted on registries; investigators should be made accountable in terms of future funding for any important deviation that could impact interpretation of trial’s results. Public funders, such as the NIH, should consider public disclosure of the results of these audits. Private sponsors such as foundations, biotech and pharmaceutical companies could also be encouraged to implement processes to ensure published or disclosed trial results followed the original protocol (and further relevant amendments); this task should be conducted by outside auditors or in-house quality assurance departments not related to those departments responsible for the conduct of trials and dissemination of results. Fulfilment of quality assessment standards should be made public, for instance, on the registry where the trial results are posted.
Next steps Current movements to increase transparency will not prevent outcome reporting bias or the ability of practicing physicians to be alert to it. Even if registration of nonregulated intervention trials is widely followed in the future, the credibility of the clinical trial enterprise and the protection of the health of the public require that journals do all that they can to insure the publication of accurate and reliable clinical trials results. ICMJE should address selective outcome reporting by mandating quality-control processes to prevent it and by regularly assessing their efficiency. Asking authors to specifically inform editorial staff at the time of manuscript submission about critical aspects of every trial, by comparing the information provided in the manuscript with that of a reliable source document (the registry or the protocol), seems appropriate, but other processes might be adequate as well. Ideally, ICMJE should propose which clinical trial information is critical. Until this happens, individual journal editors should start implementing protections that will minimize selective outcome trial reporting. Conflicts of interest The opinions expressed in this article are those of the authors and may not reflect the opinions of the organizations that they work for. Both authors reported having no conflict of interest. Funding This work required no funding. Authors contributions RDR conceived the idea and wrote the first draft of the manuscript. ALC was involved in the critical revision of the manuscript for important intellectual content. RDR and ALC approved the final version of the manuscript. Address Clinical Research BUC (Biosciences UAM+CSIC) Programme International Campus of Excellence Universidad Aut onoma de Madrid Madrid Spain (R. Dal-Re); Division of Medical Ethics Langone Medical Center New York University, New York, NY, USA (A. L. Caplan). Correspondence to: Dr Rafael Dal-Re, Investigaci on Clınica, Programa BUC (Biociencias UAM+CSIC), Campus de Excelencia Internacional, Universidad Aut onoma de Madrid, Ciudad Universitaria de Cantoblanco, c/Einstein 3, E-28049 Madrid, Spain. Tel.: 34-649410221; fax: 34-914974083; e-mail: [email protected] Received 28 April 2015; accepted 19 June 2015
European Journal of Clinical Investigation Vol 45
897
AND A. L. CAPLAN R. DAL-RE
References 1 International Committee of Medical Journal Editors. Recommendations for the conduct, reporting, editing, and publication of scholarly work in medical journals. Dec 2014. Available at: www.icmje.org/icmje-recommendations.pdf. Accessed on 3 July 2015. 2 World Health Organization. International clinical trials registry platform (ICTRP). Available at: www.who.int/ictrp/en/. Accessed on 3 July 2015. 3 Food and Drug Administration Amendments Act of 2007. Public Law 110-85 www.gpo.gov/fdsys/pkg/PLAW-110publ85/pdf/ PLAW-110publ85.pdf. Accessed on 3 July 2015. 4 European Commission. Directive 2001/20/EC of the European parliament and of the council of 4 April 2001 on the approximation of the laws, regulations, and administrative provisions of the member states relating to the implementation of good clinical practice in the conduct of clinical trials on medicinal products for human use. May 2001. Available at: http://ec.europa.eu/ health/files/eudralex/vol-1/dir_2001_20/dir_2001_20_en.pdf. Accessed on 3 July 2015. 5 European Medicines Agency. Posting of clinical trial summary results in European Clinical Trials Database (EudraCT) to become mandatory for sponsors as of 21 July 2014. Available at: www. ema.europa.eu/ema/index.jsp?curl=pages/news_and_events/ news/2014/06/news_detail_002127.jsp&mid=WC0b01ac05800 4d5c1. Accessed on 3 July 2015. 6 Chan AW, Upshur R, Singh JA, Ghersi D, Chapuis F, Altman DG. Research protocols: waiving confidentiality for the greater good. BMJ 2006;332:1086–9. 7 Hrynaszkiewicz I, Altman DG. Towards agreement on best practice for publishing raw clinical trial data. Trials 2009;10:17. 8 Ebrahim S, Sohani ZN, Montoya L, Agarwal A, Thorlund K, Mills EJ et al. Reanalyses of randomized clinical trial data. JAMA 2014;312:1024–32. 9 Rathi VK, Strait KM, Gross CP, Hrynaszkiewicz I, Joffe S, Krumholz HM et al. Predictors of clinical trial data sharing: exploratory analysis of a cross-sectional survey. Trials 2014;15:384. 10 Wellcome Trust. Public Health Research Data Forum. Available at: http://www.wellcome.ac.uk/About-us/Policy/Spotlight-issues/ Data-sharing/Public-health-and-epidemiology/WTDV030689.htm. Accessed on 3 July 2015. 11 Clinical Study Data Request.com https://www.clinicalstudydatare quest.com/. Accessed on 3 July 2015. 12 European Medicines Agency. European Medicines Agency policy on publication of clinical data for medicinal products for human use. 2 October 2014. EMA/240810/2013. Policy/0070. Available at: http://www.ema.europa.eu/docs/en_GB/document_library/Other/ 2014/10/WC500174796.pdf. Accessed on 3 July 2015. 13 PLOS Data policy summary. Available at: http://blogs.lshtm.ac.uk/ rdmss/files/2014/03/PLOS-Data-Policy-Summary.pdf. Accessed on 3 July 2015. 14 Godlee F, Groves T. The new BMJ policy on sharing data from drug and device trials. BMJ 2012;345:e7888. 15 Dwan K, Gamble C, Williamson PR, Kirkham JJ, for the Reporting Bias Group. Systematic review of the empirical evidence of study publication bias and outcome reporting bias - an updated review. PLoS ONE 2013;8:e66844.
898
www.ejci-online.com
16 Page MJ, McKenzie JE, Kirkham J et al. Bias due to selective inclusion and reporting of outcomes and analyses in systematic reviews of randomised trials of healthcare interventions. Cochrane Database Syst Rev 2014;10:MR000035. 17 Dwan K, Altman DG, Cresswell L et al. Comparison of protocols and registry entries to published reports for randomised controlled trials. Cochrane Database Syst Rev 2011;1:MR000031. 18 Dal-Re R. Improving transparency of clinical trials. Trends Pharmacol Sci 2015;36:323–5. 19 Ioannidis JPA. Clinical trials: what a waste (editorial). BMJ 2014;349: g7089. 20 Chan AW, Song F, Vickers A, Jefferson T, Dickersin K, Gøtzsche PC et al. Increasing value and reducing waste: addressing inaccessible research. Lancet 2014;383:257–66. 21 Naci H, Ioannidis JP. How good is ‘evidence’ from clinical studies of drug effects and why might such evidence fail in the prediction of the clinical utility of drugs? Annu Rev Pharmacol Toxicol 2015;55:169– 89. 22 Becker JE, Krumholz HM, Ben-Josef G, Ross JS. Reporting of results in ClinicalTrials.gov and high-impact journals. JAMA 2014;311:1063–5. 23 COPE. Committee on publication ethics. Code of conduct and best practice guidelines for journal editors. March 2011. Available at: http://publicationethics.org/resources/code-conduct. Accessed on 3 July 2015. 24 WAME Publication Ethics Committee. Recommendations on publication ethics policies for medical journals. World Association of Medical Editors. Available at: http://www.wame.org/about/ recommendations-on-publication-ethics-policie. Accessed on 3 July 2015. 25 Altman DG, Moher D. Declaration of transparency for each research article. BMJ 2013;347:f4796. 26 Dal-Re R, Caplan A. Time to ensure that clinical trial appropriate results are actually published. Eur J Clin Pharmacol 2014;70:491–3. 27 Mathieu S, Chan A-W, Ravaud P. Use of trial register information during the peer review process. PLoS ONE 2013;8:e59910. 28 Viergever RF, Karam G, Reis A et al. The quality of registration of clinical trials: still a problem. PLoS ONE 2014;9:e84727. 29 Huser V, Cimino JJ. Evaluating adherence to the International Committee of Medical Journal Editors’ policy of mandatory, timely clinical trial registration. J Am Med Inform Assoc 2013;20(e1):e169–74. 30 Evans S. When and how can endpoints be changed after initiation of a randomized clinical trial? PLoS Clin Trials 2007;2:e18. 31 Institute of Medicine. Sharing of Clinical Trial Data: Maximizing benefits, minimizing risk. 14 January 2015. Available at: https:// www.iom.edu/Reports/2015/Sharing-Clinical-Trial-Data.aspx. Accessed on 3 July 2015. 32 Zarin DA, Tse T, Sheehan J. The proposed rule for US clinical trial registration and results submission. N Engl J Med 2015;372:174–80. 33 Dal-Re R, Ioannidis JPA, Bracken MB. A call to improve the transparency of trials of non-regulated interventions. BMJ 2015;350: h1323. 34 World Health Organization. WHO Statement on Public Disclosure of Clinical Trial Results. Available at: http://www.who.int/ictrp/ results/reporting/en/. Accessed on 3 July 2015. 35 Watson R. European ombudsman questions European Medicines Agency over Abbvie redactions. BMJ 2014;349:g6904.
ª 2015 Stichting European Society for Clinical Investigation Journal Foundation
PERSPECTIVE Journal editors impasse with outcome reporting bias * and Arthur L. Caplan† Rafael Dal-Re noma *Clinical Research, BUC (Biosciences UAM+CSIC) Programme, International Campus of Excellence, Universidad Auto de Madrid, Madrid, Spain, †Division of Medical Ethics, Langone Medical Center, New York University, New York, NY, USA
KEY POINTS Several requirements and regulations have been issued to promote clinical trial transparency through prospective registration of trials, disclosure of results, access to trial reports submitted to regulatory agencies and access to anonymized patient-level data. Clinical trial results are disseminated through articles. Yet, many present outcome reporting bias. Open access to trial data will help to deter outcome reporting bias. However, this is not enough to clinicians. Access to trial protocols by journal staff has proven rather inefficacious in preventing outcome reporting bias. Two proposals have been suggested or implemented to tackle outcome reporting bias that are discussed in this article. Editors should implement quality-control processes aiming at preventing outcome reporting bias. Readers should be informed as to the efficiency of the implemented process. Eur J Clin Invest 2015; 45 (9): 895–898 In 2015, hardly anyone could seriously argue against open access to clinical trial results, the type of research design that could most impact people’s health. Yet, publication bias and outcomes reporting bias prompted different stakeholders, for example the International Committee of Medical Journal Editors (ICMJE) [1], the World Health Organization (WHO) [2], the U.S. Food and Drug Administration [3] and the European Medicines Agency (EMA) [4,5], to issue a number of requirements and regulations asking for both prospective registration of trials and disclosure of trial results aimed at strengthening trial transparency. Investigators willing to work in a completely transparent environment called for access to research protocols [6] and to individual participant data (raw data) [7]. Having access to trial protocols (or information included on registries) will help to deter publication bias and outcome reporting bias; having access to raw data will facilitate the conduct of meta-analyses, re-analysis of already published endpoints and analyses of those that were not originally performed and avoid wasting resources for redundant studies. These activities are of critical importance. 35% of published re-analyses led to conclusions different from those of the original article [8].
Open access to participant-level data or extensive clinical trial reports All critical stakeholders involved in the clinical trial enterprise have moved towards trying to fulfil the demands for
trial transparency. Trialists publishing in high impact journals strongly support data sharing [9]. Twenty private and public funders belonging to the Public Health Research Data Forum [10] have agreed to promote greater access to data. Eleven pharmaceutical companies have a mechanism to provide access to anonymized patient-level data of trials upon request [11]. Beginning 1 January 2015, the EMA is disclosing Clinical Study Reports (a 1000 + pages long document) of each trial on medicines conducted in the European Union [12]. This report –which includes the trial protocol, the statistical analysis, trial results and a summary in lay language – is accessible upon request. In the near future, the EMA will address how to provide participant-level data [12]. Finally, PLOS journals [13] and The BMJ [14] require authors to make all data available.
Selective reporting of outcomes in clinical trials Readers expect published trial results to be accurate and reliable. However, outcome reporting bias is common [15–17], is present in subtle ways [18] and is seriously impacting the credibility of clinical trials [19]. Furthermore, selective reporting of outcomes contributes to the waste of scarce resources available for clinical research [20]. It is one of the factors that could contribute to the lack of clinical utility of medicines once they are commercially available [21].
European Journal of Clinical Investigation Vol 45
895
AND A. L. CAPLAN R. DAL-RE
Trial registration has not deterred the selective reporting of outcomes: is still happening even in the most prestigious journals [22]. Open access to participant-level data even if fully achieved is not sufficient to prevent outcome reporting bias. Moreover, it is almost useless for busy clinicians that rarely have time to check an article’s information against that provided on a database.
Two key journal editor’s responsibilities Following the Committee on Publication Ethics’ Code of conduct [23], journal editors are accountable for every article published in their journals. This means that they have to strive to meet readers’ expectations about the reliability of the trial data they publish. They must ensure the quality and integrity of all articles involving clinical trials [23]. In addition, the publication of deceptive selective trial outcomes is a form of data falsification – a recognized form of misconduct [24]. Journal editors should support initiatives that aim to reduce publication misconduct and to educate clinical investigators about publication ethics [23]. To the best of our knowledge with the exception of The BMJ and the BMJ Open, no journal has implemented a specific process to tackle selective reporting of trial outcomes. A number of top medical journals ask authors to submit the trial protocol when submitting a clinical trial results manuscript. However, access to protocols by journal staff has proven to be inadequate in preventing outcome reporting bias [17,22]. Journals unfortunately are failing in their obligation to minimize selective reporting of trial outcomes.
Focus on outcome reporting bias prevention Two proposals have been suggested that journals can institute to minimize bias in clinical trials. One, currently utilized by The BMJ and the BMJ Open, requires a declaration of transparency by the lead author, stating the manuscript shows an ‘honest, accurate and transparent account of the study being reported’ [25]. Lead authors of research articles are currently assuring readers that ‘no important aspects of the research have been omitted from the published article and any discrepancies from the study as planned (and, if relevant, registered) have been explained’ [25]. This attestation, however, is not enough. It will not have much sway with investigators intent on duplicate publication or plagiarism. Both could be present even when authors are required to formally state that the information provided in the submitted manuscript is original and has not been already published. And, the usefulness of this attestation to prevent outcome reporting bias is almost impossible to assess. The second proposed method tries to fulfil this requirement. The author has to submit part of the information provided on
896
www.ejci-online.com
the public trials registry – where it was initially registered – and an explanation of any discrepancy between the information provided in the manuscript with that included on the registry [26]. Without providing this information, the manuscript should not be processed for publication. The registry information proposed to be provided includes 8 of the 20 World Health Organization items of The Trial Registration Data Set [26]. As most reviewers are not willing to cross-check the information in the manuscript with that on the registry [27], journal editorial staff should conduct this task prior to any decisions about suitability for peer-review. Granted, the usefulness of a crosschecking requirement depends on the quality of the registered information, which in many trials is deficient [28]. Lack of timely registration is important with regard to preventing (or documenting) outcome reporting bias. Recent data showed that although there is a notable trend of increasing timely registration, in 19% of all 24,769 trials started in 20082011 registered on ClinicalTrials.gov [29] it cannot be assured that trial outcomes were not registered to favour the results obtained in the trial through, for instance, conducting interim analyses. Editors should consider the publication on a regular basis of a report on the number of manuscripts that were rejected because of unacceptable discrepancies between submitted manuscripts and registry data. Thus, for instance, omission, modification or introduction of a primary end-point in a manuscript not reflected in the registry – present in 4% to 50% of articles [17] – could suggest, unless clearly explained and strongly supported [30], a lack of integrity: there is evidence of manipulation of outcomes analysis and reporting [19]. Such a report could also comment on the number of those trial results published with acceptable discrepancies and on the number having no discrepancies from the time of manuscript submission. Submission of the trial protocol (and relevant amendments) at the time of manuscript submission and of an explanation for any discrepancy between the information provided in the manuscript with that included in the protocol could be an appropriate alternative. However, registry information, if appropriately provided, seems more adequate as it is public from the time of trial registration and registries like ClinicalTrials.gov provide for the tracking of all changes introduced throughout the life of a trial. The regular publication of the number of manuscripts that were rejected by the journal and reasons supporting it will inform readers about the extent of the outcome reporting bias problem in clinical trials and thereby will probably help deterring this type of misconduct. In addition, journals should state in their instructions to authors that if important discrepancies are observed between a published article and the information provided by third parties (e.g. research based on data from Regulatory Agencies or open-access databases), an
ª 2015 Stichting European Society for Clinical Investigation Journal Foundation
EDITORS AND OUTCOME REPORTING BIAS
Editorial ‘expression of concern’ could be published or, the article might be retracted. Implementing any quality-control process generates additional costs for journals. The key question is the price they and their subscribers and publishers are willing to invest to strive to make their journal free of selective outcome reporting.
Clinical trial transparency: looking forward Recent developments are reshaping clinical trials transparency landscape. Thus, on the one hand, the Institute of Medicine recommendations on trial data sharing will probably have a strong influence in many arenas, even outside the US, with the result that hopefully data sharing will soon be the expected norm [31]. The published U.S. National Institutes of Health (NIH) draft policy for public comment requiring prospective registration and results disclosure of all NIH-funded trials [32] is an another important step towards the transparency of all trials – including those conducted with nonregulated interventions such as behaviour, surgery or physiotherapy. This requirement should be followed by many other funders and other stakeholders [33] to be really meaningful. Finally, the WHO has recently called for open disclosure of results of all clinical trials, including past trials [34]. On the other hand, the EMA has disappointed the expectations of many concerning securing full access to all clinical trial data. Although a major step forward, the new EMA policy [12] is only applicable to marketing authorizations submitted after January 1st 2015. In addition, as the trial sponsor is primarily responsible for redacting Clinical Study Reports, some information could remain hidden [35]. Journal editors should work together with other stakeholders and consider new approaches aiming to increase transparency and deter outcome reporting bias. Thus, funding agencies could audit (randomly or following prespecified criteria) the reliability and completeness of trial results published and/or posted on registries; investigators should be made accountable in terms of future funding for any important deviation that could impact interpretation of trial’s results. Public funders, such as the NIH, should consider public disclosure of the results of these audits. Private sponsors such as foundations, biotech and pharmaceutical companies could also be encouraged to implement processes to ensure published or disclosed trial results followed the original protocol (and further relevant amendments); this task should be conducted by outside auditors or in-house quality assurance departments not related to those departments responsible for the conduct of trials and dissemination of results. Fulfilment of quality assessment standards should be made public, for instance, on the registry where the trial results are posted.
Next steps Current movements to increase transparency will not prevent outcome reporting bias or the ability of practicing physicians to be alert to it. Even if registration of nonregulated intervention trials is widely followed in the future, the credibility of the clinical trial enterprise and the protection of the health of the public require that journals do all that they can to insure the publication of accurate and reliable clinical trials results. ICMJE should address selective outcome reporting by mandating quality-control processes to prevent it and by regularly assessing their efficiency. Asking authors to specifically inform editorial staff at the time of manuscript submission about critical aspects of every trial, by comparing the information provided in the manuscript with that of a reliable source document (the registry or the protocol), seems appropriate, but other processes might be adequate as well. Ideally, ICMJE should propose which clinical trial information is critical. Until this happens, individual journal editors should start implementing protections that will minimize selective outcome trial reporting. Conflicts of interest The opinions expressed in this article are those of the authors and may not reflect the opinions of the organizations that they work for. Both authors reported having no conflict of interest. Funding This work required no funding. Authors contributions RDR conceived the idea and wrote the first draft of the manuscript. ALC was involved in the critical revision of the manuscript for important intellectual content. RDR and ALC approved the final version of the manuscript. Address Clinical Research BUC (Biosciences UAM+CSIC) Programme International Campus of Excellence Universidad Aut onoma de Madrid Madrid Spain (R. Dal-Re); Division of Medical Ethics Langone Medical Center New York University, New York, NY, USA (A. L. Caplan). Correspondence to: Dr Rafael Dal-Re, Investigaci on Clınica, Programa BUC (Biociencias UAM+CSIC), Campus de Excelencia Internacional, Universidad Aut onoma de Madrid, Ciudad Universitaria de Cantoblanco, c/Einstein 3, E-28049 Madrid, Spain. Tel.: 34-649410221; fax: 34-914974083; e-mail: [email protected] Received 28 April 2015; accepted 19 June 2015
European Journal of Clinical Investigation Vol 45
897
AND A. L. CAPLAN R. DAL-RE
References 1 International Committee of Medical Journal Editors. Recommendations for the conduct, reporting, editing, and publication of scholarly work in medical journals. Dec 2014. Available at: www.icmje.org/icmje-recommendations.pdf. Accessed on 3 July 2015. 2 World Health Organization. International clinical trials registry platform (ICTRP). Available at: www.who.int/ictrp/en/. Accessed on 3 July 2015. 3 Food and Drug Administration Amendments Act of 2007. Public Law 110-85 www.gpo.gov/fdsys/pkg/PLAW-110publ85/pdf/ PLAW-110publ85.pdf. Accessed on 3 July 2015. 4 European Commission. Directive 2001/20/EC of the European parliament and of the council of 4 April 2001 on the approximation of the laws, regulations, and administrative provisions of the member states relating to the implementation of good clinical practice in the conduct of clinical trials on medicinal products for human use. May 2001. Available at: http://ec.europa.eu/ health/files/eudralex/vol-1/dir_2001_20/dir_2001_20_en.pdf. Accessed on 3 July 2015. 5 European Medicines Agency. Posting of clinical trial summary results in European Clinical Trials Database (EudraCT) to become mandatory for sponsors as of 21 July 2014. Available at: www. ema.europa.eu/ema/index.jsp?curl=pages/news_and_events/ news/2014/06/news_detail_002127.jsp&mid=WC0b01ac05800 4d5c1. Accessed on 3 July 2015. 6 Chan AW, Upshur R, Singh JA, Ghersi D, Chapuis F, Altman DG. Research protocols: waiving confidentiality for the greater good. BMJ 2006;332:1086–9. 7 Hrynaszkiewicz I, Altman DG. Towards agreement on best practice for publishing raw clinical trial data. Trials 2009;10:17. 8 Ebrahim S, Sohani ZN, Montoya L, Agarwal A, Thorlund K, Mills EJ et al. Reanalyses of randomized clinical trial data. JAMA 2014;312:1024–32. 9 Rathi VK, Strait KM, Gross CP, Hrynaszkiewicz I, Joffe S, Krumholz HM et al. Predictors of clinical trial data sharing: exploratory analysis of a cross-sectional survey. Trials 2014;15:384. 10 Wellcome Trust. Public Health Research Data Forum. Available at: http://www.wellcome.ac.uk/About-us/Policy/Spotlight-issues/ Data-sharing/Public-health-and-epidemiology/WTDV030689.htm. Accessed on 3 July 2015. 11 Clinical Study Data Request.com https://www.clinicalstudydatare quest.com/. Accessed on 3 July 2015. 12 European Medicines Agency. European Medicines Agency policy on publication of clinical data for medicinal products for human use. 2 October 2014. EMA/240810/2013. Policy/0070. Available at: http://www.ema.europa.eu/docs/en_GB/document_library/Other/ 2014/10/WC500174796.pdf. Accessed on 3 July 2015. 13 PLOS Data policy summary. Available at: http://blogs.lshtm.ac.uk/ rdmss/files/2014/03/PLOS-Data-Policy-Summary.pdf. Accessed on 3 July 2015. 14 Godlee F, Groves T. The new BMJ policy on sharing data from drug and device trials. BMJ 2012;345:e7888. 15 Dwan K, Gamble C, Williamson PR, Kirkham JJ, for the Reporting Bias Group. Systematic review of the empirical evidence of study publication bias and outcome reporting bias - an updated review. PLoS ONE 2013;8:e66844.
898
www.ejci-online.com
16 Page MJ, McKenzie JE, Kirkham J et al. Bias due to selective inclusion and reporting of outcomes and analyses in systematic reviews of randomised trials of healthcare interventions. Cochrane Database Syst Rev 2014;10:MR000035. 17 Dwan K, Altman DG, Cresswell L et al. Comparison of protocols and registry entries to published reports for randomised controlled trials. Cochrane Database Syst Rev 2011;1:MR000031. 18 Dal-Re R. Improving transparency of clinical trials. Trends Pharmacol Sci 2015;36:323–5. 19 Ioannidis JPA. Clinical trials: what a waste (editorial). BMJ 2014;349: g7089. 20 Chan AW, Song F, Vickers A, Jefferson T, Dickersin K, Gøtzsche PC et al. Increasing value and reducing waste: addressing inaccessible research. Lancet 2014;383:257–66. 21 Naci H, Ioannidis JP. How good is ‘evidence’ from clinical studies of drug effects and why might such evidence fail in the prediction of the clinical utility of drugs? Annu Rev Pharmacol Toxicol 2015;55:169– 89. 22 Becker JE, Krumholz HM, Ben-Josef G, Ross JS. Reporting of results in ClinicalTrials.gov and high-impact journals. JAMA 2014;311:1063–5. 23 COPE. Committee on publication ethics. Code of conduct and best practice guidelines for journal editors. March 2011. Available at: http://publicationethics.org/resources/code-conduct. Accessed on 3 July 2015. 24 WAME Publication Ethics Committee. Recommendations on publication ethics policies for medical journals. World Association of Medical Editors. Available at: http://www.wame.org/about/ recommendations-on-publication-ethics-policie. Accessed on 3 July 2015. 25 Altman DG, Moher D. Declaration of transparency for each research article. BMJ 2013;347:f4796. 26 Dal-Re R, Caplan A. Time to ensure that clinical trial appropriate results are actually published. Eur J Clin Pharmacol 2014;70:491–3. 27 Mathieu S, Chan A-W, Ravaud P. Use of trial register information during the peer review process. PLoS ONE 2013;8:e59910. 28 Viergever RF, Karam G, Reis A et al. The quality of registration of clinical trials: still a problem. PLoS ONE 2014;9:e84727. 29 Huser V, Cimino JJ. Evaluating adherence to the International Committee of Medical Journal Editors’ policy of mandatory, timely clinical trial registration. J Am Med Inform Assoc 2013;20(e1):e169–74. 30 Evans S. When and how can endpoints be changed after initiation of a randomized clinical trial? PLoS Clin Trials 2007;2:e18. 31 Institute of Medicine. Sharing of Clinical Trial Data: Maximizing benefits, minimizing risk. 14 January 2015. Available at: https:// www.iom.edu/Reports/2015/Sharing-Clinical-Trial-Data.aspx. Accessed on 3 July 2015. 32 Zarin DA, Tse T, Sheehan J. The proposed rule for US clinical trial registration and results submission. N Engl J Med 2015;372:174–80. 33 Dal-Re R, Ioannidis JPA, Bracken MB. A call to improve the transparency of trials of non-regulated interventions. BMJ 2015;350: h1323. 34 World Health Organization. WHO Statement on Public Disclosure of Clinical Trial Results. Available at: http://www.who.int/ictrp/ results/reporting/en/. Accessed on 3 July 2015. 35 Watson R. European ombudsman questions European Medicines Agency over Abbvie redactions. BMJ 2014;349:g6904.
ª 2015 Stichting European Society for Clinical Investigation Journal Foundation
