International Journal of Cardiology 198 (2015) 47–48
Contents lists available at ScienceDirect
International Journal of Cardiology journal homepage: www.elsevier.com/locate/ijcard
Disclosure of negative trial results. A call for action Annalisa Capuano a,⁎, Andrew J.S. Coats b, Cristina Scavone a, Francesco Rossi a, Giuseppe M.C. Rosano c,d a Second University of Naples, Campania Regional Centre for Pharmacovigilance and Pharmacoepidemiology, Department of Experimental Medicine, Section of Pharmacology L. Donatelli of Medicine and Surgery, Naples, Italy b Monash University, Australia and University of Warwick, UK c IRCCS San Raffaele, Roma, Italy d Cardiovascular & Cell Sciences Institute, St George's University of London, UK
a r t i c l e
i n f o
Article history: Received 1 June 2015 Accepted 27 June 2015 Available online 2 July 2015 Keywords: File drawer effect Clinical trial Withholding results
In the past few years the scientific community has discussed on how to tackle research misconduct. However, little progress has been made in another important and relevant issue that can lead to incorrect assessments to be on the clinical efficacy of treatments, the reporting of negative clinical trials. Today the ethical scientific community is confronted with two urgent tasks: to ensure that negative clinical trial results are published and to prevent such results ending up filed “in a drawer”. The “file-drawer effect” influences publication of research results more than rejection by journals [1]. The “file drawer effect” is the nonreporting of results, often negative or neutral, from clinical trials in a specific area of research. This phenomenon is of extreme relevance as studies not published could potentially give different results from those that are published and alter the overall perception of the results of a treatment in a given treatment area [1]. It may even be possible that a given treatment is ineffective but that is only those of studies that by chance show that a statistically significant result is published, while the remaining trials where the null hypothesis was not rejected are not published. This will lead to the endorsement of the benefit of the treatment even though a true benefit is not there. There is evidence to support that editors tend to reject research where the results are not statistically significant. Furthermore, it has been suggested that clinical trials with significant positive results had
⁎ Corresponding author. E-mail address: [email protected] (A. Capuano).
http://dx.doi.org/10.1016/j.ijcard.2015.06.157 0167-5273/© 2015 Elsevier Ireland Ltd. All rights reserved.
a higher probability of being included in meta-analyses than studies showing negative results [2]. Dickersin investigated the association between trial characteristics, findings, and publication of NIH funded trials and found that trials with “significant” results were 12 folds more likely to be published than those showing “non-significant” results. Of interest, no other factor was positively associated with publication [3]. The trend to publication bias, the so called “positive-outcome bias”, is very common because reviewers and editors are less likely to publish negative results and findings that are not statistically significant. Issues related to publication bias enclose under-reporting research and hyperclaiming and concern the investigators who manipulate the actual results of their research in order to get published [1,3]. Withholding results are an example of misconduct research which can result in the use of drug therapies that are harmful, ineffective, unethical, and expensive; since clinical research that doesn't confirm the expected benefits of those treatments and their related clinical trial data remain unpublished [4]. Moreover, underreporting research is not only restricted to the investigator's failure to report negative results, but also to the failure to report the results with enough detail to allow a full assessment of their validity. In order to improve the quality of published data and, at the same time, allow a full understanding of research findings, it has been suggested that investigators should in particular report their methods and analyses; in that way, reviewers can draft reasonable conclusions about the external validity of their research findings [5]. Of course, the majority of clinical trials are sponsored by the pharmaceutical industry and funding sources can exert considerable pressure on scientists to suppress or minimize inconclusive results and reports of adverse effects. The key to avoiding underreporting and hyper-claiming research should be to motivate investigators to submit all good quality studies for publication and, to contribute to the education of journal editors, reviewers, and investigators alike that null or negative findings are as important as positive ones. For all the above mentioned reasons, the only consideration underlying the decision to publish a trial should be the quality of the study design and conduct [6]. In truth, it seems that the scientific literature is mainly oriented towards positive results, some of which by such a selection may prove to be misleading, while the negative result trails that reflect truly negative treatment effects are vanishing. Thus the “ethical scientific community” has a responsibility to change this situation.
48
A. Capuano et al. / International Journal of Cardiology 198 (2015) 47–48
In this context, on the 14th of April 2015, in a press statement, WHO said that “the withholding of results from clinical trials is unethical” [7]. The WHO proposed that the main results of every clinical study should be submitted to a journal within 12 months after data collection and that the studies should be published within 24 months. WHO have also advised researchers to consult the CONSORT Statement, an evidencebased minimum set of recommendations for reporting the results of randomized clinical trials. CONSORT Statement can help the authors to prepare reports of trial findings, and direct them to complete and transparent reporting which allow critical appraisal and interpretation. Furthermore, WHO ask that the “key outcomes” (number of patients enrolled, key results, adverse events) are detectable in a clinical trials registry by 12 months after the end of the study [8]. To date, one of the most significant concerns of clinical trial databases, such as trail registries, consists in the difficulty of reporting research findings held by the sponsor. Therefore, the data are often not available for public access. Unfortunately, only two registries currently existing (Clinicaltrials.gov and the EU Clinical Trials Register) work to ensure clinical trial documentation [7,8]. In order to avoid “withholding results”, there is an urgent need to improve the policies of scientific Journals for publication and access to clinical trial data.
Conflict of interest The authors report no relationships that could be construed as a conflict of interest.
References [1] K. Dickersin, Y.I. Min, C.L. Meinert, Factors influencing publication of research results. Follow-up of applications submitted to two institutional review boards, JAMA 267 (1992) 374–378. [2] M. Kicinski, Publication bias in recent meta-analyses, PLoS One 8 (2013) e81823. [3] K. Dickersin, Publication bias: recognizing the problem, understanding its origins and scope, and preventing harm, in: H. Rothstein, A. Sutton, M. Borenstein (Eds.), Handbook of Publication Bias, Wiley, New York, 2004. [4] I. Chalmers, In the dark: drug companies should be forced to publish all the results of clinical trials, New Sci. 181 (2004) 19. [5] B.D. Sales, S. Folkman, Ethics in Research with Human Participants, American Psychological Association, Washington, DC, 2000. 13–26. [6] A. Ioannou, Publication bias: a threat to the objective report of research results, ERIC Database, 2009. [7] World Health Organization, WHO statement on public disclosure of clinical trial results, Accessed 14 April 2015. http://www.who.int/ictrp/results/reporting. [8] S.M. Vasee, K. Ghassan, K.S. Vannice, M.P. Kieny, Rationale for WHO's new position calling for prompt reporting and public disclosure of interventional clinical trial results, 12 (2015) e1001819.
Contents lists available at ScienceDirect
International Journal of Cardiology journal homepage: www.elsevier.com/locate/ijcard
Disclosure of negative trial results. A call for action Annalisa Capuano a,⁎, Andrew J.S. Coats b, Cristina Scavone a, Francesco Rossi a, Giuseppe M.C. Rosano c,d a Second University of Naples, Campania Regional Centre for Pharmacovigilance and Pharmacoepidemiology, Department of Experimental Medicine, Section of Pharmacology L. Donatelli of Medicine and Surgery, Naples, Italy b Monash University, Australia and University of Warwick, UK c IRCCS San Raffaele, Roma, Italy d Cardiovascular & Cell Sciences Institute, St George's University of London, UK
a r t i c l e
i n f o
Article history: Received 1 June 2015 Accepted 27 June 2015 Available online 2 July 2015 Keywords: File drawer effect Clinical trial Withholding results
In the past few years the scientific community has discussed on how to tackle research misconduct. However, little progress has been made in another important and relevant issue that can lead to incorrect assessments to be on the clinical efficacy of treatments, the reporting of negative clinical trials. Today the ethical scientific community is confronted with two urgent tasks: to ensure that negative clinical trial results are published and to prevent such results ending up filed “in a drawer”. The “file-drawer effect” influences publication of research results more than rejection by journals [1]. The “file drawer effect” is the nonreporting of results, often negative or neutral, from clinical trials in a specific area of research. This phenomenon is of extreme relevance as studies not published could potentially give different results from those that are published and alter the overall perception of the results of a treatment in a given treatment area [1]. It may even be possible that a given treatment is ineffective but that is only those of studies that by chance show that a statistically significant result is published, while the remaining trials where the null hypothesis was not rejected are not published. This will lead to the endorsement of the benefit of the treatment even though a true benefit is not there. There is evidence to support that editors tend to reject research where the results are not statistically significant. Furthermore, it has been suggested that clinical trials with significant positive results had
⁎ Corresponding author. E-mail address: [email protected] (A. Capuano).
http://dx.doi.org/10.1016/j.ijcard.2015.06.157 0167-5273/© 2015 Elsevier Ireland Ltd. All rights reserved.
a higher probability of being included in meta-analyses than studies showing negative results [2]. Dickersin investigated the association between trial characteristics, findings, and publication of NIH funded trials and found that trials with “significant” results were 12 folds more likely to be published than those showing “non-significant” results. Of interest, no other factor was positively associated with publication [3]. The trend to publication bias, the so called “positive-outcome bias”, is very common because reviewers and editors are less likely to publish negative results and findings that are not statistically significant. Issues related to publication bias enclose under-reporting research and hyperclaiming and concern the investigators who manipulate the actual results of their research in order to get published [1,3]. Withholding results are an example of misconduct research which can result in the use of drug therapies that are harmful, ineffective, unethical, and expensive; since clinical research that doesn't confirm the expected benefits of those treatments and their related clinical trial data remain unpublished [4]. Moreover, underreporting research is not only restricted to the investigator's failure to report negative results, but also to the failure to report the results with enough detail to allow a full assessment of their validity. In order to improve the quality of published data and, at the same time, allow a full understanding of research findings, it has been suggested that investigators should in particular report their methods and analyses; in that way, reviewers can draft reasonable conclusions about the external validity of their research findings [5]. Of course, the majority of clinical trials are sponsored by the pharmaceutical industry and funding sources can exert considerable pressure on scientists to suppress or minimize inconclusive results and reports of adverse effects. The key to avoiding underreporting and hyper-claiming research should be to motivate investigators to submit all good quality studies for publication and, to contribute to the education of journal editors, reviewers, and investigators alike that null or negative findings are as important as positive ones. For all the above mentioned reasons, the only consideration underlying the decision to publish a trial should be the quality of the study design and conduct [6]. In truth, it seems that the scientific literature is mainly oriented towards positive results, some of which by such a selection may prove to be misleading, while the negative result trails that reflect truly negative treatment effects are vanishing. Thus the “ethical scientific community” has a responsibility to change this situation.
48
A. Capuano et al. / International Journal of Cardiology 198 (2015) 47–48
In this context, on the 14th of April 2015, in a press statement, WHO said that “the withholding of results from clinical trials is unethical” [7]. The WHO proposed that the main results of every clinical study should be submitted to a journal within 12 months after data collection and that the studies should be published within 24 months. WHO have also advised researchers to consult the CONSORT Statement, an evidencebased minimum set of recommendations for reporting the results of randomized clinical trials. CONSORT Statement can help the authors to prepare reports of trial findings, and direct them to complete and transparent reporting which allow critical appraisal and interpretation. Furthermore, WHO ask that the “key outcomes” (number of patients enrolled, key results, adverse events) are detectable in a clinical trials registry by 12 months after the end of the study [8]. To date, one of the most significant concerns of clinical trial databases, such as trail registries, consists in the difficulty of reporting research findings held by the sponsor. Therefore, the data are often not available for public access. Unfortunately, only two registries currently existing (Clinicaltrials.gov and the EU Clinical Trials Register) work to ensure clinical trial documentation [7,8]. In order to avoid “withholding results”, there is an urgent need to improve the policies of scientific Journals for publication and access to clinical trial data.
Conflict of interest The authors report no relationships that could be construed as a conflict of interest.
References [1] K. Dickersin, Y.I. Min, C.L. Meinert, Factors influencing publication of research results. Follow-up of applications submitted to two institutional review boards, JAMA 267 (1992) 374–378. [2] M. Kicinski, Publication bias in recent meta-analyses, PLoS One 8 (2013) e81823. [3] K. Dickersin, Publication bias: recognizing the problem, understanding its origins and scope, and preventing harm, in: H. Rothstein, A. Sutton, M. Borenstein (Eds.), Handbook of Publication Bias, Wiley, New York, 2004. [4] I. Chalmers, In the dark: drug companies should be forced to publish all the results of clinical trials, New Sci. 181 (2004) 19. [5] B.D. Sales, S. Folkman, Ethics in Research with Human Participants, American Psychological Association, Washington, DC, 2000. 13–26. [6] A. Ioannou, Publication bias: a threat to the objective report of research results, ERIC Database, 2009. [7] World Health Organization, WHO statement on public disclosure of clinical trial results, Accessed 14 April 2015. http://www.who.int/ictrp/results/reporting. [8] S.M. Vasee, K. Ghassan, K.S. Vannice, M.P. Kieny, Rationale for WHO's new position calling for prompt reporting and public disclosure of interventional clinical trial results, 12 (2015) e1001819.
