Downloaded from http://heart.bmj.com/ on June 1, 2015 - Published by group.bmj.com
Heart Online First, published on January 30, 2015 as 10.1136/heartjnl-2015-307458 Editorial
Research integrity: we are all accountable Catherine M Otto,1 Steven M Bradley,2,3 David E Newby4 Advances in medical care depend on rigorous, reliable and reproducible basic, clinical and translational research. The scientific method used in medical research is predicated on testing a hypothesis using a well-thought-out study design with meticulous data recording and appropriate statistical analysis of the results. In addition, core scientific research values include truthful, objective, unbiased and complete reporting of the study results. Unfortunately, the veracity of research findings is sometimes called into question, typically after publication in a peer-reviewed journal. Sometimes the concern is simply suboptimal data presentation or statistical analysis, but other cases involve dubious or even fraudulent data.1–3 This type of research misconduct harms the scientific community and may harm patient care. Preventing research misconduct requires active participation and interaction between the researcher, the research institution and the medical journal.
RESEARCH MISCONDUCT We repeatedly see examples of research misconduct in clinical cardiology research, resulting in retraction of papers and potentially leading to suboptimal, or even detrimental, patient care. From the point of view of the research community, it is not enough to retract publications after scientific misconduct has been identified. Instead, we must prevent research misconduct—defined broadly to include publication of fraudulent data, inadequate record keeping, lack of full patient consent, plagiarism and poorly designed research protocols. Reviewing where things went wrong can be helpful, but we also need new strategies to prevent publication of research that fails to meet the highest standards of integrity. Research misconduct wastes research funding, damages the 1
Division of Cardiology, University of Washington School of Medicine, Seattle, Washington, USA; 2VA Eastern Colorado Health Care System, Denver, Colorado, USA; 3University of Colorado School of Medicine, Aurora, Colorado, USA; 4Centre for Cardiovascular Science, University of Edinburgh, Edinburgh, UK Correspondence to Professor Catherine M Otto, Division of Cardiology, University of Washington School of Medicine, Box 356422, Seattle, WA 98195, USA; [email protected]
standing of the scientific community, undermines the altruistic contribution of subjects enrolled in the study and endangers patients subsequently receiving treatment based on fraudulent data.4
ENSURING RESEARCH INTEGRITY The basic principles of research integrity have been clearly articulated in numerous formats by professional organisations, research institutions and individual thought leaders. We know what we need to do; the challenge is how to do it. A common reaction to this type of problem is to create more rules and regulations. The downside of this approach is an increased administrative burden that might stifle innovative research, as well as incurring time and costs for documentation and review. In addition, we are sceptical that more rules and regulations will accomplish our goals; clearly most people involved in scientific misconduct do not follow the rules anyway and often repeatedly engage in scientific misconduct.5
ROLE OF THE RESEARCH INSTITUTION There is no simple solution to ensuring research integrity but a big first step would be to consider the researcher and the research institution equally responsible when scientific misconduct occurs. Research institutions should provide education and training in research ethics and the principles of research integrity. The institution also is expected to investigate possible scientific misconduct, impose corrective measures and ensure that fraudulent data are retracted promptly. However, this is not enough. Instead of punishing individual researchers, we need to minimise the likelihood of scientific misconduct and accept institutional, as well as individual, responsibility when misconduct does occur. High-quality truthful research results from a never-ending iterative cycle of education, training, discussion, oversight and review. This iterative cycle occurs at multiple levels, starting with the research mentor–trainee relationship and extending to research group internal reviews, institutional programmes and oversight, and presentations of preliminary results at national meetings. Hopefully, any incipient cases of research misconduct or poor Otto CM, et al. Heart Month 2015 Vol 0 No 0
data documentations will be identified early by the research mentor, members of the research team, coauthors of the paper, the research group leader or peer researchers when preliminary data are presented. Research institutions should be undertaking periodic audits or internal reviews of ongoing research to prevent misconduct and elevate research quality. Clinical trial audit is a requirement of study sponsors by national competent authorities, such as the Medicines and Healthcare Products Regulatory Authority (MHRA) or the Food and Drug Administration (FDA). This independent audit and data verification is a mark of a high-quality study that should be valued by investigators, the academic community and journals. Accreditation of research laboratories or groups might be another option that would require consensus and compatibility with the principles of Good Clinical Practice. Ultimately, the principles of research integrity should be so fully embedded in our institutional culture that individual researchers follow these principles as a matter of course. We are all responsible for ensuring that research misconduct does not occur and for speaking up when we are concerned that research is not being conducted properly.
DUE PROCESS FOR ALLEGED SCIENTIFIC MISCONDUCT On the other hand, there is a potential downside to encouraging people to speak up about suspected research misconduct. Sometimes misguided individuals with incomplete or incorrect information will raise unjustified questions. In the worstcase scenario, malicious individuals will make false accusations. For an innocent researcher, the mere suggestion of scientific misconduct can have a detrimental effect on future career and research opportunities.6 Further, the time, cost and stress of responding to a false accusation detract from ongoing investigations. Research institutions must provide a fair, timely and transparent process for evaluating potential scientific misconduct. In addition, the results of those deliberations must be widely disseminated and available to the public. This protects the innocent and also publicises individual and institutional misconduct that can be linked to previous publications.
REWARD SCIENTIFIC INTEGRITY We need to move from a purely reactive approach that punishes those who are found guilty of scientific misconduct to a proactive approach that rewards those 1
Copyright Article author (or their employer) 2015. Produced by BMJ Publishing Group Ltd (& BCS) under licence.
Downloaded from http://heart.bmj.com/ on June 1, 2015 - Published by group.bmj.com
Editorial who are good citizens of the scientific community.7 Academic promotion should be based on the quality and integrity of their scientific work. Currently, a key measure of scientific quality is publication in high-impact journals. These journals often require exciting or controversial findings to reach a high enough priority for publication. As a result, studies with suboptimal design and flaws in execution might still be published in a high-impact journal if the findings are news worthy. In contrast, a scientifically rigorous, carefully thought through and perfectly executed study with a neutral or unexciting result is unlikely to be accepted by a high-impact journal. This tension may lead investigators to inflate the importance of their work to achieve publication thresholds needed to support career aspirations. However, if the findings cannot be replicated or if a substantial mistake is identified, this short-term ‘gain’ is rapidly lost and is damaging to the long-term reputation of the investigator, the institution and the journal. Subscribers to journals want to read about exciting findings but will readily complain when such reports turn out to be false, or will become disinterested if journals predominantly publish neutral or confirmatory findings. This social paradox is not unique to scientific media but is one of the most difficult issues for any journal, particularly for higher impact journals like Heart. Open access publication helps but how do we give more kudos for work that is performed to a very high-quality standard but without exciting findings? Somehow, we need to find a way to provide equal recognition to researchers who perform highquality research regardless of the study findings or who carefully replicate previous findings. The integrity of medical research also depends on a scientific community with each individual contributing to the peer-review process, providing editorial comments, participating in national and international scientific meetings and being involved in post-publication discussions in a variety of formats. All of these activities should be valued at the institutional level
2
and should be considered core elements of an academic position.
ROLE OF THE MEDICAL JOURNAL Publication of medical research relies on trust in the scientists who submit papers and the institutions where the research was conducted. Peer review ensures the presented research is relevant, important and of high scientific calibre. In addition, editors and reviewers look for data inconsistencies, determine if the data are presented clearly with appropriate statistical analysis, require precision in language and help ensure the conclusions are supported by the research findings. Plagiarism detection software can detect overlap in wording with previous publications. However, editors and reviewers are unable to directly verify whether the data are true. Journals rely on authors to tell the truth and on institutions to provide appropriate approval and oversight of research protocols: an issue of particular importance for clinical research, often with multiple institutions involved but none taking primary responsibility.
CAN JOURNALS DO MORE? Even with a culture of research integrity and optimal institutional oversight, a few instances of research misconduct inevitably will make it through all these layers of review and be submitted to scientific journals for publication. Unfortunately a few of these papers will go through the review process and be published because journals assume truthfulness by authors and appropriate oversight by the research institution. Once a journal becomes aware of potential scientific misconduct related to a published paper, the journal again must rely on the research institution to investigate and verify or dismiss those concerns. Options then range from simple correction of honest errors to retraction of papers with fraudulent data, including editorial commentary to indicate what was retracted and the rationale for this action. We can do even more. Although perhaps not ideal, pre-publication peer review is the best current option and we should seek to strengthen this process, including formal statistical review before publication. Post-publication peer review
includes communications between readers and authors in the form of E-letters or other online venues. Authors have the responsibility to respond to reader’s questions and to provide data supporting their research conclusions when requested. Online data repositories will further improve transparency, allowing evaluation of the original data and additional data analysis. Journals are not the best option for developing and maintaining data repositories but could provide a link from the publication to the data source. We also now have options, such as our sister open access journal Open Heart, for publication of well-designed studies with negative results or studies that replicate previous research findings, which should help reduce the publication bias against these types of studies. Finally, as we engage in medical research, we each need to strive constantly towards the highest standards of honesty and integrity and to articulate repeatedly those principles to those around us. Contributors All authors contributed to writing the editorial and all have seen and approved the final version. Competing interests None. Provenance and peer review Commissioned; internally peer reviewed. To cite Otto CM, Bradley SM, Newby DE. Heart Published Online First: [ please include Day Month Year] doi:10.1136/heartjnl-2015-307458 Heart 2015;0:1–2. doi:10.1136/heartjnl-2015-307458
REFERENCES 1 2
3
4
5
6 7
Braunwald E. On analysing scientific fraud. Nature 1987;325:215–16. Lüscher TF. The codex of science: honesty, precision, and truth—and its violations. Eur Heart J 2013;34:1018–23. Cole GD, Francis DP. The challenge of delivering reliable science and guidelines: opportunities for all to participate. Eur Heart J 2014;35:2435–40. Steen RG. Retractions in the scientific literature: do authors deliberately commit research fraud? J Med Ethics 2011;37:113–17. Stern AM, Casadevall A, Steen RG, et al. Financial costs and personal consequences of research misconduct resulting in retracted publications. Elife 2014;3:e02956. Kennedy D. The Baltimore affair: let’s not forget what went wrong. Nat Med 1996;2:843–4. Ioannidis JP. How to make more published research true. PLoS Med 2014;11:e1001747.
Otto CM, et al. Heart Month 2015 Vol 0 No 0
Downloaded from http://heart.bmj.com/ on June 1, 2015 - Published by group.bmj.com
Research integrity: we are all accountable Catherine M Otto, Steven M Bradley and David E Newby Heart published online January 30, 2015
Updated information and services can be found at: http://heart.bmj.com/content/early/2015/01/30/heartjnl-2015-307458
These include:
References Email alerting service
This article cites 7 articles, 3 of which you can access for free at: http://heart.bmj.com/content/early/2015/01/30/heartjnl-2015-307458 #BIBL Receive free email alerts when new articles cite this article. Sign up in the box at the top right corner of the online article.
Notes
To request permissions go to: http://group.bmj.com/group/rights-licensing/permissions To order reprints go to: http://journals.bmj.com/cgi/reprintform To subscribe to BMJ go to: http://group.bmj.com/subscribe/
Heart Online First, published on January 30, 2015 as 10.1136/heartjnl-2015-307458 Editorial
Research integrity: we are all accountable Catherine M Otto,1 Steven M Bradley,2,3 David E Newby4 Advances in medical care depend on rigorous, reliable and reproducible basic, clinical and translational research. The scientific method used in medical research is predicated on testing a hypothesis using a well-thought-out study design with meticulous data recording and appropriate statistical analysis of the results. In addition, core scientific research values include truthful, objective, unbiased and complete reporting of the study results. Unfortunately, the veracity of research findings is sometimes called into question, typically after publication in a peer-reviewed journal. Sometimes the concern is simply suboptimal data presentation or statistical analysis, but other cases involve dubious or even fraudulent data.1–3 This type of research misconduct harms the scientific community and may harm patient care. Preventing research misconduct requires active participation and interaction between the researcher, the research institution and the medical journal.
RESEARCH MISCONDUCT We repeatedly see examples of research misconduct in clinical cardiology research, resulting in retraction of papers and potentially leading to suboptimal, or even detrimental, patient care. From the point of view of the research community, it is not enough to retract publications after scientific misconduct has been identified. Instead, we must prevent research misconduct—defined broadly to include publication of fraudulent data, inadequate record keeping, lack of full patient consent, plagiarism and poorly designed research protocols. Reviewing where things went wrong can be helpful, but we also need new strategies to prevent publication of research that fails to meet the highest standards of integrity. Research misconduct wastes research funding, damages the 1
Division of Cardiology, University of Washington School of Medicine, Seattle, Washington, USA; 2VA Eastern Colorado Health Care System, Denver, Colorado, USA; 3University of Colorado School of Medicine, Aurora, Colorado, USA; 4Centre for Cardiovascular Science, University of Edinburgh, Edinburgh, UK Correspondence to Professor Catherine M Otto, Division of Cardiology, University of Washington School of Medicine, Box 356422, Seattle, WA 98195, USA; [email protected]
standing of the scientific community, undermines the altruistic contribution of subjects enrolled in the study and endangers patients subsequently receiving treatment based on fraudulent data.4
ENSURING RESEARCH INTEGRITY The basic principles of research integrity have been clearly articulated in numerous formats by professional organisations, research institutions and individual thought leaders. We know what we need to do; the challenge is how to do it. A common reaction to this type of problem is to create more rules and regulations. The downside of this approach is an increased administrative burden that might stifle innovative research, as well as incurring time and costs for documentation and review. In addition, we are sceptical that more rules and regulations will accomplish our goals; clearly most people involved in scientific misconduct do not follow the rules anyway and often repeatedly engage in scientific misconduct.5
ROLE OF THE RESEARCH INSTITUTION There is no simple solution to ensuring research integrity but a big first step would be to consider the researcher and the research institution equally responsible when scientific misconduct occurs. Research institutions should provide education and training in research ethics and the principles of research integrity. The institution also is expected to investigate possible scientific misconduct, impose corrective measures and ensure that fraudulent data are retracted promptly. However, this is not enough. Instead of punishing individual researchers, we need to minimise the likelihood of scientific misconduct and accept institutional, as well as individual, responsibility when misconduct does occur. High-quality truthful research results from a never-ending iterative cycle of education, training, discussion, oversight and review. This iterative cycle occurs at multiple levels, starting with the research mentor–trainee relationship and extending to research group internal reviews, institutional programmes and oversight, and presentations of preliminary results at national meetings. Hopefully, any incipient cases of research misconduct or poor Otto CM, et al. Heart Month 2015 Vol 0 No 0
data documentations will be identified early by the research mentor, members of the research team, coauthors of the paper, the research group leader or peer researchers when preliminary data are presented. Research institutions should be undertaking periodic audits or internal reviews of ongoing research to prevent misconduct and elevate research quality. Clinical trial audit is a requirement of study sponsors by national competent authorities, such as the Medicines and Healthcare Products Regulatory Authority (MHRA) or the Food and Drug Administration (FDA). This independent audit and data verification is a mark of a high-quality study that should be valued by investigators, the academic community and journals. Accreditation of research laboratories or groups might be another option that would require consensus and compatibility with the principles of Good Clinical Practice. Ultimately, the principles of research integrity should be so fully embedded in our institutional culture that individual researchers follow these principles as a matter of course. We are all responsible for ensuring that research misconduct does not occur and for speaking up when we are concerned that research is not being conducted properly.
DUE PROCESS FOR ALLEGED SCIENTIFIC MISCONDUCT On the other hand, there is a potential downside to encouraging people to speak up about suspected research misconduct. Sometimes misguided individuals with incomplete or incorrect information will raise unjustified questions. In the worstcase scenario, malicious individuals will make false accusations. For an innocent researcher, the mere suggestion of scientific misconduct can have a detrimental effect on future career and research opportunities.6 Further, the time, cost and stress of responding to a false accusation detract from ongoing investigations. Research institutions must provide a fair, timely and transparent process for evaluating potential scientific misconduct. In addition, the results of those deliberations must be widely disseminated and available to the public. This protects the innocent and also publicises individual and institutional misconduct that can be linked to previous publications.
REWARD SCIENTIFIC INTEGRITY We need to move from a purely reactive approach that punishes those who are found guilty of scientific misconduct to a proactive approach that rewards those 1
Copyright Article author (or their employer) 2015. Produced by BMJ Publishing Group Ltd (& BCS) under licence.
Downloaded from http://heart.bmj.com/ on June 1, 2015 - Published by group.bmj.com
Editorial who are good citizens of the scientific community.7 Academic promotion should be based on the quality and integrity of their scientific work. Currently, a key measure of scientific quality is publication in high-impact journals. These journals often require exciting or controversial findings to reach a high enough priority for publication. As a result, studies with suboptimal design and flaws in execution might still be published in a high-impact journal if the findings are news worthy. In contrast, a scientifically rigorous, carefully thought through and perfectly executed study with a neutral or unexciting result is unlikely to be accepted by a high-impact journal. This tension may lead investigators to inflate the importance of their work to achieve publication thresholds needed to support career aspirations. However, if the findings cannot be replicated or if a substantial mistake is identified, this short-term ‘gain’ is rapidly lost and is damaging to the long-term reputation of the investigator, the institution and the journal. Subscribers to journals want to read about exciting findings but will readily complain when such reports turn out to be false, or will become disinterested if journals predominantly publish neutral or confirmatory findings. This social paradox is not unique to scientific media but is one of the most difficult issues for any journal, particularly for higher impact journals like Heart. Open access publication helps but how do we give more kudos for work that is performed to a very high-quality standard but without exciting findings? Somehow, we need to find a way to provide equal recognition to researchers who perform highquality research regardless of the study findings or who carefully replicate previous findings. The integrity of medical research also depends on a scientific community with each individual contributing to the peer-review process, providing editorial comments, participating in national and international scientific meetings and being involved in post-publication discussions in a variety of formats. All of these activities should be valued at the institutional level
2
and should be considered core elements of an academic position.
ROLE OF THE MEDICAL JOURNAL Publication of medical research relies on trust in the scientists who submit papers and the institutions where the research was conducted. Peer review ensures the presented research is relevant, important and of high scientific calibre. In addition, editors and reviewers look for data inconsistencies, determine if the data are presented clearly with appropriate statistical analysis, require precision in language and help ensure the conclusions are supported by the research findings. Plagiarism detection software can detect overlap in wording with previous publications. However, editors and reviewers are unable to directly verify whether the data are true. Journals rely on authors to tell the truth and on institutions to provide appropriate approval and oversight of research protocols: an issue of particular importance for clinical research, often with multiple institutions involved but none taking primary responsibility.
CAN JOURNALS DO MORE? Even with a culture of research integrity and optimal institutional oversight, a few instances of research misconduct inevitably will make it through all these layers of review and be submitted to scientific journals for publication. Unfortunately a few of these papers will go through the review process and be published because journals assume truthfulness by authors and appropriate oversight by the research institution. Once a journal becomes aware of potential scientific misconduct related to a published paper, the journal again must rely on the research institution to investigate and verify or dismiss those concerns. Options then range from simple correction of honest errors to retraction of papers with fraudulent data, including editorial commentary to indicate what was retracted and the rationale for this action. We can do even more. Although perhaps not ideal, pre-publication peer review is the best current option and we should seek to strengthen this process, including formal statistical review before publication. Post-publication peer review
includes communications between readers and authors in the form of E-letters or other online venues. Authors have the responsibility to respond to reader’s questions and to provide data supporting their research conclusions when requested. Online data repositories will further improve transparency, allowing evaluation of the original data and additional data analysis. Journals are not the best option for developing and maintaining data repositories but could provide a link from the publication to the data source. We also now have options, such as our sister open access journal Open Heart, for publication of well-designed studies with negative results or studies that replicate previous research findings, which should help reduce the publication bias against these types of studies. Finally, as we engage in medical research, we each need to strive constantly towards the highest standards of honesty and integrity and to articulate repeatedly those principles to those around us. Contributors All authors contributed to writing the editorial and all have seen and approved the final version. Competing interests None. Provenance and peer review Commissioned; internally peer reviewed. To cite Otto CM, Bradley SM, Newby DE. Heart Published Online First: [ please include Day Month Year] doi:10.1136/heartjnl-2015-307458 Heart 2015;0:1–2. doi:10.1136/heartjnl-2015-307458
REFERENCES 1 2
3
4
5
6 7
Braunwald E. On analysing scientific fraud. Nature 1987;325:215–16. Lüscher TF. The codex of science: honesty, precision, and truth—and its violations. Eur Heart J 2013;34:1018–23. Cole GD, Francis DP. The challenge of delivering reliable science and guidelines: opportunities for all to participate. Eur Heart J 2014;35:2435–40. Steen RG. Retractions in the scientific literature: do authors deliberately commit research fraud? J Med Ethics 2011;37:113–17. Stern AM, Casadevall A, Steen RG, et al. Financial costs and personal consequences of research misconduct resulting in retracted publications. Elife 2014;3:e02956. Kennedy D. The Baltimore affair: let’s not forget what went wrong. Nat Med 1996;2:843–4. Ioannidis JP. How to make more published research true. PLoS Med 2014;11:e1001747.
Otto CM, et al. Heart Month 2015 Vol 0 No 0
Downloaded from http://heart.bmj.com/ on June 1, 2015 - Published by group.bmj.com
Research integrity: we are all accountable Catherine M Otto, Steven M Bradley and David E Newby Heart published online January 30, 2015
Updated information and services can be found at: http://heart.bmj.com/content/early/2015/01/30/heartjnl-2015-307458
These include:
References Email alerting service
This article cites 7 articles, 3 of which you can access for free at: http://heart.bmj.com/content/early/2015/01/30/heartjnl-2015-307458 #BIBL Receive free email alerts when new articles cite this article. Sign up in the box at the top right corner of the online article.
Notes
To request permissions go to: http://group.bmj.com/group/rights-licensing/permissions To order reprints go to: http://journals.bmj.com/cgi/reprintform To subscribe to BMJ go to: http://group.bmj.com/subscribe/
