PERSPECTIVE
Addressing conflict of interest in non-pharmacological research Linked Comment: Citrome. Int J Clin Pract 2015; 69: 267–8. Linked Comment: Mayes et al. Int J Clin Pract 2015; 69: 384–5. Linked Comment: Stossel et al. Int J Clin Pract 2015; 69: 385–6.
evaluate these interventions through research methodology and publication. Conflict arises in such situations when the same researchers (with clear vested in interests in success) also hold multiple roles in the design, evaluation and reporting of a N-Ph intervention. While this may appear unduly pessimistic, studies in which researchers both develop and evaluate interventions tend to have larger effect sizes (11) and studies with these conflicts of interest are more likely to report positive findings (11). This potentially biases the published scientific evidence base. Conflicts of interest create conscious and unconscious biases in how research is managed. Territorial conflicts can affect how data from studies by other researchers are interpreted and handled in the scientific discourse. Undue credence can be given to studies based on the perceived status of an individual or team (12), or their alignment with other researchers in a field. Unsupported findings can be arbitrarily dismissed for methodological reasons while convenient findings are championed as provision of confirmative proof, irrespective of method or quality. These conflicts of interest can shape researchers’ basic assumptions and influence the direction of subsequent research. For example, the assumption that particular ‘best types’ of interventions can be identified is common in debates about N-Ph interventions, but is also highly questionable (13). Publishing systems and their norms, disciplines, organisations and individuals inevitably frame and influence the individual researcher’s positioning of their work in the wider scientific discourse (14). A discourse that – as this illustrates – can be coloured by conflicts. To address these conflicts successfully it is important to understand why they occur.
Conflicts of interest abound in research. Yet almost exclusive attention has been paid to the conflicts arising around industry involvement in pharmacological and device research where data are routinely or deliberately withheld from the public domain (1,2), or when ghostwriters are used to mask the role of industry (3,4).
Conflict of interest arises in published research studies because personal interests conflict with values important to science: impartiality, fairness and evidencedriven progress
Conflicts also occur due to bias arising from the need for people to be successful in financial and scientific terms (5). Similarly conflicts of interest in non-pharmacological (N-Ph) research are not uncommon, and are equally damaging to the quality of evidence and the credibility of science. These conflicts are now widely publicised, having been associated with numerous so-called ‘scandals’, most notably in psychology (6). The high degree of public concern about these conflicts and scandals was featured as the front-page story of The Economist in late 2013 (5,6). Only ~20% of the N-Ph studies published in medical journals formally report conflicts (7) and the public are as concerned about these conflicts as those arising from pharmacological studies (8). How conflicts of interest are managed in N-Ph research is then important for scientists, science and society. As well as being unethical and potentially dangerous, any poor handling (real or perceived) erodes public trust in health professionals, scientists and science. How then can those involved in N-Ph research successfully address these concerns? (9).
Types of conflicts of interest in non-pharmacological research Financial and career conflicts are traditionally linked to N-Ph studies. However, conflicts in N-Ph research also arise because individuals may gain from the success of an intervention in financial or career terms (6). This can occur when the favourable results of particular studies are excessively linked to career advancement, or financial benefits arise from wider adoption of a proprietary N-Ph intervention by a healthcare provider following an affirmative study (10). Some N-Ph researchers have developed lucrative side interests through widely marketing and contracting out proprietary interventions, while simultaneously continuing to
270
How conflicts of interest occur in non-pharmacological research Concerns over the conflicts of interest in research were expressed as far back as the 19th century (15). They are inevitable in academic and clinical systems where the unit of evaluation is authorship, and contributorship is detailed and collated in ‘curriculum ª 2015 John Wiley & Sons Ltd Int J Clin Pract, March 2015, 69, 3, 270–272. doi: 10.1111/ijcp.12569
Perpective
vitae’ or a team’s progress to date. In addition to providing descriptions, the items reported in these documents represent scholarly reputation and academic capital – important currencies that strongly influence the attainment of future funds, and the progress within the complex, hierarchical organisational structures where research is conducted. These systems also reward particular types of research and results. There are especially strong incentives to submit affirmative findings because studies with statistically significant positive results are more likely to be accepted for publication (15,16) in higher impact journals (17), and subsequently cited (15). For example, one prominent researcher in social psychology skewed results to gain undue publicity and status via counterintuitive, and thus head-line grabbing, results (12). With an over reliance on statistically significant tests (18), even negative findings have been spun into positive results in up to 40% of papers (19). Such positive spins can be produced through various methodological adjustments, such as stopping data collection early or excluding outlying data using unclear or inconsistent post hoc criteria (20). In a recent survey, nearly 70% of psychology researchers noted that data collection was stopped early, based on the results of an interim analysis (21). Such techniques have been described as the ‘steroids of scientific competition, artificially enhancing performance while providing considerable latitude for rationalization and selfdeception. . .’ (21, p. 524). They encroach on a ‘grey zone’ of misconduct termed ‘questionable research practices.’(21) While such conduct is ethically wrong, its lure is understandable in a global economic climate where researchers face the real prospect of losing jobs, and when research funding is increasingly competitive and based on reputation and past ‘successes’. In N-Ph studies, conflicts also arise from the widening needs of organisations and disciplines (8,14) and the social and economic nature of scientific processes (13). Disciplines, organisations, teams and individuals have much to gain or lose relative to each other based on their research findings and funding. As individual, team and organisational reputations are developed, disciplinary territory is asserted over particular kinds of successful interventions; and funding and publications increase perceived status, reputations, and legacies. The apparent ‘success’, esteem and increased profiles of researchers are beneficial to both the individual and their place of work. Academic cultures have ‘currency’ that goes beyond the intrinsic aim to advance knowledge. Researcher’s publications, particularly those in highly respected journals have direct and indirect effects on individual career progression and also on the perceived status, power, and funding of organisations. Academic organisations are ª 2015 John Wiley & Sons Ltd Int J Clin Pract, March 2015, 69, 3, 270–272
highly attuned to media profile, return on investment, visible responsiveness to stakeholders, and public accountability for their worth (8). These organisational motivators in turn lead to increased pressure on investigators and thus to heightened potential for individual conflict of interest.
Handling conflict of interest in nonpharmacological research better Conflicts of interest in N-Ph research are inevitable but ignoring them is harmful. These conflicts are best addressed through personal and organisational values and practices that strive for balance, integrity and transparency. While legislative attempts to ensure conflicts of interest do not excessively influence science – such as the full disclosure of financial and career interests – these cannot substitute for authentic openness and fairness towards inquiry. The following ideas are suggestions on how to better address conflicts of interest in N-Ph research. First, peer-reviewed publications are a key source of prestige and reputation for researchers, their organisations and disciplines. Thus, journals can play a vital role in addressing competing interests. Most journal editors are aware of the importance of addressing conflict in the review process, and have requirements in place for disclosures of financial conflicts, contributions and funding sources. However, other steps can also address conflicts better. Negative findings should be as likely to be published as affirmative findings, assuming that the methods of studies are equally robust. Peer review can be fully blinded to reduce the conflicts that can occur during manuscript review when authors’ identities are known– a practice common to many journals, but paradoxically more absent from the highest impact journals in general medicine. Even in instances in which papers are not able to be anonymised, disclosure of reviewers’ identities to authors increases accountability and transparency arising from reviewers who may be competitors of the authors and or/those have a vested interest in rejecting the publication. Second, funding for the evaluation of interventions, particularly randomised trials, should be dependent on the use of independent assessors who undertake evaluation at arms length from those who design interventions and may benefit from study results. The use of third party evaluators mitigates potential assessment bias and provides the important assurance that authors who may gain credibility and opportunity from the success of a particular intervention were not directly also involved in study measurements. Conflict of interest arises in published research studies because personal interests conflict with values important to science: impartiality, fairness and evidence-driven progress. Understanding when and how
271
272
Perpective
these interests can conflict is vital. As with other academic processes – such as undertaking qualitative research or serving on departmental promotion committees and peer-review funding panels – the onus is predominantly on the individual to consider openly, fairly and reflexively whether and how their own individual position, interests and stances influence their contributions. Thus, thirdly, reflexivity involves being more aware of one’s own subjectivity, biases, and preconceptions, and how these could and should influence these contributions. Being reflexive involves the researcher reflecting on how data collection, analysis and interpretation are affected by their own identities, experiences, and emotions. It is important that processes and requirements in journals and organisations take account of all possible conflicts of interest in N-Ph research. However, these ultimately cannot substitute for the integrity of the individual in taking an open, accountable and fair approach to the conflicts they have, and being aware of how these influence their conduct and what then to do. To promote accountability, scientists working around N-Ph interventions should consider and confront the possibility of all kind of conflicts of interest in their debate and discourse. The values and practices fostered in young or early career scientists through socialisation and curricula often focus on technical skills, career strategy and disciplinary proficiencies. Yet with the ambition and advancement common to this sub-population, these traits may actually exacerbate
References 1 Turner EH, Matthews AM, Linardatos E, Tell RA, Rosenthal R. Selective publication of antidepressant trials and its influence on apparent efficacy. N Engl J Med 2008; 358: 252–60. 2 Tagore A. Drug promotion tactics-yet another pharma deception? Int J Clin Pract 2014; 68: 662–5. 3 Singer N. Medical Papers by Ghostwriters Pushed Therapy. New York Times 2009; 06/08/2009. 4 Barton D, Stossel T, Stell L. After 20 years, industry critics bury skeptics, despite empirical vacuum. Int J Clin Pract 2014; 68: 666–73. 5 Bekelman J, Li Y, Gross C. Scope and impact of financial conflicts of interest in biomedical research. J Am Med Assoc 2003; 289: 454–65. 6 Luscher T. Good publishing practice. Eur Heart J 2012; 33: 557–61. 7 Friedman L, Richter E. Relationship between conflict of interest and research results. J Gen Int Med 2004; 19: 51–6. 8 Bion J. Financial and intellectual conflicts of interest: confusion and clarity. Curr Opin Crit Care 2009; 15: 583–90. 9 Schneider N, Lingner H, Schwartz F. Disclosing conflicts of interest in german publications concerning health services research. BMC Health Serv Res 2007; 7: 78. 10 Petrosino A, Soydan H. The impact of program developers as evaluators on criminal recidivism: results
11
12
13
14 15
16
17
18
19
the impact of conflicts of interest on science. Training and other postgraduate programmes in the basic and human sciences should address the nature and importance of reflexivity in science and to the scientific process and the ethical importance of openness, candour and accountability around these conflicts. Organisations have powerful formal and informal roles in shaping research practices, through regulations and through culture. Counter-intuitively, creating a culture of openness around conflicts of interest in research may be a step towards more frankness among researchers about these conflicts. Working to normalise the multiple pulls researchers experience as ‘competing’ rather than ‘conflicting’ interests may remove stigma and guilt that create reluctance to report and discuss issues related to conflicts of interest in N-Ph interventions. A. M. Clark,1 A. Choby,1 K. Ainsworth,2 D. R. Thompson3 1 University of Alberta, Edmonton, AB, Canada 2 University of Glasgow, Glasgow, UK 3 Cardiovascular Research Centre, Australian Catholic University, Melbourne, Vic., Australia Correspondence to: Alexander M. Clark, Professor & Associate Dean, Level 3 ECHA, University of Alberta, Edmonton, AB, Canada T6G 1C9 Tel.: + 780 492 6764 Fax: + 780 492 2551 Email: [email protected]
from meta-analyses of experimental and quasi-experimental research. J Exp Criminol 2005; 1: 435–50. Gorman D, Conde E. Conflict of interest in the evaluation and dissemination of “model” schoolbased drug and violence prevention programs. Eval Program Plann 2007; 30: 422–9. Bartlett T. The Fraud Who Fooled (Almost) Everyone. Chronicle of Higher Education 2011; November 3, 2011, 1:37 pm. Clark AM, Thompson DR. What type of heart failure program is best? Wrong question, wrong assumption. Eur J Heart Fail 2010; 12: 1271–3. Dolby RGA. Uncertain Knowledge. London: Cambridge University Press, 1996. Fanelli D. Do pressures to publish increase scientists’ bias? an empirical support from US States data. PLoS ONE 2010; 5: e10271. Nickerson R. Confirmation bias: a ubiquitous phenomenon in many guises. Rev Gen Psychol 1998; 2: 175–220. Etter JF, Stapleton J. Citations to trials of nicotine replacement therapy were biased toward positive results and high-impact-factor journals. J Clin Epidemiol 2009; 62: 831–7. Ziliak ST, McCloskey DN. The Cult of Statistical Significance. Ann Arbor: University of Michigan Press, 2008. Boutron I, Dutton S, Ravaud P, Altman DG. Reporting and interpretation of randomized controlled tri-
als with statistically nonsignificant results for primary outcomes. JAMA 2010; 303: 2058. 20 Simmons J, Nelson L, Simonsohn U. False-positive psychology: undisclosed flexibility in data collection and analysis allows presenting anything as significant. Psychol Sci 2011; 22 (11): 1359–66. doi:10.1177/ 0956797611417632. 21 John L, Loewenstein G, Drazen P. Measuring the prevalence of questionable research practices with incentives for truth-telling. Psychol Sci 2012; 23 (5): 524–32.
Acknowledgements No financial support received.
Disclosure None to disclose.
Author Contributions The article was conceived by AMC with DRT, AC and KA providing important content at every stage. AMC guarantors the article. Paper received July 2014, accepted September 2014
ª 2015 John Wiley & Sons Ltd Int J Clin Pract, March 2015, 69, 3, 270–272
Addressing conflict of interest in non-pharmacological research Linked Comment: Citrome. Int J Clin Pract 2015; 69: 267–8. Linked Comment: Mayes et al. Int J Clin Pract 2015; 69: 384–5. Linked Comment: Stossel et al. Int J Clin Pract 2015; 69: 385–6.
evaluate these interventions through research methodology and publication. Conflict arises in such situations when the same researchers (with clear vested in interests in success) also hold multiple roles in the design, evaluation and reporting of a N-Ph intervention. While this may appear unduly pessimistic, studies in which researchers both develop and evaluate interventions tend to have larger effect sizes (11) and studies with these conflicts of interest are more likely to report positive findings (11). This potentially biases the published scientific evidence base. Conflicts of interest create conscious and unconscious biases in how research is managed. Territorial conflicts can affect how data from studies by other researchers are interpreted and handled in the scientific discourse. Undue credence can be given to studies based on the perceived status of an individual or team (12), or their alignment with other researchers in a field. Unsupported findings can be arbitrarily dismissed for methodological reasons while convenient findings are championed as provision of confirmative proof, irrespective of method or quality. These conflicts of interest can shape researchers’ basic assumptions and influence the direction of subsequent research. For example, the assumption that particular ‘best types’ of interventions can be identified is common in debates about N-Ph interventions, but is also highly questionable (13). Publishing systems and their norms, disciplines, organisations and individuals inevitably frame and influence the individual researcher’s positioning of their work in the wider scientific discourse (14). A discourse that – as this illustrates – can be coloured by conflicts. To address these conflicts successfully it is important to understand why they occur.
Conflicts of interest abound in research. Yet almost exclusive attention has been paid to the conflicts arising around industry involvement in pharmacological and device research where data are routinely or deliberately withheld from the public domain (1,2), or when ghostwriters are used to mask the role of industry (3,4).
Conflict of interest arises in published research studies because personal interests conflict with values important to science: impartiality, fairness and evidencedriven progress
Conflicts also occur due to bias arising from the need for people to be successful in financial and scientific terms (5). Similarly conflicts of interest in non-pharmacological (N-Ph) research are not uncommon, and are equally damaging to the quality of evidence and the credibility of science. These conflicts are now widely publicised, having been associated with numerous so-called ‘scandals’, most notably in psychology (6). The high degree of public concern about these conflicts and scandals was featured as the front-page story of The Economist in late 2013 (5,6). Only ~20% of the N-Ph studies published in medical journals formally report conflicts (7) and the public are as concerned about these conflicts as those arising from pharmacological studies (8). How conflicts of interest are managed in N-Ph research is then important for scientists, science and society. As well as being unethical and potentially dangerous, any poor handling (real or perceived) erodes public trust in health professionals, scientists and science. How then can those involved in N-Ph research successfully address these concerns? (9).
Types of conflicts of interest in non-pharmacological research Financial and career conflicts are traditionally linked to N-Ph studies. However, conflicts in N-Ph research also arise because individuals may gain from the success of an intervention in financial or career terms (6). This can occur when the favourable results of particular studies are excessively linked to career advancement, or financial benefits arise from wider adoption of a proprietary N-Ph intervention by a healthcare provider following an affirmative study (10). Some N-Ph researchers have developed lucrative side interests through widely marketing and contracting out proprietary interventions, while simultaneously continuing to
270
How conflicts of interest occur in non-pharmacological research Concerns over the conflicts of interest in research were expressed as far back as the 19th century (15). They are inevitable in academic and clinical systems where the unit of evaluation is authorship, and contributorship is detailed and collated in ‘curriculum ª 2015 John Wiley & Sons Ltd Int J Clin Pract, March 2015, 69, 3, 270–272. doi: 10.1111/ijcp.12569
Perpective
vitae’ or a team’s progress to date. In addition to providing descriptions, the items reported in these documents represent scholarly reputation and academic capital – important currencies that strongly influence the attainment of future funds, and the progress within the complex, hierarchical organisational structures where research is conducted. These systems also reward particular types of research and results. There are especially strong incentives to submit affirmative findings because studies with statistically significant positive results are more likely to be accepted for publication (15,16) in higher impact journals (17), and subsequently cited (15). For example, one prominent researcher in social psychology skewed results to gain undue publicity and status via counterintuitive, and thus head-line grabbing, results (12). With an over reliance on statistically significant tests (18), even negative findings have been spun into positive results in up to 40% of papers (19). Such positive spins can be produced through various methodological adjustments, such as stopping data collection early or excluding outlying data using unclear or inconsistent post hoc criteria (20). In a recent survey, nearly 70% of psychology researchers noted that data collection was stopped early, based on the results of an interim analysis (21). Such techniques have been described as the ‘steroids of scientific competition, artificially enhancing performance while providing considerable latitude for rationalization and selfdeception. . .’ (21, p. 524). They encroach on a ‘grey zone’ of misconduct termed ‘questionable research practices.’(21) While such conduct is ethically wrong, its lure is understandable in a global economic climate where researchers face the real prospect of losing jobs, and when research funding is increasingly competitive and based on reputation and past ‘successes’. In N-Ph studies, conflicts also arise from the widening needs of organisations and disciplines (8,14) and the social and economic nature of scientific processes (13). Disciplines, organisations, teams and individuals have much to gain or lose relative to each other based on their research findings and funding. As individual, team and organisational reputations are developed, disciplinary territory is asserted over particular kinds of successful interventions; and funding and publications increase perceived status, reputations, and legacies. The apparent ‘success’, esteem and increased profiles of researchers are beneficial to both the individual and their place of work. Academic cultures have ‘currency’ that goes beyond the intrinsic aim to advance knowledge. Researcher’s publications, particularly those in highly respected journals have direct and indirect effects on individual career progression and also on the perceived status, power, and funding of organisations. Academic organisations are ª 2015 John Wiley & Sons Ltd Int J Clin Pract, March 2015, 69, 3, 270–272
highly attuned to media profile, return on investment, visible responsiveness to stakeholders, and public accountability for their worth (8). These organisational motivators in turn lead to increased pressure on investigators and thus to heightened potential for individual conflict of interest.
Handling conflict of interest in nonpharmacological research better Conflicts of interest in N-Ph research are inevitable but ignoring them is harmful. These conflicts are best addressed through personal and organisational values and practices that strive for balance, integrity and transparency. While legislative attempts to ensure conflicts of interest do not excessively influence science – such as the full disclosure of financial and career interests – these cannot substitute for authentic openness and fairness towards inquiry. The following ideas are suggestions on how to better address conflicts of interest in N-Ph research. First, peer-reviewed publications are a key source of prestige and reputation for researchers, their organisations and disciplines. Thus, journals can play a vital role in addressing competing interests. Most journal editors are aware of the importance of addressing conflict in the review process, and have requirements in place for disclosures of financial conflicts, contributions and funding sources. However, other steps can also address conflicts better. Negative findings should be as likely to be published as affirmative findings, assuming that the methods of studies are equally robust. Peer review can be fully blinded to reduce the conflicts that can occur during manuscript review when authors’ identities are known– a practice common to many journals, but paradoxically more absent from the highest impact journals in general medicine. Even in instances in which papers are not able to be anonymised, disclosure of reviewers’ identities to authors increases accountability and transparency arising from reviewers who may be competitors of the authors and or/those have a vested interest in rejecting the publication. Second, funding for the evaluation of interventions, particularly randomised trials, should be dependent on the use of independent assessors who undertake evaluation at arms length from those who design interventions and may benefit from study results. The use of third party evaluators mitigates potential assessment bias and provides the important assurance that authors who may gain credibility and opportunity from the success of a particular intervention were not directly also involved in study measurements. Conflict of interest arises in published research studies because personal interests conflict with values important to science: impartiality, fairness and evidence-driven progress. Understanding when and how
271
272
Perpective
these interests can conflict is vital. As with other academic processes – such as undertaking qualitative research or serving on departmental promotion committees and peer-review funding panels – the onus is predominantly on the individual to consider openly, fairly and reflexively whether and how their own individual position, interests and stances influence their contributions. Thus, thirdly, reflexivity involves being more aware of one’s own subjectivity, biases, and preconceptions, and how these could and should influence these contributions. Being reflexive involves the researcher reflecting on how data collection, analysis and interpretation are affected by their own identities, experiences, and emotions. It is important that processes and requirements in journals and organisations take account of all possible conflicts of interest in N-Ph research. However, these ultimately cannot substitute for the integrity of the individual in taking an open, accountable and fair approach to the conflicts they have, and being aware of how these influence their conduct and what then to do. To promote accountability, scientists working around N-Ph interventions should consider and confront the possibility of all kind of conflicts of interest in their debate and discourse. The values and practices fostered in young or early career scientists through socialisation and curricula often focus on technical skills, career strategy and disciplinary proficiencies. Yet with the ambition and advancement common to this sub-population, these traits may actually exacerbate
References 1 Turner EH, Matthews AM, Linardatos E, Tell RA, Rosenthal R. Selective publication of antidepressant trials and its influence on apparent efficacy. N Engl J Med 2008; 358: 252–60. 2 Tagore A. Drug promotion tactics-yet another pharma deception? Int J Clin Pract 2014; 68: 662–5. 3 Singer N. Medical Papers by Ghostwriters Pushed Therapy. New York Times 2009; 06/08/2009. 4 Barton D, Stossel T, Stell L. After 20 years, industry critics bury skeptics, despite empirical vacuum. Int J Clin Pract 2014; 68: 666–73. 5 Bekelman J, Li Y, Gross C. Scope and impact of financial conflicts of interest in biomedical research. J Am Med Assoc 2003; 289: 454–65. 6 Luscher T. Good publishing practice. Eur Heart J 2012; 33: 557–61. 7 Friedman L, Richter E. Relationship between conflict of interest and research results. J Gen Int Med 2004; 19: 51–6. 8 Bion J. Financial and intellectual conflicts of interest: confusion and clarity. Curr Opin Crit Care 2009; 15: 583–90. 9 Schneider N, Lingner H, Schwartz F. Disclosing conflicts of interest in german publications concerning health services research. BMC Health Serv Res 2007; 7: 78. 10 Petrosino A, Soydan H. The impact of program developers as evaluators on criminal recidivism: results
11
12
13
14 15
16
17
18
19
the impact of conflicts of interest on science. Training and other postgraduate programmes in the basic and human sciences should address the nature and importance of reflexivity in science and to the scientific process and the ethical importance of openness, candour and accountability around these conflicts. Organisations have powerful formal and informal roles in shaping research practices, through regulations and through culture. Counter-intuitively, creating a culture of openness around conflicts of interest in research may be a step towards more frankness among researchers about these conflicts. Working to normalise the multiple pulls researchers experience as ‘competing’ rather than ‘conflicting’ interests may remove stigma and guilt that create reluctance to report and discuss issues related to conflicts of interest in N-Ph interventions. A. M. Clark,1 A. Choby,1 K. Ainsworth,2 D. R. Thompson3 1 University of Alberta, Edmonton, AB, Canada 2 University of Glasgow, Glasgow, UK 3 Cardiovascular Research Centre, Australian Catholic University, Melbourne, Vic., Australia Correspondence to: Alexander M. Clark, Professor & Associate Dean, Level 3 ECHA, University of Alberta, Edmonton, AB, Canada T6G 1C9 Tel.: + 780 492 6764 Fax: + 780 492 2551 Email: [email protected]
from meta-analyses of experimental and quasi-experimental research. J Exp Criminol 2005; 1: 435–50. Gorman D, Conde E. Conflict of interest in the evaluation and dissemination of “model” schoolbased drug and violence prevention programs. Eval Program Plann 2007; 30: 422–9. Bartlett T. The Fraud Who Fooled (Almost) Everyone. Chronicle of Higher Education 2011; November 3, 2011, 1:37 pm. Clark AM, Thompson DR. What type of heart failure program is best? Wrong question, wrong assumption. Eur J Heart Fail 2010; 12: 1271–3. Dolby RGA. Uncertain Knowledge. London: Cambridge University Press, 1996. Fanelli D. Do pressures to publish increase scientists’ bias? an empirical support from US States data. PLoS ONE 2010; 5: e10271. Nickerson R. Confirmation bias: a ubiquitous phenomenon in many guises. Rev Gen Psychol 1998; 2: 175–220. Etter JF, Stapleton J. Citations to trials of nicotine replacement therapy were biased toward positive results and high-impact-factor journals. J Clin Epidemiol 2009; 62: 831–7. Ziliak ST, McCloskey DN. The Cult of Statistical Significance. Ann Arbor: University of Michigan Press, 2008. Boutron I, Dutton S, Ravaud P, Altman DG. Reporting and interpretation of randomized controlled tri-
als with statistically nonsignificant results for primary outcomes. JAMA 2010; 303: 2058. 20 Simmons J, Nelson L, Simonsohn U. False-positive psychology: undisclosed flexibility in data collection and analysis allows presenting anything as significant. Psychol Sci 2011; 22 (11): 1359–66. doi:10.1177/ 0956797611417632. 21 John L, Loewenstein G, Drazen P. Measuring the prevalence of questionable research practices with incentives for truth-telling. Psychol Sci 2012; 23 (5): 524–32.
Acknowledgements No financial support received.
Disclosure None to disclose.
Author Contributions The article was conceived by AMC with DRT, AC and KA providing important content at every stage. AMC guarantors the article. Paper received July 2014, accepted September 2014
ª 2015 John Wiley & Sons Ltd Int J Clin Pract, March 2015, 69, 3, 270–272
