Ò
PAIN xxx (2014) xxx–xxx
www.elsevier.com/locate/pain
Reporting of intention-to-treat analyses in recent analgesic clinical trials: ACTTION systematic review and recommendations Jennifer S. Gewandter a,⇑, Michael P. McDermott b, Andrew McKeown a, Shannon M. Smith a, Joseph R. Pawlowski a, Joseph J. Poli a, Daniel Rothstein a, Mark R. Williams a, Shay Bujanover c, John T. Farrar d, Ian Gilron e, Nathaniel P. Katz f,g, Michael C. Rowbotham h, Dennis C. Turk i, Robert H. Dworkin a a
Department of Anesthesiology, University of Rochester School of Medicine and Dentistry, Rochester, NY, USA Departments of Biostatistics and Computational Biology and Neurology, University of Rochester School of Medicine and Dentistry, Rochester, NY, USA c Depomed, Inc., Newark, CA, USA d University of Pennsylvania, Philadelphia, PA, USA e Queen’s University, Kingston, Ontario, Canada f Analgesic Solutions, Natick, MA, USA g Tufts University, Boston, MA, USA h California Pacific Medical Center, San Francisco, CA, USA i Department of Anesthesiology and Pain Medicine, University of Washington, Seattle, WA, USA b
Sponsorships or competing interests that may be relevant to content are disclosed at the end of this article.
a r t i c l e
i n f o
Article history: Received 25 June 2014 Received in revised form 8 September 2014 Accepted 29 September 2014
Keywords: ACTTION Intention to treat Systematic review
a b s t r a c t The intention-to-treat (ITT) principle states that all subjects in a randomized clinical trial (RCT) should be analyzed in the group to which they were assigned, regardless of compliance with assigned treatment. Analyses performed according to the ITT principle preserve the benefits of randomization and are recommended by regulators and statisticians for analyses of RCTs. The objective of this study was to determine the frequency with which publications of analgesic RCTs in 3 major pain journals report an ITT analysis and the percentage of the author-declared ITT analyses that include all randomized subjects and thereby fulfill the most common interpretation of the ITT principle. RCTs investigating noninvasive, pharmacologic and interventional (eg, nerve blocks, implantable pumps, spinal cord stimulators, surgery) treatments for pain, published between January 2006 and June 2013 (n = 173), were included. None of the trials using experimental pain models reported an ITT analysis; 47% of trials investigating clinical pain conditions reported an ITT analysis, and 5% reported a modified ITT analysis. Of the analyses reported as ITT, 67% reported reasons for excluding subjects from the analysis, and 18% of those listing reasons for exclusion did not do so in the Methods section. Such mislabeling can make it difficult to identify traditional ITT analyses for inclusion in meta-analyses. We hope that deficiencies in reporting identified in this study will encourage authors, reviewers, and editors to promote more consistent use of the term ‘‘intention to treat’’ for more accurate reporting of RCT-based evidence for pain treatments. Ó 2014 International Association for the Study of Pain. Published by Elsevier B.V. All rights reserved.
1. Introduction A randomized clinical trial (RCT) is considered the gold standard of evidence for causal inference of a treatment effect on a health outcome. Randomization eliminates bias in treatment effect esti⇑ Corresponding author. Address: University of Rochester School of Medicine and Dentistry, Department of Anesthesiology, 601 Elmwood Av, Box 604, Rochester, NY 14609, USA. Tel.: +1 (585) 276 5661; fax: +1 (585) 244 7271. E-mail address: [email protected] (J.S. Gewandter).
mates that arise from inherent differences between treatment groups, which can occur in nonrandomized studies. The intention-to-treat (ITT) principle states that randomized subjects should be analyzed in the group to which they were randomized, no matter what treatment they actually receive or use [20]. As discussed further below, an analysis that adheres to the ITT principle addresses the effect of a treatment policy or the ‘‘efficacy of prescribing the treatment,’’ not necessarily the efficacy of the treatment if it is taken as prescribed, which many believe to be more clinically relevant [35,36]. However, eliminating participants from
http://dx.doi.org/10.1016/j.pain.2014.09.039 0304-3959/Ó 2014 International Association for the Study of Pain. Published by Elsevier B.V. All rights reserved.
Ò
J.S. Gewandter et al. / PAIN 155 (2014) 2714–2719
the analysis based on protocol adherence can bias treatment effect estimates such that apparent treatment effects may be due to group differences in participant characteristics rather than the treatment. Despite its potential limitations, the ITT principle preserves the benefits of randomization and an ITT analysis is recommended, at least as a sensitivity analysis, for RCTs by regulators and statisticians [8,39]. An ITT analysis should include all randomized subjects in the group to which they were randomized, but there are typically subjects with missing data in clinical trials. A systematic review of methodological literature found that almost all authors agreed that the ITT analysis should not eliminate subjects with missing outcome data, and that missing data should be accommodated using methods such as imputation or statistical modeling [2]. More detailed information on the benefits and limitations of various methods to accommodate missing data can be found elsewhere [11,22,26]. Previous studies have investigated the frequency with which ITT analyses have been identified by authors or with which analyses have included all randomized participants in RCT reports in general medical [14,18,19,23,27,31] or specialty [1,4,15,30,32,38,40] journals. Fewer studies have evaluated whether analyses that are identified as ITT by authors actually include data from all randomized participants [4,14,17,32]. The frequency with which an ITT analysis was identified and all randomized participants were included in the analysis varied greatly depending on the medical specialty being evaluated (author-identified ITT range, 15% to 67%; all participants analyzed range, 22% to 45%). The purpose of our study was to determine the percentage of RCTs for pain treatments published between 2006 and 2013 in several major pain journals that reported an ITT analysis, whether the numbers of subjects analyzed were clearly reported, and whether those ITT analyses included all randomized subjects. 2. Methods 2.1. Article selection Reports of all interventional (ie, invasive—epidural steroid injections, intravenous infusions, spinal cord stimulators, or implantable pumps) [7] and noninvasive pharmacologic clinical trials published between 2006 and 2012 were selected from the European Journal of Pain, Journal of Pain, and Pain as reported by Gewandter et al. [12]. In brief, selected articles reported trials that were randomized, double-blind, compared at least 2 treatments (one of which could be placebo), and had a primary or secondary outcome variable related to pain. Articles published between January 2006 and June 2013 meeting these criteria were identified independently by 2 authors (JSG and AM) by hand searching each journal’s table of contents, and were added to complete the sample for this study. 2.2. Data extraction A coding manual was developed to investigate the use of the ITT principle and clarity of reporting of details relevant to ITT in analgesic RCTs. The coding manual was pretested and modified in 3 rounds. In each round, 3 authors (JSG, AM, and SMS) coded articles separately and discussed any discrepancies or confusion with the manual and modified the manual as necessary. Each article was coded by 2 authors in a randomized order. One author (AM) coded all of the articles; 3 authors (DR, JJP, JRP) provided the duplicate coding, with each coding approximately one-third of the articles. After the articles were coded, JSG reviewed the data
2715
and corrected discrepancies due to obvious oversight. Discrepancies that were due to alternative interpretations were discussed by JSG and AM and a consensus was reached. The method used to accommodate missing data, if any, was coded as previously described [11]. For the purposes of these analyses, a complete case analysis (ie, an analysis that only includes subjects who completed the study) was not considered a method to accommodate missing data. Journal, study design, year, and number of participants were also extracted for each article. In addition, information regarding trial type (experimental pain model or clinical pain condition), sponsor (industry, government, professional organization, or not reported), and trial size (P100 or
PAIN xxx (2014) xxx–xxx
www.elsevier.com/locate/pain
Reporting of intention-to-treat analyses in recent analgesic clinical trials: ACTTION systematic review and recommendations Jennifer S. Gewandter a,⇑, Michael P. McDermott b, Andrew McKeown a, Shannon M. Smith a, Joseph R. Pawlowski a, Joseph J. Poli a, Daniel Rothstein a, Mark R. Williams a, Shay Bujanover c, John T. Farrar d, Ian Gilron e, Nathaniel P. Katz f,g, Michael C. Rowbotham h, Dennis C. Turk i, Robert H. Dworkin a a
Department of Anesthesiology, University of Rochester School of Medicine and Dentistry, Rochester, NY, USA Departments of Biostatistics and Computational Biology and Neurology, University of Rochester School of Medicine and Dentistry, Rochester, NY, USA c Depomed, Inc., Newark, CA, USA d University of Pennsylvania, Philadelphia, PA, USA e Queen’s University, Kingston, Ontario, Canada f Analgesic Solutions, Natick, MA, USA g Tufts University, Boston, MA, USA h California Pacific Medical Center, San Francisco, CA, USA i Department of Anesthesiology and Pain Medicine, University of Washington, Seattle, WA, USA b
Sponsorships or competing interests that may be relevant to content are disclosed at the end of this article.
a r t i c l e
i n f o
Article history: Received 25 June 2014 Received in revised form 8 September 2014 Accepted 29 September 2014
Keywords: ACTTION Intention to treat Systematic review
a b s t r a c t The intention-to-treat (ITT) principle states that all subjects in a randomized clinical trial (RCT) should be analyzed in the group to which they were assigned, regardless of compliance with assigned treatment. Analyses performed according to the ITT principle preserve the benefits of randomization and are recommended by regulators and statisticians for analyses of RCTs. The objective of this study was to determine the frequency with which publications of analgesic RCTs in 3 major pain journals report an ITT analysis and the percentage of the author-declared ITT analyses that include all randomized subjects and thereby fulfill the most common interpretation of the ITT principle. RCTs investigating noninvasive, pharmacologic and interventional (eg, nerve blocks, implantable pumps, spinal cord stimulators, surgery) treatments for pain, published between January 2006 and June 2013 (n = 173), were included. None of the trials using experimental pain models reported an ITT analysis; 47% of trials investigating clinical pain conditions reported an ITT analysis, and 5% reported a modified ITT analysis. Of the analyses reported as ITT, 67% reported reasons for excluding subjects from the analysis, and 18% of those listing reasons for exclusion did not do so in the Methods section. Such mislabeling can make it difficult to identify traditional ITT analyses for inclusion in meta-analyses. We hope that deficiencies in reporting identified in this study will encourage authors, reviewers, and editors to promote more consistent use of the term ‘‘intention to treat’’ for more accurate reporting of RCT-based evidence for pain treatments. Ó 2014 International Association for the Study of Pain. Published by Elsevier B.V. All rights reserved.
1. Introduction A randomized clinical trial (RCT) is considered the gold standard of evidence for causal inference of a treatment effect on a health outcome. Randomization eliminates bias in treatment effect esti⇑ Corresponding author. Address: University of Rochester School of Medicine and Dentistry, Department of Anesthesiology, 601 Elmwood Av, Box 604, Rochester, NY 14609, USA. Tel.: +1 (585) 276 5661; fax: +1 (585) 244 7271. E-mail address: [email protected] (J.S. Gewandter).
mates that arise from inherent differences between treatment groups, which can occur in nonrandomized studies. The intention-to-treat (ITT) principle states that randomized subjects should be analyzed in the group to which they were randomized, no matter what treatment they actually receive or use [20]. As discussed further below, an analysis that adheres to the ITT principle addresses the effect of a treatment policy or the ‘‘efficacy of prescribing the treatment,’’ not necessarily the efficacy of the treatment if it is taken as prescribed, which many believe to be more clinically relevant [35,36]. However, eliminating participants from
http://dx.doi.org/10.1016/j.pain.2014.09.039 0304-3959/Ó 2014 International Association for the Study of Pain. Published by Elsevier B.V. All rights reserved.
Ò
J.S. Gewandter et al. / PAIN 155 (2014) 2714–2719
the analysis based on protocol adherence can bias treatment effect estimates such that apparent treatment effects may be due to group differences in participant characteristics rather than the treatment. Despite its potential limitations, the ITT principle preserves the benefits of randomization and an ITT analysis is recommended, at least as a sensitivity analysis, for RCTs by regulators and statisticians [8,39]. An ITT analysis should include all randomized subjects in the group to which they were randomized, but there are typically subjects with missing data in clinical trials. A systematic review of methodological literature found that almost all authors agreed that the ITT analysis should not eliminate subjects with missing outcome data, and that missing data should be accommodated using methods such as imputation or statistical modeling [2]. More detailed information on the benefits and limitations of various methods to accommodate missing data can be found elsewhere [11,22,26]. Previous studies have investigated the frequency with which ITT analyses have been identified by authors or with which analyses have included all randomized participants in RCT reports in general medical [14,18,19,23,27,31] or specialty [1,4,15,30,32,38,40] journals. Fewer studies have evaluated whether analyses that are identified as ITT by authors actually include data from all randomized participants [4,14,17,32]. The frequency with which an ITT analysis was identified and all randomized participants were included in the analysis varied greatly depending on the medical specialty being evaluated (author-identified ITT range, 15% to 67%; all participants analyzed range, 22% to 45%). The purpose of our study was to determine the percentage of RCTs for pain treatments published between 2006 and 2013 in several major pain journals that reported an ITT analysis, whether the numbers of subjects analyzed were clearly reported, and whether those ITT analyses included all randomized subjects. 2. Methods 2.1. Article selection Reports of all interventional (ie, invasive—epidural steroid injections, intravenous infusions, spinal cord stimulators, or implantable pumps) [7] and noninvasive pharmacologic clinical trials published between 2006 and 2012 were selected from the European Journal of Pain, Journal of Pain, and Pain as reported by Gewandter et al. [12]. In brief, selected articles reported trials that were randomized, double-blind, compared at least 2 treatments (one of which could be placebo), and had a primary or secondary outcome variable related to pain. Articles published between January 2006 and June 2013 meeting these criteria were identified independently by 2 authors (JSG and AM) by hand searching each journal’s table of contents, and were added to complete the sample for this study. 2.2. Data extraction A coding manual was developed to investigate the use of the ITT principle and clarity of reporting of details relevant to ITT in analgesic RCTs. The coding manual was pretested and modified in 3 rounds. In each round, 3 authors (JSG, AM, and SMS) coded articles separately and discussed any discrepancies or confusion with the manual and modified the manual as necessary. Each article was coded by 2 authors in a randomized order. One author (AM) coded all of the articles; 3 authors (DR, JJP, JRP) provided the duplicate coding, with each coding approximately one-third of the articles. After the articles were coded, JSG reviewed the data
2715
and corrected discrepancies due to obvious oversight. Discrepancies that were due to alternative interpretations were discussed by JSG and AM and a consensus was reached. The method used to accommodate missing data, if any, was coded as previously described [11]. For the purposes of these analyses, a complete case analysis (ie, an analysis that only includes subjects who completed the study) was not considered a method to accommodate missing data. Journal, study design, year, and number of participants were also extracted for each article. In addition, information regarding trial type (experimental pain model or clinical pain condition), sponsor (industry, government, professional organization, or not reported), and trial size (P100 or
