bs_bs_banner
Pain Medicine 2014; 15: 539–540 Wiley Periodicals, Inc.
EDITORIAL Efficacy vs Effectiveness and Explanatory vs Pragmatic: Where Is the Balance Point in Pain Medicine Research?
In this issue of Pain Medicine, there is a study by Cohen and colleagues that investigates the effect of procedural sedation on diagnostic and therapeutic outcomes for sacroiliac joint and sympathetic nerve block procedures [1]. They found that procedural sedation is associated with an increased rate of false positive blocks and had no significant effects on patient satisfaction or clinical outcomes at 1 month. Although this is a crucial issue relevant to the day-to-day practice of pain medicine, this elegantly designed study also broaches in a sophisticated manner the divide between explanatory and pragmatic trials. Their study design results in findings that are of high scientific quality and yet broadly applicable to routine interventional pain medicine care. Randomized trials can be broadly characterized as an effectiveness or efficacy trial, and the corresponding terms “pragmatic” vs “explanatory” are often used [2]. Pragmatic trials help providers decide between options for care, and explanatory trials test research hypotheses under ideal study circumstances. Pragmatic trials seek answers to whether an intervention will work under usual conditions, whereas explanatory trials are best suited to test causal hypotheses in a setting where confounders can be minimized to the greatest degree possible. Typically, the findings from pragmatic trials have high generalizability (i.e., they are likely to hold true in most clinical circumstances), whereas explanatory trials are the most likely laboratory experiments where confidence is high that any differences found between study conditions (e.g., treatment vs control) were the result of scientific testing under rigorously controlled conditions. Generally, the control condition for purely pragmatic trials is usual care, whereas for explanatory trials, it is placebo or an active comparator condition. Hence, there are differences between the “effectiveness” and “efficacy” of any treatment, and these terms are not interchangeable. A treatment can be effective (resulting in improvement in most patients vs not treating) but may not be efficacious (i.e., treatment outcome is not significantly different than that of placebo treatment). Most trials are neither completely pragmatic (effectiveness) or explanatory (efficacy) and lie instead on a continuum between these poles. The movement toward more comparative effectiveness research is also part of this
quest for balance. For interventional pain medicine, as in most areas of medicine, we need studies that achieve the optimum balance between these perspectives, and that balance point will differ from study to study depending on the question of investigation [3]. For instance, in Cohen et al., the primary question was, “does the use of sedation confer a greater chance of a false positive results for diagnostic sacroiliac joint and sympathetic nerve blocks?” To these ends, pragmatic components of the trial included the following: 1) broad inclusion criteria—any patient judged by routine history and physical examination, with pain of a least 1 month duration and greater than 3/10, was eligible for inclusion; 2) practitioners and setting—this was a multicenter trial with multiple physicians delivering the interventions in standard pain medicine clinical environments; and 3) sedation—midazolam and/or fentanyl, but no standardized dose, as is common in clinical practice. Explanatory components of the trial included the following: 1) a randomized crossover design; 2) agreement among providers on the procedural techniques; 3) rigorous outcome assessment—a comprehensive baseline assessment battery and a 6-hour postprocedure assessment period, performed every 30 minutes, followed by a 1-month repeated comprehensive outcome assessment; and 4) primary data analysis was not “intention to treat.” In other words, it did not include all enrolled patients regardless of whether they had completed the study. The primary analysis was only performed in those subjects who had injections with and without sedation (N = 46). In sum, this study tilts toward the explanatory but has a sufficient degree of pragmatic qualities to suggest high generalizability. Furthermore, the explanatory and pragmatic components reinforce each other. For instance, in the study context of a standard clinical environment, rigorous outcome assessment, using repeated measures with high validity, accuracy, and sensitivity to change, adds confidence that the findings will most likely hold true in the usual care of patients. Thus, the optimal balance point between the poles of pragmatic and explanatory qualities is where these approaches mutually strengthen each other to create the most robust framework for a clinical trial design. Doctors Cohen and colleagues are to be congratulated for finding this difficult balance with a blended methodological approach yielding results likely to withstand the test of time. 539
Wasan AJAY D. WASAN, MD, MSc Departments of Anesthesiology and Psychiatry, University of Pittsburgh School of Medicine, Pittsburgh, Pennsylvania, USA
References 1 Cohen SP, Hameed H, Kurihara C, et al. The effect of sedation on the accuracy and treatment outcomes
540
for diagnostic injections: A randomized, controlled crossover study. Pain Med 2014; 15(5):588–602. 2 Thorpe KE, Zwarenstein M, Oxman AD, et al. A pragmatic-explanatory continuum indicator summary (PRECIS): A tool to help trial designers. J Clin Epidemiol 2009;62(5):464–75. 3 Rowbotham MC, Gilron I, Glazer C, et al. Can pragmatic trials help us better understand chronic pain and improve treatment? Pain 2013;154(5):643–6.
Pain Medicine 2014; 15: 539–540 Wiley Periodicals, Inc.
EDITORIAL Efficacy vs Effectiveness and Explanatory vs Pragmatic: Where Is the Balance Point in Pain Medicine Research?
In this issue of Pain Medicine, there is a study by Cohen and colleagues that investigates the effect of procedural sedation on diagnostic and therapeutic outcomes for sacroiliac joint and sympathetic nerve block procedures [1]. They found that procedural sedation is associated with an increased rate of false positive blocks and had no significant effects on patient satisfaction or clinical outcomes at 1 month. Although this is a crucial issue relevant to the day-to-day practice of pain medicine, this elegantly designed study also broaches in a sophisticated manner the divide between explanatory and pragmatic trials. Their study design results in findings that are of high scientific quality and yet broadly applicable to routine interventional pain medicine care. Randomized trials can be broadly characterized as an effectiveness or efficacy trial, and the corresponding terms “pragmatic” vs “explanatory” are often used [2]. Pragmatic trials help providers decide between options for care, and explanatory trials test research hypotheses under ideal study circumstances. Pragmatic trials seek answers to whether an intervention will work under usual conditions, whereas explanatory trials are best suited to test causal hypotheses in a setting where confounders can be minimized to the greatest degree possible. Typically, the findings from pragmatic trials have high generalizability (i.e., they are likely to hold true in most clinical circumstances), whereas explanatory trials are the most likely laboratory experiments where confidence is high that any differences found between study conditions (e.g., treatment vs control) were the result of scientific testing under rigorously controlled conditions. Generally, the control condition for purely pragmatic trials is usual care, whereas for explanatory trials, it is placebo or an active comparator condition. Hence, there are differences between the “effectiveness” and “efficacy” of any treatment, and these terms are not interchangeable. A treatment can be effective (resulting in improvement in most patients vs not treating) but may not be efficacious (i.e., treatment outcome is not significantly different than that of placebo treatment). Most trials are neither completely pragmatic (effectiveness) or explanatory (efficacy) and lie instead on a continuum between these poles. The movement toward more comparative effectiveness research is also part of this
quest for balance. For interventional pain medicine, as in most areas of medicine, we need studies that achieve the optimum balance between these perspectives, and that balance point will differ from study to study depending on the question of investigation [3]. For instance, in Cohen et al., the primary question was, “does the use of sedation confer a greater chance of a false positive results for diagnostic sacroiliac joint and sympathetic nerve blocks?” To these ends, pragmatic components of the trial included the following: 1) broad inclusion criteria—any patient judged by routine history and physical examination, with pain of a least 1 month duration and greater than 3/10, was eligible for inclusion; 2) practitioners and setting—this was a multicenter trial with multiple physicians delivering the interventions in standard pain medicine clinical environments; and 3) sedation—midazolam and/or fentanyl, but no standardized dose, as is common in clinical practice. Explanatory components of the trial included the following: 1) a randomized crossover design; 2) agreement among providers on the procedural techniques; 3) rigorous outcome assessment—a comprehensive baseline assessment battery and a 6-hour postprocedure assessment period, performed every 30 minutes, followed by a 1-month repeated comprehensive outcome assessment; and 4) primary data analysis was not “intention to treat.” In other words, it did not include all enrolled patients regardless of whether they had completed the study. The primary analysis was only performed in those subjects who had injections with and without sedation (N = 46). In sum, this study tilts toward the explanatory but has a sufficient degree of pragmatic qualities to suggest high generalizability. Furthermore, the explanatory and pragmatic components reinforce each other. For instance, in the study context of a standard clinical environment, rigorous outcome assessment, using repeated measures with high validity, accuracy, and sensitivity to change, adds confidence that the findings will most likely hold true in the usual care of patients. Thus, the optimal balance point between the poles of pragmatic and explanatory qualities is where these approaches mutually strengthen each other to create the most robust framework for a clinical trial design. Doctors Cohen and colleagues are to be congratulated for finding this difficult balance with a blended methodological approach yielding results likely to withstand the test of time. 539
Wasan AJAY D. WASAN, MD, MSc Departments of Anesthesiology and Psychiatry, University of Pittsburgh School of Medicine, Pittsburgh, Pennsylvania, USA
References 1 Cohen SP, Hameed H, Kurihara C, et al. The effect of sedation on the accuracy and treatment outcomes
540
for diagnostic injections: A randomized, controlled crossover study. Pain Med 2014; 15(5):588–602. 2 Thorpe KE, Zwarenstein M, Oxman AD, et al. A pragmatic-explanatory continuum indicator summary (PRECIS): A tool to help trial designers. J Clin Epidemiol 2009;62(5):464–75. 3 Rowbotham MC, Gilron I, Glazer C, et al. Can pragmatic trials help us better understand chronic pain and improve treatment? Pain 2013;154(5):643–6.
