SPINE Volume 39, Number 5, pp E305-E310 ©2014, Lippincott Williams & Wilkins
EDITORIAL
How Can We Design Low Back Pain Intervention Studies to Better Explain the Effects of Treatment? Gemma Mansell, MSc,* Jonathan C. Hill, PhD,* Steven J. Kamper, PhD,† Peter Kent, PhD,‡ Chris Main, PhD,* and Danielle A. van der Windt, PhD*
I
n this article, we consider whether adjustments to study designs, particularly of clinical trials, might provide important information to help explain how interventions for patients with low back pain (LBP) work. Back pain clinical trials have typically shown small effects associated with various treatments as compared with no treatment or current usual care, which could mean that these treatments are not very effective,1,2 or that the treatments are only effective for specific clinical subgroups and interventions therefore need better targeting.3 However, researchers also need to consider new methods to optimize treatment outcomes by investigating the processes which explain how or why specific treatments work.4 Such studies have been referred to as mediation analyses,5 and this article seeks to highlight why and how future studies should be designed to incorporate mediation analyses. The primary importance of mediation analysis is that it can help us understand how to improve treatments for patients with back pain. For example, psychological factors such as fear avoidance and self-efficacy are known to be predictive of patient outcomes such as disability and return to work, but it is less clear whether targeting an intervention specifically at reducing fear avoidance or increasing self-efficacy is able to further improve such outcomes. By analyzing the causal pathway between treatment, potential mediators and outcomes, we can identify the key factors to focus on during treatment and From the *Arthritis Research UK Primary Care Centre, Primary Care Sciences, Keele University, Keele, Staffordshire, United Kingdom; †Musculoskeletal Division, The George Institute for Global Health, University of Sydney, Australia and EMGO+ Institute, VU University Medical Centre, Amsterdam, the Netherlands; and ‡Spine Centre of Southern Denmark, Hospital Lillibælt, Institute of Regional Health Services Research, University of Southern Denmark, Odense, Denmark. Acknowledgment date: July 3, 2013. Revision date: September 16, 2013. Acceptance date: November 15, 2013. The manuscript submitted does not contain information about medical device(s)/drug(s). The National Institute for Health Research (NIHR) under its Programme Grants for Applied Research Programme (grant reference no. RP-PG-0707-10131) funds were received in support of this work. Relevant financial activities outside the submitted work: employment, grants, royalties. Address correspondence and reprint requests to Gemma Mansell, MSc, Arthritis Research UK Primary Care Centre, Primary Care Sciences, Keele University, Keele, Staffordshire, United Kingdom, ST5 5BG; E-mail: [email protected] DOI: 10.1097/BRS.0000000000000144 Spine
also identify which factors do not mediate outcome, to further streamline and improve intervention efficacy and efficiency. At the Twelfth International Low Back Pain Primary Care Research Forum in Odense, Denmark 2012, a dedicated workshop was held in which various aspects of mediation study design were explored and methodological considerations discussed. The workshop brought together experienced and internationally recognized LBP researchers and clinicians to provide an overview of the importance of mediation analyses and explore its usefulness in testing our assumptions about how our interventions may be influencing outcomes. We provide a summary of the workshop findings and discuss the implications for embedding mediation analysis methodology within future interventional research. We also offer definitions for some commonly used and misused terms, provide examples of mediation research using different study designs, and incorporate information on the methodology of mediation analysis from the wider literature, to formulate recommendations for future mediation research in the field of LBP. It is important to acknowledge that while the focus of this article is on mediators of specific treatment effects, the identification of mediator variables often comes from a program of work, which may involve different study designs. This article therefore includes a discussion of research using observational designs to explore mediating processes, although the limitations of evidence from such studies is noted and ultimately randomized controlled trials (RCTs) are required to confirm which factors mediate specific treatment effects.
CONCEPT DEFINITIONS When longitudinal data from an observational study (single groups of individuals observed over time) or an intervention study (2 or more groups of individuals, exposed to different treatment conditions) are analyzed, some factors may be consistently predictive of outcome. However, although such factors may show a statistical association with outcome following treatment, the relationship can take a number of different forms such as follows: a prognostic factor, a treatment effect moderator, or a treatment effect mediator.
Prognostic Factors Prognostic factors are baseline characteristics that are associated with outcome regardless of treatment, such as high baseline pain and high baseline activity limitation, which www.spinejournal.com
E305
Copyright © 2014 Lippincott Williams & Wilkins. Unauthorized reproduction of this article is prohibited. SPINEE1302.indd E305
01/02/14 9:30 AM
EDITORIAL
LBP Intervention Studies to Explain the Effects of Treatment • Mansell et al
are generally known to be associated with poor outcome in patients with spinal pain across a variety of different treatment interventions. Therefore, in an RCT, a prognostic factor might predict outcome for patients in both the intervention and control groups. In intervention studies, prognostic factors have also been called nonspecific predictors of treatment outcome.5 In statistical terms, they are baseline variables that are associated with outcome (they have a significant main effect only) but do not correlate or interact with treatment allocation.5
Treatment Effect Moderators A treatment effect moderator is a baseline variable that identifies people who do better or worse with a given treatment. These factors specify for whom and under what circumstances a treatment is most effective and are therefore central to stratified care that targets specific subgroups of people. For example, treatment A may work better for people with high fear of movement (who) or in people seeking primary care rather than secondary care (what circumstances) compared with treatment B. In statistical terms, treatment effect moderators stratify or subgroup the effect of a particular treatment and are indicated by a statistical interaction between treatment allocation (treatment A vs. B), time and subgroup (e.g., high vs. low fear).5 In some fields of research, treatment effect moderators are also referred to as effect modifiers or predictors of differential treatment effect, but these terms refer to the same thing.6
Treatment Effect Mediators In contrast to prognostic factors and treatment effect moderators, which are both measured only at baseline, treatment effect mediators are factors that change in response to a specific treatment and should therefore not only be measured at baseline but also during and after treatment. These factors help explain the relationship between treatment and outcome, and may therefore help identify the causal mechanisms by which treatments work. The identification of mediators can potentially improve treatment effects by strengthening the influence of the mediating factors involved in producing the treatment effect. Hypothetically it might be that in order for cognitive behavioral therapy (the treatment) to reduce activity limitation (the outcome), a patient’s self-efficacy (the mediator) needs to be strengthened, and that in the absence of self-efficacy improving, cognitive behavioral therapy will not be as effective as it otherwise might be. In statistical terms, changes in mediator scores are associated with specific treatment allocation (i.e., differ between intervention groups), and change in the mediator has a direct association with change in treatment outcome5 (Figure 1).
STUDY DESIGN IN MEDIATION ANALYSIS An attractive aspect of mediation analysis is its applicability to different study designs. The analytical process can be applied to data from cross-sectional studies, longitudinal observational cohorts and RCTs. However, different study designs do impose different limitations on the nature of the research question that can be answered and also the certainty with which causal inferences can be made. E306
www.spinejournal.com
Figure 1. Treatment and mediating effects. CBT indicates cognitive behavioral therapy. *Analogous to the main effect of treatment.
Cross-sectional Designs Mediation analysis has often been used with data from cross-sectional studies. With a convincing biological rationale and a priori defined hypotheses, these data can provide some evidence in support of pre-existing theory or be used to identify promising potential pathways that can then be tested and confirmed via more robust designs such as RCTs. However, causal inference requires a specified temporal (time-based) ordering of the investigated variables, and in cross-sectional studies where participants are only assessed at 1 time point, no temporal link can be made. Many publications that report the use of this design do acknowledge that the use of cross-sectional data precludes the inference of causation.7–9
Longitudinal Designs A number of authors have stipulated that longitudinal designs are required for the assessment of change over time.10,11 Longitudinal data enable the researcher to take control of previous levels of the mediator and outcome variables that can strengthen the claim of a potential causal association, but do not account for the potential influence of confounding variables that are not measured or included in the mediation analysis. Confounding is a key issue in mediation analysis using longitudinal and cross-sectional observational designs. Confounders are variables that are correlated with treatment exposure (i.e., show different values between treatment groups) or the independent variable and are associated with outcome, but unlike mediating variables are not on a causal pathway between treatment and outcome. A series of studies on people with spinal pain in Australia used longitudinal data from observational cohorts12,13 or pooled data from an RCT14 to investigate the mediating influence of psychological factors in the relationship between pain and disability. Three or more waves of data may be useful for assessing temporality as this provides stronger evidence in support of a causal pathway linking the variables,15–17 although it is acknowledged that this does increase March 2014
Copyright © 2014 Lippincott Williams & Wilkins. Unauthorized reproduction of this article is prohibited. SPINEE1302.indd E306
01/02/14 9:30 AM
EDITORIAL the complexity of the analysis and requires more complex modeling procedures.17
Experimental Designs Mediation analysis aims to identify causal associations, and experimental designs where patients are randomly allocated to a certain intervention at a point in time provides perhaps the strongest support for causal inference. It should be noted however that RCTs and observational studies seek to address different types of questions in mediation research. Studies using observational data investigate the relationships between variables and outcomes relevant to people with a particular condition, and may support the generation of hypotheses regarding potential mediators. Mediation studies that include allocation to 1 treatment or another as the independent variable investigate the mechanism of action of a particular treatment on outcome. It is this aspect of experimental designs which is of particular benefit in mediation analyses as it allows the investigation of the key variables (mediators) to target in future intervention studies. For example, studies of mediation analysis in RCTs have found that graded exposure treatment exerted part of its treatment effect in people with chronic LBP via reduction of catastrophizing,18 and showed that improvement in back pain was due to an increase in lumbar muscle endurance after an exercise program.19 Another advantage of experimental designs is their ability to reduce the effect of confounding. When an RCT design is used, randomization helps ensure that intervention groups have similar characteristics at baseline and so any differences in outcome are less likely to be as a result of 1 group differing substantially from the other to start with.16,20,21 This can help researchers to see whether any mediating effects would have occurred in spite of treatment (e.g., natural resolution) or whether they are due to the treatment being given and are therefore on a causal pathway.21 However, it has been recognized that the RCT design does not necessarily account for all unmeasured variables that could impact on outcome.22 This is because although participants may have been randomized to the intervention itself, change in the mediator(s) and outcome(s) occurs after randomization has taken place (if the intervention is effective),15 meaning that there may still be factors other than treatment that affect the relationship between the mediator and outcome variables.23 For example, although patients may have similar levels of fear avoidance at baseline due to the randomization process, during treatment some patients may develop better therapeutic relationships with their therapist than others, which may lead to a greater change in fear avoidance, and therefore a greater change in outcome for these particular patients. In this case, the therapeutic relationship is a potential confounder because it may have an impact on fear avoidance in addition to the impact of treatment.
ISSUES AROUND HETEROGENEITY IN PRIMARY CARE POPULATIONS Studies carried out in secondary or tertiary care populations, in which the condition is usually clearly established, unambiguously identified, and of sufficient severity or impact to Spine
LBP Intervention Studies to Explain the Effects of Treatment • Mansell et al
enable the identification of therapeutic change, may report larger and more consistent effect sizes than those conducted with primary care populations.24 A problem when investigating primary care populations may be the greater clinical variability in symptoms and their impact on the population under study. This could also influence mediation analyses as greater heterogeneity in potential mediators and outcomes results in larger standard deviations in the scores in the study population, making it harder to demonstrate a difference between treatments. Perhaps more importantly, greater clinical variability may also imply that the mechanism explaining treatment effect varies across patients, which can translate into difficulty in identifying the role of mediators in explaining treatment effects. To overcome this, larger sample sizes might be required so that the assessment of the effect of mediators in different subgroups can be undertaken. A specific advantage of primary care populations is that, from a public health perspective, the results may be more widely generalizable.
AREAS FOR IMPROVEMENT TO THE DESIGN OF FUTURE MEDIATION STUDIES A number of ways to incorporate mediation analysis into future RCTs by making small changes to their current design have been suggested. The list below outlines the main suggestions but is by no means exhaustive; more detailed reviews of the mediation literature, relating to both design and analysis issues, can be found elsewhere.11,23,25–27
Importance of a Theoretical Foundation Selecting mediators based on theory can help researchers choose the key factors to focus on,28 and will help formulate clear hypotheses regarding how a treatment might work before the study begins, which may subsequently be supported or refuted by the results of the mediation analysis.29 A theoretical basis will also help with the issue of temporality described in the earlier text because it provides a framework for showing how variables might relate to each other in a particular order.30
Design of Studies to Test Specifically for Mediation It is suggested that where an intervention seeks to change a particular mediator or set of mediators, measures of the proposed mediators should be included in the study.11 However, investigators should limit data collection to only include mediators that can potentially be modified by treatment and for which there is some evidence for a mediating relationship with the outcome measure.28 For example, researchers might hypothesize that an exercise program is effective due to its action of improving aerobic endurance in patients, or alternatively via improvements in self-efficacy. In these cases, measures of aerobic capacity and/or self-efficacy should be incorporated into the study design to specifically test these hypotheses. Sample size obviously has an effect on the power of a study to detect mediating effects. Although the required sample size is dependent on the choice of statistical technique, variability www.spinejournal.com
E307
Copyright © 2014 Lippincott Williams & Wilkins. Unauthorized reproduction of this article is prohibited. SPINEE1302.indd E307
01/02/14 9:30 AM
EDITORIAL
LBP Intervention Studies to Explain the Effects of Treatment • Mansell et al
of the measures and the expected strength of the associations between the variables, a substantial sample may be necessary. It is recommended that sophisticated mediation analysis methods require at least 15031 to 20032 participants to provide reliable results. One review33 found that of 166 mediation analysis studies published in 2 key journals in their field, 113 studies (68%) included sample sizes of above 150.
intervention may work differently for different people and the effects could occur at different times.11 Although attempting to untangle this issue adds considerable complexity, multiple appropriately timed measurements are important for exploring mediators of treatment.
Timing of Assessments
Measurement error is a particularly important problem when trying to assess whether change in a particular factor is responsible for the subsequent change in outcome, as in mediation analysis. Measurements need to be highly reliable when measuring change34 to ensure accurate estimates. Measurement error can become magnified when a raw change score (difference between pretest and post-test) is calculated as there is likely to be error associated with both measurements.34 A measure’s sensitivity to change is also important because this relates to whether or not a measure can detect change over time and separate this from measurement error.35 The use of measures with poor reliability or responsiveness in mediation studies may affect the user's ability to identify effects of mediating variables.
To assess temporality, it has been suggested that both the mediator and outcome should be assessed frequently during the period of time that change is expected to occur.16,24 This again shows the importance of considering treatment mediators a priori because a clear hypothesis regarding when a mediator is likely to change is important in planning the optimal timing for study measurement time points. Although studies that include assessments pre- and postintervention may provide some useful information on what mediating factors are related to outcome,16 they are not able to determine which variable (mediator or outcome) changed first.11 Following the example of the exercise intervention as proposed in the “Design of Studies to Test Specifically for Mediation” section, while changes in aerobic capacity are likely to take several weeks to occur, self-efficacy gains may be realized in a shorter time frame. These considerations will need careful thought to plan appropriate timing of study measurements. Also, an
Inclusion of Study Measures With Appropriate Measurement Properties
Incorporation of Exploratory Analysis In the workshop on mediation analysis held during the Twelfth Forum in Odense 2012 there was general consensus
Box 1. Recommendations to support and improve research into treatment effect mediators.
E308
www.spinejournal.com
March 2014
Copyright © 2014 Lippincott Williams & Wilkins. Unauthorized reproduction of this article is prohibited. SPINEE1302.indd E308
01/02/14 9:30 AM
EDITORIAL that, aside from the points mentioned in the earlier text, there is a need to return to exploratory analysis (e.g., mediation analyses in uncontrolled studies) and qualitative analysis to pinpoint the concepts that are most important in investigating treatment mechanisms. The importance of qualitative research in helping identify key factors to test as mediators has been highlighted previously,11 suggesting that this can be especially helpful in areas where no theories are available and can be used initially to explore therapeutic processes and their impact on therapist and patient. This implies that a program of research will often be needed to identify key mediators, with replication of findings to confirm the importance of those mediators being an essential part of that process.19,28
SUMMARY AND CONCLUSION The majority of trials investigating the effectiveness of primary care interventions for back pain have shown small or at best, moderate effects of treatment36,37 and the field is looking for better ways to improve outcomes for patients with back pain. Mediation analysis aims to provide better insight into the causal pathways underlying treatment effects, explaining why treatments work or do not work and potentially offering new opportunities to improve patient outcomes by optimizing the content or delivery of treatment. Until recently, mediation analysis in clinical research was often limited to a descriptive evaluation of the processes potentially underlying the effects of treatment in trials.38,39 During the past few years, interest in the use of more sophisticated approaches to mediation analysis has increased, often guided by methods described in the psychological literature. In this article, we have summarized the concepts and different designs used in mediation analysis and explained the importance of experimental designs when investigating mediators of treatment effect. We have also emphasized the importance of other considerations such as defining and understanding constructs, selecting study measures with appropriate measurement properties, and ensuring study measurement time points are appropriately selected, to investigate the longitudinal associations between mediating and outcome variables. We have also outlined the relevance of observational and qualitative research in identifying potential mediating factors. On the basis of the discussions during the 2012 workshop and supported by the literature, we have proposed a set of recommendations to support and improve the design of mediation analysis in back pain research (Box 1), with the ultimate aim to improve the design and delivery of intervention studies and optimize outcomes for patients with back pain.
Acknowledgment Author Chris Main thanks Dr. Kelvin Jordan, for his statistical advice.
References
1. van der Windt D, Hay E, Jellema P, et al. Psychosocial interventions for low back pain in primary care: lessons learnt from recent trials. Spine 2008;33:81–9.
Spine
LBP Intervention Studies to Explain the Effects of Treatment • Mansell et al
2. Guzman J, Esmail R, Karjalainen K, et al. Multidisciplinary rehabilitation for chronic low back pain: systematic review. BMJ 2001;322:1511–6. 3. Haldorsen EM, Grasdal AL, Skouen JS, et al. Is there a right treatment for a particular patient group? Comparison of ordinary treatment, light multidisciplinary treatment, and extensive multidisciplinary treatment for long-term sick-listed employees with musculoskeletal pain. Pain 2002;95:49–63. 4. Shaw WS, Linton SJ, Pransky G. Reducing sickness absence from work due to low back pain: how well do intervention strategies match modifiable risk factors? J Occup Rehabil 2006;16:591–605. 5. Kraemer HC, Wilson GT, Fairburn CG, et al. Mediators and moderators of treatment effects in randomized clinical trials. Arch Gen Psychiatry 2002;59:877–83. 6. Kraemer HC, Stice E, Kazdin A, et al. How do risk factors work together? Mediators, moderators, and independent, overlapping, and proxy risk factors. Am J Psychiatry 2001;158:848–56. 7. Woby SR, Roach NK, Urmston M, et al. The relation between cognitive factors and levels of pain and disability in chronic low back pain patients presenting for physiotherapy. Eur J Pain 2007;11:869–77. 8. Arnstein P, Caudill M, Mandle CL, et al. Self-efficacy as a mediator of the relationship between pain intensity, disability and depression in chronic pain patients. Pain 1999;80:483–91. 9. Gheldof EL, Vinck J, Van den Bussche E, et al. Pain and painrelated fear are associated with functional and social disability in an occupational setting: evidence of mediation by pain-related fear. Eur J Pain 2006;10:513–25. 10. Liu LC, Flay BR. Aban Aya Investigators. Evaluating mediation in longitudinal multivariate data: mediation effects for the Aban Aya Youth Project drug prevention program. Prev Sci 2009;10: 197–207. 11. Kazdin AE. Mediators and mechanisms of change in psychotherapy research. Annu Rev Clin Psychol 2007;3:1–27. 12. Kamper SJ, Maher CG, Costa Lda CM, et al. Does fear of movement mediate the relationship between pain intensity and disability in patients following whiplash injury? A prospective longitudinal study. Pain 2012;153:113–9. 13. Costa Lda CM, Maher CG, McAuley JH, et al. Self-efficacy is more important than fear of movement in mediating the relationship between pain and disability in chronic low back pain. Eur J Pain 2011;15: 213–9. 14. Hall AM, Kamper SJ, Maher CG, et al. Symptoms of depression and stress mediate the effect of pain on disability. Pain 2011;152: 1044–51. 15. Collins LM, Graham JW, Flaherty BP. An alternative framework for defining mediation. Multivar Behav Res 1998;33:295–312. 16. Lockhart G, MacKinnon DP, Ohlrich V. Mediation analysis in psychosomatic medicine research. Psychosom Med 2011;73:29–43. 17. Maric M, Wiers RW, Prins PJ. Ten ways to improve the use of statistical mediation analysis in the practice of child and adolescent treatment research. Clin Child Fam Psychol Rev 2012;15:177–91. 18. Leeuw M, Goossens ME, van Breukelen GJ, et al. Exposure in vivo versus operant graded activity in chronic low back pain patients: results of a randomized controlled trial. Pain 2008;138:192–207. 19. Mulroy SJ, Winstein CJ, Kulig K, et al. Secondary mediation and regression analyses of the PTClinResNet database: Determining causal relationships among the international classification of functioning, disability and health levels for four physical therapy intervention trials. Phys Ther 2011;91:1766–79. 20. Bullock JG, Green DP, Ha SE. Yes, but what’s the mechanism? (don’t expect an easy answer). J Pers Soc Psychol 2010;98:550–58. 21. Murphy R, Cooper Z, Hollon SD, et al. How do psychological treatments work? Investigating mediators of change. Behav Res Ther 2009;47:1–5. 22. Judd CM, Kenny DA. Process analysis: Estimating mediation in treatment evaluations. Eval Rev 1981;5:602–19. 23. Emsley R, Dunn G, White IR. Mediation and moderation of treatment effects in randomised controlled trials of complex interventions. Stat Methods Med Res 2010;19:237–70. www.spinejournal.com
E309
Copyright © 2014 Lippincott Williams & Wilkins. Unauthorized reproduction of this article is prohibited. SPINEE1302.indd E309
01/02/14 9:30 AM
EDITORIAL 24. Grissom RJ, Kim JJ. Effect Sizes for Research: Univariate and Multivariate Applications. 2nd ed. New York: Taylor & Francis (Routledge); 2012. 25. Hayes AF. Beyond Baron and Kenny: statistical mediation analysis in the new millennium. Commun Monogr 2009;76:408–20. 26. MacKinnon DP, Fairchild AJ, Fritz MS. Mediation analysis. Ann Rev Psychol 2007;58:593–614. 27. MacKinnon DP, Fairchild AJ. Current directions in mediation analysis. Curr Dir Psychol Sci 2009;18:16–20. 28. MacKinnon DP. Analysis of mediating variables in prevention and intervention research. Natl Inst Drug Abuse Monogr 1994;139:127–53. 29. MacKinnon DP. Integrating mediators and moderators in research design. Res Social Work Pract 2011;21:675–81. 30. Mathieu JE, Taylor SR. Clarifying conditions and decision points for mediational type inferences in organizational behavior. J Org Behav 2006;27: 1031–56. 31. Holbert RL, Stephenson MT. Structural equation modeling in the communication sciences, 1995–2000. J Hum Commun Res 2008;28:531–51. 32. Tomarken AJ, Waller NG. Potential problems with “well fitting” models. J Abnormal Psychol 2003;112:578–98.
E310
www.spinejournal.com
LBP Intervention Studies to Explain the Effects of Treatment • Mansell et al
33. Fritz MS, MacKinnon DP. Required sample size to detect the mediated effect. Psychol Sci 2007;18:233–9. 34. Streiner DL, Norman GR. Health Measurement Scales: A Practical Guide to Their Development and Use. 4th ed Oxford: Oxford University Press; 2008. 35. Terwee CB, Bot SD, de Boer MR, et al. Quality criteria were proposed for measurement properties of health status questionnaires. J Clin Epidemiol 2007;60:34–42. 36. Keller A, Hayden J, Bombardier C, et al. Effect sizes of nonsurgical treatments of nonspecific low back pain. Eur Spine J 2007;16: 1776–88. 37. Machedo LA, Kamper SJ, Herbert RD, et al. Analgesic effects of treatments for nonspecific low back pain: a meta-analysis of placebo-controlled randomized trials. Rheumatology (Oxford) 2009;48:520–7. 38. Jellema P, van der Windt DA, van der Horst HE, et al. Why is a treatment aimed at psychosocial factors not effective in patients with (sub)acute low back pain? Pain 2005;118: 350–9. 39. Thorn BE, Day MA, Burns J, et al. Randomized trial of group cognitive behavioral therapy compared with a pain education control for low-literacy rural people with chronic pain. Pain 2011;152: 2710–20.
March 2014
Copyright © 2014 Lippincott Williams & Wilkins. Unauthorized reproduction of this article is prohibited. SPINEE1302.indd E310
01/02/14 9:30 AM
EDITORIAL
How Can We Design Low Back Pain Intervention Studies to Better Explain the Effects of Treatment? Gemma Mansell, MSc,* Jonathan C. Hill, PhD,* Steven J. Kamper, PhD,† Peter Kent, PhD,‡ Chris Main, PhD,* and Danielle A. van der Windt, PhD*
I
n this article, we consider whether adjustments to study designs, particularly of clinical trials, might provide important information to help explain how interventions for patients with low back pain (LBP) work. Back pain clinical trials have typically shown small effects associated with various treatments as compared with no treatment or current usual care, which could mean that these treatments are not very effective,1,2 or that the treatments are only effective for specific clinical subgroups and interventions therefore need better targeting.3 However, researchers also need to consider new methods to optimize treatment outcomes by investigating the processes which explain how or why specific treatments work.4 Such studies have been referred to as mediation analyses,5 and this article seeks to highlight why and how future studies should be designed to incorporate mediation analyses. The primary importance of mediation analysis is that it can help us understand how to improve treatments for patients with back pain. For example, psychological factors such as fear avoidance and self-efficacy are known to be predictive of patient outcomes such as disability and return to work, but it is less clear whether targeting an intervention specifically at reducing fear avoidance or increasing self-efficacy is able to further improve such outcomes. By analyzing the causal pathway between treatment, potential mediators and outcomes, we can identify the key factors to focus on during treatment and From the *Arthritis Research UK Primary Care Centre, Primary Care Sciences, Keele University, Keele, Staffordshire, United Kingdom; †Musculoskeletal Division, The George Institute for Global Health, University of Sydney, Australia and EMGO+ Institute, VU University Medical Centre, Amsterdam, the Netherlands; and ‡Spine Centre of Southern Denmark, Hospital Lillibælt, Institute of Regional Health Services Research, University of Southern Denmark, Odense, Denmark. Acknowledgment date: July 3, 2013. Revision date: September 16, 2013. Acceptance date: November 15, 2013. The manuscript submitted does not contain information about medical device(s)/drug(s). The National Institute for Health Research (NIHR) under its Programme Grants for Applied Research Programme (grant reference no. RP-PG-0707-10131) funds were received in support of this work. Relevant financial activities outside the submitted work: employment, grants, royalties. Address correspondence and reprint requests to Gemma Mansell, MSc, Arthritis Research UK Primary Care Centre, Primary Care Sciences, Keele University, Keele, Staffordshire, United Kingdom, ST5 5BG; E-mail: [email protected] DOI: 10.1097/BRS.0000000000000144 Spine
also identify which factors do not mediate outcome, to further streamline and improve intervention efficacy and efficiency. At the Twelfth International Low Back Pain Primary Care Research Forum in Odense, Denmark 2012, a dedicated workshop was held in which various aspects of mediation study design were explored and methodological considerations discussed. The workshop brought together experienced and internationally recognized LBP researchers and clinicians to provide an overview of the importance of mediation analyses and explore its usefulness in testing our assumptions about how our interventions may be influencing outcomes. We provide a summary of the workshop findings and discuss the implications for embedding mediation analysis methodology within future interventional research. We also offer definitions for some commonly used and misused terms, provide examples of mediation research using different study designs, and incorporate information on the methodology of mediation analysis from the wider literature, to formulate recommendations for future mediation research in the field of LBP. It is important to acknowledge that while the focus of this article is on mediators of specific treatment effects, the identification of mediator variables often comes from a program of work, which may involve different study designs. This article therefore includes a discussion of research using observational designs to explore mediating processes, although the limitations of evidence from such studies is noted and ultimately randomized controlled trials (RCTs) are required to confirm which factors mediate specific treatment effects.
CONCEPT DEFINITIONS When longitudinal data from an observational study (single groups of individuals observed over time) or an intervention study (2 or more groups of individuals, exposed to different treatment conditions) are analyzed, some factors may be consistently predictive of outcome. However, although such factors may show a statistical association with outcome following treatment, the relationship can take a number of different forms such as follows: a prognostic factor, a treatment effect moderator, or a treatment effect mediator.
Prognostic Factors Prognostic factors are baseline characteristics that are associated with outcome regardless of treatment, such as high baseline pain and high baseline activity limitation, which www.spinejournal.com
E305
Copyright © 2014 Lippincott Williams & Wilkins. Unauthorized reproduction of this article is prohibited. SPINEE1302.indd E305
01/02/14 9:30 AM
EDITORIAL
LBP Intervention Studies to Explain the Effects of Treatment • Mansell et al
are generally known to be associated with poor outcome in patients with spinal pain across a variety of different treatment interventions. Therefore, in an RCT, a prognostic factor might predict outcome for patients in both the intervention and control groups. In intervention studies, prognostic factors have also been called nonspecific predictors of treatment outcome.5 In statistical terms, they are baseline variables that are associated with outcome (they have a significant main effect only) but do not correlate or interact with treatment allocation.5
Treatment Effect Moderators A treatment effect moderator is a baseline variable that identifies people who do better or worse with a given treatment. These factors specify for whom and under what circumstances a treatment is most effective and are therefore central to stratified care that targets specific subgroups of people. For example, treatment A may work better for people with high fear of movement (who) or in people seeking primary care rather than secondary care (what circumstances) compared with treatment B. In statistical terms, treatment effect moderators stratify or subgroup the effect of a particular treatment and are indicated by a statistical interaction between treatment allocation (treatment A vs. B), time and subgroup (e.g., high vs. low fear).5 In some fields of research, treatment effect moderators are also referred to as effect modifiers or predictors of differential treatment effect, but these terms refer to the same thing.6
Treatment Effect Mediators In contrast to prognostic factors and treatment effect moderators, which are both measured only at baseline, treatment effect mediators are factors that change in response to a specific treatment and should therefore not only be measured at baseline but also during and after treatment. These factors help explain the relationship between treatment and outcome, and may therefore help identify the causal mechanisms by which treatments work. The identification of mediators can potentially improve treatment effects by strengthening the influence of the mediating factors involved in producing the treatment effect. Hypothetically it might be that in order for cognitive behavioral therapy (the treatment) to reduce activity limitation (the outcome), a patient’s self-efficacy (the mediator) needs to be strengthened, and that in the absence of self-efficacy improving, cognitive behavioral therapy will not be as effective as it otherwise might be. In statistical terms, changes in mediator scores are associated with specific treatment allocation (i.e., differ between intervention groups), and change in the mediator has a direct association with change in treatment outcome5 (Figure 1).
STUDY DESIGN IN MEDIATION ANALYSIS An attractive aspect of mediation analysis is its applicability to different study designs. The analytical process can be applied to data from cross-sectional studies, longitudinal observational cohorts and RCTs. However, different study designs do impose different limitations on the nature of the research question that can be answered and also the certainty with which causal inferences can be made. E306
www.spinejournal.com
Figure 1. Treatment and mediating effects. CBT indicates cognitive behavioral therapy. *Analogous to the main effect of treatment.
Cross-sectional Designs Mediation analysis has often been used with data from cross-sectional studies. With a convincing biological rationale and a priori defined hypotheses, these data can provide some evidence in support of pre-existing theory or be used to identify promising potential pathways that can then be tested and confirmed via more robust designs such as RCTs. However, causal inference requires a specified temporal (time-based) ordering of the investigated variables, and in cross-sectional studies where participants are only assessed at 1 time point, no temporal link can be made. Many publications that report the use of this design do acknowledge that the use of cross-sectional data precludes the inference of causation.7–9
Longitudinal Designs A number of authors have stipulated that longitudinal designs are required for the assessment of change over time.10,11 Longitudinal data enable the researcher to take control of previous levels of the mediator and outcome variables that can strengthen the claim of a potential causal association, but do not account for the potential influence of confounding variables that are not measured or included in the mediation analysis. Confounding is a key issue in mediation analysis using longitudinal and cross-sectional observational designs. Confounders are variables that are correlated with treatment exposure (i.e., show different values between treatment groups) or the independent variable and are associated with outcome, but unlike mediating variables are not on a causal pathway between treatment and outcome. A series of studies on people with spinal pain in Australia used longitudinal data from observational cohorts12,13 or pooled data from an RCT14 to investigate the mediating influence of psychological factors in the relationship between pain and disability. Three or more waves of data may be useful for assessing temporality as this provides stronger evidence in support of a causal pathway linking the variables,15–17 although it is acknowledged that this does increase March 2014
Copyright © 2014 Lippincott Williams & Wilkins. Unauthorized reproduction of this article is prohibited. SPINEE1302.indd E306
01/02/14 9:30 AM
EDITORIAL the complexity of the analysis and requires more complex modeling procedures.17
Experimental Designs Mediation analysis aims to identify causal associations, and experimental designs where patients are randomly allocated to a certain intervention at a point in time provides perhaps the strongest support for causal inference. It should be noted however that RCTs and observational studies seek to address different types of questions in mediation research. Studies using observational data investigate the relationships between variables and outcomes relevant to people with a particular condition, and may support the generation of hypotheses regarding potential mediators. Mediation studies that include allocation to 1 treatment or another as the independent variable investigate the mechanism of action of a particular treatment on outcome. It is this aspect of experimental designs which is of particular benefit in mediation analyses as it allows the investigation of the key variables (mediators) to target in future intervention studies. For example, studies of mediation analysis in RCTs have found that graded exposure treatment exerted part of its treatment effect in people with chronic LBP via reduction of catastrophizing,18 and showed that improvement in back pain was due to an increase in lumbar muscle endurance after an exercise program.19 Another advantage of experimental designs is their ability to reduce the effect of confounding. When an RCT design is used, randomization helps ensure that intervention groups have similar characteristics at baseline and so any differences in outcome are less likely to be as a result of 1 group differing substantially from the other to start with.16,20,21 This can help researchers to see whether any mediating effects would have occurred in spite of treatment (e.g., natural resolution) or whether they are due to the treatment being given and are therefore on a causal pathway.21 However, it has been recognized that the RCT design does not necessarily account for all unmeasured variables that could impact on outcome.22 This is because although participants may have been randomized to the intervention itself, change in the mediator(s) and outcome(s) occurs after randomization has taken place (if the intervention is effective),15 meaning that there may still be factors other than treatment that affect the relationship between the mediator and outcome variables.23 For example, although patients may have similar levels of fear avoidance at baseline due to the randomization process, during treatment some patients may develop better therapeutic relationships with their therapist than others, which may lead to a greater change in fear avoidance, and therefore a greater change in outcome for these particular patients. In this case, the therapeutic relationship is a potential confounder because it may have an impact on fear avoidance in addition to the impact of treatment.
ISSUES AROUND HETEROGENEITY IN PRIMARY CARE POPULATIONS Studies carried out in secondary or tertiary care populations, in which the condition is usually clearly established, unambiguously identified, and of sufficient severity or impact to Spine
LBP Intervention Studies to Explain the Effects of Treatment • Mansell et al
enable the identification of therapeutic change, may report larger and more consistent effect sizes than those conducted with primary care populations.24 A problem when investigating primary care populations may be the greater clinical variability in symptoms and their impact on the population under study. This could also influence mediation analyses as greater heterogeneity in potential mediators and outcomes results in larger standard deviations in the scores in the study population, making it harder to demonstrate a difference between treatments. Perhaps more importantly, greater clinical variability may also imply that the mechanism explaining treatment effect varies across patients, which can translate into difficulty in identifying the role of mediators in explaining treatment effects. To overcome this, larger sample sizes might be required so that the assessment of the effect of mediators in different subgroups can be undertaken. A specific advantage of primary care populations is that, from a public health perspective, the results may be more widely generalizable.
AREAS FOR IMPROVEMENT TO THE DESIGN OF FUTURE MEDIATION STUDIES A number of ways to incorporate mediation analysis into future RCTs by making small changes to their current design have been suggested. The list below outlines the main suggestions but is by no means exhaustive; more detailed reviews of the mediation literature, relating to both design and analysis issues, can be found elsewhere.11,23,25–27
Importance of a Theoretical Foundation Selecting mediators based on theory can help researchers choose the key factors to focus on,28 and will help formulate clear hypotheses regarding how a treatment might work before the study begins, which may subsequently be supported or refuted by the results of the mediation analysis.29 A theoretical basis will also help with the issue of temporality described in the earlier text because it provides a framework for showing how variables might relate to each other in a particular order.30
Design of Studies to Test Specifically for Mediation It is suggested that where an intervention seeks to change a particular mediator or set of mediators, measures of the proposed mediators should be included in the study.11 However, investigators should limit data collection to only include mediators that can potentially be modified by treatment and for which there is some evidence for a mediating relationship with the outcome measure.28 For example, researchers might hypothesize that an exercise program is effective due to its action of improving aerobic endurance in patients, or alternatively via improvements in self-efficacy. In these cases, measures of aerobic capacity and/or self-efficacy should be incorporated into the study design to specifically test these hypotheses. Sample size obviously has an effect on the power of a study to detect mediating effects. Although the required sample size is dependent on the choice of statistical technique, variability www.spinejournal.com
E307
Copyright © 2014 Lippincott Williams & Wilkins. Unauthorized reproduction of this article is prohibited. SPINEE1302.indd E307
01/02/14 9:30 AM
EDITORIAL
LBP Intervention Studies to Explain the Effects of Treatment • Mansell et al
of the measures and the expected strength of the associations between the variables, a substantial sample may be necessary. It is recommended that sophisticated mediation analysis methods require at least 15031 to 20032 participants to provide reliable results. One review33 found that of 166 mediation analysis studies published in 2 key journals in their field, 113 studies (68%) included sample sizes of above 150.
intervention may work differently for different people and the effects could occur at different times.11 Although attempting to untangle this issue adds considerable complexity, multiple appropriately timed measurements are important for exploring mediators of treatment.
Timing of Assessments
Measurement error is a particularly important problem when trying to assess whether change in a particular factor is responsible for the subsequent change in outcome, as in mediation analysis. Measurements need to be highly reliable when measuring change34 to ensure accurate estimates. Measurement error can become magnified when a raw change score (difference between pretest and post-test) is calculated as there is likely to be error associated with both measurements.34 A measure’s sensitivity to change is also important because this relates to whether or not a measure can detect change over time and separate this from measurement error.35 The use of measures with poor reliability or responsiveness in mediation studies may affect the user's ability to identify effects of mediating variables.
To assess temporality, it has been suggested that both the mediator and outcome should be assessed frequently during the period of time that change is expected to occur.16,24 This again shows the importance of considering treatment mediators a priori because a clear hypothesis regarding when a mediator is likely to change is important in planning the optimal timing for study measurement time points. Although studies that include assessments pre- and postintervention may provide some useful information on what mediating factors are related to outcome,16 they are not able to determine which variable (mediator or outcome) changed first.11 Following the example of the exercise intervention as proposed in the “Design of Studies to Test Specifically for Mediation” section, while changes in aerobic capacity are likely to take several weeks to occur, self-efficacy gains may be realized in a shorter time frame. These considerations will need careful thought to plan appropriate timing of study measurements. Also, an
Inclusion of Study Measures With Appropriate Measurement Properties
Incorporation of Exploratory Analysis In the workshop on mediation analysis held during the Twelfth Forum in Odense 2012 there was general consensus
Box 1. Recommendations to support and improve research into treatment effect mediators.
E308
www.spinejournal.com
March 2014
Copyright © 2014 Lippincott Williams & Wilkins. Unauthorized reproduction of this article is prohibited. SPINEE1302.indd E308
01/02/14 9:30 AM
EDITORIAL that, aside from the points mentioned in the earlier text, there is a need to return to exploratory analysis (e.g., mediation analyses in uncontrolled studies) and qualitative analysis to pinpoint the concepts that are most important in investigating treatment mechanisms. The importance of qualitative research in helping identify key factors to test as mediators has been highlighted previously,11 suggesting that this can be especially helpful in areas where no theories are available and can be used initially to explore therapeutic processes and their impact on therapist and patient. This implies that a program of research will often be needed to identify key mediators, with replication of findings to confirm the importance of those mediators being an essential part of that process.19,28
SUMMARY AND CONCLUSION The majority of trials investigating the effectiveness of primary care interventions for back pain have shown small or at best, moderate effects of treatment36,37 and the field is looking for better ways to improve outcomes for patients with back pain. Mediation analysis aims to provide better insight into the causal pathways underlying treatment effects, explaining why treatments work or do not work and potentially offering new opportunities to improve patient outcomes by optimizing the content or delivery of treatment. Until recently, mediation analysis in clinical research was often limited to a descriptive evaluation of the processes potentially underlying the effects of treatment in trials.38,39 During the past few years, interest in the use of more sophisticated approaches to mediation analysis has increased, often guided by methods described in the psychological literature. In this article, we have summarized the concepts and different designs used in mediation analysis and explained the importance of experimental designs when investigating mediators of treatment effect. We have also emphasized the importance of other considerations such as defining and understanding constructs, selecting study measures with appropriate measurement properties, and ensuring study measurement time points are appropriately selected, to investigate the longitudinal associations between mediating and outcome variables. We have also outlined the relevance of observational and qualitative research in identifying potential mediating factors. On the basis of the discussions during the 2012 workshop and supported by the literature, we have proposed a set of recommendations to support and improve the design of mediation analysis in back pain research (Box 1), with the ultimate aim to improve the design and delivery of intervention studies and optimize outcomes for patients with back pain.
Acknowledgment Author Chris Main thanks Dr. Kelvin Jordan, for his statistical advice.
References
1. van der Windt D, Hay E, Jellema P, et al. Psychosocial interventions for low back pain in primary care: lessons learnt from recent trials. Spine 2008;33:81–9.
Spine
LBP Intervention Studies to Explain the Effects of Treatment • Mansell et al
2. Guzman J, Esmail R, Karjalainen K, et al. Multidisciplinary rehabilitation for chronic low back pain: systematic review. BMJ 2001;322:1511–6. 3. Haldorsen EM, Grasdal AL, Skouen JS, et al. Is there a right treatment for a particular patient group? Comparison of ordinary treatment, light multidisciplinary treatment, and extensive multidisciplinary treatment for long-term sick-listed employees with musculoskeletal pain. Pain 2002;95:49–63. 4. Shaw WS, Linton SJ, Pransky G. Reducing sickness absence from work due to low back pain: how well do intervention strategies match modifiable risk factors? J Occup Rehabil 2006;16:591–605. 5. Kraemer HC, Wilson GT, Fairburn CG, et al. Mediators and moderators of treatment effects in randomized clinical trials. Arch Gen Psychiatry 2002;59:877–83. 6. Kraemer HC, Stice E, Kazdin A, et al. How do risk factors work together? Mediators, moderators, and independent, overlapping, and proxy risk factors. Am J Psychiatry 2001;158:848–56. 7. Woby SR, Roach NK, Urmston M, et al. The relation between cognitive factors and levels of pain and disability in chronic low back pain patients presenting for physiotherapy. Eur J Pain 2007;11:869–77. 8. Arnstein P, Caudill M, Mandle CL, et al. Self-efficacy as a mediator of the relationship between pain intensity, disability and depression in chronic pain patients. Pain 1999;80:483–91. 9. Gheldof EL, Vinck J, Van den Bussche E, et al. Pain and painrelated fear are associated with functional and social disability in an occupational setting: evidence of mediation by pain-related fear. Eur J Pain 2006;10:513–25. 10. Liu LC, Flay BR. Aban Aya Investigators. Evaluating mediation in longitudinal multivariate data: mediation effects for the Aban Aya Youth Project drug prevention program. Prev Sci 2009;10: 197–207. 11. Kazdin AE. Mediators and mechanisms of change in psychotherapy research. Annu Rev Clin Psychol 2007;3:1–27. 12. Kamper SJ, Maher CG, Costa Lda CM, et al. Does fear of movement mediate the relationship between pain intensity and disability in patients following whiplash injury? A prospective longitudinal study. Pain 2012;153:113–9. 13. Costa Lda CM, Maher CG, McAuley JH, et al. Self-efficacy is more important than fear of movement in mediating the relationship between pain and disability in chronic low back pain. Eur J Pain 2011;15: 213–9. 14. Hall AM, Kamper SJ, Maher CG, et al. Symptoms of depression and stress mediate the effect of pain on disability. Pain 2011;152: 1044–51. 15. Collins LM, Graham JW, Flaherty BP. An alternative framework for defining mediation. Multivar Behav Res 1998;33:295–312. 16. Lockhart G, MacKinnon DP, Ohlrich V. Mediation analysis in psychosomatic medicine research. Psychosom Med 2011;73:29–43. 17. Maric M, Wiers RW, Prins PJ. Ten ways to improve the use of statistical mediation analysis in the practice of child and adolescent treatment research. Clin Child Fam Psychol Rev 2012;15:177–91. 18. Leeuw M, Goossens ME, van Breukelen GJ, et al. Exposure in vivo versus operant graded activity in chronic low back pain patients: results of a randomized controlled trial. Pain 2008;138:192–207. 19. Mulroy SJ, Winstein CJ, Kulig K, et al. Secondary mediation and regression analyses of the PTClinResNet database: Determining causal relationships among the international classification of functioning, disability and health levels for four physical therapy intervention trials. Phys Ther 2011;91:1766–79. 20. Bullock JG, Green DP, Ha SE. Yes, but what’s the mechanism? (don’t expect an easy answer). J Pers Soc Psychol 2010;98:550–58. 21. Murphy R, Cooper Z, Hollon SD, et al. How do psychological treatments work? Investigating mediators of change. Behav Res Ther 2009;47:1–5. 22. Judd CM, Kenny DA. Process analysis: Estimating mediation in treatment evaluations. Eval Rev 1981;5:602–19. 23. Emsley R, Dunn G, White IR. Mediation and moderation of treatment effects in randomised controlled trials of complex interventions. Stat Methods Med Res 2010;19:237–70. www.spinejournal.com
E309
Copyright © 2014 Lippincott Williams & Wilkins. Unauthorized reproduction of this article is prohibited. SPINEE1302.indd E309
01/02/14 9:30 AM
EDITORIAL 24. Grissom RJ, Kim JJ. Effect Sizes for Research: Univariate and Multivariate Applications. 2nd ed. New York: Taylor & Francis (Routledge); 2012. 25. Hayes AF. Beyond Baron and Kenny: statistical mediation analysis in the new millennium. Commun Monogr 2009;76:408–20. 26. MacKinnon DP, Fairchild AJ, Fritz MS. Mediation analysis. Ann Rev Psychol 2007;58:593–614. 27. MacKinnon DP, Fairchild AJ. Current directions in mediation analysis. Curr Dir Psychol Sci 2009;18:16–20. 28. MacKinnon DP. Analysis of mediating variables in prevention and intervention research. Natl Inst Drug Abuse Monogr 1994;139:127–53. 29. MacKinnon DP. Integrating mediators and moderators in research design. Res Social Work Pract 2011;21:675–81. 30. Mathieu JE, Taylor SR. Clarifying conditions and decision points for mediational type inferences in organizational behavior. J Org Behav 2006;27: 1031–56. 31. Holbert RL, Stephenson MT. Structural equation modeling in the communication sciences, 1995–2000. J Hum Commun Res 2008;28:531–51. 32. Tomarken AJ, Waller NG. Potential problems with “well fitting” models. J Abnormal Psychol 2003;112:578–98.
E310
www.spinejournal.com
LBP Intervention Studies to Explain the Effects of Treatment • Mansell et al
33. Fritz MS, MacKinnon DP. Required sample size to detect the mediated effect. Psychol Sci 2007;18:233–9. 34. Streiner DL, Norman GR. Health Measurement Scales: A Practical Guide to Their Development and Use. 4th ed Oxford: Oxford University Press; 2008. 35. Terwee CB, Bot SD, de Boer MR, et al. Quality criteria were proposed for measurement properties of health status questionnaires. J Clin Epidemiol 2007;60:34–42. 36. Keller A, Hayden J, Bombardier C, et al. Effect sizes of nonsurgical treatments of nonspecific low back pain. Eur Spine J 2007;16: 1776–88. 37. Machedo LA, Kamper SJ, Herbert RD, et al. Analgesic effects of treatments for nonspecific low back pain: a meta-analysis of placebo-controlled randomized trials. Rheumatology (Oxford) 2009;48:520–7. 38. Jellema P, van der Windt DA, van der Horst HE, et al. Why is a treatment aimed at psychosocial factors not effective in patients with (sub)acute low back pain? Pain 2005;118: 350–9. 39. Thorn BE, Day MA, Burns J, et al. Randomized trial of group cognitive behavioral therapy compared with a pain education control for low-literacy rural people with chronic pain. Pain 2011;152: 2710–20.
March 2014
Copyright © 2014 Lippincott Williams & Wilkins. Unauthorized reproduction of this article is prohibited. SPINEE1302.indd E310
01/02/14 9:30 AM
