Journal of Clinical Epidemiology
The hierarchy of evidence and quantum theory Adam La Caze* School of Pharmacy, Pharmacy Australia Centre of Excellence, The University of Queensland, Level 4, 20 Cornwall Street Woolloongabba, 4102 Brisbane, St Lucia, QLD 4072, Australia Accepted 17 June 2015; Published online xxxx
Medicine is complex, and evidence regarding the effects of interventions is subject to multiple sources of bias. The dominant view is that the ‘‘best available evidence’’ for the effects of interventions comes from randomized trials or systematic reviews of randomized trials. For instance, evidence-based medicine provides a hierarchy of evidence for assessing the benefits of treatments, which places well-conducted randomized trials and systematic reviews of randomized trials above other sources of evidence including observational studies. The most recent version of the levels of evidence from the Oxford Centre for Evidence-based Medicine provides additional advice for when evidence may be graded up or down a level based on the characteristics of the study . The hierarchy of evidence is justified on the basis of internal validityd methods listed higher in the hierarchy have the capacity to reduce more sources of bias than those listed lower . Although the limitations of randomized trials are well recognized [3e5], the idea that medical decisions in a wide variety of contexts are best informed by the results of randomized trials remains prevalent. In this edition of the journal, Walach and Loef  argue that the theoretical foundations of the hierarchy of evidence are incorrect and should be replaced with a theoretical structure analogous to quantum theory. Walach and Loef argue that a nonlinear, noncommutative theoretical structure would provide a better foundation for amalgamating the different types of evidence that inform therapeutic decisions. The theoretical structure Walach and Loef propose represents a bold departure from current thinking on medical evidence and medical evidence amalgamation. Bold positions need compelling arguments. I think Walach and Loef are yet to provide these arguments, but there are benefits to closely considering their position and possible alternatives. Walach and Loef correctly highlight the problems of relying solely on the hierarchy of evidence to inform therapeutic decisions. Therapeutic decisions require a judgment
Conflict of interest: None. * Corresponding author. Tel.: þ61-7-3346-1985; Fax: þ61-7-3346-1999. E-mail address: [email protected]
http://dx.doi.org/10.1016/j.jclinepi.2015.06.014 0895-4356/Ó 2015 Elsevier Inc. All rights reserved.
of both efficacy and effectiveness and as such an assessment of evidence in terms of both internal and external validity. The hierarchy of evidence as a hierarchy of internal validity helps to inform the assessment of whether the treatment is efficacious. Because therapeutic decisions require consideration of harms and the applicability of evidence to individuals (among other things), it is often necessary to amalgamate evidence from a range of sources. Although internal validity is important, it is not sufficient for therapeutic decisions and Walach and Loef are right to call for a more inclusive approach. But it is possible to do better in this respect without accepting that a quantum theoretical structure underpins the relationship between different types of biomedical evidence. Walach and Loef argue that there are aspects of biomedical evidence, and the way in which it is used to inform therapeutic decisions, that are importantly analogous to the theoretical structure of quantum physics. Specifically, that there exists incompatibility and noncommutativity in the sense required of quantum theoretical structures. Walach and von Stillfried provide a guide to the way in which the authors wish to generalize quantum theory . An example they provide from quantum mechanics is the challenge of describing the physical properties of a particular quantum-level particle. The description requires the location and the momentum of the particle. The problem is that it is not possible to simultaneously measure the location and momentum of the particle: our attempts to measure the location of the particle will affect the particle’s momentum, and vice versa. Measuring the location and momentum of the particle is incompatible at a fundamental level. In areas of classical physics, it is possible to independently measure location and momentum and the order in which they are measured can be considered irrelevant. This is not the case on the quantum level, the sequence in which we measure the location and momentum influences our observations of the particledin this way, the sequence of operations is noncommuting. The challenge for Walach and Loef is to demonstrate how incompatibility and noncommutativity arise in applying biomedical evidence to therapeutic decisions. Incompatibility, they argue, arises via internal and external
A. La Caze / Journal of Clinical Epidemiology
validity. This is because many of the methods used to improve the internal validity of a study reduce the generalizability and thus external validity of the study. There is no doubt that internal and external validity are often in tension, but suggesting they are incompatible in the sense required for quantum theory surely stretches the analogy too far. Internal and external validity are judgments that can be made of every study. Judging the internal validity of a specific result is relatively self-contained in the sense that with the help of a little background information, many of the details that influence the internal validity of the observed result relate to the methods used in the study. External validity, by contrast, is best viewed as a causal assessment and as such requires consideration of a broad range of evidence [8,9]. Relying solely on randomized trials to assess external validity is problematic. But the lesson here is not that internal and external validity are incompatible in some fundamental sense, rather, that the hierarchy of evidence informs judgments of internal validity better than it informs judgments of external validity. The argument provided for noncommutativity of biomedical evidence also stretches the analogy with quantum theory. Walach and Loef argue that the sequence of clinical studiesdspecifically, whether observational studies precede or follow randomized trialsdinfluences the findings of the studies. The kind of examples Walach and Loef have in mind regard traditional healing practices or interventions such as meditation that require an active participant, preferably an individual who believes that the intervention will be effective. The idea is that many of these interventions are unsuccessful when tested in randomized trials because the kind of participants in whom the intervention will be successful are more likely to refuse random allocation to treatment or control. This has the effect that ‘‘passive’’ participants are preferentially enrolled, and as a result, any estimate of the effect of the intervention is unreliable. Walach and Loef suggest that it is the long period of observed effectiveness among practitioners and recipients of these therapies that undermines the validity of randomized trials. This, they argue, highlights the importance of the sequence of clinical studies on the observed results. A second part of the argument for noncommutativity focuses on the assumption of randomized trials that the specific effects of an intervention can be isolated from, and added to, the nonspecific effects. The so-called ‘‘efficacy paradox’’ arises when treatment A, which is considered effective by practitioners and recipients of a treatment, but is not effective when compared with its own placebo, is contrasted to treatment B, which possesses moderatespecific effects but small nonspecific effects. The suggestion is that treatment A is more effective than treatment B, but that only treatment B will be deemed effective in randomized trials. The authors refer to recent acupuncture trials as an illustration of the paradox [10e12]. In two of the trials, acupuncture and sham acupuncture were more effective than conventional therapy for chronic low back
pain and osteoarthritis of the knee, respectively, but in none of the trials was acupuncture superior to sham acupuncture. Walach and Loef interpret these trials as evidence that specific and nonspecific effects are synergistic rather than additive, and this is taken as further evidence that the sequence of studies is important. ‘‘Patients need to first believe, through previous studies and their public perception or through a popular myth, that an intervention is effective, before a strong placebo-response can be produced. If one were to take away the specific effects and trust the placebocomponent only, the latter would collapse as well.’’ Walach and Loef fail to demonstrate noncommutativity in biomedical evidence. A simpler interpretation is available for each of the arguments that they provide. It is important to acknowledge that the participant’s willingness to be randomly allocated treatment can introduce bias into a clinical trial. But this, like many other sources of bias can be mitigated and/or acknowledged when interpreting the results. If it is thought that not receiving a preferred treatment will make the trial unattractive to potential participants, a crossover design can be used, or alternatively all participants can be offered the active intervention at the conclusion of the trial. Psychological interventions require active participation and have demonstrated benefit when tested in randomized trials . Likewise, there are significant examples in which the specific effects of an intervention have been demonstrated despite these specific effects being in the opposite direction to extensive observational studies and wide-spread beliefs . The possible interactions between specific and nonspecific effects of an intervention are complex and important to consider. But the cases Walach and Loef discuss provide no empirical evidence of synergy. The standard additive assumption of randomized trials provides a more straightforward interpretation of the German acupuncture trials. The nonspecific effects of acupuncture and sham acupuncture were greater than the specific plus nonspecific effects of conventional therapy. This is likely to be a mixture of participant beliefs, the selected patient population, the effects of touch or other aspects of acupuncture/sham acupuncture, and the ineffectiveness of conventional treatments for chronic osteoarthritis of the knee and low back pain . Nonspecific effects should not be ignored but nor should they be imbued with mysterious properties. Perhaps, the findings observed in the acupuncture trials will help identify specific effects that can be isolated, tested, and replicated. Walach and Loef are arguing for a paradigm shift. They wish to replace the theoretical foundation of ‘‘evidencebased’’ decision making with something considerably more complex. A paradigm shift is needed when the prevailing paradigm is unable to accommodate important findings, and a new paradigm is available that accommodates the
A. La Caze / Journal of Clinical Epidemiology
anomalous observations and provides resources for new directions of research. This does not appear to be the situation we are in. We need ways to better conceptualize judgments of external validity and effectiveness. Similarly, we need better methods to amalgamate and communicate the diverse range of evidence that informs important therapeutic decisions. It seems unlikely, however, that we will achieve these aims by adopting quantum theory as a model for thinking about evidence in medicine.
References  OCEBM Levels of Evidence Working Group. Oxford Centre for Evidence-Based Medicine 2011 Levels of Evidence [Internet] 2011: Available at: http://www.cebm.net/ocebm-levels-of-evidence. Accessed July 21, 2015.  La Caze A. Evidence-based medicine must be. J Med Philos 2009;34: 509e27.  Vandenbroucke JP, Psaty BM. Benefits and risks of drug treatments: how to combine the best evidence on benefits with the best data about adverse effects. JAMA 2008;300:2417e9.  Black N. Why we need observational studies to evaluate the effectiveness of health care. BMJ 1996;312:1215e8.  Rothwell PM. Assessment of the external validity of randomised controlled trials. In: Rothwell PM, editor. Treating individuals: From randomised trials to personalised medicine. London: Elsevier; 2007: 61e82.  Walach H, Loef M. Using a matrix-analytical approach to synthesising evidence solved incompatibility problem in the hierarchy of
evidence. J Clin Epidemiol 2015. http://dx.doi.org/10.1016/j.jclinepi. 2015.03.027. [Epub ahead of print]. Walach H, von Stillfried N. Generalised quantum theory-basic idea and general intuition: a background story and overview. Axiomathes 2011;21:185e209. Rothman KJ, Greenland S, Lash TL. Validity in epidemiologic studies. In: Rothman KJ, Greenland S, Lash TL, editors. Modern epidemiology. 3rd ed. Philadelphia: Lippincott Williams & Wilkins; 2008:128e47. Cartwright N. Predicting what will happen when we act. What counts for warrant? Prev Med 2011;53:221e4. Haake M, Muller H-H, Schade-Brittinger C, Basler HD, Schafer H, Maier C, et al. German acupuncture trials for chronic low back pain. Arch Intern Med 2007;167:1892e8. Diener HC, Kronfeld K, Boewing G, Lungenhausen M, Maier C, Molsberger A, et al. Efficacy of acupuncture for the prophylaxis of migraine: a multicentre randomised controlled clinical trial. Lancet Neurol 2006;5:310e6. Scharf HP, Mansmann U, Streitberger K, Witte S, Kramer J, Maier C, et al. Acupuncture and knee osteoarthritis. Ann Intern Med 2006;145: 12e20. ~ Linde K, Sigterman K, Kriston L, RAijcker G, Jamil S, Meissner K, et al. Effectiveness of psychological treatments for depressive disorders in primary care: systematic review and meta-analysis. Ann Fam Med 2015;13:56e68. Women’s Health Initiative Investigators. Risks and benefits of estrogen plus progestin in healthy postmenopausal women. JAMA 2002; 288:321e33. Williams CM, Maher CG, Latimer J, McLachlan AJ, Hancock MJ, Day RO, et al. Efficacy of paracetamol for acute low-back pain: a double-blind, randomised controlled trial. Lancet 2014;384: 1586e96.