Eur J Anaesthesiol 2015; 32:77–78
EDITORIAL
Minimal clinically important difference A novel approach to measure changes in outcome in perioperative medicine Bernhard Walder, Julien Maillard and Anne Lu¨bbeke
European Journal of Anaesthesiology 2015, 32:77–78
Traditionally perioperative medicine concentrates on postoperative mortality and postoperative morbidity, most often measured at 30 days. There are, however, other important outcomes, in particular in older patients, such as functional and physical health and quality of life. Typically these outcomes are evaluated as comparisons between the pre and postinterventional state. As patients, clinicians and researchers, it is precisely the change resulting from an intervention that is of greatest interest. Examples of postintervention change might be postoperative disability after cerebral surgery, reduced exercise capacity and health-related quality of life (HRQoL) after orthopaedic surgery or it could be individual differences in response to treatment (e.g. pain treatment). Standardised questionnaires or scores, also called patient-reported outcomes (PROs), assist the collection of relevant patient data. The HRQoL is one example of these. Their acceptance as an important adjunct to the traditional outcomes has steadily increased amongst health professionals, not least because they are convenient for collecting large amounts of information, for instance, through the use of patient data management systems. A major challenge is to detect and describe clinically relevant changes of health status measures over time (responsiveness). These clinically relevant or significant changes have to be distinguished from statistically significant changes, which may be clinically irrelevant. For instance, a pain treatment may decrease pain score from 2.2 to 2.0 (measured with a valid numeric rating scale; minimal 0, maximal 10) and the result may in a larger sample be statistically significant, but this degree of change is, for most patients, imperceptible.
The concept of ‘minimal clinically important difference’ (MCID) has been introduced in an effort to define what is the smallest meaningful change that a patient can detect with confidence. Using pain treatment again as an example, a decrease in pain score from 5.0 to 3.0, measured with the same numeric rating scale, would be, for most patients, perceptible and clinically relevant, even if the result proved to be statistically insignificant due to a large variability and small sample size.1 An important assumption in this example is that the instrument ‘numerical rating scale’ can indeed reliably detect this change in (subjective) pain. Another example is a change in the HRQoL score in which, in patients with chronic disease, the MCID of the physical component score (SF-36) has been identified as 5 units.2 In other words, a change less than 5 units is likely to go undetected, although an increase or decrease equal to or greater than 5 will be perceived as either beneficial or harmful, with the potential for bringing about a change in management.3 MCIDs, therefore, improve the clinical interpretability of patient outcomes.4 Two approaches to measuring the clinical significance of a change in health status and defining the MCID have been proposed. These are termed distribution-based and anchor-based methods.5 The former includes methods based on statistical significance, sample variability (distribution) or measurement precision.6 Distribution-based methods are easy to generate. The most widely used distribution-based method, the sample variability method, is based on the postulate that there is a relationship between the MCID and variation (standard deviation) across a specific content area; the larger the standard deviation, the larger the MCID. Cohen in his influential publication from 1988 used effect sizes as a distribution-based measure of clinical significance.7 He suggested that a change of 0.2 standard deviation units
From the Division of Anaesthesiology (BW, JM), and Division of Orthopaedic Surgery (AL), Geneva University Hospitals, Geneva, Switzerland Correspondence to Prof Bernhard Walder, Division of Anaesthesiology, Department APSI, Geneva University Hospitals, Rue Gabrielle-Perret-Gentil 4, CH-1211 Gene`ve 14, Switzerland Tel: +41 22 382 30 63; fax: +41 22 382 75 51; e-mail: [email protected] 0265-0215 ß 2015 Copyright European Society of Anaesthesiology
DOI:10.1097/EJA.0000000000000147
Copyright © European Society of Anaesthesiology. Unauthorized reproduction of this article is prohibited.
78 Walder et al.
corresponded to a small clinically meaningful change (small effect size), a change of 0.5 units to a moderate change (moderate effect size) and a change of 0.8 standard deviation units corresponded to a large clinically meaningful change (large effect size). Others have confirmed Cohen’s suggested benchmarks for the estimation of clinically important differences.8,9 Anchor-based methods link the instrument change to a meaningful external anchor (valid independent measure). For instance, for HRQoL scores MCIDs can be estimated using the anchor ‘Karnofsky performance status’, a clinical rating of a patient’s functional capacity.10 The MCID should be based on a clinical anchor that has a correlation with the HRQoL between at least 0.30 and at least 0.50.11,12 On the basis of the anchor instrument, three groups are generally built: improvement, no change, deterioration. It has been suggested that MCIDs should preferably be determined by anchor-based, rather than distributionbased methods.11 However, the anchor instrument may have limitations (ceiling effect).10 Furthermore, the difference between distribution-based and anchor-based methods may be minor, at least in certain domains.13 When there is a difference, MCID ranges seem to be slightly larger with distribution-based methods.14 Therefore, distribution-based methods may overestimate MCID. Nevertheless, as no single anchor is ideal and no single method is perfect (no gold standard), it is recommended that multiple approaches from both anchor-based and distribution-based methods are used to estimate the MCID for patient outcome instruments.11 The variability of the MCID may be large as different outcomes will have their own specific MCID. The MCID for pain, or health-related quality of life, or disability will vary and not be constant. Furthermore, the MCID may vary according to the baseline score, at least in some instances.15 The MCID may also not distinguish between improvement and importance. Last, but not least, the MICD may vary in cohorts with different case-mixes and in different cultural contexts. As a result these limitations, better studies should use more than one instrument with its own MCID for a single particular clinical context (more than one point estimate per context). Further research in perioperative medicine should assess patient outcomes and use MCIDs to guide sample size estimation. Moreover, the reporting of the results should include measures of the clinical meaningfulness of the reported change expressed for example, as the effect size of a change or the proportion of patients with a clinically perceptible improvement based on a predefined MCID. Finally, the proportions provided by MCIDs allow for calculating the number needed-to-treat (NNT), a powerful measure to evaluate treatment effects.16
In future, relevant research in perioperative medicine will include more systematic use of PROs. The reporting on changes in PROs in perioperative medicine should be based on clinical relevance and not on statistical significance. The MCID concept is a promising way of improving outcome research in perioperative medicine and the results obtained are more intuitive for clinicians and patients.
Acknowledgements relating to this article Assistance with the Editorial: none. Financial support and sponsorship: none. Conflict of interest: none. Comment from the Editor: this Editorial was checked by the editors but was not sent for external peer review. BW is a Deputy Editor-inChief of the European Journal of Anaesthesiology.
References 1
2
3
4
5
6
7 8
9
10
11
12
13
14
15
16
Farrar JT, Young JP, LaMoreaux L, et al. Clinical importance of changes in chronic pain intensity measured on an 11-point numerical pain rating scale. Pain 2001; 94:149–158. Ware J, Kosinski M, Keller SD. A 12-Item Short-Form Health Survey: construction of scales and preliminary tests of reliability and validity. Med Care 1996; 34:220–233. Revicki DA, Cella D, Hays RD, et al. Responsiveness and minimal important differences for patient reported outcomes. Health Qual Life Outcomes 2006; 4:70. Johnston BC, Thorlund K, Schu¨nemann HJ, et al. Improving the interpretation of quality of life evidence in meta-analyses: the application of minimal important difference units. Health Qual Life Outcomes 2010; 8:116. Guyatt GH, Osoba D, Wu AW, et al., Clinical Significance Consensus Meeting Group. Methods to explain the clinical significance of health status measures. Mayo Clin Proc Mayo Clin 2002; 77:371–383. Crosby RD, Kolotkin RL, Williams GR. Defining clinically meaningful change in health-related quality of life. J Clin Epidemiol 2003; 56:395– 407. Cohen J. Statistical power analysis for the behavioral sciences. New Jersey, USA: Lawrence Erlbaum Associates; 1988. Norman GR, Sloan JA, Wyrwich KW. Interpretation of changes in healthrelated quality of life: the remarkable universality of half a standard deviation. Med Care 2003; 41:582–592. Samsa G, Edelman D, Rothman ML, et al. Determining clinically important differences in health status measures: a general approach with illustration to the Health Utilities Index Mark II. Pharmacoeconomics 1999; 15:141– 155. Sagberg LM, Jakola AS, Solheim O. Quality of life assessed with EQ-5D in patients undergoing glioma surgery: what is the responsiveness and minimal clinically important difference? Qual Life Res 2014; 23:1427– 1434. Revicki D, Hays RD, Cella D, Sloan J. Recommended methods for determining responsiveness and minimally important differences for patient-reported outcomes. J Clin Epidemiol 2008; 61:102–109. Turner D, Schu¨nemann HJ, Griffith LE, et al. The minimal detectable change cannot reliably replace the minimal important difference. J Clin Epidemiol 2010; 63:28–36. Johnston BC, Thorlund K, da Costa BR, et al. New methods can extend the use of minimal important difference units in meta-analyses of continuous outcome measures. J Clin Epidemiol 2012; 65:817–826. Le QA, Doctor JN, Zoellner LA, Feeny NC. Minimal clinically important differences for the EQ-5D and QWB-SA in Posttraumatic Stress Disorder (PTSD): results from a Doubly Randomized Preference Trial (DRPT). Health Qual Life Outcomes 2013; 11:59. Lauridsen HH, Hartvigsen J, Manniche C, et al. Responsiveness and minimal clinically important difference for pain and disability instruments in low back pain patients. BMC Musculoskelet Disord 2006; 7:82. Cook RJ, Sackett DL. The number needed to treat: a clinically useful measure of treatment effect. BMJ 1995; 310:452–454.
Eur J Anaesthesiol 2015; 32:77–78 Copyright © European Society of Anaesthesiology. Unauthorized reproduction of this article is prohibited.
EDITORIAL
Minimal clinically important difference A novel approach to measure changes in outcome in perioperative medicine Bernhard Walder, Julien Maillard and Anne Lu¨bbeke
European Journal of Anaesthesiology 2015, 32:77–78
Traditionally perioperative medicine concentrates on postoperative mortality and postoperative morbidity, most often measured at 30 days. There are, however, other important outcomes, in particular in older patients, such as functional and physical health and quality of life. Typically these outcomes are evaluated as comparisons between the pre and postinterventional state. As patients, clinicians and researchers, it is precisely the change resulting from an intervention that is of greatest interest. Examples of postintervention change might be postoperative disability after cerebral surgery, reduced exercise capacity and health-related quality of life (HRQoL) after orthopaedic surgery or it could be individual differences in response to treatment (e.g. pain treatment). Standardised questionnaires or scores, also called patient-reported outcomes (PROs), assist the collection of relevant patient data. The HRQoL is one example of these. Their acceptance as an important adjunct to the traditional outcomes has steadily increased amongst health professionals, not least because they are convenient for collecting large amounts of information, for instance, through the use of patient data management systems. A major challenge is to detect and describe clinically relevant changes of health status measures over time (responsiveness). These clinically relevant or significant changes have to be distinguished from statistically significant changes, which may be clinically irrelevant. For instance, a pain treatment may decrease pain score from 2.2 to 2.0 (measured with a valid numeric rating scale; minimal 0, maximal 10) and the result may in a larger sample be statistically significant, but this degree of change is, for most patients, imperceptible.
The concept of ‘minimal clinically important difference’ (MCID) has been introduced in an effort to define what is the smallest meaningful change that a patient can detect with confidence. Using pain treatment again as an example, a decrease in pain score from 5.0 to 3.0, measured with the same numeric rating scale, would be, for most patients, perceptible and clinically relevant, even if the result proved to be statistically insignificant due to a large variability and small sample size.1 An important assumption in this example is that the instrument ‘numerical rating scale’ can indeed reliably detect this change in (subjective) pain. Another example is a change in the HRQoL score in which, in patients with chronic disease, the MCID of the physical component score (SF-36) has been identified as 5 units.2 In other words, a change less than 5 units is likely to go undetected, although an increase or decrease equal to or greater than 5 will be perceived as either beneficial or harmful, with the potential for bringing about a change in management.3 MCIDs, therefore, improve the clinical interpretability of patient outcomes.4 Two approaches to measuring the clinical significance of a change in health status and defining the MCID have been proposed. These are termed distribution-based and anchor-based methods.5 The former includes methods based on statistical significance, sample variability (distribution) or measurement precision.6 Distribution-based methods are easy to generate. The most widely used distribution-based method, the sample variability method, is based on the postulate that there is a relationship between the MCID and variation (standard deviation) across a specific content area; the larger the standard deviation, the larger the MCID. Cohen in his influential publication from 1988 used effect sizes as a distribution-based measure of clinical significance.7 He suggested that a change of 0.2 standard deviation units
From the Division of Anaesthesiology (BW, JM), and Division of Orthopaedic Surgery (AL), Geneva University Hospitals, Geneva, Switzerland Correspondence to Prof Bernhard Walder, Division of Anaesthesiology, Department APSI, Geneva University Hospitals, Rue Gabrielle-Perret-Gentil 4, CH-1211 Gene`ve 14, Switzerland Tel: +41 22 382 30 63; fax: +41 22 382 75 51; e-mail: [email protected] 0265-0215 ß 2015 Copyright European Society of Anaesthesiology
DOI:10.1097/EJA.0000000000000147
Copyright © European Society of Anaesthesiology. Unauthorized reproduction of this article is prohibited.
78 Walder et al.
corresponded to a small clinically meaningful change (small effect size), a change of 0.5 units to a moderate change (moderate effect size) and a change of 0.8 standard deviation units corresponded to a large clinically meaningful change (large effect size). Others have confirmed Cohen’s suggested benchmarks for the estimation of clinically important differences.8,9 Anchor-based methods link the instrument change to a meaningful external anchor (valid independent measure). For instance, for HRQoL scores MCIDs can be estimated using the anchor ‘Karnofsky performance status’, a clinical rating of a patient’s functional capacity.10 The MCID should be based on a clinical anchor that has a correlation with the HRQoL between at least 0.30 and at least 0.50.11,12 On the basis of the anchor instrument, three groups are generally built: improvement, no change, deterioration. It has been suggested that MCIDs should preferably be determined by anchor-based, rather than distributionbased methods.11 However, the anchor instrument may have limitations (ceiling effect).10 Furthermore, the difference between distribution-based and anchor-based methods may be minor, at least in certain domains.13 When there is a difference, MCID ranges seem to be slightly larger with distribution-based methods.14 Therefore, distribution-based methods may overestimate MCID. Nevertheless, as no single anchor is ideal and no single method is perfect (no gold standard), it is recommended that multiple approaches from both anchor-based and distribution-based methods are used to estimate the MCID for patient outcome instruments.11 The variability of the MCID may be large as different outcomes will have their own specific MCID. The MCID for pain, or health-related quality of life, or disability will vary and not be constant. Furthermore, the MCID may vary according to the baseline score, at least in some instances.15 The MCID may also not distinguish between improvement and importance. Last, but not least, the MICD may vary in cohorts with different case-mixes and in different cultural contexts. As a result these limitations, better studies should use more than one instrument with its own MCID for a single particular clinical context (more than one point estimate per context). Further research in perioperative medicine should assess patient outcomes and use MCIDs to guide sample size estimation. Moreover, the reporting of the results should include measures of the clinical meaningfulness of the reported change expressed for example, as the effect size of a change or the proportion of patients with a clinically perceptible improvement based on a predefined MCID. Finally, the proportions provided by MCIDs allow for calculating the number needed-to-treat (NNT), a powerful measure to evaluate treatment effects.16
In future, relevant research in perioperative medicine will include more systematic use of PROs. The reporting on changes in PROs in perioperative medicine should be based on clinical relevance and not on statistical significance. The MCID concept is a promising way of improving outcome research in perioperative medicine and the results obtained are more intuitive for clinicians and patients.
Acknowledgements relating to this article Assistance with the Editorial: none. Financial support and sponsorship: none. Conflict of interest: none. Comment from the Editor: this Editorial was checked by the editors but was not sent for external peer review. BW is a Deputy Editor-inChief of the European Journal of Anaesthesiology.
References 1
2
3
4
5
6
7 8
9
10
11
12
13
14
15
16
Farrar JT, Young JP, LaMoreaux L, et al. Clinical importance of changes in chronic pain intensity measured on an 11-point numerical pain rating scale. Pain 2001; 94:149–158. Ware J, Kosinski M, Keller SD. A 12-Item Short-Form Health Survey: construction of scales and preliminary tests of reliability and validity. Med Care 1996; 34:220–233. Revicki DA, Cella D, Hays RD, et al. Responsiveness and minimal important differences for patient reported outcomes. Health Qual Life Outcomes 2006; 4:70. Johnston BC, Thorlund K, Schu¨nemann HJ, et al. Improving the interpretation of quality of life evidence in meta-analyses: the application of minimal important difference units. Health Qual Life Outcomes 2010; 8:116. Guyatt GH, Osoba D, Wu AW, et al., Clinical Significance Consensus Meeting Group. Methods to explain the clinical significance of health status measures. Mayo Clin Proc Mayo Clin 2002; 77:371–383. Crosby RD, Kolotkin RL, Williams GR. Defining clinically meaningful change in health-related quality of life. J Clin Epidemiol 2003; 56:395– 407. Cohen J. Statistical power analysis for the behavioral sciences. New Jersey, USA: Lawrence Erlbaum Associates; 1988. Norman GR, Sloan JA, Wyrwich KW. Interpretation of changes in healthrelated quality of life: the remarkable universality of half a standard deviation. Med Care 2003; 41:582–592. Samsa G, Edelman D, Rothman ML, et al. Determining clinically important differences in health status measures: a general approach with illustration to the Health Utilities Index Mark II. Pharmacoeconomics 1999; 15:141– 155. Sagberg LM, Jakola AS, Solheim O. Quality of life assessed with EQ-5D in patients undergoing glioma surgery: what is the responsiveness and minimal clinically important difference? Qual Life Res 2014; 23:1427– 1434. Revicki D, Hays RD, Cella D, Sloan J. Recommended methods for determining responsiveness and minimally important differences for patient-reported outcomes. J Clin Epidemiol 2008; 61:102–109. Turner D, Schu¨nemann HJ, Griffith LE, et al. The minimal detectable change cannot reliably replace the minimal important difference. J Clin Epidemiol 2010; 63:28–36. Johnston BC, Thorlund K, da Costa BR, et al. New methods can extend the use of minimal important difference units in meta-analyses of continuous outcome measures. J Clin Epidemiol 2012; 65:817–826. Le QA, Doctor JN, Zoellner LA, Feeny NC. Minimal clinically important differences for the EQ-5D and QWB-SA in Posttraumatic Stress Disorder (PTSD): results from a Doubly Randomized Preference Trial (DRPT). Health Qual Life Outcomes 2013; 11:59. Lauridsen HH, Hartvigsen J, Manniche C, et al. Responsiveness and minimal clinically important difference for pain and disability instruments in low back pain patients. BMC Musculoskelet Disord 2006; 7:82. Cook RJ, Sackett DL. The number needed to treat: a clinically useful measure of treatment effect. BMJ 1995; 310:452–454.
Eur J Anaesthesiol 2015; 32:77–78 Copyright © European Society of Anaesthesiology. Unauthorized reproduction of this article is prohibited.
