The minimum clinically important difference for EQ-5D index: a critical review.

Expert Review of Pharmacoeconomics & Outcomes Research Downloaded from informahealthcare.com by Nyu Medical Center on 02/16/15 For personal use only.

Review

The minimum clinically important difference for EQ-5D index: a critical review Expert Rev. Pharmacoecon. Outcomes Res. 14(2), 221–233 (2014)

Silvia Coretti*1, Matteo Ruggeri2 and Paul McNamee3 1 Dr. Catholic University of Sacred Hearth – Faculty of Economics, L.go F. Vito 1, Rome 00168, Italy 2 Universita` cattolica del sacro cuore, largo francesco vito 1, roma 00168, Italy 3 University of Aberdeen, Aberdeen AB24 3FX, UK *Author for correspondence: [email protected]

The European Quality of Life-5 Dimensions Questionnaire (EQ-5D) is the most common instrument to value health outcomes under the patient’s perspective. Several studies have investigated whether observed changes are meaningful to patients, using a variety of approaches to estimate the minimum clinically important difference (MCID). This study provides an overview of the state of art of the estimation of the MCID for the three-level EQ-5D index based on the UK scoring algorithm, critically assessing the available evidence. The interest in estimation of MCID for the EQ-5D has been increasing in recent years. However, some additional standardization in the estimation procedures may be of value, in order to enhance the ability to make comparisons across measures and disease areas. Further methodological research might also contribute to reducing gaps between theory and practice. KEYWORDS: EQ-5D • Euro QoL • minimum clinically important difference • minimum relevant difference • review • smallest worthwhile effect

The concept of evidence-based medicine calls for an explicit and judicious use of the best available clinical evidence in the decisionmaking process. To implement this effectively, the physician’s personal expertise should be combined with the best clinical evidence derived from a systematic research process [1]. On the other hand, the current tendency to shift from a physician-centered to a patient-centered decision-making model and thus from a parental model of patient care to a shared decisionmaking process requires a closer attention to patient-important outcomes. It means that besides clinical relevance, it is increasingly important to take into account patients’ preferences in appraising clinical outcomes [2]. Coherent with this framework, there is increasing use of health-related quality of life (HRQoL) measures in clinical trials. Such measures derive from a comprehensive concept of health and take into account patient’s preferences on health states. Moreover, generic HRQoL measures allow direct comparison between treatments across different therapeutic areas and thereby provide a useful tool for decision-makers wishing to use the results from economic evaluations to inform their policy and practice decisions. However, there informahealthcare.com

10.1586/14737167.2014.894462

is often limited evidence available on the importance of treatment effects observed from generic HRQoL measures from the perspective of patients [3,4]. The minimal important difference (MID) or minimal clinical important difference (MCID) represents the smallest amount of benefit that the patient can recognize and value. Such a definition bridges the patient perspective with evidence-based medicine [4]. Similarly, Ferreira et al. [5] define the smallest worthwhile effect (SWE) as the smallest beneficial effect justifying costs, risks and inconveniences of an intervention. Therefore, SWE can be considered as a threshold effect beyond which an intervention could be indicated. This definition differs from the one originally developed by Jaeschke et al. for the introduction of a cost–benefit trade-off [5,6]. Likewise, Barret et al. [4] use the expression ‘sufficiently important difference’ to mean the smallest effect big enough to offset costs and risks related to the intervention. The above definitions imply coexistence of three main features of the MCID [4–7]: • MCID needs to be evaluated by the potential beneficiaries of care. Thus, judgments on whether the effects of interventions are sufficient to offset its costs and inconveniences

2014 Informa UK Ltd

ISSN 1473-7167

221


Review

Coretti, Ruggeri & McNamee

should reflect patients’ and community preferences and not physicians’ or researchers’ ones [5]. It means that patients involved in the study should be asked about their judgment on the changes they experienced and such pieces of information, together with patient or general population (i.e., community) values of health states, should be utilized to compute MCID; • Being based on the trade-off between benefits and costs, discomfort and risks, it is generally intervention-specific. The monetary and opportunity costs of the intervention should be considered, as well as the inconvenience and loss of time. Similarly, the risks of side effects and adverse events should be accounted for; • For studies of interventions, MCID should represent the hypothetical difference between the outcome obtained through the intervention under study and the outcome that the same patient would have experienced undergoing the alternative regimen (i.e., the comparator). Unadjusted treatment outcomes or longitudinal changes cannot be considered as measures of SWE because they can be affected by many factors other than the intervention, namely temporal changes, regression to the mean and placebo effects; The measurement of the MCID plays a key role in sample size calculations for clinical trials as well as in interpretation of results. Ideally, clinical trials should be powered to detect the MCID; for example, a randomized controlled trial (RCT) could be powered to detect effects that are at least as large as the mean, the median or the i-th percentile MCID. According to Barret et al. the hypothesized effect sizes for randomized trials should be set at or near the median sufficiently important difference; thereby, the trial would be able to detect the degreeof-benefit sufficient for at least half of potential recipients [4]. However, the lack of a counterfactual usually impedes to carry out such an analysis at the individual level. Generally, RCTs are powered to detect effect sizes reported in the previous studies or chosen arbitrarily by the investigators [7]. Thus, ‘clinical significance’ is a broader concept than ‘statistical significance’. For instance, a large trial can detect statistically significant treatment effects that are so small to be clinically insignificant. Conversely, a small trial can fail to detect clinically significant treatment effects [4,7]. Methods to compute the MCID

The SWE is traditionally determined through either anchorbased methods or distribution-based methods. Distributional methods express the effect sizes in terms of between-patient variability and are mostly based on standard deviation statistics. So far, the minimum detectable change (MDC) and the standard error of the measurement have been the most commonly used distribution-based approaches utilized in the estimation of meaningful changes for European QoL-5 Dimensions Questionnaire (EQ-5D) [5]. However, standard error and effect sizes are often used for several types of generic and specific measures. The MDC can be computed through the limits of agreement method. Such a method is based on Bland–Altman plots 222

that allow analysis of the agreement between two different assays showing the distribution of their difference with respect to the whole range of possible scores [8]. The standard error of measurement (SEM) can be seen as a proxy of the MID, or as a tool to calculate the MDC. For example, given a 95% confidence level, the MDC is computed as 1.96 21/2 SEM, where the SEM is obtained by the formula: sx (1–rx)1/2. The term rx represents the reliability of the measure often derived from relevant literature. This method assumes that the smallest change in the health-related outcome corresponds to the component of variation that exceeds the random variation in the sample as well as the variation induced by the measurement error of the instrument [9,10]. The estimated effect size is another broadly used method. It consists of multiplying the effect size considered as small (0.2) or moderate (0.5) according to Cohen’s criteria by the standard deviation of the baseline score [11,12]. The mentioned measures are usually utilized to explore the clinimetric properties of an outcome measure, but they fail to generate good estimates of the minimum clinically relevant effect because they do not take into account the patient’s perspective. According to Revicki et al., who produced a set of guidelines for the estimation of MCID, longitudinal data are needed to assess the responsiveness of HRQoL measures [13]. Moreover, some criterion is needed to judge whether any changes that have occurred are large enough to be viewed as important. Such a criterion is called an ‘anchor’ and is applied within the anchor-based methods. One may choose clinical endpoints, global improvement scales, changes in other patient reported outcome measures or a combination of patient- and clinical-based outcomes as an anchor. An anchor needs to have a nontrivial association with change in the outcome of interest. In other words, if the correlation between the anchor and the patient-reported outcome is zero, then the anchor is not suitable. Such an association is usually investigated, in terms of strength and direction, through the Sperman’s correlation coefficient [13]. Once the anchor has been chosen, different methods can be applied to compute the MCID; the best-established ones are summarized in TABLE 1. Anchor-based methods are generally regarded as the most accurate because they are patient-centered, relying on measures of patient’s perception of the health improvement attributable to a treatment. However, although patients are usually supposed to provide information about their perceived improvement, they are rarely asked to state whether or not they consider such an improvement to be worthwhile in the light of its cost, risks and inconveniences [5,7]. The methods previously described are undoubtedly the most widely practiced. However, even when they are interventionspecific and focused on the patient’s perception, they may still fail to capture the smallest clinically relevant effect, as defined by the most recent literature [4,5,7], because they do not include any specific assessment of potential costs and harms. Essentially, they are not designed to answer the question ‘How Expert Rev. Pharmacoecon. Outcomes Res. 14(2), (2014)

The minimum clinically important difference for EQ-5D index

Review


Table 1. Anchor-based methods: a comparison. Name

Procedure

Output

Ref.

Regression methods

Ordinary least squares or generalized linear models are used to regress the change in the score obtained in the instrument under study on the change in the anchor score. Alternatively, the anchor score can be expressed as a dummy variable meaning ‘improvement’ or ‘response to treatment’. A threshold value can be elicited from patient’s preferences or stated by the analyst. Other covariates can be added

The b coefficient of the change in the anchor regressor is the MCID. Interval estimation can be performed

Average change approach

Take the average of the score change in the instrument of interest observed in the cohort of patients classified as responders according to the anchor. Bootstrap can be performed

The average value is the MCID. Interval estimation can be performed

[25]

Minimum detectable change approach

The MDC is the smallest change above the measurement error given a certain level of confidence. Take the average of the score change in the instrument of interest observed in the cohort of patients classified as non-responders according to the anchor

The upper limit of the confidence interval around the average change detected in non-responders is the MDC

[25]

Change difference approach

Take the difference between the average score change seen in the cohort of responders and the average score change seen in the cohort of non-responders

The MCID is the difference. Interval estimation can be performed

[25]

ROC curves

The ROC is a plot of the sensitivity of the instruments against 1- its specificity. In this context, sensitivity is considered as the proportion of patients who report an improvement in the anchor and a score in the instrument of interest above some pre-identified threshold value. The threshold value serves to dichotomizing the change in the instrument score if it is a continuous variable. Median splitting is usually adopted for this purpose. Instead, specificity is the proportion of patients who do not record an improvement in the anchor and show a score in the instrument of interest below the pre-identified threshold value

The MCID is the change value that provides greatest sensitivity and specificity for a positive response. The area under the curve provides the probability that a patient is correctly classified

[25]

[12,16]

MCID: Minimum clinically important difference; MDC: Minimum detectable change; ROC: Receiver operating characteristics curve.

much benefit is needed in order to justify the costs and risks of a given treatment?’ [4]. In order to overcome these limits, the benefit–harm trade-off method has recently been proposed. It is interview-based, with respondents being presented with the possible benefits and harms of an intervention. Harms include the overall costs (monetary and opportunity losses), risks and inconveniences of the intervention. Interviewees are supposed to comment on whether they would undergo the intervention or not given a certain pattern of costs and benefits. Then, holding all the other variables constant, the expected benefit is varied by a small amount in order to check whether the patient changes his or her mind about the worthiness of the intervention. This process is repeated until it is possible to elicit the smallest level of threshold benefit for which the intervention is deemed worthwhile. This technique requires the benefit to be expressed in terms of incremental benefit with respect to a relevant comparator. There are only a few examples of this approach in informahealthcare.com

terms of trial design and analysis [4,5,7], and it appears that no studies have applied this approach to HRQoL measures. Aim of the study

The current review aims to provide an overview of the state of the art of the estimation of the MCID in relation to the threelevel version of EQ-5D index based on the UK scoring algorithm, and to critically appraise the relevant empirical literature in order to assess the quality of the evidence base. This study focuses on estimates of MCID to be used in the interpretation of group-level comparisons, that is to understand whether the difference in HRQoL as assessed by EQ-5D between the intervention group and the control group is quantitatively important. Methods Data sources & research strategy

The literature search was performed by consulting PubMed, Cinahl and Embase and screening the references of the papers 223


Review


included in the review. The literature search was conducted by identifying a set of potentially useful key words to use on each database. In particular, in the search strategy, the minimum clinically important difference was referred to as: ‘smallest worthwhile effect’, ‘minimal/minimum clinically significant change’, ‘minimal/minimum clinically significant difference’, ‘minimum/minimal clinically relevant change’ ‘minimum/ minimal relevant effect’, ‘smallest detectable change’, ‘minimum/ minimal detectable change’ and ‘minimum/minimal important difference’. Boolean operators were used to link the key words. The full list of key words utilized to identify papers in each database is reported in the APPENDIX (supplementary material can be found online at www.informahealthcare.com/suppl/10.1586/ 14737167.2014.894462). In each of the mentioned searches, the selected papers were supposed to fulfill the following requirements: abstract available; written in English language; published in the last 10 years; and key words reported either in the title or in the abstract. Inclusion/exclusion criteria

The eligibility of the studies was ascertained by two steps: a preliminary screening was done on the basis of title and abstract. Afterward, the studies included in this prior assessment were selected on the basis of the full text at the second screening. Only primary reports were selected in the first phase; hence literature reviews and model-based economic evaluations were discarded, though some were retained as potentially useful sources of background information. Moreover, methodological papers, position papers and case studies were excluded. Randomized controlled trials and observational studies could also be discarded if they did not contain in the title or in the abstract information relevant to the aim of our study. In order to be considered eligible, the studies overcoming the first screening needed to report some estimate of the minimum clinically important effect when considering the threelevel version of EQ-5D index based on the UK scoring algorithm as an outcome measure. The procedure that led to such an estimate needed to be explained in detail. When several scoring algorithms were utilized in a paper, only estimates referring to the UK one were taken into account. Data extraction

Standardized data extraction forms were used. Each record yielded by the search strategy was attached an ID code in order to be easily tracked. Such a code was composed of a letter, namely P, or C or E, indicating the database and a three-digit number corresponding to its position in the list. For each paper, regardless of its subsequent inclusion in the review, first author, title, source and study design were recorded. For included studies, several pieces of information were recorded in order to investigate to which extent their estimate of the MCID was consistent with the definition provided by existing literature [4–7]. Since the analysis was not restricted to a single clinical condition, data extraction form contained: 224

• Information about the clinical condition of interest, the number of patients involved in the study, the interventions assessed. In particular, the number of patients actually included in the analysis (withdrawals excluded) was reported; • Information about the definition of MCID adopted and the method used to compute it. This set of information includes the indication of whether the estimate is intervention or disease-specific, if the analysis is longitudinal or cross-sectional, and whether patient preferences for the health states were elicited; • Finally, the estimate of the MCID for EQ-5D was reported. When several methods were used, several different values or intervals were considered; The selected studies were grouped according to the therapeutic area of interest. Results Search strategy results

The research strategy yielded 796 results: 480 from PubMed, 211 from Cinahl and 105 from Embase. A preliminary screening based on the title and abstract identified 173 potentially relevant works. The remaining 623 were excluded: 81 were duplicates, 408 were considered not relevant to the aim of the study because they did not involve the estimation of the MCID or they did not use the UK version of the three-level EQ-5D index as an outcome measure. Moreover, 15 publications were discarded for being based on a research protocol and not having any available results yet. Finally, 118 studies were excluded because they were not primary reports. The potentially relevant works were reviewed more accurately at the second screening, carried out by reading the full text: 107 studies were discarded because the MCID had not been investigated at all; 24 studies were excluded because the MCID was estimated for outcome measures other than EQ-5D (UK version). In such cases, the EQ-5D index was usually adopted as a secondary outcome measure. Finally, 24 studies were excluded because they did not provide an estimation of the MCID, but they use the MCID computed within other studies as a benchmark to compare their clinical results and to assess their clinical relevance. The references of the latter papers were screened in order to identify the primary source of the MCID value reported. Most of them refer to some of the studies included in the current review [10,11,14], the remaining part referred to a study by Dolan et al. [15]. Eighteen works ultimately met the inclusion criteria. The selection process is depicted in FIGURE 1. General description of the selected studies

Eighteen studies were eventually included in the review. The selected studies were very heterogeneous in terms of design. The sample was composed of four (22%) cross-sectional studies [8,9,11,16], one (6%) non-randomized controlled trial [17], six (33%) prospective cohort studies [18–23], two (11%) randomized controlled trials [12,24] and four (22%) retrospective cohort studies [10,25–27]. In addition to these papers, a further study by Expert Rev. Pharmacoecon. Outcomes Res. 14(2), (2014)



Review

Walters et al. of 2005 was included: here PubMed: Cinahl: Embase: MCID was estimated on a sample con480 results 211 results 105 results structed by pooling data collected on 2283 patients affected by different health conditions and assessed in eight different 796 abstracts longitudinal studies [14]. Thus, even Abstract-based screening though it is not properly a primary report, it was included in the analysis for two reasons: first estimates have been 623 excluded: computed ex novo based on previously – Duplicates (82) published data; second, the estimates pro– Not relevant (408) 173 potentially relevant vided by the authors are often used as a – Results not available (15) – Study design not suitable (118) threshold to establish the clinical signifiFull text-based screening cance of outcomes in subsequent studies. More than half of the selected studies (56%) were published after 2010. Further, the majority of studies have been conducted in the field of musculoskeletal dis155 excluded: 18 eligible studies – MCID not estimated (24) orders: 12 out of 18 selected studies refer – MICD not investigated for EQ- 5D (24) to this therapeutic area [8,9,11,17–22,25–27]. – MCID not investigated not at all (107) Two studies focused on oncology patients. In particular, the paper by Kvam et al. [23] Figure 1. Study selection process. investigated the minimum important difEQ-5D: European Quality of Life-5 Dimensions Questionnaire; MCID: Minimum clinically ference for EQ-5D in patients with multiimportant difference. ple myeloma, while in the study by Pickard et al. [10] an estimate was based on a large cohort of patients with advanced cancer (stage 3 or 4), are patient-centered because they seek to associate a certain who underwent at least two cycles of chemotherapy, regardless of score in the measure of interest with some scale of well-being the cancer site. Both these works recur frequently in the referen- or severity score, this association does not directly reflect ces of other studies as a benchmark to measure the clinical signifi- patient valuation of that specific health state or intervention. cance of HRQoL outcomes. The paper by Walters et al. [14] is On the contrary, it is usually established by the researcher on not disease-specific. The remaining three papers focus on hetero- the basis of relevant literature or as a result of quantitative analgeneous conditions, namely post-traumatic stress disorder [12], ysis. In the current review, only six studies provide estimates based on patient’s judgment [16,20,22,25–27]. In such studies, bowel inflammatory disease [16] and psoriasis [24]. The MCID is referred to in different ways: clinically relevant patient’s valuation is obtained either through explicit questions difference, minimum important difference, minimum clinically about patient’s satisfaction with their global health improveimportant difference (most commonly used), cut-off value of ment or through questionnaires aimed at investigating patient’s success, minimum clinical important change, MDC, meaning- perception of the improvement for specific symptoms. Howful differences and smallest detectable difference. However, all ever, four of these studies [22,25,26] also exploit other scales in the procedures adopted to compute the SWE are very similar which the threshold for improvement is defined by and can be classified as either anchor-based methods or the researcher. Second, it has been recommended that the estimate needs to distribution-based methods. Half of the studies use anchorbased methods to investigate MCID; only four studies use be intervention-specific, that is, made with reference to the distribution-based methods; finally, five studies combine risks and the costs associated with the intervention. Only five anchor-based with distribution-based methods, with the aim of of the selected studies provide intervention-specific estimates. Much more common is the identification of the SWE for providing more accurate estimates. patients affected by a specific health problem (11 studies). In one of the studies [14], the authors seek to estimate an Quality of the evidence base Turning now to the issue of the quality of the evidence base, instrument-specific MCID, applicable across several conditions we assess the studies based on recommendations emerging from and interventions. Moreover, even when intervention-specific estimates are computed, no explicit reference is done to the existing literature [5–7]. There are three key criteria. First, it is considered best practice for the estimation to be costs or to the risks associated with the intervention. Finally, it is argued that the estimate of the SWE should be based on patient’s judgment. This means that the patient must be involved in the identification of the size of the effect that based on between-group differences, since this has a clearer can be considered as meaningful. Even if anchor-based methods focus on comparative effectiveness between the proposed informahealthcare.com

225

Review



intervention and an alternative comparator intervention. Among the selected studies, only two are intervention-specific and explicitly compute the MCID in terms of difference between two groups of patients undergoing alternative treatment. One of these studies is a non-randomized controlled trial [17] and the other one is an RCT [24]. Estimates of the minimal clinically relevant effect

Considering now the estimation methods utilized and the results obtained, we focus on the two usual techniques, distribution-based methods and anchor-based methods: Distribution-based methods

Nine studies used distribution-based methods. Overall, five different distributional methods were identified. The study by Larsen et al. [17] did not provide much detail about the methods used to estimate the MCID, probably because it was not the main topic of the study. The QoL of two groups of patients was assessed at baseline and after 3 months follow-up. A multivariate analysis was performed by regressing the change in QoL on several variables namely treatment group, gender, age, diagnosis and implant type. The MCID was estimated as the impact of the treatment group on the difference in QoL (b coefficient of the regression). Overall, five studies used effect sizes based on Cohen’s criteria to assess the MID. In particular, in two of them the baseline standard error of the measurement was multiplied by 0.2, meaning small effect [11,23]; in one study [10] a coefficient of 0.5 was chosen, meaning moderate effect. Finally, two studies [12,14] report estimates obtained using both the coefficients in order to provide a range of values. One study is based on the limits of agreement method [8]. In three studies [9,10,24], the SEM was applied to calculate the MDC. In addition, Walters et al. [14] used also the effect size and the standardized response mean computed as the ratio between the raw score change from the first to the second assessment for each patient and its standard deviation. Anchor based methods

Fourteen studies investigated the SWE using anchor-based methods. Amongst these, four studies also used distributionbased approaches as well. Moreover, even when only anchorbased methods were adopted, several anchors were used in the attempt of obtaining more accurate estimates. Only 5 out of 14 studies utilize a unique anchor to estimate MCID. Both generic and disease-specific outcome measures have been used as anchors in the selected studies. Eight studies combine both types of measure in an attempt to provide more accurate estimates or a range of values. In five studies, MCID has been computed using only disease-specific anchors. Only one study provided an estimate based on a single generic anchor. General disability scales (e.g., Oswestry Disability Index, Global Rating of Change, Clinical Global Impression [CGI]Severity Scale, CGI-Index [CGI-I], Postraumatic Stress Disorder 226

Symptom Scale-Interview) and disease-specific scales assessed by physicians (e.g., Eastern Cooperative Oncology Group, Functional Assessment of Cancer Therapy-General, Psoriasis Area and Severity Index) were often used to build anchors [10,14,22–24]. Sometimes, these tools were combined with questionnaires aimed at assessing patients’ satisfaction [10,14,22,23]. The health transition index of the SF- 36 health survey is the most commonly used anchor [25–27]. It requires patients to rate their current health compared with their health in the past through a four-item Likert scale ranging from ‘worse’ to ‘markedly better’. Patients answering ‘slightly better’ or ‘markedly better’ are classified as responders, whereas those answering ‘unchanged’ or ‘worse’ are classified as non-responders. In the same studies, a patient-centered anchor was also derived by asking patients whether they were satisfied with the results of their surgery. Patients answering ‘yes’ were classified as responders, whereas those answering ‘no’ were classified as non-responders. Anchors adopted in the selected studies are listed in TABLE 2. In the study by Le et al. [12], the methods used to test the goodness of the anchors are fully described. Patients were classified as responders or non-responders according to the score obtained in CGI-I and Postraumatic Stress Disorder Symptom Scale-Interview after receiving the treatment. Correlation coefficients between changes in EQ-5D scores and changes in anchor measures were computed. The methodological guidelines proposed by Revicki et al. suggest that such coefficients need to be ‡0.3 in order to be considered a good anchor [13]. The estimation of MCID was performed through a linear regression model (ordinary least sqaures) in which changes in the EQ-5D were regressed on the transformed CGI-I. The MCID corresponds to the b coefficients of the regression. Stark et al. [16] take a similar approach and use econometric methods to estimate the MCID. The difference in EQ-5D score was the dependent variable, and the answers to the transition question whether health status was better or worse were entered as dummy variables. Their coefficients estimate the incremental difference in scores when health status shifts to better or worse (according to the patient’s view) compared to the stable health status. Several studies [12,18–21,25–27] identify MCID by means of a receiver operating characteristics curve. This method has been widely employed in the assessment of responsiveness of outcome measures. However, a complicating factor is that although the area under the curve is always regarded as a measure of the instrument responsiveness, different decision rules are adopted to identify the MCID. Solberg et al. identify the MCID as the upper corner of the curve, which is assumed to be the best cut-off score distinguishing between success and not [18]. Soer et al. define the MCID as the point in which the sum of sensitivity plus 1-specificity is maximized [19]. Conversely, five studies [18,20,25–27] compute the MCID as the point of the receiver operating characteristics curve in which the sum of sensitivity and specificity is maximized. In the study by Impellizzeri et al. [21], the 80% specificity method has been Expert Rev. Pharmacoecon. Outcomes Res. 14(2), (2014)


Review


Table 2. Number and kind of anchor used. Study (year)

Anchors (n)

Anchors

Solberg et al. (2013)

1

Global Perceived Scale Of Change

Soer et al. (2012)

3

Roland Morris Disability Questionnaire Numeric rating scale

Parker et al. (2012)

1


2

Impellizzeri et al. (2012)

Generic

Ref.

Disease-specific [18]

Pain Disability Index

[19]

North America Spine Society Patient Satisfaction Scale

[20]

Health transition index of SF-36

Patient’s satisfaction after the surgery

[27]

2

Symptoms specific well-being

Global Treatment Outcome

[21]


2


Patient’s satisfaction after the treatment

[26]


2


Patient’s satisfaction after the treatment

[25]

McDonough et al. (2011)

4

Symptom satisfaction Self-perceived health

Oswestry Disability Index Progress rating

[22]

Patient’s perceived improvement

[23]

Eastern Cooperative Oncology Group performance status ratings Functional Assessment of Cancer Therapy-General score

[10]

Treatment response status

[12]

Kvam et al. (2011) Pickard et al. (2007)

2

Le et al. (2013)

3

Stark et al. (2010)

1

Patient’s perceived improvement after the treatment

[16]

Shiriak et al. (2010)

2

Psoriasis Area and Severity Index Physician’s Global Assessment

[24]

Walters et al. (2005)

1

Global Rating of Change

[14]

Clinical Global Impression-Severity Clinical Global Impression-Index

adopted; the MCID corresponds to the point of the curve that exhibits the best sensitivity for a response while still achieving at last 80% specificity. Average change, MDC and change difference approaches are also used in the identified studies. The average change approach has been used in seven studies [20,22–27]. The MDC approach has been applied in four studies [20,25–27]. Finally, the change difference has been used in five studies [14,20,25–27]. Four studies [20,25–27] have exactly the same structure and apply the same procedures to estimate MCID. As the existing literature offers no clear guidance regarding which of the possible methods is the most accurate, most of the studies combine different techniques and apply triangulation to provide a range of values rather than a single estimate. In particular, average change, MDC and change difference approaches are usually associated. Comparison of methods & results

The figures provided in the selected studies are very variable. Indeed, no disease-specific inclusion criteria were adopted; thus part of this variability is likely to be due to the heterogeneity of the study populations. However, even within the same clinical area, large variability in the results is observed. This suggests that informahealthcare.com

the variation in method may be a factor that gives rise to differences in values. Further research using meta-analysis techniques, controlling for both patient and study heterogeneity, could help to understand the extent to which this variability can be attributed to the utilization of different methods. No relationship has been observed between the sample size and the MCID. Overall, the estimates of MCID range from 0.03, estimated by Soer et al. [19] for patients with low back pain, to 0.52 estimated by Parker et al. [26] for patients with recurrent lumbar stenosis. Both these extreme values fall in the clinical area of musculoskeletal disorders. Only two studies dealing with oncologic conditions have been included [10,23]. Both of them assessed the SWE through several methods. Their results are quite close, ranging from 0.07 to 0.12. The lack of studies clearly impedes any sensible comparison across different diagnoses. The available evidence is also too limited to draw conclusions about any trend of the estimates delivered by each method (e.g., whether distribution-based methods yield systematically smaller estimates). Neither it is possible to infer which tool is the most accurate among those belonging to the same category (e.g., whether disease-specific scales are more suitable anchors than general well-being scales). 227

228

Study design

Cross-sectional study

Prospective cohort study



Retrospective cohort study


Marra et al. (2005)

Solberg et al. (2013)

Soer et al. (2012)



Impellizzeri et al. (2012)

99 consecutive patients with femoroacetabular impingement who underwent surgery

47 patients undergoing revision fusion for pseudoarthrosisassociated back pain

69 patients with cervical radiculopathy who underwent anterior cervical discectomy and fusion

151 patients with low back pain undergoing minimal intervention, rehabilitation, anesthesiology, combination of treatments

692 patients undergoing lumbar disc herniation surgery

313 patients with diagnosis of rheumatoid arthritis, receiving any care

98 elective primary hip arthroplasty patients undergoing either a standard procedure or an accelerated perioperative procedure

Population and intervention(s)

A (ROC)

A (ROC)

A (AC, MDC, CD, ROC)


A (ROC)

MCIC

MCID

MCID

MCIC

D (SD)

MID

CVS

D (REG)

Method of elicitation

CRD

Terminology

Longitudinal

Longitudinal

Longitudinal

Longitudinal

Longitudinal

Cross-section

Cross-section

Cross-section/ longitudinal

DIS

INT

INT

DIS

INT/DIS

DIS

INT

Intervention/ disease specific

RES

PTS/RES

PTS

RES

RES

RES

RES

Whose preferences?

0.16, 95% CI: (0.686-0.857)

0.14–0.24

0.24

0.03

0.3

0.05

0.08, 95% CI: (0.01–0.15)

Results

[21]

[27]

[20]

[19]

[18]

[11]

[17]

Ref.

A: Anchor-based method; AC: Average change; CD: Change difference; CRD: Clinically relevant difference; CVS: Cut-off values of success; D: Distribution-based method; DIS: Disease-specific estimate; ES: Effect size; INT: Intervention-specific estimate; LA: Limits of agreement; MCIC: Minimum clinical important change; MCID: Minimum clinically important difference; MD: Meaningful differences; MDC: Minimum detectable change; MDS: Mean difference score; MID: Minimum important difference; N/A: Not applicable; PTS: Patient; REG: Regression analysis; RES: Researcher; ROC: Receiver operating characteristics curve; SD: 0.2 or 0.5 standard deviation; SDD: Smallest detectable difference; SEM: Standard error of measurement; SRM: Standardized response mean.

Non-randomized trial

Larsen et al. (2008)

Musculoskeletal disorders

Study (year)

Table 3. Characteristics of measures of the smallest worthwhile effect.


Review Coretti, Ruggeri & McNamee

Expert Rev. Pharmacoecon. Outcomes Res. 14(2), (2014)

Study design

informahealthcare.com




McDonough et al. (2011)

Boonen et al. (2007)

Staerkle et al. (2011)

239 patients with multiple myeloma

51 patients with inguinal hernia undergoing surgery

120 patients with ankylosing spondylitis who underwent either 3-week spa treatment (n = 80) with usual care (n = 40)

1000 persons with intervertebral disc herniation

45 patients undergoing transforaminal lumbar interbody fusion for low-grade degenerative lumbar spondylolisthesisassociated back and leg pain

53 patients undergoing revision surgery for same-level recurrent lumbar stenosis–associated back and leg pain


MD

A (AC)/D(SD)

D (SEM)

D (LA)

SDD

MDC

A (AC)


MCID

MID



MCID

Terminology

Longitudinal

Longitudinal

Cross-section

Longitudinal

Longitudinal

Longitudinal


DIS

INT

DIS

DIS

INT

INT


RES

RES

RES

RES/PTS

PTS

PTS/RES

Whose preferences?

0.08–0.10

0.36

0.36

ODI = 0.12 (0.09,0.16) Satisfaction = 0.25 (0.20, 0.30); Progress = 0.15 (0.10, 0.20); Self-perceived health = 0.14 (0.09, 0.19)

0.15–0.54

0.29–0.52

Results

[23]

[9]

[8]

[22]

[25]

[26]

Ref.


Kvam et al. (2011)




Oncology



Musculoskeletal disorders (cont.)

Study (year)

Table 3. Characteristics of measures of the smallest worthwhile effect (cont.).



Review

229

230

Study design



Randomized controlled trial

Stark et al. (2010)

Shikiar et al. (2006)

Longitudinal study

2283 patients from 8 longitudinal studies dealing with different diseases

147 patients with moderate-to-severe plaque psoriasis receiving either adalimumab-based pharmaceutical treatment or placebo

502 patients with bowel inflammatory disease

200 patients with primary posttraumatic stress disorder undergoing either cognitive behavioral therapy or sertraline-based treatment

534 patients with advanced cancer of the bladder, brain, breast, colon/rectum, head/neck, liver/ pancreas, kidney, lung, lymphoma, ovary, and prostate who underwent at least two cycles of chemotherapy


A (CD)/D (ES, SRM, SD)

A (AC)/D (SEM)

MID/MCID

MID

A (REG)

A (REG, ROC)/D (SD)

A (MDS)/D (SEM, SD)


MD

MCID

MID

Terminology

Longitudinal

Cross-section

Longitudinal

Longitudinal

Cross-section


N/A

INT

DIS

DIS

DIS


RES

RES

PTS

RES

RES

Whose preferences?

0.074 (-0.011 to 0.140)

MID-1: 0.10 (0.24) MID-2: 0.20 (0.21) MID-3: 0.09 SEM: 0.22 0.5 SD: 0.14

Improved health: 0.076 Deteriorated health: 0.109

A: 0.05–0.08 D: 0.04–0.10

D: 0.10–0.12 both for all subgroups; A: 0.09–0.10 for all cancers, and 0.07– 0.08 for lung cancer

Results

[14]

[24]

[16]

[12]

[10]

Ref.


Walters & Brazier (2005)

Not disease/intervention-specific

Randomized controlled trial

Le et al. (2013)

Other clinical areas

Pickard et al. (2007)

Oncology (cont.)

Study (year)

Table 3. Characteristics of measures of the smallest worthwhile effect (cont.).


Review Coretti, Ruggeri & McNamee



The features of the selected studies relevant to the current analysis are reported in TABLE 3.


Discussion

General measures of QoL are increasingly used in clinical trials to enhance the delivery of patient-centered care. Suitable estimates of MCID for such measures are important to inform trial design and interpretation of results. The purpose of the current study was to assess the state of art of the investigation of MCID for the three-level EQ-5D index, based on the UK scoring algorithm. A structured literature review was conducted. Overall, 796 abstracts were screened and 18 studies were eventually reviewed. The review pointed out that interest around MCID of EQ-5D has been increasing in the last few years and that the largest work has been done in the field of musculoskeletal disorders. Anchor-based methods are often preferred in the estimation of MCID as they are focused on the patient’s perspective. So far, no studies have investigated the MCID for EQ-5D through the benefit–harm trade-off method, although it offers the most direct method of accounting for both benefits and harms from the patient perspective. The literature review provides an insight into the remarkable heterogeneity in the methods adopted to estimate the MCID as well as into its definition. The estimates range from 0.03 to 0.52. Furthermore, the reading of some of the excluded studies revealed that many authors use the estimates of MCID from other works to assess the clinical significance of their findings. This comparison represents an example of how MCID can be used in the interpretation of results. Estimates provided in three of the included studies are usually considered as a benchmark by other authors [10,14,23]. Our literature review exhibits some limitations that deserve mention. First, the review only focused on papers published in peer-reviewed journals. This ensures a certain standard of quality of the included papers but, occasionally, the inclusion of grey literature can be a useful source of relevant information. Likewise, the choice of including only English language papers might have caused the loss of some relevant studies. Second, the process of selection of relevant works and the data extrapolation and synthesis has been performed by a single reviewer. Third, in this review, we only focused on anchor-based and distribution-based methods as listed and explained by existing literature. However, MCID can also be computed through alternative methods. For instance, Luo et al. used instrumentdefined health state transitions to estimate MCID, assuming that changes in preference scores associated with the smallest health transitions are minimally important. The authors estimated a MID of 0.082 for the UK algorithm and of 0.040 for the US algorithm [28]. In terms of being able to recommend which approach to use for MCID estimation for the three-level EQ-5D index, although anchor-based approaches are considered superior, through incorporation of patient and community preferences, it is interesting informahealthcare.com

Review

to note that, in the disease-specific case, estimates of MCID derived from the two approaches are often similar [29]. Many of the studies included in our review combine anchor-based and distribution-based methods and obtain a range of values through triangulation, coherent with the guidelines produced by Revicki et al. [13]. This triangulation may enhance the confidence of some analysts in the use of those estimates. In the context of disease-specific instruments that use a seven-point Likert-type scale, Juniper et al. found that a change of 0.5 per item often corresponds to the MCID, irrespective of estimation method. It is a moot point, however, as to whether the same holds for a generic instrument such as the EQ-5D, where unequal weighting across items is to be expected [30]. Expert commentary

Interest in the study of the minimum clinically important difference for EQ-5D has been increasing in recent years. Such a measure is in widespread use in assessing cost-effectiveness. The EQ-5D index is based on a combination of patients’ judgments over their own health status and wider societal values regarding which health states are better than others. It allows comparison across treatments that address different pathologic conditions. The widespread use of quality-adjusted life years in the assessment of the effectiveness and cost-effectiveness of new health technologies calls for a reliable estimation of the minimum clinically important difference for the EQ-5D index, which would be useful to adequately design clinical trials and to critically appraise study results. So far, few studies set their optimal sample size based on an empirically derived estimate of the MCID for the particular health condition in question. Conversely, many commentators compare the treatment effects resulting from clinical studies with some estimate of the MCID in order to establish whether the incremental effectiveness of the treatment under study is clinically relevant. However, there appears to be some disconnect between estimates currently used as a benchmark and the three requirements for MCID highlighted by existing literature [5–7] (i.e., centered on patient’s preferences, based on between-groups comparison and accounting for benefit and harms of the treatment of interest). In particular, costs and potential harms are never fully accounted for by the estimates currently available, probably because their computation would require additional data compared to that usually collected during a clinical trial. Indeed, the benefit– harm trade-off is the only method able to capture the full patient’s evaluations including their judgment on what constitutes a fair trade-off between benefit and costs, in their widest meaning. The application of the benefit–harm trade-off method is possible however through the implementation of discrete choice experiments. There are no examples that we are aware of that have conducted these methods in the context of sample size determination. Moreover, our review shows that patients’ preferences are often not fully accounted for in the computation of MCID, since thresholds for treatment success are often defined by clinicians or arbitrarily by the analyst. This is surprising as one 231


Review


would expect treatment success—measured by the EQ-5D – to vary across disease conditions and disease severity, as well as by individual factors specific to the patients that are unobservable to the clinician and to the analyst. One potential simple solution to this problem is to adopt an additional patient satisfaction scale or global health improvement scale (e.g., a Likert scale) in clinical studies and exploit traditional methods for computation. Five-year view

Our review highlighted a remarkable gap between theory and practice in the estimation of the minimum clinically important difference for EQ-5D index as well as a substantial lack of standardization in methodology. However, the interest around this topic has been growing in the last few years. In the future, it would be beneficial to perform further methodological research in order to establish some standardized procedure for computation. In effect, many methods for computation are available, but what is really lacking is evidence stating which of the existing method performs best in fulfilling the requirements of MCID already strongly supported by theoretical literature. Improved availability of widely acknowledged guidelines for estimation would enhance standardization and avoid duplication of effort.

Moreover, much work has to be done so far in the field of musculoskeletal diseases, but little evidence is currently available for other clinical areas characterized by high investment in clinical research such as oncology, cardiovascular diseases and infectious diseases. In those clinical areas, research usually results in the commercialization of drugs with high costs and small incremental innovation. It would be advantageous to have a deeper investigation of the MCID for EQ-5D in these areas in order to better appreciate the results of regulatory trials. Disclaimer

The views expressed here are those of the authors and not necessarily those of the funders. Financial & competing interests disclosure

The Health Economics Research Unit is funded in part by the Chief Scientist Office of the Scottish Government Health and Social Care Directorates. The authors have other no relevant affiliations or financial involvement with any organization or entity with a financial interest in or financial conflict with the subject matter or materials discussed in the manuscript. No writing assistance was utilized in the production of this manuscript.

Key issues • Health-related quality of life measures and especially European Quality of Life-5 Dimensions Questionnaire index are increasingly used in clinical studies. • The minimum clinically important difference (MCID) is the minimum difference that the patient is able to recognize and appreciate. • MCID can be used to define the optimal sample size in a clinical as well as to interpret results. • Little empirical work has been done to estimate MCID for European Quality of Life-5 Dimensions Questionnaire index despite the widespread use of this measure. • Many computational methods exist for MCID, leading to different results. • Future research should focus on both methodological and empirical aspects in order to enhance knowledge and understanding, leading to potentially greater standardization.

the framework of clinical significance. Med Decis Making 2005;25(3):250-61

References 1.

2.

3.

4.

Sackett DL, Rosenberg WM, Gray JA, et al. Evidence based medicine: what it is and what it isn’t. BMJ 1996;312(7023): 71-2 Guyatt G, Montori V, Devereaux PJ, et al. Patients at the center: in our practice, and in our use of language. ACP J Club 2004; 140(1):A11-12 Dolan P. The measurement of health-related quality of life for use in resource allocation decisions in health care. In: Culyer AJ, editor. Handbook of health economics. 1st edition. Elsevier, Holland; 2000. p. 1723-60 Barrett B, Brown D, Mundt M, Brown R. Sufficiently important difference: expanding

232

5.

6.

7.

Ferreira ML, Herbert RD, Ferreira PH, et al. A critical review of methods used to determine the smallest worthwhile effect of interventions for low back pain. J Clin Epidemiol 2012;65(3):253-61 Jaeschke R, Singer J, Guyatt GH. Measurement of health status. Ascertaining the minimal clinically important difference. Control Clin Trials 1989;10(4):407-15 Ferreira ML, Ferreira PH, Herbert RD, Latimer J. People with low back pain typically need to feel ’much better’ to consider intervention worthwhile: an observational study. Aust J Physiother 2009;55(2):123-7

8.

Boonen A, van der Heijde D, Landewe´ R, et al. How do the EQ-5D, SF-6D and the well-being rating scale compare in patients with ankylosing spondylitis? Ann Rheum Dis 2007;66(6):771-7

9.

Staerkle RF VP. Simple questionnaire for assessing core outcomes in inguinal hernia repair. Br J Surg 2011;98(1):148-55

10.

Pickard AS, Neary MP, Cella D. Estimation of minimally important differences in EQ-5D utility and VAS scores in cancer. Health Qual Life Outcomes 2007;21(5): 70-8

11.

Marra CA, Woolcott JC, Kopec JA, et al. A comparison of generic, indirect utility measures (the HUI2, HUI3, SF-6D, and the EQ-5D) and disease-specific instruments



(the RAQoL and the HAQ) in rheumatoid arthritis. Soc Sci Med 2005;60(7):1571-82


12.

13.

14.

Le QA, Doctor JN, Zoellner LA, Feeny NC. Minimal clinically important differences for the EQ-5D and QWB-SA in Post-traumatic Stress Disorder (PTSD): results from a Doubly Randomized Preference Trial (DRPT). Health Qual Life Outcomes 2013; 12(11):59-68 Revicki D, Hays RD, Cella D, Sloan J. Recommended methods for determining responsiveness and minimally important differences for patient-reported outcomes. J Clin Epidemiol 2008;61(2):102-9 Walters SJ, Brazier JE. Comparison of the minimally important difference for two health state utility measures: EQ-5D and SF-6D. Qual Life Res 2005;14(6):1523-32

15.

Dolan P. Modelling valuations for EuroQol health states. Med Care 1997;35(11): 1095-108

16.

Stark RG, Reitmeir P, Leidl R, Ko¨nig HH. Validity, reliability, and responsiveness of the EQ-5D in inflammatory bowel disease in Germany. Inflamm Bowel Dis 2010; 16(1):42-51

17.

18.

Larsen K, Hansen TB, Søballe K. Hiparthroplasty patients benefit from accelerated perioperative care and rehabilitation: a quasi-experimental study of 98 patients. Acta Orthop 2008;79(5): 624-30 Solberg T, Johnsen LG, Nygaard ØP, Grotle M. Can we define success criteria for lumbar disc surgery? : estimates for a substantial amount of improvement in core outcome measures. Acta Orthop 2013; 84(2):196-201

informahealthcare.com

19.

Soer R, Reneman MF, Speijer BL, et al. Clinimetric properties of the EuroQol-5D in patients with chronic low back pain. Spine J 2012;12(11):1035-9

20.

Parker SL, Godil SS, Shau DN, et al. Assessment of the minimum clinically important difference in pain, disability, and quality of life after anterior cervical discectomy and fusion: clinical article. J Neurosurg Spine 2013;18(2):154-60

Review

25.

Parker SL, Adogwa O, Paul AR, et al. Utility of minimum clinically important difference in assessing pain, disability, and health state after transforaminal lumbar interbody fusion for degenerative lumbar spondylolisthesis. J Neurosurg Spine 2011; 14(5):598-604

26.

Parker SL, Mendenhall SK, Shau DN, et al. Minimum clinically important difference in pain, disability, and quality of life after neural decompression and fusion for same-level recurrent lumbar stenosis: understanding clinical versus statistical significance. J Neurosurg Spine 2012;16(5): 471-8

21.

Impellizzeri FM, Mannion AF, Naal FD, et al. The early outcome of surgical treatment for femoroacetabular impingement: success depends on how you measure it. Osteoarthritis Cartilage 2012; 20(7):638-45

27.

22.

McDonough CM, Tosteson TD, Tosteson AN, et al. A longitudinal comparison of 5 preference-weighted health state classification systems in persons with intervertebral disk herniation. Med Decis Making 2011;31(2):270-80

Parker SL, Adogwa O, Mendenhall SK, et al. Determination of minimum clinically important difference (MCID) in pain, disability, and quality of life after revision fusion for symptomatic pseudo arthrosis. Spine J 2012;12(12):1122-8

28.

Luo N, Johnson J, Coons SJ. Using instrument-defined health state transitions to estimate minimally important differences for four preference-based health-related quality of life instruments. Med Care 2010; 48(4):365-71

29.

Wyrwich KW, Nienaber NA, Tierney WM, Wolinsky FD. Linking clinical relevance and statistical significance in evaluating intra-individual changes in health-related quality of life. Med Care 1999;37(5):469-78

30.

Juniper EF, Guyatt GH, Willan A, Griffith LE. Determining a minimal important change in a disease-specific Quality of Life Questionnaire. J Clin Epidemiol 1994;47(1):81-7

23.

24.

Kvam AK, Fayers PM, Wisloff F. Responsiveness and minimal important score differences in quality-of-life questionnaires: a comparison of the EORTC QLQ-C30 cancer-specific questionnaire to the generic utility questionnaires EQ-5D and 15D in patients with multiple myeloma. Eur J Haematol 2011;87(4):330-7 Shikiar R, Willian MK, Okun MM, et al. The validity and responsiveness of three quality of life measures in the assessment of psoriasis patients: results of a phase II study. Health Qual Life Outcomes 2006;27(4): 71-82

233

Minimum clinically important difference in medical studies.

Brief pain inventory--facial minimum clinically important difference.

Minimum clinically important difference for the COPD Assessment Test: a prospective analysis.

Estimation and application of the minimum clinically important difference in COPD.

THE MINIMUM CLINICALLY IMPORTANT DIFFERENCE ON THE VISA-A AND LEFS FOR PATIENTS WITH INSERTIONAL ACHILLES TENDINOPATHY.

Pain relief that matters to patients: systematic review of empirical studies assessing the minimum clinically important difference in acute pain.

Physiology-based minimum clinically important difference thresholds in adult laryngotracheal stenosis.

Erratum to: The Minimum Clinically Important Difference of the Patient-rated Wrist Evaluation Score for Patients With Distal Radius Fractures.

The Minimum Clinically Important Difference of the Patient-rated Wrist Evaluation Score for Patients With Distal Radius Fractures.

Estimating a minimal clinically important difference for the EuroQol 5-Dimension health status index in persons with multiple sclerosis.

Usefulness of minimum clinically important difference for assessing patients with subaxial degenerative cervical spine disease: statistical versus substantial clinical benefit.

Determining the minimal clinically important difference and responsiveness of the Dermatology Life Quality Index (DLQI): further data.

Minimal clinically important difference: defining what really matters to patients.

Values of the Minimal Clinically Important Difference for the Neuropsychiatric Inventory Questionnaire in Individuals with Dementia.

Health status instruments for patients with COPD in pulmonary rehabilitation: defining a minimal clinically important difference.

Validity of minimal clinically important difference values for the multiple sclerosis walking scale-12?

Minimum clinically important difference in diffusing capacity of the lungs for carbon monoxide among patients with severe and very severe chronic obstructive pulmonary disease.

The minimum clinically important improvement in the incremental shuttle walk test following cardiac rehabilitation.

Choosing a "yardstick" to determine the minimal clinically important difference: reply to Johnson and Currow.

Impact of age on the likelihood of reaching a minimum clinically important difference in 374 three-column spinal osteotomies: clinical article.

Challenges for defining minimal clinically important difference (MCID) after spinal cord injury.

Review of clinical outcomes-based anchors of minimum clinically important differences in hip and knee registry-based reports and publications.

The EQ-5D-5L health status questionnaire in COPD: validity, responsiveness and minimum important difference.

Invited commentary on 'different minimal clinically important difference (MCID) scores lead to different clinical prediction rules for the Oswestry disability index when using the same sample of patients'.