Osteoarthritis and Cartilage 22 (2014) 1107e1110
Brief Report
Concordance between important change and acceptable symptom state following knee arthroplasty: the role of baseline scores A. Escobar y z, D.L. Riddle x * y Research Unit, Hospital Universitario Basurto, Avda. Montevideo 18, 48013 Bilbao, Spain n en Servicios de Salud en Enfermedades Cro nicas (REDISSEC), Spain z Red de Investigacio x Departments of Physical Therapy and Orthopaedic Surgery, Virginia Commonwealth University, Richmond, VA, USA
a r t i c l e i n f o
s u m m a r y
Article history: Received 12 March 2014 Accepted 10 June 2014
Objective: To assess the influence of baseline score on concordance between the Minimal Clinically Important Difference (MCID) and Patient Acceptable Symptom State (PASS) estimates obtained on persons following total knee arthroplasty (TKA). Design: Patients scheduled for TKA in 15 hospitals in Spain were recruited and provided pre-operative and 1-year postoperative Western Ontario and McMaster Universities Osteoarthritis Index (WOMAC) Pain and Function scores. 1 year following surgery, patients completed questionnaires that determined the importance of any changes in status since surgery as well as the extent of satisfaction with the 1-year outcome. Kappa (k) was used to determine the extent of concordance between MCID and PASS measures for the entire sample and after splitting the sample based on quartiles of the baseline WOMAC scores. Results: A total of 923 patients participated in the study. The extent of concordance between MCID and PASS without regard to baseline score was k ¼ 0.41 (95% Confidence Interval (CI) ¼ 0.36, 0.47) for WOMAC Function and after stratifying baseline scores into quartiles, k ¼ 0.72 (95%CI ¼ 0.68, 0.77). Similar estimates were obtained for WOMAC Pain. Conclusions: Baseline score has substantial influence on the extent of concordance between MCID and PASS for patients undergoing TKA. Clinicians should account for baseline score when interpreting either MCID or PASS and the extent to which these measures agree. These findings have potential to influence the interpretation of outcome following TKA. © 2014 Osteoarthritis Research Society International. Published by Elsevier Ltd. All rights reserved.
Keywords: Knee Arthroplasty PASS MCID
Two commonly recommended evidence-based methods for judging response to treatment are the Minimal Clinically Important Difference (MCID), also referred to as Minimal Clinically Important Improvement (MCII) and the Patient Acceptable Symptom State (PASS)1. These approaches are conceptually distinct but complementary. The MCID (or MCII) describes the magnitude of change associated with patient-perceived important change with treatment while the PASS describes patients' satisfaction or acceptability with an outcome at a point in time following an intervention. Some have described MCID as a journey2 that results in the patient “feeling better”3 while PASS equates to an acceptable destination and “feeling good” following treatment.
* Address correspondence and reprint requests to: D.L. Riddle, Otto D. Payton Professor, Departments of Physical Therapy and Orthopaedic Surgery, Virginia Commonwealth University, Richmond, VA 23298-0224, USA. Tel: 1-804-828-0234; Fax: 1-804-828-8111. E-mail addresses: [email protected] (A. Escobar), [email protected] (D.L. Riddle).
The patient's baseline score has been repeatedly shown to be an important modifier of both MCID and PASS estimates in several studies of persons with knee osteoarthritis (OA) (3e7). Patients whose knee pain is worse prior to treatment, for example, tend to perceive larger changes in pain as being important and they tend to be satisfied with greater pain following treatment as compared to persons with less pre-treatment pain. An important line of inquiry is to determine the extent to which MCID and PASS estimates are equivalent. That is, if a patient achieves the MCID, how often does this same patient achieve PASS? If these two approaches align at the level of the individual, interpretation of outcome would be much simpler and clinicians would only need to use a single approach. When PASS and MCID do not agree for a patient, say the patient does not meet the MCID but achieves PASS, interpretation is more nuanced and potentially confusing for both the clinician and patient. For example, which of these two attributes is more important or should they be weighted the same? Tubach and colleagues have suggested that MCID and PASS are equivalent for persons with knee OA in that the MCID subtracted
http://dx.doi.org/10.1016/j.joca.2014.06.006 1063-4584/© 2014 Osteoarthritis Research Society International. Published by Elsevier Ltd. All rights reserved.
1108
A. Escobar, D.L. Riddle / Osteoarthritis and Cartilage 22 (2014) 1107e1110
from the baseline score was equivalent to PASS, even when adjusting for baseline score3. A limitation of the approach by Tubach and colleagues is the use of group-based methods for comparing PASS to MCID. Group means may be similar but there may be substantial discordance between PASS and MCID at the level of the individual. Maxwell and colleagues, on the other hand, reported a third of patients undergoing total knee arthroplasty (TKA) were discordant on PASS and MCID estimates4. A limitation of the approach by Maxwell and colleagues is that baseline score was not adjusted for in the analysis. Baseline score influences both PASS and MCID3,5e7. The MCID and PASS are commonly recommended methods for judging response to treatment both in clinical practice8 and in clinical trials9. However, the literature has not established whether these alternative methods of judging outcome produce similar findings at the level of individual patients following TKA. If MCID and PASS estimates are different at the individual patient level, clinicians should assess both given the distinct inferences associated with the two approaches. The purpose of our study was to assess the impact of baseline stratification on the extent to which either Western Ontario and McMaster Universities Osteoarthritis Index (WOMAC) Function or WOMAC Pain PASS and MCID estimates are concordant for persons undergoing TKA. Methods We have combined two cohorts of patients waiting for TKA, with similar sample characteristics, and which have been described elsewhere10. Consecutive patients placed on the waiting list to undergo primary TKA for OA and managed in the 15 participant hospitals were eligible for the study. All patients received a letter informing them about the study and asking for their voluntary participation. Questionnaires were sent to consenting patients at baseline and 12 months post-surgery. Of the 1616 persons who were offered participation, a total of 923 (57.1%) completed all 12month questionnaires. The Institutional Review Boards of the Hospitals approved the studies. We used the WOMAC, a disease-specific, self-administered questionnaire11. It is a multidimensional scale made up of 24 items grouped into three dimensions: pain (5 items), stiffness (2 items), and physical function (17 items). Our focus in the current study was the WOMAC Pain and Function dimensions. We used the Likert version with five response levels for each item, representing different degrees of intensity (none, mild, moderate, severe or extreme) and that were scored from 0 to 4. The final scores were determined by adding the corresponding items for each dimension, and standardizing to a range of values from 0 to 100. We used the reverse option, where 0 represents the worst and 100 the best status. The WOMAC has been translated and validated into Spanish12. Statistical analysis Descriptive statistics included frequencies and percentages for categorical variables and means and standard deviations (sd) for continuous variables. MCID has been defined1 as the smallest difference between the scores in a questionnaire that the patient perceives to be beneficial. All patients had to answer two questions, raw transition items (RTI), about their improvement or deterioration, one about pain and another about function 1 year after TKA. The five possible responses were “a great deal better”, “somewhat better”, “equal”, “somewhat worse” and “a great deal worse”. The MCID was estimated for pain and function by the mean change score for those patients who declared changes “somewhat better” in the RTI regarding their pain and function taking the difference
between the score at 1 year and at the baseline. The usefulness of our RTI in establishing the MCID has been previously described10. Following the paper of Tubach et al.13, we used the concept of PASS. It has been studied using the question: “If you had to be the rest of your life with the symptoms you have now, how would you feel?” In lieu of a “yes” or “no” response as defined by Tubach et al.13, categories in our study were very satisfied, somewhat satisfied, somewhat dissatisfied and very dissatisfied. Patients who were “very satisfied” or “somewhat satisfied” were considered as satisfied while the remainder was considered as unsatisfied (Yes/ No). As we have used the reverse option for WOMAC scores where 0 represents the worst status and 100 the best possible status, we defined the PASS as the twenty-fifth percentile of the final score at 1 year instead of the seventy-fifth percentile. We analyzed the extent of agreement between PASS and MCID by means of kappa coefficient (k) along with 95% Confidence Intervals (CIs). Analyses were performed globally (i.e., without baseline score stratification) and according to baseline severity quartiles. We stratified by quartiles because our relatively large sample size allowed for reasonably narrow confidence limits. The influence of the baseline score on concordance was judged by the extent to which, if any, the 95% CIs of the k coefficients for one measure (e.g., baseline scores analyzed in quartiles, k for WOMAC Function) demonstrated overlap with the point estimate of the other measure (e.g., global k for WOMAC Function). All statistical analyses were performed using SPSS version 17 and Confidence Interval Analysis (CIA) v.2.2. Results We had 923 patients in our sample, 70% were female and the average age was 71.6 (sd ¼ 6.8) years. Average WOMAC Function scores were 38.7 (sd ¼ 17.5) at baseline and 70.7 (sd ¼ 20.10) 1-year following TKA. The WOMAC Pain average score prior to TKA was 44.5 (sd ¼ 18.1) at baseline and 78.5 (sd ¼ 18.3) 1-year later. Global and baseline scores analyzed in quartiles for MCID and PASS scores are reported in Table I and the percentage of patients in each combination MCID/PASS subgroup and global scores along with k coefficients for WOMAC Pain and Function scales are reported in Table II. Kappa (k) for the global MCID and PASS scores was 0.41 (95%CI ¼ 0.36, 0.47) for WOMAC Function and 0.46 (95%CI ¼ 0.40, 0.52) for WOMAC Pain. For the baseline scores analyzed in quartiles, MCID and PASS scores, k for WOMAC Function was 0.72 (95% CI ¼ 0.68, 0.77) and 0.75 (95%CI ¼ 0.70, 0.79) for WOMAC Pain. Additional analyses of the MCID and PASS are reported in the online Appendix. Discussion Interpreting changes in patient status following TKA is most commonly judged using MCID and/or PASS. Both potentially provide useful information regarding either the improvement/worsening since surgery (MCID) or the final outcome status (PASS). These measures, although complementary can sometimes provide conflicting information and the literature is unclear on how common conflicting results occur. We found that adjustment for baseline status is critically important when making interpretations from MCID and PASS estimates as a composite indicator of both journey and destination outcomes following TKA. Point estimates of the extent of concordance along with 95% CIs were substantially higher when baseline scores were accounted for as compared to global scores, as seen in Table II. When a clinician accounts for baseline scores, agreement between MCID and PASS is substantial14 and disagreement is infrequent. As noted in Table I, the magnitude of PASS and MCID estimates are substantially influenced by baseline score quartile. For example, the WOMAC Function MCID is 47.5 for
A. Escobar, D.L. Riddle / Osteoarthritis and Cartilage 22 (2014) 1107e1110
1109
Table I WOMAC Pain and Function scores, MCID and PASS scores for total sample and by quartiles of baseline severity
Quartiles scores WOMAC Function Baseline score PASS twenty-fifth (95% CI) MCID mean (95% CI) Quartiles scores WOMAC Pain Baseline score PASS twenty-fifth (95% CI) MCID mean (95% CI) *
Quartile 1 (worst pain or functional loss)
Quartile 2
Quartile 3
Quartile 4 (least pain or functional loss)
Global*
50.0
e
16.7 (15.8e17.6) 58.8 (54.4e63.2) 47.5 (43.4e51.6)
33.0 (32.6e33.5) 64.7 (61.8e67.6) 37.2 (34.7e39.7)
44.9 (44.4e45.3) 69.1 (67.6e72.1) 27.6 (24.6e30.7)
61.6 (60.5e62.7) 72.1 (67.7e75.0) 15.9 (13.3e18.4)
38.7 (37.5e39.7) 66.2 (64.7e67.7) 32.4 (30.5e34.2)
55.0
e
20.7 (19.5e21.9) 70.0 (65.0e75.0) 47.9 (43.6e52.3)
39.6 (39.2e40.1) 75.0 (70.0e80.0) 32.2 (29.1e35.2)
51.1 (50.7e51.6) 75.0 (70.0e80.0) 25.1 (22.5e27.7)
67.6 (66.5e68.8) 80.0 (75.0e85.0) 12.4 (9.2e15.5)
44.5 (43.3e45.6) 75.0 (70.0e80.0) 29.0 (27.0e31.1)
Global scores did not account for baseline WOMAC score severity.
the worst baseline score quartile and 15.9 for the least baseline score quartile. If a clinician does not account for baseline score, clinical interpretation will suffer and conflicting findings among PASS and MCID will be more common. While we studied the MCID and PASS of self-reported measures, these methods for interpreting changes over time also are relevant for performance measures which are commonly conducted on persons with TKA. Ours is not the first study to quantify the importance of baseline score when interpreting outcome. Davis and colleagues, for example, found that misclassification bias increases if the baseline score is not accounted for when interpreting change (i.e., MCID) in patients following TKA6. Baseline status is an important predictor of both change and final outcome and our study, for the first time, demonstrates the important role of baseline status in interpreting MCID and PASS concordance for individual patients undergoing TKA. Others have reported on similarities between MCID and PASS estimates for groups of patients3,7 but the utility of this information is limited when applied to individuals. Maxwell and colleagues compared WOMAC PASS and MCID estimates and found a 67% agreement for PASS and MCID measures for 228 patients with TKA. The authors did not account for baseline scores. We calculated kappa (k) from this data and found a k ¼ 0.37 (95%CI ¼ 0.26, 0.49). This estimate was very similar to our global comparison of MCID and PASS (k ¼ 0.41 (95%CI ¼ 0.36e0.47)). Our results combined with the findings of Maxwell and colleagues provide consensus evidence indicating that PASS and MCID show only moderate agreement if baseline scores are not accounted for. Our estimates of PASS/MCID concordance for our analyses of baseline quartile scores support the notion that clinicians should
Table II Percentage of patients in each combination of PASS and MCID and kappa coefficients for WOMAC Pain and WOMAC Function PASS/MCID
WOMAC Function PASS þ MCID þ* PASS þ MCID y PASS MCID þ PASS MCID Kappa (95%CI) WOMAC Pain PASS þ MCID þ PASS þ MCID PASS MCID þ PASS MCID Kappa (95%CI) * y
Total sample cut-off
Cut-off by quartiles
41.0% 23.0% 6.8% 29.2% 0.41 (0.36e0.47)
52.6% 11.9% 1.5% 34.0% 0.72 (0.68e0.77)
54.4% 16.3% 7.8% 21.5% 0.46 (0.40e0.52)
62.3% 8.2% 2.7% 26.7% 0.75 (0.70e0.79)
PASS/MCID þ: patients who attained cut-off scores. PASS/MCID : patients who did not reach the cut-off scores.
routinely account for baseline scores for both PASS and MCID when making comprehensive judgments about progress and outcome following TKA. The evidence demonstrating the role of baseline score in influencing MCID estimates3,6,10 and PASS estimates15 combined with our evidence supporting the role of baseline score on estimates of concordance between PASS and MCID make a compelling case for the routine use of baseline scores by both clinicians and researchers in interpreting changes and outcomes following TKA. While we found good concordance between MCID and PASS, we still endorse use of both estimates in daily practice because while agreement was strong (i.e., 86.6% agreement and k ¼ 0.72 for WOMAC Function and 89% agreement and k ¼ 0.75 for WOMAC Pain), there will be some patients in whom there is discordance. For those patients in whom discordance exists, the clinician will need to make clinical judgments separately for MCID and PASS and consider possible reasons for why one indicator was met and the other was not. Importantly, our study was not designed to determine whether MCID or PASS is more important for clinical decisions. Until consensus is reached among key arthroplasty stakeholders on relative importance of PASS vs MCID, it is our opinion that both indicators of outcome be used routinely in clinical practice to ascertain a comprehensive understanding of likely surgical effects. Contributions AE and DLR conceived of and designed the study, AE acquired and analyzed the data and AE and DLR interpreted the data. AE and DLR drafted the article and revised it critically for important intellectual content and both AE and DLR gave final approval of the version to be submitted. Role of the funding source This study was supported by grants from the Fondo de Invesn Sanitaria: First cohort: FIS01/184, and Second cohort: tigacio FIS (PI 04/0938, PI 04/0542, and PI 04/2577) and from the Basque Country Health Department (200411012). Both public sources have only been funders of the research projects. The study sponsors played no role in the analysis or drafting of the manuscript. Competing interest statement DLR and AE declare no competing interests. Acknowledgments None to declare.
1110
A. Escobar, D.L. Riddle / Osteoarthritis and Cartilage 22 (2014) 1107e1110
Supplementary data Supplementary data related to this article can be found at http:// dx.doi.org/10.1016/j.joca.2014.06.006. References 1. Kvien TK, Heiberg T, Hagen KB. Minimal clinically important improvement/difference (MCII/MCID) and patient acceptable symptom state (PASS): what do these concepts mean? Ann Rheum Dis 2007;66(Suppl 3):iii40e1. 2. Losina E, Katz JN. Total knee replacement: pursuit of the paramount result. Rheumatology (Oxford) 2012;51:1735e6. 3. Tubach F, Dougados M, Falissard B, Baron G, Logeart I, Ravaud P. Feeling good rather than feeling better matters more to patients. Arthritis Rheum 2006;55:526e30. 4. Maxwell JL, Felson DT, Niu J, Wise B, Nevitt MC, Singh JA, et al. Does clinically important change in function after knee replacement guarantee good absolute function? The multicenter osteoarthritis study. J Rheumatol 2014;41:60e4. 5. Escobar A, Gonzalez M, Quintana JM, Vrotsou K, Bilbao A, Herrera-Espineira C, et al. Patient acceptable symptom state and OMERACT-OARSI set of responder criteria in joint replacement. Identification of cut-off values. Osteoarthritis Cartilage 2012;20:87e92. 6. Davis AM, Perruccio AV, Lohmander LS. Minimally clinically important improvement: all non-responders are not really non-responders an illustration from total knee replacement. Osteoarthritis Cartilage 2012;20:364e7. 7. Judge A, Arden NK, Cooper C, Kassim JM, Carr AJ, Field RE, et al. Predictors of outcomes of total knee replacement surgery. Rheumatology (Oxford) 2012;51:1804e13. 8. Perrot S, Bertin P. “Feeling better” or “feeling well” in usual care of hip and knee osteoarthritis pain: determination of
9.
10.
11.
12.
13.
14. 15.
cutoff points for patient acceptable symptom state (PASS) and minimal clinically important improvement (MCII) at rest and on movement in a national multicenter cohort study of 2414 patients with painful osteoarthritis. Pain 2013;154: 248e56. Meyer RJ. U.S. regulatory perspective on the minimal clinically important difference in chronic obstructive pulmonary disease. COPD 2005;2:47e9. Escobar A, Garcia PL, Herrera-Espineira C, Aizpuru F, Sarasqueta C, Gonzalez Saenz de TM, et al. Total knee replacement; minimal clinically important differences and responders. Osteoarthritis Cartilage 2013;21:2006e12. Bellamy N, Buchanan WW, Goldsmith CH, Campbell J, Stitt LW. Validation study of WOMAC: a health status instrument for measuring clinically important patient relevant outcomes to antirheumatic drug therapy in patients with osteoarthritis of the hip or knee. J Rheumatol 1988;15:1833e40. Escobar A, Quintana JM, Bilbao A, Azkarate J, Guenaga JI. Validation of the Spanish version of the WOMAC questionnaire for patients with hip or knee osteoarthritis. Western Ontario and McMaster Universities Osteoarthritis Index. Clin Rheumatol 2002;21:466e71. Tubach F, Ravaud P, Baron G, Falissard B, Logeart I, Bellamy N, et al. Evaluation of clinically relevant states in patient reported outcomes in knee and hip osteoarthritis: the patient acceptable symptom state. Ann Rheum Dis 2005;64:34e7. Landis JR, Koch GG. The measurement of observer agreement for categorical data. Biometrics 1977;33:159e74. Judge A, Arden NK, Kiran A, Price A, Javaid MK, Beard D, et al. Interpretation of patient-reported outcomes for hip and knee replacement surgery: identification of thresholds associated with satisfaction with surgery. J Bone Joint Surg Br 2012;94: 412e8.
Brief Report
Concordance between important change and acceptable symptom state following knee arthroplasty: the role of baseline scores A. Escobar y z, D.L. Riddle x * y Research Unit, Hospital Universitario Basurto, Avda. Montevideo 18, 48013 Bilbao, Spain n en Servicios de Salud en Enfermedades Cro nicas (REDISSEC), Spain z Red de Investigacio x Departments of Physical Therapy and Orthopaedic Surgery, Virginia Commonwealth University, Richmond, VA, USA
a r t i c l e i n f o
s u m m a r y
Article history: Received 12 March 2014 Accepted 10 June 2014
Objective: To assess the influence of baseline score on concordance between the Minimal Clinically Important Difference (MCID) and Patient Acceptable Symptom State (PASS) estimates obtained on persons following total knee arthroplasty (TKA). Design: Patients scheduled for TKA in 15 hospitals in Spain were recruited and provided pre-operative and 1-year postoperative Western Ontario and McMaster Universities Osteoarthritis Index (WOMAC) Pain and Function scores. 1 year following surgery, patients completed questionnaires that determined the importance of any changes in status since surgery as well as the extent of satisfaction with the 1-year outcome. Kappa (k) was used to determine the extent of concordance between MCID and PASS measures for the entire sample and after splitting the sample based on quartiles of the baseline WOMAC scores. Results: A total of 923 patients participated in the study. The extent of concordance between MCID and PASS without regard to baseline score was k ¼ 0.41 (95% Confidence Interval (CI) ¼ 0.36, 0.47) for WOMAC Function and after stratifying baseline scores into quartiles, k ¼ 0.72 (95%CI ¼ 0.68, 0.77). Similar estimates were obtained for WOMAC Pain. Conclusions: Baseline score has substantial influence on the extent of concordance between MCID and PASS for patients undergoing TKA. Clinicians should account for baseline score when interpreting either MCID or PASS and the extent to which these measures agree. These findings have potential to influence the interpretation of outcome following TKA. © 2014 Osteoarthritis Research Society International. Published by Elsevier Ltd. All rights reserved.
Keywords: Knee Arthroplasty PASS MCID
Two commonly recommended evidence-based methods for judging response to treatment are the Minimal Clinically Important Difference (MCID), also referred to as Minimal Clinically Important Improvement (MCII) and the Patient Acceptable Symptom State (PASS)1. These approaches are conceptually distinct but complementary. The MCID (or MCII) describes the magnitude of change associated with patient-perceived important change with treatment while the PASS describes patients' satisfaction or acceptability with an outcome at a point in time following an intervention. Some have described MCID as a journey2 that results in the patient “feeling better”3 while PASS equates to an acceptable destination and “feeling good” following treatment.
* Address correspondence and reprint requests to: D.L. Riddle, Otto D. Payton Professor, Departments of Physical Therapy and Orthopaedic Surgery, Virginia Commonwealth University, Richmond, VA 23298-0224, USA. Tel: 1-804-828-0234; Fax: 1-804-828-8111. E-mail addresses: [email protected] (A. Escobar), [email protected] (D.L. Riddle).
The patient's baseline score has been repeatedly shown to be an important modifier of both MCID and PASS estimates in several studies of persons with knee osteoarthritis (OA) (3e7). Patients whose knee pain is worse prior to treatment, for example, tend to perceive larger changes in pain as being important and they tend to be satisfied with greater pain following treatment as compared to persons with less pre-treatment pain. An important line of inquiry is to determine the extent to which MCID and PASS estimates are equivalent. That is, if a patient achieves the MCID, how often does this same patient achieve PASS? If these two approaches align at the level of the individual, interpretation of outcome would be much simpler and clinicians would only need to use a single approach. When PASS and MCID do not agree for a patient, say the patient does not meet the MCID but achieves PASS, interpretation is more nuanced and potentially confusing for both the clinician and patient. For example, which of these two attributes is more important or should they be weighted the same? Tubach and colleagues have suggested that MCID and PASS are equivalent for persons with knee OA in that the MCID subtracted
http://dx.doi.org/10.1016/j.joca.2014.06.006 1063-4584/© 2014 Osteoarthritis Research Society International. Published by Elsevier Ltd. All rights reserved.
1108
A. Escobar, D.L. Riddle / Osteoarthritis and Cartilage 22 (2014) 1107e1110
from the baseline score was equivalent to PASS, even when adjusting for baseline score3. A limitation of the approach by Tubach and colleagues is the use of group-based methods for comparing PASS to MCID. Group means may be similar but there may be substantial discordance between PASS and MCID at the level of the individual. Maxwell and colleagues, on the other hand, reported a third of patients undergoing total knee arthroplasty (TKA) were discordant on PASS and MCID estimates4. A limitation of the approach by Maxwell and colleagues is that baseline score was not adjusted for in the analysis. Baseline score influences both PASS and MCID3,5e7. The MCID and PASS are commonly recommended methods for judging response to treatment both in clinical practice8 and in clinical trials9. However, the literature has not established whether these alternative methods of judging outcome produce similar findings at the level of individual patients following TKA. If MCID and PASS estimates are different at the individual patient level, clinicians should assess both given the distinct inferences associated with the two approaches. The purpose of our study was to assess the impact of baseline stratification on the extent to which either Western Ontario and McMaster Universities Osteoarthritis Index (WOMAC) Function or WOMAC Pain PASS and MCID estimates are concordant for persons undergoing TKA. Methods We have combined two cohorts of patients waiting for TKA, with similar sample characteristics, and which have been described elsewhere10. Consecutive patients placed on the waiting list to undergo primary TKA for OA and managed in the 15 participant hospitals were eligible for the study. All patients received a letter informing them about the study and asking for their voluntary participation. Questionnaires were sent to consenting patients at baseline and 12 months post-surgery. Of the 1616 persons who were offered participation, a total of 923 (57.1%) completed all 12month questionnaires. The Institutional Review Boards of the Hospitals approved the studies. We used the WOMAC, a disease-specific, self-administered questionnaire11. It is a multidimensional scale made up of 24 items grouped into three dimensions: pain (5 items), stiffness (2 items), and physical function (17 items). Our focus in the current study was the WOMAC Pain and Function dimensions. We used the Likert version with five response levels for each item, representing different degrees of intensity (none, mild, moderate, severe or extreme) and that were scored from 0 to 4. The final scores were determined by adding the corresponding items for each dimension, and standardizing to a range of values from 0 to 100. We used the reverse option, where 0 represents the worst and 100 the best status. The WOMAC has been translated and validated into Spanish12. Statistical analysis Descriptive statistics included frequencies and percentages for categorical variables and means and standard deviations (sd) for continuous variables. MCID has been defined1 as the smallest difference between the scores in a questionnaire that the patient perceives to be beneficial. All patients had to answer two questions, raw transition items (RTI), about their improvement or deterioration, one about pain and another about function 1 year after TKA. The five possible responses were “a great deal better”, “somewhat better”, “equal”, “somewhat worse” and “a great deal worse”. The MCID was estimated for pain and function by the mean change score for those patients who declared changes “somewhat better” in the RTI regarding their pain and function taking the difference
between the score at 1 year and at the baseline. The usefulness of our RTI in establishing the MCID has been previously described10. Following the paper of Tubach et al.13, we used the concept of PASS. It has been studied using the question: “If you had to be the rest of your life with the symptoms you have now, how would you feel?” In lieu of a “yes” or “no” response as defined by Tubach et al.13, categories in our study were very satisfied, somewhat satisfied, somewhat dissatisfied and very dissatisfied. Patients who were “very satisfied” or “somewhat satisfied” were considered as satisfied while the remainder was considered as unsatisfied (Yes/ No). As we have used the reverse option for WOMAC scores where 0 represents the worst status and 100 the best possible status, we defined the PASS as the twenty-fifth percentile of the final score at 1 year instead of the seventy-fifth percentile. We analyzed the extent of agreement between PASS and MCID by means of kappa coefficient (k) along with 95% Confidence Intervals (CIs). Analyses were performed globally (i.e., without baseline score stratification) and according to baseline severity quartiles. We stratified by quartiles because our relatively large sample size allowed for reasonably narrow confidence limits. The influence of the baseline score on concordance was judged by the extent to which, if any, the 95% CIs of the k coefficients for one measure (e.g., baseline scores analyzed in quartiles, k for WOMAC Function) demonstrated overlap with the point estimate of the other measure (e.g., global k for WOMAC Function). All statistical analyses were performed using SPSS version 17 and Confidence Interval Analysis (CIA) v.2.2. Results We had 923 patients in our sample, 70% were female and the average age was 71.6 (sd ¼ 6.8) years. Average WOMAC Function scores were 38.7 (sd ¼ 17.5) at baseline and 70.7 (sd ¼ 20.10) 1-year following TKA. The WOMAC Pain average score prior to TKA was 44.5 (sd ¼ 18.1) at baseline and 78.5 (sd ¼ 18.3) 1-year later. Global and baseline scores analyzed in quartiles for MCID and PASS scores are reported in Table I and the percentage of patients in each combination MCID/PASS subgroup and global scores along with k coefficients for WOMAC Pain and Function scales are reported in Table II. Kappa (k) for the global MCID and PASS scores was 0.41 (95%CI ¼ 0.36, 0.47) for WOMAC Function and 0.46 (95%CI ¼ 0.40, 0.52) for WOMAC Pain. For the baseline scores analyzed in quartiles, MCID and PASS scores, k for WOMAC Function was 0.72 (95% CI ¼ 0.68, 0.77) and 0.75 (95%CI ¼ 0.70, 0.79) for WOMAC Pain. Additional analyses of the MCID and PASS are reported in the online Appendix. Discussion Interpreting changes in patient status following TKA is most commonly judged using MCID and/or PASS. Both potentially provide useful information regarding either the improvement/worsening since surgery (MCID) or the final outcome status (PASS). These measures, although complementary can sometimes provide conflicting information and the literature is unclear on how common conflicting results occur. We found that adjustment for baseline status is critically important when making interpretations from MCID and PASS estimates as a composite indicator of both journey and destination outcomes following TKA. Point estimates of the extent of concordance along with 95% CIs were substantially higher when baseline scores were accounted for as compared to global scores, as seen in Table II. When a clinician accounts for baseline scores, agreement between MCID and PASS is substantial14 and disagreement is infrequent. As noted in Table I, the magnitude of PASS and MCID estimates are substantially influenced by baseline score quartile. For example, the WOMAC Function MCID is 47.5 for
A. Escobar, D.L. Riddle / Osteoarthritis and Cartilage 22 (2014) 1107e1110
1109
Table I WOMAC Pain and Function scores, MCID and PASS scores for total sample and by quartiles of baseline severity
Quartiles scores WOMAC Function Baseline score PASS twenty-fifth (95% CI) MCID mean (95% CI) Quartiles scores WOMAC Pain Baseline score PASS twenty-fifth (95% CI) MCID mean (95% CI) *
Quartile 1 (worst pain or functional loss)
Quartile 2
Quartile 3
Quartile 4 (least pain or functional loss)
Global*
50.0
e
16.7 (15.8e17.6) 58.8 (54.4e63.2) 47.5 (43.4e51.6)
33.0 (32.6e33.5) 64.7 (61.8e67.6) 37.2 (34.7e39.7)
44.9 (44.4e45.3) 69.1 (67.6e72.1) 27.6 (24.6e30.7)
61.6 (60.5e62.7) 72.1 (67.7e75.0) 15.9 (13.3e18.4)
38.7 (37.5e39.7) 66.2 (64.7e67.7) 32.4 (30.5e34.2)
55.0
e
20.7 (19.5e21.9) 70.0 (65.0e75.0) 47.9 (43.6e52.3)
39.6 (39.2e40.1) 75.0 (70.0e80.0) 32.2 (29.1e35.2)
51.1 (50.7e51.6) 75.0 (70.0e80.0) 25.1 (22.5e27.7)
67.6 (66.5e68.8) 80.0 (75.0e85.0) 12.4 (9.2e15.5)
44.5 (43.3e45.6) 75.0 (70.0e80.0) 29.0 (27.0e31.1)
Global scores did not account for baseline WOMAC score severity.
the worst baseline score quartile and 15.9 for the least baseline score quartile. If a clinician does not account for baseline score, clinical interpretation will suffer and conflicting findings among PASS and MCID will be more common. While we studied the MCID and PASS of self-reported measures, these methods for interpreting changes over time also are relevant for performance measures which are commonly conducted on persons with TKA. Ours is not the first study to quantify the importance of baseline score when interpreting outcome. Davis and colleagues, for example, found that misclassification bias increases if the baseline score is not accounted for when interpreting change (i.e., MCID) in patients following TKA6. Baseline status is an important predictor of both change and final outcome and our study, for the first time, demonstrates the important role of baseline status in interpreting MCID and PASS concordance for individual patients undergoing TKA. Others have reported on similarities between MCID and PASS estimates for groups of patients3,7 but the utility of this information is limited when applied to individuals. Maxwell and colleagues compared WOMAC PASS and MCID estimates and found a 67% agreement for PASS and MCID measures for 228 patients with TKA. The authors did not account for baseline scores. We calculated kappa (k) from this data and found a k ¼ 0.37 (95%CI ¼ 0.26, 0.49). This estimate was very similar to our global comparison of MCID and PASS (k ¼ 0.41 (95%CI ¼ 0.36e0.47)). Our results combined with the findings of Maxwell and colleagues provide consensus evidence indicating that PASS and MCID show only moderate agreement if baseline scores are not accounted for. Our estimates of PASS/MCID concordance for our analyses of baseline quartile scores support the notion that clinicians should
Table II Percentage of patients in each combination of PASS and MCID and kappa coefficients for WOMAC Pain and WOMAC Function PASS/MCID
WOMAC Function PASS þ MCID þ* PASS þ MCID y PASS MCID þ PASS MCID Kappa (95%CI) WOMAC Pain PASS þ MCID þ PASS þ MCID PASS MCID þ PASS MCID Kappa (95%CI) * y
Total sample cut-off
Cut-off by quartiles
41.0% 23.0% 6.8% 29.2% 0.41 (0.36e0.47)
52.6% 11.9% 1.5% 34.0% 0.72 (0.68e0.77)
54.4% 16.3% 7.8% 21.5% 0.46 (0.40e0.52)
62.3% 8.2% 2.7% 26.7% 0.75 (0.70e0.79)
PASS/MCID þ: patients who attained cut-off scores. PASS/MCID : patients who did not reach the cut-off scores.
routinely account for baseline scores for both PASS and MCID when making comprehensive judgments about progress and outcome following TKA. The evidence demonstrating the role of baseline score in influencing MCID estimates3,6,10 and PASS estimates15 combined with our evidence supporting the role of baseline score on estimates of concordance between PASS and MCID make a compelling case for the routine use of baseline scores by both clinicians and researchers in interpreting changes and outcomes following TKA. While we found good concordance between MCID and PASS, we still endorse use of both estimates in daily practice because while agreement was strong (i.e., 86.6% agreement and k ¼ 0.72 for WOMAC Function and 89% agreement and k ¼ 0.75 for WOMAC Pain), there will be some patients in whom there is discordance. For those patients in whom discordance exists, the clinician will need to make clinical judgments separately for MCID and PASS and consider possible reasons for why one indicator was met and the other was not. Importantly, our study was not designed to determine whether MCID or PASS is more important for clinical decisions. Until consensus is reached among key arthroplasty stakeholders on relative importance of PASS vs MCID, it is our opinion that both indicators of outcome be used routinely in clinical practice to ascertain a comprehensive understanding of likely surgical effects. Contributions AE and DLR conceived of and designed the study, AE acquired and analyzed the data and AE and DLR interpreted the data. AE and DLR drafted the article and revised it critically for important intellectual content and both AE and DLR gave final approval of the version to be submitted. Role of the funding source This study was supported by grants from the Fondo de Invesn Sanitaria: First cohort: FIS01/184, and Second cohort: tigacio FIS (PI 04/0938, PI 04/0542, and PI 04/2577) and from the Basque Country Health Department (200411012). Both public sources have only been funders of the research projects. The study sponsors played no role in the analysis or drafting of the manuscript. Competing interest statement DLR and AE declare no competing interests. Acknowledgments None to declare.
1110
A. Escobar, D.L. Riddle / Osteoarthritis and Cartilage 22 (2014) 1107e1110
Supplementary data Supplementary data related to this article can be found at http:// dx.doi.org/10.1016/j.joca.2014.06.006. References 1. Kvien TK, Heiberg T, Hagen KB. Minimal clinically important improvement/difference (MCII/MCID) and patient acceptable symptom state (PASS): what do these concepts mean? Ann Rheum Dis 2007;66(Suppl 3):iii40e1. 2. Losina E, Katz JN. Total knee replacement: pursuit of the paramount result. Rheumatology (Oxford) 2012;51:1735e6. 3. Tubach F, Dougados M, Falissard B, Baron G, Logeart I, Ravaud P. Feeling good rather than feeling better matters more to patients. Arthritis Rheum 2006;55:526e30. 4. Maxwell JL, Felson DT, Niu J, Wise B, Nevitt MC, Singh JA, et al. Does clinically important change in function after knee replacement guarantee good absolute function? The multicenter osteoarthritis study. J Rheumatol 2014;41:60e4. 5. Escobar A, Gonzalez M, Quintana JM, Vrotsou K, Bilbao A, Herrera-Espineira C, et al. Patient acceptable symptom state and OMERACT-OARSI set of responder criteria in joint replacement. Identification of cut-off values. Osteoarthritis Cartilage 2012;20:87e92. 6. Davis AM, Perruccio AV, Lohmander LS. Minimally clinically important improvement: all non-responders are not really non-responders an illustration from total knee replacement. Osteoarthritis Cartilage 2012;20:364e7. 7. Judge A, Arden NK, Cooper C, Kassim JM, Carr AJ, Field RE, et al. Predictors of outcomes of total knee replacement surgery. Rheumatology (Oxford) 2012;51:1804e13. 8. Perrot S, Bertin P. “Feeling better” or “feeling well” in usual care of hip and knee osteoarthritis pain: determination of
9.
10.
11.
12.
13.
14. 15.
cutoff points for patient acceptable symptom state (PASS) and minimal clinically important improvement (MCII) at rest and on movement in a national multicenter cohort study of 2414 patients with painful osteoarthritis. Pain 2013;154: 248e56. Meyer RJ. U.S. regulatory perspective on the minimal clinically important difference in chronic obstructive pulmonary disease. COPD 2005;2:47e9. Escobar A, Garcia PL, Herrera-Espineira C, Aizpuru F, Sarasqueta C, Gonzalez Saenz de TM, et al. Total knee replacement; minimal clinically important differences and responders. Osteoarthritis Cartilage 2013;21:2006e12. Bellamy N, Buchanan WW, Goldsmith CH, Campbell J, Stitt LW. Validation study of WOMAC: a health status instrument for measuring clinically important patient relevant outcomes to antirheumatic drug therapy in patients with osteoarthritis of the hip or knee. J Rheumatol 1988;15:1833e40. Escobar A, Quintana JM, Bilbao A, Azkarate J, Guenaga JI. Validation of the Spanish version of the WOMAC questionnaire for patients with hip or knee osteoarthritis. Western Ontario and McMaster Universities Osteoarthritis Index. Clin Rheumatol 2002;21:466e71. Tubach F, Ravaud P, Baron G, Falissard B, Logeart I, Bellamy N, et al. Evaluation of clinically relevant states in patient reported outcomes in knee and hip osteoarthritis: the patient acceptable symptom state. Ann Rheum Dis 2005;64:34e7. Landis JR, Koch GG. The measurement of observer agreement for categorical data. Biometrics 1977;33:159e74. Judge A, Arden NK, Kiran A, Price A, Javaid MK, Beard D, et al. Interpretation of patient-reported outcomes for hip and knee replacement surgery: identification of thresholds associated with satisfaction with surgery. J Bone Joint Surg Br 2012;94: 412e8.
