Journal of Clinical Epidemiology

-

(2014)

-

ORIGINAL ARTICLE

Minimal clinically important difference (MCID) for the functional assessment of cancer therapy: Cognitive function (FACT-Cog) in breast cancer patients Yin Ting Cheunga,b, Yu Lee Fooa, Maung Shwea, Yee Pin Tanc, Gilbert Fanc, Wei Sean Yongd, Preetha Madhukumard, Wei Seong Ooie, Wen Yee Chaye, Rebecca A. Dente,f, Soo Fan Ange, Soo Kien Loe, Yoon Sim Yape, Raymond Nge,f, Alexandre Chana,b,* a

Department of Pharmacy, National University of Singapore, 18 Science Drive 4, Singapore 117543 Department of Pharmacy, National Cancer Centre Singapore, 11 Hospital Drive, Singapore 169610 c Department of Psychosocial Oncology, National Cancer Centre Singapore, 11 Hospital Drive, Singapore 169610 d Department of Surgical Oncology, National Cancer Centre Singapore, 11 Hospital Drive, Singapore 169610 e Department of Medical Oncology, National Cancer Centre Singapore, 11 Hospital Drive, Singapore 169610 f Department of Clinical Sciences, Duke-NUS Graduate Medical School Singapore, 8 College Road Singapore 169857 b

Accepted 18 December 2013; Published online xxxx

Abstract Objectives: This is the first reported study to determine the minimal clinically important difference (MCID) of Functional Assessment of Cancer TherapyeCognitive Function (FACT-Cog), a validated subjective neuropsychological instrument designed to evaluate cancer patients’ perceived cognitive deterioration. Study Design and Setting: Breast cancer patients (n 5 220) completed FACT-Cog and European Organization for Research and Treatment of Cancer Quality of Life Questionnaire Core 30 (EORTC-QLQ-C30) at baseline and at least 3 months later. Anchor-based approach used the validated EORTC-QLQ-C30eCognitive Functioning scale (EORTC-CF) as the anchor for patients who showed minimal deterioration and a receiver operating characteristic (ROC) curve to identify the optimal MCID cutoff for deterioration. Distribution-based approach used one-third standard deviation (SD), half SD, and one standard error of measurement (SEM) of the total FACT-Cog score (148 points). Results: There was a moderate correlation between changes in FACT-Cog and EORTC-CF scores (r 5 0.43; P ! 0.001). The EORTCCFeanchored MCID was 9.6 points (95% confidence interval: 4.4, 14.8). The MCID from the ROC method was 7.5 points (area under the curve: 0.75; sensitivity: 75.6%; specificity: 68.8%). For the distribution-based approach, the MCIDs corresponding to one-third SD, half SD, and one SEM were 6.9, 10.3, and 10.6 points, respectively. Combining the approaches, the MCID identified for FACT-Cog ranged from 6.9 to 10.6 points (4.7e7.2% of the total score). Conclusion: The estimates of 6.9e10.6 points as MCID can facilitate the interpretation of patient-reported cognitive deterioration and sample size estimates in future studies. Ó 2014 Elsevier Inc. All rights reserved. Keywords: Chemotherapy; Cognitive function; Minimal clinically important difference; FACT-Cog; Quality of life; Breast cancer

1. Introduction Recent studies have revealed that cancer patients experience chemotherapy-associated cognitive changes [1e7]. Studies have focused primarily on breast cancer patients, Conflict of interest: All authors have no conflict of interest to declare. Funding: This study was financed by research grants awarded by the National University of Singapore (R-148-000-166-112) and the National Cancer Centre Singapore (NRFCB12131). This study was presented as a poster at the 2013 American Society for Clinical Oncology (ASCO) Meeting in Chicago, IL, USA (abstract #6589). * Corresponding author. Tel.: þ65-6436-8133; fax: þ65-6220-2573. E-mail address: [email protected] (A. Chan). 0895-4356/$ - see front matter Ó 2014 Elsevier Inc. All rights reserved. http://dx.doi.org/10.1016/j.jclinepi.2013.12.011

among which a prevalence of moderate-to-severe cognitive impairment ranging widely from 16% to 75% has been reported [6,8e10]. Manifestations of cognitive deficits typically present as subtle impairment in the cognitive domains of concentration, memory, processing speed, language, and executive function [1e5,11]. Studies have revealed that these cognitive changes can be distressing, leading to the disruption of functional abilities [1,3e5,11]. The Functional Assessment of Cancer TherapyeCognitive Function (FACT-Cog) is a subjective neuropsychological instrument designed to evaluate the effects of cancer patients’ perceived cognitive deterioration in their health-related quality of life (HRQoL) [12].

2

Y.T. Cheung et al. / Journal of Clinical Epidemiology

What is new? Key findings  Using both anchor- and distribution-based approaches, a reasonable range for the minimal clinically important difference (MCID) of the Functional Assessment of Cancer TherapyeCognitive Function (FACT-Cog; version 3) was identified.  A 6.9- to 10.6-point reduction of the FACT-Cog score corresponds to the smallest clinically relevant self-reported cognitive deterioration in Asian breast cancer patients. What this adds to what was known?  These estimates can be applied to gauge the clinical relevance of patient-reported cognitive changes. What is the implication and what should change now?  Future clinical studies with self-reported cognitive deterioration as end points can also use this MCID range for sample size estimation.

-

(2014)

-

for future studies. Through adopting the anchor- and distribution-based approaches, this study was designed to identify the MCID estimates for FACT-Cog in breast cancer patients. 2. Methods 2.1. Subjects and study design This was a prospective longitudinal study conducted at the outpatient clinics of the National Cancer Centre, Singapore, between November 2010 and December 2012. Inclusion criteria were a diagnosis of breast cancer, an age more than 21 years, had completed surgical treatment for breast cancer (mastectomy or lumpectomy), scheduled to receive chemotherapy treatment (anthracycline- or taxane based), and had not received radiotherapy at the point of recruitment. Patients were excluded if they were diagnosed with brain metastasis and/or any neuropsychiatric illness that might impair their cognitive function. The study protocol was approved by the SingHealth Institutional Review Board, and written informed consent was obtained from all the participants. 2.2. Study procedure

Conventionally, the results of patient-reported outcomes are analyzed and interpreted using statistical testing in clinical research. Tests of statistical significance are used to determine whether an observed difference is attributable to chance alone. Although such statistical comparisons can reflect a statistical change in the measured outcome, they do not indicate whether the difference is clinically relevant to patients. In essence, the clinical meaning of cognitive deterioration cannot be adequately determined from statistical significance alone. To facilitate the interpretation of the clinical relevance of score changes, the concept of minimal clinically important difference (MCID) was proposed [13]. For a given instrument, the MCID represents the smallest difference in scores that is considered clinically significant. Identification of the MCID can guide the clinical interpretation of patientreported cognitive changes by providing recognizable end points. In the literature, two approaches are identified to estimate the MCID: (1) anchor-based and (2) distributionbased [13e18]. The anchor-based approach uses an external criterion and calculates the MCID by comparing the reported scores in distinct anchor-defined groups. Conversely, the distribution approach uses the underlying statistical property of a data set to establish the clinically meaningful change. Both approaches are frequently used in complement to ensure the robustness of the MCID estimates. Establishing the MCID for FACT-Cog is expected to aid in the assessment of the clinical significance of cognitive changes observed in clinical trials and sample size estimates

All the recruited patients were evaluated at baseline (T1) and at a follow-up point in time at least 3 months later (T2). This time frame was chosen as clinicians at our institution arrange follow-up consultations with stable ambulatory patients at least once every 3 months. In addition, observations suggest that these breast cancer patients may experience small but significant changes in their HRQoL and cognitive functioning after a span of 3 months [1,19]. Clinical data such as patient demographics, current medications for cancer treatment, and concurrent medical conditions were retrieved from the clinical database. Patient performance status was rated according to the Eastern Cooperative Oncology Group criteria [20]. At both T1 and T2, the patients completed two questionnaires: the FACT-Cog (version 3) [12] and the European Organization for Research and Treatment of Cancer Quality of Life Questionnaire Core 30 (EORTC-QLQ-C30) [21]. English and Chinese versions were available for all questionnaires, and they were administered to the Englishand Chinese-speaking groups, respectively. Completion of the questionnaires took approximately 20 minutes. 2.3. Instruments 2.3.1. FACT-Cog (version 3) This 37-item questionnaire is divided into six cognitive domains: memory, concentration, mental acuity, verbal fluency, functional interference, and multitasking ability. In addition, the tool includes two other subscales, namely ‘‘noticeability’’ (comments from others) and ‘‘effect of perceived cognitive impairment on quality of life.’’ The respondents indicate on a 5-point Likert scale ranging from 0, ‘‘never,’’

Y.T. Cheung et al. / Journal of Clinical Epidemiology

to 4, ‘‘several times a day,’’ the frequency of each occurrence over the 7 days preceding the test. In the subscales of perceived cognitive abilities and the effect of cognitive impairment on quality of life, the responses are rated on a 5-point severity scale ranging from 0, ‘‘not at all,’’ to 4, ‘‘very much.’’ The individual subscale scores are summed to determine the FACT-Cog total score, ranging from 0 to 148, with higher scores indicating better cognitive functioning. Both versions of FACT-Cog (English and Chinese) have been validated within the Asian breast cancer population; the internal consistencies within FACT-Cog and its cognitive domains were high (Cronbach a 5 0.707e0.929), and testeretest reliability was satisfactory for both versions (Intraclass correlation coefficient [ICC] 5 0.762 and 0.697, respectively) [22]. The measurement equivalence of the English and Chinese versions has been confirmed, supporting the aggregation of data for analyses [22]. 2.3.2. EORTC-QLQ-C30 This 30-item questionnaire is designed to evaluate HRQoL in cancer patients [21]. It consists of five functioning scales, nine individual symptom scales, and a single global health status/HRQoL scale. Of interest in this study was the cognitive functioning scale (EORTC-CF), consisting of two items that assess the cognitive domains of concentration and memory. Items are scored on a 4-point Likert scale with response categories ranging from ‘‘not at all’’ to ‘‘very much.’’ The transformed scores on the EORTC-CF range from 0 to 100, with higher scores indicating better perceived cognitive function. The English and Chinese versions of the EORTC-QLQ-C30 were previously validated in cancer patients in Singapore, with the majority of whom were breast cancer patients [23,24]. 2.4. Statistical analysis Descriptive statistics were used to characterize patient demographics, clinical variables, and score distributions for FACT-Cog and EORTC-CF. The scoring of the questionnaires was performed according to the manuals published by the respective developers. For the comparison of scores across T1 and T2, paired t-test and Wilcoxon signed-rank test were used for parametric and nonparametric data, respectively. For perspective, the floor and ceiling effects of FACT-Cog within the sample were examined by reporting the percentages of patients who scored 0 and 148 points on FACT-Cog, respectively. The internal consistencies of the overall FACT-Cog scale were also evaluated using Cronbach a. A Cronbach a of 0.7 and above is considered strong [25]. Anchor- and distribution-based approaches were used to obtain the MCID estimates [18,26e28]. 2.5. Anchor-based approach The EORTC-CF was used as an external criterion against which changes in FACT-Cog were anchored and calibrated.

-

(2014)

-

3

Spearman rank correlation coefficient was used to quantify the association between FACT-Cog and EORTC-CF. To establish an MCID, it has been recommended that the change scores of the anchor and the instrument being examined have a correlation threshold of r  j0.30j [28]. Change scores in the EORTC-CF could range from 100 to 100, with 13 possible change values across the range. The delineation of clinically distinct anchor-defined categories was based on a one-step change from baseline, corresponding to the smallest possible change score of 16.67. Each patient was assigned to one of these categories, accordingly: ‘‘much worse’’ (less than 16.67), ‘‘minimally worse’’ (16.67), ‘‘no change’’ (0), ‘‘minimally better’’ (16.67), or ‘‘much better’’ (O16.67). In one landmark study that evaluated the clinical significance of QLQ-C30 score changes, it was observed that for patients who indicated ‘‘a moderate’’ change for either better or worse, the mean change in scores was between 10 and 20 points [29]. Hence, we hypothesized that a 16.67 change on EORTC-CF is a valid estimate to capture a significant change of cognitive functioning status. To obtain the MCID for deterioration, the mean difference in scores for ‘‘no change’’ was subtracted from the mean difference in scores for ‘‘minimally worse.’’ Likewise, the average difference scores for patients classified as ‘‘minimally better’’ were compared with those for ‘‘no change’’ to compute the MCID estimate for improvement. The associated effect sizes (ESs) were determined for the respective clinical categories by dividing the mean change in scores by the overall baseline standard deviations (SDs) for the sample. To avoid overestimating the MCID, FACT-Cog change scores corresponding to the ‘‘much worse’’ and ‘‘much better’’ categories were not considered in the analysis. Another anchor-based method used involved a sensitivityand specificity-based analysis [15]. Receiver operating characteristic (ROC) curves were constructed, and the area under the curve (AUC) represents the ability of the instrument to distinguish between patients who did and did not undergo a clinically important change. The ROC cutoff point with the highest sensitivity and specificity corresponds to the optimal FACT-Cog change score that best discriminated between patients who changed and those who showed no change on the anchor. This ROC threshold value was then selected as the MCID estimate for FACT-Cog. 2.6. Distribution-based approach The magnitude of the MCID for the FACT-Cog total scores was estimated with ESs. An ES is a standardized index of change that represents the number of SDs by which the scores have changed from T1 to T2. Cohen developed benchmarks to facilitate the interpretations of ES [30]; a value of half SD was associated with a medium-sized effect and has been suggested as a default threshold for defining important patient-perceived change on HRQoL measures [31]. Based on previous MCID determination studies for other neuropsychological tools and FACIT instruments

4

Y.T. Cheung et al. / Journal of Clinical Epidemiology

(Functional Assessment of Chronic Illness Therapy), onethird and half SDs were considered likely approximations of the MCID [14,16,32e35]. Thus, one-third and half SDs were calculated at T1, T2, and for the T1eT2 score changes, and the mean values were computed. Additionally, the standard error of measurement (SEM)ebased criterion was used to provide an MCID estimate that exceeded the measurement imprecision of the instrument [36]. SEM was computed using the following formula: SEM 5 sO(1  rxx) [36,37], where s denotes the SD of the FACT-Cog scores and rxx is the testeretest reliability of the instrument. The value of rxx, 0.737, was obtained from a previous validation study for FACT-Cog [38]. A one-SEM standard has been shown to closely approximate the minimum clinically important intraindividual change [37]. The SEM was calculated for T1 and T2 scores and the mean computed. 2.7. Estimation of the overall MCID To account for the method- and sample-dependent variations, it is recommended that a plausible range of MCIDs be presented, rather than an absolute single threshold [39]. Thus, an MCID range was derived based on a combination of anchor- and distribution-based approaches. 2.8. Estimation of the domain MCID Because the EORTC-CF anchor only consists of two items assessing the domains of memory and concentration, each domain-specific item was used as an individual anchor to determine the MCIDs for the memory and concentration domains of the FACT-Cog. A one-step change, corresponding to a onepoint difference on the domain-specific item, was considered a minimal significant change. Patients were categorized into the respective anchor-defined categories accordingly.

-

(2014)

-

3. Results Three hundred thirty breast cancer patients participated at baseline (T1), of whom 220 (66.7%) completed the follow-up assessment (T2) and were included in the analysis (Table 1). Patients were lost to follow-up mainly because they had moved to another country or cancer institution or they had defaulted follow-up consultation. Overall, the mean age 6 SD was 50.9 6 9.5 years. Most of the patients were Chinese (79.6%) and married (73.2%). Approximately half of the patients were stage II patients (47.3%), postmenopausal (53.2%), and were receiving antihormonal therapy (45.5%). The mean duration from T1 to T2 was 9.7 6 6.8 months. More than half of the assessments (59.5%) were completed in English. 3.1. Questionnaire scoring The score distributions for the FACT-Cog and the EORTC-CF at both time points are summarized in Table 2. The mean total scores of the FACT-Cog at T1 and T2 were 127.6 6 18.1 and 119.0 6 23.3, respectively, with an average change score of 8.6 6 20.4 (P ! 0.001). For the EORTC-CF, the mean scores were 89.8 6 14.8 and 82.0 6 19.6 at T1 and T2, respectively. The mean change in the FACT-Cog total scores did not differ significantly, after stratifying by survey language (P 5 0.389). None of the patients scored 0 on the FACT-Cog for both time points. The lowest score obtained was 64 points (T1) and 54 points (T2). A total of 5.5% (T1) and 3.7% (T2) of the patients reported 148 points (perfect score) on the FACT-Cog. Internal consistencies within FACT-Cog were satisfactory at both time points (Cronbach a 5 0.840 and 0.832, respectively). 3.2. Anchor-based analysis

2.9. Exploratory analysis of baseline dependency of MCID To investigate whether estimates of MCID differed according to baseline clinical or demographic characteristics, patients were stratified according to their baseline perceived cognitive performance (using the median FACT-Cog score at T1 as cutoff to define high vs. low baseline scores), age (using the cutoff age of 60 years to define old vs. young), education status (elementary education and below vs. senior high school education and above), disease severity (stages I and II vs. stages III and IV), and type of chemotherapy treatment (anthracycline- vs. taxane based). Calculation of anchor-based MCID was performed within each stratum, provided the sample size of every anchordefined category within each stratum was more than 10 patients to allow for meaningful estimation of MCID. Statistical analyses were conducted using the SPSS software (IBM SPSS Statistics for Windows, Version 20.0, Armonk, NY). All the statistical tests were two tailed, with a significance level of 0.05.

There was a moderate correlation between the change scores of the FACT-Cog and the EORTC-CF (r 5 0.43; P ! 0.001). The EORTC-CFeanchored MCID estimates are listed in Table 3. The FACT-Cog was markedly more responsive to deterioration than improvement, as demonstrated by an ES of 0.66 for ‘‘minimally worse’’ vs. an ES of !0.20 for ‘‘minimally better.’’ These ESs represented moderate and negligible effects, respectively. The MCID for deterioration was actually statistically significant. A decrement of 9.6 points [95% confidence interval (CI): 4.4, 14.8] in the FACT-Cog total score corresponded to a minimal important change, defined in the EORTC-CF. For improvement, an increase of 2.2 points (95% CI: 5.8, 10.3) in the FACT-Cog total score corresponded to a minimal important improvement in patients’ cognitive functioning, but statistical significance was not observed between the adjacent categories of ‘‘minimally better’’ and ‘‘no change,’’ as well as ‘‘much better’’ and ‘‘no change.’’ Consequently, an ROC analysis was only performed to identify an MCID estimate for deterioration (Fig. 1). The

Y.T. Cheung et al. / Journal of Clinical Epidemiology Table 1. Patient characteristics (n 5 220) Characteristic Demographic variables Age (yr), mean 6 SD Age (range in yr) !30 31e40 41e50 51e60 O60 Education No formal education Elementary Junior high school Senior high school University/postuniversity degree Ethnicity Chinese Malay Indian Othera Marital status Single Married Divorced Widowed Clinical variables Breast cancer staging 0 I II III IV ECOG status 0 5 Fully ambulatory without symptoms 1 5 Fully ambulatory with symptoms Comorbidity Presence of comorbidity Types of comorbidityb Cardiovascular Gastrointestinal Endocrine Respiratory Others Receipt of antihormonal therapy Tamoxifen or aromatase inhibitors Complementary alternative medicine (CAM) Traditional medicines or health supplements Menopausal status Postmenopausal Chemotherapy protocol Anthracycline based (AC, FEC) Taxane based (DC) Hemoglobin level (g/dL), mean 6 SD Methodological variables Duration between T1 and T2 (mo), mean 6 SD Language of administration English Chinese

Frequency (%) 50.9 6 9.5 6 24 71 86 33

(2.7) (10.9) (32.3) (39.0) (15.1)

4 48 102 25 41

(1.8) (21.8) (46.4) (11.4) (18.6)

175 26 9 10

(79.6) (11.8) (4.1) (4.5)

43 161 9 7

(19.5) (73.2) (4.1) (3.2)

2 45 104 57 12

(0.9) (20.4) (47.3) (25.9) (5.5)

186 (84.5) 34 (15.5) 84 (38.2) 72 6 6 3 3

(85.7) (7.1) (7.1) (3.6) (3.6)

100 (45.5) 144 (65.5) 117 (53.2) 164 (74.5) 56 (25.5) 11.5 6 1.2 9.7 6 6.8 131 (59.5) 89 (40.5)

Abbreviations: SD, standard deviation; ECOG, Eastern Cooperative Oncology Group; AC, doxorubicin and cyclophosphamide; FEC, 5-fluorouracil, epirubicin, and cyclophosphamide; DC, docetaxel and cyclophosphamide. a ‘‘Other’’ races included Filipino, Burmese, and Arab participants. b Percentages add up to more than 100% because of patients having more than one comorbidity.

-

(2014)

-

5

AUC anchored by the EORTC-CF for all the FACT-Cog change scores was 0.75 (95% CI: 0.67, 0.82), suggesting that the FACT-Cog can fairly distinguish between patients who deteriorated and those who did not change on the EORTC-CF. The optimal MCID cutoff of 7.5 points for deterioration in perceived cognitive functioning on the FACT-Cog corresponded to a sensitivity of 75.6% and a specificity of 68.8%. 3.3. Distribution-based analysis The means for one-third and half SDs produced MCID estimates that ranged from 6.9 to 10.3 points (Table 4), respectively. The one-SEM criterion resulted in an MCID estimate of 10.6 points for the FACT-Cog total score. 3.4. Estimation of the overall MCID Collectively, the MCID estimates established using different approaches suggest that a decrease of 6.9e10.6 points (4.7e7.2% of the total score) in the FACT-Cog corresponds to the threshold for clinically significant cognitive deterioration in breast cancer patients. Dividing the MCID estimates by the number of items in the FACT-Cog yields an average of 0.19e0.29 points per item. 3.5. Estimation of the domain MCID The association between the change scores in the FACTCog memory domain and the memory-specific EORTC-CF anchor was moderate (r 5 0.41; P ! 0.001), whereas for the FACT-Cog concentration domain, the observed change correlation was relatively weak (r 5 0.27; P ! 0.001). Anchoring the deterioration scores of the FACT-Cog memory domain against the corresponding items in the EORTC-CF yields an MCID estimate of 3.2 points (95% CI: 1.9, 4.5). For the FACT-Cog concentration domain, the threshold result was 1.8 points (95% CI: 0.7, 2.9). These MCID values translated to 11.4% and 11.3% of the domain ranges for the worsening of the memory and concentration domains, respectively. 3.6. Exploratory analysis of the baseline dependency of MCID When patients were stratified on the basis of their initial FACT-Cog scores (median cutoff point of 132), the threshold of clinically meaningful deterioration for patients with lower baseline scores (N 5 110) was estimated as 12.0 points (95% CI: 2.9, 21.1). In contrast, patients who reported higher cognitive functioning at baseline (N 5 110) required a smaller degree of deterioration to perceive that change as important, with an MCID estimate of 9.7 points (95% CI: 4.4, 15.1). However, because of the skewed data and limited sample size within anchor-defined categories for the different stratified subgroups of age, disease severity, and education status,

6

Y.T. Cheung et al. / Journal of Clinical Epidemiology

-

(2014)

-

Table 2. Summary statistics for FACT-Cog and EORTC-CF (n 5 220) Baseline (T1) Mean ± SD

Instrument scales (theoretical range) FACT-Cog domains/subscales Memory (0-24) Concentration (0-16) Mental acuity (0-16) Verbal fluency (0-24) Functional interference (0-16) Multitasking ability (0-14) Noticeability (0-16) Impact on quality of life (0-16) FACT-Cog total (0-148) EORTC-CF (0-100)

23.7 13.7 13.6 21.3 13.7 13.1 15.3 13.4 127.6 89.8

6 6 6 6 6 6 6 6 6 6

3.9 2.7 2.6 3.2 2.4 2.6 1.7 3.7 18.1 14.8

Follow-up (T2)

Median

Q1

Q3

25.0 14.0 14.0 22.0 14.0 14.0 16.0 15.0 132.0 100

21.0 12.0 12.0 20.0 13.0 11.3 15.0 12.0 120.0 83.3

27.0 16.0 16.0 23.8 15.0 15.0 16.0 16.0 141.0 100.0

Mean ± SD 21.9 12.6 12.3 19.9 13.1 12.0 14.9 12.5 119.0 82.0

6 6 6 6 6 6 6 6 6 6

4.7 3.1 3.2 4.1 2.8 3.4 2.1 3.9 23.3 19.6

Average change

Median

Q1

Q3

23.0 13.5 13.0 21.0 14.0 12.5 16.0 14.0 125.0 83.3

19.0 10.0 10.0 18.0 11.3 10.0 15.0 10.0 105.3 66.7

26.0 15.0 15.0 23.0 15.0 14.0 16.0 16.0 138.0 100

Mean ± SD 1.8 1.1 1.3 1.3 0.6 1.1 0.4 0.9 8.6 7.7

6 6 6 6 6 6 6 6 6 6

4.5a 3.2a 3.1a 3.7a 2.4a 3.2a 2.1b 4.4b 20.4a 18.5a

Abbreviations: FACT-Cog, Functional Assessment of Cancer TherapyeCognitive Function; EORTC-CF, European Organization for Research and Treatment of Cancer Quality of Life Questionnaire Core 30eCognitive Functioning scale; SD, standard deviation; Q1, lower quartile; Q3, upper quartile. a Denotes change at a statistical significance level of P ! 0.001. b Denotes change at a statistical significance level of P ! 0.01.

the decline in scores is clinically relevant. Concerning this research interest, one potential application of the MCID is to help investigators assess the clinical significance of the score differences observed between groups receiving different types or doses of chemotherapy regimens. Using the established MCID estimates, results in terms of the proportion of patients experiencing clinically meaningful deterioration in each treatment arm can be meaningfully interpreted. Furthermore, knowledge of the MCID can also be used to guide sample size calculations in future trials. In designing a clinical trial, investigators may use a known MCID to estimate the minimum number of subjects needed in each study arm to detect a clinically significant change in cognition. The anchor- and distribution-based approaches complement one another because they each possess advantages and disadvantages [26]. Basing the MCID entirely on distributional approaches has been criticized as reflecting little clinical relevance in the change [15,26,27]. Thus, it is recommended that an anchor-based estimate be given greater emphasis when determining the MCID because it is directly linked to patients’ perceptions of change [17]. It was observed that the MCID estimates derived from half SD and one SEM were in close agreement with the EORTCCFeanchored MCID, supporting the two distributional criteria to approximate the minimal threshold of clinically

it was not possible to compare the resultant MCID estimates based on these clinical/demographic factors.

4. Discussion To the best of our knowledge, this is the first reported study to establish an MCID for the FACT-Cog. Our findings suggest that a reduction of 6.9e10.6 points on the FACTCog is the smallest difference that constitutes clinically meaningful deterioration. That is, when the FACT-Cog change scores between two assessments reach 4.7e7.2% of the instrument range, the change can be considered clinically significant. No MCID estimate was established for cognitive improvement because the associated ES was too small (ES ! 0.2) to merit consideration, and the number of patients who demonstrated cognitive functioning improvement was too small in this study. Moreover, the outcome of clinical interest lies in the trajectory of canceror chemotherapy-associated cognitive deterioration and the resulting effect on cancer patients’ quality of life. The primary significance of establishing an MCID for the FACT-Cog is to enhance the clinical interpretability of patient-reported cognitive changes. In addition to achieving statistical rigor, the outcome that is more practical in trials that evaluate cognitive end points is whether

Table 3. Anchor-based (EORTC-CF) MCID estimates EORTC-CF category Much better Minimally better No change Minimally worse Much worse

n 8 20 109 56 27

Mean change ± SD 0.3 0.1 2.3 11.9 15.8

6 6 6 6 6

13.2 17.8 16.5 14.8 16.6

MCID (95% CI)

ES

3.4 (2.9, 11.2) 2.2 (5.8, 10.3)

0.006 0.003 0.127 0.658 0.897

9.6 (14.8, 4.4)a 15.8 (19.7, 6.5)a

Abbreviations: EORTC-CF, European Organization for Research and Treatment of Cancer Quality of Life Questionnaire Core 30eCognitive Functioning scale; n, sample size; SD, standard deviation; MCID, minimal clinically important difference; CI, confidence interval; ES, effect size. a Denotes that the mean change of FACT-Cog scores at statistically significance level of P ! 0.01.

Y.T. Cheung et al. / Journal of Clinical Epidemiology

-

(2014)

-

7

Table 4. Distribution-based MCID estimates Distribution-based criteria Total FACT-Cog score

SD

One-third SD

Half SD

1 SEM

Baseline (T1) Follow-up (T2) Baseline to follow-up Mean

18.1 23.3 20.4 20.6

6.0 7.8 6.8 6.9

9.1 11.6 10.2 10.3

9.3 11.9 10.6

Abbreviations: MCID, minimal clinically important difference; FACT-Cog, Functional Assessment of Cancer TherapyeCognitive Function; SD, standard deviation; SEM, standard error of measurement. Bold text denotes the mean values used to calculate the overall MCID estimates from the distribution-based approach.

Fig. 1. Receiver operating characteristic curve for FACT-Cog change scores anchored by EORTC-CF; area under the curve 5 0.75. FACTCog, Functional Assessment of Cancer TherapyeCognitive Function; EORTC-CF, European Organization for Research and Treatment of Cancer Quality of Life Questionnaire Core 30eCognitive Functioning scale.

relevant change. The results also present important evidence demonstrating the responsiveness of the FACT-Cog to cognitive deterioration. The ES associated with ‘‘no change’’ (ES 5 0.13), ‘‘minimally worse’’ (ES 5 0.66), and ‘‘much worse’’ (ES 5 1.79) corresponded to negligible, moderate, and large effects, respectively, indicating that the magnitudes of mean change scores were in the expected direction. The findings of previous studies illustrate the stability of MCID estimates for FACIT scales and subscales across patient populations [40]. This consistency supports devising general guidelines for estimating MCIDs for other FACIT instruments. A rule of thumb for the MCID of the FACIT total scale is 0.15e0.25 points per item and 4e6% of the instrument scale breadth [40]. The MCID range identified in this study equates to 0.19e0.29 points per item and 4.7e7.2% of the instrument range, consistent with the guidelines. The FACT-Cog MCID range is also in line with a percentage change of 5e10% in the EORTC-QLQ-C30 scales, which was proposed as a minimal clinically significant change [29,41]. A remarkable convergence of MCID estimates on half SD or 6e7% of the instrument range across a variety of disease conditions and HRQoL measures was observed [42]. In common with these findings, the EORTC-CFeanchored MCID was 6.5% of the total FACT-Cog score, further supporting the validity of our estimates. The degree of concordance between our results and the work of others is consistent with the notion that patients may be able to recognize a constant change using different instruments [43].

We also attempted to determine the MCID for the individual cognitive domains of the FACT-Cog. It is important to note that both the MCID estimates for the memory and concentration domains were approximately 10% of the domain range, suggesting that this threshold of minimally significant difference may be extrapolated to other cognitive domains. The identification of domain MCIDs enables the interpretation of FACT-Cog score differences pertinent to individual cognitive domains, which may be useful in trials targeting the examination of specific cognitive domains. Available literature has asserted the presence of asymmetry between improvement and deterioration in patientreported outcomes [43e47]. In particular, among patients with cancer, it was observed that MCID estimates for deterioration were larger than those for improvement across the subscales of the FACIT instruments [43,44,47]. Although cognitive improvement did not achieve statistical significance in our study, the results showed similar tendency, suggesting that the meaningfulness of change is dependent on the direction of change. For a comparable EORTC-CF level, a larger degree of worsening is required to constitute clinical relevance than that of improvement. One explanation that has been postulated concerning the difference in meaning patients placed on change is associated with the response shift in individuals’ perceptions of change [44]. The challenging ordeal of undergoing cancer therapy may have led patients to readjust their expectancies of perceived HRQoL in general. As observed in our previous focus group study, many breast cancer patients have adopted self-help or compensatory strategies to cope with their cognitive changes [1], thereby attenuating the impact of perceived cognitive deterioration on their HRQoL. Taken together, we reason that this change in patients’ internal reference standards over time may have caused them to adaptively downplay the perceived impact of cognitive changes. Baseline demographic and clinical characteristics might contribute to different threshold of perceived changes [46,48]. Because of the limited sample size, stratification within the variables such as age, disease severity, fatigue, and global health status was restricted. However, our exploratory analysis suggested that the baseline status may be relevant in ascertaining clinically meaningful change. That is, an outcome of ‘‘no change’’ may carry different significance

8

Y.T. Cheung et al. / Journal of Clinical Epidemiology

depending on where the patients were positioned along the continuum of cognitive functioning (low or high) at baseline. In comparison with patients with higher perceived cognitive functioning, patients who reported greater cognitive disturbances at baseline had to register a larger degree of deterioration to be labeled for experiencing an important change. This finding sheds light on the context-specific nature of meaningful change, suggesting that certain groups of patients may be more sensitive to deterioration in perceived cognition over time. In light of this, it is reasonable to expect that the threshold of change may shift according to other differences of characteristics such as age, disease severity, and global health status. This study has some limitations. The EORTC-CF consists of only two domain-specific items, regarding memory and concentration. Furthermore, there was only modest correlation between the change scores of the FACT-Cog and the EORTC-CF (r 5 0.43). Patients’ self-evaluation of cognition may also have been guided by their preexisting knowledge of chemotherapy-associated cognitive changes [49], resulting in an overestimation of the cognitive deterioration they actually experienced. Although objective neuropsychological instruments may provide readily interpretable scores, strong evidence points to the lack of a correlation between subjective and objective cognitive dysfunction [9,50,51], and this precludes objective measures as useful anchors for considering the possibility of underestimating the MCID. Thus, the psychometrically validated EORTC-CF was selected to prospectively measure the change in patients’ perceived cognitive status. Despite these limitations, the anchor is deemed credible because these domains are clinically tied to the end point being evaluated. Some studies compare scores of the construct of interest to the patients’ answers with another subjective assessment for the purpose of evaluating patient-reported outcomes, typically a global assessment rating in which the patients rate the status of that particular construct of interest as ‘‘better,’’ ‘‘unchanged,’’ or ‘‘worse’’ [27,28,52]. Although this is a recommended approach by the US Food and Drug Administration [52], a global rating assessment was not adopted in this study because based on observations from our previous studies, it was found that Asian breast cancer patients had difficulty comprehending what constitutes ‘‘cognitive functioning’’ [1]. Chinese-speaking patients, in particular, had difficulties comprehending the terms ‘‘ren zhi gong neng’’ (the Chinese equivalent of ‘‘cognitive function’’). Having considered this observation, we decided that the use of global ratings is not the ideal within the context of our patient population. Moreover, global rating assessment is also limited by its unknown validity and reliability [27], and patients’ judgment of change in that particular construct (in this case, cognitive functioning) may be heavily influenced by the perception of other health status (such as physical functioning, pain, and emotional functioning) at the follow-up point in time. However, it is recommended that for future studies conducted on other cancer populations

-

(2014)

-

or ethnic groups, the use of a global rating assessment can be adopted. Given that existing evidence in the literature has established the importance of cultural adaptation and considerations in the interpretation of patient-reported outcomes [1,53,54], the MCID identified in this study might not be applicable to cancer patients of other ethnic groups. We also acknowledge that the time frame of assessment varied among patients, ranging from 3 to 22 months. However, we did not notice a clear trend in the magnitude and direction of score changes, with respect to time passed from baseline. A minimum duration of 3 months between assessments was considered suitable to quantify a subtle yet important change. Lastly, the considerable dropout rate of 30% might add a potential selection bias to this study. However, a retrospective analysis of the causes of patients not completing the questionnaires at T2 show that they were mainly not disease- or treatment related; the baseline characteristics of the respondents and nonrespondents to follow-up were fairly similar. 5. Conclusions A 6.9- to 10.6-point reduction in the FACT-Cog score was established as the smallest meaningful self-reported cognitive deterioration. We believe that the identified estimates are clinically and methodologically valid because they were derived using a combination of approaches. A reasonable range for the MCID is provided, and the estimates can be applied as a yardstick to aid in the interpretation of clinical relevance in patient-reported cognitive changes and sample size estimates for future studies. These estimates of meaningful difference will also provide further insights into the effect that cancer patients’ subjective cognitive deterioration has on their quality of life. These current estimates should be further validated and refined by the use of larger data sets and other recommended methodological approaches and on different patient groups and ethnic populations.

Acknowledgments The authors acknowledge the contributions of all the study participants. The authors also thank Mr. Yuan Chuan Kee (research assistant) and Mr. Si Rong Lim, Ms. Yi Fan Zhang, and Mr. Desmond Tan (undergraduate students from the Department of Pharmacy, NUS) for their assistance in data collection. The simplified Chinese version of the FACT-Cog is available at http://www.facit.org/FACITOrg/Questionnaires. References [1] Cheung YT, Shwe M, Tan YP, Fan G, Ng R, Chan A. Cognitive changes in multiethnic Asian breast cancer patients: a focus group study. Ann Oncol 2012;23:2547e52. [2] Collins B, Mackenzie J, Tasca GA, Scherling C, Smith A. Cognitive effects of chemotherapy in breast cancer patients: a dose-response study. Psychooncology 2013;22:1517e27.

Y.T. Cheung et al. / Journal of Clinical Epidemiology [3] Mitchell T, Turton P. ‘Chemobrain’: concentration and memory effects in people receiving chemotherapyda descriptive phenomenological study. Eur J Cancer Care (Engl) 2011;20:539e48. [4] Von Ah D, Habermann B, Carpenter JS, Schneider BL. Impact of perceived cognitive impairment in breast cancer survivors. Eur J Oncol Nurs 2013;17:236e41. [5] Shilling V, Jenkins V. Self-reported cognitive problems in women receiving adjuvant therapy for breast cancer. Eur J Oncol Nurs 2007;11:6e15. [6] Vardy J. Cognitive function in breast cancer survivors. Cancer Treat Res 2009;151:387e419. [7] Ando-Tanabe N, Iwamitsu Y, Kuranami M, Okazaki S, Yasuda H, Nakatani Y, et al. Cognitive function in women with breast cancer receiving adjuvant chemotherapy and healthy controls. Breast Cancer 2012:http://dx.doi.org/10.1007/s12282-012-0405-7. [Epub ahead of print]. [8] Wefel JS, Schagen SB. Chemotherapy-related cognitive dysfunction. Curr Neurol Neurosci Rep 2012;12:267e75. [9] Pullens MJ, De Vries J, Roukema JA. Subjective cognitive dysfunction in breast cancer patients: a systematic review. Psychooncology 2010;19:1127e38. [10] Cheung YT, Chui WK, Chan A. Neuro-cognitive impairment in breast cancer patients: pharmacological considerations. Crit Rev Oncol Hematol 2012;83:99e111. [11] Munir F, Burrows J, Yarker J, Kalawsky K, Bains M. Women’s perceptions of chemotherapy-induced cognitive side affects on work ability: a focus group study. J Clin Nurs 2010;19:1362e70. [12] Wagner LI, Sweet J, Butt Z, Lai J-S, Cella D. Measuring patient self-reported cognitive function: development of the functional assessment of cancer therapy-cognitive function instrument. J Support Oncol 2009;7:W32e9. [13] Jaeschke R, Singer J, Guyatt GH. Measurement of health status. Ascertaining the minimal clinically important difference. Control Clin Trials 1989;10:407e15. [14] Cella D, Eton DT, Lai JS, Peterman AH, Merkel DE. Combining anchor and distribution-based methods to derive minimal clinically important differences on the Functional Assessment of Cancer Therapy (FACT) anemia and fatigue scales. J Pain Symptom Manage 2002;24:547e61. [15] de Vet HC, Ostelo RW, Terwee CB, van der Roer N, Knol DL, Beckerman H, et al. Minimally important change determined by a visual method integrating an anchor-based and a distribution-based approach. Qual Life Res 2007;16:131e42. [16] Eton DT, Cella D, Yost KJ, Yount SE, Peterman AH, Neuberg DS, et al. A combination of distribution- and anchor-based approaches determined minimally important differences (MIDs) for four endpoints in a breast cancer scale. J Clin Epidemiol 2004;57: 898e910. [17] Revicki DA, Cella D, Hays RD, Sloan JA, Lenderking WR, Aaronson NK. Responsiveness and minimal important differences for patient reported outcomes. Health Qual Life Outcomes 2006; 4:70. [18] Wells G, Beaton D, Shea B, Boers M, Simon L, Strand V, et al. Minimal clinically important differences: review of methods. J Rheumatol 2001;28:406e12. [19] Cheung YT, Shwe M, Kee YC, Chui WK, Tan YP, Yong WS, et al. Post-chemotherapy cognitive changes in Asian breast cancer patients: a prospective cohort Study. Support Care Cancer 2013;21:S209. [20] Oken MM, Creech RH, Tormey DC, Horton J, Davis TE, McFadden ET, et al. Toxicity and response criteria of the Eastern Cooperative Oncology Group. Am J Clin Oncol 1982;5:649e55. [21] Aaronson NK, Ahmedzai S, Bergman B, Bullinger M, Cull A, Duez NJ, et al. The European Organization for Research and Treatment of Cancer QLQ-C30: a quality-of-life instrument for use in international clinical trials in oncology. J Natl Cancer Inst 1993;85: 365e76.

-

(2014)

-

9

[22] Cheung YT, Lim SR, Shwe M, Tan YP, Chan A. Psychometric properties and measurement equivalence of the English and Chinese versions of the functional assessment of cancer therapycognitive in Asian patients with breast cancer. Value Health 2013;16:1001e13. [23] Luo N, Fones CS, Lim SE, Xie F, Thumboo J, Li SC. The European Organization for Research and Treatment of Cancer Quality of Life Questionnaire (EORTC QLQ-c30): validation of English version in Singapore. Qual Life Res 2005;14:1181e6. [24] Cheung YB, Thumboo J, Goh C, Khoo KS, Che W, Wee J. The equivalence and difference between the English and Chinese versions of two major, cancer-specific, health-related quality-of-life questionnaires. Cancer 2004;101:2874e80. [25] Cronbach LJ. Coefficient alpha and the internal structure of tests. Psychometrika 1951;16:297e334. [26] Crosby RD, Kolotkin RL, Williams GR. Defining clinically meaningful change in health-related quality of life. J Clin Epidemiol 2003;56:395e407. [27] Copay AG, Subach BR, Glassman SD, Polly DW Jr, Schuler TC. Understanding the minimum clinically important difference: a review of concepts and methods. Spine J 2007;7:541e6. [28] Revicki D, Hays RD, Cella D, Sloan J. Recommended methods for determining responsiveness and minimally important differences for patient-reported outcomes. J Clin Epidemiol 2008;61: 102e9. [29] Osoba D, Rodrigues G, Myles J, Zee B, Pater J. Interpreting the significance of changes in health-related quality-of-life scores. J Clin Oncol 1998;16:139e44. [30] Cohen J. Statistical power analysis for the behavioral sciences. 2nd ed. Hillsdale, NJ: L. Erlbaum Associates; 1988. [31] Norman GR, Sloan JA, Wyrwich KW. Interpretation of changes in health-related quality of life: the remarkable universality of half a standard deviation. Med Care 2003;41:582e92. [32] Yost KJ, Cella D, Chawla A, Holmgren E, Eton DT, Ayanian JZ, et al. Minimally important differences were estimated for the Functional Assessment of Cancer Therapy-Colorectal (FACT-C) instrument using a combination of distribution- and anchor-based approaches. J Clin Epidemiol 2005;58:1241e51. [33] Cella D, Eton DT, Fairclough DL, Bonomi P, Heyes AE, Silberman C, et al. What is a clinically meaningful change on the Functional Assessment of Cancer Therapy-Lung (FACT-L) questionnaire? Results from Eastern Cooperative Oncology Group (ECOG) study 5592. J Clin Epidemiol 2002;55:285e95. [34] Maringwa J, Quinten C, King M, Ringash J, Osoba D, Coens C, et al. Minimal clinically meaningful differences for the EORTC QLQ-C30 and EORTC QLQ-BN20 scales in brain cancer patients. Ann Oncol 2011;22:2107e12. [35] Howard R, Phillips P, Johnson T, O’Brien J, Sheehan B, Lindesay J, et al. Determining the minimum clinically important differences for outcomes in the DOMINO trial. Int J Geriatr Psychiatry 2011;26: 812e7. [36] Wyrwich KW, Nienaber NA, Tierney WM, Wolinsky FD. Linking clinical relevance and statistical significance in evaluating intraindividual changes in health-related quality of life. Med Care 1999;37:469e78. [37] Wyrwich KW, Tierney WM, Wolinsky FD. Further evidence supporting an SEM-based criterion for identifying meaningful intraindividual changes in health-related quality of life. J Clin Epidemiol 1999;52:861e73. [38] Cheung YT, Maung Shwe HG, Chan A. Cognitive functioningrelated quality of life in breast cancer patients: validation of the Functional Assessment of Cancer Therapy-Cognitive (FACT-Cog). Support Care Cancer 2012;20:S140. [39] Hays RD, Woolley JM. The concept of clinically meaningful difference in health-related quality-of-life research. How meaningful is it? Pharmacoeconomics 2000;18:419e23.

10

Y.T. Cheung et al. / Journal of Clinical Epidemiology

[40] Yost KJ, Eton DT. Combining distribution- and anchor-based approaches to determine minimally important differences: the FACIT experience. Eval Health Prof 2005;28:172e91. [41] King MT. The interpretation of scores from the EORTC quality of life questionnaire QLQ-C30. Qual Life Res 1996;5:555e67. [42] Osoba D, King M. Meaningful differences. In: Fayers PM, Hays RD, editors. Assessing quality of life in clinical trials. New York, NY: Oxford University Press Inc.; 2005:243e57. [43] Ringash J, O’Sullivan B, Bezjak A, Redelmeier DA. Interpreting clinically significant changes in patient-reported outcomes. Cancer 2007;110:196e202. [44] Cella D, Hahn EA, Dineen K. Meaningful change in cancer-specific quality of life scores: differences between improvement and worsening. Qual Life Res 2002;11:207e21. [45] Cocks K, King MT, Velikova G, de Castro G Jr, Martyn St-James M, Fayers PM, et al. Evidence-based guidelines for interpreting change scores for the European Organisation for the Research and Treatment of Cancer Quality of Life Questionnaire Core 30. Eur J Cancer 2012; 48:1713e21. [46] Adamchic I, Tass PA, Langguth B, Hauptmann C, Koller M, Schecklmann M, et al. Linking the Tinnitus Questionnaire and the subjective Clinical Global Impression: which differences are clinically important? Health Qual Life Outcomes 2012;10:79. [47] Ringash J, Bezjak A, O’Sullivan B, Redelmeier DA. Interpreting differences in quality of life: the FACT-H&N in laryngeal cancer patients. Qual Life Res 2004;13:725e33.

-

(2014)

-

[48] Crosby RD, Kolotkin RL, Williams GR. An integrated method to determine meaningful changes in health-related quality of life. J Clin Epidemiol 2004;57:1153e60. [49] Schagen SB, Das E, van Dam FS. The influence of priming and preexisting knowledge of chemotherapy-associated cognitive complaints on the reporting of such complaints in breast cancer patients. Psychooncology 2009;18:674e8. [50] Joly F, Lange M, Rigal O, Correia H, Giffard B, Beaumont JL, et al. French version of the Functional Assessment of Cancer TherapyCognitive Function (FACT-Cog) version 3. Support Care Cancer 2012;20:3297e305. [51] Jacobs SR, Jacobsen PB, Booth-Jones M, Wagner LI, Anasetti C. Evaluation of the functional assessment of cancer therapy cognitive scale with hematopoietic stem cell transplant patients. J Pain Symptom Manage 2007;33:13e23. [52] McLeod LD, Coon CD, Martin SA, Fehnel SE, Hays RD. Interpreting patient-reported outcome results: US FDA guidance and emerging methods. Expert Rev Pharmacoecon Outcomes Res 2011;11:163e9. [53] Wild D, Grove A, Martin M, Eremenco S, McElroy S, VerjeeLorenz A, et al. Principles of good practice for the translation and cultural adaptation process for patient-reported outcomes (PRO) measures: report of the ISPOR task force for translation and cultural adaptation. Value Health 2005;8:94e104. [54] Ketay S, Aron A, Hedden T. Culture and attention: evidence from brain and behavior. Prog Brain Res 2009;178:79e92.