Dement Geriatr Cogn Disord 2014;37:223–231 DOI: 10.1159/000353541 Accepted after revision: May 21, 2013 Published online: November 2, 2013
© 2013 S. Karger AG, Basel 1420–8008/14/0374–0223$39.50/0 www.karger.com/dem
Original Research Article
Validation of the Chinese Version of Addenbrooke’s Cognitive ExaminationRevised for Screening Mild Alzheimer’s Disease and Mild Cognitive Impairment Rong Fang a Gang Wang a Yue Huang c Jun-Peng Zhuang a Hui-Dong Tang a Ying Wang a Yu-Lei Deng a Wei Xu a Sheng-Di Chen a, b Ru-Jing Ren a a
Department of Neurology and Institute of Neurology, Ruijin Hospital, Shanghai Jiao Tong University School of Medicine, and b Laboratory of Neurodegenerative Diseases, Institute of Health Science, Shanghai Institutes of Biological Sciences, Chinese Academy of Science and Shanghai Jiao Tong University School of Medicine, Shanghai, PR China; c Neuroscience Research Australia and the University of New South Wales, Randwick, N.S.W., Australia
Key Words Alzheimer’s disease · Addenbrooke’s Cognitive Examination-Revised · Mild cognitive impairment · Mini-Mental State Examination · Chinese · Screening Abstract Background/Aims: As a suitable test to screen for Alzheimer’s disease (AD) or mild cognitive impairment (MCI), studies to validate the Chinese version of Addenbrooke’s Cognitive Examination-Revised (ACE-R) are rare. Methods: A total of 151 subjects were recruited and the neuropsychological assessments were employed. One-way analysis of variance and Bonferroni correction were used to compare scores of different psychometric scales. Intraclass correlation coefficient (ICC) and Cronbach’s coefficient α were used to evaluate the reliability of psychometric scales. The validity of ACE-R to screen for mild AD and amnestic subtype of MCI (a-MCI) was assessed by receiver operating characteristic (ROC) curves. Results: The Chinese ACE-R had good reliability (inter-rater ICC = 0.994; test-retest ICC = 0.967) as well as reliable internal consistency (Cronbach’s coefficient α = 0.859). With its cutoff of 67/68, the sensitivity (0.920) and specificity (0.857) were lower than for the Mini-Mental State Examination (MMSE) cutoff (sensitivity 1.000 and specificity 0.937) to screen for mild AD. However, the sensitivity of ACE-R to screen for a-MCI was superior to the MMSE with a cutoff of 85/86. The specificity
Sheng-Di Chen and Ru-Jing Ren Department of Neurology and Institute of Neurology Ruijin Hospital, Shanghai Jiao Tong University School of Medicine Shanghai 200025 (PR China) E-Mail chen_sd @ medmail.com.cn; doctorren2001 @ 126.com
Downloaded by: Univ. of California San Diego 132.239.1.231 - 6/4/2015 4:34:02 AM
S.-D.C. and R.-J.R. contributed equally to this work.
224
Dement Geriatr Cogn Disord 2014;37:223–231 DOI: 10.1159/000353541
© 2013 S. Karger AG, Basel www.karger.com/dem
Fang et al.: Validation of the Chinese Version of Addenbrooke’s Cognitive ExaminationRevised for Screening Mild Alzheimer’s Disease and Mild Cognitive Impairment
of ACE-R was lower than that of the MMSE to screen for a-MCI. The area under the ROC curve of ACE-R was much larger than that of the MMSE (0.836 and 0.751) for detecting a-MCI rather than mild AD. Conclusion: The Chinese ACE-R is a reliable assessment tool for cognitive impairment. It is more sensitive and accurate in screening for a-MCI rather than for AD compared to the MMSE. © 2013 S. Karger AG, Basel
Introduction
With an aging population, the number of patients with Alzheimer’s disease (AD) and other cognitive disorders is growing dramatically [1]. The concept of mild cognitive impairment (MCI) was first proposed by Reisberg and his colleagues in 1988 [2], and Petersen et al. [3] advanced its diagnostic criteria in 1999. Petersen and Morris [4] further divided MCI into 4 subtypes [amnestic MCI-single domain (sa-MCI), amnestic MCI-multiple domains (ma-MCI), nonamnestic MCI-single domain (sna-MCI), and nonamnestic MCI-multiple domains (mna-MCI)]. Sa-MCI and ma-MCI could be combined into amnestic subtype of MCI (a-MCI), which correlates more with AD, as they are characterized predominantly by memory impairment [5]. Previous studies showed that the annual conversion rate from MCI to AD was 10–15%, and approximately 50% of MCI patients will convert to AD within 4 years [6]. Therefore, MCI is considered to be a transition phase from normal aging to AD [7]. Yet, the lack of relatively accurate screening tests for detecting early-stage AD and MCI impedes early treatment. Hence, great efforts have been made to achieve an early diagnosis of AD and associated cognitive disorders. Due to the high conversion rate to AD [8] and the lack of effective tools to identify MCI in a normal population, it has been a hot topic to identify appropriate neuropsychological screening tests with a high accuracy in detecting a-MCI. Similarly, a screening test with a high sensitivity and specificity for mild AD is also required. As the most widely used screening tool, the Mini-Mental State Examination (MMSE) has significant shortcomings, including weak executive, visuospatial, and language items as well as ceiling effects without enough sensitivity to detect MCI or mild AD. These limitations are more obvious when it is applied to individuals with a high level of education. Addenbrooke’s Cognitive Examination-Revised (ACE-R) is a brief cognitive test revised by Mioshi et al. [9] in 2006, and, by contrast, recent series of studies recommended ACE-R as a superior tool for screening for early dementia and a-MCI in normal populations as well as for discriminating different types of dementias [10, 11]. At present, ACE-R has been translated into many languages and has been validated in different clinical sites in the UK, Australia, Spain, Greece, Germany, Italy, Portugal, Japan, and Korea [5, 12–20]. However, the psychometric properties of the Chinese version have not yet been examined. Therefore, our aim is to investigate the reliability and validity of the Chinese ACE-R and to discuss the optimal screening cutoff for mild AD as well as a-MCI in the Chinese population.
Participants and Assessment Procedure A total of 151 individuals were enrolled at the Memory Clinic of the Department of Neurology, Ruijin Hospital Affiliated to Shanghai Jiao Tong University School of Medicine, from December 2010 to March 2012. The sample was divided into three groups – mild AD group (n = 25), a-MCI group (n = 75), and normal control group (n = 51) – according to the patients’ clinical manifestations, neuropsychological tests, laboratory tests, and imaging examinations, conducted collaboratively by two neurologists. The diagnostic criteria for each group were the following:
Downloaded by: Univ. of California San Diego 132.239.1.231 - 6/4/2015 4:34:02 AM
Materials and Methods
225
Dement Geriatr Cogn Disord 2014;37:223–231 DOI: 10.1159/000353541
© 2013 S. Karger AG, Basel
Fang et al.: Validation of the Chinese Version of Addenbrooke’s Cognitive ExaminationRevised for Screening Mild Alzheimer’s Disease and Mild Cognitive Impairment
Neuropsychological Assessment The following neuropsychological tests were used in this study: (1) The Chinese ACE-R. The scale was translated by Dr. Yue Huang and administration documents were translated by Dr. Gang Wang and Dr. Shengdi Chen. There are 16 cognitive items divided into 5 domains in ACE-R, including orientation, registration, attention and concentration, memory (recall, anterograde memory, retrograde memory, recognition), verbal fluency, language (comprehension, writing, repetition, naming, comprehension, reading), visuospatial abilities, and perceptual abilities. The total score is 100 points allocated to the 5 cognitive domains separately as follows: 18 points for attention and orientation, 26 points for memory, 14 points for fluency, 26 points for language, and 16 points for visuospatial abilities. A higher score indicates better cognitive function. Some items of the original ACE-R were adapted for a better understanding during administration within the Chinese population. We replaced ‘lemon’ with ‘orange’ in the item of 3 words for registration and recall. The name and address in anterograde memory, recall, and recognition was replaced by a Chinese name and address. The last question in retrograde memory was replaced by ‘who is the only female emperor in Chinese history?’ We asked subjects to speak as many words as possible with the Chinese character ‘che’ during 1 min instead of the letter ‘P’ in the item for verbal fluency. All the words and sentences were replaced by Chinese phrases in the repetition item. The comprehension part of the domain ‘language’ has 4 questions, the first one of which was replaced by ‘which thing is used for filling with water?’ The second question was revised into ‘which animal lives in Australia?’ Finally, we asked the patients to read out 8 Chinese characters instead of 5 English words in the reading item. (2) CDR [23–26]. This is a semi-structured scale, which involves 2 separate interviews with the subjects and with the relatives/caregivers of the subject. It has 6 items including memory, orientation, the abilities of judgment and solving problems, social business, housework and hobbies, and self-care abilities. Each item has 5 possible grades (0, 0.5, 1, 2, and 3). The scoring of the CDR is: 0 for normal cognition, 0.5 for very mild dementia, 1 for mild dementia, 2 for moderate dementia, and 3 for severe dementia. A professional neuropsychology evaluator interviewed the subjects and recorded the contents of the interview. Another evaluator rated the subjects according to the record, thereby classifying them into the appropriate groups (normal, MCI, or mild AD). (3) AVLT [22, 27–29]. There are 4 categories including 12 words altogether to be recalled in this test. After the evaluator has read these words in 1-second intervals between words, the subject needs to recall
Downloaded by: Univ. of California San Diego 132.239.1.231 - 6/4/2015 4:34:02 AM
(1) Mild AD group: meet the criteria of probable AD of the National Institute of Neurological and Communicative Disorders and Stroke-Alzheimer’s Disease and Related Disorders Association (NINCDS-ADRDA) [21] and Clinical Dementia Rating scale (CDR) = 1; (2) a-MCI group, as proposed by Petersen et al. [3] in 1999: memory has become worse gradually, as reported by the subject or his/her caregiver; objective evidence of impaired memory compared with normal controls matched for age, gender, and education; auditory verbal learning test (AVLT) ≤4 for individuals between 51–60 years, ≤3 for individuals between 61 and 70 years, and ≤2 for individuals over 71 years [22]; CDR = 0.5; normal activities of daily living, and not demented; (3) Control group: no memory complaints; CDR = 0, and normal activities of daily living. All the subjects received detailed inquiries including age, educational background, method of contact, marital status, medical history (especially brain disease, concomitant diseases), personal habits, mental conditions such as anxiety and depression, collateral history from caregivers to establish their profile and exclude secondary memory impairment confounders such as anxiety, depression, schizophrenia, and drug abuse. They also needed to be able to complete the ACE-R and AVLT tests. Cranial computed tomography (CT) or magnetic resonance imaging (MRI) scans were performed to exclude other central nervous system diseases such as cerebral infarction, brain tumors, normal-pressure hydrocephalus, and so on. The patients in the mild AD and a-MCI groups had routine blood biochemical tests including folic acid, vitamin B12, and thyroid function (FT3, FT4, and TSH). All participants were of unrelated Chinese Han descent. Written consent was obtained from all participants or their guardians. The study protocol was approved by the Research Ethics Committee, Ruijin Hospital Affiliated to Shanghai Jiao Tong University School of Medicine, Shanghai, China. We selected 30 subjects at random (10 out of each group) to evaluate the inter-rater reliability of ACE-R. Then we selected 24 subjects randomly (8 out of each group) to evaluate the test-retest reliability of ACE-R. The repetition of the tests was performed 4 weeks after the initial assessment. All subjects consented to participate in the retest.
226
Dement Geriatr Cogn Disord 2014;37:223–231 DOI: 10.1159/000353541
© 2013 S. Karger AG, Basel www.karger.com/dem
Fang et al.: Validation of the Chinese Version of Addenbrooke’s Cognitive ExaminationRevised for Screening Mild Alzheimer’s Disease and Mild Cognitive Impairment
them. This procedure is repeated 3 times. Then, after a 5-min nonverbal test, every subject is asked to recall these words for the fourth time. The final occasion to recall these words is after another 20-min nonverbal test. At the last occasion, it is judged whether the subject is a-MCI according to the number of recalled words. The cutoff point of a-MCI is ≤4 between 51 and 60 years of age, ≤3 between 61 and 70 years, and ≤2 over 71 years [22]. Statistical Analysis Statistical analyses were performed using SPSS software. First, χ2 analysis was used for categorical data such as gender. One-way analysis of variance and Bonferroni correction were used for enumeration data such as age and years of education. The scores of the MMSE, ACE-R, and the sub-scores of ACE-R were also analyzed by one-way analysis of variance and Bonferroni correction. Single linear regressions were used to detect the relationship between demographic variables and neuropsychological tests (ACE-R and MMSE). Secondly, we examined the reliability of ACE-R. The intraclass correlation coefficient (ICC) was used to detect the interrater and test-retest reliability. Cronbach’s coefficient α was used to evaluate the internal consistency reliability of ACE-R. Finally, we compared the validity of ACE-R and MMSE in screening for mild AD and a-MCI separately. Receiver operating characteristic (ROC) curves were graphed and we compared the area under the curves. We determined the cutoff for the best sensitivity and specificity according to the maximum Youden index (Youden index = sensitivity + specificity – 1). Different positive predictive values (PPVs) and negative predictive values (NPVs) of ACE-R and MMSE were calculated based on different prevalence at 1, 5, 10, 20, and 40%. The statistically significant cutoff was selected to be p < 0.05 in all the tests.
Results
Demographics of Mild AD, a-MCI, and Control Groups The results of the comparison of demographics, scores of the MMSE, ACE-R, and subitems of ACE-R in the three groups are shown in table 1. There are no significant differences in age [F(2,148) = 2.81, pT = 0.063] and gender [χ2(2) = 0.326, pT = 0.850] between the three groups, but there is a significant difference in education years [F(2,148) = 3.126, pT = 0.047]. Hence, we analyzed age and education years using single linear regressions to find whether these variables had an influence on the results of the neuropsychological tests. Our results showed that age had a negative linear influence on ACE-R (standard partial regression coefficient β = –0.326, p < 0.001) and MMSE (β = –0.266, p = 0.001), whilst educational years had a positive linear influence on ACE-R (β = 0.378, p < 0.001) and MMSE (β = 0.193, p = 0.014). Among the three groups matched for age and gender, the scores of the MMSE, ACE-R, and the 5 sub-scores of ACE-R were ranked as follows: control group > a-MCI group > mild AD group. All the differences were significant (p < 0.05).
Validity of ACE-R in Screening for Mild AD and a-MCI Compared to MMSE For a differentiation between a-MCI patients and the nondemented control group, the area under the ROC curve of ACE-R was 0.836, while the area under the ROC curve of the MMSE was 0.751 (fig. 1). When the cutoff of ACE-R is 85/86 (≤85 for a-MCI and ≥86 for normal controls), the sensitivity is 0.867 and the specificity is 0.706 to detect a-MCI indi-
Downloaded by: Univ. of California San Diego 132.239.1.231 - 6/4/2015 4:34:02 AM
Reliability of the Chinese ACE-R Compared to the MMSE The inter-rater reliability and the test-retest reliability of ACE-R (ICC = 0.994 and 0.967, respectively) were comparable with the MMSE (ICC = 0.956 and 0.977, respectively). ACE-R has a good internal consistency reliability with Cronbach’s coefficient α = 0.859. Cronbach’s coefficient α of the 5 sub-items of ACE-R was 0.808 for attention and orientation, 0.803 for memory, 0.818 for fluency, 0.767 for language, and 0.797 for visuospatial abilities, indicating that every item is needed for ACE-R.
227
Dement Geriatr Cogn Disord 2014;37:223–231 © 2013 S. Karger AG, Basel
DOI: 10.1159/000353541
ROC curve for screening for a-MCI 1.0
0.8
0.8
0.6
0..6
Sensitivity
Sensitivity
ROC curve for screening for mild AD 1.0
0.4
0.2
0
a
0.4 ACE-R MMSE
0.2
0
0.2
0.4 0.6 1 – specificity
0.8
Color version available online
Fang et al.: Validation of the Chinese Version of Addenbrooke’s Cognitive ExaminationRevised for Screening Mild Alzheimer’s Disease and Mild Cognitive Impairment
1.0
0
b
0
0.2
0.4 0.6 1 – specificity
0.8
1.0
Fig. 1. ROC curves of ACE-R and MMSE for screening mild AD (a) and a-MCI (b).
Table 1. Demographics, scores of the MMSE and ACE-R, and sub-scores of ACE-R in mild AD, a-MCI, and control groups
Age, yearsa Gender, male/femaleb Education, yearsa MMSEa ACE-Ra Attention and orientationa (18 points maximum) Memorya (26 points maximum) Fluencya (14 points maximum) Languagea (26 points maximum) Visuospatiala (16 points maximum)
Mild AD (n = 25)
a-MCI (n = 75)
Control (n = 51)
p12
p13
p23
pT
73.32 ± 8.13 11/14 9.68 ± 5.01 20.12 ± 1.56 55.72 ± 9.20
69.52 ± 9.69 37/38 10.07 ± 4.41 27.03 ± 2.16 76.56 ± 10.31
68.16 ± 8.18 23/28 11.77 ± 3.46 28.77 ± 1.11 87.59 ± 7.68
0.205 0.818 1.000 *** ***
0.059 1.000 0.135 *** ***
1.000 0.717 0.085 *** ***
0.063 0.850 * *** ***
12.72 ± 1.57 12.48 ± 3.65 4.64 ± 3.12 15.44 ± 3.77 10.16 ± 3.42
16.23 ± 1.88 18.52 ± 4.20 8.09 ± 1.97 20.40 ± 4.10 13.36 ± 2.81
17.24 ± 1.16 22.63 ± 3.05 9.49 ± 2.09 23.41 ± 2.69 14.84 ± 1.62
*** *** *** *** ***
*** *** *** *** ***
** *** ** *** **
*** *** *** *** ***
viduals. The sensitivity of the MMSE is lower (0.502), yet the specificity is higher (0.863) when its cutoff is 27/28 (table 2). For screening patients with mild AD, the area under the ROC curve of ACE-R is smaller than that of the MMSE (0.945 and 0.996, respectively). When the cutoff of ACE-R is 67/68, the sensitivity is 0.920 and the specificity is 0.857. The sensitivity and specificity of the MMSE are both higher (1.000 and 0.937, respectively) when the cutoff is 23/24.
Downloaded by: Univ. of California San Diego 132.239.1.231 - 6/4/2015 4:34:02 AM
Values are means ± SD unless otherwise indicated. p12 = Comparison between mild AD and a-MCI groups; p13 = comparison between mild AD and control groups; p23 = comparison between a-MCI and control groups; pT = comparison between the three groups. * p < 0.05; ** p < 0.01; *** p < 0.001. a One-way analysis of variance and Bonferroni correction; b χ2 analysis.
228
Dement Geriatr Cogn Disord 2014;37:223–231 © 2013 S. Karger AG, Basel www.karger.com/dem
DOI: 10.1159/000353541
Fang et al.: Validation of the Chinese Version of Addenbrooke’s Cognitive ExaminationRevised for Screening Mild Alzheimer’s Disease and Mild Cognitive Impairment
Table 2. The cutoff, sensitivity, specificity and Youden index of the MMSE and ACE-R for screening for mild AD and a-MCI
MMSE ACE-R
Screening for mild AD
Screening for a-MCI
cutoff sensitivity specificity Youden index
cutoff
23/24 1.000 67/68 0.920
27/28 0.520 85/86 0.867
0.937 0.857
0.937 0.777
sensitivity specificity Youden index 0.863 0.706
0.383 0.573
Table 3. PPV and NPV of the MMSE and ACE-R for screening for mild AD and a-MCI at different prevalence
rates Different prevalence rates 1% PPV
NPV
5% PPV
NPV
10% PPV
NPV
20% PPV
NPV
40% PPV
NPV
Screening for mild AD MMSE 0.138 1.000 ACE-R 0.061 0.999
0.455 1.000 0.253 0.995
0.638 1.000 0.417 0.990
0.799 1.000 0.617 0.977
0.914 1.000 0.811 0.941
Screening for a-MCI MMSE 0.037 0.994 ACE-R 0.029 0.998
0.167 0.972 0.134 0.990
0.297 0.942 0.247 0.980
0.487 0.878 0.424 0.955
0.717 0.730 0.663 0.888
The results of PPV and NPV of ACE-R and MMSE for screening for a-MCI or mild AD at different prevalence rates (1, 5, 10, 20, and 40%) are shown in table 3. The PPVs and NPVs of the MMSE for detecting mild AD are both higher than for ACE-R. The NPVs of ACE-R for detecting a-MCI are higher than for the MMSE. The PPVs of ACE-R are slightly lower than for the MMSE when detecting a-MCI.
The goal of this study was mainly to investigate the reliability and validity of the Chinese ACE-R version to screen for mild AD as well as a-MCI. The results showed that the Chinese ACE-R is suitable for clinical application with an excellent inter-rater and test-retest reliability. Its internal consistency is also sound. Our findings show that total scores and subscores of the ACE-R are all ranked as control group > MCI group > mild AD (p < 0.01), which is consistent with earlier research results [30]. In accordance with the studies reported previously [5, 24], our study indicates that the Chinese ACE-R has a better sensitivity (0.867), accuracy (0.836 of area under the ROC curve), as well as NPV (73–99.4%) than the MMSE to screen for MCI in a general population. Some of the results are similar to those from two recent studies [7, 31]. One study with 242 subjects including 39 MCI subjects and 73 controls conducted by Yoshida et al. [31] in Japan indicated that the cutoff of ACE-R was 88/89 to screen for MCI (our cutoff is 85/86). Compared with the MMSE, ACE-R had a higher accuracy (0.952 vs. 0.868 of area under the ROC curve for ACE-R and MMSE, respectively) and a higher sensitivity (0.87 vs. 0.41 for ACE-R and MMSE, respectively) to screen for MCI [31]. Consistent with our findings, the specificity of ACE-R was lower than for the MMSE (0.92 vs. 0.99 for ACE-R and MMSE, respectively). However, the
Downloaded by: Univ. of California San Diego 132.239.1.231 - 6/4/2015 4:34:02 AM
Discussion
229
Dement Geriatr Cogn Disord 2014;37:223–231 DOI: 10.1159/000353541
© 2013 S. Karger AG, Basel
Japanese study reported a higher value of specificity (0.92) than our study (0.706) [31], possibly due to the high level of education (>11 years) and fewer MCI subjects (n = 39) in the Japanese cohort. Another study carried out by Ahmed et al. [7] in the UK showed that the cutoff of the ACE-R to screen for MCI was also 88/89, and the area under the ROC curve was 0.822, which is similar to our data. Meanwhile, our findings show that the accuracy of ACE-R when screening for AD was not higher than that of the MMSE (0.945 vs. 0.996 of area under the ROC curve, respectively), which is the same result as in a Germany study conducted by Alexopoulos et al. [17]. Interestingly, our results indicate that the sensitivity and specificity of ACE-R is 0.92 and 0.875, respectively. These results are different from most other studies, as they showed a higher sensitivity (>94%) [9, 14, 16, 31] and a higher specificity (>88%) [9, 11, 14, 16, 24]. In addition, the cutoff for screening for AD in the majority of studies was above 80 [14, 16, 21, 31], which was much higher than ours at 67/68. Besides differences in culture and language background, the following are some probable reasons to explain the significant differences between our results: (1) the sample of AD patients in this study is comparatively smaller (n = 25); (2) our group is comprised of mainly mild AD subjects, while the majority of the previous studies included both moderate AD subjects and even patients with other forms of dementia such as frontotemporal dementia or dementia with Lewy bodies, which could be more easily separated from normal controls, and (3) the mean number of education years of the AD cohort (9.68 ± 5.01 years) is less than in most of the aforementioned studies (≥10 years). There is already evidence that ACE-R cannot detect AD with a higher accuracy in people who received
© 2013 S. Karger AG, Basel 1420–8008/14/0374–0223$39.50/0 www.karger.com/dem
Original Research Article
Validation of the Chinese Version of Addenbrooke’s Cognitive ExaminationRevised for Screening Mild Alzheimer’s Disease and Mild Cognitive Impairment Rong Fang a Gang Wang a Yue Huang c Jun-Peng Zhuang a Hui-Dong Tang a Ying Wang a Yu-Lei Deng a Wei Xu a Sheng-Di Chen a, b Ru-Jing Ren a a
Department of Neurology and Institute of Neurology, Ruijin Hospital, Shanghai Jiao Tong University School of Medicine, and b Laboratory of Neurodegenerative Diseases, Institute of Health Science, Shanghai Institutes of Biological Sciences, Chinese Academy of Science and Shanghai Jiao Tong University School of Medicine, Shanghai, PR China; c Neuroscience Research Australia and the University of New South Wales, Randwick, N.S.W., Australia
Key Words Alzheimer’s disease · Addenbrooke’s Cognitive Examination-Revised · Mild cognitive impairment · Mini-Mental State Examination · Chinese · Screening Abstract Background/Aims: As a suitable test to screen for Alzheimer’s disease (AD) or mild cognitive impairment (MCI), studies to validate the Chinese version of Addenbrooke’s Cognitive Examination-Revised (ACE-R) are rare. Methods: A total of 151 subjects were recruited and the neuropsychological assessments were employed. One-way analysis of variance and Bonferroni correction were used to compare scores of different psychometric scales. Intraclass correlation coefficient (ICC) and Cronbach’s coefficient α were used to evaluate the reliability of psychometric scales. The validity of ACE-R to screen for mild AD and amnestic subtype of MCI (a-MCI) was assessed by receiver operating characteristic (ROC) curves. Results: The Chinese ACE-R had good reliability (inter-rater ICC = 0.994; test-retest ICC = 0.967) as well as reliable internal consistency (Cronbach’s coefficient α = 0.859). With its cutoff of 67/68, the sensitivity (0.920) and specificity (0.857) were lower than for the Mini-Mental State Examination (MMSE) cutoff (sensitivity 1.000 and specificity 0.937) to screen for mild AD. However, the sensitivity of ACE-R to screen for a-MCI was superior to the MMSE with a cutoff of 85/86. The specificity
Sheng-Di Chen and Ru-Jing Ren Department of Neurology and Institute of Neurology Ruijin Hospital, Shanghai Jiao Tong University School of Medicine Shanghai 200025 (PR China) E-Mail chen_sd @ medmail.com.cn; doctorren2001 @ 126.com
Downloaded by: Univ. of California San Diego 132.239.1.231 - 6/4/2015 4:34:02 AM
S.-D.C. and R.-J.R. contributed equally to this work.
224
Dement Geriatr Cogn Disord 2014;37:223–231 DOI: 10.1159/000353541
© 2013 S. Karger AG, Basel www.karger.com/dem
Fang et al.: Validation of the Chinese Version of Addenbrooke’s Cognitive ExaminationRevised for Screening Mild Alzheimer’s Disease and Mild Cognitive Impairment
of ACE-R was lower than that of the MMSE to screen for a-MCI. The area under the ROC curve of ACE-R was much larger than that of the MMSE (0.836 and 0.751) for detecting a-MCI rather than mild AD. Conclusion: The Chinese ACE-R is a reliable assessment tool for cognitive impairment. It is more sensitive and accurate in screening for a-MCI rather than for AD compared to the MMSE. © 2013 S. Karger AG, Basel
Introduction
With an aging population, the number of patients with Alzheimer’s disease (AD) and other cognitive disorders is growing dramatically [1]. The concept of mild cognitive impairment (MCI) was first proposed by Reisberg and his colleagues in 1988 [2], and Petersen et al. [3] advanced its diagnostic criteria in 1999. Petersen and Morris [4] further divided MCI into 4 subtypes [amnestic MCI-single domain (sa-MCI), amnestic MCI-multiple domains (ma-MCI), nonamnestic MCI-single domain (sna-MCI), and nonamnestic MCI-multiple domains (mna-MCI)]. Sa-MCI and ma-MCI could be combined into amnestic subtype of MCI (a-MCI), which correlates more with AD, as they are characterized predominantly by memory impairment [5]. Previous studies showed that the annual conversion rate from MCI to AD was 10–15%, and approximately 50% of MCI patients will convert to AD within 4 years [6]. Therefore, MCI is considered to be a transition phase from normal aging to AD [7]. Yet, the lack of relatively accurate screening tests for detecting early-stage AD and MCI impedes early treatment. Hence, great efforts have been made to achieve an early diagnosis of AD and associated cognitive disorders. Due to the high conversion rate to AD [8] and the lack of effective tools to identify MCI in a normal population, it has been a hot topic to identify appropriate neuropsychological screening tests with a high accuracy in detecting a-MCI. Similarly, a screening test with a high sensitivity and specificity for mild AD is also required. As the most widely used screening tool, the Mini-Mental State Examination (MMSE) has significant shortcomings, including weak executive, visuospatial, and language items as well as ceiling effects without enough sensitivity to detect MCI or mild AD. These limitations are more obvious when it is applied to individuals with a high level of education. Addenbrooke’s Cognitive Examination-Revised (ACE-R) is a brief cognitive test revised by Mioshi et al. [9] in 2006, and, by contrast, recent series of studies recommended ACE-R as a superior tool for screening for early dementia and a-MCI in normal populations as well as for discriminating different types of dementias [10, 11]. At present, ACE-R has been translated into many languages and has been validated in different clinical sites in the UK, Australia, Spain, Greece, Germany, Italy, Portugal, Japan, and Korea [5, 12–20]. However, the psychometric properties of the Chinese version have not yet been examined. Therefore, our aim is to investigate the reliability and validity of the Chinese ACE-R and to discuss the optimal screening cutoff for mild AD as well as a-MCI in the Chinese population.
Participants and Assessment Procedure A total of 151 individuals were enrolled at the Memory Clinic of the Department of Neurology, Ruijin Hospital Affiliated to Shanghai Jiao Tong University School of Medicine, from December 2010 to March 2012. The sample was divided into three groups – mild AD group (n = 25), a-MCI group (n = 75), and normal control group (n = 51) – according to the patients’ clinical manifestations, neuropsychological tests, laboratory tests, and imaging examinations, conducted collaboratively by two neurologists. The diagnostic criteria for each group were the following:
Downloaded by: Univ. of California San Diego 132.239.1.231 - 6/4/2015 4:34:02 AM
Materials and Methods
225
Dement Geriatr Cogn Disord 2014;37:223–231 DOI: 10.1159/000353541
© 2013 S. Karger AG, Basel
Fang et al.: Validation of the Chinese Version of Addenbrooke’s Cognitive ExaminationRevised for Screening Mild Alzheimer’s Disease and Mild Cognitive Impairment
Neuropsychological Assessment The following neuropsychological tests were used in this study: (1) The Chinese ACE-R. The scale was translated by Dr. Yue Huang and administration documents were translated by Dr. Gang Wang and Dr. Shengdi Chen. There are 16 cognitive items divided into 5 domains in ACE-R, including orientation, registration, attention and concentration, memory (recall, anterograde memory, retrograde memory, recognition), verbal fluency, language (comprehension, writing, repetition, naming, comprehension, reading), visuospatial abilities, and perceptual abilities. The total score is 100 points allocated to the 5 cognitive domains separately as follows: 18 points for attention and orientation, 26 points for memory, 14 points for fluency, 26 points for language, and 16 points for visuospatial abilities. A higher score indicates better cognitive function. Some items of the original ACE-R were adapted for a better understanding during administration within the Chinese population. We replaced ‘lemon’ with ‘orange’ in the item of 3 words for registration and recall. The name and address in anterograde memory, recall, and recognition was replaced by a Chinese name and address. The last question in retrograde memory was replaced by ‘who is the only female emperor in Chinese history?’ We asked subjects to speak as many words as possible with the Chinese character ‘che’ during 1 min instead of the letter ‘P’ in the item for verbal fluency. All the words and sentences were replaced by Chinese phrases in the repetition item. The comprehension part of the domain ‘language’ has 4 questions, the first one of which was replaced by ‘which thing is used for filling with water?’ The second question was revised into ‘which animal lives in Australia?’ Finally, we asked the patients to read out 8 Chinese characters instead of 5 English words in the reading item. (2) CDR [23–26]. This is a semi-structured scale, which involves 2 separate interviews with the subjects and with the relatives/caregivers of the subject. It has 6 items including memory, orientation, the abilities of judgment and solving problems, social business, housework and hobbies, and self-care abilities. Each item has 5 possible grades (0, 0.5, 1, 2, and 3). The scoring of the CDR is: 0 for normal cognition, 0.5 for very mild dementia, 1 for mild dementia, 2 for moderate dementia, and 3 for severe dementia. A professional neuropsychology evaluator interviewed the subjects and recorded the contents of the interview. Another evaluator rated the subjects according to the record, thereby classifying them into the appropriate groups (normal, MCI, or mild AD). (3) AVLT [22, 27–29]. There are 4 categories including 12 words altogether to be recalled in this test. After the evaluator has read these words in 1-second intervals between words, the subject needs to recall
Downloaded by: Univ. of California San Diego 132.239.1.231 - 6/4/2015 4:34:02 AM
(1) Mild AD group: meet the criteria of probable AD of the National Institute of Neurological and Communicative Disorders and Stroke-Alzheimer’s Disease and Related Disorders Association (NINCDS-ADRDA) [21] and Clinical Dementia Rating scale (CDR) = 1; (2) a-MCI group, as proposed by Petersen et al. [3] in 1999: memory has become worse gradually, as reported by the subject or his/her caregiver; objective evidence of impaired memory compared with normal controls matched for age, gender, and education; auditory verbal learning test (AVLT) ≤4 for individuals between 51–60 years, ≤3 for individuals between 61 and 70 years, and ≤2 for individuals over 71 years [22]; CDR = 0.5; normal activities of daily living, and not demented; (3) Control group: no memory complaints; CDR = 0, and normal activities of daily living. All the subjects received detailed inquiries including age, educational background, method of contact, marital status, medical history (especially brain disease, concomitant diseases), personal habits, mental conditions such as anxiety and depression, collateral history from caregivers to establish their profile and exclude secondary memory impairment confounders such as anxiety, depression, schizophrenia, and drug abuse. They also needed to be able to complete the ACE-R and AVLT tests. Cranial computed tomography (CT) or magnetic resonance imaging (MRI) scans were performed to exclude other central nervous system diseases such as cerebral infarction, brain tumors, normal-pressure hydrocephalus, and so on. The patients in the mild AD and a-MCI groups had routine blood biochemical tests including folic acid, vitamin B12, and thyroid function (FT3, FT4, and TSH). All participants were of unrelated Chinese Han descent. Written consent was obtained from all participants or their guardians. The study protocol was approved by the Research Ethics Committee, Ruijin Hospital Affiliated to Shanghai Jiao Tong University School of Medicine, Shanghai, China. We selected 30 subjects at random (10 out of each group) to evaluate the inter-rater reliability of ACE-R. Then we selected 24 subjects randomly (8 out of each group) to evaluate the test-retest reliability of ACE-R. The repetition of the tests was performed 4 weeks after the initial assessment. All subjects consented to participate in the retest.
226
Dement Geriatr Cogn Disord 2014;37:223–231 DOI: 10.1159/000353541
© 2013 S. Karger AG, Basel www.karger.com/dem
Fang et al.: Validation of the Chinese Version of Addenbrooke’s Cognitive ExaminationRevised for Screening Mild Alzheimer’s Disease and Mild Cognitive Impairment
them. This procedure is repeated 3 times. Then, after a 5-min nonverbal test, every subject is asked to recall these words for the fourth time. The final occasion to recall these words is after another 20-min nonverbal test. At the last occasion, it is judged whether the subject is a-MCI according to the number of recalled words. The cutoff point of a-MCI is ≤4 between 51 and 60 years of age, ≤3 between 61 and 70 years, and ≤2 over 71 years [22]. Statistical Analysis Statistical analyses were performed using SPSS software. First, χ2 analysis was used for categorical data such as gender. One-way analysis of variance and Bonferroni correction were used for enumeration data such as age and years of education. The scores of the MMSE, ACE-R, and the sub-scores of ACE-R were also analyzed by one-way analysis of variance and Bonferroni correction. Single linear regressions were used to detect the relationship between demographic variables and neuropsychological tests (ACE-R and MMSE). Secondly, we examined the reliability of ACE-R. The intraclass correlation coefficient (ICC) was used to detect the interrater and test-retest reliability. Cronbach’s coefficient α was used to evaluate the internal consistency reliability of ACE-R. Finally, we compared the validity of ACE-R and MMSE in screening for mild AD and a-MCI separately. Receiver operating characteristic (ROC) curves were graphed and we compared the area under the curves. We determined the cutoff for the best sensitivity and specificity according to the maximum Youden index (Youden index = sensitivity + specificity – 1). Different positive predictive values (PPVs) and negative predictive values (NPVs) of ACE-R and MMSE were calculated based on different prevalence at 1, 5, 10, 20, and 40%. The statistically significant cutoff was selected to be p < 0.05 in all the tests.
Results
Demographics of Mild AD, a-MCI, and Control Groups The results of the comparison of demographics, scores of the MMSE, ACE-R, and subitems of ACE-R in the three groups are shown in table 1. There are no significant differences in age [F(2,148) = 2.81, pT = 0.063] and gender [χ2(2) = 0.326, pT = 0.850] between the three groups, but there is a significant difference in education years [F(2,148) = 3.126, pT = 0.047]. Hence, we analyzed age and education years using single linear regressions to find whether these variables had an influence on the results of the neuropsychological tests. Our results showed that age had a negative linear influence on ACE-R (standard partial regression coefficient β = –0.326, p < 0.001) and MMSE (β = –0.266, p = 0.001), whilst educational years had a positive linear influence on ACE-R (β = 0.378, p < 0.001) and MMSE (β = 0.193, p = 0.014). Among the three groups matched for age and gender, the scores of the MMSE, ACE-R, and the 5 sub-scores of ACE-R were ranked as follows: control group > a-MCI group > mild AD group. All the differences were significant (p < 0.05).
Validity of ACE-R in Screening for Mild AD and a-MCI Compared to MMSE For a differentiation between a-MCI patients and the nondemented control group, the area under the ROC curve of ACE-R was 0.836, while the area under the ROC curve of the MMSE was 0.751 (fig. 1). When the cutoff of ACE-R is 85/86 (≤85 for a-MCI and ≥86 for normal controls), the sensitivity is 0.867 and the specificity is 0.706 to detect a-MCI indi-
Downloaded by: Univ. of California San Diego 132.239.1.231 - 6/4/2015 4:34:02 AM
Reliability of the Chinese ACE-R Compared to the MMSE The inter-rater reliability and the test-retest reliability of ACE-R (ICC = 0.994 and 0.967, respectively) were comparable with the MMSE (ICC = 0.956 and 0.977, respectively). ACE-R has a good internal consistency reliability with Cronbach’s coefficient α = 0.859. Cronbach’s coefficient α of the 5 sub-items of ACE-R was 0.808 for attention and orientation, 0.803 for memory, 0.818 for fluency, 0.767 for language, and 0.797 for visuospatial abilities, indicating that every item is needed for ACE-R.
227
Dement Geriatr Cogn Disord 2014;37:223–231 © 2013 S. Karger AG, Basel
DOI: 10.1159/000353541
ROC curve for screening for a-MCI 1.0
0.8
0.8
0.6
0..6
Sensitivity
Sensitivity
ROC curve for screening for mild AD 1.0
0.4
0.2
0
a
0.4 ACE-R MMSE
0.2
0
0.2
0.4 0.6 1 – specificity
0.8
Color version available online
Fang et al.: Validation of the Chinese Version of Addenbrooke’s Cognitive ExaminationRevised for Screening Mild Alzheimer’s Disease and Mild Cognitive Impairment
1.0
0
b
0
0.2
0.4 0.6 1 – specificity
0.8
1.0
Fig. 1. ROC curves of ACE-R and MMSE for screening mild AD (a) and a-MCI (b).
Table 1. Demographics, scores of the MMSE and ACE-R, and sub-scores of ACE-R in mild AD, a-MCI, and control groups
Age, yearsa Gender, male/femaleb Education, yearsa MMSEa ACE-Ra Attention and orientationa (18 points maximum) Memorya (26 points maximum) Fluencya (14 points maximum) Languagea (26 points maximum) Visuospatiala (16 points maximum)
Mild AD (n = 25)
a-MCI (n = 75)
Control (n = 51)
p12
p13
p23
pT
73.32 ± 8.13 11/14 9.68 ± 5.01 20.12 ± 1.56 55.72 ± 9.20
69.52 ± 9.69 37/38 10.07 ± 4.41 27.03 ± 2.16 76.56 ± 10.31
68.16 ± 8.18 23/28 11.77 ± 3.46 28.77 ± 1.11 87.59 ± 7.68
0.205 0.818 1.000 *** ***
0.059 1.000 0.135 *** ***
1.000 0.717 0.085 *** ***
0.063 0.850 * *** ***
12.72 ± 1.57 12.48 ± 3.65 4.64 ± 3.12 15.44 ± 3.77 10.16 ± 3.42
16.23 ± 1.88 18.52 ± 4.20 8.09 ± 1.97 20.40 ± 4.10 13.36 ± 2.81
17.24 ± 1.16 22.63 ± 3.05 9.49 ± 2.09 23.41 ± 2.69 14.84 ± 1.62
*** *** *** *** ***
*** *** *** *** ***
** *** ** *** **
*** *** *** *** ***
viduals. The sensitivity of the MMSE is lower (0.502), yet the specificity is higher (0.863) when its cutoff is 27/28 (table 2). For screening patients with mild AD, the area under the ROC curve of ACE-R is smaller than that of the MMSE (0.945 and 0.996, respectively). When the cutoff of ACE-R is 67/68, the sensitivity is 0.920 and the specificity is 0.857. The sensitivity and specificity of the MMSE are both higher (1.000 and 0.937, respectively) when the cutoff is 23/24.
Downloaded by: Univ. of California San Diego 132.239.1.231 - 6/4/2015 4:34:02 AM
Values are means ± SD unless otherwise indicated. p12 = Comparison between mild AD and a-MCI groups; p13 = comparison between mild AD and control groups; p23 = comparison between a-MCI and control groups; pT = comparison between the three groups. * p < 0.05; ** p < 0.01; *** p < 0.001. a One-way analysis of variance and Bonferroni correction; b χ2 analysis.
228
Dement Geriatr Cogn Disord 2014;37:223–231 © 2013 S. Karger AG, Basel www.karger.com/dem
DOI: 10.1159/000353541
Fang et al.: Validation of the Chinese Version of Addenbrooke’s Cognitive ExaminationRevised for Screening Mild Alzheimer’s Disease and Mild Cognitive Impairment
Table 2. The cutoff, sensitivity, specificity and Youden index of the MMSE and ACE-R for screening for mild AD and a-MCI
MMSE ACE-R
Screening for mild AD
Screening for a-MCI
cutoff sensitivity specificity Youden index
cutoff
23/24 1.000 67/68 0.920
27/28 0.520 85/86 0.867
0.937 0.857
0.937 0.777
sensitivity specificity Youden index 0.863 0.706
0.383 0.573
Table 3. PPV and NPV of the MMSE and ACE-R for screening for mild AD and a-MCI at different prevalence
rates Different prevalence rates 1% PPV
NPV
5% PPV
NPV
10% PPV
NPV
20% PPV
NPV
40% PPV
NPV
Screening for mild AD MMSE 0.138 1.000 ACE-R 0.061 0.999
0.455 1.000 0.253 0.995
0.638 1.000 0.417 0.990
0.799 1.000 0.617 0.977
0.914 1.000 0.811 0.941
Screening for a-MCI MMSE 0.037 0.994 ACE-R 0.029 0.998
0.167 0.972 0.134 0.990
0.297 0.942 0.247 0.980
0.487 0.878 0.424 0.955
0.717 0.730 0.663 0.888
The results of PPV and NPV of ACE-R and MMSE for screening for a-MCI or mild AD at different prevalence rates (1, 5, 10, 20, and 40%) are shown in table 3. The PPVs and NPVs of the MMSE for detecting mild AD are both higher than for ACE-R. The NPVs of ACE-R for detecting a-MCI are higher than for the MMSE. The PPVs of ACE-R are slightly lower than for the MMSE when detecting a-MCI.
The goal of this study was mainly to investigate the reliability and validity of the Chinese ACE-R version to screen for mild AD as well as a-MCI. The results showed that the Chinese ACE-R is suitable for clinical application with an excellent inter-rater and test-retest reliability. Its internal consistency is also sound. Our findings show that total scores and subscores of the ACE-R are all ranked as control group > MCI group > mild AD (p < 0.01), which is consistent with earlier research results [30]. In accordance with the studies reported previously [5, 24], our study indicates that the Chinese ACE-R has a better sensitivity (0.867), accuracy (0.836 of area under the ROC curve), as well as NPV (73–99.4%) than the MMSE to screen for MCI in a general population. Some of the results are similar to those from two recent studies [7, 31]. One study with 242 subjects including 39 MCI subjects and 73 controls conducted by Yoshida et al. [31] in Japan indicated that the cutoff of ACE-R was 88/89 to screen for MCI (our cutoff is 85/86). Compared with the MMSE, ACE-R had a higher accuracy (0.952 vs. 0.868 of area under the ROC curve for ACE-R and MMSE, respectively) and a higher sensitivity (0.87 vs. 0.41 for ACE-R and MMSE, respectively) to screen for MCI [31]. Consistent with our findings, the specificity of ACE-R was lower than for the MMSE (0.92 vs. 0.99 for ACE-R and MMSE, respectively). However, the
Downloaded by: Univ. of California San Diego 132.239.1.231 - 6/4/2015 4:34:02 AM
Discussion
229
Dement Geriatr Cogn Disord 2014;37:223–231 DOI: 10.1159/000353541
© 2013 S. Karger AG, Basel
Japanese study reported a higher value of specificity (0.92) than our study (0.706) [31], possibly due to the high level of education (>11 years) and fewer MCI subjects (n = 39) in the Japanese cohort. Another study carried out by Ahmed et al. [7] in the UK showed that the cutoff of the ACE-R to screen for MCI was also 88/89, and the area under the ROC curve was 0.822, which is similar to our data. Meanwhile, our findings show that the accuracy of ACE-R when screening for AD was not higher than that of the MMSE (0.945 vs. 0.996 of area under the ROC curve, respectively), which is the same result as in a Germany study conducted by Alexopoulos et al. [17]. Interestingly, our results indicate that the sensitivity and specificity of ACE-R is 0.92 and 0.875, respectively. These results are different from most other studies, as they showed a higher sensitivity (>94%) [9, 14, 16, 31] and a higher specificity (>88%) [9, 11, 14, 16, 24]. In addition, the cutoff for screening for AD in the majority of studies was above 80 [14, 16, 21, 31], which was much higher than ours at 67/68. Besides differences in culture and language background, the following are some probable reasons to explain the significant differences between our results: (1) the sample of AD patients in this study is comparatively smaller (n = 25); (2) our group is comprised of mainly mild AD subjects, while the majority of the previous studies included both moderate AD subjects and even patients with other forms of dementia such as frontotemporal dementia or dementia with Lewy bodies, which could be more easily separated from normal controls, and (3) the mean number of education years of the AD cohort (9.68 ± 5.01 years) is less than in most of the aforementioned studies (≥10 years). There is already evidence that ACE-R cannot detect AD with a higher accuracy in people who received
