Research Investigations
Validation of Montreal Cognitive Assessment and Discriminant Power of Montreal Cognitive Assessment Subtests in Patients With Mild Cognitive Impairment and Alzheimer Dementia in Turkish Population
Journal of Geriatric Psychiatry and Neurology 1-7 ª The Author(s) 2014 Reprints and permission: sagepub.com/journalsPermissions.nav DOI: 10.1177/0891988714522701 jgpn.sagepub.com
Yıldız Kaya, MD1, Ozlem Erden Aki, MD2, Ufuk Anik Can, MD1, Eda Derle, MD1, Seda Kibarog˘lu, MD1, and Anil Barak, PhD3
Abstract Montreal Cognitive Assessment (MoCA) is a new cognitive tool developed for screening mild cognitive impairment (MCI). The authors examined validity of MoCA and discriminating power of subtests in a Turkish population comprising of 474 participants (246 healthy controls, 114 subjects with MCI and 114 subjects with dementia). The ANCOVAs showed that age and education had a main effect on MoCA scores. Cut scores were computed according to different education levels. The overall cut-off values for MCI and dementia were found to be lower compared to western studies. MoCA was found to have good internal consistency. The subtests most useful in discriminating MCI from healthy controls were recall, visuospatial and language, while in discriminating dementia from MCI were visuospatial, orientation and attention subtests. The results demonstrated that MoCA is a valid and reliable instrument in screening MCI, and compared with the MMSE, MoCA was proved to have superior sensitivity and specificity in detecting MCI. Keywords cognitive screening, dementia, memory, neuropsychological testing, cognitive impairment
Introduction 1
A spectrum of cognitive changes occurs with aging. Mild cognitive impairment (MCI) is a concept developed to identify individuals at an elevated risk of dementia.2 Many investigators view MCI as a transitional state between normal cognitive functioning and dementia including Alzheimer disease.3 Amnestic-type MCI is regarded as the predementia state of AD.4 Early diagnosis of MCI offers possibilities for potential treatment with the aim of preventing or delaying the onset of dementia. Neuropsychological tests have been used to assess and diagnose patients with MCI but this is a time-consuming task and many clinicians lack the time and expertise to recruit these testing procedures. In the assessment of patients with dementia, Mini-Mental Sate Examination (MMSE) is widely used as a screening instrument although it lacks sensitivity in detecting MCI or early stages of dementia.5 To address these problems, The Montreal Cognitive Assessment (MoCA) was developed as a brief cognitive screening instrument for detection of milder forms of cognitive impairment.6 Nasreddine et al showed that MoCA has a high sensitivity
and specificity in detecting patients with MCI performing in the normal range on MMSE.6 Many other studies have also shown that MoCA is useful and accurate in identification of MCI and MoCA is superior to the MMSE in detection of patients with cognitive impairment at higher risk of incident dementia.7-9 The MoCA is a 1-page 30-point test administered approximately in 10 to 15 minutes. It allows a comprehensive assessment of major cognitive domains: short-term memory, visuospatial abilities, executive functions, attention, concentration, working memory, language, and orientation to time
1
Department of Neurology, School of Medicine, Baskent University, Ankara, Turkey 2 Department of Psychiatry, School of Medicine, Hacettepe University, Ankara, Turkey 3 Department of Statistics, Hacettepe University, Ankara, Turkey Corresponding Author: Ozlem Erden Aki, Department of Psychiatry, School of Medicine, Hacettepe University, Sihhiye 06100 Ankara, Turkey. Email: [email protected]
Downloaded from jgp.sagepub.com at GEORGIAN COURT UNIV on January 26, 2015
2
Journal of Geriatric Psychiatry and Neurology
and place. The MoCA has been adapted to different languages and validated in many countries.7,10-13 The results of several studies can be accessed through the official Web page of the MoCA group (http://www.mocatest.org). The MoCA was adapted into Turkish by Selekler et al.14 They also evaluated the predictive validity of the test in Turkish patients with MCI and dementia. The aim of the present study was to determine reliability and validity of MoCA test for patients with MCI and dementia in a larger Turkish sample through analysis of diagnostic accuracy and to establish the optimal cutoff points according to different educational levels for detection of MCI and dementia. The second aim was to identify the most prominent subtests of MoCA in determining patients with MCI and dementia.
Methods The present research was approved by the Ethical Board of Bas¸ kent University Hospitals. A written informed consent was obtained from the participants. For the patients with dementia who were incapable of providing consent on his or her behalf, a legal representative or a next of kin was asked to provide it.
Participants In the current study, the whole sample comprised 474 participants which formed 3 groups: 1. healthy elderly (normal control, NC) group, 246 individuals; 2. mild cognitive impairment (MCI) group, 114 patients; 3. Alzheimer’s dementia (AD) group, 114 patients. Age ranged from 50 to 92 years. All patients had a minimum of 5 years of education; 3 levels of education were determined: 5 years (primary education), 6 to 11 years (secondary education), and 12 years and older (high school and university education). These levels are in accordance with Turkish education system and correspond to primary school, secondary school, and high school and university education, respectively. Control group individuals were recruited through local advertisements; they had no memory complaints and all were in the normal range for neuropsychological tests used in the study. Patients with any medical, neurological, or psychiatric condition that could affect the neurocognitive functions and patients with a substance use disorder were not recruited. Patients with MCI and AD were recruited from patients visiting the outpatient neurology clinic of Bas¸ kent University Hospital between January 2010 and December 2012 with the complaint of cognitive decline. Case definitions for MCI were based on Petersen et al criteria.15 All patients of the MCI group had amnestic-type MCI. Objective memory impairment was determined by setting the cutoff level for memory performance at the 1.5 standard deviations (SDs) below the mean for the healthy age-matched groups in relevant neuropsychological tests assessing memory as will be reviewed subsequently. Diagnosis of patients with AD was given according to the
National Institute of Neurological and Communicative Disorders and Stroke-Alzheimer’s Disease and Related Disorders Association diagnostic criteria.16 Mild and moderate dementia cases were included. Final determination of the diagnoses of 2 patient groups was based on a consensus team’s decision which comprised neurologists experienced in dementia and neuropsychological assessments’ results. The participants with acute and chronic infections, inflammatory diseases like tuberculosis, with diseases that could lead to cognitive dysfunction like degenerative neurological syndromes, that is, Parkinson disease, other degenerative dementia types (vascular dementia, Lewy Body dementia, etc) cerebrovascular diseases, epilepsy, with a history of central nervous system infection and significant depression or other psychiatric disorders assessed by the Diagnostic and Statistical Manual of Mental Disorders, Fourth Edition, and participants on psychoactive drugs or substances were excluded from the study.
Neuropsychological Assessment and Procedures The Turkish version of MoCA test which was adapted into Turkish by Selekler et al was used in this study.14 This version was accessed through the official Web page of the MoCA group. Turkish version included 7 cognitive domains: visuospatial abilities (clock-drawing, cube copy, and alternation task adapted from the Trail-Making B task), naming (confrontation naming of 3 animals), attention (including the sum of attention, concentration, and working memory items), language (the sum of repetition of sentences and verbal fluency task scores), abstract thinking/executive functions (the 2-item verbal abstraction), short-term memory/recall, and orientation. We used total raw scores for all education groups on contrary to the original study and preferred to compute cutoff scores for every group with 3 education levels.6 The participants were assessed with the MMSE and MoCA in a counterbalanced method. The MMSE is a brief cognitive test used for screening dementia. Cognitive areas such as orientation to time and place, immediate and short-term memory, attention, language, and praxis are included in MMSE.5 We used Turkish form of MMSE in the present study.17 Assessment with MoCA and MMSE was followed by a full battery of neuropsychological tests covering cognitive domains associated with dementia syndromes (including episodic memory, semantic memory, working memory, executive function, and processing speed). Rey Auditory-Verbal Learning Test (RVLT) is a verbal list learning task used to evaluate components of verbal memory, including registration, recall, and recognition.18 Turkish version was adapted and validated by Oktem-Tanor et al.19 All the participants from the MCI group were required to have scores 1.5 SD below the mean for the healthy age and education adjusted normative values in RVLT. Other neurocognitive tests used in the diagnosis of the groups include Turkish adapted forms of Rey Complex Figure Test,20,21 Trail Making Test A and B,20,22 Stroop Test,20,23 Digit Span Test forward and backward tasks,24 and Controlled Oral Word Association Task20 (Turkish adaptation was conducted by Aki et al and is in publication process).
Downloaded from jgp.sagepub.com at GEORGIAN COURT UNIV on January 26, 2015
Kaya et al
3
Table 1. Mean Scores of MMSE and MoCA for Healthy Control, MCI, and Dementia (AD) Groups.
MMSE scores MoCA scores
AD, Mean + SD
MCI, Mean + SD
NC, Mean + SD
19.92 (4.54) 11.97 (4.14)
25.55 (2.09) 18.91 (3.30)
28.21 (1.75) 23.27 (3.06)
Abbreviations: MMSE, Mini-Mental State Examination; MoCA, Montreal Cognitive Assessment; AD, Alzheimer dementia group; MCI, mild cognitive impairment group; NC, normal control group.
Statistics SPSS for Windows v 17.0 was used for the data analyses. Differences between groups regarding demographic variables (age and education) were investigated using Kruskal-Wallis test and chisquare analysis, depending on the measurement of variables. Group (NC, MCI, and AD), gender, and education effects on MoCA total scores and subtest scores were investigated by analysis of covariance (ANCOVA); age was treated as covariate. Bonferroni test was used for post hoc comparisons. Cronbach a was computed to measure the internal consistency of MoCA test . Pearson correlation coefficients were computed among MoCA, MMSE, and MoCA subtest scores to test the concurrent validity. The ability of MMSE, MoCA, and MoCA subtest scores to differentiate all 3 groups was analyzed using receiver–operating characteristics (ROC) curve analysis. Larger area under the curves indicated improved diagnostic performance. For significant cutoff scores, the sensitivity and specificity values were presented. A 5% type-1 error level was used to accept a statistically significant value throughout the analyses.
Results Demographic Characteristics of Groups and ANCOVA The study was comprised of a total of 474 participants: 114 patients with AD (65 female and 49 male), 114 patients with MCI (49 female and 65 male), and 246 NC individuals (148 female and 98 male). Mean scores of MMSE and MoCA for every group are summarized in Table 1. Mean ages of NC, MCI, and AD groups were different ¼ (F2, 473 ¼ 72.699, P < .001). The AD group (mean age X 77.22, SD ¼ 9.10) was older than the MCI group (mean age X ¼ 74.20, SD ¼ 8.83) and NC group (mean age X ¼ 68.00, SD ¼ 10.34). Distribution of education levels of the participants at 3 groups also differs, w2(4) ¼ 29.89, P < .001. The MCI and NC groups had more participants with higher education levels than the AD group. In order to determine the effects of groups (AD, MCI, and NC), education (3 levels), and gender on MoCA total scores, 3 3 2 ANCOVA was conducted; since age differs within the 3 groups, age was treated as covariate and analyses revealed
25 Mean MoCA Scores
Geriatric Depression Scale (GDS) was also used to exclude depressive symptoms in all the patients. The GDS is a 30-item self-report test used to evaluate cognitive symptoms of depression in elderly patients.25 Turkish validity and reliability study was done by Ertan and Eker.26
20 15
Gender Female
10
Male
5 0 NC
MCI Group
AD
Figure 1. Gender group interaction effect on MoCA scores. MoCA indicates Montreal Cognitive Assessment; NC, normal control group; MCI, mild cognitive impairment; AD, Alzheimer’s dementia group.
group main effect (F2, 454 ¼ 251.78, P < .001). Post hoc analyses revealed that scores of all groups differed from each other. As it is seen from Table 1, the AD group had the lowest score whereas the control individuals performed the best. The ANCOVA also showed that education had a main effect on scores (F2, 454 ¼ 26.76, P < .001] while gender did not. Post hoc tests showed that performance of all education groups differed from each other (for primary education X ¼ 15.44, SD ¼ 5.36; secondary education X ¼ 19.34, SD ¼ 5.38 and for high school and university education X ¼ 21.91, SD ¼ 4.82, respectively). Gender group (F4, 454 ¼ 3.82, P ¼ .023) and education group (F4, 454 ¼ 2.66, P ¼ .032) interaction effects were also significant and are shown in Figures 1 and 2, respectively. As it is indicated in Figure 1, in the AD group women had lower MoCA scores than men while there was no difference in scores according to gender in other groups (Figure 1). Regarding education group effect, in the NC group there was a difference in MoCA scores according to education levels, in the MCI group this difference persisted, and in the AD group the MoCA total scores were not different according to educational levels (P ¼ .18; Figure 2). Montreal Cognitive Assessment total scores according to education levels for NC, MCI, and AD groups are presented in Table 2.
Internal Consistency of MoCA Cronbach a was found as .81 for MoCA. This result implicates a considerable level of internal consistency for the Turkish version of MoCA.
Downloaded from jgp.sagepub.com at GEORGIAN COURT UNIV on January 26, 2015
4
Journal of Geriatric Psychiatry and Neurology
30
Mean MoCA Scores
25 20
Education Levels 1st level
15
2nd level
10
3rd level
5 0
NC
MCI
AD
Group
Figure 2. Education-group interaction effect on MoCA scores. Education levels: 1, primary school; 2, secondary school and high school; and 3, university. MoCA indicates Montreal Cognitive Assessment; NC, normal control group; MCI, mild cognitive impairment; AD, Alzheimer’s dementia group. Table 2. MoCA Mean Scores and SDs for Patients in NC, MCI, and AD Groups According to Different Levels of Education. Groups NC
MCI
AD
Education Levelsa 1 2 3 Total 1 2 3 Total 1 2 3 Total
MoCA Scores, Mean + SD 19.76 22.98 24.70 23.27 16.37 18.78 20.17 18.91 11.00 12.31 13.07 11.97
+ 3.42 + 2.47 + 2.15 + 3.06 + 3.83 + 3.07 + 2.47 + 3.30 + 3.86 + 4.08 + 4.47 + 4.14
Abbreviations: MoCA, Montreal Cognitive Assessment; NC, normal control group; MCI, mild cognitive impairment group; AD, Alzheimer dementia group; SD, standard deviation. a Education levels: 1, primary school; 2, secondary school and high school; and 3, university.
Also, correlations of subtests and total scores of MoCA for each group were calculated and are presented in Table 3. As seen from Table 3, all the correlation coefficients were significant demonstrating a good consistency. In order to test the concurrent validity of MoCA, correlation coefficients between MMSE and MoCA total scores were computed for NC, MCI, and AD groups; these were as follows: r ¼ .626, P < .001, r ¼ .598, P < .001, r ¼ .807, P < .001, respectively.
Sensitivity and Specificity As demonstrated by the scatterplot in Figure 3, a substantial proportion of patients with MCI scored above the Turkish cutoff of 23/24 of MMSE. In contrast, a few of the patients with MCI scored in the normal range of MoCA.
Receiver–operating characteristics curves for both MMSE and MoCA were drawn for NC group versus MCI group, NC group versus AD group, and MCI versus AD groups to determine the discriminatory validity of MoCA. Areas under the curve for MoCA and MMSE for 3 comparison conditions are presented in Table 4. Education had an important effect on the scores of MoCA. The ROC analyses to determine discriminatory validity of MoCA for MCI versus NC groups according to 3 educational levels (group with 5-8 years of education, 9-11 years of education, and 12 years of education) were also drawn and are as follows: first group (AUC: 0.763, 95% confidence interval [CI]: 0.641-0.885, P < .001), second group (AUC: 0.869, 95% CI: 0.794-0.943, P < .001), and third group (AUC: 0.918, 95% CI: 0.876-0.960, P < .001) respectively. We computed cutoff points according to 3 levels of education. We used raw scores for all education groups. Cutoff points with sensitivity and specificity levels are presented in Table 5. Discriminant validity of MoCA subtests for MCI and for AD was also computed. The following subtests discriminated best participants with MCI from NC: recall (AUC: 0.758, 95% CI: 0.705-0.811, P < .001), visuospatial (AUC: 0.735, 95% CI: 0.678-0.791, P < .001), and language (AUC: 0.715, 95% CI: 0.657-0.773, P < .001). The patients with dementia were best discriminated from the patients with MCI by the following tests subsequently: visuospatial (AUC: 0.806, 95% CI: 0.748-0.864, P < .001), orientation (AUC: 0.784, 95% CI: 0.723-0.844, P < .001), and attention (AUC: 0.778, 95% CI: 0.718-0.838, P < .001).
Discussion The MoCA adaptation and validity study for a Turkish population was conducted by Selekler et al.14 In this study, a group main effect had been detected but the age and education had no main effect on the MoCA scores; a group–age interaction effect was also found. Smaller sample size comprised 20 patients with MCI and 20 patients with dementia and as the authors have stated the limited leveling of education might have played a role in these results. However, similar to our study, they also found lower mean values and lower cut scores compared to the Western studies. In the present study, effects of demographic factors on MoCA scores, concurrent validity, cutoff scores for different educational levels for MCI and dementia cases, and discriminant validity of MoCA subtest scores were determined in a larger sample of participants comprising healthy controls and patients with MCI and dementia. As seen with many neurocognitive tests, MoCA performance was also found to be affected by education, gender, and age. Previous studies in other cultures and languages reported similar effects regarding education, with lower education levels leading to lower mean scores and lower cut scores.12,27 Gender had no main effect in our study, many other studies also reported no effect of gender.12,27,28 But a gender group interaction effect was found and female patients in the dementia group had lower scores
Downloaded from jgp.sagepub.com at GEORGIAN COURT UNIV on January 26, 2015
Kaya et al
5
Table 3. Correlations of MoCA Subtest Scores With MoCA Total Scores in NC, MCI, and Dementia (AD) Groups.
NC MCI AD
MoCA total MoCA total MoCA total
Visuospatial Domain
Naming
Attention
Language
Abstraction
Recall
Orientation
0.590a 0.592a 0.671a
0.389a 0.479a 0.498a
0.641a 0.622a 0.785a
0.593a 0.455a 0.364a
0.370a 0.525a 0.326a
0.594a 0.394a 0.222b
0.279a 0.315a 0.638a
Abbreviations: MoCA, Montreal Cognitive Assessment; NC, normal control group; MCI, mild cognitive impairment group; AD, Alzheimer dementia group. a P < .001 level. b P < 0.05 level.
Figure 3. Scatterplot of Montreal Cognitive Assessment (MoCA) and Mini-Mental State Examination (MMSE) scores in normal controls (NCs), mild cognitive impairment (MCI), and Alzheimer’s dementia (AD) groups.
compared to males in the dementia group, while there was no difference regarding gender in other 2 groups. The reasons for this interaction effect in dementia group are not clear. Lu et al reported a gender effect on MoCA scores, men performing better than women among individuals with lesser education.29 In our sample, due to the lower levels of education in the participants with dementia, gender effect may have emerged only in the dementia group. Therefore, education and exposure to schooling should be taken into account while using MoCA in these populations. Age also had an effect on test scores in our sample, older age related with lower MoCA scores. Lu et al reported age effect while Lee and Hu did not find age effect on MoCA scores in their samples.12,27,29 Average MoCA scores for healthy controls and patients with MCI in this study (23.27 and 18.91 respectively) were lower than the original study of Nasreddine.6 Reasons may include the lack of familiarity of our especially primary education level patients with some of the subtests, like trail making and cube
copying. One item of the naming of 3 animals (rhinoceros) also was not familiar both semantically and phonemically for Turkish sample. Similar difficulties in those subtests were reported in other studies.27,29,30 Improvement in the Turkish version of the MoCA test may be needed, finding a more suitable item in the naming subtest. Many later research conducted in different cultures and languages found lower mean scores and also lower cutoff points for MoCA test.31,32 Waldron-Perrine explained this finding by the use of an older male veteran adult population from an urban area and recommended caution in different populations determining cut scores.32 Lu et al also found lower mean scores for healthy Chinese individuals in their study. They explained this finding by the lower level of education including individuals with no formal education in their sample and also with being unfamiliar with some of the tasks of MoCA test. In our sample, at least 5 years of education was required to be included in the study and literacy was needed for some subtests of MoCA like trail making and phonemic fluency. This offers a limitation for the applicability of MoCA in Turkish samples because there are still many illiterate individuals, especially in the older populations in Turkey. Cutoff points for both MCI and dementia were computed in the present study for different education levels. Patients with higher education levels had higher cut scores with better sensitivity and specificity than the patients with lower education levels. Presenting different cutoff scores for different levels of education would increase the utility of MoCA in Turkish elderly population. The MoCA was found to have a good concurrent validity with the MMSE in accordance with some other studies.12,33 The significant correlations between MoCA total score and subtest scores also demonstrated a good consistency. The MoCA discriminated patients with MCI and dementia slightly better than MMSE as shown in the AUC values. But its power in discriminating patients with MCI than healthy controls was more pronounced in better educated groups, which implicates MoCA is a useful screening tool in groups where ceiling effect is an issue. We also made analyses to determine the best domains of MoCA in discriminating patients with MCI and dementia. Short-term memory, visuospatial domain and language best discriminated patients with MCI from healthy controls. Memoria et al and Freitas et al found similar results in their sample.33,34 All these results are in accordance with the knowledge of memory disorder being the most prominent deficit in
Downloaded from jgp.sagepub.com at GEORGIAN COURT UNIV on January 26, 2015
6
Journal of Geriatric Psychiatry and Neurology
Table 4. Area Under the Curve Values for MoCA and MMSE Tests in Different Group Comparisons.
MCI vs NC AD vs NC MCI vs AD
MMSE MoCA MMSE MoCA MMSE MoCA
AUC
SE
P
95% CI
0.837 0.846 0.978 0.987 0.892 0.910
0.022 0.021 0.007 0.005 0.022 0.019
Validation of Montreal Cognitive Assessment and Discriminant Power of Montreal Cognitive Assessment Subtests in Patients With Mild Cognitive Impairment and Alzheimer Dementia in Turkish Population
Journal of Geriatric Psychiatry and Neurology 1-7 ª The Author(s) 2014 Reprints and permission: sagepub.com/journalsPermissions.nav DOI: 10.1177/0891988714522701 jgpn.sagepub.com
Yıldız Kaya, MD1, Ozlem Erden Aki, MD2, Ufuk Anik Can, MD1, Eda Derle, MD1, Seda Kibarog˘lu, MD1, and Anil Barak, PhD3
Abstract Montreal Cognitive Assessment (MoCA) is a new cognitive tool developed for screening mild cognitive impairment (MCI). The authors examined validity of MoCA and discriminating power of subtests in a Turkish population comprising of 474 participants (246 healthy controls, 114 subjects with MCI and 114 subjects with dementia). The ANCOVAs showed that age and education had a main effect on MoCA scores. Cut scores were computed according to different education levels. The overall cut-off values for MCI and dementia were found to be lower compared to western studies. MoCA was found to have good internal consistency. The subtests most useful in discriminating MCI from healthy controls were recall, visuospatial and language, while in discriminating dementia from MCI were visuospatial, orientation and attention subtests. The results demonstrated that MoCA is a valid and reliable instrument in screening MCI, and compared with the MMSE, MoCA was proved to have superior sensitivity and specificity in detecting MCI. Keywords cognitive screening, dementia, memory, neuropsychological testing, cognitive impairment
Introduction 1
A spectrum of cognitive changes occurs with aging. Mild cognitive impairment (MCI) is a concept developed to identify individuals at an elevated risk of dementia.2 Many investigators view MCI as a transitional state between normal cognitive functioning and dementia including Alzheimer disease.3 Amnestic-type MCI is regarded as the predementia state of AD.4 Early diagnosis of MCI offers possibilities for potential treatment with the aim of preventing or delaying the onset of dementia. Neuropsychological tests have been used to assess and diagnose patients with MCI but this is a time-consuming task and many clinicians lack the time and expertise to recruit these testing procedures. In the assessment of patients with dementia, Mini-Mental Sate Examination (MMSE) is widely used as a screening instrument although it lacks sensitivity in detecting MCI or early stages of dementia.5 To address these problems, The Montreal Cognitive Assessment (MoCA) was developed as a brief cognitive screening instrument for detection of milder forms of cognitive impairment.6 Nasreddine et al showed that MoCA has a high sensitivity
and specificity in detecting patients with MCI performing in the normal range on MMSE.6 Many other studies have also shown that MoCA is useful and accurate in identification of MCI and MoCA is superior to the MMSE in detection of patients with cognitive impairment at higher risk of incident dementia.7-9 The MoCA is a 1-page 30-point test administered approximately in 10 to 15 minutes. It allows a comprehensive assessment of major cognitive domains: short-term memory, visuospatial abilities, executive functions, attention, concentration, working memory, language, and orientation to time
1
Department of Neurology, School of Medicine, Baskent University, Ankara, Turkey 2 Department of Psychiatry, School of Medicine, Hacettepe University, Ankara, Turkey 3 Department of Statistics, Hacettepe University, Ankara, Turkey Corresponding Author: Ozlem Erden Aki, Department of Psychiatry, School of Medicine, Hacettepe University, Sihhiye 06100 Ankara, Turkey. Email: [email protected]
Downloaded from jgp.sagepub.com at GEORGIAN COURT UNIV on January 26, 2015
2
Journal of Geriatric Psychiatry and Neurology
and place. The MoCA has been adapted to different languages and validated in many countries.7,10-13 The results of several studies can be accessed through the official Web page of the MoCA group (http://www.mocatest.org). The MoCA was adapted into Turkish by Selekler et al.14 They also evaluated the predictive validity of the test in Turkish patients with MCI and dementia. The aim of the present study was to determine reliability and validity of MoCA test for patients with MCI and dementia in a larger Turkish sample through analysis of diagnostic accuracy and to establish the optimal cutoff points according to different educational levels for detection of MCI and dementia. The second aim was to identify the most prominent subtests of MoCA in determining patients with MCI and dementia.
Methods The present research was approved by the Ethical Board of Bas¸ kent University Hospitals. A written informed consent was obtained from the participants. For the patients with dementia who were incapable of providing consent on his or her behalf, a legal representative or a next of kin was asked to provide it.
Participants In the current study, the whole sample comprised 474 participants which formed 3 groups: 1. healthy elderly (normal control, NC) group, 246 individuals; 2. mild cognitive impairment (MCI) group, 114 patients; 3. Alzheimer’s dementia (AD) group, 114 patients. Age ranged from 50 to 92 years. All patients had a minimum of 5 years of education; 3 levels of education were determined: 5 years (primary education), 6 to 11 years (secondary education), and 12 years and older (high school and university education). These levels are in accordance with Turkish education system and correspond to primary school, secondary school, and high school and university education, respectively. Control group individuals were recruited through local advertisements; they had no memory complaints and all were in the normal range for neuropsychological tests used in the study. Patients with any medical, neurological, or psychiatric condition that could affect the neurocognitive functions and patients with a substance use disorder were not recruited. Patients with MCI and AD were recruited from patients visiting the outpatient neurology clinic of Bas¸ kent University Hospital between January 2010 and December 2012 with the complaint of cognitive decline. Case definitions for MCI were based on Petersen et al criteria.15 All patients of the MCI group had amnestic-type MCI. Objective memory impairment was determined by setting the cutoff level for memory performance at the 1.5 standard deviations (SDs) below the mean for the healthy age-matched groups in relevant neuropsychological tests assessing memory as will be reviewed subsequently. Diagnosis of patients with AD was given according to the
National Institute of Neurological and Communicative Disorders and Stroke-Alzheimer’s Disease and Related Disorders Association diagnostic criteria.16 Mild and moderate dementia cases were included. Final determination of the diagnoses of 2 patient groups was based on a consensus team’s decision which comprised neurologists experienced in dementia and neuropsychological assessments’ results. The participants with acute and chronic infections, inflammatory diseases like tuberculosis, with diseases that could lead to cognitive dysfunction like degenerative neurological syndromes, that is, Parkinson disease, other degenerative dementia types (vascular dementia, Lewy Body dementia, etc) cerebrovascular diseases, epilepsy, with a history of central nervous system infection and significant depression or other psychiatric disorders assessed by the Diagnostic and Statistical Manual of Mental Disorders, Fourth Edition, and participants on psychoactive drugs or substances were excluded from the study.
Neuropsychological Assessment and Procedures The Turkish version of MoCA test which was adapted into Turkish by Selekler et al was used in this study.14 This version was accessed through the official Web page of the MoCA group. Turkish version included 7 cognitive domains: visuospatial abilities (clock-drawing, cube copy, and alternation task adapted from the Trail-Making B task), naming (confrontation naming of 3 animals), attention (including the sum of attention, concentration, and working memory items), language (the sum of repetition of sentences and verbal fluency task scores), abstract thinking/executive functions (the 2-item verbal abstraction), short-term memory/recall, and orientation. We used total raw scores for all education groups on contrary to the original study and preferred to compute cutoff scores for every group with 3 education levels.6 The participants were assessed with the MMSE and MoCA in a counterbalanced method. The MMSE is a brief cognitive test used for screening dementia. Cognitive areas such as orientation to time and place, immediate and short-term memory, attention, language, and praxis are included in MMSE.5 We used Turkish form of MMSE in the present study.17 Assessment with MoCA and MMSE was followed by a full battery of neuropsychological tests covering cognitive domains associated with dementia syndromes (including episodic memory, semantic memory, working memory, executive function, and processing speed). Rey Auditory-Verbal Learning Test (RVLT) is a verbal list learning task used to evaluate components of verbal memory, including registration, recall, and recognition.18 Turkish version was adapted and validated by Oktem-Tanor et al.19 All the participants from the MCI group were required to have scores 1.5 SD below the mean for the healthy age and education adjusted normative values in RVLT. Other neurocognitive tests used in the diagnosis of the groups include Turkish adapted forms of Rey Complex Figure Test,20,21 Trail Making Test A and B,20,22 Stroop Test,20,23 Digit Span Test forward and backward tasks,24 and Controlled Oral Word Association Task20 (Turkish adaptation was conducted by Aki et al and is in publication process).
Downloaded from jgp.sagepub.com at GEORGIAN COURT UNIV on January 26, 2015
Kaya et al
3
Table 1. Mean Scores of MMSE and MoCA for Healthy Control, MCI, and Dementia (AD) Groups.
MMSE scores MoCA scores
AD, Mean + SD
MCI, Mean + SD
NC, Mean + SD
19.92 (4.54) 11.97 (4.14)
25.55 (2.09) 18.91 (3.30)
28.21 (1.75) 23.27 (3.06)
Abbreviations: MMSE, Mini-Mental State Examination; MoCA, Montreal Cognitive Assessment; AD, Alzheimer dementia group; MCI, mild cognitive impairment group; NC, normal control group.
Statistics SPSS for Windows v 17.0 was used for the data analyses. Differences between groups regarding demographic variables (age and education) were investigated using Kruskal-Wallis test and chisquare analysis, depending on the measurement of variables. Group (NC, MCI, and AD), gender, and education effects on MoCA total scores and subtest scores were investigated by analysis of covariance (ANCOVA); age was treated as covariate. Bonferroni test was used for post hoc comparisons. Cronbach a was computed to measure the internal consistency of MoCA test . Pearson correlation coefficients were computed among MoCA, MMSE, and MoCA subtest scores to test the concurrent validity. The ability of MMSE, MoCA, and MoCA subtest scores to differentiate all 3 groups was analyzed using receiver–operating characteristics (ROC) curve analysis. Larger area under the curves indicated improved diagnostic performance. For significant cutoff scores, the sensitivity and specificity values were presented. A 5% type-1 error level was used to accept a statistically significant value throughout the analyses.
Results Demographic Characteristics of Groups and ANCOVA The study was comprised of a total of 474 participants: 114 patients with AD (65 female and 49 male), 114 patients with MCI (49 female and 65 male), and 246 NC individuals (148 female and 98 male). Mean scores of MMSE and MoCA for every group are summarized in Table 1. Mean ages of NC, MCI, and AD groups were different ¼ (F2, 473 ¼ 72.699, P < .001). The AD group (mean age X 77.22, SD ¼ 9.10) was older than the MCI group (mean age X ¼ 74.20, SD ¼ 8.83) and NC group (mean age X ¼ 68.00, SD ¼ 10.34). Distribution of education levels of the participants at 3 groups also differs, w2(4) ¼ 29.89, P < .001. The MCI and NC groups had more participants with higher education levels than the AD group. In order to determine the effects of groups (AD, MCI, and NC), education (3 levels), and gender on MoCA total scores, 3 3 2 ANCOVA was conducted; since age differs within the 3 groups, age was treated as covariate and analyses revealed
25 Mean MoCA Scores
Geriatric Depression Scale (GDS) was also used to exclude depressive symptoms in all the patients. The GDS is a 30-item self-report test used to evaluate cognitive symptoms of depression in elderly patients.25 Turkish validity and reliability study was done by Ertan and Eker.26
20 15
Gender Female
10
Male
5 0 NC
MCI Group
AD
Figure 1. Gender group interaction effect on MoCA scores. MoCA indicates Montreal Cognitive Assessment; NC, normal control group; MCI, mild cognitive impairment; AD, Alzheimer’s dementia group.
group main effect (F2, 454 ¼ 251.78, P < .001). Post hoc analyses revealed that scores of all groups differed from each other. As it is seen from Table 1, the AD group had the lowest score whereas the control individuals performed the best. The ANCOVA also showed that education had a main effect on scores (F2, 454 ¼ 26.76, P < .001] while gender did not. Post hoc tests showed that performance of all education groups differed from each other (for primary education X ¼ 15.44, SD ¼ 5.36; secondary education X ¼ 19.34, SD ¼ 5.38 and for high school and university education X ¼ 21.91, SD ¼ 4.82, respectively). Gender group (F4, 454 ¼ 3.82, P ¼ .023) and education group (F4, 454 ¼ 2.66, P ¼ .032) interaction effects were also significant and are shown in Figures 1 and 2, respectively. As it is indicated in Figure 1, in the AD group women had lower MoCA scores than men while there was no difference in scores according to gender in other groups (Figure 1). Regarding education group effect, in the NC group there was a difference in MoCA scores according to education levels, in the MCI group this difference persisted, and in the AD group the MoCA total scores were not different according to educational levels (P ¼ .18; Figure 2). Montreal Cognitive Assessment total scores according to education levels for NC, MCI, and AD groups are presented in Table 2.
Internal Consistency of MoCA Cronbach a was found as .81 for MoCA. This result implicates a considerable level of internal consistency for the Turkish version of MoCA.
Downloaded from jgp.sagepub.com at GEORGIAN COURT UNIV on January 26, 2015
4
Journal of Geriatric Psychiatry and Neurology
30
Mean MoCA Scores
25 20
Education Levels 1st level
15
2nd level
10
3rd level
5 0
NC
MCI
AD
Group
Figure 2. Education-group interaction effect on MoCA scores. Education levels: 1, primary school; 2, secondary school and high school; and 3, university. MoCA indicates Montreal Cognitive Assessment; NC, normal control group; MCI, mild cognitive impairment; AD, Alzheimer’s dementia group. Table 2. MoCA Mean Scores and SDs for Patients in NC, MCI, and AD Groups According to Different Levels of Education. Groups NC
MCI
AD
Education Levelsa 1 2 3 Total 1 2 3 Total 1 2 3 Total
MoCA Scores, Mean + SD 19.76 22.98 24.70 23.27 16.37 18.78 20.17 18.91 11.00 12.31 13.07 11.97
+ 3.42 + 2.47 + 2.15 + 3.06 + 3.83 + 3.07 + 2.47 + 3.30 + 3.86 + 4.08 + 4.47 + 4.14
Abbreviations: MoCA, Montreal Cognitive Assessment; NC, normal control group; MCI, mild cognitive impairment group; AD, Alzheimer dementia group; SD, standard deviation. a Education levels: 1, primary school; 2, secondary school and high school; and 3, university.
Also, correlations of subtests and total scores of MoCA for each group were calculated and are presented in Table 3. As seen from Table 3, all the correlation coefficients were significant demonstrating a good consistency. In order to test the concurrent validity of MoCA, correlation coefficients between MMSE and MoCA total scores were computed for NC, MCI, and AD groups; these were as follows: r ¼ .626, P < .001, r ¼ .598, P < .001, r ¼ .807, P < .001, respectively.
Sensitivity and Specificity As demonstrated by the scatterplot in Figure 3, a substantial proportion of patients with MCI scored above the Turkish cutoff of 23/24 of MMSE. In contrast, a few of the patients with MCI scored in the normal range of MoCA.
Receiver–operating characteristics curves for both MMSE and MoCA were drawn for NC group versus MCI group, NC group versus AD group, and MCI versus AD groups to determine the discriminatory validity of MoCA. Areas under the curve for MoCA and MMSE for 3 comparison conditions are presented in Table 4. Education had an important effect on the scores of MoCA. The ROC analyses to determine discriminatory validity of MoCA for MCI versus NC groups according to 3 educational levels (group with 5-8 years of education, 9-11 years of education, and 12 years of education) were also drawn and are as follows: first group (AUC: 0.763, 95% confidence interval [CI]: 0.641-0.885, P < .001), second group (AUC: 0.869, 95% CI: 0.794-0.943, P < .001), and third group (AUC: 0.918, 95% CI: 0.876-0.960, P < .001) respectively. We computed cutoff points according to 3 levels of education. We used raw scores for all education groups. Cutoff points with sensitivity and specificity levels are presented in Table 5. Discriminant validity of MoCA subtests for MCI and for AD was also computed. The following subtests discriminated best participants with MCI from NC: recall (AUC: 0.758, 95% CI: 0.705-0.811, P < .001), visuospatial (AUC: 0.735, 95% CI: 0.678-0.791, P < .001), and language (AUC: 0.715, 95% CI: 0.657-0.773, P < .001). The patients with dementia were best discriminated from the patients with MCI by the following tests subsequently: visuospatial (AUC: 0.806, 95% CI: 0.748-0.864, P < .001), orientation (AUC: 0.784, 95% CI: 0.723-0.844, P < .001), and attention (AUC: 0.778, 95% CI: 0.718-0.838, P < .001).
Discussion The MoCA adaptation and validity study for a Turkish population was conducted by Selekler et al.14 In this study, a group main effect had been detected but the age and education had no main effect on the MoCA scores; a group–age interaction effect was also found. Smaller sample size comprised 20 patients with MCI and 20 patients with dementia and as the authors have stated the limited leveling of education might have played a role in these results. However, similar to our study, they also found lower mean values and lower cut scores compared to the Western studies. In the present study, effects of demographic factors on MoCA scores, concurrent validity, cutoff scores for different educational levels for MCI and dementia cases, and discriminant validity of MoCA subtest scores were determined in a larger sample of participants comprising healthy controls and patients with MCI and dementia. As seen with many neurocognitive tests, MoCA performance was also found to be affected by education, gender, and age. Previous studies in other cultures and languages reported similar effects regarding education, with lower education levels leading to lower mean scores and lower cut scores.12,27 Gender had no main effect in our study, many other studies also reported no effect of gender.12,27,28 But a gender group interaction effect was found and female patients in the dementia group had lower scores
Downloaded from jgp.sagepub.com at GEORGIAN COURT UNIV on January 26, 2015
Kaya et al
5
Table 3. Correlations of MoCA Subtest Scores With MoCA Total Scores in NC, MCI, and Dementia (AD) Groups.
NC MCI AD
MoCA total MoCA total MoCA total
Visuospatial Domain
Naming
Attention
Language
Abstraction
Recall
Orientation
0.590a 0.592a 0.671a
0.389a 0.479a 0.498a
0.641a 0.622a 0.785a
0.593a 0.455a 0.364a
0.370a 0.525a 0.326a
0.594a 0.394a 0.222b
0.279a 0.315a 0.638a
Abbreviations: MoCA, Montreal Cognitive Assessment; NC, normal control group; MCI, mild cognitive impairment group; AD, Alzheimer dementia group. a P < .001 level. b P < 0.05 level.
Figure 3. Scatterplot of Montreal Cognitive Assessment (MoCA) and Mini-Mental State Examination (MMSE) scores in normal controls (NCs), mild cognitive impairment (MCI), and Alzheimer’s dementia (AD) groups.
compared to males in the dementia group, while there was no difference regarding gender in other 2 groups. The reasons for this interaction effect in dementia group are not clear. Lu et al reported a gender effect on MoCA scores, men performing better than women among individuals with lesser education.29 In our sample, due to the lower levels of education in the participants with dementia, gender effect may have emerged only in the dementia group. Therefore, education and exposure to schooling should be taken into account while using MoCA in these populations. Age also had an effect on test scores in our sample, older age related with lower MoCA scores. Lu et al reported age effect while Lee and Hu did not find age effect on MoCA scores in their samples.12,27,29 Average MoCA scores for healthy controls and patients with MCI in this study (23.27 and 18.91 respectively) were lower than the original study of Nasreddine.6 Reasons may include the lack of familiarity of our especially primary education level patients with some of the subtests, like trail making and cube
copying. One item of the naming of 3 animals (rhinoceros) also was not familiar both semantically and phonemically for Turkish sample. Similar difficulties in those subtests were reported in other studies.27,29,30 Improvement in the Turkish version of the MoCA test may be needed, finding a more suitable item in the naming subtest. Many later research conducted in different cultures and languages found lower mean scores and also lower cutoff points for MoCA test.31,32 Waldron-Perrine explained this finding by the use of an older male veteran adult population from an urban area and recommended caution in different populations determining cut scores.32 Lu et al also found lower mean scores for healthy Chinese individuals in their study. They explained this finding by the lower level of education including individuals with no formal education in their sample and also with being unfamiliar with some of the tasks of MoCA test. In our sample, at least 5 years of education was required to be included in the study and literacy was needed for some subtests of MoCA like trail making and phonemic fluency. This offers a limitation for the applicability of MoCA in Turkish samples because there are still many illiterate individuals, especially in the older populations in Turkey. Cutoff points for both MCI and dementia were computed in the present study for different education levels. Patients with higher education levels had higher cut scores with better sensitivity and specificity than the patients with lower education levels. Presenting different cutoff scores for different levels of education would increase the utility of MoCA in Turkish elderly population. The MoCA was found to have a good concurrent validity with the MMSE in accordance with some other studies.12,33 The significant correlations between MoCA total score and subtest scores also demonstrated a good consistency. The MoCA discriminated patients with MCI and dementia slightly better than MMSE as shown in the AUC values. But its power in discriminating patients with MCI than healthy controls was more pronounced in better educated groups, which implicates MoCA is a useful screening tool in groups where ceiling effect is an issue. We also made analyses to determine the best domains of MoCA in discriminating patients with MCI and dementia. Short-term memory, visuospatial domain and language best discriminated patients with MCI from healthy controls. Memoria et al and Freitas et al found similar results in their sample.33,34 All these results are in accordance with the knowledge of memory disorder being the most prominent deficit in
Downloaded from jgp.sagepub.com at GEORGIAN COURT UNIV on January 26, 2015
6
Journal of Geriatric Psychiatry and Neurology
Table 4. Area Under the Curve Values for MoCA and MMSE Tests in Different Group Comparisons.
MCI vs NC AD vs NC MCI vs AD
MMSE MoCA MMSE MoCA MMSE MoCA
AUC
SE
P
95% CI
0.837 0.846 0.978 0.987 0.892 0.910
0.022 0.021 0.007 0.005 0.022 0.019
