RESEARCH ARTICLE
Validation of the Montreal Cognitive Assessment (MoCA) in Spanish as a screening tool for mild cognitive impairment and mild dementia in patients over 65 years old in Bogotá, Colombia Laura Gil1, Carolina Ruiz de Sánchez1,2, Fabián Gil3, Sara Julieta Romero2 and Felipe Pretelt Burgos1 1
Neurosciences Department, Hospital San Ignacio, Pontificia Universidad Javeriana, Bogotá D.C., Colombia Memory Clinic Intellectus, Memory and Cognition Center, Hospital San Ignacio, Pontificia Universidad Javeriana, Bogotá D.C., Colombia 3 Clinical Epidemiology and Biostatistics Department, Pontificia Universidad Javeriana, Bogotá D.C., Colombia Correspondence to: Dr. L. Gil, E-mail: [email protected] 2
The Montreal Cognitive Assessment (MoCA) was developed as a simple screening tool for cognitive impairment. This study is the first validation in Latin America of the MoCA in Spanish (MoCA-S), which was developed in Colombia (South America). Methods: Aiming to perform the first validation of the MoCA-S, we developed a study of concordance by conformity to assess the MoCA-S compared with diagnostic consensus by interdisciplinary assessment in the Memory Clinic (the best diagnostic method available) and to evaluate the psychometric properties of the MoCA-S. A total of 193 subjects were evaluated, 109 of whom were patients, including 26 who met the mild cognitive impairment (MCI) clinical criteria, based on neuropsychological testing, and 83 who had mild dementia (MD). The remaining 84 participants were healthy subjects from the community. Results: The psychometric evaluation of the MoCA-S was appropriate. Using a cutoff score of ≥23, the MoCA had sensitivities of 76.0% to detect MCI and 92.7% to detect MD and a specificity of 79.8%. The percentage of patients clearly labeled by the MoCA-S was 85%. Conclusion: The MoCA-S is a valid screening tool and is useful for identifying MCI and MD in Colombia. The MoCA-S is valid and adequate for application in Colombia with good internal consistency, interobserver reliability, and content validity. However, the average educational level was high in this study; thus, caution should be exercised when extrapolating these results to individuals with lower educational levels. Copyright # 2014 John Wiley & Sons, Ltd. Objective:
Key words: dementia; mild cognitive impairment; Alzheimer’s disease; validation studies; MoCA-S; neuropsychological test; memory disorders History: Received 10 November 2013; Accepted 31 July 2014; Published online in Wiley Online Library (wileyonlinelibrary.com) DOI: 10.1002/gps.4199
Introduction The Montreal Cognitive Assessment (MoCA) test was developed by Nasreddine et al. (2005) as a simple instrument with which to screen for mild cognitive impairment (MCI) and mild Alzheimer’s disease (AD). This test was developed to overcome the difficulties imposed by the Mini-mental state examination (MMSE) in the diagnosis and differentiation of mild Copyright # 2014 John Wiley & Sons, Ltd.
AD and MCI (Nasreddine et al., 2005). The MMSE exhibits difficulties in detecting dementia in its initial stages (Folstein et al., 1975), and most of the patients who meet the clinical criteria for MCI have normal scores on the MMSE (Nasreddine et al., 2005). Several studies worldwide have assessed the MoCA and have found it to be a useful instrument for the screening and diagnosis of MCI and mild AD (Nasreddine et al., 2005; Lee et al., 2008; Wong et al., Int J Geriatr Psychiatry 2014
L. Gil et al.
2008; Lozano et al., 2009; Rahman and El Gaafary, 2009; Tangwongchai et al., 2009; Duro et al., 2010; Fujiwara et al., 2010; Thissen et al., 2010; Magierska et al., 2012; Tsai et al., 2012; Freitas et al., 2013) or of mild dementia (MD) (Smith et al., 2007; Lozano et al., 2009) with adequate validity, test–retest reliability, inter-observer reliability, and internal consistency (Nasreddine et al., 2005; Smith et al., 2007; Lee et al., 2008; Wong et al., 2008; Lozano et al., 2009; Rahman and El Gaafary, 2009; Tangwongchai et al., 2009; Duro et al., 2010; Fujiwara et al., 2010; Thissen et al., 2010; Magierska et al., 2012; Tsai et al., 2012; Freitas et al., 2013). The Spanish translation of the MoCA (MoCA-S) was evaluated in Spain by Lozano et al. (2009), who collected preliminary data showing that the MoCA-S is effective for detecting dementia but is less effective than the original version for the diagnosis of MCI. In Brazil, this scale has been validated in Portuguese with adequate validity and reliability using a cutoff score of 25, yielding a sensitivity of 81% and specificity of 77% (Memória et al., 2013). Until now, there have been no validations of the MoCA-S in Spanishspeaking countries of Latin America. A study in Colombia was conducted to assess the performance of the MoCA-S in an older population with a low educational level. That study revealed that the MoCA-S had high reliability in older Colombians with low educational levels but that the scores were strongly dependent on educational level (Gómez et al., 2013). The initial validation of the scale with a cutoff score of ≥26 reached a sensitivity for the diagnosis of MCI and mild AD of 90% and 100%, respectively, and a specificity of 87% for the exclusion of older individuals who were cognitively healthy (Nasreddine et al., 2005). The cutoff score has varied in the different validations of the scale (Lee et al., 2008; Waldron-Perrine and Axelrod, 2012; Yu et al., 2012; Freitas et al., 2013; Memória et al., 2013). In several studies, the specificity was improved by lowering the cutoff score (Luis et al., 2009; Larner, 2011). Furthermore, in a study performed by Rossetti et al. in 2653 subjects with cardiovascular disease from a population encompassing several ethnicities, after excluding those subjects with complaints of memory and neurological alterations, the average value of the MoCA was 23.4, suggesting a lower cutoff point than that initially proposed for the original scale (Rossetti et al., 2011). The cutoff point of this scale has clearly not yet been established, as it varies according to the study population and the educational level. For this reason, the aim of this study was to validate the MoCA-S in Colombia and propose a cutoff score. The MoCA-S Copyright # 2014 John Wiley & Sons, Ltd.
is a screening instrument that will help select patients with memory complaints who must undergo a complete neurological and neuropsychological study, thereby optimizing the use of health resources in our country. In addition, MCI has become a very important concept in recent years because it represents a window of opportunity for earlier diagnosis and treatment before evolving to frank dementia (Ghost et al., 2014). MCI is a state of transition between a normal cognitive state and a pathological cognitive state, that is, dementia. Therefore, its identification is essential. The clinical criteria for MCI proposed by Petersen et al. (1999) state that MCI is a transitional condition of cognitive decline that does not compromise the activities of daily living enough to warrant the diagnosis of dementia and the patient must have a score of 1.5 standard deviations below the level expected according to age and educational status in one domain of the neuropsychological evaluation, mainly memory, or subtle impairment in multiple domains of 0.5 to 1 standard deviation below the levels expected according to age and educational status (Petersen et al., 2001; Petersen, 2004; Ghost et al. 2014). The study by Petersen et al. (1999) showed that patients with MCI and MD have an average of 2.7 and 3.4, respectively, on the Global Deterioration Scale (GDS) of the Reisberg scale. No screening test for MCI is currently available in Colombia; therefore, we want to validate the Spanish translation of the MoCA. Methods The MoCA-S was used; this test is available from the website http://www.mocatest.org. It consists of a 30-point scale, divided into seven domains that assess the following functions: visuospatial/executive (five points), nomination (three points), attention (six points), language (three points), abstraction (two points), memory (five points), and orientation (six points). No modification was performed to the Spanish translation of the original scale, and the Nasreddine group was notified regarding our use of the test. This study was performed at the San Ignacio University Hospital of the Pontificia Universidad Javeriana with the support of the Memory Clinic Intellectus, Center of Memory and Cognition, and Neuroscience Department. All participants in this study provided their informed consent authorizing their voluntary participation. The project was approved by the Research and Ethics Committee of the School of Medicine at the Pontificia Universidad Javeriana. Int J Geriatr Psychiatry 2014
Spanish MoCA validation in Colombia
Design
Assessment
With the purpose of validating the MoCA-S, an agreement by consensus type of study (Kotter et al., 2011) was planned to compare the execution of the MoCA-S with the clinical diagnosis obtained through consensus of the Memory Clinic Intellectus (the best diagnostic method available). Consequently, the study was intended to evaluate the psychometric properties of the Spanish version of the test. The main objective of the study was to determine whether the MoCA-S instrument was useful for discriminating between cognitively healthy subjects and subjects with cognitive alterations associated with MCI and MD in Colombia.
All subjects took the MoCA-S. Those subjects with cognitive alterations were assessed at the Memory Clinic by an interdisciplinary team comprising specialists in neurology, psychiatry, geriatrics, psychogeriatrics, and neuropsychology who applied the clinical diagnostic criteria for patients with MCI or MD. The diagnoses were supported by the neuropsychological test profiles, functional assessments, and, in most cases, neuroimaging evaluations. No cases of dementia were confirmed by genetic or pathological biomarkers. The specialists applied the following scales: Minimental state exam, Hashinski Scale, Barthel and Lawton Scales, Hamilton Test, Columbia, Yesavage Scale, and Cornell Scale. They also performed a complete general, mental, and neurological exam of the patients. The neuropsychology tests chosen were those recognized as assessing the cognitive functions implicated in MCI and MD. The aim of the clinical battery was to gain information in areas relevant to the determination of MCI and MD, without recourse to information from the neuropsychology battery to permit the independent assessment of each. Additionally, all patients were evaluated by a battery of standardized neuropsychological tests adapted in Colombia (Jacquier et al., 1997), which included some tests from the Consortium to Establish a Registry for AD (Aguirre-Acevedo et al., 2007). The tools included the Verbal Fluency Test, Boston Naming Test, Word List Memory, Constructional Praxis, Word List Recall, Word List Recognition, Recall of Constructional Praxis, and additional tests such as the Grober–Buschke test, Digit Symbol Substitution Test, Frontal Behavioral Inventory Kertesz, INECO Test, Sayings and Similarities Test, and Retrospective Memory Test. The patients were also evaluated by a neurologist, who administered the MoCA-S. The evaluator who administered the MoCA-S did not know the clinical diagnosis or results of the neuropsychological assessment. The neuropsychological assessment was performed before evaluating the patients’ medical record. Both measurement techniques were applied on the same day to ensure that the subjects’ condition had not changed. The healthy subjects were assessed by specialists in neurology and neuropsychology using the same neuropsychological battery.
Subjects
The subjects younger than 65 years old were excluded from the study. Two groups were formed (Table 1). The first group included 109 patients with cognitive impairment, and the second group included 84 healthy subjects recruited from the community who had normal neurological and neuropsychological assessments and no memory complaints. The patient group was subdivided into the MCI subgroup (includes amnestic and multiple-domain MCI), comprised of 26 subjects who met Petersen’s diagnostic criteria for MCI (cognitive complaints and neuropsychological evaluation with an execution of 1.5 standard deviations below the level expected according to age and educational status; Petersen et al., 1999) and a GDS of Reisberg score of 2 or 3 (Reisberg et al., 1982), and the MD subgroup, comprised of 83 subjects who met the clinical criteria for dementia of the DSM-IV (American Psychiatric Association, 2000) and were found to be in the MD stage with a GDS score of 3 or 4 or to have compromises in their advanced and instrumental activities of daily living. Table 1 Demographic features, namely age, gender, and education level, and the corresponding diagnostic percentages Subject demographics
No.
%
Gender
68 125 123 57 13 9 81 39 64 84 26 83 193
35.3 64.7 63.7 29.6 6.7 4.6 42 20.2 33.2 43.5 13.5 43.0 100.0
Age Educational level
Diagnosis
Male Female 65–75 years 76–85 years >86 years 89% and a specificity of >79% were considered to indicate optimal screening performance.
MoCA vs MMSE 37.5
30.
28.6
27.3 25.2
24.5 20.7
22.5
16.1
MMSE MOCA
15.
7.5
0.
Results
Control group
Subject demographics
The MoCA-S was applied to 193 subjects, 64.7% of whom were women, with an age range between 65 and 94 years. Most patients were between 66 and 75 years old. The average educational level of the assessed population was 12 years. See Tables 1 and 2 for detailed demographic information. Within all study subgroups, namely, the control group and MCI and MD patient groups, no differences were found with respect to age (p = 0.33). The educational level was also similar, although the MD group had a significantly lower educational level than the MCI and control groups (p = 0.0004). There were no significant differences between the healthy subjects and the MCI patients (p = 0.10). Such differences in the educational level have been found by other authors (Nasreddine et al., 2005) without invalidating the results. In the group of subjects with cognitive impairment, the average scores were 17.2 on the MoCA-S and 25.2 on the MMSE. The healthy group showed average scores of 25.2 on the MoCA-S and 28.6 on the MMSE. When subdividing the group of subjects with cognitive impairment, the MCI subgroup had an average MoCA-S score of 20.7 and an average MMSE score of 27.3 (normal range). In contrast, the MD subgroup showed an average MoCA-S score of 16.1 (abnormal range) and an average MMSE score of 24.5 (abnormal range; Figure 1). These findings Table 2 Average age and educational level according to diagnosis Demographic features
Subgroup Control MCI MD
Average age (years)
Average educational level (years)
Women (%)
68 (SD: 10.38) 65 (SD: 13.4) 73 (SD: 7.5)
14 (SD: 4.7) 13 (SD: 5.1) 11 (SD: 5.1)
69.4 65.9 57.4
MCI, mild cognitive impairment; MD, mild dementia; SD, standard deviation.
Copyright # 2014 John Wiley & Sons, Ltd.
MCI
MD
Figure 1 Average of the test scores of Montreal Cognitive Assessment Spanish version (MoCA-S) versus Mini-mental state examination (MMSE) in patients with mild dementia (MD) or mild cognitive impairment (MCI) or in healthy subjects (control group).
reveal that the MoCA-S showed a large score difference between the different subgroups, whereas the MMSE remained in the normal range for the MCI subgroup and may remain in the normal range for the MD subgroup. The MoCA-S test score was significantly different between the different subgroups (p < 0.0001). The majority of patients with MD had AD (66.2%) with a GDS score of 3 or 4. Psychometric properties of the Montreal Cognitive Assessment Scale assessment. The inter-observer reliability was assessed through the application of the test on the first 10 patients in the study by two independent and blinded evaluators. A Lin correlation of coefficient and concordance of 0.093 was obtained with a 95% CI of [0.78, 1.00]. These correlation and concordance coefficients are considered almost perfect. The translation of the scale to Spanish was evaluated, and it was found to meet several requirements that support its construct validity. It has an adequate appearance and content validity. Furthermore, its internal consistency was assessed through the Cronbach alpha index, which yielded a score of 0.8463 for the different items assessed. This result is similar to that obtained by the authors in the validation of the original scale. The criterion validity of the test was assessed by comparing the result of the Spanish translation of the MoCA with the interdisciplinary diagnosis of the Memory Clinic, and good agreement between both measurements was found (kappa of 0.69; 95% CI [0.59, 0.80]). Additionally, an analysis of the variance was performed to correlate the score averages of the MoCA-S between the three different subgroups, namely, Int J Geriatr Psychiatry 2014
Spanish MoCA validation in Colombia
the control, MCI, and MD groups, considering that the scores differed by group. A significant difference was found between the three subgroups (p < 0.0001). Finally, a strong correlation with the MMSE was found through the Spearman correlation index (0.755; 95% CI [0.687, 0.811]). This result demonstrates that the scale represents a good measurement of the cognitive performance of these subjects. Agreement of the Montreal Cognitive Assessment in Spanish with the results of the interdisciplinary assessment by the Memory Clinic: sensitivity and specificity
The sensitivities, specificities, positive predictive value, negative predictive value, and accuracy of the MoCA-S at different cutoff values are presented in Table 3. The agreement of the classification of the subjects according to the results of the MoCA-S test, considering a healthy patient cutoff score of ≥23, with the diagnosis by the Memory Clinic was adequate (kappa 0.69; 95% CI [0.59, 0.80]). The percentage of patients correctly classified by the MoCA-S was 85%. The ROC curve is presented in Figure 2, and the area under the ROC curve was 0.93 (95% CI [0.89, 0.96]; Figure 2). The sensitivity and specificity of the test for the population sample were optimized when using a cutoff score of ≥23 (any score of ≤22 was considered to be an abnormal result). With this cutoff score, a better balance between sensitivity and specificity was achieved. The sensitivity of the test to detect cognitive impairment (MCI or MD) was 89%, with a specificity of 79.8%. Similarly, the positive predictive value was 85%, and the negative predictive value was 84.8%. As the cutoff score was increased to 24 or higher, the sensitivity improved, but the specificity worsened because many healthy patients were then classified as
Figure 2 Receiver operating characteristic (ROC) curve analysis of the Spanish version of the Montreal Cognitive Assessment for the detection of normal cognitive and mild cognitive impairment/mild dementia. CI, confidence interval.
ill by the MoCA-S. In contrast, if the cutoff point was decreased to 22 or less, the sensitivity decreased, allowing more patients with cognitive impairment to be classified as normal by the MoCA-S (Table 3). When the sensitivity calculation was performed only for the subgroup of subjects with MD, a sensitivity of 92.7% was established, with only six false positives. These cases included three patients with AD, two patients with behavioral variant frontotemporal dementia, and one patient with mixed dementia. This cutoff value guarantees a low possibility of the test to classify a subject with MD as normal. Effect of educational level
The educational level has an effect on test performance (Table 4). The scores of MoCA and MMSE are dropping along with the education level in the three
Table 3 Sensitivity, specificity, positive predictive value (PPV), negative predictive value (NPV), and accuracy of the Spanish version Montreal Cognitive Assessment for detection of normal cognitive and mild cognitive impairment/mild dementia Cutoff ≥19 ≥20 ≥21 ≥22 ≥23 ≥24 ≥25 ≥26 ≥27 ≥28 ≥29 30
Sensitivity (%)
Specificity (%)
PPV (%)
NPV (%)
Accuracy (%)
56.9 62.4 72.5 80.7 89.0 93.6 96.3 99.1 99.1 100.0 100.0 100.0
97.6 95.2 88.1 84.5 79.8 76.2 61.9 52.4 35.7 20.2 13.1 10.7
96.8 94.4 88.7 87.1 85 83.6 76.6 72.2 66.6 59.8 59.8 59.2
44.2 71.1 31.2 77.1 84.8 90.1 92.8 97.7 96.7 100.0 100.0 100.0
74.6 76.7 79.3 82.4 85.0 86.0 81.3 78.8 71.5 65.3 62.2 56.5
Bold values indicate cutoff point used in the study with values of sensitivity and specificity.
Copyright # 2014 John Wiley & Sons, Ltd.
Int J Geriatr Psychiatry 2014
L. Gil et al. Table 4 Average MoCA (without correction) and MMSE scores according to educational level and diagnostic group Education level
MoCA MMSE
Control MCI MD Control MCI MD
Validation of the Montreal Cognitive Assessment (MoCA) in Spanish as a screening tool for mild cognitive impairment and mild dementia in patients over 65 years old in Bogotá, Colombia Laura Gil1, Carolina Ruiz de Sánchez1,2, Fabián Gil3, Sara Julieta Romero2 and Felipe Pretelt Burgos1 1
Neurosciences Department, Hospital San Ignacio, Pontificia Universidad Javeriana, Bogotá D.C., Colombia Memory Clinic Intellectus, Memory and Cognition Center, Hospital San Ignacio, Pontificia Universidad Javeriana, Bogotá D.C., Colombia 3 Clinical Epidemiology and Biostatistics Department, Pontificia Universidad Javeriana, Bogotá D.C., Colombia Correspondence to: Dr. L. Gil, E-mail: [email protected] 2
The Montreal Cognitive Assessment (MoCA) was developed as a simple screening tool for cognitive impairment. This study is the first validation in Latin America of the MoCA in Spanish (MoCA-S), which was developed in Colombia (South America). Methods: Aiming to perform the first validation of the MoCA-S, we developed a study of concordance by conformity to assess the MoCA-S compared with diagnostic consensus by interdisciplinary assessment in the Memory Clinic (the best diagnostic method available) and to evaluate the psychometric properties of the MoCA-S. A total of 193 subjects were evaluated, 109 of whom were patients, including 26 who met the mild cognitive impairment (MCI) clinical criteria, based on neuropsychological testing, and 83 who had mild dementia (MD). The remaining 84 participants were healthy subjects from the community. Results: The psychometric evaluation of the MoCA-S was appropriate. Using a cutoff score of ≥23, the MoCA had sensitivities of 76.0% to detect MCI and 92.7% to detect MD and a specificity of 79.8%. The percentage of patients clearly labeled by the MoCA-S was 85%. Conclusion: The MoCA-S is a valid screening tool and is useful for identifying MCI and MD in Colombia. The MoCA-S is valid and adequate for application in Colombia with good internal consistency, interobserver reliability, and content validity. However, the average educational level was high in this study; thus, caution should be exercised when extrapolating these results to individuals with lower educational levels. Copyright # 2014 John Wiley & Sons, Ltd. Objective:
Key words: dementia; mild cognitive impairment; Alzheimer’s disease; validation studies; MoCA-S; neuropsychological test; memory disorders History: Received 10 November 2013; Accepted 31 July 2014; Published online in Wiley Online Library (wileyonlinelibrary.com) DOI: 10.1002/gps.4199
Introduction The Montreal Cognitive Assessment (MoCA) test was developed by Nasreddine et al. (2005) as a simple instrument with which to screen for mild cognitive impairment (MCI) and mild Alzheimer’s disease (AD). This test was developed to overcome the difficulties imposed by the Mini-mental state examination (MMSE) in the diagnosis and differentiation of mild Copyright # 2014 John Wiley & Sons, Ltd.
AD and MCI (Nasreddine et al., 2005). The MMSE exhibits difficulties in detecting dementia in its initial stages (Folstein et al., 1975), and most of the patients who meet the clinical criteria for MCI have normal scores on the MMSE (Nasreddine et al., 2005). Several studies worldwide have assessed the MoCA and have found it to be a useful instrument for the screening and diagnosis of MCI and mild AD (Nasreddine et al., 2005; Lee et al., 2008; Wong et al., Int J Geriatr Psychiatry 2014
L. Gil et al.
2008; Lozano et al., 2009; Rahman and El Gaafary, 2009; Tangwongchai et al., 2009; Duro et al., 2010; Fujiwara et al., 2010; Thissen et al., 2010; Magierska et al., 2012; Tsai et al., 2012; Freitas et al., 2013) or of mild dementia (MD) (Smith et al., 2007; Lozano et al., 2009) with adequate validity, test–retest reliability, inter-observer reliability, and internal consistency (Nasreddine et al., 2005; Smith et al., 2007; Lee et al., 2008; Wong et al., 2008; Lozano et al., 2009; Rahman and El Gaafary, 2009; Tangwongchai et al., 2009; Duro et al., 2010; Fujiwara et al., 2010; Thissen et al., 2010; Magierska et al., 2012; Tsai et al., 2012; Freitas et al., 2013). The Spanish translation of the MoCA (MoCA-S) was evaluated in Spain by Lozano et al. (2009), who collected preliminary data showing that the MoCA-S is effective for detecting dementia but is less effective than the original version for the diagnosis of MCI. In Brazil, this scale has been validated in Portuguese with adequate validity and reliability using a cutoff score of 25, yielding a sensitivity of 81% and specificity of 77% (Memória et al., 2013). Until now, there have been no validations of the MoCA-S in Spanishspeaking countries of Latin America. A study in Colombia was conducted to assess the performance of the MoCA-S in an older population with a low educational level. That study revealed that the MoCA-S had high reliability in older Colombians with low educational levels but that the scores were strongly dependent on educational level (Gómez et al., 2013). The initial validation of the scale with a cutoff score of ≥26 reached a sensitivity for the diagnosis of MCI and mild AD of 90% and 100%, respectively, and a specificity of 87% for the exclusion of older individuals who were cognitively healthy (Nasreddine et al., 2005). The cutoff score has varied in the different validations of the scale (Lee et al., 2008; Waldron-Perrine and Axelrod, 2012; Yu et al., 2012; Freitas et al., 2013; Memória et al., 2013). In several studies, the specificity was improved by lowering the cutoff score (Luis et al., 2009; Larner, 2011). Furthermore, in a study performed by Rossetti et al. in 2653 subjects with cardiovascular disease from a population encompassing several ethnicities, after excluding those subjects with complaints of memory and neurological alterations, the average value of the MoCA was 23.4, suggesting a lower cutoff point than that initially proposed for the original scale (Rossetti et al., 2011). The cutoff point of this scale has clearly not yet been established, as it varies according to the study population and the educational level. For this reason, the aim of this study was to validate the MoCA-S in Colombia and propose a cutoff score. The MoCA-S Copyright # 2014 John Wiley & Sons, Ltd.
is a screening instrument that will help select patients with memory complaints who must undergo a complete neurological and neuropsychological study, thereby optimizing the use of health resources in our country. In addition, MCI has become a very important concept in recent years because it represents a window of opportunity for earlier diagnosis and treatment before evolving to frank dementia (Ghost et al., 2014). MCI is a state of transition between a normal cognitive state and a pathological cognitive state, that is, dementia. Therefore, its identification is essential. The clinical criteria for MCI proposed by Petersen et al. (1999) state that MCI is a transitional condition of cognitive decline that does not compromise the activities of daily living enough to warrant the diagnosis of dementia and the patient must have a score of 1.5 standard deviations below the level expected according to age and educational status in one domain of the neuropsychological evaluation, mainly memory, or subtle impairment in multiple domains of 0.5 to 1 standard deviation below the levels expected according to age and educational status (Petersen et al., 2001; Petersen, 2004; Ghost et al. 2014). The study by Petersen et al. (1999) showed that patients with MCI and MD have an average of 2.7 and 3.4, respectively, on the Global Deterioration Scale (GDS) of the Reisberg scale. No screening test for MCI is currently available in Colombia; therefore, we want to validate the Spanish translation of the MoCA. Methods The MoCA-S was used; this test is available from the website http://www.mocatest.org. It consists of a 30-point scale, divided into seven domains that assess the following functions: visuospatial/executive (five points), nomination (three points), attention (six points), language (three points), abstraction (two points), memory (five points), and orientation (six points). No modification was performed to the Spanish translation of the original scale, and the Nasreddine group was notified regarding our use of the test. This study was performed at the San Ignacio University Hospital of the Pontificia Universidad Javeriana with the support of the Memory Clinic Intellectus, Center of Memory and Cognition, and Neuroscience Department. All participants in this study provided their informed consent authorizing their voluntary participation. The project was approved by the Research and Ethics Committee of the School of Medicine at the Pontificia Universidad Javeriana. Int J Geriatr Psychiatry 2014
Spanish MoCA validation in Colombia
Design
Assessment
With the purpose of validating the MoCA-S, an agreement by consensus type of study (Kotter et al., 2011) was planned to compare the execution of the MoCA-S with the clinical diagnosis obtained through consensus of the Memory Clinic Intellectus (the best diagnostic method available). Consequently, the study was intended to evaluate the psychometric properties of the Spanish version of the test. The main objective of the study was to determine whether the MoCA-S instrument was useful for discriminating between cognitively healthy subjects and subjects with cognitive alterations associated with MCI and MD in Colombia.
All subjects took the MoCA-S. Those subjects with cognitive alterations were assessed at the Memory Clinic by an interdisciplinary team comprising specialists in neurology, psychiatry, geriatrics, psychogeriatrics, and neuropsychology who applied the clinical diagnostic criteria for patients with MCI or MD. The diagnoses were supported by the neuropsychological test profiles, functional assessments, and, in most cases, neuroimaging evaluations. No cases of dementia were confirmed by genetic or pathological biomarkers. The specialists applied the following scales: Minimental state exam, Hashinski Scale, Barthel and Lawton Scales, Hamilton Test, Columbia, Yesavage Scale, and Cornell Scale. They also performed a complete general, mental, and neurological exam of the patients. The neuropsychology tests chosen were those recognized as assessing the cognitive functions implicated in MCI and MD. The aim of the clinical battery was to gain information in areas relevant to the determination of MCI and MD, without recourse to information from the neuropsychology battery to permit the independent assessment of each. Additionally, all patients were evaluated by a battery of standardized neuropsychological tests adapted in Colombia (Jacquier et al., 1997), which included some tests from the Consortium to Establish a Registry for AD (Aguirre-Acevedo et al., 2007). The tools included the Verbal Fluency Test, Boston Naming Test, Word List Memory, Constructional Praxis, Word List Recall, Word List Recognition, Recall of Constructional Praxis, and additional tests such as the Grober–Buschke test, Digit Symbol Substitution Test, Frontal Behavioral Inventory Kertesz, INECO Test, Sayings and Similarities Test, and Retrospective Memory Test. The patients were also evaluated by a neurologist, who administered the MoCA-S. The evaluator who administered the MoCA-S did not know the clinical diagnosis or results of the neuropsychological assessment. The neuropsychological assessment was performed before evaluating the patients’ medical record. Both measurement techniques were applied on the same day to ensure that the subjects’ condition had not changed. The healthy subjects were assessed by specialists in neurology and neuropsychology using the same neuropsychological battery.
Subjects
The subjects younger than 65 years old were excluded from the study. Two groups were formed (Table 1). The first group included 109 patients with cognitive impairment, and the second group included 84 healthy subjects recruited from the community who had normal neurological and neuropsychological assessments and no memory complaints. The patient group was subdivided into the MCI subgroup (includes amnestic and multiple-domain MCI), comprised of 26 subjects who met Petersen’s diagnostic criteria for MCI (cognitive complaints and neuropsychological evaluation with an execution of 1.5 standard deviations below the level expected according to age and educational status; Petersen et al., 1999) and a GDS of Reisberg score of 2 or 3 (Reisberg et al., 1982), and the MD subgroup, comprised of 83 subjects who met the clinical criteria for dementia of the DSM-IV (American Psychiatric Association, 2000) and were found to be in the MD stage with a GDS score of 3 or 4 or to have compromises in their advanced and instrumental activities of daily living. Table 1 Demographic features, namely age, gender, and education level, and the corresponding diagnostic percentages Subject demographics
No.
%
Gender
68 125 123 57 13 9 81 39 64 84 26 83 193
35.3 64.7 63.7 29.6 6.7 4.6 42 20.2 33.2 43.5 13.5 43.0 100.0
Age Educational level
Diagnosis
Male Female 65–75 years 76–85 years >86 years 89% and a specificity of >79% were considered to indicate optimal screening performance.
MoCA vs MMSE 37.5
30.
28.6
27.3 25.2
24.5 20.7
22.5
16.1
MMSE MOCA
15.
7.5
0.
Results
Control group
Subject demographics
The MoCA-S was applied to 193 subjects, 64.7% of whom were women, with an age range between 65 and 94 years. Most patients were between 66 and 75 years old. The average educational level of the assessed population was 12 years. See Tables 1 and 2 for detailed demographic information. Within all study subgroups, namely, the control group and MCI and MD patient groups, no differences were found with respect to age (p = 0.33). The educational level was also similar, although the MD group had a significantly lower educational level than the MCI and control groups (p = 0.0004). There were no significant differences between the healthy subjects and the MCI patients (p = 0.10). Such differences in the educational level have been found by other authors (Nasreddine et al., 2005) without invalidating the results. In the group of subjects with cognitive impairment, the average scores were 17.2 on the MoCA-S and 25.2 on the MMSE. The healthy group showed average scores of 25.2 on the MoCA-S and 28.6 on the MMSE. When subdividing the group of subjects with cognitive impairment, the MCI subgroup had an average MoCA-S score of 20.7 and an average MMSE score of 27.3 (normal range). In contrast, the MD subgroup showed an average MoCA-S score of 16.1 (abnormal range) and an average MMSE score of 24.5 (abnormal range; Figure 1). These findings Table 2 Average age and educational level according to diagnosis Demographic features
Subgroup Control MCI MD
Average age (years)
Average educational level (years)
Women (%)
68 (SD: 10.38) 65 (SD: 13.4) 73 (SD: 7.5)
14 (SD: 4.7) 13 (SD: 5.1) 11 (SD: 5.1)
69.4 65.9 57.4
MCI, mild cognitive impairment; MD, mild dementia; SD, standard deviation.
Copyright # 2014 John Wiley & Sons, Ltd.
MCI
MD
Figure 1 Average of the test scores of Montreal Cognitive Assessment Spanish version (MoCA-S) versus Mini-mental state examination (MMSE) in patients with mild dementia (MD) or mild cognitive impairment (MCI) or in healthy subjects (control group).
reveal that the MoCA-S showed a large score difference between the different subgroups, whereas the MMSE remained in the normal range for the MCI subgroup and may remain in the normal range for the MD subgroup. The MoCA-S test score was significantly different between the different subgroups (p < 0.0001). The majority of patients with MD had AD (66.2%) with a GDS score of 3 or 4. Psychometric properties of the Montreal Cognitive Assessment Scale assessment. The inter-observer reliability was assessed through the application of the test on the first 10 patients in the study by two independent and blinded evaluators. A Lin correlation of coefficient and concordance of 0.093 was obtained with a 95% CI of [0.78, 1.00]. These correlation and concordance coefficients are considered almost perfect. The translation of the scale to Spanish was evaluated, and it was found to meet several requirements that support its construct validity. It has an adequate appearance and content validity. Furthermore, its internal consistency was assessed through the Cronbach alpha index, which yielded a score of 0.8463 for the different items assessed. This result is similar to that obtained by the authors in the validation of the original scale. The criterion validity of the test was assessed by comparing the result of the Spanish translation of the MoCA with the interdisciplinary diagnosis of the Memory Clinic, and good agreement between both measurements was found (kappa of 0.69; 95% CI [0.59, 0.80]). Additionally, an analysis of the variance was performed to correlate the score averages of the MoCA-S between the three different subgroups, namely, Int J Geriatr Psychiatry 2014
Spanish MoCA validation in Colombia
the control, MCI, and MD groups, considering that the scores differed by group. A significant difference was found between the three subgroups (p < 0.0001). Finally, a strong correlation with the MMSE was found through the Spearman correlation index (0.755; 95% CI [0.687, 0.811]). This result demonstrates that the scale represents a good measurement of the cognitive performance of these subjects. Agreement of the Montreal Cognitive Assessment in Spanish with the results of the interdisciplinary assessment by the Memory Clinic: sensitivity and specificity
The sensitivities, specificities, positive predictive value, negative predictive value, and accuracy of the MoCA-S at different cutoff values are presented in Table 3. The agreement of the classification of the subjects according to the results of the MoCA-S test, considering a healthy patient cutoff score of ≥23, with the diagnosis by the Memory Clinic was adequate (kappa 0.69; 95% CI [0.59, 0.80]). The percentage of patients correctly classified by the MoCA-S was 85%. The ROC curve is presented in Figure 2, and the area under the ROC curve was 0.93 (95% CI [0.89, 0.96]; Figure 2). The sensitivity and specificity of the test for the population sample were optimized when using a cutoff score of ≥23 (any score of ≤22 was considered to be an abnormal result). With this cutoff score, a better balance between sensitivity and specificity was achieved. The sensitivity of the test to detect cognitive impairment (MCI or MD) was 89%, with a specificity of 79.8%. Similarly, the positive predictive value was 85%, and the negative predictive value was 84.8%. As the cutoff score was increased to 24 or higher, the sensitivity improved, but the specificity worsened because many healthy patients were then classified as
Figure 2 Receiver operating characteristic (ROC) curve analysis of the Spanish version of the Montreal Cognitive Assessment for the detection of normal cognitive and mild cognitive impairment/mild dementia. CI, confidence interval.
ill by the MoCA-S. In contrast, if the cutoff point was decreased to 22 or less, the sensitivity decreased, allowing more patients with cognitive impairment to be classified as normal by the MoCA-S (Table 3). When the sensitivity calculation was performed only for the subgroup of subjects with MD, a sensitivity of 92.7% was established, with only six false positives. These cases included three patients with AD, two patients with behavioral variant frontotemporal dementia, and one patient with mixed dementia. This cutoff value guarantees a low possibility of the test to classify a subject with MD as normal. Effect of educational level
The educational level has an effect on test performance (Table 4). The scores of MoCA and MMSE are dropping along with the education level in the three
Table 3 Sensitivity, specificity, positive predictive value (PPV), negative predictive value (NPV), and accuracy of the Spanish version Montreal Cognitive Assessment for detection of normal cognitive and mild cognitive impairment/mild dementia Cutoff ≥19 ≥20 ≥21 ≥22 ≥23 ≥24 ≥25 ≥26 ≥27 ≥28 ≥29 30
Sensitivity (%)
Specificity (%)
PPV (%)
NPV (%)
Accuracy (%)
56.9 62.4 72.5 80.7 89.0 93.6 96.3 99.1 99.1 100.0 100.0 100.0
97.6 95.2 88.1 84.5 79.8 76.2 61.9 52.4 35.7 20.2 13.1 10.7
96.8 94.4 88.7 87.1 85 83.6 76.6 72.2 66.6 59.8 59.8 59.2
44.2 71.1 31.2 77.1 84.8 90.1 92.8 97.7 96.7 100.0 100.0 100.0
74.6 76.7 79.3 82.4 85.0 86.0 81.3 78.8 71.5 65.3 62.2 56.5
Bold values indicate cutoff point used in the study with values of sensitivity and specificity.
Copyright # 2014 John Wiley & Sons, Ltd.
Int J Geriatr Psychiatry 2014
L. Gil et al. Table 4 Average MoCA (without correction) and MMSE scores according to educational level and diagnostic group Education level
MoCA MMSE
Control MCI MD Control MCI MD
