ORIGINAL ARTICLE
Comparison of an Electronic and Paper-based Montreal Cognitive Assessment Tool Anne Snowdon, PhD,* Abdulkadir Hussein, PhD,w Robert Kent, PhD,z Lou Pino, PhD,* and Vladimir Hachinski, MD, FRCPy
Abstract: This pilot study compared a novel electronic Montreal Cognitive Assessment (eMoCA) tool to the original paper-based MoCA. Potential participants were approached at primary care practices, a geriatric day hospital, and a university campus. Each of the 401 participants were randomly assigned to either the eMoCA (N = 182) or MoCA (N = 219). Scores were adjusted by selfreported demographic and health information using regression analysis. The difference in average scores (26.21 ± 3.11 for the MoCA group and 24.84 ± 4.21 for the eMoCA group) was found to be statistically significant. Controlling for the effect of potential covariate factors with regression analyses, the adjusted difference is 0.90 (95% confidence interval, 1.45 to 0.35). This difference may be due to factors related to use of the electronic device or software usability. However, the standardized, self-administered eMoCA may offer an opportunity for health systems to screen for early changes in cognitive function in primary care settings and offer greater access to assessment for rural or remote communities. Population-level research may be required to identify whether the score difference between test versions requires a downward adjustment to the eMoCA score taken as indicative of cognitive impairment. Key Words: Montreal Cognitive Assessment, cognitive impairment, dementia, electronic screening
(Alzheimer Dis Assoc Disord 2015;29:325–329)
I
t is estimated that the number of Canadians living with cognitive impairment, including dementia, could double to 1.4 million by 2031.1 Most patients with cognitive impairment are referred to memory clinics, geriatricians, or neuropsychologists. However, specialist care and diagnosis may be delayed as wait times of 6 to 12 months are not uncommon.2 Early detection of mild cognitive impairment (MCI) is particularly relevant for vascular dementia as it may be possible to treat and prevent the progression of vascular changes in the brain that are commonly associated with hypertension and diabetes.3 Vascular dementia is the second most prevalent form of dementia after Alzheimer disease and represents up to 20% of all cases.3 Early prevention and screening for vascular dementia offers patients the opportunity to slow the progression of cognitive changes associated with deterioration in vascular Received for publication May 5, 2014; accepted October 7, 2014. From the *Ivey Business School; yDepartment of Clinical Neurological Sciences, Schulich Medical School, Western University, London; Departments of wMathematics and Statistics; and zComputer Science, University of Windsor, Windsor, ON, Canada. Supported by the Alzheimer Society of Canada. The authors declare no conflicts of interest. Reprints: Anne Snowdon, PhD, Ivey School of Business, Western University, 1151 Richmond St, London, ON, Canada N6A 3K7 (e-mail: [email protected]). Copyright r 2014 Wolters Kluwer Health, Inc. All rights reserved.
Alzheimer Dis Assoc Disord
health. Primary care providers are well positioned to screen for early signs of dementia in their patients; however, some studies report that >60% of patients with dementia remain undiagnosed in early phases of the disease.4,5 Early diagnosis with appropriate interventions may reduce cognitive deterioration and lessen disease burden.6–8 In a survey conducted by Iracleous et al9 of Canadian physicians, 89% of participants “Agreed” or “Strongly Agreed” that cognitive impairment assessment in primary care is important. They further identified that the key attributes of an ideal cognitive assessment tool include validity/accuracy, ease of administration, time required to complete, and acceptability to patients.9 Digital health assessment tools are evolving quickly in health systems as health information technologies advance and offer patients increasing opportunity to manage their health and wellness using online or digital health assessment tools. Screening tools that are easy to use and interface with electronic medical records are emerging in primary care clinical settings. Digital assessment tools may be particularly suited to screening in primary care settings as it has the potential to meet these key attributes while offering the potential for greater efficiency related to reduced demands on health providers to administer paper-based tools such as the Montreal Cognitive Assessment (MoCA).10 In addition, digital assessment tools offer the advantage of standardizing the administration and data collection processes; offers reliable and randomized presentation of stimuli over multiple trials and repeat administrations; and unobtrusive measurement of cognitive skills and response times during all aspects of the assessment process.11 To examine the validity and reliability of a digital assessment tool, particularly one for which the end-users are the elderly population, the ease of use and user experience must be carefully considered. Several studies have shown that older adults rate computer-based tests as understandable and easy to use,12 and more acceptable than paper and pencil tests.13 Accuracy of the analysis of test outcomes using digital tools may offer when advantage as digital algorithms are built into the software to automatically score the results of the testing, and offer insights into trending of results when testing is repeated over time. Digital assessment tools may also have the added advantage of standardizing administration of the tool due to the reduction in variation in implementation (eg, health care professionals’ body language that elicits certain responses or occasional “helpful hints” offered to patients when completing the assessment tool. MoCA is a simple and effective test for detecting MCI.14–16 It is a paper-based assessment tool that is typically administered by a clinician and takes approximately 10 minutes to complete. A cognitively healthy score on this 30-point test is considered to be Z26.14 An electronic tablet-based version of this assessment tool (eMoCA) was
Volume 29, Number 4, October–December 2015
www.alzheimerjournal.com |
Copyright r 2015 Wolters Kluwer Health, Inc. All rights reserved.
325
Alzheimer Dis Assoc Disord
Snowdon et al
developed using a prerecorded voice designed to support self-administration of the assessment tool, using verbal instructions and an electronic stylus to engage the patient in responding to questions and drawing pictures required to respond to questions on the MoCA instrument. We hypothesized that there would be no statistically significant difference between eMoCA or MoCA participant scores. In this study, 401 volunteers were invited to complete either the electronic or paper-based test versions and differences in their scores were analyzed.
Volume 29, Number 4, October–December 2015
w2 tests were used for comparing the 2 groups for categorical variables. For continuous variables, means were compared using a 2-sample t test. Education was grouped into the following categories: less than high school (12 y). Age cohorts for the analysis were 20 years and younger, 21 to 40 years, 41 to 60 years, and 60 years and older. Regression analyses were carried out to control for the effect of demographic and health-related variables for each of the sample groups. Initially, the SAS software’s
METHODS The eMoCA tool was designed in collaboration with Dr Ziad Nasreddine to emulate the MoCA tool,12 for use on android tablets. The application featured prerecorded verbal instructions accompanied by on-screen instructions and practice questions using the stylus to support selfassessment by patients. The tablet records all of the participants’ responses and after the assessment has been completed, a trained individual can assign a score based on the established 30-point scale developed for the MoCA. Participants were recruited for this study from 9 primary care practices in Ontario, a geriatric day hospital in Nova Scotia and a University campus in Ontario. Multiple testing locations enabled access to a variety of participants with a wide range of demographic characteristics. All participants were over the age of 18 years and fluent in English. Those with significant sight or hearing challenges were excluded from the study. The eMoCA and MoCA were administered to participants randomly assigned to eMoCA and paperbased MoCA groups, at each location. The study took place between July 7, 2011 and September 30, 2012 and was approved by the Western University Research Ethics Board. A demographic questionnaire was built into the eMoCA tool that asked participants questions about personal and family history, demographics (eg, age, sex, education level), current medications, lifestyle (eg, alcohol and tobacco use, physical activity), and comorbidities including diagnoses of dementia, high blood pressure, diabetes, history of stroke/cardiovascular disease, and high blood lipids. Once the participant completed the demographic questions on the tablet, the software randomly assigned the participant to either the MoCA or eMoCA group. Participants randomized to the eMoCA were provided the tablet and independently completed the eMoCA tool using the device. Research assistants were nearby to respond to questions if needed. For the participants randomized to the paper version of the MoCA, research assistants administered the MoCA tool, by following a script which was identical to the instructions built into the eMoCA, to minimize variance in participant instruction across the 2 test groups. After test completion, participants were given their MoCA scores and directed to speak with their primary care provider about any concerns. Time required to complete the MoCA tool was recorded. The software recorded the time required to complete the test for the eMoCA and the research assistant recorded the start time and completion time for each participant completing the paper version of the MoCA.
Statistical Analyses Descriptive statistics for demographic and health-related variables were analyzed using the SPSS software. Table 1 reports univariate comparisons of the demographic and health-related variables for each of the 2 testing regimes.
TABLE 1. Demographic Characteristics
MoCA (N = 219) Final score [mean (SD)] 26.2 Time to complete test [mean 10.27 (SD)] (min) Age [mean (SD)] 51.6 Age categories [N (%)] r20 27 21-40 45 41-60 62 Z61 85 Sex [N (%)] Female 123 Male 88 Marital status [N (%)] Single 67 Married 105 Divorced 20 Widowed 19 Household income [N (%)] < $20K 37 $20K-$40K 44 $40K-$60K 28 $60-$80K 33 > $80K 32 Education [N (%)] Grade school 3 Some high school 10 High school graduate 67 Some post-high school 24 College 36 University degree 70 Less than high school 13 More than high school 130 Frequency of computer use [N (%)] Never 11 Rarely 15 Sometimes 10 Frequently 175 Cerebrovascular risk factors [N (%)] Normal 91 Risk factor(s) 128 Depression (2-item score r5) [N (%)] Normal 163 Depressed 51 Smoker [N (%)] Smoker 20 Nonsmoker 188
(3.0) (3.1) (20.8) (12.3) (20.6) (28.3) (38.8) (58.3) (41.7)
eMoCA (N = 182)
P
24.8 (4.2) 15.45 (3.6)
0.00017 0.00001
51.7 (21.2) 15 49 49 69
(8.24) (26.9) (26.9) (37.9)
115 (65.0) 62 (35)
0.946 0.334
0.178
(31.8) (49.8) (9.5) (9)
59 78 20 20
(33.3) (44.6) (11.3) (11.3)
0.671
(21.3) (25.3) (16.1) (19.0) (18.4)
41 35 27 15 35
(26.8) (22.9) (17.6) (9.8) (22.9)
(1.4) (4.8) (31.9) (11.4) (17.1) (33.3) (6.2) (61.8)
4 13 53 20 31 56 17 107
(2.3) (7.3) (29.9) (11.3) (17.5) (31.6) (9.6) (60.4)
(5.2) (7.1) (4.7) (82.9)
13 8 19 138
(7.3) (4.5) (10.7) (77.5)
0.051
(41.6) (58.4)
71 (39.0) 111 (61.0)
0.606
(76.2) (23.8)
136 (74.7) 46 (25.3)
0.739
(9.6) (90.4)
20 (11.4) 155 (88.6)
0.563
0.146
0.894
0.433
eMoCA indicates electronic Montreal Cognitive Assessment; MoCA, Montreal Cognitive Assessment.
326 | www.alzheimerjournal.com Copyright r 2014 Wolters Kluwer Health, Inc. All rights reserved. Copyright r 2015 Wolters Kluwer Health, Inc. All rights reserved.
Alzheimer Dis Assoc Disord
Volume 29, Number 4, October–December 2015
GLMSELECT procedure was used to select the best subset of variables in Table 1 and their interactions in a linear regression model. To accommodate the skewed distribution of scores and down weight any extreme outliers, a robust regression procedure was used to compare the final model with a model that used all the variables reported in Table 1. The final model fitted the data satisfactorily (Table 2).
Comparison of an Electronic and Paper-based MoCA
still estimated at 0.90 (95% confidence interval, 1.45 to 0.35; P = 0.9844). That is, if 2 individuals measured the same for all other factors that appear in Table 2, but differed only in the test type, then the individual taking the electronic version of MoCA is expected to have a score approximately 1 unit lower than participants who completed the paper version of the MoCA.
RESULTS
DISCUSSION
A total of 401 participants were recruited for the study with 268 from primary care practices, 19 from the geriatric day hospital and 114 from the university. A baseline comparison of the 2 groups of participants showed no statistically significant differences in demographic characteristics, or/and other self-reported measures between the 2 test type populations, indicating that randomization was effective (Table 1). Of the 182 participants who took the eMoCA, 46.7% had a final score of
Comparison of an Electronic and Paper-based Montreal Cognitive Assessment Tool Anne Snowdon, PhD,* Abdulkadir Hussein, PhD,w Robert Kent, PhD,z Lou Pino, PhD,* and Vladimir Hachinski, MD, FRCPy
Abstract: This pilot study compared a novel electronic Montreal Cognitive Assessment (eMoCA) tool to the original paper-based MoCA. Potential participants were approached at primary care practices, a geriatric day hospital, and a university campus. Each of the 401 participants were randomly assigned to either the eMoCA (N = 182) or MoCA (N = 219). Scores were adjusted by selfreported demographic and health information using regression analysis. The difference in average scores (26.21 ± 3.11 for the MoCA group and 24.84 ± 4.21 for the eMoCA group) was found to be statistically significant. Controlling for the effect of potential covariate factors with regression analyses, the adjusted difference is 0.90 (95% confidence interval, 1.45 to 0.35). This difference may be due to factors related to use of the electronic device or software usability. However, the standardized, self-administered eMoCA may offer an opportunity for health systems to screen for early changes in cognitive function in primary care settings and offer greater access to assessment for rural or remote communities. Population-level research may be required to identify whether the score difference between test versions requires a downward adjustment to the eMoCA score taken as indicative of cognitive impairment. Key Words: Montreal Cognitive Assessment, cognitive impairment, dementia, electronic screening
(Alzheimer Dis Assoc Disord 2015;29:325–329)
I
t is estimated that the number of Canadians living with cognitive impairment, including dementia, could double to 1.4 million by 2031.1 Most patients with cognitive impairment are referred to memory clinics, geriatricians, or neuropsychologists. However, specialist care and diagnosis may be delayed as wait times of 6 to 12 months are not uncommon.2 Early detection of mild cognitive impairment (MCI) is particularly relevant for vascular dementia as it may be possible to treat and prevent the progression of vascular changes in the brain that are commonly associated with hypertension and diabetes.3 Vascular dementia is the second most prevalent form of dementia after Alzheimer disease and represents up to 20% of all cases.3 Early prevention and screening for vascular dementia offers patients the opportunity to slow the progression of cognitive changes associated with deterioration in vascular Received for publication May 5, 2014; accepted October 7, 2014. From the *Ivey Business School; yDepartment of Clinical Neurological Sciences, Schulich Medical School, Western University, London; Departments of wMathematics and Statistics; and zComputer Science, University of Windsor, Windsor, ON, Canada. Supported by the Alzheimer Society of Canada. The authors declare no conflicts of interest. Reprints: Anne Snowdon, PhD, Ivey School of Business, Western University, 1151 Richmond St, London, ON, Canada N6A 3K7 (e-mail: [email protected]). Copyright r 2014 Wolters Kluwer Health, Inc. All rights reserved.
Alzheimer Dis Assoc Disord
health. Primary care providers are well positioned to screen for early signs of dementia in their patients; however, some studies report that >60% of patients with dementia remain undiagnosed in early phases of the disease.4,5 Early diagnosis with appropriate interventions may reduce cognitive deterioration and lessen disease burden.6–8 In a survey conducted by Iracleous et al9 of Canadian physicians, 89% of participants “Agreed” or “Strongly Agreed” that cognitive impairment assessment in primary care is important. They further identified that the key attributes of an ideal cognitive assessment tool include validity/accuracy, ease of administration, time required to complete, and acceptability to patients.9 Digital health assessment tools are evolving quickly in health systems as health information technologies advance and offer patients increasing opportunity to manage their health and wellness using online or digital health assessment tools. Screening tools that are easy to use and interface with electronic medical records are emerging in primary care clinical settings. Digital assessment tools may be particularly suited to screening in primary care settings as it has the potential to meet these key attributes while offering the potential for greater efficiency related to reduced demands on health providers to administer paper-based tools such as the Montreal Cognitive Assessment (MoCA).10 In addition, digital assessment tools offer the advantage of standardizing the administration and data collection processes; offers reliable and randomized presentation of stimuli over multiple trials and repeat administrations; and unobtrusive measurement of cognitive skills and response times during all aspects of the assessment process.11 To examine the validity and reliability of a digital assessment tool, particularly one for which the end-users are the elderly population, the ease of use and user experience must be carefully considered. Several studies have shown that older adults rate computer-based tests as understandable and easy to use,12 and more acceptable than paper and pencil tests.13 Accuracy of the analysis of test outcomes using digital tools may offer when advantage as digital algorithms are built into the software to automatically score the results of the testing, and offer insights into trending of results when testing is repeated over time. Digital assessment tools may also have the added advantage of standardizing administration of the tool due to the reduction in variation in implementation (eg, health care professionals’ body language that elicits certain responses or occasional “helpful hints” offered to patients when completing the assessment tool. MoCA is a simple and effective test for detecting MCI.14–16 It is a paper-based assessment tool that is typically administered by a clinician and takes approximately 10 minutes to complete. A cognitively healthy score on this 30-point test is considered to be Z26.14 An electronic tablet-based version of this assessment tool (eMoCA) was
Volume 29, Number 4, October–December 2015
www.alzheimerjournal.com |
Copyright r 2015 Wolters Kluwer Health, Inc. All rights reserved.
325
Alzheimer Dis Assoc Disord
Snowdon et al
developed using a prerecorded voice designed to support self-administration of the assessment tool, using verbal instructions and an electronic stylus to engage the patient in responding to questions and drawing pictures required to respond to questions on the MoCA instrument. We hypothesized that there would be no statistically significant difference between eMoCA or MoCA participant scores. In this study, 401 volunteers were invited to complete either the electronic or paper-based test versions and differences in their scores were analyzed.
Volume 29, Number 4, October–December 2015
w2 tests were used for comparing the 2 groups for categorical variables. For continuous variables, means were compared using a 2-sample t test. Education was grouped into the following categories: less than high school (12 y). Age cohorts for the analysis were 20 years and younger, 21 to 40 years, 41 to 60 years, and 60 years and older. Regression analyses were carried out to control for the effect of demographic and health-related variables for each of the sample groups. Initially, the SAS software’s
METHODS The eMoCA tool was designed in collaboration with Dr Ziad Nasreddine to emulate the MoCA tool,12 for use on android tablets. The application featured prerecorded verbal instructions accompanied by on-screen instructions and practice questions using the stylus to support selfassessment by patients. The tablet records all of the participants’ responses and after the assessment has been completed, a trained individual can assign a score based on the established 30-point scale developed for the MoCA. Participants were recruited for this study from 9 primary care practices in Ontario, a geriatric day hospital in Nova Scotia and a University campus in Ontario. Multiple testing locations enabled access to a variety of participants with a wide range of demographic characteristics. All participants were over the age of 18 years and fluent in English. Those with significant sight or hearing challenges were excluded from the study. The eMoCA and MoCA were administered to participants randomly assigned to eMoCA and paperbased MoCA groups, at each location. The study took place between July 7, 2011 and September 30, 2012 and was approved by the Western University Research Ethics Board. A demographic questionnaire was built into the eMoCA tool that asked participants questions about personal and family history, demographics (eg, age, sex, education level), current medications, lifestyle (eg, alcohol and tobacco use, physical activity), and comorbidities including diagnoses of dementia, high blood pressure, diabetes, history of stroke/cardiovascular disease, and high blood lipids. Once the participant completed the demographic questions on the tablet, the software randomly assigned the participant to either the MoCA or eMoCA group. Participants randomized to the eMoCA were provided the tablet and independently completed the eMoCA tool using the device. Research assistants were nearby to respond to questions if needed. For the participants randomized to the paper version of the MoCA, research assistants administered the MoCA tool, by following a script which was identical to the instructions built into the eMoCA, to minimize variance in participant instruction across the 2 test groups. After test completion, participants were given their MoCA scores and directed to speak with their primary care provider about any concerns. Time required to complete the MoCA tool was recorded. The software recorded the time required to complete the test for the eMoCA and the research assistant recorded the start time and completion time for each participant completing the paper version of the MoCA.
Statistical Analyses Descriptive statistics for demographic and health-related variables were analyzed using the SPSS software. Table 1 reports univariate comparisons of the demographic and health-related variables for each of the 2 testing regimes.
TABLE 1. Demographic Characteristics
MoCA (N = 219) Final score [mean (SD)] 26.2 Time to complete test [mean 10.27 (SD)] (min) Age [mean (SD)] 51.6 Age categories [N (%)] r20 27 21-40 45 41-60 62 Z61 85 Sex [N (%)] Female 123 Male 88 Marital status [N (%)] Single 67 Married 105 Divorced 20 Widowed 19 Household income [N (%)] < $20K 37 $20K-$40K 44 $40K-$60K 28 $60-$80K 33 > $80K 32 Education [N (%)] Grade school 3 Some high school 10 High school graduate 67 Some post-high school 24 College 36 University degree 70 Less than high school 13 More than high school 130 Frequency of computer use [N (%)] Never 11 Rarely 15 Sometimes 10 Frequently 175 Cerebrovascular risk factors [N (%)] Normal 91 Risk factor(s) 128 Depression (2-item score r5) [N (%)] Normal 163 Depressed 51 Smoker [N (%)] Smoker 20 Nonsmoker 188
(3.0) (3.1) (20.8) (12.3) (20.6) (28.3) (38.8) (58.3) (41.7)
eMoCA (N = 182)
P
24.8 (4.2) 15.45 (3.6)
0.00017 0.00001
51.7 (21.2) 15 49 49 69
(8.24) (26.9) (26.9) (37.9)
115 (65.0) 62 (35)
0.946 0.334
0.178
(31.8) (49.8) (9.5) (9)
59 78 20 20
(33.3) (44.6) (11.3) (11.3)
0.671
(21.3) (25.3) (16.1) (19.0) (18.4)
41 35 27 15 35
(26.8) (22.9) (17.6) (9.8) (22.9)
(1.4) (4.8) (31.9) (11.4) (17.1) (33.3) (6.2) (61.8)
4 13 53 20 31 56 17 107
(2.3) (7.3) (29.9) (11.3) (17.5) (31.6) (9.6) (60.4)
(5.2) (7.1) (4.7) (82.9)
13 8 19 138
(7.3) (4.5) (10.7) (77.5)
0.051
(41.6) (58.4)
71 (39.0) 111 (61.0)
0.606
(76.2) (23.8)
136 (74.7) 46 (25.3)
0.739
(9.6) (90.4)
20 (11.4) 155 (88.6)
0.563
0.146
0.894
0.433
eMoCA indicates electronic Montreal Cognitive Assessment; MoCA, Montreal Cognitive Assessment.
326 | www.alzheimerjournal.com Copyright r 2014 Wolters Kluwer Health, Inc. All rights reserved. Copyright r 2015 Wolters Kluwer Health, Inc. All rights reserved.
Alzheimer Dis Assoc Disord
Volume 29, Number 4, October–December 2015
GLMSELECT procedure was used to select the best subset of variables in Table 1 and their interactions in a linear regression model. To accommodate the skewed distribution of scores and down weight any extreme outliers, a robust regression procedure was used to compare the final model with a model that used all the variables reported in Table 1. The final model fitted the data satisfactorily (Table 2).
Comparison of an Electronic and Paper-based MoCA
still estimated at 0.90 (95% confidence interval, 1.45 to 0.35; P = 0.9844). That is, if 2 individuals measured the same for all other factors that appear in Table 2, but differed only in the test type, then the individual taking the electronic version of MoCA is expected to have a score approximately 1 unit lower than participants who completed the paper version of the MoCA.
RESULTS
DISCUSSION
A total of 401 participants were recruited for the study with 268 from primary care practices, 19 from the geriatric day hospital and 114 from the university. A baseline comparison of the 2 groups of participants showed no statistically significant differences in demographic characteristics, or/and other self-reported measures between the 2 test type populations, indicating that randomization was effective (Table 1). Of the 182 participants who took the eMoCA, 46.7% had a final score of
