Article
Assessment of Instrumental Activities of Daily Living in Dementia: Diagnostic Value of the Amsterdam Instrumental Activities of Daily Living Questionnaire
Journal of Geriatric Psychiatry and Neurology 26(4) 244-250 ª The Author(s) 2013 Reprints and permission: sagepub.com/journalsPermissions.nav DOI: 10.1177/0891988713509139 jgpn.sagepub.com
Sietske A. M. Sikkes, PhD1,2,3, Yolande A. L. Pijnenburg, MD, PhD1,3, Dirk L. Knol, PhD2, Elly S. M. de Lange-de Klerk, MD, PhD2, Philip Scheltens, MD, PhD1,3, and Bernard M. J. Uitdehaag, MD, PhD2,3
Abstract Background: Measuring impairments in ‘‘instrumental activities of daily living’’ (IADL) is important in dementia, but challenging due to the lack of reliable and valid instruments. We recently developed the Amsterdam Instrumental Activities of Daily Living Questionnaire (A-IADL-Q; note 1). We aim to investigate the diagnostic accuracy of the A-IADL-Q for dementia in a memory clinic setting. Methods: Patients visiting the Alzheimer Center of the VU University Medical Center with their informants between 2009 and 2011 were included (N ¼ 278). Diagnoses were established in a multidisciplinary consensus meeting, independent of the A-IADL-Q scores. An optimal A-IADL-Q cutoff point was determined, and sensitivity and specificity were calculated. Area under the curves (AUCs) were compared between A-IADL-Q and ‘‘disability assessment of dementia’’ (DAD). The additional diagnostic value of the A-IADL-Q to Mini-Mental State Examination (MMSE) was examined using logistic regression analyses. Results: Dementia prevalence was 50.5%. Overall diagnostic accuracy based on the AUC was 0.75 (95% confidence interval [CI]: 0.70-0.81) for the A-IADL-Q and 0.70 (95% CI: 0.63-0.77) for the DAD, which did not differ significantly. The optimal cutoff score for the A-IADL-Q was 51.4, resulting in sensitivity of 0.74 and specificity of 0.64. Combining the A-IADL-Q with the MMSE improved specificity (0.94), with a decline in sensitivity (0.55). Logistic regression models showed that adding A-IADL-Q improved the diagnostic accuracy (Z ¼ 2.55, P ¼ .011), whereas the DAD did not. Conclusions: In this study, we showed a fair diagnostic accuracy for A-IADL-Q and an additional value in the diagnosis of dementia. These results support the role of A-IADLQ as a valuable diagnostic tool. Keywords dementia, sensitivity and specificity, Alzheimer disease, activities of daily living, questionnaire, informant, diagnosis Received April 17, 2013. Received revised August 20, 2013. Accepted for publication September 2, 2013.
Introduction In dementia, interference in everyday functioning is generally measured using informant-based questionnaires aimed at instrumental activities in daily living (IADL).1 Because this interference in everyday functioning is part of the diagnostic criteria of dementia,2 it is important to measure IADL optimally, in a reliable and valid way. However, in several reviews, it was found that the quality of the currently used informant-based IADL questionnaires was limited.3-5 Based on this observation, we started with the development of the Amsterdam IADL Questionnaire (A-IADL-Q; note 1) with input from patients, caregivers, neurologists, geriatricians, specialist nurses, occupational therapists, neuropsychologists, and epidemiologists.6 Activities were chosen to be suitable for both
men and women and for different age groups. In addition, more up-to-date items, such as items related to everyday technology use (computer use and mobile phone use), were included. In a
1
Alzheimer Center, VU University Medical Center, Amsterdam, the Netherlands 2 Department of Epidemiology and Biostatistics, VU University Medical Center, Amsterdam, the Netherlands 3 Department of Neurology, VU University Medical Center, Amsterdam, the Netherlands Corresponding Author: Sietske A. M. Sikkes, Alzheimer Center, VU University Medical Center, PK-1Z035, PO Box 7057, 1007 MB Amsterdam, the Netherlands. Email: [email protected]
Downloaded from jgp.sagepub.com at GEORGIAN COURT UNIV on March 18, 2015
Sikkes et al
245
Figure 1. Study flow.
previous study, we demonstrated good test–retest reliability, content validity, construct validity, and high internal consistency.6,7 In the current study, we aim to investigate the diagnostic usefulness of this newly developed questionnaire. The measurement of IADL has been widely investigated as a possible screening tool for dementia, either in the general population or in the general practice.8-13 However, only a limited number of studies report on the diagnostic accuracy of IADL in a memory clinic setting despite the widespread use. The results of these diagnostic accuracy studies diverge, possibly explained by differences in study design.14-16 In particular, the inclusion of
healthy controls as a reference group might have led to an overestimation of diagnostic accuracy.14 In the current study, we chose a pragmatic design to investigate the diagnostic accuracy by including all consecutive patients visiting a memory clinic. We hypothesized that the A-IADL-Q would be able to discriminate between patients with and patients without dementia. Second, we expected the A-IADL-Q to be a better discriminator than a traditional IADL measure. Third, we hypothesized that the A-IADL-Q would have an additional diagnostic effect on the Mini-Mental State Examination (MMSE) and sociodemographic characteristics.
Downloaded from jgp.sagepub.com at GEORGIAN COURT UNIV on March 18, 2015
246
Journal of Geriatric Psychiatry and Neurology 26(4)
Methods All consecutive informants of patients who visited the Alzheimer Center of the VU University Medical Center for dementia screening were enrolled during the following 3 periods due to practical reasons: October 2009 to May 2010, January to April 2011, and January to February 2012. The eligibility criteria were (1) the presence of an informal informant, (2) the ability of the informant to complete the questionnaire, and (3) an established diagnosis. Figure 1 shows the number of patients included and excluded in the current study. The study was approved by the ethics committee of the VU University Medical Center. All patients gave written informed consent, and all informants gave oral informed consent prior to participation in this study.
Materials Diagnostic Process Patients underwent a standardized dementia assessment including clinical history, medical and neurological examination, routine blood tests to rule out treatable causes, a neuropsychological test battery, magnetic resonance imaging, and electroencephalogram. Clinical diagnosis was established in a multidisciplinary consensus meeting according to the standard diagnostic criteria.2,17-21 Participants of this meeting were unaware of the A-IADL-Q results. However, they were aware of the results of the disability assessment of dementia (DAD). All diagnostic tests (both index and reference tests) were administered on the same day.
The A-IADL-Q The A-IADL-Q is a disease-specific IADL questionnaire, aimed at measuring IADL problems in early dementia. The questionnaire was self-administered by the informant on a tablet computer. We used 2 Web-based platforms for the data collection, Examine22 for the data collection until 2011 and Qualtrics (Qualtrics Labs Inc, Provo, Utah) for the data collection in 2012. It is a branched questionnaire, with a minimum of 47 and a maximum of 70 activities. As a result, no detailed questions are asked when the patient does not perform a specific activity.6 Each item has a 5-point scale response option (scored 0-4). The scoring of the AIADL-Q is calculated using item response theory as described in detail elsewhere.7 The calculated total score for the A-IADL-Q is a y score with a mean of 0 and standard deviation (SD) of 1. This score is transformed to a mean of 50 and SD of 10 (10 y þ 50), resulting in a scoring range from 20 to 80, with lower scores indicating poorer performance.
Disability Assessment for Dementia (DAD) The DAD was reviewed in a systematic review as having reasonable psychometric properties,3 and we therefore chose to compare the A-IADL-Q with the DAD. The DAD is a disease-specific interview-based questionnaire aimed at evaluating functional
disability in community-dwelling persons with Alzheimer disease.23 The DAD consists of 40 items, related to both IADL and basic activities of daily living. A total weighted percentage score is calculated by adding item scores and dividing this by the number of items answered, excluding the nonapplicable answers. Lower scores indicate more ADL and IADL dysfunction. A specialist nurse and research associate conducted the DAD interview.
Statistical Analysis We used SPSS (IBM SPSS Statistics for Windows, Version 20.0; Armonk, New York), Mplus,24 and R25 for data analyses. Patient and informant characteristics between different diagnostic groups were compared using t tests, analysis of variance, chi-square tests, or Mann-Whitney U tests as appropriate. We compared A-IADL-Q scores between patients with and without dementia using an independent t test. To investigate the diagnostic accuracy, receiver–operating characteristic (ROC) curves were created for both the A-IADL-Q and the DAD. Area under the curves (AUCs) were calculated with 95% confidence intervals (CIs) as a further measure of diagnostic accuracy. For an AUC, a value of 0.5 indicates a random guess and a value of 1 perfect classification. The AUCs were further classified as having low accuracy (values between .50 and .70), moderate accuracy (values between .71 and .90), and high accuracy (values higher than .91).26 Differences between ROC curves were tested using the method of DeLong et al27 incorporated in R.28 For the A-IADL-Q, the best possible cutoff score was determined based on the Youden index, and sensitivity and specificity values were calculated. The A-IADL-Q was combined with the MMSE,29 both in series (positive result if both tests are positive) and in parallel (positive result if one of the tests is positive). For tests in series and parallel, sensitivity, specificity, likelihood ratios, and diagnostic odds ratios were calculated. Finally, we used logistic regression models to investigate the additional diagnostic value of the A-IADL-Q. Two models were tested, one with basic clinical and demographic information (MMSE, age, sex, and education) and the second model combining the clinical and demographic information with the A-IADL-Q. The ROC curves were created for both the models, and AUCs were compared using the DeLong method in R.27,28 The significance level was set at P < .05.
Results A total of 278 informants of patients completed the A-IADL-Q. Patient were aged 27 to 86 years (M ¼ 63.9, SD ¼ 9.5), and 140 (50.5%) patients were diagnosed with dementia. Alzheimer disease was the most common type of dementia (n ¼ 105, 75.0%), followed by frontotemporal lobe dementia (n ¼ 17, 12.1%) and Lewy Body dementia (n ¼ 11, 7.9%). The remaining 138 (49.6%) patients were diagnosed as ‘‘not demented.’’ Figure 1 shows the diagnoses for all patients included in this study, and Table 1 shows the patient and informant characteristics. As expected, patients with dementia were older (258.8) ¼ 3.05, P ¼ .003, and had lower MMSE scores (Z ¼ 10.35, P < .001).
Downloaded from jgp.sagepub.com at GEORGIAN COURT UNIV on March 18, 2015
Sikkes et al
247
Table 1. Patient and Informant Characteristics of Patients With and Without Dementia.a Patients (N ¼ 278)
Age Female gender Level of educationb, c, d MMSEe Relationship spouse Relationship > 10 yearsf Living togetherg
Informants (N ¼ 278)
Dementia (n ¼ 140)
No Dementia (n ¼ 138)
P Value
65.6 (8.2) 51 (36.4%) 5 (4-6) 22 (17-24)
62.2 (10.5) 41 (29.7%) 5 (4-6) 27 (25-29)
.003 .23 .43
Assessment of Instrumental Activities of Daily Living in Dementia: Diagnostic Value of the Amsterdam Instrumental Activities of Daily Living Questionnaire
Journal of Geriatric Psychiatry and Neurology 26(4) 244-250 ª The Author(s) 2013 Reprints and permission: sagepub.com/journalsPermissions.nav DOI: 10.1177/0891988713509139 jgpn.sagepub.com
Sietske A. M. Sikkes, PhD1,2,3, Yolande A. L. Pijnenburg, MD, PhD1,3, Dirk L. Knol, PhD2, Elly S. M. de Lange-de Klerk, MD, PhD2, Philip Scheltens, MD, PhD1,3, and Bernard M. J. Uitdehaag, MD, PhD2,3
Abstract Background: Measuring impairments in ‘‘instrumental activities of daily living’’ (IADL) is important in dementia, but challenging due to the lack of reliable and valid instruments. We recently developed the Amsterdam Instrumental Activities of Daily Living Questionnaire (A-IADL-Q; note 1). We aim to investigate the diagnostic accuracy of the A-IADL-Q for dementia in a memory clinic setting. Methods: Patients visiting the Alzheimer Center of the VU University Medical Center with their informants between 2009 and 2011 were included (N ¼ 278). Diagnoses were established in a multidisciplinary consensus meeting, independent of the A-IADL-Q scores. An optimal A-IADL-Q cutoff point was determined, and sensitivity and specificity were calculated. Area under the curves (AUCs) were compared between A-IADL-Q and ‘‘disability assessment of dementia’’ (DAD). The additional diagnostic value of the A-IADL-Q to Mini-Mental State Examination (MMSE) was examined using logistic regression analyses. Results: Dementia prevalence was 50.5%. Overall diagnostic accuracy based on the AUC was 0.75 (95% confidence interval [CI]: 0.70-0.81) for the A-IADL-Q and 0.70 (95% CI: 0.63-0.77) for the DAD, which did not differ significantly. The optimal cutoff score for the A-IADL-Q was 51.4, resulting in sensitivity of 0.74 and specificity of 0.64. Combining the A-IADL-Q with the MMSE improved specificity (0.94), with a decline in sensitivity (0.55). Logistic regression models showed that adding A-IADL-Q improved the diagnostic accuracy (Z ¼ 2.55, P ¼ .011), whereas the DAD did not. Conclusions: In this study, we showed a fair diagnostic accuracy for A-IADL-Q and an additional value in the diagnosis of dementia. These results support the role of A-IADLQ as a valuable diagnostic tool. Keywords dementia, sensitivity and specificity, Alzheimer disease, activities of daily living, questionnaire, informant, diagnosis Received April 17, 2013. Received revised August 20, 2013. Accepted for publication September 2, 2013.
Introduction In dementia, interference in everyday functioning is generally measured using informant-based questionnaires aimed at instrumental activities in daily living (IADL).1 Because this interference in everyday functioning is part of the diagnostic criteria of dementia,2 it is important to measure IADL optimally, in a reliable and valid way. However, in several reviews, it was found that the quality of the currently used informant-based IADL questionnaires was limited.3-5 Based on this observation, we started with the development of the Amsterdam IADL Questionnaire (A-IADL-Q; note 1) with input from patients, caregivers, neurologists, geriatricians, specialist nurses, occupational therapists, neuropsychologists, and epidemiologists.6 Activities were chosen to be suitable for both
men and women and for different age groups. In addition, more up-to-date items, such as items related to everyday technology use (computer use and mobile phone use), were included. In a
1
Alzheimer Center, VU University Medical Center, Amsterdam, the Netherlands 2 Department of Epidemiology and Biostatistics, VU University Medical Center, Amsterdam, the Netherlands 3 Department of Neurology, VU University Medical Center, Amsterdam, the Netherlands Corresponding Author: Sietske A. M. Sikkes, Alzheimer Center, VU University Medical Center, PK-1Z035, PO Box 7057, 1007 MB Amsterdam, the Netherlands. Email: [email protected]
Downloaded from jgp.sagepub.com at GEORGIAN COURT UNIV on March 18, 2015
Sikkes et al
245
Figure 1. Study flow.
previous study, we demonstrated good test–retest reliability, content validity, construct validity, and high internal consistency.6,7 In the current study, we aim to investigate the diagnostic usefulness of this newly developed questionnaire. The measurement of IADL has been widely investigated as a possible screening tool for dementia, either in the general population or in the general practice.8-13 However, only a limited number of studies report on the diagnostic accuracy of IADL in a memory clinic setting despite the widespread use. The results of these diagnostic accuracy studies diverge, possibly explained by differences in study design.14-16 In particular, the inclusion of
healthy controls as a reference group might have led to an overestimation of diagnostic accuracy.14 In the current study, we chose a pragmatic design to investigate the diagnostic accuracy by including all consecutive patients visiting a memory clinic. We hypothesized that the A-IADL-Q would be able to discriminate between patients with and patients without dementia. Second, we expected the A-IADL-Q to be a better discriminator than a traditional IADL measure. Third, we hypothesized that the A-IADL-Q would have an additional diagnostic effect on the Mini-Mental State Examination (MMSE) and sociodemographic characteristics.
Downloaded from jgp.sagepub.com at GEORGIAN COURT UNIV on March 18, 2015
246
Journal of Geriatric Psychiatry and Neurology 26(4)
Methods All consecutive informants of patients who visited the Alzheimer Center of the VU University Medical Center for dementia screening were enrolled during the following 3 periods due to practical reasons: October 2009 to May 2010, January to April 2011, and January to February 2012. The eligibility criteria were (1) the presence of an informal informant, (2) the ability of the informant to complete the questionnaire, and (3) an established diagnosis. Figure 1 shows the number of patients included and excluded in the current study. The study was approved by the ethics committee of the VU University Medical Center. All patients gave written informed consent, and all informants gave oral informed consent prior to participation in this study.
Materials Diagnostic Process Patients underwent a standardized dementia assessment including clinical history, medical and neurological examination, routine blood tests to rule out treatable causes, a neuropsychological test battery, magnetic resonance imaging, and electroencephalogram. Clinical diagnosis was established in a multidisciplinary consensus meeting according to the standard diagnostic criteria.2,17-21 Participants of this meeting were unaware of the A-IADL-Q results. However, they were aware of the results of the disability assessment of dementia (DAD). All diagnostic tests (both index and reference tests) were administered on the same day.
The A-IADL-Q The A-IADL-Q is a disease-specific IADL questionnaire, aimed at measuring IADL problems in early dementia. The questionnaire was self-administered by the informant on a tablet computer. We used 2 Web-based platforms for the data collection, Examine22 for the data collection until 2011 and Qualtrics (Qualtrics Labs Inc, Provo, Utah) for the data collection in 2012. It is a branched questionnaire, with a minimum of 47 and a maximum of 70 activities. As a result, no detailed questions are asked when the patient does not perform a specific activity.6 Each item has a 5-point scale response option (scored 0-4). The scoring of the AIADL-Q is calculated using item response theory as described in detail elsewhere.7 The calculated total score for the A-IADL-Q is a y score with a mean of 0 and standard deviation (SD) of 1. This score is transformed to a mean of 50 and SD of 10 (10 y þ 50), resulting in a scoring range from 20 to 80, with lower scores indicating poorer performance.
Disability Assessment for Dementia (DAD) The DAD was reviewed in a systematic review as having reasonable psychometric properties,3 and we therefore chose to compare the A-IADL-Q with the DAD. The DAD is a disease-specific interview-based questionnaire aimed at evaluating functional
disability in community-dwelling persons with Alzheimer disease.23 The DAD consists of 40 items, related to both IADL and basic activities of daily living. A total weighted percentage score is calculated by adding item scores and dividing this by the number of items answered, excluding the nonapplicable answers. Lower scores indicate more ADL and IADL dysfunction. A specialist nurse and research associate conducted the DAD interview.
Statistical Analysis We used SPSS (IBM SPSS Statistics for Windows, Version 20.0; Armonk, New York), Mplus,24 and R25 for data analyses. Patient and informant characteristics between different diagnostic groups were compared using t tests, analysis of variance, chi-square tests, or Mann-Whitney U tests as appropriate. We compared A-IADL-Q scores between patients with and without dementia using an independent t test. To investigate the diagnostic accuracy, receiver–operating characteristic (ROC) curves were created for both the A-IADL-Q and the DAD. Area under the curves (AUCs) were calculated with 95% confidence intervals (CIs) as a further measure of diagnostic accuracy. For an AUC, a value of 0.5 indicates a random guess and a value of 1 perfect classification. The AUCs were further classified as having low accuracy (values between .50 and .70), moderate accuracy (values between .71 and .90), and high accuracy (values higher than .91).26 Differences between ROC curves were tested using the method of DeLong et al27 incorporated in R.28 For the A-IADL-Q, the best possible cutoff score was determined based on the Youden index, and sensitivity and specificity values were calculated. The A-IADL-Q was combined with the MMSE,29 both in series (positive result if both tests are positive) and in parallel (positive result if one of the tests is positive). For tests in series and parallel, sensitivity, specificity, likelihood ratios, and diagnostic odds ratios were calculated. Finally, we used logistic regression models to investigate the additional diagnostic value of the A-IADL-Q. Two models were tested, one with basic clinical and demographic information (MMSE, age, sex, and education) and the second model combining the clinical and demographic information with the A-IADL-Q. The ROC curves were created for both the models, and AUCs were compared using the DeLong method in R.27,28 The significance level was set at P < .05.
Results A total of 278 informants of patients completed the A-IADL-Q. Patient were aged 27 to 86 years (M ¼ 63.9, SD ¼ 9.5), and 140 (50.5%) patients were diagnosed with dementia. Alzheimer disease was the most common type of dementia (n ¼ 105, 75.0%), followed by frontotemporal lobe dementia (n ¼ 17, 12.1%) and Lewy Body dementia (n ¼ 11, 7.9%). The remaining 138 (49.6%) patients were diagnosed as ‘‘not demented.’’ Figure 1 shows the diagnoses for all patients included in this study, and Table 1 shows the patient and informant characteristics. As expected, patients with dementia were older (258.8) ¼ 3.05, P ¼ .003, and had lower MMSE scores (Z ¼ 10.35, P < .001).
Downloaded from jgp.sagepub.com at GEORGIAN COURT UNIV on March 18, 2015
Sikkes et al
247
Table 1. Patient and Informant Characteristics of Patients With and Without Dementia.a Patients (N ¼ 278)
Age Female gender Level of educationb, c, d MMSEe Relationship spouse Relationship > 10 yearsf Living togetherg
Informants (N ¼ 278)
Dementia (n ¼ 140)
No Dementia (n ¼ 138)
P Value
65.6 (8.2) 51 (36.4%) 5 (4-6) 22 (17-24)
62.2 (10.5) 41 (29.7%) 5 (4-6) 27 (25-29)
.003 .23 .43
