Article

Validity and Reliability of the Neuropsychiatric Inventory Questionnaire Version in Patients With Stroke or Transient Ischemic Attack Having Cognitive Impairment

Journal of Geriatric Psychiatry and Neurology 2014, Vol. 27(4) 247-252 ª The Author(s) 2014 Reprints and permission: sagepub.com/journalsPermissions.nav DOI: 10.1177/0891988714532017 jgpn.sagepub.com

Adrian Wong, PhD1,2, Sheung-Tak Cheng, PhD3, Eugene S. K. Lo, BSocSc4, Pauline W. L. Kwan, MSSc4, Lorraine S. N. Law, BSocSc4, Anne Y. Y. Chan, MD4, Lawrence Ka-Sing Wong, MD4, and Vincent Mok, MD4

Abstract This study examined the validity and reliability of the Neuropsychiatric Inventory Questionnaire version (NPI-Q), a proxyreported format of the interview-based NPI, in assessing neuropsychiatric symptoms in 173 patients with stroke or transient ischemic attack (TIA) having cognitive impairment. The NPI-Q was validated against the NPI as a gold standard. Informants took approximately 7 minutes to complete the NPI-Q. Bland-Altman analysis revealed a bias of 0.7 points, with 95% limits of agreement between 8.6 and 10.0 between the total symptom scores of the NPI and NPI-Q. The NPI-Q correlated significantly with the NPI in individual and total symptom scores and caregiver distress scores. In predicting presence of symptoms on the NPI, the NPI-Q yielded, on average, sensitivity of 74.1% and specificity of 79.5%. On the NPI-Q, informants tended to overreport symptoms in patients with less severe symptoms but underreport with increasing symptom severity. Internal consistency of the NPI-Q was acceptable (Cronbach’s a ¼ 0.756). One-week test–retest reliability of the NPI-Q was excellent (intraclass correlation coefficient ¼ .990). The NPI-Q is a valid and reliable instrument for screening neuropsychiatric symptoms in patients with stroke and TIA. Keywords neuropsychiatric inventory questionnaire, behavioral and psychological symptoms of dementia, vascular cognitive impairment, vascular dementia, stroke, clinical assessment

Introduction Neuropsychiatric symptoms are common in cognitive disorders. Commonly referred to as behavioral and psychological symptoms of dementia, these symptoms cause significant distress to caregivers1; predict functional dependency, poor quality of life, and nursing home placement in patients with dementia2,3; and predict progression of cognitive decline in mild cognitive impairment.4 Among patients with stroke, more than half of them exhibit neuropsychiatric symptoms, with prevalence and manifestations of symptoms interacting with time after stroke onset.5 Unfortunately, these symptoms are sometimes unrecognized and undermanaged in patients with stroke, as physical and cognitive problems often take precedence over mood and behavioral complaints. The Neuropsychiatry Inventory (NPI) is a interview-based instrument designed for the assessment of common psychiatric and behavioral symptoms in dementing illnesses.6 It is administered to informants to evaluate the frequency, severity, and caregiver distress of 12 neuropsychiatric symptoms including

delusions, hallucinations, agitation/aggression, dysphoria/ depression, anxiety, euphoria/elation, apathy/indifference, disinhibition, irritability/lability, aberrant motor behaviors, nighttime disturbances, and appetite/eating disturbances in patients. The NPI adopts a screening algorithm to reduce administration time. Under this algorithm, individual symptoms are first 1

Department of Psychological Studies, The Hong Kong Institute of Education, Tai-po, Hong Kong SAR, China 2 Center for Psychosocial Health and Aging, The Hong Kong Institute of Education, Tai-po, Hong Kong SAR, China 3 Department of Health and Physical Education, The Hong Kong Institute of Education, Tai-po, Hong Kong SAR, China 4 Department of Medicine and Therapeutics, The Chinese University of Hong Kong, Shatin, Hong Kong SAR, China Corresponding Author: Vincent Mok, Department of Medicine and Therapeutics, 9/F Clinical Sciences Building, Prince of Wales Hospital, The Chinese University of Hong Kong, Shatin, New Territories, Hong Kong SAR, China. Email: [email protected]

Downloaded from jgp.sagepub.com at UNIV OF CONNECTICUT on June 5, 2015

248

Journal of Geriatric Psychiatry and Neurology 27(4)

screened using a short script-based screening question. Detailed inquiries about the frequency and severity of problem behaviors are made only when the response to the screening question is positive. The NPI has been adopted in large-scale clinical trials to measure treatment effects in stroke patients.7 However, despite the time saving algorithm, the NPI still requires direct interview from clinicians. In view of this, the NPI questionnaire version (NPI-Q) was developed.8 The NPI-Q uses a proxy-reported questionnaire format to assess the same 12 behavior symptoms. The major advantage of the NPI-Q over the NPI is that its proxy-reported questionnaire format significantly minimizes the interview effort of a clinician or an interviewer. In general, symptoms are more briefly described on the NPI-Q than on the NPI. In terms of differences in scoring, the NPI-Q omits the symptom frequency rating in the NPI and assesses only the severity of symptoms and caregiver distress. Individual item scores are calculated as symptom severity (03) score on the NPI-Q and as symptom frequency (0-4) multiplied by severity (0-3) on the NPI. Accordingly, the range of the total symptom score is reduced in the NPI-Q (0-36) when compared to the NPI (0-144). On the other hand, scoring of caregiver distress rating (0-60) is retained in the NPI-Q. In the original validation study, the NPI-Q correlated very well with the NPI in both symptom (r ¼ .91) and caregiver distress (r ¼ .92) ratings.8 The brevity of the NPI-Q makes it well suited for surveying neuropsychiatric symptoms in busy stroke clinics or large epidemiological studies to reduce clinicians’ resources in evaluating these symptoms. To date, the NPI-Q is not yet available in Chinese. The objective of this study is to develop a Chinese version of the NPI-Q and determine its validity and reliability in Chinese patients with cognitive impairment after ischemic stroke or transient ischemic attack (TIA).

Methods Participants Participants were patient–informant pairs from the STroke Registry Investigating Cognitive DEcline (STRIDE) study,9 which is a longitudinal study examining cognitive changes in consecutive patients admitted to a university-affiliated hospital for ischemic stroke or TIA. Patients with global rating 0.5 in Clinical Dementia Rating Scale (CDR)10 were randomly selected and included in this study because patients with cognitive impairment are more likely to experience neuropsychiatric symptoms when compared to their cognitively normal counterparts. Patients were assessed on an outpatient basis at 3 months or more after stroke onset. This study complied with the Declaration of Helsinki and was approved by the Joint Chinese University of Hong Kong—New Territories East Cluster Clinical Research Ethics Committee. Written informed consent was given by each patient for participation in the parent STRIDE study. Proxy consent was sought for patients with diminished capacity (eg, significant cognitive impairment) for giving informed consent. Verbal consent was obtained from the informant before filling out the NPI-Q.

Measures Cognitive functions of patients were assessed using the Cantonese Mini-Mental State examination (MMSE)11 and the CDR. The MMSE is a widely used cognitive screening test for general cognitive functions. The CDR was rated on the basis of all available clinical information, and patients were classified into 5 severity levels of cognitive impairment, denoted by 0 (no dementia); 0.5 (very mild dementia); 1 (mild dementia); 2 (moderate dementia); and 3 (severe dementia). The validated Chinese NPI interview version12 was completed as the standard assessment of neuropsychiatric symptoms in the STRIDE study. The English version of the NPI-Q was translated into Chinese by a bilingual research staff experienced in dementia research and then back-translated by an experienced clinical psychologist.

Procedure The informant was brought to a waiting area of an outpatient research facility. A trained research assistant explained the instructions of NPI-Q to the informant, who then completed the NPI-Q alone. At the same time, the patient was administered the MMSE in a separate room. After the completion of cognitive test with the patient, the NPI-Q was collected from the informant. Afterward, the NPI and CDR were administered by interviewing the same informant by another research assistant in the absence of the patient. The research assistant administering the NPI was blind to the results of the NPI-Q. Test–retest reliability was assessed in 20 randomly selected patients over a period of 1 week. Informants were given a copy of the NPI-Q to bring home and to complete it after 1 week and then mail it back to the study center in a postage-paid envelope provided to them. All procedures were carried out in the Cantonese language, which is the language most commonly spoken in Hong Kong.

Statistical Analysis Interscale agreement between the NPI and the NPI-Q total scores was examined using the Bland-Altman method, which is widely used in the evaluation of agreement of 2 continuous measurements.13 This method calculates the mean difference (ie, bias) as well as the 95% limits of agreement (LoA) between NPI and NPI-Q total scores. Good interscale agreement is denoted by smaller mean difference and width of 95% LoA. In addition, correlations between the NPI and NPI-Q were assessed using Pearson r for the total scores; and in view of the ordinal nature of the ratings and the restricted range for the item scores, Spearman r was calculated for the 12 individual item scores. Correlation analysis was repeated using the NPI symptom severity score alone to exclude the effect of symptom frequency upon the interscale correlations. To assess the relation between the NPI-Q and the severity of cognitive impairment, correlations between the NPI-Q total scores and the CDR and MMSE were computed using the Spearman r and Pearson r,

Downloaded from jgp.sagepub.com at UNIV OF CONNECTICUT on June 5, 2015

Wong et al

249

Table 1. Demographic, Clinical and Cognitive Characteristics of Patients.a 173 74.4 (10.5) 61.3% 4.1 (4.3) 91.3% 8.7% 6.3 (5.9)

15 +1.96 SD 10.0

10 NPI-Q Total

N Age, years Female Education, years Ischemic stroke TIA NIHSS CDR 0.5 1 2 3 MMSE NPI Total symptom score Caregiver distress score NPI-Q Total symptom score Caregiver distress score

20

5

Mean 0.7

0 -5

-1.96 SD -8.6

-10 69.9% 15.6% 17% 8% 19.7 (6.7) 5.4 (6.3) 2.5 (3.9) 4.7 (4.3) 5.3 (6.0)

Abbreviations: TIA, transient ischemic attack; NIHSS, National Institute of Health Stroke Scale; CDR, Clinical Dementia Rating; MMSE, Mini-Mental State Examination; NPI, Neuropsychiatric Inventory; NPI-Q, Neuropsychiatric Inventory Questionnaire version. a Data are shown in mean (standard deviation) or % as appropriate.

respectively. Prevalence of each individual symptom was defined by its presence or absence on the NPI and NPI-Q. Discrepancy in symptom prevalence between the NPI and the NPIQ was calculated by subtracting the prevalence of NPI-Q from that of the NPI. Sensitivity and specificity values of the NPI-Q were calculated. Sensitivity refers to the proportion of participants identified as positive on the NPI that is also identified as positive on the NPI-Q, whereas specificity refers to the proportion of participants identified as negative on the NPI that is also identified as negative on the NPI-Q. Sensitivity and specificity values range from 0% to 100%, with higher values denoting better diagnostic performance. Internal consistency between the 12 symptom scores on the NPI-Q was assessed using the Cronbach’s a. Test–retest reliability of the NPI-Q was expressed in intraclass correlation coefficient. Statistical analysis was performed using IBM SPSS Statistics version 20 and MedCalc version 12. Statistical significance was set at P  .05.

Results One hundred and seventy-three patient–informant pairs were recruited. Table 1 shows the demographic and cognitive characteristics of the patients. Compared to the STRIDE participants not included in this study (n ¼ 876), patients in this study were older (mean age of patients not included ¼ 69.3 years [standard deviation, SD ¼ 12.1], P < .001), had a higher proportion of female (43.2%, P < .001), were less educated (mean education ¼ 5.5 years [SD ¼ 4.8], P < .01), had more severe strokes (mean National Institute of Health Stroke Scale ¼ 4.9 [SD ¼ 5.4], P < .01), and poorer cognition (mean

-15 -20 0

5

10

15 NPI Total

20

25

30

Figure 1. Bland Altman plot depicting interscale agreement between Neuropsychiatric Inventory (NPI) and NPI Questionnaire version (NPI-Q) total Scores.

MMSE ¼ 24.0 [SD ¼ 6.1], P < .001). In this study, informants were mainly woman (65.9%) and had a mean age of 49.9 (SD ¼ 13.0) years. Relationships of informants to the patients were spouses (22.5%), adult children (63.6%), grandchildren (2.3%), other relatives (8.1%), friends (0.6%), and undisclosed relationships (2.9%). In all, 79.6% had at least secondary level of education. On average, informants took about 7 minutes to complete the NPI-Q, whereas the administration of the interview-based NPI required approximately 15 minutes. The NPI-Q total scores correlated positively with patients’ CDR rating (r ¼ 0.451, P < .001) and inversely with MMSE scores (r ¼ .269, P < .001) but not with patients’ age, gender, or education. Bland-Altman analysis for interscale agreement revealed a mean difference of 0.7 points (1.44-0.02) between the NPI and the NPI-Q total (as calculated by NPI  NPI-Q total). The 95% LoA were between 0.86 (9.80 to 7.36) and 10.0 (8.82-11.26). The Bland-Altman plot (Figure 1) suggests a slight proportional bias of the NPI-Q as individual interscale differences tended to increase in positive direction with increasing severity of symptoms measured on the NPI. Table 2 shows the results of interscale correlations of symptom and caregiver distress scores and discrepancy in symptom prevalence as well as the in sensitivity and specificity values of the NPI-Q. The total symptom and caregiver distress of the NPI and NPI-Q were all positively correlated (r ¼ .661, P < .001 for symptom score; r ¼ .579, P < .001 for caregiver distress score). For individual item scores across the 12 domains, Spearman r ranged between .241 to .667, P < .01, for symptom scores and .176 to .590, P < .05, for caregiver distress scores. No appreciable difference was noted in the interscale correlations when the NPI severity score was used. Overall, symptoms were 12.1% more prevalent on the NPI-Q than on the NPI. Symptom prevalence was consistently higher on the NPI-Q than on NPI across all 12 symptoms, with discrepancy ranging between 4.1% (hallucinations) and 21.3% (appetite/

Downloaded from jgp.sagepub.com at UNIV OF CONNECTICUT on June 5, 2015

250

Journal of Geriatric Psychiatry and Neurology 27(4)

Table 2. Intercorrelations of Symptom Scores and Caregiver Distress Scores, Prevalence Discrepancy of Symptom Between the NPI and the NPI-Q, Sensitivity and Specificity, and Predictive Values of NPI-Q.a Symptom Score

Symptom

Caregiver Distress

NPI With NPI-Q r

NPI Severity With NPI-Q r

Prevalence Discrepancy %

Sensitivity %

Specificity %

NPI With NPI-Q r

.667b .637b .411b .508b .241b .291b .318b .313b .475b .336b .351b .276b .661b

.652b .636b .401b .508b .224b .291b .319b .367b .443b .315b .431b .242b .626b

6.4 4.1 18.5 10.4 19.6 5.8 10.9 11.6 19.1 7.5 15 21.3 12.1

88.5 81.2 75.8 77.8 60.9 100 57.9 71.4 84.8 53.3 72.3 65 94.3

90.5 93.6 71.4 70.9 71.9 94.2 74.1 86.7 68.5 87.3 73.7 71.2 44

.490b .459b .445b .502b .177b .340b .320b .313b .547b .198b .268b .176c .579b

Delusions Hallucinations Agitation/aggression Depression Anxiety Elation/euphoria Apathy/indifference Disinhibition Irritability/lability Aberrant motor behaviors Sleep and night time disturbance Appetite and eating disturbance Totald

50 45 40 35 30 25 20 15 10 5 0 Appetite and eating disturbance

Sleep and night time disturbance

Aberant motor behaviors

Irritability/lability

Disinhibition

Apathy/Indifference

Elation/Euphoria

Anxiety

Depression

Agitation/Aggression

Hallucinations

NPI NPI-Q

Delusions

Prevalence (%)

Abbreviations: NPI, Neuropsychiatric Inventory; NPI-Q, Neuropsychiatric Inventory Questionnaire version. a Prevalence discrepancy is calculated by subtracting the symptom prevalence on NPI-Q from NPI. b P < .01. c P < .05. d Intercorrelations of the total symptom and caregiver distress scores between the NPI-Q and the NPI are calculated using Pearson r coefficient.

Neuropsychiatric Symptom Figure 2. Prevalence of symptoms on Neuropsychiatric Inventory (NPI) and NPI Questionnaire version (NPI-Q), expressed in %.

eating disturbance). The mean sensitivity and specificity across the 12 symptoms of the NPI-Q were 74.1% and 79.5%, respectively. Figure 2 shows the prevalence of the 12 symptoms on the NPI and the NPI-Q. Cronbach a, a measure of internal consistency between the 12 symptom scores, was .756. Twenty informants completed the NPI-Q at home with a mean number of 7.5 (SD ¼ 1.9) days after the first administration. Intraclass correlation between the 2 administrations were .990 (.976-.996), P < .001.

Discussion Our study showed that the Chinese NPI-Q had modest to moderate concurrent validity in measuring symptoms and caregiver distress in patients with stroke or TIA. The NPI-Q also has satisfactory internal consistency and excellent test–retest reliability.14 Using the NPI as the gold standard, the interscale agreement between the total symptom score of the NPI and NPI-Q was

Downloaded from jgp.sagepub.com at UNIV OF CONNECTICUT on June 5, 2015

Wong et al

251

acceptable, as shown by a mean bias of 0.7 point, with a relatively narrow 95% LoA of 18.6-point difference. Consistent with findings from the validation study of the English version of the NPI-Q,8 our data also revealed a significant informant bias toward overreporting symptoms on the NPI-Q, given the relatively high interscale prevalence discrepancy in several symptoms. The largest discrepancies were observed in appetite/eating disturbance (21.3%), anxiety (19.6%) irritability/lability (19.1%), and agitation/aggression (18.5%). These symptoms fall into clusters of mood disturbance (anxiety and appetite/eating disturbance) and behavioral problems (irritability/lability and agitation/aggression) as determined in a prior confirmatory factor analysis study in local patients with dementia.15 A closer examination of the Bland-Altman plot (Figure 1) suggested that this bias was proportional to the severity of symptoms, that is, informants tended to rate more severely on the NPI-Q than the NPI in patients with low scores on the NPI, whereas a reversed pattern was observed among those with higher scores on the NPI. This suggested that overreporting mainly occurred in those with less severe symptoms. In contrast, informants tended to underreport symptoms as the severity of symptoms increased, suggesting that the NPI-Q may be less sensitive than the NPI at the more severe end of the spectrum. There are several possible reasons for reporting bias on the NPI-Q. First, the brief behavioral descriptors of the symptoms might not be adequate for informants to fully grasp the essential features of the symptoms, thus they might confuse normal behaviors with symptoms and vice versa. Moreover, physical sequelae of stroke might result in behavioral changes mimicking neuropsychiatric symptoms described in the NPI-Q. For example, dysarthria or dietary management may result in changes in eating habits described as symptoms of appetite/eating disturbance. On a more global level, reporting bias might also reflect the level of knowledge the general public has regarding the constructs concerned. In Hong Kong, public education on stroke traditionally focuses on the physical and cognitive aspects and very little information is given on its mood and behavioral sequelae; thus informants may not be able to accurately appreciate the features of symptoms without proper guidance from a professional. Despite these limitations of the NPI-Q, the brevity and predictive profile makes it suitable for use as a preliminary method to screen neuropsychiatric symptoms for further evaluation by clinicians. It can be filled out by informants in the waiting room within a few minutes, and symptoms with a positive rating on the NPI-Q can be further assessed by a clinician in a formal diagnostic interview. The ease of use of the NPI-Q will likely increase motivation for clinicians to assess these symptoms to facilitate their management plans for the patients and their caregivers as well as to generate useful prognostic information regarding cognitive decline. There are a number of limitations in this study. First of all, we used the NPI as the gold standard measure for the NPI-Q to compare against. However, the NPI, in itself, is also a brief

screening instrument with its own limitations.16 Moreover, the fact that informants were interviewed for NPI shortly after they had completed the NPI-Q might have biased the NPI ratings and inflated the concurrent validity. Furthermore, we did not assess the level of knowledge the informant had upon the patient behaviors and thus the representativeness of ratings. Likewise, variables such as level of caregiving burden, caregiver’s own mood, and recalling bias that could potentially color their perceptions on patients’ behaviors were not collected and accounted for in the analyses. Moreover, compared to the STRIDE study participants not included in this study, those included in this study had a higher proportion of female, were older, less educated, and more impaired in terms of stroke severity and cognition, and thus the findings may not be entirely applicable to the general stroke population. Finally, the sample size for test–retest reliability was relatively small. In conclusion, the Chinese NPI-Q has demonstrated modest to moderate concurrent validity and excellent test–retest reliability. There is a tendency for informants to overreport symptoms in patients with less severe symptoms and to underreport among those with more severe symptoms. Given its brevity and predictive profile, the NPI-Q may serve as a useful preliminary screening tool to screen symptoms for further assessment by the clinician. Authors’ Note Adrian Wong—conceptualization of study, data analysis and manuscript preparation; Sheung-Tak Cheng—data analysis and manuscript preparation; Eugene S.K. Lo—data collection and analysis; Pauline W.L. Kwan—data collection and analysis; Lorraine S.N. Law—data collection and analysis; Anne Y.Y. Chan—conceptualization of study; Lawrence K.S. Wong—conceptualization of study; Vincent Mok— conceptualization of study and manuscript preparation.

Acknowledgments This work was supported by the Neurology Research Fund, Department of Medicine and Therapeutics, The Chinese University of Hong Kong. We would like to thank our research assistants Ms Christine Lau and Ms Joan Khoo for their valuable contributions to this study.

Declaration of Conflicting Interests The author(s) declared no potential conflicts of interest with respect to the research, authorship, and/or publication of this article.

Funding The author(s) received no financial support for the research, authorship, and/or publication of this article.

References 1. Tan LL, Wong HB, Allen H. The impact of neuropsychiatric symptoms of dementia on distress in family and professional caregivers in Singapore. Int Psychogeriatr. 2005;17(2):253-263. 2. Samus QM, Rosenblatt A, Onyike C, et al. Correlates of caregiver-rated quality of life in assisted living: the Maryland Assisted Living study. J Gerontol B Psychol Sci Soc Sci. 2006; 61(5):P311-P314.

Downloaded from jgp.sagepub.com at UNIV OF CONNECTICUT on June 5, 2015

252

Journal of Geriatric Psychiatry and Neurology 27(4)

3. Royall DR, Lauterbach EC, Cummings JL, et al. Executive control function: a review of its promise and challenges for clinical research. A report from the committee on research of the American neuropsychiatric association. J Neuropsychiatry Clin Neurosci. 2002;14(4):377-405. 4. Palmer K, Di Iulio F, Varsi AE, et al. Neuropsychiatric predictors of progression from amnestic-mild cognitive impairment to Alzheimer’s disease: the role of depression and apathy. J Alzheimers Dis. 2010;20(1):175-183. 5. Angelelli P, Paolucci S, Bivona U, et al. Development of neuropsychiatric symptoms in poststroke patients: a cross-sectional study. Acta Psychiatr Scand. 2004;110(1):55-63. 6. Cummings JL. The Neuropsychiatric Inventory: assessing psychopathology in dementia patients. Neurology. 1997;48(5 suppl 6):S10-S16. 7. Black SE, Doody R, Li H, et al. Donepezil preserves cognition and global function in patients with severe Alzheimer disease. Neurology. 2007;69(5):459-469. 8. Kaufer DI, Cummings JL, Ketchel P, et al. Validation of the NPIQ, a brief clinical form of the Neuropsychiatric Inventory. J Neuropsychiatry Clin Neurosci. 2000;12(2):233-239. 9. Yang J, Wong A, Wang Z, et al. Risk factors for incident dementia after stroke and transient ischemic attack [published online March 3, 2014]. Alzheimers Dement. 2014.

10. Hughes CP, Berg L, Danziger WL, Coben LA, Martin RL. A new clinical scale for the staging of dementia. Br J Psychiatry. 1982; 140:566-572. 11. Chiu HFK, Kee HC, Chung WS, Kwong PK. Reliability and validity of the cantonese version of mini-mental state examination a preliminary study. J Hong Kong College Psychiatry. 1994; 4(S2):25-28. 12. Leung VP, Lam LC, Chiu HF, Cummings JL, Chen QL. Validation study of the Chinese version of the neuropsychiatric inventory (CNPI). Int J Geriatr Psychiatry. 2001;16(8): 789-793. 13. Bland JM, Altman DG. Statistical methods for assessing agreement between two methods of clinical measurement. Lancet. 1986;1(8476):307-310. 14. Bland JM, Altman DG. Cronbach’s alpha. BMJ. 1997;314(7080): 572. 15. Cheng S-T, Kwok T, Lam LC. Neuropsychiatric symptom clusters of Alzheimer’s disease in Hong Kong Chinese: prevalence and confirmatory factor analysis of the Neuropsychiatric Inventory. Int Psychogeriatr. 2012;24(9):1465-1473. 16. Mayer LS, Bay RC, Politis A, et al. Comparison of three rating scales as outcome measures for treatment trials of depression in Alzheimer disease: findings from DIADS. Int J Geriatr Psychiatry. 2006;21(10):930-936.

Downloaded from jgp.sagepub.com at UNIV OF CONNECTICUT on June 5, 2015