Cog-4 Has Limited Diagnostic Test Accuracy and Validity for Cognitive Assessment in Stroke Survivors Rosalind Lees, BA,* Jane Lua, MBChB,† Emily Melling, MBChB,† Yen Miao, MBChB,† Jia Tan, MBChB,† and Terence J. Quinn, MD*
Background: Guidelines recommend cognitive screening for all stroke survivors but do not suggest a preferred tool. Certain elements (orientation, executive function, language, and inattention) of the impairment scale, National Institutes of Health Stroke Scale (NIHSS), have been suggested as a short cognitive screening test— Cog-4. We aimed to describe accuracy and validity of Cog-4 against a more detailed cognitive assessment (Montreal Cognitive Assessment [MoCA]). Methods: We assessed consecutive acute stroke unit admissions in 2 hospitals over 3 months. Four independent blinded assessors performed NIHSS and MoCA between days 1 and 4 poststroke. We described test properties of Cog-4 for MoCA-defined cognitive impairment using usual thresholds (Cog-4 $ 1 and MoCA , 26 of 30) and described the correlations of individual Cog-4 components with broadly equivalent MoCA domains. Results: We assessed 173 participants; 166 had Cog-4 data and 148 MoCA. MoCA described 84% (n 5 124) of assessed participants as having cognitive impairment and the Cog-4, 37% (n 5 62). Cog-4 had a sensitivity of .36 (95% confidence interval [CI]: .28-.45) and a specificity of .96 (95% CI: .80-.99) (positive predictive value: .98, negative predictive value: .23) for MoCA-defined cognitive impairment. Individual Cog-4 items correlated with certain MoCA domains, but the strength of association was modest (r 5 2.44 orientation, 2.37 language, 2.19 for inattention, and no significant correlation for executive function, P 5 .72). Conclusions: Our data suggest that many stroke survivors with MoCA-defined cognitive problems would not be detected by Cog-4. Subtest correlations suggest that Cog-4 may not be a valid measure of the cognitive domains that it purports to describe. Other brief cognitive screening tests may be better suited to acute stroke. Key Words: Cognitive impairment—Cog-4—MoCA—sensitivity—scales— specificity—stroke—screening. Ó 2014 by National Stroke Association
Introduction From the *Institute of Cardiovascular and Medical Sciences, School of Medicine, University of Glasgow; and †Undergraduate Medical School, School of Medicine, University of Glasgow, Glasgow, UK. Received October 8, 2013; revision received November 14, 2013; accepted December 29, 2013. Funding: R.L. is supported by a Chest Heart and Stroke Scotland research grant; this study was supported by a ‘‘start up’’ grant from the British Geriatrics Society (Scotland). Conflicts of interest: None declared. Address correspondence to T. J. Quinn, Department of Academic Geriatric Medicine, Walton Building, Glasgow Royal Infirmary, Glasgow, UK G4 0SF. E-mail: [email protected]. 1052-3057/$ - see front matter Ó 2014 by National Stroke Association http://dx.doi.org/10.1016/j.jstrokecerebrovasdis.2013.12.042
Cognitive impairment affects around one third of stroke survivors.1,2 Cognitive deficits can negatively affect many aspects of patient recovery. Poststroke cognitive impairment is associated with poor functional recovery and increased mortality.3-5 Stroke survivors with cognitive deficits may have improved outcomes if diagnosis is made at an early stage. Timely diagnosis of potential cognitive issues will allow for appropriate intervention and follow-up.6 Clinical guidelines in the United Kingdom and other countries, therefore, recommend that all acute stroke admissions are screened for cognitive deficits.7
Journal of Stroke and Cerebrovascular Diseases, Vol. -, No. - (---), 2014: pp 1-7
1
R. LEES ET AL.
2
A variety of tools are available for measuring cognitive impairment. However, there is no consensus as to preferred test strategy. Reviews of cognitive scales used in both clinical and research settings have shown substantial heterogeneity in choice and application of testing instrument.8,9 The ideal would be detailed specialist assessment, supplemented by a comprehensive neuropsychological battery. Such an approach is not feasible in a busy acute setting. Usual practice is to employ a 2-step process for assessment: initial brief screening followed by furthermore detailed assessment by a specialist if screening detects problems. The National Institutes Stroke Scale (NIHSS) is an impairment scale commonly employed in stroke care. NIHSS assesses neurologic function through measurement of consciousness, language, neglect, visual fields, eye movement, facial symmetry, motor strength, sensation, and co-ordination.10 Certain elements of the NIHSS, (1b) orientation, (1c) executive function, (9) language, and (11) inattention, have been suggested as a short cognitive screening test—the Cog-4.11 The Cog-4 has been shown to provide some indication of cognitive impairment at 18 months poststroke, although it performs less well than the more detailed screening tool of Folstein MiniMental State Examination.11 However, Mini-Mental State Examination may not be the ideal comparator as it has limited ability to describe the executive impairments commonly seen in stroke cohorts.12,13 Studies of responsiveness of Cog-4 has suggested that Cog-4 assesses only a limited range of possible cognitive outcomes with a marked ‘‘floor’’ effect14,15 and lacks sensitivity depending on the hemisphere affected.14-16 However, other aspects of the Cog-4 are desirable for the acute stroke setting; the assessment is short in duration, does not add to test burden, and can be derived from routinely collected data. Cognitive assessment in acute stroke can be complicated by speech disturbance,17 physical impairments, and concomitant medical complications including delirium.18-20 A valid assessment of the test properties of the Cog-4 as a screening tool must include subjects representative of those stroke patients likely to be assessed in routine clinical practice. Our intention was to mirror ‘‘usual practice’’ through comparison of as a brief tool with a clinically accepted measure in a representative sample at the acute stage. To describe test accuracy and validity of the Cog-4, we used a multidomain, cognitive assessment recommended for all-cause vascular cognitive impairment, the Montreal Cognitive Assessment (MoCA).21-23 The MoCA was originally developed as a brief screening tool to identify older adults with cognitive impairment that score in ‘‘normal range’’ on other commonly used assessments.22 Although not specifically designed for use in stroke, MoCA has many features that make it a useful scale for assessment of stroke survivors and is increasingly used
21-23
in this population. In specific relation to our study, MoCA was chosen as a cognitive assessment as its test domains allow for individual component analysis of Cog-4 items while still offering a valid global test of cognitive function.23 Our primary aims were to test the accuracy of Cog-4 for detection of MoCA-defined cognitive impairment and to describe correlation of individual Cog-4 items and broadly corresponding MoCA cognitive domains.
Subjects and Methods We devised the study in line with methodological (ie, assessment administration) and reporting guidance for diagnostic test accuracy studies, including the dementia-specific guidance STARDdem.24-26 We accept that STARDdem is most appropriate to those studies that use a dementia diagnosis reference standard. Although our reference standard was cognitive impairment on MoCA rather than clinical dementia, the best practice described in STARDdem is still applicable to our study. The study had approval from the local research ethics committee.
Setting The study was conducted across the stroke units of 2 urban teaching hospitals (Glasgow Royal Infirmary and Western Infirmary Glasgow, UK). Data were collected in 2 cohorts between April 2012 to June 2012 and December 2012 to April 2013. Both units admit all suspected strokes regardless of age, prestroke function, or severity of stroke.
Participants Patients were consecutive inpatient stroke survivors. Verbal consent was attained before assessment that took place between days 1 and 4 poststroke (based on day of admission). We did not set a priori exclusion criteria relating to medical instability; the treating clinical team made a decision on whether the patient was too unwell or unsuitable for any form of cognitive assessment, for example, coma or end of life care. Clinical and demographic details were independently collected from the case notes. Stroke classifications were based on the Oxford Community Project Classification,27 lacunar stroke, partial anterior circulation stroke (PACS), posterior circulation stroke, total anterior circulation stroke (TACS), transient ischemic attack, and other/unclassified. We assessed various risk factors for cognitive impairment, all based on previous diagnosis extracted from patient health records: previous dementia, depression, and prestroke visual/hearing impairments.
Assessments Our index tests were the subscales of the NIHSS (orientation, executive function, language, and inattention) that
PROPERTIES OF COG-4 ASSESSMENT
3
Table 1. The Cog-4 assessment, domains and scoring Cog-4 1 (b)
1 (c)
9
11
Level of consciousness— questions (month and age) Answers both correctly Answers one question correctly Answers neither questions correctly Level of consciousness— commands (open and close eyes; grip and release hand) Performs both task correctly Performs 1 task correctly Performs neither task correctly Best language No aphasia Mild-to-moderate aphasia Severe aphasia Mute and global aphasia Extinction and inattention No inattention Mild inattention Severe inattention
Score
0 1 2
0 1 2 0 1 2 3 0 1 2
Abbreviation: Cog-4, 4 cognitive areas of the National Institutes of Health Stroke Scale.
make up the Cog-4.11 Each component is scored from 0 to 2 or 3 with more than 0 being considered abnormal (Table 1); increasing scores describe greater severity of impairment within that domain.14 Our reference standard was the MoCA. Component domain scores range from 1 to 6 and include visuospatial/executive function, animal naming, short- and long-term memory, attention, language, abstraction, and orientation.
Figure 1. Recruitment flow chart. Abbreviations: Cog-4, 4 cognitive areas of the National Institute of Health Stroke Scale; MoCA, Montreal Cognitive Assessment.
In Cog-4, elements of the NIHSS are taken as proxies of cognitive domains: NIH 1b, orientation; NIH 1c, executive function; NIH 9, language; and NIH 11 (visual), inattention. We compared with corresponding domains of MoCA. Cog4 and MoCA had certain domains that purported to measure the same construct, orientation, executive function, and language; the Cog4 domain of (visual) inattention had no direct equivalent within MoCA, and we chose visuospatial function as being closest in nature.28 We employed the recommended standard test cut points of 1 or more for Cog-414 and less than 26 of 30 for the MoCA22 as testing positive for cognitive impairment. Assessments were not modified for patients with specific impairments; however, disabilities that may have impeded scores were noted.
Procedure Both measures were performed on each patient on the same day in a randomized order by different researchers who were blinded to each other’s results. Where no score was available in the Cog-4 or MoCA, these participants were removed from analysis. The total scores were calculated and classified from available data, where participants who were unable to complete a particular section of the test were scored as 0.
Analysis Data extraction from proformas and analyses were performed by a single researcher (R.L.). Our primary analysis was the test accuracy of total Cog-4 for the binary outcome cognitive impairment/no cognitive impairment on MoCA. Secondary analyses described correlation between individual components of the Cog-4 and the corresponding domains of the MoCA.
R. LEES ET AL.
4
Table 2. Test accuracy of Cog-4 and MoCA as ‘‘3 3 3’’ table MoCA MoCA 1ve MoCA –ve untestable Cog-4 1ve Cog-4 2ve Cog-4 untestable
45 79 0
1 23 0
17 1 7
Abbreviations: Cog-4, 4 cognitive areas of the National Institutes of Health Stroke Scale, diagnostic threshold used was more than 1 of 12; MoCA, Montreal Cognitive Assessment, diagnostic threshold used was less than 26 of 30; 1ve, case positive for cognitive impairment; 2ve, case negative for cognitive impairment.
We created a 2 3 2 and 3 3 3 data table for primary analyses. The 3 3 3 table allows for those unable to complete testing to be considered in the analysis and should give a better reflection of how the test performs in clinical practice.29 From these, we calculated sensitivity, specificity, and positive and negative predictive values with corresponding 95% confidence intervals (CIs). We recognized that NIHSS scores favor dominant hemisphere and anterior lesions. The effect of hemisphere on Cog-4 properties has previously been described14-16; on advice from reviewers, we performed post hoc subgroup analyses, repeating the analysis limited to those with TACS/PACS. Statistical analyses were performed using SPSS (IBM Ltd., Portsmouth, UK, version 19.0), Minitab (Minitab Ltd., State College, PA,
windows version 15), and Statsdirect software (Stats Direct Ltd., Cheshire, UK, version 2.7.9).
Results Over the 24-week study period, 203 subjects admitted were potentially eligible for assessment. Data were collected from 173 with 30 patients (15%) refusing cognitive assessment (Fig 1). Participants were 53% men (n 5 92), median age 74 years (interquartile range [IQR]: 63-83), and 70 (40.5%) had a history of stroke. Those refusing cognitive assessment had similar profiles to those recruited: median age 77 years (IQR: 67-86) and median NIHSS 4 (IQR: 2-8). The majority of participants were able to complete the screening assessments: 166 had Cog-4 data and 148 MoCA (Table 2). Some participants were only able to score part of the MoCA; in this instance, those domains that could not be scored were coded as zero; this was required in n 5 12 (7%). Stroke classifications by the Oxford Community Project Classification were lacunar stroke (n 5 38, 22%), PACS (n 5 54, 31%), posterior circulation stroke (n 5 12, 7%), TACS (n 5 27, 16%), TIA (n 5 10, 6%), and other/unclassified (n 5 32, 18%). Median total NIHSS was 3 (IQR: 2-6; range: 0-24) and median Cog-4 was 0 (IQR: 0-1; range: 0-8). There were various risk factors for cognitive impairment within subjects: 16 (9%) had a previous diagnosis of dementia, 15 (8.7%) had depression, and 26 (15%) had prestroke visual or hearing impairments (Table 3).
Table 3. Clinical and demographic information of participants
Participant demographics Male Age Prior stroke OCSP classification LAC PAC POC TAC TIA Unclassified Cognitive impairment risk factors Dementia Depression Visual/hearing impairment Assessments NIHSS Cog-4
Participants, n (%)
Median
IQR
Range
92 (53%) — 70 (40.5%)
— 74 —
— 63-83 —
— 28-97 —
38 (22%) 54 (31%) 12 (7%) 27 (16%) 10 (6%) 32 (18%)
— — — — — —
— — — — — —
— — — — — —
16 (9%) 15 (8.7%) 26 (15%)
— — —
— — —
— — —
— —
3 0
2-6 0-1
0-24 0-8
Abbreviations: Cog-4, 4 cognitive areas of the NIHSS; IQR, interquartile range; LAC, lacunar stroke; NIHSS, National Institutes of Health Stroke Scale; OCSP, Oxford Community Stroke Project; PACS, partial anterior circulation stroke; POCS, posterior circulation stroke; TACS, total anterior circulation stroke; TIA, transient ischemic attack.
PROPERTIES OF COG-4 ASSESSMENT
5
Table 4. Test accuracy comparisons between total Cog-4 and MoCA
Cog-4
Sensitivity
Specificity
Positive predictive value
Negative predictive value
Estimate 95% CI
.36 .28-.45
.96 .80-.99
.98 .89-1.00
.23 .16-.32
Correlation (r)
Significance (P value)
2.48
,.0001
Abbreviations: CI, confidence interval; Cog-4, 4 cognitive areas of the National Institutes of Health Stroke Scale, diagnostic threshold used was more than 1 of 12; MoCA, Montreal Cognitive Assessment, diagnostic threshold used was less than 26 of 30.
The MoCA recorded 82% (n 5 124) of participants with data available as having cognitive impairment at standard diagnostic threshold and the Cog-4 recorded 37% (n 5 62). For test accuracy, Cog-4 had a specificity of .96 (95% CI: .80-.99), sensitivity .36 (95% CI: .28-.45), positive predictive value .98 (95% CI: .89-1.00), negative predictive value .23 (95% CI: .16-.32; Table 4). The correlation between total scores on the 2 assessments was significant (r 5 2.48, P , .0001). In the individual domain analyses, significant correlations were found for orientation (P , .0001), language (P , .0001), and inattention (P 5 .02), albeit the strength of association was modest, r 5 2.44, 2.37, and 2.19, respectively (Table 5). Our post hoc subgroup analysis suggested that test properties of Cog-4 were not improved when the test was limited to those with anterior strokes (TACS and PACS): sensitivity .53 (95% CI: .41-.65), specificity .50 (95% CI: .46-.99), positive predictive value .99 (95% CI: .87-1.00), and negative predictive value .13 (95% CI: .06.28). As prevalence of MoCA-defined cognitive impairment was higher than anticipated, we performed post hoc analyses describing test properties of Cog-4 at 2 other threshold MoCA scores that have been suggested for use in stroke populations. Sensitivity improved with lower MoCA thresholds but remained suboptimal (Table 6).
Discussion We have demonstrated suboptimal validity and test accuracy of Cog-4 as a brief cognitive screening assessment for acute stroke. Our analysis showed favorable specificity but at the expense of poor sensitivity. Although total Cog-4 showed significant correlation with MoCA, the association was modest. Three individual Cog-4 domains were significantly correlated with corresponding cognitive domains (orientation, language, and inattention) although again the strength of association was at the best modest. Our data suggest that many stroke survivors with potential cognitive problems would not be picked up by Cog-4 testing and other brief cognitive screening tests may be better suited to acute stroke. The correlation
data suggest that Cog-4 items may not be robust measures of the cognitive domains they purport to measure. Our test accuracy data are in keeping with previous research that has described reduced sensitivity of Cog-4 to detect cognitive impairment.11 Our analysis was necessarily pragmatic. We recognize that MoCA is not a definitive diagnostic test rather MoCA offers a screening tool, albeit a more detailed tool than Cog-4. However, MoCA was specifically designed for assessment of the cognitive domains of interest, whereas Cog-4 extrapolates neurologic impairment data as a proxy for cognitive assessment, and so we feel our analysis of correlation and test accuracy is still valid. Our reference standard, the MoCA, recorded 82% (n 5 124) of participants as having cognitive impairment. This is higher than previous study estimates; however, previous studies were not performed in our very acute time frame.30-32 In the absence of consensus agreement for MoCA diagnostic threshold in the stroke setting, we used the standard cutoff described for MoCA. The very high prevalence of cognitive impairment recorded suggests that this threshold may be too high. Other authors have commented that similar and alternative cut points have been suggested30,33 as some domain impairments may be transient and not indicatory of long-term cognitive problems.34,35 In our post hoc analysis, lower cut points in the MoCA improve the properties of the Cog-4 to an extent but not sufficiently to recommend the use of the Cog-4. The substantial early cognitive burden in acute stroke we have demonstrated is a factor to be mindful of in context of recommendations for cognitive screening in acute stroke and an area that would benefit from further research to
Table 5. Correlations between subdomains of Cog-4 and MoCA
Cog-4 sub-domain
Correlation (r)
Significance (P value)
Orientation Executive function Language Inattention
2.44 2.03 2.37 2.19
,.0001 .72 ,.0001 .02
Abbreviations: Cog-4, 4 cognitive areas of the National Institute of Health Stroke Scale; MoCA, Montreal Cognitive Assessment.
R. LEES ET AL.
6
Table 6. Test accuracy comparisons between the total Cog-4 and MoCA at different cut points Cog-4
Sensitivity , 20
Specificity , 20
Sensitivity , 24
Specificity , 24
Estimate 95% CI
.49 .37-.60
.86 .76-.92
.40 .31-.50
.89 .77-.95
Abbreviations: Cog-4, 4 cognitive areas of the National Institutes of Health Stroke Scale, diagnostic threshold used was 1 or more of 12; MoCA, Montreal Cognitive Assessment, diagnostic threshold used was less than 20 of 30 and less than 24 of 30.
describe the incidence and natural history of these early cognitive impairments. The limitations of this work are that assessors although trained in the cognitive tools were not experienced stroke clinicians. However, part of an effective screening test is the ability to be used by any member of the stroke care team. Our assessments were focussed on test accuracy and comparative validity, and we did not describe metrics relating to feasibility (eg, time taken to complete assessments) or management changes resulting from cognitive assessment results. Strengths of this work are the clinical setting and inclusive approach, which should improve external validity. We present data relevant to an under-researched but topical area of stroke care, with data collected and presented following best practice guidelines. Our data suggest that although cognitive screening within the acute stroke setting is generally feasible, Cog4 may not be suited to this purpose. In particular, Cog-4 may be insufficiently sensitive to cognitive impairment in stroke survivors, and certain domains of Cog-4 may not be valid measures of the cognitive constructs they purport to describe. We recommend formal diagnostic test accuracy analysis of other short screening tools to inform the choice of optimal assessment at various stages of the stroke survivor journey. Acknowledgment: We would like to thank the stroke care teams for their co-operation and help in identifying appropriate patients. We thank Anna Noel-Storr, Cochrane Dementia and Cognitive Improvement Group, for sharing STARDdem reporting guidance.
References 1. Patel M, Coshall C, Rudd AG, et al. Natural history of cognitive impairment after stroke and factors associated with its recovery. Clin Rehabil 2003;17:158-166. 2. Lawrence ES, Coshall C, Dundas R, et al. Estimates of the prevalence of acute stroke impairments and disability in a multiethnic population. Stroke 2001;32:1279-1284. 3. Robertson IH, Ridgeway V, Greenfield E, et al. Motor recovery after stroke depends on intact sustained attention: a 2-year follow-up study. Neuropsychology 1997; 11:290-295. 4. Robinson RG, Bollawilson K, Kaplan E, et al. Depression influences intellectual impairment in stroke patients. Br J Psychiatry 1986;148:541-547.
5. Patel MD, Coshall C, Rudd AG, et al. Cognitive impairment after stroke: clinical determinants and its associations with long-term stroke outcomes. J Am Geriatrics Soc 2002;50:700-706. 6. Zinn S, Dudley TK, Bosworth HB, et al. The effect of poststroke cognitive impairment on rehabilitation process and functional outcome. Arch Phys Med Rehabil 2004; 85:1084-1090. 7. Scottish Intercollegiate Guidelines Network (SIGN). Management of patients with stroke. Rehabilitation, prevention and management of complications and discharge planning. In: Scottish Intercollegiate Guidelines Network, ed. SIGN Publication Guideline 2010 . Edinburgh , UK: Healthcare Improvement Scotland (HIS), 2010. 8. Lees R, Fearon P, Harrison JK, et al. Cognitive and mood assessment in stroke research: focused review of contemporary studies. Stroke 2012;43:1678-1680. 9. Lees R, Quinn T, Broomfield N. Questionnaire assessment of usual practice in mood and cognitive assessment in Scottish stroke units. Disability Rehabil. http://dx.doi. org/10.3109/09638288.2013.791728. 10. Brott T, Adams HP Jr, Olinger CP, et al. Measurements of acute cerebral infarction: a clinical examination scale. Stroke 1989;20:864-870. 11. Cumming TB, Blomstrand C, Bernhardt J, et al. The NIH stroke scale can establish cognitive function after stroke. Cerebrovasc Dis 2010;30:7-14. 12. Bour A, Rasquin S, Boreas A, et al. How predictive is the MMSE for cognitive performance after stroke? J Neurol 2010;257:630-637. 13. Nys GM, van Zandvoort MJ, de Kort PL, et al. Restrictions of the Mini-Mental State Examination in acute stroke. Arch Clin Neuropsychol 2005;20:623-629. 14. Ankolekar S, Renton C, Sprigg N, et al. The Cog-4 subset of the National Institutes of Health Stroke Scale as a measure of cognition: relationship with baseline factors and functional outcome after stroke using data from the virtual International Stroke Trials Archive. Stroke Res Treat 2013;2013:6. 15. Hajjar K, Fulton RL, Diener HC, et al. Does the cognitive measure Cog-4 show improvement among patients treated with thrombolysis after acute stroke? Int J Stroke 2013;8:652-656. 16. Gottesman RF, Kleinman JT, Davis C, et al. The NIHSSplus: improving cognitive assessment with the NIHSS. Behav Neurol 2010;22:11-15. 17. Thomas SA, Lincoln NB. Factors relating to depression after stroke. Br J Clin Psychol 2006;45:49-61. 18. Laska AC, Martensson B, Kahan T, et al. Recognition of depression in aphasic stroke patients. Cerebrovasc Dis 2007;24:74-79. 19. Lees R, Corbet S, Johnston C, et al. Test accuracy of short screening tests for diagnosis of delirium or cognitive impairment in an acute stroke setting. Stroke 2013; 44:3078-3083.
PROPERTIES OF COG-4 ASSESSMENT 20. Quaranta D, Marra C, Gainotti G. Mood disorders after stroke: diagnostic validation of the poststroke depression rating scale. Cerebrovasc Dis 2008;26:237-243. 21. Hachinski V, Iadecola C, Petersen RC, et al. National Institute of Neurological Disorders and StrokeCanadian Stroke Network vascular cognitive impairment harmonization standards. Stroke 2006;37:2220-2241. 22. Nasreddine ZS, Phillips NA, Bedirian V, et al. The Montreal Cognitive Assessment, MoCA: a brief screening tool for mild cognitive impairment. J Am Geriatr Soc 2005;53:695-699. 23. Pendlebury ST, Mariz J, Bull L, et al. MoCA, ACE-R, and MMSE versus the National Institute of Neurological Disorders and Stroke-Canadian Stroke Network Vascular Cognitive Impairment Harmonization Standards Neuropsychological Battery after TIA and stroke. Stroke 2012; 43:464-469. 24. Bossuyt PM, Reitsma JB, Bruns DE, et al. Towards complete and accurate reporting of studies of diagnostic accuracy: the STARD initiative. Br Med J 2003; 326:41-44. 25. Whiting PF, Rutjes AW, Westwood ME, et al. QUADAS-2: a revised tool for the quality assessment of diagnostic accuracy studies. Ann Intern Med 2011;155:529-536. 26. STARDdem Initiative. Available at: http://dementia. cochrane.org. Accessed January 2014. 27. Bamford J, Sandercock P, Dennis M, et al. Classification and natural history of clinically identifiable subtypes of cerebral infarction. Lancet 1991;337:1521-1526. 28. Damian AM, Jacobson SA, Hentz JG, et al. The Montreal Cognitive Assessment and the mini-mental state exami-
7
29.
30.
31.
32.
33.
34.
35.
nation as screening instruments for cognitive impairment: item analyses and threshold scores. Dementia Geriatr Cogn Disord 2011;31:126-131. Schuetz GM, Schlattmann P, Dewey M. Use of 3 3 2 tables with an intention to diagnose approach to assess clinical performance of diagnostic tests: meta-analytical evaluation of coronary CT angiography studies. Br Med J 2012;345. Dong YH, Sharma VK, Chan BPL, et al. The Montreal Cognitive Assessment (MoCA) is superior to the MiniMental State Examination (MMSE) for the detection of vascular cognitive impairment after acute stroke. J Neurol Sci 2010;299:15-18. Riepe MW, Riss S, Bittner D, et al. Screening for cognitive impairment in patients with acute stroke. Dementia Geriatr Cogn Disord 2004;17:49-53. Schmidt R, Mechtler L, Kinkel PR, et al. Cognitive impairment after acute supratentorial stroke: a 6-month followup clinical and computed tomographic study. Eur Arch Psychiatry Clin Neurosci 1993;243:11-15. Godefroy O, Fickl A, Roussel M, et al. Is the Montreal Cognitive Assessment superior to the Mini-Mental State examination to detect poststroke cognitive impairment? A study with neuropsychological evaluation. Stroke 2011;42:1712-1716. Hurford R, Charidimou A, Fox Z, et al. Domain-specific trends in cognitive impairment after acute ischaemic stroke. J Neurol 2013;260:237-241. Pendlebury ST, Wadling S, Silver LE, et al. Transient cognitive impairment in TIA and minor stroke. Stroke 2011;42:3116-3121.
Background: Guidelines recommend cognitive screening for all stroke survivors but do not suggest a preferred tool. Certain elements (orientation, executive function, language, and inattention) of the impairment scale, National Institutes of Health Stroke Scale (NIHSS), have been suggested as a short cognitive screening test— Cog-4. We aimed to describe accuracy and validity of Cog-4 against a more detailed cognitive assessment (Montreal Cognitive Assessment [MoCA]). Methods: We assessed consecutive acute stroke unit admissions in 2 hospitals over 3 months. Four independent blinded assessors performed NIHSS and MoCA between days 1 and 4 poststroke. We described test properties of Cog-4 for MoCA-defined cognitive impairment using usual thresholds (Cog-4 $ 1 and MoCA , 26 of 30) and described the correlations of individual Cog-4 components with broadly equivalent MoCA domains. Results: We assessed 173 participants; 166 had Cog-4 data and 148 MoCA. MoCA described 84% (n 5 124) of assessed participants as having cognitive impairment and the Cog-4, 37% (n 5 62). Cog-4 had a sensitivity of .36 (95% confidence interval [CI]: .28-.45) and a specificity of .96 (95% CI: .80-.99) (positive predictive value: .98, negative predictive value: .23) for MoCA-defined cognitive impairment. Individual Cog-4 items correlated with certain MoCA domains, but the strength of association was modest (r 5 2.44 orientation, 2.37 language, 2.19 for inattention, and no significant correlation for executive function, P 5 .72). Conclusions: Our data suggest that many stroke survivors with MoCA-defined cognitive problems would not be detected by Cog-4. Subtest correlations suggest that Cog-4 may not be a valid measure of the cognitive domains that it purports to describe. Other brief cognitive screening tests may be better suited to acute stroke. Key Words: Cognitive impairment—Cog-4—MoCA—sensitivity—scales— specificity—stroke—screening. Ó 2014 by National Stroke Association
Introduction From the *Institute of Cardiovascular and Medical Sciences, School of Medicine, University of Glasgow; and †Undergraduate Medical School, School of Medicine, University of Glasgow, Glasgow, UK. Received October 8, 2013; revision received November 14, 2013; accepted December 29, 2013. Funding: R.L. is supported by a Chest Heart and Stroke Scotland research grant; this study was supported by a ‘‘start up’’ grant from the British Geriatrics Society (Scotland). Conflicts of interest: None declared. Address correspondence to T. J. Quinn, Department of Academic Geriatric Medicine, Walton Building, Glasgow Royal Infirmary, Glasgow, UK G4 0SF. E-mail: [email protected]. 1052-3057/$ - see front matter Ó 2014 by National Stroke Association http://dx.doi.org/10.1016/j.jstrokecerebrovasdis.2013.12.042
Cognitive impairment affects around one third of stroke survivors.1,2 Cognitive deficits can negatively affect many aspects of patient recovery. Poststroke cognitive impairment is associated with poor functional recovery and increased mortality.3-5 Stroke survivors with cognitive deficits may have improved outcomes if diagnosis is made at an early stage. Timely diagnosis of potential cognitive issues will allow for appropriate intervention and follow-up.6 Clinical guidelines in the United Kingdom and other countries, therefore, recommend that all acute stroke admissions are screened for cognitive deficits.7
Journal of Stroke and Cerebrovascular Diseases, Vol. -, No. - (---), 2014: pp 1-7
1
R. LEES ET AL.
2
A variety of tools are available for measuring cognitive impairment. However, there is no consensus as to preferred test strategy. Reviews of cognitive scales used in both clinical and research settings have shown substantial heterogeneity in choice and application of testing instrument.8,9 The ideal would be detailed specialist assessment, supplemented by a comprehensive neuropsychological battery. Such an approach is not feasible in a busy acute setting. Usual practice is to employ a 2-step process for assessment: initial brief screening followed by furthermore detailed assessment by a specialist if screening detects problems. The National Institutes Stroke Scale (NIHSS) is an impairment scale commonly employed in stroke care. NIHSS assesses neurologic function through measurement of consciousness, language, neglect, visual fields, eye movement, facial symmetry, motor strength, sensation, and co-ordination.10 Certain elements of the NIHSS, (1b) orientation, (1c) executive function, (9) language, and (11) inattention, have been suggested as a short cognitive screening test—the Cog-4.11 The Cog-4 has been shown to provide some indication of cognitive impairment at 18 months poststroke, although it performs less well than the more detailed screening tool of Folstein MiniMental State Examination.11 However, Mini-Mental State Examination may not be the ideal comparator as it has limited ability to describe the executive impairments commonly seen in stroke cohorts.12,13 Studies of responsiveness of Cog-4 has suggested that Cog-4 assesses only a limited range of possible cognitive outcomes with a marked ‘‘floor’’ effect14,15 and lacks sensitivity depending on the hemisphere affected.14-16 However, other aspects of the Cog-4 are desirable for the acute stroke setting; the assessment is short in duration, does not add to test burden, and can be derived from routinely collected data. Cognitive assessment in acute stroke can be complicated by speech disturbance,17 physical impairments, and concomitant medical complications including delirium.18-20 A valid assessment of the test properties of the Cog-4 as a screening tool must include subjects representative of those stroke patients likely to be assessed in routine clinical practice. Our intention was to mirror ‘‘usual practice’’ through comparison of as a brief tool with a clinically accepted measure in a representative sample at the acute stage. To describe test accuracy and validity of the Cog-4, we used a multidomain, cognitive assessment recommended for all-cause vascular cognitive impairment, the Montreal Cognitive Assessment (MoCA).21-23 The MoCA was originally developed as a brief screening tool to identify older adults with cognitive impairment that score in ‘‘normal range’’ on other commonly used assessments.22 Although not specifically designed for use in stroke, MoCA has many features that make it a useful scale for assessment of stroke survivors and is increasingly used
21-23
in this population. In specific relation to our study, MoCA was chosen as a cognitive assessment as its test domains allow for individual component analysis of Cog-4 items while still offering a valid global test of cognitive function.23 Our primary aims were to test the accuracy of Cog-4 for detection of MoCA-defined cognitive impairment and to describe correlation of individual Cog-4 items and broadly corresponding MoCA cognitive domains.
Subjects and Methods We devised the study in line with methodological (ie, assessment administration) and reporting guidance for diagnostic test accuracy studies, including the dementia-specific guidance STARDdem.24-26 We accept that STARDdem is most appropriate to those studies that use a dementia diagnosis reference standard. Although our reference standard was cognitive impairment on MoCA rather than clinical dementia, the best practice described in STARDdem is still applicable to our study. The study had approval from the local research ethics committee.
Setting The study was conducted across the stroke units of 2 urban teaching hospitals (Glasgow Royal Infirmary and Western Infirmary Glasgow, UK). Data were collected in 2 cohorts between April 2012 to June 2012 and December 2012 to April 2013. Both units admit all suspected strokes regardless of age, prestroke function, or severity of stroke.
Participants Patients were consecutive inpatient stroke survivors. Verbal consent was attained before assessment that took place between days 1 and 4 poststroke (based on day of admission). We did not set a priori exclusion criteria relating to medical instability; the treating clinical team made a decision on whether the patient was too unwell or unsuitable for any form of cognitive assessment, for example, coma or end of life care. Clinical and demographic details were independently collected from the case notes. Stroke classifications were based on the Oxford Community Project Classification,27 lacunar stroke, partial anterior circulation stroke (PACS), posterior circulation stroke, total anterior circulation stroke (TACS), transient ischemic attack, and other/unclassified. We assessed various risk factors for cognitive impairment, all based on previous diagnosis extracted from patient health records: previous dementia, depression, and prestroke visual/hearing impairments.
Assessments Our index tests were the subscales of the NIHSS (orientation, executive function, language, and inattention) that
PROPERTIES OF COG-4 ASSESSMENT
3
Table 1. The Cog-4 assessment, domains and scoring Cog-4 1 (b)
1 (c)
9
11
Level of consciousness— questions (month and age) Answers both correctly Answers one question correctly Answers neither questions correctly Level of consciousness— commands (open and close eyes; grip and release hand) Performs both task correctly Performs 1 task correctly Performs neither task correctly Best language No aphasia Mild-to-moderate aphasia Severe aphasia Mute and global aphasia Extinction and inattention No inattention Mild inattention Severe inattention
Score
0 1 2
0 1 2 0 1 2 3 0 1 2
Abbreviation: Cog-4, 4 cognitive areas of the National Institutes of Health Stroke Scale.
make up the Cog-4.11 Each component is scored from 0 to 2 or 3 with more than 0 being considered abnormal (Table 1); increasing scores describe greater severity of impairment within that domain.14 Our reference standard was the MoCA. Component domain scores range from 1 to 6 and include visuospatial/executive function, animal naming, short- and long-term memory, attention, language, abstraction, and orientation.
Figure 1. Recruitment flow chart. Abbreviations: Cog-4, 4 cognitive areas of the National Institute of Health Stroke Scale; MoCA, Montreal Cognitive Assessment.
In Cog-4, elements of the NIHSS are taken as proxies of cognitive domains: NIH 1b, orientation; NIH 1c, executive function; NIH 9, language; and NIH 11 (visual), inattention. We compared with corresponding domains of MoCA. Cog4 and MoCA had certain domains that purported to measure the same construct, orientation, executive function, and language; the Cog4 domain of (visual) inattention had no direct equivalent within MoCA, and we chose visuospatial function as being closest in nature.28 We employed the recommended standard test cut points of 1 or more for Cog-414 and less than 26 of 30 for the MoCA22 as testing positive for cognitive impairment. Assessments were not modified for patients with specific impairments; however, disabilities that may have impeded scores were noted.
Procedure Both measures were performed on each patient on the same day in a randomized order by different researchers who were blinded to each other’s results. Where no score was available in the Cog-4 or MoCA, these participants were removed from analysis. The total scores were calculated and classified from available data, where participants who were unable to complete a particular section of the test were scored as 0.
Analysis Data extraction from proformas and analyses were performed by a single researcher (R.L.). Our primary analysis was the test accuracy of total Cog-4 for the binary outcome cognitive impairment/no cognitive impairment on MoCA. Secondary analyses described correlation between individual components of the Cog-4 and the corresponding domains of the MoCA.
R. LEES ET AL.
4
Table 2. Test accuracy of Cog-4 and MoCA as ‘‘3 3 3’’ table MoCA MoCA 1ve MoCA –ve untestable Cog-4 1ve Cog-4 2ve Cog-4 untestable
45 79 0
1 23 0
17 1 7
Abbreviations: Cog-4, 4 cognitive areas of the National Institutes of Health Stroke Scale, diagnostic threshold used was more than 1 of 12; MoCA, Montreal Cognitive Assessment, diagnostic threshold used was less than 26 of 30; 1ve, case positive for cognitive impairment; 2ve, case negative for cognitive impairment.
We created a 2 3 2 and 3 3 3 data table for primary analyses. The 3 3 3 table allows for those unable to complete testing to be considered in the analysis and should give a better reflection of how the test performs in clinical practice.29 From these, we calculated sensitivity, specificity, and positive and negative predictive values with corresponding 95% confidence intervals (CIs). We recognized that NIHSS scores favor dominant hemisphere and anterior lesions. The effect of hemisphere on Cog-4 properties has previously been described14-16; on advice from reviewers, we performed post hoc subgroup analyses, repeating the analysis limited to those with TACS/PACS. Statistical analyses were performed using SPSS (IBM Ltd., Portsmouth, UK, version 19.0), Minitab (Minitab Ltd., State College, PA,
windows version 15), and Statsdirect software (Stats Direct Ltd., Cheshire, UK, version 2.7.9).
Results Over the 24-week study period, 203 subjects admitted were potentially eligible for assessment. Data were collected from 173 with 30 patients (15%) refusing cognitive assessment (Fig 1). Participants were 53% men (n 5 92), median age 74 years (interquartile range [IQR]: 63-83), and 70 (40.5%) had a history of stroke. Those refusing cognitive assessment had similar profiles to those recruited: median age 77 years (IQR: 67-86) and median NIHSS 4 (IQR: 2-8). The majority of participants were able to complete the screening assessments: 166 had Cog-4 data and 148 MoCA (Table 2). Some participants were only able to score part of the MoCA; in this instance, those domains that could not be scored were coded as zero; this was required in n 5 12 (7%). Stroke classifications by the Oxford Community Project Classification were lacunar stroke (n 5 38, 22%), PACS (n 5 54, 31%), posterior circulation stroke (n 5 12, 7%), TACS (n 5 27, 16%), TIA (n 5 10, 6%), and other/unclassified (n 5 32, 18%). Median total NIHSS was 3 (IQR: 2-6; range: 0-24) and median Cog-4 was 0 (IQR: 0-1; range: 0-8). There were various risk factors for cognitive impairment within subjects: 16 (9%) had a previous diagnosis of dementia, 15 (8.7%) had depression, and 26 (15%) had prestroke visual or hearing impairments (Table 3).
Table 3. Clinical and demographic information of participants
Participant demographics Male Age Prior stroke OCSP classification LAC PAC POC TAC TIA Unclassified Cognitive impairment risk factors Dementia Depression Visual/hearing impairment Assessments NIHSS Cog-4
Participants, n (%)
Median
IQR
Range
92 (53%) — 70 (40.5%)
— 74 —
— 63-83 —
— 28-97 —
38 (22%) 54 (31%) 12 (7%) 27 (16%) 10 (6%) 32 (18%)
— — — — — —
— — — — — —
— — — — — —
16 (9%) 15 (8.7%) 26 (15%)
— — —
— — —
— — —
— —
3 0
2-6 0-1
0-24 0-8
Abbreviations: Cog-4, 4 cognitive areas of the NIHSS; IQR, interquartile range; LAC, lacunar stroke; NIHSS, National Institutes of Health Stroke Scale; OCSP, Oxford Community Stroke Project; PACS, partial anterior circulation stroke; POCS, posterior circulation stroke; TACS, total anterior circulation stroke; TIA, transient ischemic attack.
PROPERTIES OF COG-4 ASSESSMENT
5
Table 4. Test accuracy comparisons between total Cog-4 and MoCA
Cog-4
Sensitivity
Specificity
Positive predictive value
Negative predictive value
Estimate 95% CI
.36 .28-.45
.96 .80-.99
.98 .89-1.00
.23 .16-.32
Correlation (r)
Significance (P value)
2.48
,.0001
Abbreviations: CI, confidence interval; Cog-4, 4 cognitive areas of the National Institutes of Health Stroke Scale, diagnostic threshold used was more than 1 of 12; MoCA, Montreal Cognitive Assessment, diagnostic threshold used was less than 26 of 30.
The MoCA recorded 82% (n 5 124) of participants with data available as having cognitive impairment at standard diagnostic threshold and the Cog-4 recorded 37% (n 5 62). For test accuracy, Cog-4 had a specificity of .96 (95% CI: .80-.99), sensitivity .36 (95% CI: .28-.45), positive predictive value .98 (95% CI: .89-1.00), negative predictive value .23 (95% CI: .16-.32; Table 4). The correlation between total scores on the 2 assessments was significant (r 5 2.48, P , .0001). In the individual domain analyses, significant correlations were found for orientation (P , .0001), language (P , .0001), and inattention (P 5 .02), albeit the strength of association was modest, r 5 2.44, 2.37, and 2.19, respectively (Table 5). Our post hoc subgroup analysis suggested that test properties of Cog-4 were not improved when the test was limited to those with anterior strokes (TACS and PACS): sensitivity .53 (95% CI: .41-.65), specificity .50 (95% CI: .46-.99), positive predictive value .99 (95% CI: .87-1.00), and negative predictive value .13 (95% CI: .06.28). As prevalence of MoCA-defined cognitive impairment was higher than anticipated, we performed post hoc analyses describing test properties of Cog-4 at 2 other threshold MoCA scores that have been suggested for use in stroke populations. Sensitivity improved with lower MoCA thresholds but remained suboptimal (Table 6).
Discussion We have demonstrated suboptimal validity and test accuracy of Cog-4 as a brief cognitive screening assessment for acute stroke. Our analysis showed favorable specificity but at the expense of poor sensitivity. Although total Cog-4 showed significant correlation with MoCA, the association was modest. Three individual Cog-4 domains were significantly correlated with corresponding cognitive domains (orientation, language, and inattention) although again the strength of association was at the best modest. Our data suggest that many stroke survivors with potential cognitive problems would not be picked up by Cog-4 testing and other brief cognitive screening tests may be better suited to acute stroke. The correlation
data suggest that Cog-4 items may not be robust measures of the cognitive domains they purport to measure. Our test accuracy data are in keeping with previous research that has described reduced sensitivity of Cog-4 to detect cognitive impairment.11 Our analysis was necessarily pragmatic. We recognize that MoCA is not a definitive diagnostic test rather MoCA offers a screening tool, albeit a more detailed tool than Cog-4. However, MoCA was specifically designed for assessment of the cognitive domains of interest, whereas Cog-4 extrapolates neurologic impairment data as a proxy for cognitive assessment, and so we feel our analysis of correlation and test accuracy is still valid. Our reference standard, the MoCA, recorded 82% (n 5 124) of participants as having cognitive impairment. This is higher than previous study estimates; however, previous studies were not performed in our very acute time frame.30-32 In the absence of consensus agreement for MoCA diagnostic threshold in the stroke setting, we used the standard cutoff described for MoCA. The very high prevalence of cognitive impairment recorded suggests that this threshold may be too high. Other authors have commented that similar and alternative cut points have been suggested30,33 as some domain impairments may be transient and not indicatory of long-term cognitive problems.34,35 In our post hoc analysis, lower cut points in the MoCA improve the properties of the Cog-4 to an extent but not sufficiently to recommend the use of the Cog-4. The substantial early cognitive burden in acute stroke we have demonstrated is a factor to be mindful of in context of recommendations for cognitive screening in acute stroke and an area that would benefit from further research to
Table 5. Correlations between subdomains of Cog-4 and MoCA
Cog-4 sub-domain
Correlation (r)
Significance (P value)
Orientation Executive function Language Inattention
2.44 2.03 2.37 2.19
,.0001 .72 ,.0001 .02
Abbreviations: Cog-4, 4 cognitive areas of the National Institute of Health Stroke Scale; MoCA, Montreal Cognitive Assessment.
R. LEES ET AL.
6
Table 6. Test accuracy comparisons between the total Cog-4 and MoCA at different cut points Cog-4
Sensitivity , 20
Specificity , 20
Sensitivity , 24
Specificity , 24
Estimate 95% CI
.49 .37-.60
.86 .76-.92
.40 .31-.50
.89 .77-.95
Abbreviations: Cog-4, 4 cognitive areas of the National Institutes of Health Stroke Scale, diagnostic threshold used was 1 or more of 12; MoCA, Montreal Cognitive Assessment, diagnostic threshold used was less than 20 of 30 and less than 24 of 30.
describe the incidence and natural history of these early cognitive impairments. The limitations of this work are that assessors although trained in the cognitive tools were not experienced stroke clinicians. However, part of an effective screening test is the ability to be used by any member of the stroke care team. Our assessments were focussed on test accuracy and comparative validity, and we did not describe metrics relating to feasibility (eg, time taken to complete assessments) or management changes resulting from cognitive assessment results. Strengths of this work are the clinical setting and inclusive approach, which should improve external validity. We present data relevant to an under-researched but topical area of stroke care, with data collected and presented following best practice guidelines. Our data suggest that although cognitive screening within the acute stroke setting is generally feasible, Cog4 may not be suited to this purpose. In particular, Cog-4 may be insufficiently sensitive to cognitive impairment in stroke survivors, and certain domains of Cog-4 may not be valid measures of the cognitive constructs they purport to describe. We recommend formal diagnostic test accuracy analysis of other short screening tools to inform the choice of optimal assessment at various stages of the stroke survivor journey. Acknowledgment: We would like to thank the stroke care teams for their co-operation and help in identifying appropriate patients. We thank Anna Noel-Storr, Cochrane Dementia and Cognitive Improvement Group, for sharing STARDdem reporting guidance.
References 1. Patel M, Coshall C, Rudd AG, et al. Natural history of cognitive impairment after stroke and factors associated with its recovery. Clin Rehabil 2003;17:158-166. 2. Lawrence ES, Coshall C, Dundas R, et al. Estimates of the prevalence of acute stroke impairments and disability in a multiethnic population. Stroke 2001;32:1279-1284. 3. Robertson IH, Ridgeway V, Greenfield E, et al. Motor recovery after stroke depends on intact sustained attention: a 2-year follow-up study. Neuropsychology 1997; 11:290-295. 4. Robinson RG, Bollawilson K, Kaplan E, et al. Depression influences intellectual impairment in stroke patients. Br J Psychiatry 1986;148:541-547.
5. Patel MD, Coshall C, Rudd AG, et al. Cognitive impairment after stroke: clinical determinants and its associations with long-term stroke outcomes. J Am Geriatrics Soc 2002;50:700-706. 6. Zinn S, Dudley TK, Bosworth HB, et al. The effect of poststroke cognitive impairment on rehabilitation process and functional outcome. Arch Phys Med Rehabil 2004; 85:1084-1090. 7. Scottish Intercollegiate Guidelines Network (SIGN). Management of patients with stroke. Rehabilitation, prevention and management of complications and discharge planning. In: Scottish Intercollegiate Guidelines Network, ed. SIGN Publication Guideline 2010 . Edinburgh , UK: Healthcare Improvement Scotland (HIS), 2010. 8. Lees R, Fearon P, Harrison JK, et al. Cognitive and mood assessment in stroke research: focused review of contemporary studies. Stroke 2012;43:1678-1680. 9. Lees R, Quinn T, Broomfield N. Questionnaire assessment of usual practice in mood and cognitive assessment in Scottish stroke units. Disability Rehabil. http://dx.doi. org/10.3109/09638288.2013.791728. 10. Brott T, Adams HP Jr, Olinger CP, et al. Measurements of acute cerebral infarction: a clinical examination scale. Stroke 1989;20:864-870. 11. Cumming TB, Blomstrand C, Bernhardt J, et al. The NIH stroke scale can establish cognitive function after stroke. Cerebrovasc Dis 2010;30:7-14. 12. Bour A, Rasquin S, Boreas A, et al. How predictive is the MMSE for cognitive performance after stroke? J Neurol 2010;257:630-637. 13. Nys GM, van Zandvoort MJ, de Kort PL, et al. Restrictions of the Mini-Mental State Examination in acute stroke. Arch Clin Neuropsychol 2005;20:623-629. 14. Ankolekar S, Renton C, Sprigg N, et al. The Cog-4 subset of the National Institutes of Health Stroke Scale as a measure of cognition: relationship with baseline factors and functional outcome after stroke using data from the virtual International Stroke Trials Archive. Stroke Res Treat 2013;2013:6. 15. Hajjar K, Fulton RL, Diener HC, et al. Does the cognitive measure Cog-4 show improvement among patients treated with thrombolysis after acute stroke? Int J Stroke 2013;8:652-656. 16. Gottesman RF, Kleinman JT, Davis C, et al. The NIHSSplus: improving cognitive assessment with the NIHSS. Behav Neurol 2010;22:11-15. 17. Thomas SA, Lincoln NB. Factors relating to depression after stroke. Br J Clin Psychol 2006;45:49-61. 18. Laska AC, Martensson B, Kahan T, et al. Recognition of depression in aphasic stroke patients. Cerebrovasc Dis 2007;24:74-79. 19. Lees R, Corbet S, Johnston C, et al. Test accuracy of short screening tests for diagnosis of delirium or cognitive impairment in an acute stroke setting. Stroke 2013; 44:3078-3083.
PROPERTIES OF COG-4 ASSESSMENT 20. Quaranta D, Marra C, Gainotti G. Mood disorders after stroke: diagnostic validation of the poststroke depression rating scale. Cerebrovasc Dis 2008;26:237-243. 21. Hachinski V, Iadecola C, Petersen RC, et al. National Institute of Neurological Disorders and StrokeCanadian Stroke Network vascular cognitive impairment harmonization standards. Stroke 2006;37:2220-2241. 22. Nasreddine ZS, Phillips NA, Bedirian V, et al. The Montreal Cognitive Assessment, MoCA: a brief screening tool for mild cognitive impairment. J Am Geriatr Soc 2005;53:695-699. 23. Pendlebury ST, Mariz J, Bull L, et al. MoCA, ACE-R, and MMSE versus the National Institute of Neurological Disorders and Stroke-Canadian Stroke Network Vascular Cognitive Impairment Harmonization Standards Neuropsychological Battery after TIA and stroke. Stroke 2012; 43:464-469. 24. Bossuyt PM, Reitsma JB, Bruns DE, et al. Towards complete and accurate reporting of studies of diagnostic accuracy: the STARD initiative. Br Med J 2003; 326:41-44. 25. Whiting PF, Rutjes AW, Westwood ME, et al. QUADAS-2: a revised tool for the quality assessment of diagnostic accuracy studies. Ann Intern Med 2011;155:529-536. 26. STARDdem Initiative. Available at: http://dementia. cochrane.org. Accessed January 2014. 27. Bamford J, Sandercock P, Dennis M, et al. Classification and natural history of clinically identifiable subtypes of cerebral infarction. Lancet 1991;337:1521-1526. 28. Damian AM, Jacobson SA, Hentz JG, et al. The Montreal Cognitive Assessment and the mini-mental state exami-
7
29.
30.
31.
32.
33.
34.
35.
nation as screening instruments for cognitive impairment: item analyses and threshold scores. Dementia Geriatr Cogn Disord 2011;31:126-131. Schuetz GM, Schlattmann P, Dewey M. Use of 3 3 2 tables with an intention to diagnose approach to assess clinical performance of diagnostic tests: meta-analytical evaluation of coronary CT angiography studies. Br Med J 2012;345. Dong YH, Sharma VK, Chan BPL, et al. The Montreal Cognitive Assessment (MoCA) is superior to the MiniMental State Examination (MMSE) for the detection of vascular cognitive impairment after acute stroke. J Neurol Sci 2010;299:15-18. Riepe MW, Riss S, Bittner D, et al. Screening for cognitive impairment in patients with acute stroke. Dementia Geriatr Cogn Disord 2004;17:49-53. Schmidt R, Mechtler L, Kinkel PR, et al. Cognitive impairment after acute supratentorial stroke: a 6-month followup clinical and computed tomographic study. Eur Arch Psychiatry Clin Neurosci 1993;243:11-15. Godefroy O, Fickl A, Roussel M, et al. Is the Montreal Cognitive Assessment superior to the Mini-Mental State examination to detect poststroke cognitive impairment? A study with neuropsychological evaluation. Stroke 2011;42:1712-1716. Hurford R, Charidimou A, Fox Z, et al. Domain-specific trends in cognitive impairment after acute ischaemic stroke. J Neurol 2013;260:237-241. Pendlebury ST, Wadling S, Silver LE, et al. Transient cognitive impairment in TIA and minor stroke. Stroke 2011;42:3116-3121.
