Modest Association between the Discharge Modified Rankin Scale Score and Symptomatic Intracerebral Hemorrhage after Intravenous Thrombolysis David Asuzu, PhD,* Karin Nystrom, APRN,† Hardik Amin, MD,† Joseph Schindler, MD,*† Charles Wira, MD,*† David Greer, MD, MA, FAHA, FCCM, FNCS, FAAN,*† Nai Fang Chi, MD,‡ Janet Halliday, RN,† and Kevin N. Sheth, MD, FAHA, FCCM, FNCS*†
Background: Thirty- and 90-day modified Rankin Scale (mRS) scores are used to monitor adverse outcome or symptomatic intracerebral hemorrhage (sICH) in ischemic stroke patients after intravenous (IV) thrombolytic therapy. Discharge mRS scores are more readily available and could serve as a proxy for 30- or 90-day mRS data. Our goal was to evaluate agreement between the discharge mRS score and sICH. Additionally, we tested for correlations between the discharge mRS score and 8 clinical scores developed to predict sICH or adverse outcomes based on 90-day mRS data. Methods: Clinical data were analyzed from 210 patients receiving IV thrombolysis from January 2009 till December 2013 at the Yale New Haven Hospital. Agreement between sICH and the discharge mRS score was assessed using linear kappa. Eight clinical scores were calculated for each patient and compared with the discharge mRS score by univariate logistic regression. Goodness of fit was tested by receiver operating characteristic (ROC) analysis and by Hosmer–Lemeshow statistics. Results: We found only modest agreement between sICH and unfavorable discharge mRS scores (mRS $5), with kappa .22, P 5 .0001. All 8 clinical scores tested showed good agreement with discharge mRS score of 5 or more (ROC area ..7). Conclusions: The discharge mRS score shows only modest agreement with sICH and therefore cannot be recommended as a proxy for 30- or 90-day mRS data. However, the discharge mRS score correlates strongly with clinical scores predicting long-term adverse outcome; therefore, assessment of discharge mRS scores may be of some clinical benefit. Key Words: Ischemic stroke—hemorrhagic transformation—IV thrombolysis—alteplase—modified Rankin scale—symptomatic intracerebral hemorrhage. Ó 2015 by National Stroke Association
From the *Yale School of Medicine, New Haven, Connecticut; †Yale Emergency Medicine, Division of Vascular Neurology, Division of Neurocritical Care and Emergency Neurology, Yale-New Haven Hospital, New Haven, Connecticut; and ‡Department of Neurology, Taipei Medical University Hospital and Shuang Ho Hospital, Taipei, Taiwan. Received August 4, 2014; accepted September 29, 2014. Address correspondence to Kevin N. Sheth, MD, FAHA, FCCM, FNCS, 20 York Street, New Haven, CT 06510. E-mail: kevin.sheth@ yale.edu. 1052-3057/$ - see front matter Ó 2015 by National Stroke Association http://dx.doi.org/10.1016/j.jstrokecerebrovasdis.2014.09.034
548
The Rankin Scale was modified by the United Kingdom Transient Ischaemic Attack study group to measure overall independence after a stroke.1 This modified Rankin Scale (mRS) gives a better indication of the patient’s ability to look after themselves in daily life and shows moderate-to-good interobserver agreement.2 Thirty- or 90-day mRS scores are routinely used to measure longterm functional deficits after ischemic stroke. They are particularly useful for monitoring adverse outcomes associated with symptomatic intracerebral hemorrhage (sICH) in ischemic stroke patients treated with intravenous (IV) thrombolysis (rt-PA), because standard dose
Journal of Stroke and Cerebrovascular Diseases, Vol. 24, No. 3 (March), 2015: pp 548-553
MODEST PREDICTIVE ABILITY OF mRS SCORE AT DISCHARGE
rt-PA administration carries a significant risk of subsequent sICH—up to 6%-30% depending on the sICH definition in use.3-5 Long-term mRS scores are not readily available at many stroke centers because they require structured interviews performed by trained and certified raters.6 Discharge mRS scores on the other hand are routinely assessed at most stroke centers and are strong independent predictors of disability at 90 days, although they showed only modest agreement with 90-day mRS scores in 1 study.7 Discharge mRS scores are an attractive alternative to long-term mRS scores for assessing functional outcome in ischemic stroke patients after rt-PA therapy. A number of other clinical scores have been developed to predict sICH or adverse outcome in patients undergoing IV thrombolytic therapy, including the StrokeThrombolytic Predictive Instrument (Stroke-TPI),8 iSCORE,9 DRAGON ([hyper]Dense cerebral artery sign/ early infarct signs on admission CT scan, prestroke modified Rankin Scale (mRS) score, Age, Glucose level at baseline, Onset-to-treatment time, and baseline National Institutes of Health Stroke Scale),10 Stroke Prognostication using Age and NIH Stroke Scale-100 (SPAN-100),11 Acute Stroke Registry and Analysis of Lausanne (ASTRAL),12 Post-thrombolysis Risk Score (PRS),13 Hemorrhage After Thrombolysis,14 SEDAN (Sugar, Early infarct signs and [hyper]Dense cerebral artery sign on admission computed tomography scan, Age, and NIH Stroke Scale on admission),15 and Safe Implementation of Treatments in Stroke Symptomatic Intracerebral Hemorrhage (SITSICH) scores.16 Several of these scores were developed to predict patient outcomes based on 90-day mRS scores; however, their correlation with discharge mRS scores have not been previously tested. In this study, we investigated agreement between discharge mRS scores and the presence of sICH. Additionally, we tested for correlations between discharge mRS scores and 8 clinical scores for predicting sICH or long-term adverse outcome after rtPA administration.
Methods Patient Data Demographic and clinical data were retrospectively analyzed from all consecutive patients (n 5 210) who had a principal or secondary discharge diagnosis of ischemic stroke and received IV rt-PA therapy from January 2009 until July 2013 at Yale New Haven Hospital.17 Patients transferred from an outside institution after IV rt-PA administration were excluded, as were patients who subsequently underwent intra-arterial rt-PA or neurointerventional procedures with mechanical clot retrieval devices. One patient was excluded because of incomplete data. Eligibility criteria for IV rt-PA treatment were applied following the American Heart Association guidelines.18 This study was approved by the Yale
549
Human Investigation Committee and the Yale Human Research Protection Program.
Imaging Data Computed tomography (CT) or magnetic resonance imaging scans were performed on each patient before IV rt-PA treatment, at 24 hours after treatment, and subsequent to any observed clinical deterioration. Neuroradiologic assessment of hypodense CT lesions, hyperdense vessels, and intracerebral hemorrhage was performed on each patient by a trained neurologist (H.A.). Stroke severity was assessed by National Institute of Health Stroke Scale (NIHSS) score at baseline.
Clinical Scores and Outcome Data The discharge mRS score was calculated for each patient as a continuous variable and discharge mRS scores of 5 or more and 4 or more as binary variables. Eight clinical scores were also calculated for each patient: Stroke-TPI, DRAGON, SPAN-100, ASTRAL, PRS, Hemorrhage After Thrombolysis, SEDAN, and SITS-ICH. Detailed derivations of each score have been published elsewhere8,9,11-15 and are summarized in Table 1. The iSCORE was excluded because, similar to other authors’ concerns,19 it required detailed patient records including history of renal dialysis and congestive heart failure, which are not readily available in the hyperacute stroke setting. For the Stroke-TPI score, parameter estimates for predicting mRS scores of 5 or more without ASPECTS scores were used.8 Rather than deriving probabilities from the inverse logit function, parameter estimates were calculated directly for each patient and summed to generate a raw score, which allowed more meaningful comparison with the other scores. All other clinical scores were calculated as previously published.
Statistical Analysis Agreement between sICH and discharge mRS score of 5 or more or mRS scores of 4 or more was assessed using linear weighted kappa, which measures agreement between 2 raters or metrics after excluding the effect of chance.20 Univariate logistic regression was performed using each clinical score as an independent variable and discharge mRS, discharge mRS scores of 5 or more, or discharge mRS scores of 4 or more as the dependent variable. Goodness of fit was tested using Hosmer– Lemeshow statistics and areas under the receiver operating characteristic (ROC) curve using the DeLong method.21 P values less than .05 (2-tailed) were considered statistically significant. All analyses were performed using STATA 13 software package (StataCorp LP, College Station, TX).
D. ASUZU ET AL.
550
Table 1. Patient characteristics used to derive clinical scores Score
End points
Components
Stroke-TPI DRAGON
90-day mRS scores 90-day mRS scores
SPAN-100
ICH and 90-day composite of mRS scores, NIHSS, Barthel index and Glasgow Outcome Scale scores 90-day mRS scores
ASTRAL PRS HAT SEDAN SITS-ICH
ICH, sICH, and PH ICH, sICH, and hemorrhages with final fatal outcome sICH sICH
Age, NIHSS, glucose Age, prestroke mRS, HDMCA or early CT infarct, glucose, OTT, admission NIHSS Age, admission NIHSS
Age, admission NIHSS, OTT, decreased level of consciousness, visual field defects, glucose Age, admission NIHSS, glucose, platelets DM or glucose, admission NIHSS, early CT hypodensity Age, NIHSS, glucose, HDMCA sign, early CT infarct Age, weight, hypertension, aspirin/clopidogrel, admission NIHSS, systolic BP, glucose, OTT
Abbreviations: ASTRAL, Acute Stroke Registry and Analysis of Lausanne; BP, blood pressure; CT, computed tomography; DM, diabetes mellitus; HAT, Hemorrhage After Thrombolysis; HDMCA, hyperdense middle cerebral artery sign; ICH, intracerebral hemorrhage; mRS, modified Rankin Scale; NIHSS, National Institute of Health Stroke Scale; OTT, onset-to-treatment; PH, parenchymal hemorrhage; PRS, post-thrombolysis risk score; sICH, symptomatic intracerebral hemorrhage; SITS-ICH, Safe Implementation of Treatments in Stroke Symptomatic Intracerebral Hemorrhage; SPAN, Stroke Prognostication using Age and NIH Stroke Scale; TPI, Thrombolytic Predictive Instrument. References.8,10-16
Results A total of 210 patients received rt-PA therapy at Yale New Haven Hospital between January 2009 and December 2013. A total of 48.7% were male. Forty-nine patients (23.3%) had a history of diabetes, 155 (73.8%) had hypertension, 46 (21.9%) had a prior TIA or stroke, and 97 (46.2%) were on aspirin or clopidogrel. Median baseline NIHSS score was 10, and mean onset-totreatment (OTT) time was 148 minutes (Table S1 in Appendix).17 Symptomatic ICH occurred in 12 patients (5.71%). Thirty-seven patients (17.6%) had discharge mRS scores of 5 or more, and 151 patients (71.9%) had discharge mRS scores of 4 or more. First, we assessed the correlation between discharge mRS scores and sICH. There was 83.4% agreement be-
tween patients with sICH and discharge mRS scores of 5 or more (kappa .22, P , .001). When discharge mRS was relaxed to include scores of 4 (discharge mRS $4), agreement decreased to 34.1% (kappa .05, P 5 .01). Next, we performed logistic regression and ROC curve analysis using discharge mRS scores of 5 or more or mRS scores of 4 or more as the dependent variable and each of the 8 clinical scores as independent variables. All clinical scores showed good agreement with discharge mRS scores of 5 or more (ROC area ..7, Table 2). The 2 clinical scores showing the best agreement with discharge mRS scores of 5 or more were Stroke-TPI (ROC, .86 6 .07) and ASTRAL (ROC, .85 6 .07), with odds ratios of 1.3 6 .44 and .17 6 .06, respectively. SPAN-100 showed the least agreement with discharge mRS scores of 5 or more (ROC, .71 6 .09) with odds ratio 2.09 6 .79.
Table 2. Comparison of clinical scores for correlation with discharge mRS scores using mRS $5 Clinical score
Odds ratio
Z statistic
P . jzj
Hosmer–Lemeshow, c2
Groups
P . c2
ROC area
Stroke-TPI ASTRAL DRAGON SEDAN PRS HAT SITS-ICH SPAN-100
1.30 (.86-1.73) .17 (.12-.23) 1.66 (1.35-2.05) .83 (.47-1.21) 1.45 (.86-2.04) .94 (.59-1.29) .46 (.24-.67) 2.09 (1.30-2.87)
5.84 5.91 4.76 4.42 4.84 5.28 4.21 5.21
,.001 ,.001 ,.001 ,.001 ,.001 ,.001 ,.001 ,.001
9.62 5.96 6.05 4.41 .84 .34 10.44 0
10 10 7 5 4 5 7 2
.29 .65 .30 .22 .66 .95 .06 1
.86 (.80-.94) .85 (.79-.93) .76 (.67-.85) .75 (.67-.82) .76 (.69-.83) .76 (.67-.85) .73 (.65-.81) .71 (.62-.79)
Abbreviations: ASTRAL, Acute Stroke Registry and Analysis of Lausanne; HAT, Hemorrhage After Thrombolysis; mRS, modified Rankin Scale; PRS, post-thrombolysis risk score; SITS-ICH, Safe Implementation of Treatments in Stroke Symptomatic Intracerebral Hemorrhage; SPAN, Stroke Prognostication using Age and NIH Stroke Scale; TPI, Thrombolytic Predictive Instrument. Values presented as mean (95% confidence interval).
MODEST PREDICTIVE ABILITY OF mRS SCORE AT DISCHARGE
551
Table 3. Comparison of clinical scores for correlation with discharge mRS scores using mRS $4 Clinical score
Odds ratio
Z statistic
P . jzj
Hosmer–Lemeshow, c2
Groups
P . c2
ROC area
ASTRAL Stroke-TPI DRAGON SEDAN HAT PRS SITS-ICH SPAN-100
.12 (.08-.17) .81 (.53-1.09) 1.41 (1.19-1.68) .88 (.53-1.23) .58 (.22-.94) 1.06 (.65-1.47) .36 (.18-.54) 2.33 (.87-3.78)
5.38 5.72 3.89 4.97 3.16 5.06 3.87 3.14
,.001 ,.001 ,.001 ,.001 .002 ,.001 ,.001 .002
4.96 3.56 2.66 5.51 3.62 1.6 5.62 0
10 10 7 5 5 4 7 2
.76 .89 .75 .14 .31 .45 .35 1
.77 (.70-.84) .78 (.71-.84) .68 (.60-.76) .73 (.66-.81) .62 (.55-.70) .72 (.64-.79) .67 (.59-.76) .62 (.57-.66)
Abbreviations: ASTRAL, Acute Stroke Registry and Analysis of Lausanne; HAT, Hemorrhage After Thrombolysis; mRS, modified Rankin Scale; PRS, post-thrombolysis risk score; SITS-ICH, Safe Implementation of Treatments in Stroke Symptomatic Intracerebral Hemorrhage; SPAN, Stroke Prognostication using Age and NIH Stroke Scale; TPI, Thrombolytic Predictive Instrument. Values presented as mean (95% confidence interval).
When the discharge mRS cutoff was relaxed to mRS scores of 4 or more, only 4 showed good agreement with mRS scores of 4 or more (ASTRAL, Stroke-TPI, SEDAN, and PRS; Table 3). Stroke-TPI and ASTRAL still showed the best agreement (ROC, .78 6 .07; odds ratio, .81 6 .28 and .77 6 .07; odds ratio, .12 6 .05, respectively), and SPAN-100 showed the least agreement with discharge mRS scores of 4 or more (ROC, .62 6 .05) with odds ratio 2.33 6 1.46. When discharge mRS was tested as a continuous variable, none of the clinical scores showed good agreement with mRS (R2 , .3 for each, Table 4).
Discussion In this study, we found only modest agreement between discharge mRS scores and sICH in ischemic stroke patients undergoing rt-PA therapy. However, we found strong correlations between discharge mRS scores and 8 clinical scores developed to predict long-term adverse outcomes after rt-PA administration. rt-PA remains the only Food and Drug Administrationapproved medical therapy for ischemic stroke patients; however, patients receiving IV rt-PA are at increased
risk for subsequent sICH and adverse clinical outcomes. Clinical scores to assess and predict sICH are critical for the optimal management of ischemic stroke patients receiving rt-PA therapy. The discharge mRS score is more readily available at clinical centers than 30- and 90-day mRS scores and is less susceptible to data collection errors, making it attractive as a proxy for 30- or 90-day mRS data. A recent study compared discharge mRS scores with 30- or 90-day mRS data but found only fair agreement with 90-day mRS, although patients with higher discharge mRS scores were more than twice as likely to suffer disability at 90 days.7 Our results similarly indicate only modest agreement between discharge mRS scores and sICH, yet we found strong correlations between the discharge mRS score and clinical scores predicting adverse longterm outcomes. We also found poor agreement between sICH and delta mRS, defined as the difference between admission mRS scores and discharge mRS scores (results not shown). The discharge mRS score therefore remains a useful tool for monitoring functional deficits at discharge but cannot be recommended as a proxy for long-term adverse outcome.
Table 4. Comparison of clinical scores for correlation with discharge mRS scores using mRS as a continuous variable Clinical score
Odds ratio
t statistic
P . jtj
R square
F statistic
P . jFj
Stroke-TPI ASTRAL DRAGON SEDAN PRS HAT SITS-ICH SPAN-100
.65 (.51-.78) .10 (.07-.12) .35 (.24-.47) .70 (.46-.87) .94 (.68-1.20) .52 (.30-.74) .30 (.18-.42) 1.5 (.93-2.10)
9.22 8.62 6.1 6.45 7.17 4.7 4.83 5.09
,.001 ,.001 ,.001 ,.001 ,.001 ,.001 ,.001 ,.001
.29 .26 .15 .17 .2 .1 .1 .11
85.1 74.3 37.19 41.55 51.4 22.1 23.34 25.89
,.001 ,.001 ,.001 ,.001 ,.001 ,.001 ,.001 ,.001
Abbreviations: ASTRAL, Acute Stroke Registry and Analysis of Lausanne; HAT, Hemorrhage After Thrombolysis; mRS, modified Rankin Scale; PRS, post-thrombolysis risk score; SITS-ICH, Safe Implementation of Treatments in Stroke Symptomatic Intracerebral Hemorrhage; SPAN, Stroke Prognostication using Age and NIH Stroke Scale; TPI, Thrombolytic Predictive Instrument. Values presented as mean (95% confidence interval).
D. ASUZU ET AL.
552
Discharge status may still carry some predictive information on patients’ long-term clinical outcome. For instance, the discharge mRS score partly determines a patient’s destination after discharge, and discharge destination is an independent predictor of 30- and 90-day outcomes.22 Furthermore, the discharge mRS score independently predicts 90-day disability7 and correlates strongly with scores predicting long-term adverse outcome (this study). It therefore appears that certain aspects of patients’ disposition at discharge could determine their long-term clinical status. Further studies are needed to identify discharge criteria, which correlate independently with long-term clinical outcomes and can thus serve as proxies for 30- and 90-day mRS scores. In this study, 2 clinical scores, Stroke-TPI and ASTRAL, showed the best agreement with discharge mRS by ROC analysis at cutoffs of mRS scores of 5 or more and mRS scores of 4 or more. These 2 clinical scores are derived using similar clinical parameters. The Stroke-TPI score is derived using age, NIHSS score, and serum glucose (Table 1). ASTRAL includes these 3 parameters in addition to symptom OTT duration, decreased level of consciousness, and visual field defects.8,12 The similarity in predictive ability between these 2 scores can therefore be attributed to the clinical parameters they share. However, the Stroke-TPI score correlates positively with discharge mRS scores whereas ASTRAL correlates negatively, suggesting that either symptom OTT duration, decreased level of consciousness, or visual field defects may be protective for mRS status at discharge. This observation warrants further clarification in future studies. Our study has a number of limitations. We were unable to directly compare discharge mRS scores with 30- and 90-day mRS data because of unavailability of follow-up mRS data at our center over the duration of this study. This reflects current limitations in availability of reliable follow-up mRS data at many centers and underscores the need for readily available discharge criteria, which can predict long-term clinical outcome. We also defined sICH in this study according to the National Institute of Neurological Diseases and Stroke trial definition. The National Institute of Neurological Diseases and Stroke trial defined a hemorrhage as symptomatic if blood was not seen on a CT scan before thrombolytic therapy and there had subsequently been either a suspicion of hemorrhage or any decline in neurologic status.3 More narrow sICH definitions such as the SITS-MOST and European Cooperative Acute Stroke Study II definitions may have shown different levels of agreement with discharge mRS scores but were not readily estimable from our data set.
Conclusions In this study we found only modest agreement between discharge mRS scores and sICH. However, we found
strong correlations between discharge mRS scores and 8 clinical scores for predicting sICH or long-term adverse outcome. Discharge mRS scores therefore contain useful information about functional status at discharge, but cannot be recommended as a proxy for 30- or 90-day mRS scores.
Supplementary Data Supplementary data related to this article can be found at http://dx.doi.org/10.1016/j.jstrokecerebrovas dis.2014.09.034.
References 1. United Kingdom transient ischaemic attack (UK-TIA) aspirin trial: interim results. UK-TIA Study Group. Br Med J (Clin Res Ed) 1988;296:316-320. 2. van Swieten JC, Koudstaal PJ, Visser MC, et al. Interobserver agreement for the assessment of handicap in stroke patients. Stroke 1988;19:604-607. 3. Tissue plasminogen activator for acute ischemic stroke. The National Institute of Neurological Disorders and Stroke rt-PA Stroke Study Group. N Engl J Med 1995; 333:1581-1587. 4. Hacke W, Kaste M, Fieschi C, et al. Intravenous thrombolysis with recombinant tissue plasminogen activator for acute hemispheric stroke. The European Cooperative Acute Stroke Study (ECASS). JAMA 1995;274:1017-1025. 5. Hacke W, Kaste M, Fieschi C, et al. Randomised doubleblind placebo-controlled trial of thrombolytic therapy with intravenous alteplase in acute ischaemic stroke (ECASS II). Second European-Australasian Acute Stroke Study Investigators. Lancet 1998;352:1245-1251. 6. Banks JL, Marotta CA. Outcomes validity and reliability of the modified Rankin scale: implications for stroke clinical trials: a literature review and synthesis. Stroke 2007; 38:1091-1096. 7. Thompson MP, Reeves M. Abstract 168: assessing the utility of the modified Rankin Scale (mRS) at discharge to predict day 90 outcomes in acute stroke registries. Circulation 2012;5(3 Supplem):A168. 8. Kent DM, Selker HP, Ruthazer R, et al. The strokethrombolytic predictive instrument: a predictive instrument for intravenous thrombolysis in acute ischemic stroke. Stroke 2006;37:2957-2962. 9. Saposnik G, Raptis S, Kapral MK, et al. The iScore predicts poor functional outcomes early after hospitalization for an acute ischemic stroke. Stroke 2011;42:3421-3428. 10. Strbian D, Meretoja A, Ahlhelm FJ, et al. Predicting outcome of IV thrombolysis-treated ischemic stroke patients: the DRAGON score. Neurology 2012;78:427-432. 11. Saposnik G, Guzik AK, Reeves M, et al. Stroke Prognostication using Age and NIH Stroke Scale: SPAN-100. Neurology 2013;80:21-28. 12. Ntaios G, Faouzi M, Ferrari J, et al. An integer-based score to predict functional outcome in acute ischemic stroke: the ASTRAL score. Neurology 2012;78: 1916-1922. 13. Cucchiara B, Tanne D, Levine SR, et al. A risk score to predict intracranial hemorrhage after recombinant tissue plasminogen activator for acute ischemic stroke. J Stroke Cerebrovasc Dis 2008;17:331-333.
MODEST PREDICTIVE ABILITY OF mRS SCORE AT DISCHARGE 14. Lou M, Safdar A, Mehdiratta M, et al. The HAT Score: a simple grading scale for predicting hemorrhage after thrombolysis. Neurology 2008;71:1417-1423. 15. Strbian D, Engelter S, Michel P, et al. Symptomatic intracranial hemorrhage after stroke thrombolysis: the SEDAN score. Ann Neurol 2012;71:634-641. 16. Mazya M, Egido JA, Ford GA, et al. Predicting the risk of symptomatic intracerebral hemorrhage in ischemic stroke treated with intravenous alteplase: Safe Implementation of Treatments in Stroke (SITS) symptomatic intracerebral hemorrhage risk score. Stroke 2012;43:1524-1531. 17. Asuzu D, Nystrom K, Amin H, et al. Comparison of 8 scores for predicting symptomatic intracerebral hemorrhage after IV thrombolysis. Neurocritical Care 2014 [Epub ahead of print]. 18. Adams HP Jr, del Zoppo G, Alberts MJ, et al. Guidelines for the early management of adults with ischemic stroke: a guideline from the American Heart Association/American Stroke Association Stroke Council, Clinical Cardiology Council, Cardiovascular Radiology and Intervention Council, and the Athero-
19.
20.
21.
22.
553 sclerotic Peripheral Vascular Disease and Quality of Care Outcomes in Research Interdisciplinary Working Groups: The American Academy of Neurology affirms the value of this guideline as an educational tool for neurologists. Circulation 2007;115:e478-e534. Sung SF, Chen SC, Lin HJ, et al. Comparison of riskscoring systems in predicting symptomatic intracerebral hemorrhage after intravenous thrombolysis. Stroke 2013;44:1561-1566. Cohen J. Weighted kappa: nominal scale agreement with provision for scaled disagreement or partial credit. Psychol Bull 1968;70:213-220. DeLong ER, DeLong DM, Clarke-Pearson DL. Comparing the areas under two or more correlated receiver operating characteristic curves: a nonparametric approach. Biometrics 1988;44:837-845. Qureshi AI, Chaudhry SA, Sapkota BL, et al. Discharge destination as a surrogate for modified Rankin Scale defined outcomes at 3- and 12-months poststroke among stroke survivors. Arch Phys Med Rehabil 2012;93: 1408-1413.e1.
Background: Thirty- and 90-day modified Rankin Scale (mRS) scores are used to monitor adverse outcome or symptomatic intracerebral hemorrhage (sICH) in ischemic stroke patients after intravenous (IV) thrombolytic therapy. Discharge mRS scores are more readily available and could serve as a proxy for 30- or 90-day mRS data. Our goal was to evaluate agreement between the discharge mRS score and sICH. Additionally, we tested for correlations between the discharge mRS score and 8 clinical scores developed to predict sICH or adverse outcomes based on 90-day mRS data. Methods: Clinical data were analyzed from 210 patients receiving IV thrombolysis from January 2009 till December 2013 at the Yale New Haven Hospital. Agreement between sICH and the discharge mRS score was assessed using linear kappa. Eight clinical scores were calculated for each patient and compared with the discharge mRS score by univariate logistic regression. Goodness of fit was tested by receiver operating characteristic (ROC) analysis and by Hosmer–Lemeshow statistics. Results: We found only modest agreement between sICH and unfavorable discharge mRS scores (mRS $5), with kappa .22, P 5 .0001. All 8 clinical scores tested showed good agreement with discharge mRS score of 5 or more (ROC area ..7). Conclusions: The discharge mRS score shows only modest agreement with sICH and therefore cannot be recommended as a proxy for 30- or 90-day mRS data. However, the discharge mRS score correlates strongly with clinical scores predicting long-term adverse outcome; therefore, assessment of discharge mRS scores may be of some clinical benefit. Key Words: Ischemic stroke—hemorrhagic transformation—IV thrombolysis—alteplase—modified Rankin scale—symptomatic intracerebral hemorrhage. Ó 2015 by National Stroke Association
From the *Yale School of Medicine, New Haven, Connecticut; †Yale Emergency Medicine, Division of Vascular Neurology, Division of Neurocritical Care and Emergency Neurology, Yale-New Haven Hospital, New Haven, Connecticut; and ‡Department of Neurology, Taipei Medical University Hospital and Shuang Ho Hospital, Taipei, Taiwan. Received August 4, 2014; accepted September 29, 2014. Address correspondence to Kevin N. Sheth, MD, FAHA, FCCM, FNCS, 20 York Street, New Haven, CT 06510. E-mail: kevin.sheth@ yale.edu. 1052-3057/$ - see front matter Ó 2015 by National Stroke Association http://dx.doi.org/10.1016/j.jstrokecerebrovasdis.2014.09.034
548
The Rankin Scale was modified by the United Kingdom Transient Ischaemic Attack study group to measure overall independence after a stroke.1 This modified Rankin Scale (mRS) gives a better indication of the patient’s ability to look after themselves in daily life and shows moderate-to-good interobserver agreement.2 Thirty- or 90-day mRS scores are routinely used to measure longterm functional deficits after ischemic stroke. They are particularly useful for monitoring adverse outcomes associated with symptomatic intracerebral hemorrhage (sICH) in ischemic stroke patients treated with intravenous (IV) thrombolysis (rt-PA), because standard dose
Journal of Stroke and Cerebrovascular Diseases, Vol. 24, No. 3 (March), 2015: pp 548-553
MODEST PREDICTIVE ABILITY OF mRS SCORE AT DISCHARGE
rt-PA administration carries a significant risk of subsequent sICH—up to 6%-30% depending on the sICH definition in use.3-5 Long-term mRS scores are not readily available at many stroke centers because they require structured interviews performed by trained and certified raters.6 Discharge mRS scores on the other hand are routinely assessed at most stroke centers and are strong independent predictors of disability at 90 days, although they showed only modest agreement with 90-day mRS scores in 1 study.7 Discharge mRS scores are an attractive alternative to long-term mRS scores for assessing functional outcome in ischemic stroke patients after rt-PA therapy. A number of other clinical scores have been developed to predict sICH or adverse outcome in patients undergoing IV thrombolytic therapy, including the StrokeThrombolytic Predictive Instrument (Stroke-TPI),8 iSCORE,9 DRAGON ([hyper]Dense cerebral artery sign/ early infarct signs on admission CT scan, prestroke modified Rankin Scale (mRS) score, Age, Glucose level at baseline, Onset-to-treatment time, and baseline National Institutes of Health Stroke Scale),10 Stroke Prognostication using Age and NIH Stroke Scale-100 (SPAN-100),11 Acute Stroke Registry and Analysis of Lausanne (ASTRAL),12 Post-thrombolysis Risk Score (PRS),13 Hemorrhage After Thrombolysis,14 SEDAN (Sugar, Early infarct signs and [hyper]Dense cerebral artery sign on admission computed tomography scan, Age, and NIH Stroke Scale on admission),15 and Safe Implementation of Treatments in Stroke Symptomatic Intracerebral Hemorrhage (SITSICH) scores.16 Several of these scores were developed to predict patient outcomes based on 90-day mRS scores; however, their correlation with discharge mRS scores have not been previously tested. In this study, we investigated agreement between discharge mRS scores and the presence of sICH. Additionally, we tested for correlations between discharge mRS scores and 8 clinical scores for predicting sICH or long-term adverse outcome after rtPA administration.
Methods Patient Data Demographic and clinical data were retrospectively analyzed from all consecutive patients (n 5 210) who had a principal or secondary discharge diagnosis of ischemic stroke and received IV rt-PA therapy from January 2009 until July 2013 at Yale New Haven Hospital.17 Patients transferred from an outside institution after IV rt-PA administration were excluded, as were patients who subsequently underwent intra-arterial rt-PA or neurointerventional procedures with mechanical clot retrieval devices. One patient was excluded because of incomplete data. Eligibility criteria for IV rt-PA treatment were applied following the American Heart Association guidelines.18 This study was approved by the Yale
549
Human Investigation Committee and the Yale Human Research Protection Program.
Imaging Data Computed tomography (CT) or magnetic resonance imaging scans were performed on each patient before IV rt-PA treatment, at 24 hours after treatment, and subsequent to any observed clinical deterioration. Neuroradiologic assessment of hypodense CT lesions, hyperdense vessels, and intracerebral hemorrhage was performed on each patient by a trained neurologist (H.A.). Stroke severity was assessed by National Institute of Health Stroke Scale (NIHSS) score at baseline.
Clinical Scores and Outcome Data The discharge mRS score was calculated for each patient as a continuous variable and discharge mRS scores of 5 or more and 4 or more as binary variables. Eight clinical scores were also calculated for each patient: Stroke-TPI, DRAGON, SPAN-100, ASTRAL, PRS, Hemorrhage After Thrombolysis, SEDAN, and SITS-ICH. Detailed derivations of each score have been published elsewhere8,9,11-15 and are summarized in Table 1. The iSCORE was excluded because, similar to other authors’ concerns,19 it required detailed patient records including history of renal dialysis and congestive heart failure, which are not readily available in the hyperacute stroke setting. For the Stroke-TPI score, parameter estimates for predicting mRS scores of 5 or more without ASPECTS scores were used.8 Rather than deriving probabilities from the inverse logit function, parameter estimates were calculated directly for each patient and summed to generate a raw score, which allowed more meaningful comparison with the other scores. All other clinical scores were calculated as previously published.
Statistical Analysis Agreement between sICH and discharge mRS score of 5 or more or mRS scores of 4 or more was assessed using linear weighted kappa, which measures agreement between 2 raters or metrics after excluding the effect of chance.20 Univariate logistic regression was performed using each clinical score as an independent variable and discharge mRS, discharge mRS scores of 5 or more, or discharge mRS scores of 4 or more as the dependent variable. Goodness of fit was tested using Hosmer– Lemeshow statistics and areas under the receiver operating characteristic (ROC) curve using the DeLong method.21 P values less than .05 (2-tailed) were considered statistically significant. All analyses were performed using STATA 13 software package (StataCorp LP, College Station, TX).
D. ASUZU ET AL.
550
Table 1. Patient characteristics used to derive clinical scores Score
End points
Components
Stroke-TPI DRAGON
90-day mRS scores 90-day mRS scores
SPAN-100
ICH and 90-day composite of mRS scores, NIHSS, Barthel index and Glasgow Outcome Scale scores 90-day mRS scores
ASTRAL PRS HAT SEDAN SITS-ICH
ICH, sICH, and PH ICH, sICH, and hemorrhages with final fatal outcome sICH sICH
Age, NIHSS, glucose Age, prestroke mRS, HDMCA or early CT infarct, glucose, OTT, admission NIHSS Age, admission NIHSS
Age, admission NIHSS, OTT, decreased level of consciousness, visual field defects, glucose Age, admission NIHSS, glucose, platelets DM or glucose, admission NIHSS, early CT hypodensity Age, NIHSS, glucose, HDMCA sign, early CT infarct Age, weight, hypertension, aspirin/clopidogrel, admission NIHSS, systolic BP, glucose, OTT
Abbreviations: ASTRAL, Acute Stroke Registry and Analysis of Lausanne; BP, blood pressure; CT, computed tomography; DM, diabetes mellitus; HAT, Hemorrhage After Thrombolysis; HDMCA, hyperdense middle cerebral artery sign; ICH, intracerebral hemorrhage; mRS, modified Rankin Scale; NIHSS, National Institute of Health Stroke Scale; OTT, onset-to-treatment; PH, parenchymal hemorrhage; PRS, post-thrombolysis risk score; sICH, symptomatic intracerebral hemorrhage; SITS-ICH, Safe Implementation of Treatments in Stroke Symptomatic Intracerebral Hemorrhage; SPAN, Stroke Prognostication using Age and NIH Stroke Scale; TPI, Thrombolytic Predictive Instrument. References.8,10-16
Results A total of 210 patients received rt-PA therapy at Yale New Haven Hospital between January 2009 and December 2013. A total of 48.7% were male. Forty-nine patients (23.3%) had a history of diabetes, 155 (73.8%) had hypertension, 46 (21.9%) had a prior TIA or stroke, and 97 (46.2%) were on aspirin or clopidogrel. Median baseline NIHSS score was 10, and mean onset-totreatment (OTT) time was 148 minutes (Table S1 in Appendix).17 Symptomatic ICH occurred in 12 patients (5.71%). Thirty-seven patients (17.6%) had discharge mRS scores of 5 or more, and 151 patients (71.9%) had discharge mRS scores of 4 or more. First, we assessed the correlation between discharge mRS scores and sICH. There was 83.4% agreement be-
tween patients with sICH and discharge mRS scores of 5 or more (kappa .22, P , .001). When discharge mRS was relaxed to include scores of 4 (discharge mRS $4), agreement decreased to 34.1% (kappa .05, P 5 .01). Next, we performed logistic regression and ROC curve analysis using discharge mRS scores of 5 or more or mRS scores of 4 or more as the dependent variable and each of the 8 clinical scores as independent variables. All clinical scores showed good agreement with discharge mRS scores of 5 or more (ROC area ..7, Table 2). The 2 clinical scores showing the best agreement with discharge mRS scores of 5 or more were Stroke-TPI (ROC, .86 6 .07) and ASTRAL (ROC, .85 6 .07), with odds ratios of 1.3 6 .44 and .17 6 .06, respectively. SPAN-100 showed the least agreement with discharge mRS scores of 5 or more (ROC, .71 6 .09) with odds ratio 2.09 6 .79.
Table 2. Comparison of clinical scores for correlation with discharge mRS scores using mRS $5 Clinical score
Odds ratio
Z statistic
P . jzj
Hosmer–Lemeshow, c2
Groups
P . c2
ROC area
Stroke-TPI ASTRAL DRAGON SEDAN PRS HAT SITS-ICH SPAN-100
1.30 (.86-1.73) .17 (.12-.23) 1.66 (1.35-2.05) .83 (.47-1.21) 1.45 (.86-2.04) .94 (.59-1.29) .46 (.24-.67) 2.09 (1.30-2.87)
5.84 5.91 4.76 4.42 4.84 5.28 4.21 5.21
,.001 ,.001 ,.001 ,.001 ,.001 ,.001 ,.001 ,.001
9.62 5.96 6.05 4.41 .84 .34 10.44 0
10 10 7 5 4 5 7 2
.29 .65 .30 .22 .66 .95 .06 1
.86 (.80-.94) .85 (.79-.93) .76 (.67-.85) .75 (.67-.82) .76 (.69-.83) .76 (.67-.85) .73 (.65-.81) .71 (.62-.79)
Abbreviations: ASTRAL, Acute Stroke Registry and Analysis of Lausanne; HAT, Hemorrhage After Thrombolysis; mRS, modified Rankin Scale; PRS, post-thrombolysis risk score; SITS-ICH, Safe Implementation of Treatments in Stroke Symptomatic Intracerebral Hemorrhage; SPAN, Stroke Prognostication using Age and NIH Stroke Scale; TPI, Thrombolytic Predictive Instrument. Values presented as mean (95% confidence interval).
MODEST PREDICTIVE ABILITY OF mRS SCORE AT DISCHARGE
551
Table 3. Comparison of clinical scores for correlation with discharge mRS scores using mRS $4 Clinical score
Odds ratio
Z statistic
P . jzj
Hosmer–Lemeshow, c2
Groups
P . c2
ROC area
ASTRAL Stroke-TPI DRAGON SEDAN HAT PRS SITS-ICH SPAN-100
.12 (.08-.17) .81 (.53-1.09) 1.41 (1.19-1.68) .88 (.53-1.23) .58 (.22-.94) 1.06 (.65-1.47) .36 (.18-.54) 2.33 (.87-3.78)
5.38 5.72 3.89 4.97 3.16 5.06 3.87 3.14
,.001 ,.001 ,.001 ,.001 .002 ,.001 ,.001 .002
4.96 3.56 2.66 5.51 3.62 1.6 5.62 0
10 10 7 5 5 4 7 2
.76 .89 .75 .14 .31 .45 .35 1
.77 (.70-.84) .78 (.71-.84) .68 (.60-.76) .73 (.66-.81) .62 (.55-.70) .72 (.64-.79) .67 (.59-.76) .62 (.57-.66)
Abbreviations: ASTRAL, Acute Stroke Registry and Analysis of Lausanne; HAT, Hemorrhage After Thrombolysis; mRS, modified Rankin Scale; PRS, post-thrombolysis risk score; SITS-ICH, Safe Implementation of Treatments in Stroke Symptomatic Intracerebral Hemorrhage; SPAN, Stroke Prognostication using Age and NIH Stroke Scale; TPI, Thrombolytic Predictive Instrument. Values presented as mean (95% confidence interval).
When the discharge mRS cutoff was relaxed to mRS scores of 4 or more, only 4 showed good agreement with mRS scores of 4 or more (ASTRAL, Stroke-TPI, SEDAN, and PRS; Table 3). Stroke-TPI and ASTRAL still showed the best agreement (ROC, .78 6 .07; odds ratio, .81 6 .28 and .77 6 .07; odds ratio, .12 6 .05, respectively), and SPAN-100 showed the least agreement with discharge mRS scores of 4 or more (ROC, .62 6 .05) with odds ratio 2.33 6 1.46. When discharge mRS was tested as a continuous variable, none of the clinical scores showed good agreement with mRS (R2 , .3 for each, Table 4).
Discussion In this study, we found only modest agreement between discharge mRS scores and sICH in ischemic stroke patients undergoing rt-PA therapy. However, we found strong correlations between discharge mRS scores and 8 clinical scores developed to predict long-term adverse outcomes after rt-PA administration. rt-PA remains the only Food and Drug Administrationapproved medical therapy for ischemic stroke patients; however, patients receiving IV rt-PA are at increased
risk for subsequent sICH and adverse clinical outcomes. Clinical scores to assess and predict sICH are critical for the optimal management of ischemic stroke patients receiving rt-PA therapy. The discharge mRS score is more readily available at clinical centers than 30- and 90-day mRS scores and is less susceptible to data collection errors, making it attractive as a proxy for 30- or 90-day mRS data. A recent study compared discharge mRS scores with 30- or 90-day mRS data but found only fair agreement with 90-day mRS, although patients with higher discharge mRS scores were more than twice as likely to suffer disability at 90 days.7 Our results similarly indicate only modest agreement between discharge mRS scores and sICH, yet we found strong correlations between the discharge mRS score and clinical scores predicting adverse longterm outcomes. We also found poor agreement between sICH and delta mRS, defined as the difference between admission mRS scores and discharge mRS scores (results not shown). The discharge mRS score therefore remains a useful tool for monitoring functional deficits at discharge but cannot be recommended as a proxy for long-term adverse outcome.
Table 4. Comparison of clinical scores for correlation with discharge mRS scores using mRS as a continuous variable Clinical score
Odds ratio
t statistic
P . jtj
R square
F statistic
P . jFj
Stroke-TPI ASTRAL DRAGON SEDAN PRS HAT SITS-ICH SPAN-100
.65 (.51-.78) .10 (.07-.12) .35 (.24-.47) .70 (.46-.87) .94 (.68-1.20) .52 (.30-.74) .30 (.18-.42) 1.5 (.93-2.10)
9.22 8.62 6.1 6.45 7.17 4.7 4.83 5.09
,.001 ,.001 ,.001 ,.001 ,.001 ,.001 ,.001 ,.001
.29 .26 .15 .17 .2 .1 .1 .11
85.1 74.3 37.19 41.55 51.4 22.1 23.34 25.89
,.001 ,.001 ,.001 ,.001 ,.001 ,.001 ,.001 ,.001
Abbreviations: ASTRAL, Acute Stroke Registry and Analysis of Lausanne; HAT, Hemorrhage After Thrombolysis; mRS, modified Rankin Scale; PRS, post-thrombolysis risk score; SITS-ICH, Safe Implementation of Treatments in Stroke Symptomatic Intracerebral Hemorrhage; SPAN, Stroke Prognostication using Age and NIH Stroke Scale; TPI, Thrombolytic Predictive Instrument. Values presented as mean (95% confidence interval).
D. ASUZU ET AL.
552
Discharge status may still carry some predictive information on patients’ long-term clinical outcome. For instance, the discharge mRS score partly determines a patient’s destination after discharge, and discharge destination is an independent predictor of 30- and 90-day outcomes.22 Furthermore, the discharge mRS score independently predicts 90-day disability7 and correlates strongly with scores predicting long-term adverse outcome (this study). It therefore appears that certain aspects of patients’ disposition at discharge could determine their long-term clinical status. Further studies are needed to identify discharge criteria, which correlate independently with long-term clinical outcomes and can thus serve as proxies for 30- and 90-day mRS scores. In this study, 2 clinical scores, Stroke-TPI and ASTRAL, showed the best agreement with discharge mRS by ROC analysis at cutoffs of mRS scores of 5 or more and mRS scores of 4 or more. These 2 clinical scores are derived using similar clinical parameters. The Stroke-TPI score is derived using age, NIHSS score, and serum glucose (Table 1). ASTRAL includes these 3 parameters in addition to symptom OTT duration, decreased level of consciousness, and visual field defects.8,12 The similarity in predictive ability between these 2 scores can therefore be attributed to the clinical parameters they share. However, the Stroke-TPI score correlates positively with discharge mRS scores whereas ASTRAL correlates negatively, suggesting that either symptom OTT duration, decreased level of consciousness, or visual field defects may be protective for mRS status at discharge. This observation warrants further clarification in future studies. Our study has a number of limitations. We were unable to directly compare discharge mRS scores with 30- and 90-day mRS data because of unavailability of follow-up mRS data at our center over the duration of this study. This reflects current limitations in availability of reliable follow-up mRS data at many centers and underscores the need for readily available discharge criteria, which can predict long-term clinical outcome. We also defined sICH in this study according to the National Institute of Neurological Diseases and Stroke trial definition. The National Institute of Neurological Diseases and Stroke trial defined a hemorrhage as symptomatic if blood was not seen on a CT scan before thrombolytic therapy and there had subsequently been either a suspicion of hemorrhage or any decline in neurologic status.3 More narrow sICH definitions such as the SITS-MOST and European Cooperative Acute Stroke Study II definitions may have shown different levels of agreement with discharge mRS scores but were not readily estimable from our data set.
Conclusions In this study we found only modest agreement between discharge mRS scores and sICH. However, we found
strong correlations between discharge mRS scores and 8 clinical scores for predicting sICH or long-term adverse outcome. Discharge mRS scores therefore contain useful information about functional status at discharge, but cannot be recommended as a proxy for 30- or 90-day mRS scores.
Supplementary Data Supplementary data related to this article can be found at http://dx.doi.org/10.1016/j.jstrokecerebrovas dis.2014.09.034.
References 1. United Kingdom transient ischaemic attack (UK-TIA) aspirin trial: interim results. UK-TIA Study Group. Br Med J (Clin Res Ed) 1988;296:316-320. 2. van Swieten JC, Koudstaal PJ, Visser MC, et al. Interobserver agreement for the assessment of handicap in stroke patients. Stroke 1988;19:604-607. 3. Tissue plasminogen activator for acute ischemic stroke. The National Institute of Neurological Disorders and Stroke rt-PA Stroke Study Group. N Engl J Med 1995; 333:1581-1587. 4. Hacke W, Kaste M, Fieschi C, et al. Intravenous thrombolysis with recombinant tissue plasminogen activator for acute hemispheric stroke. The European Cooperative Acute Stroke Study (ECASS). JAMA 1995;274:1017-1025. 5. Hacke W, Kaste M, Fieschi C, et al. Randomised doubleblind placebo-controlled trial of thrombolytic therapy with intravenous alteplase in acute ischaemic stroke (ECASS II). Second European-Australasian Acute Stroke Study Investigators. Lancet 1998;352:1245-1251. 6. Banks JL, Marotta CA. Outcomes validity and reliability of the modified Rankin scale: implications for stroke clinical trials: a literature review and synthesis. Stroke 2007; 38:1091-1096. 7. Thompson MP, Reeves M. Abstract 168: assessing the utility of the modified Rankin Scale (mRS) at discharge to predict day 90 outcomes in acute stroke registries. Circulation 2012;5(3 Supplem):A168. 8. Kent DM, Selker HP, Ruthazer R, et al. The strokethrombolytic predictive instrument: a predictive instrument for intravenous thrombolysis in acute ischemic stroke. Stroke 2006;37:2957-2962. 9. Saposnik G, Raptis S, Kapral MK, et al. The iScore predicts poor functional outcomes early after hospitalization for an acute ischemic stroke. Stroke 2011;42:3421-3428. 10. Strbian D, Meretoja A, Ahlhelm FJ, et al. Predicting outcome of IV thrombolysis-treated ischemic stroke patients: the DRAGON score. Neurology 2012;78:427-432. 11. Saposnik G, Guzik AK, Reeves M, et al. Stroke Prognostication using Age and NIH Stroke Scale: SPAN-100. Neurology 2013;80:21-28. 12. Ntaios G, Faouzi M, Ferrari J, et al. An integer-based score to predict functional outcome in acute ischemic stroke: the ASTRAL score. Neurology 2012;78: 1916-1922. 13. Cucchiara B, Tanne D, Levine SR, et al. A risk score to predict intracranial hemorrhage after recombinant tissue plasminogen activator for acute ischemic stroke. J Stroke Cerebrovasc Dis 2008;17:331-333.
MODEST PREDICTIVE ABILITY OF mRS SCORE AT DISCHARGE 14. Lou M, Safdar A, Mehdiratta M, et al. The HAT Score: a simple grading scale for predicting hemorrhage after thrombolysis. Neurology 2008;71:1417-1423. 15. Strbian D, Engelter S, Michel P, et al. Symptomatic intracranial hemorrhage after stroke thrombolysis: the SEDAN score. Ann Neurol 2012;71:634-641. 16. Mazya M, Egido JA, Ford GA, et al. Predicting the risk of symptomatic intracerebral hemorrhage in ischemic stroke treated with intravenous alteplase: Safe Implementation of Treatments in Stroke (SITS) symptomatic intracerebral hemorrhage risk score. Stroke 2012;43:1524-1531. 17. Asuzu D, Nystrom K, Amin H, et al. Comparison of 8 scores for predicting symptomatic intracerebral hemorrhage after IV thrombolysis. Neurocritical Care 2014 [Epub ahead of print]. 18. Adams HP Jr, del Zoppo G, Alberts MJ, et al. Guidelines for the early management of adults with ischemic stroke: a guideline from the American Heart Association/American Stroke Association Stroke Council, Clinical Cardiology Council, Cardiovascular Radiology and Intervention Council, and the Athero-
19.
20.
21.
22.
553 sclerotic Peripheral Vascular Disease and Quality of Care Outcomes in Research Interdisciplinary Working Groups: The American Academy of Neurology affirms the value of this guideline as an educational tool for neurologists. Circulation 2007;115:e478-e534. Sung SF, Chen SC, Lin HJ, et al. Comparison of riskscoring systems in predicting symptomatic intracerebral hemorrhage after intravenous thrombolysis. Stroke 2013;44:1561-1566. Cohen J. Weighted kappa: nominal scale agreement with provision for scaled disagreement or partial credit. Psychol Bull 1968;70:213-220. DeLong ER, DeLong DM, Clarke-Pearson DL. Comparing the areas under two or more correlated receiver operating characteristic curves: a nonparametric approach. Biometrics 1988;44:837-845. Qureshi AI, Chaudhry SA, Sapkota BL, et al. Discharge destination as a surrogate for modified Rankin Scale defined outcomes at 3- and 12-months poststroke among stroke survivors. Arch Phys Med Rehabil 2012;93: 1408-1413.e1.
