Assessing the Performance of the Framingham Stroke Risk Score in the Reasons for Geographic and Racial Differences in Stroke Cohort Leslie A. McClure, PhD; Dawn O. Kleindorfer, MD; Brett M. Kissela, MD; Mary Cushman, MD; Elsayed Z. Soliman, MD; George Howard, DrPH Background and Purpose—The most well-known stroke risk score is the Framingham Stroke Risk Score (FSRS), which was developed during the higher stroke risk period of the 1990s and has not been validated for blacks. We assessed the performance of the FSRS among participants in the Reasons for Geographic and Racial Differences in Stroke (REGARDS) study to determine whether it is useful in both blacks and whites. Methods—Expected annualized stroke rates from the FSRS were compared with observed stroke rates overall and within strata defined by FSRS risk factors (age, sex, systolic blood pressure, use of antihypertensive medications, diabetes mellitus, smoking, atrial fibrillation, left ventricular hypertrophy, and prevalent coronary heart disease). Results—Among 27 748 participants stroke-free at baseline, 715 stroke events occurred over 5.6 years of follow-up. F SRSestimated incidence rates of stroke were 1.6× higher than observed for black men, 1.9× higher for white men, 1.7× higher for black women, and 1.7× higher for white women. This overestimation was consistent among most subgroups of FSRS factors, although the magnitude of overestimation varied by the risk factor assessed. Conclusions—Although higher FSRS was associated with higher stroke risk, the FSRS overestimated the observed stroke rates in this study, particularly in certain subgroups. This may be because of temporal declines in stroke rates, secular trends in prevention treatments, or differences in populations studied. More accurate estimates of event rates are critical for planning research, including clinical trials, and targeting health-care efforts. (Stroke. 2014;45:1716-1720.) Key Words: risk assessment ◼ risk factors ◼ stroke
C
linical tools to predict disease are important to direct treatments and to counsel individuals on behavioral changes that might modify risks. Recent reports have offered new tools for the prediction of a composite outcome of hard atherosclerotic cardiovascular disease end point, including both coronary and stroke events1; however, these new tools do not directly address risk from the individual components of the composite outcome, and acceptance of these risk functions has been controversial.2 Several risk scores are available to predict stroke,3–8 the most well known and well accepted being the Framingham Stroke Risk Score (FSRS), which was developed during the high stroke risk period of the 1990s. Factors included in the FSRS are age, sex, systolic blood pressure, use of antihypertensive medications, diabetes mellitus, smoking, atrial fibrillation, left ventricular hypertrophy, and prevalent coronary heart disease.4 The Cardiovascular Health Study (CHS) risk score includes the same risk factors and a measure of frailty.5 In the development of a risk score from the
Atherosclerosis in Communities (ARIC) study, the goal was to determine nontraditional risk factors that may improve predictability of the model; however, their analyses determined that the traditional risk factors performed reasonably well and that the addition of nontraditional risk factors did not substantially improve prediction.5 As such, the FSRS remains the standard for predicting stroke risk in the general population. Notably absent from the FSRS, the CHS risk score, and the new American College of Cardiology/American Heart Association (ACC/AHA) Pooled Cohort Risk Equation is the effect of race on stroke risk prediction. The FSRS does not include a term for race or ethnicity because there were few blacks in the study. Similarly, in CHS, at the time the risk score was developed, there was insufficient follow-up among black participants (≈15% of the cohort). Framingham, CHS, and other studies included in the new ACC/AHA instrument included relatively few blacks, and racial differences were not the focus of these studies. In ARIC, race was considered, but
Received January 22, 2014; final revision received February 24, 2014; accepted March 13, 2014. From the Department of Biostatistics, University of Alabama at Birmingham (L.A.M., G.H.); Department of Neurology, University of Cincinnati, OH (D.O.K., B.M.K.); Department of Medicine, University of Vermont, Burlington (M.C.); and Department of Epidemiology and Prevention, Wake Forest University School of Medicine, Winston-Salem, NC (E.Z.S.). Presented in part at the International Stroke Conference of the American Heart Association, Honolulu, HI, February 6-8, 2013. The online-only Data Supplement is available with this article at http://stroke.ahajournals.org/lookup/suppl/doi:10.1161/STROKEAHA. 114.004915/-/DC1. Correspondence to Leslie A. McClure, PhD, Department of Biostatistics, University of Alabama at Birmingham, 1720 Second Ave S, Birmingham, AL 35294-0022. E-mail [email protected] © 2014 American Heart Association, Inc. Stroke is available at http://stroke.ahajournals.org
DOI: 10.1161/STROKEAHA.114.004915
Downloaded from http://stroke.ahajournals.org/ at CONS CALIFORNIA DIG LIB on January 6, 2015 1716
McClure et al The Framingham Stroke Risk Score in REGARDS 1717 inclusion in modeling did not increase predictability beyond the traditional risk factors, and ARIC confounded race and geography with most of the black study participants from the single Jackson (Mississippi) study site.1 Currently available risk scores assume that the effect of risk factors is similar for blacks and whites. The Framingham Coronary Risk Score was validated in several racial/ethnic groups9; however, this work did not assess the potential that risk factors could have a differential effect on whites and blacks. To our knowledge, the performance of the FSRS has not been assessed in different racial groups, and the possibility that risk factors could have a different role in blacks and whites has not been evaluated. We assessed the performance of the FSRS among black and white participants in the Reasons for Geographic and Racial Differences in Stroke (REGARDS) study to determine whether this tool is useful in both blacks and whites.
Methods The REGARDS study is a national population-based cohort study that recruited 30 239 participants ≥45 years of age, with 45% men and 55% women, 42% black and 58% white. Recruitment began in January 2003 and was completed in October 2007. Twenty-one percent of the cohort was recruited from the buckle of the stroke belt (coastal plain region of North Carolina, South Carolina, and Georgia), 35% from the stroke belt states (remainder of North Carolina, South Carolina, and Georgia, plus Alabama, Mississippi, Tennessee, Arkansas, and Louisiana), and the remaining 44% from the other 42 contiguous states (goal was 30% belt, 20% buckle, 50% remainder of nation). Detailed study methods are published elsewhere.10 In brief, participants were selected from commercially available lists of residents and were recruited using a combination of mail notification and telephone contact. Verbal consent and baseline data, including demographics, smoking history, and cardiovascular risk factor history, were collected via computer-assisted telephone interview. After completion of the baseline interview, an in-home visit was performed to collect physical measurements, including blood pressure, blood and urine samples, ECG, and a signed informed consent. Blood was processed centrally at the University of Vermont, and ECGs were read centrally at Wake Forest University School of Medicine. Follow-up phone contact is made at 6-month intervals for surveillance of suspected stroke events, with medical records for suspected events retrieved and adjudicated by a panel of physicians. The study was approved and monitored by the institutional review boards at all participating institutions.
Framingham Stroke Risk Factors Age, sex, use of antihypertensive medications, and smoking were all based on self-report during the initial phone interview. Age was categorized as young (45–64 years), middle (65–74 years), or older (≥75 years). Heart disease was defined as self-report of myocardial infarction, bypass, angioplasty or stenting, or EGC evidence of myocardial infarction. Participants were considered diabetic if their fasting glucose was ≥126 (or ≥200 if they were not fasting) or if they self-reported medication use for diabetes mellitus. Systolic blood pressure was the average of 2 seated measurements. Determination of atrial fibrillation was by self-report of a physician diagnosis or ECG evidence, and left ventricular hypertrophy was based on ECG evidence.
were centrally adjudicated by physicians. For deaths without medical records, death certificates were examined and adjudicated, and proxy interviews undertaken. Stroke events were defined following the World Health Organization definition.11 Those events not meeting the WHO definition but with symptoms lasting >24 hours and neuroimaging consistent with acute ischemia or hemorrhage were classified as clinical strokes and were included in analyses.
Statistical Analysis We assessed the prevalence of the FSRS factors among REGARDS participants at baseline by race–sex strata, computed the FSRS, and compared the FSRS factors across race–sex strata using ANOVA or χ2 tests of association, as appropriate. We then fit Poisson regression models to estimate the observed stroke incidence rates. We compared the observed stroke rates with those predicted by the FSRS by computing the ratio of the predicted to observed risk and the 95% confidence intervals around the risk ratios. We compared the observed and predicted stroke rates overall, by levels of predicted FSRS levels and by levels of FSRS risk factors for each race–sex stratum. Because of missing stroke data due to ongoing adjudication of stroke events, as well as differential retrieval of medical records, we used multiple imputation techniques to classify in-process stroke events using a logistic function predicting the likelihood that an attempted record retrieval would result in an adjudicated stroke.12 Ten imputed data sets were used, and 22% of reported events were eligible to be imputed.
Results Among 27 748 stroke-free participants at baseline followed for an average of 5.6 years, 30% were white women, 30% were white men, 25% were black women, and 15% were black men. Table 1 presents the average 10-year FSRS and the prevalence of FSRS risk factors by race–sex strata. All factors differed significantly across race–sex strata. Predicted stroke risk was highest in black men (11.9% 10-year risk; SD, 10.0), followed by white men (11.0% 10-year risk; SD, 9.6), black women (10.2% 10-year risk; SD, 12.1), and white women (7.8% 10-year risk; SD, 10.2). A similar pattern in the association between race–sex stratum and systolic blood pressure was observed. Table 2 provides the ratio of the expected-to-observed annualized stroke risk by race–sex strata, overall (Figure 1), by quintiles of the FSRS (Figure 2), and by levels of each of the FSRS risk factors (Figures I and II in the online-only Table 1. Average FSRS and Prevalence of the FSRS Risk Factors by Race–Sex Strata White
Black
Female (n=8342)
Male (n=8198)
Female (n=7037)
Male (n=4171)
FSRS, mean percent 10-y risk (SD)
7.8 (10.2)
11.0 (9.6)
10.2 (12.1)
11.9 (10.0)
SBP, mm Hg, mean (SD)
123 (16)
127 (15)
130 (17)
132 (17)
Current smoker
1096 (13%)
936 (11%)
1080 (15%)
803 (19%)
Age, mean (SD) Heart disease
64.5 (9.6)
65.9 (9.3)
63.4 (9.3)
64.2 (9.2)
972 (12%)
2059 (25%)
828 (12%)
Determination of Stroke Events
LVH
726 (18%)
263 (4%)
82 (2%)
455 (8%)
Events reported during follow-up as possible stoke, transient ischemic attack, death, hospitalization, or emergency department visit for brain aneurysm, brain hemorrhage, stroke symptoms, or unknown reason prompted a request for medical records. Initial review by a stroke nurse excluded obvious nonstrokes; the remaining medical records
109 (4%)
Atrial fibrillation
85 (1%)
211 (3%)
Diabetes mellitus
1021 (13%)
1394 (17%)
21 (0.3%) 1963 (29%)
56 (1%) 1226 (31%)
FSRS indicates Framingham Stroke Risk Score; LVH, left ventricular hypertrophy; and SBP, systolic blood pressure.
Downloaded from http://stroke.ahajournals.org/ at CONS CALIFORNIA DIG LIB on January 6, 2015
1718 Stroke June 2014 Table 2. Ratio of Expected-to-Observed Stroke Risk (95% Confidence Interval) by Race–Sex Strata, Overall and Across Different Levels of FSRS and FSRS Risk Factors White
Black
Female (n=8342)
Male (n=8198)
Female (n=7037)
Male (n=4171)
1.71 (1.49, 1.96)
1.88 (1.66, 2.11)
1.71 (1.49, 1.96)
1.60 (1.35, 1.89)
Q1 (0 for all 4 race–sex strata. The magnitude of overestimation in the top quintile of predicted risk was about the same for white and black men, but was larger for black women than for white women. The ratio of expected-to-observed stroke risk by race–sex strata across the levels of each of the factors contributing to the FSRS indicated that across all risk factors, at all levels, the
FSRS overestimated the stroke risk relative to that observed. The overestimation was particularly large at higher levels of risk factors, such as for those with stage II hypertension and for those with atrial fibrillation. With respect to race and sex, there were no consistent patterns regarding for whom the overestimation was largest. In some cases, the overestimation was larger for blacks than for whites (eg, for women with stage II hypertension, or for female smokers), whereas in other cases, the overestimation was worse for whites than for blacks (eg, male nonsmokers or for men with left ventricular hypertrophy).
Discussion In this large cohort of black and white adults aged ≥45 years, the validity of the FSRS was confirmed by a clearly higher observed number of stroke events among those with higher
Downloaded from http://stroke.ahajournals.org/ at CONS CALIFORNIA DIG LIB on January 6, 2015
McClure et al The Framingham Stroke Risk Score in REGARDS 1719
Figure 1. Ratio of expected-to-observed stroke risk and 95% confidence interval, by race (red indicates white; and blue, black) and sex (F indicates female; and M, male).
FSRS scores; however, the FSRS overestimated the observed stroke rates, particularly in certain subgroups. Generally, the FSRS suggested there should be approximately twice as many strokes occurring than we detected, and with a few exceptions, this was consistent across strata defined by the risk factors. This overestimation of stroke risk is similar to the suggested overestimation of risk that has been central to the controversy surrounding the predicted atherosclerotic cardiovascular disease risk in the ACC/AHA guideline instrument.2 The FSRS is not the only stroke risk score that is available. Several others have been developed, including the CHS risk score,5 the ARIC risk score,6 and others.7,8 Although REGARDS does not have all of the variables contributing to each of these, it is expected that the results would be the similar to those observed with the FSRS, given the similarities among the performance of the scores.6,13 While the magnitude of the expected-to-observed ratio of strokes was reasonably consistent, there were several exceptions. For all 4 race–sex strata, the expected-to-observed ratio was higher for study participants in the highest quintile of the FSRS. This finding supports the notion that preventive treatments reduced the number of events in these individuals. This is particularly the case for white men and black women with stage II hypertension (systolic blood pressure ≥160 mm Hg). The expected-to-observed ratio was also quite
Figure 2. Ratio of expected-to-observed stroke risk and 95% confidence interval across the quintiles of the Framingham Stroke Risk Score by race (red indicates white; and blue, black) and sex (F indicates female; and M, male).
high for both black men and women with atrial fibrillation, potentially suggesting that treatment of this condition has improved with time. With these exceptions, the magnitude of the expected-to-observed ratio of events was reasonably consistent, suggesting more of a general change in risk of stroke. With REGARDS being one of the validation studies for the new ACC/AHA guideline models that showed lower observed than predicted risk, we have suggested 4 potential contributors to the overestimation of risk.14 First, several of the studies used to develop the risk equation included both self-reported events with subsequent medical record adjudication and active surveillance for events in community hospitals, whereas in REGARDS, we used only self-reported events to retrieve records for subsequent adjudication. As such, we may be systemically missing some stroke events. Second, dynamic temporal changes in the prevalence of statin use, which was not considered in the validation cohorts including REGARDS, could contribute to fewer events than expected. Third, increased use of revascularization could contribute to fewer events than expected.15 Finally, the duration of follow-up was relatively shorter in REGARDS compared with the studies used to develop the risk functions. All of these factors could potentially be playing a role in the overestimation of stroke events in this report. However, our follow-up contacts were quite good (76% of participants completed >75% of anticipated follow-up contacts), and we cast a very broad net for suspected events (>4 suspected events were reviewed for each event adjudicated to be a stroke). In addition, incident statins use have at most a 30% effect on stroke risk16 only in the subset of the population beginning treatment; carotid revascularization would have an effect a relatively small portion of the population; and our estimates are stable with >800 stroke events. Alternatively, with the Framingham study having a cohort assessed before 2000 and the REGARDS cohort assessed after 2000, the overestimation could be a product of a dramatic temporal decline in incident stroke. Between 1999 and 2010, data from the Center for Disease Control and Prevention's Wide-Ranging Online Data for Epidemiologic Research (CDC WONDER) show that there has been a decrease in age-adjusted death from stroke (International Classification of Diseases, Tenth Revision: I60–I69) from 61.6/100 000 to 39.1/100 000 (a 37% decrease),17 with most thinking that the decline in mortality is primarily attributable to a decline in incidence. We think that a substantial portion of the overestimation of stroke risk is associated with these dramatic declines in stroke risk, suggesting the need for a recalibration of the risk formulas for incident stroke and highlighting the need for ongoing epidemiological studies that can address issues such as this over time. As with all research, this study has limitations. Although the FSRS is designed to determine 10-year risk of stroke, we only have 5.6 years of follow-up thus far in REGARDS and thus have annualized stroke rates. We have made the assumption that the rates are constant with time, and this may not be true. Furthermore, we rely on participant report of stroke, which may introduce a bias due to underreporting of potential events, inability to obtain medical records for stroke adjudication, or losses to follow-up. Although we have minimized these biases by using external resources such as National Death Index searches and relying on proxy
Downloaded from http://stroke.ahajournals.org/ at CONS CALIFORNIA DIG LIB on January 6, 2015
1720 Stroke June 2014 reports, as well as by imputing events that are still in process, they still may exist. In summary, among black and white participants in the REGARDS cohort, the FSRS overestimated the actual stroke risk, across sex and race, and at all levels of observed risk factors. The ability to estimate stroke risk accurately is important for planning research and for targeting clinical efforts toward stroke prevention. Our results might be explained by secular trends in stroke incidence since the FSRS was developed. Thus, future research should focus on estimating stroke risk more accurately across races in the context of current medical care trends.
Acknowledgments The content is solely the responsibility of the authors and does not necessarily represent the official views of the National Institute of Neurological Disorders and Stroke or the National Institutes of Health. Representatives of the funding agency have been involved in the review of the article but not directly involved in the collection, management, analysis, or interpretation of the data. We thank the other investigators, the staff, and the participants of the REGARDS study for their valuable contributions. A full list of participating investigators and institutions in REGARDS can be found at http:// www.regardsstudy.org.
Sources of Funding This research project is supported by a cooperative agreement U01 NS041588 from the National Institute of Neurological Disorders and Stroke, National Institutes of Health, and Department of Health and Human Service.
Disclosures None.
References 1. Goff DC Jr, Lloyd-Jones DM, Bennett G, O’Donnell CJ, Coady S, Robinson J, et al. ACC/AHA guideline on the assessment of cardiovascular risk: a report of the American College of Cardiology/American Heart Association task force on practice guidelines [published online ahead of print November 12, 2013]. J Am Coll Cardiol. doi:10.1016/j. jacc.2013.11.005. http://www.sciencedirect.com/science/article/pii/ S0735109713060312. Accessed February 24, 2014.
2. Ridker PM, Cook NR. Statins: new American guidelines for prevention of cardiovascular disease. Lancet. 2013;382:1762–1765. 3. Wolf PA, D’Agostino RB, Belanger AJ, Kannel WB. Probability of stroke: a risk profile from the Framingham Study. Stroke. 1991;22:312–318. 4. D’Agostino RB, Wolf PA, Belanger AJ, Kannel WB. Stroke risk profile: adjustment for antihypertensive medication. The Framingham Study. Stroke. 1994;25:40–43. 5. Manolio TA, Kronmal RA, Burke GL, O’Leary DH, Price TR. Short-term predictors of incident stroke in older adults. The Cardiovascular Health Study. Stroke. 1996;27:1479–1486. 6. Chambless LE, Heiss G, Shahar E, Earp MJ, Toole J. Prediction of ischemic stroke risk in the Atherosclerosis Risk in Communities Study. Am J Epidemiol. 2004;160:259–269. 7. Gaziano TA, Young CR, Fitzmaurice G, Atwood S, Gaziano JM. Laboratory-based versus non-laboratory-based method for assessment of cardiovascular disease risk: the NHANES I Follow-up Study cohort. Lancet. 2008;371:923–931. 8. Hippisley-Cox J, Coupland C, Brindle P. Derivation and validation of QStroke score for predicting risk of ischaemic stroke in primary care and comparison with other risk scores: a prospective open cohort study. BMJ. 2013;346:f2573. 9. D’Agostino RB Sr, Grundy S, Sullivan LM, Wilson P; CHD Risk Prediction Group. Validation of the Framingham coronary heart disease prediction scores: results of a multiple ethnic groups investigation. JAMA. 2001;286:180–187. 10. Howard VJ, Cushman M, Pulley L, Gomez CR, Go RC, Prineas RJ, et al. The reasons for geographic and racial differences in stroke study: objectives and design. Neuroepidemiology. 2005;25:135–143. 11. Report of the WHO Task Force on Stroke and Cerebrovascular Diseases. Recommendations on stroke prevention, diagnosis and therapy. Stroke. 1989;20:1407–1431. 12. Howard G, McClure LA, Moy CS, Safford MM, Cushman M, Judd SE, et al. Imputation of incident events in longitudinal cohort studies. Am J Epidemiol. 2011;174:718–726. 13. Lumley T, Kronmal RA, Cushman M, Manolio TA, Goldstein S. A stroke prediction score in the elderly: validation and Web-based application. J Clin Epidemiol. 2002;55:129–136. 14. Muntner P, Safford MM, Cushman M, Howard G. Comment on the reports of over-estimation of ASCVD risk using the 2013 AHA/ACC risk equation. Circulation. 2014;129:266–267. 15. Matsen SL, Chang DC, Perler BA, Roseborough GS, Williams GM. Trends in the in-hospital stroke rate following carotid endarterectomy in California and Maryland. J Vasc Surg. 2006;44:488–495. 16. Blauw GJ, Lagaay AM, Smelt AH, Westendorp RG. Stroke, statins, and cholesterol. A meta-analysis of randomized, placebo-controlled, doubleblind trials with HMG-CoA reductase inhibitors. Stroke. 1997;28:946–950. 17. Sloan MA, Price TR, Foulkes MA, Marler JR, Mohr JP, Hier DB, et al. Circadian rhythmicity of stroke onset. Intracerebral and subarachnoid hemorrhage. Stroke. 1992;23:1420–1426.
Downloaded from http://stroke.ahajournals.org/ at CONS CALIFORNIA DIG LIB on January 6, 2015
Assessing the Performance of the Framingham Stroke Risk Score in the Reasons for Geographic and Racial Differences in Stroke Cohort Leslie A. McClure, Dawn O. Kleindorfer, Brett M. Kissela, Mary Cushman, Elsayed Z. Soliman and George Howard Stroke. 2014;45:1716-1720; originally published online April 15, 2014; doi: 10.1161/STROKEAHA.114.004915 Stroke is published by the American Heart Association, 7272 Greenville Avenue, Dallas, TX 75231 Copyright © 2014 American Heart Association, Inc. All rights reserved. Print ISSN: 0039-2499. Online ISSN: 1524-4628
The online version of this article, along with updated information and services, is located on the World Wide Web at: http://stroke.ahajournals.org/content/45/6/1716
Permissions: Requests for permissions to reproduce figures, tables, or portions of articles originally published in Stroke can be obtained via RightsLink, a service of the Copyright Clearance Center, not the Editorial Office. Once the online version of the published article for which permission is being requested is located, click Request Permissions in the middle column of the Web page under Services. Further information about this process is available in the Permissions and Rights Question and Answer document. Reprints: Information about reprints can be found online at: http://www.lww.com/reprints Subscriptions: Information about subscribing to Stroke is online at: http://stroke.ahajournals.org//subscriptions/
Downloaded from http://stroke.ahajournals.org/ at CONS CALIFORNIA DIG LIB on January 6, 2015
C
linical tools to predict disease are important to direct treatments and to counsel individuals on behavioral changes that might modify risks. Recent reports have offered new tools for the prediction of a composite outcome of hard atherosclerotic cardiovascular disease end point, including both coronary and stroke events1; however, these new tools do not directly address risk from the individual components of the composite outcome, and acceptance of these risk functions has been controversial.2 Several risk scores are available to predict stroke,3–8 the most well known and well accepted being the Framingham Stroke Risk Score (FSRS), which was developed during the high stroke risk period of the 1990s. Factors included in the FSRS are age, sex, systolic blood pressure, use of antihypertensive medications, diabetes mellitus, smoking, atrial fibrillation, left ventricular hypertrophy, and prevalent coronary heart disease.4 The Cardiovascular Health Study (CHS) risk score includes the same risk factors and a measure of frailty.5 In the development of a risk score from the
Atherosclerosis in Communities (ARIC) study, the goal was to determine nontraditional risk factors that may improve predictability of the model; however, their analyses determined that the traditional risk factors performed reasonably well and that the addition of nontraditional risk factors did not substantially improve prediction.5 As such, the FSRS remains the standard for predicting stroke risk in the general population. Notably absent from the FSRS, the CHS risk score, and the new American College of Cardiology/American Heart Association (ACC/AHA) Pooled Cohort Risk Equation is the effect of race on stroke risk prediction. The FSRS does not include a term for race or ethnicity because there were few blacks in the study. Similarly, in CHS, at the time the risk score was developed, there was insufficient follow-up among black participants (≈15% of the cohort). Framingham, CHS, and other studies included in the new ACC/AHA instrument included relatively few blacks, and racial differences were not the focus of these studies. In ARIC, race was considered, but
Received January 22, 2014; final revision received February 24, 2014; accepted March 13, 2014. From the Department of Biostatistics, University of Alabama at Birmingham (L.A.M., G.H.); Department of Neurology, University of Cincinnati, OH (D.O.K., B.M.K.); Department of Medicine, University of Vermont, Burlington (M.C.); and Department of Epidemiology and Prevention, Wake Forest University School of Medicine, Winston-Salem, NC (E.Z.S.). Presented in part at the International Stroke Conference of the American Heart Association, Honolulu, HI, February 6-8, 2013. The online-only Data Supplement is available with this article at http://stroke.ahajournals.org/lookup/suppl/doi:10.1161/STROKEAHA. 114.004915/-/DC1. Correspondence to Leslie A. McClure, PhD, Department of Biostatistics, University of Alabama at Birmingham, 1720 Second Ave S, Birmingham, AL 35294-0022. E-mail [email protected] © 2014 American Heart Association, Inc. Stroke is available at http://stroke.ahajournals.org
DOI: 10.1161/STROKEAHA.114.004915
Downloaded from http://stroke.ahajournals.org/ at CONS CALIFORNIA DIG LIB on January 6, 2015 1716
McClure et al The Framingham Stroke Risk Score in REGARDS 1717 inclusion in modeling did not increase predictability beyond the traditional risk factors, and ARIC confounded race and geography with most of the black study participants from the single Jackson (Mississippi) study site.1 Currently available risk scores assume that the effect of risk factors is similar for blacks and whites. The Framingham Coronary Risk Score was validated in several racial/ethnic groups9; however, this work did not assess the potential that risk factors could have a differential effect on whites and blacks. To our knowledge, the performance of the FSRS has not been assessed in different racial groups, and the possibility that risk factors could have a different role in blacks and whites has not been evaluated. We assessed the performance of the FSRS among black and white participants in the Reasons for Geographic and Racial Differences in Stroke (REGARDS) study to determine whether this tool is useful in both blacks and whites.
Methods The REGARDS study is a national population-based cohort study that recruited 30 239 participants ≥45 years of age, with 45% men and 55% women, 42% black and 58% white. Recruitment began in January 2003 and was completed in October 2007. Twenty-one percent of the cohort was recruited from the buckle of the stroke belt (coastal plain region of North Carolina, South Carolina, and Georgia), 35% from the stroke belt states (remainder of North Carolina, South Carolina, and Georgia, plus Alabama, Mississippi, Tennessee, Arkansas, and Louisiana), and the remaining 44% from the other 42 contiguous states (goal was 30% belt, 20% buckle, 50% remainder of nation). Detailed study methods are published elsewhere.10 In brief, participants were selected from commercially available lists of residents and were recruited using a combination of mail notification and telephone contact. Verbal consent and baseline data, including demographics, smoking history, and cardiovascular risk factor history, were collected via computer-assisted telephone interview. After completion of the baseline interview, an in-home visit was performed to collect physical measurements, including blood pressure, blood and urine samples, ECG, and a signed informed consent. Blood was processed centrally at the University of Vermont, and ECGs were read centrally at Wake Forest University School of Medicine. Follow-up phone contact is made at 6-month intervals for surveillance of suspected stroke events, with medical records for suspected events retrieved and adjudicated by a panel of physicians. The study was approved and monitored by the institutional review boards at all participating institutions.
Framingham Stroke Risk Factors Age, sex, use of antihypertensive medications, and smoking were all based on self-report during the initial phone interview. Age was categorized as young (45–64 years), middle (65–74 years), or older (≥75 years). Heart disease was defined as self-report of myocardial infarction, bypass, angioplasty or stenting, or EGC evidence of myocardial infarction. Participants were considered diabetic if their fasting glucose was ≥126 (or ≥200 if they were not fasting) or if they self-reported medication use for diabetes mellitus. Systolic blood pressure was the average of 2 seated measurements. Determination of atrial fibrillation was by self-report of a physician diagnosis or ECG evidence, and left ventricular hypertrophy was based on ECG evidence.
were centrally adjudicated by physicians. For deaths without medical records, death certificates were examined and adjudicated, and proxy interviews undertaken. Stroke events were defined following the World Health Organization definition.11 Those events not meeting the WHO definition but with symptoms lasting >24 hours and neuroimaging consistent with acute ischemia or hemorrhage were classified as clinical strokes and were included in analyses.
Statistical Analysis We assessed the prevalence of the FSRS factors among REGARDS participants at baseline by race–sex strata, computed the FSRS, and compared the FSRS factors across race–sex strata using ANOVA or χ2 tests of association, as appropriate. We then fit Poisson regression models to estimate the observed stroke incidence rates. We compared the observed stroke rates with those predicted by the FSRS by computing the ratio of the predicted to observed risk and the 95% confidence intervals around the risk ratios. We compared the observed and predicted stroke rates overall, by levels of predicted FSRS levels and by levels of FSRS risk factors for each race–sex stratum. Because of missing stroke data due to ongoing adjudication of stroke events, as well as differential retrieval of medical records, we used multiple imputation techniques to classify in-process stroke events using a logistic function predicting the likelihood that an attempted record retrieval would result in an adjudicated stroke.12 Ten imputed data sets were used, and 22% of reported events were eligible to be imputed.
Results Among 27 748 stroke-free participants at baseline followed for an average of 5.6 years, 30% were white women, 30% were white men, 25% were black women, and 15% were black men. Table 1 presents the average 10-year FSRS and the prevalence of FSRS risk factors by race–sex strata. All factors differed significantly across race–sex strata. Predicted stroke risk was highest in black men (11.9% 10-year risk; SD, 10.0), followed by white men (11.0% 10-year risk; SD, 9.6), black women (10.2% 10-year risk; SD, 12.1), and white women (7.8% 10-year risk; SD, 10.2). A similar pattern in the association between race–sex stratum and systolic blood pressure was observed. Table 2 provides the ratio of the expected-to-observed annualized stroke risk by race–sex strata, overall (Figure 1), by quintiles of the FSRS (Figure 2), and by levels of each of the FSRS risk factors (Figures I and II in the online-only Table 1. Average FSRS and Prevalence of the FSRS Risk Factors by Race–Sex Strata White
Black
Female (n=8342)
Male (n=8198)
Female (n=7037)
Male (n=4171)
FSRS, mean percent 10-y risk (SD)
7.8 (10.2)
11.0 (9.6)
10.2 (12.1)
11.9 (10.0)
SBP, mm Hg, mean (SD)
123 (16)
127 (15)
130 (17)
132 (17)
Current smoker
1096 (13%)
936 (11%)
1080 (15%)
803 (19%)
Age, mean (SD) Heart disease
64.5 (9.6)
65.9 (9.3)
63.4 (9.3)
64.2 (9.2)
972 (12%)
2059 (25%)
828 (12%)
Determination of Stroke Events
LVH
726 (18%)
263 (4%)
82 (2%)
455 (8%)
Events reported during follow-up as possible stoke, transient ischemic attack, death, hospitalization, or emergency department visit for brain aneurysm, brain hemorrhage, stroke symptoms, or unknown reason prompted a request for medical records. Initial review by a stroke nurse excluded obvious nonstrokes; the remaining medical records
109 (4%)
Atrial fibrillation
85 (1%)
211 (3%)
Diabetes mellitus
1021 (13%)
1394 (17%)
21 (0.3%) 1963 (29%)
56 (1%) 1226 (31%)
FSRS indicates Framingham Stroke Risk Score; LVH, left ventricular hypertrophy; and SBP, systolic blood pressure.
Downloaded from http://stroke.ahajournals.org/ at CONS CALIFORNIA DIG LIB on January 6, 2015
1718 Stroke June 2014 Table 2. Ratio of Expected-to-Observed Stroke Risk (95% Confidence Interval) by Race–Sex Strata, Overall and Across Different Levels of FSRS and FSRS Risk Factors White
Black
Female (n=8342)
Male (n=8198)
Female (n=7037)
Male (n=4171)
1.71 (1.49, 1.96)
1.88 (1.66, 2.11)
1.71 (1.49, 1.96)
1.60 (1.35, 1.89)
Q1 (0 for all 4 race–sex strata. The magnitude of overestimation in the top quintile of predicted risk was about the same for white and black men, but was larger for black women than for white women. The ratio of expected-to-observed stroke risk by race–sex strata across the levels of each of the factors contributing to the FSRS indicated that across all risk factors, at all levels, the
FSRS overestimated the stroke risk relative to that observed. The overestimation was particularly large at higher levels of risk factors, such as for those with stage II hypertension and for those with atrial fibrillation. With respect to race and sex, there were no consistent patterns regarding for whom the overestimation was largest. In some cases, the overestimation was larger for blacks than for whites (eg, for women with stage II hypertension, or for female smokers), whereas in other cases, the overestimation was worse for whites than for blacks (eg, male nonsmokers or for men with left ventricular hypertrophy).
Discussion In this large cohort of black and white adults aged ≥45 years, the validity of the FSRS was confirmed by a clearly higher observed number of stroke events among those with higher
Downloaded from http://stroke.ahajournals.org/ at CONS CALIFORNIA DIG LIB on January 6, 2015
McClure et al The Framingham Stroke Risk Score in REGARDS 1719
Figure 1. Ratio of expected-to-observed stroke risk and 95% confidence interval, by race (red indicates white; and blue, black) and sex (F indicates female; and M, male).
FSRS scores; however, the FSRS overestimated the observed stroke rates, particularly in certain subgroups. Generally, the FSRS suggested there should be approximately twice as many strokes occurring than we detected, and with a few exceptions, this was consistent across strata defined by the risk factors. This overestimation of stroke risk is similar to the suggested overestimation of risk that has been central to the controversy surrounding the predicted atherosclerotic cardiovascular disease risk in the ACC/AHA guideline instrument.2 The FSRS is not the only stroke risk score that is available. Several others have been developed, including the CHS risk score,5 the ARIC risk score,6 and others.7,8 Although REGARDS does not have all of the variables contributing to each of these, it is expected that the results would be the similar to those observed with the FSRS, given the similarities among the performance of the scores.6,13 While the magnitude of the expected-to-observed ratio of strokes was reasonably consistent, there were several exceptions. For all 4 race–sex strata, the expected-to-observed ratio was higher for study participants in the highest quintile of the FSRS. This finding supports the notion that preventive treatments reduced the number of events in these individuals. This is particularly the case for white men and black women with stage II hypertension (systolic blood pressure ≥160 mm Hg). The expected-to-observed ratio was also quite
Figure 2. Ratio of expected-to-observed stroke risk and 95% confidence interval across the quintiles of the Framingham Stroke Risk Score by race (red indicates white; and blue, black) and sex (F indicates female; and M, male).
high for both black men and women with atrial fibrillation, potentially suggesting that treatment of this condition has improved with time. With these exceptions, the magnitude of the expected-to-observed ratio of events was reasonably consistent, suggesting more of a general change in risk of stroke. With REGARDS being one of the validation studies for the new ACC/AHA guideline models that showed lower observed than predicted risk, we have suggested 4 potential contributors to the overestimation of risk.14 First, several of the studies used to develop the risk equation included both self-reported events with subsequent medical record adjudication and active surveillance for events in community hospitals, whereas in REGARDS, we used only self-reported events to retrieve records for subsequent adjudication. As such, we may be systemically missing some stroke events. Second, dynamic temporal changes in the prevalence of statin use, which was not considered in the validation cohorts including REGARDS, could contribute to fewer events than expected. Third, increased use of revascularization could contribute to fewer events than expected.15 Finally, the duration of follow-up was relatively shorter in REGARDS compared with the studies used to develop the risk functions. All of these factors could potentially be playing a role in the overestimation of stroke events in this report. However, our follow-up contacts were quite good (76% of participants completed >75% of anticipated follow-up contacts), and we cast a very broad net for suspected events (>4 suspected events were reviewed for each event adjudicated to be a stroke). In addition, incident statins use have at most a 30% effect on stroke risk16 only in the subset of the population beginning treatment; carotid revascularization would have an effect a relatively small portion of the population; and our estimates are stable with >800 stroke events. Alternatively, with the Framingham study having a cohort assessed before 2000 and the REGARDS cohort assessed after 2000, the overestimation could be a product of a dramatic temporal decline in incident stroke. Between 1999 and 2010, data from the Center for Disease Control and Prevention's Wide-Ranging Online Data for Epidemiologic Research (CDC WONDER) show that there has been a decrease in age-adjusted death from stroke (International Classification of Diseases, Tenth Revision: I60–I69) from 61.6/100 000 to 39.1/100 000 (a 37% decrease),17 with most thinking that the decline in mortality is primarily attributable to a decline in incidence. We think that a substantial portion of the overestimation of stroke risk is associated with these dramatic declines in stroke risk, suggesting the need for a recalibration of the risk formulas for incident stroke and highlighting the need for ongoing epidemiological studies that can address issues such as this over time. As with all research, this study has limitations. Although the FSRS is designed to determine 10-year risk of stroke, we only have 5.6 years of follow-up thus far in REGARDS and thus have annualized stroke rates. We have made the assumption that the rates are constant with time, and this may not be true. Furthermore, we rely on participant report of stroke, which may introduce a bias due to underreporting of potential events, inability to obtain medical records for stroke adjudication, or losses to follow-up. Although we have minimized these biases by using external resources such as National Death Index searches and relying on proxy
Downloaded from http://stroke.ahajournals.org/ at CONS CALIFORNIA DIG LIB on January 6, 2015
1720 Stroke June 2014 reports, as well as by imputing events that are still in process, they still may exist. In summary, among black and white participants in the REGARDS cohort, the FSRS overestimated the actual stroke risk, across sex and race, and at all levels of observed risk factors. The ability to estimate stroke risk accurately is important for planning research and for targeting clinical efforts toward stroke prevention. Our results might be explained by secular trends in stroke incidence since the FSRS was developed. Thus, future research should focus on estimating stroke risk more accurately across races in the context of current medical care trends.
Acknowledgments The content is solely the responsibility of the authors and does not necessarily represent the official views of the National Institute of Neurological Disorders and Stroke or the National Institutes of Health. Representatives of the funding agency have been involved in the review of the article but not directly involved in the collection, management, analysis, or interpretation of the data. We thank the other investigators, the staff, and the participants of the REGARDS study for their valuable contributions. A full list of participating investigators and institutions in REGARDS can be found at http:// www.regardsstudy.org.
Sources of Funding This research project is supported by a cooperative agreement U01 NS041588 from the National Institute of Neurological Disorders and Stroke, National Institutes of Health, and Department of Health and Human Service.
Disclosures None.
References 1. Goff DC Jr, Lloyd-Jones DM, Bennett G, O’Donnell CJ, Coady S, Robinson J, et al. ACC/AHA guideline on the assessment of cardiovascular risk: a report of the American College of Cardiology/American Heart Association task force on practice guidelines [published online ahead of print November 12, 2013]. J Am Coll Cardiol. doi:10.1016/j. jacc.2013.11.005. http://www.sciencedirect.com/science/article/pii/ S0735109713060312. Accessed February 24, 2014.
2. Ridker PM, Cook NR. Statins: new American guidelines for prevention of cardiovascular disease. Lancet. 2013;382:1762–1765. 3. Wolf PA, D’Agostino RB, Belanger AJ, Kannel WB. Probability of stroke: a risk profile from the Framingham Study. Stroke. 1991;22:312–318. 4. D’Agostino RB, Wolf PA, Belanger AJ, Kannel WB. Stroke risk profile: adjustment for antihypertensive medication. The Framingham Study. Stroke. 1994;25:40–43. 5. Manolio TA, Kronmal RA, Burke GL, O’Leary DH, Price TR. Short-term predictors of incident stroke in older adults. The Cardiovascular Health Study. Stroke. 1996;27:1479–1486. 6. Chambless LE, Heiss G, Shahar E, Earp MJ, Toole J. Prediction of ischemic stroke risk in the Atherosclerosis Risk in Communities Study. Am J Epidemiol. 2004;160:259–269. 7. Gaziano TA, Young CR, Fitzmaurice G, Atwood S, Gaziano JM. Laboratory-based versus non-laboratory-based method for assessment of cardiovascular disease risk: the NHANES I Follow-up Study cohort. Lancet. 2008;371:923–931. 8. Hippisley-Cox J, Coupland C, Brindle P. Derivation and validation of QStroke score for predicting risk of ischaemic stroke in primary care and comparison with other risk scores: a prospective open cohort study. BMJ. 2013;346:f2573. 9. D’Agostino RB Sr, Grundy S, Sullivan LM, Wilson P; CHD Risk Prediction Group. Validation of the Framingham coronary heart disease prediction scores: results of a multiple ethnic groups investigation. JAMA. 2001;286:180–187. 10. Howard VJ, Cushman M, Pulley L, Gomez CR, Go RC, Prineas RJ, et al. The reasons for geographic and racial differences in stroke study: objectives and design. Neuroepidemiology. 2005;25:135–143. 11. Report of the WHO Task Force on Stroke and Cerebrovascular Diseases. Recommendations on stroke prevention, diagnosis and therapy. Stroke. 1989;20:1407–1431. 12. Howard G, McClure LA, Moy CS, Safford MM, Cushman M, Judd SE, et al. Imputation of incident events in longitudinal cohort studies. Am J Epidemiol. 2011;174:718–726. 13. Lumley T, Kronmal RA, Cushman M, Manolio TA, Goldstein S. A stroke prediction score in the elderly: validation and Web-based application. J Clin Epidemiol. 2002;55:129–136. 14. Muntner P, Safford MM, Cushman M, Howard G. Comment on the reports of over-estimation of ASCVD risk using the 2013 AHA/ACC risk equation. Circulation. 2014;129:266–267. 15. Matsen SL, Chang DC, Perler BA, Roseborough GS, Williams GM. Trends in the in-hospital stroke rate following carotid endarterectomy in California and Maryland. J Vasc Surg. 2006;44:488–495. 16. Blauw GJ, Lagaay AM, Smelt AH, Westendorp RG. Stroke, statins, and cholesterol. A meta-analysis of randomized, placebo-controlled, doubleblind trials with HMG-CoA reductase inhibitors. Stroke. 1997;28:946–950. 17. Sloan MA, Price TR, Foulkes MA, Marler JR, Mohr JP, Hier DB, et al. Circadian rhythmicity of stroke onset. Intracerebral and subarachnoid hemorrhage. Stroke. 1992;23:1420–1426.
Downloaded from http://stroke.ahajournals.org/ at CONS CALIFORNIA DIG LIB on January 6, 2015
Assessing the Performance of the Framingham Stroke Risk Score in the Reasons for Geographic and Racial Differences in Stroke Cohort Leslie A. McClure, Dawn O. Kleindorfer, Brett M. Kissela, Mary Cushman, Elsayed Z. Soliman and George Howard Stroke. 2014;45:1716-1720; originally published online April 15, 2014; doi: 10.1161/STROKEAHA.114.004915 Stroke is published by the American Heart Association, 7272 Greenville Avenue, Dallas, TX 75231 Copyright © 2014 American Heart Association, Inc. All rights reserved. Print ISSN: 0039-2499. Online ISSN: 1524-4628
The online version of this article, along with updated information and services, is located on the World Wide Web at: http://stroke.ahajournals.org/content/45/6/1716
Permissions: Requests for permissions to reproduce figures, tables, or portions of articles originally published in Stroke can be obtained via RightsLink, a service of the Copyright Clearance Center, not the Editorial Office. Once the online version of the published article for which permission is being requested is located, click Request Permissions in the middle column of the Web page under Services. Further information about this process is available in the Permissions and Rights Question and Answer document. Reprints: Information about reprints can be found online at: http://www.lww.com/reprints Subscriptions: Information about subscribing to Stroke is online at: http://stroke.ahajournals.org//subscriptions/
Downloaded from http://stroke.ahajournals.org/ at CONS CALIFORNIA DIG LIB on January 6, 2015
