Downloaded from heart.bmj.com on October 18, 2014 - Published by group.bmj.com
Heart Online First, published on September 4, 2014 as 10.1136/heartjnl-2014-306062 Coronary artery disease
ORIGINAL ARTICLE
Performance of the GRACE scores in a New Zealand acute coronary syndrome cohort Aaron Lin,1 Gerry Devlin,2 Mildred Lee,1 Andrew J Kerr1,3 1
Department of Cardiology, Middlemore Hospital, Auckland, New Zealand 2 Department of Cardiology, Waikato Hospital, Hamilton, New Zealand 3 Section of Epidemiology and Biostatistics, University of Auckland, Auckland, New Zealand Correspondence to Dr Andrew Kerr, Department of Cardiology, Middlemore Hospital, Otahuhu, Auckland 93311, New Zealand; [email protected] Received 16 April 2014 Revised 22 June 2014 Accepted 31 July 2014
ABSTRACT Background Risk stratification after acute coronary syndrome (ACS) event is recommended to guide intensity and timing of investigation and management. The Global Registry of Acute Coronary Events (GRACE) investigators have published several scores for predicting patient risk both at hospital admission and discharge. Objective To evaluate the performance of the admission-to-6-month and discharge-to-6-month GRACE scores for predicting myocardial infarction (MI) and mortality in a contemporary cohort of patients admitted with ACS. Methods The cohort comprised 3743 consecutive patients admitted to cardiology services in two large New Zealand hospitals with an ACS between 2007 and 2011. Risk score data was collected in an electronic registry and linked anonymously to national hospitalisation and mortality records. Results Between admission and 6 months, 160 patients died and another 269 were rehospitalised with an MI. The GRACE admission-to-6-month total mortality and mortality/MI scores both overestimated event rates approximately twofold. The discharge-to-6-month mortality equation was better calibrated. Global discrimination was very good for both admission-to-6month and discharge-to-6-month mortality scores (c=0.805 and c=0.795, respectively) and moderately good for the corresponding mortality/MI equations (c=0.652 and c=0.624, respectively). Conclusions In a contemporary ACS cohort, the GRACE discharge-to-6-month mortality score has very good discrimination and accurately predicts mortality rates, whereas the admission-to-6-month equation, despite good discrimination, overestimated risk. Recalibration or more dynamic modelling of inhospital risk which includes variables such as time from admission to risk assessment are needed to support use of ACS risk assessment inhospital.
INTRODUCTION
To cite: Lin A, Devlin G, Lee M, et al. Heart Published Online First: [please include Day Month Year] doi:10.1136/heartjnl2014-306062
Early risk stratification plays a pivotal role in the management of acute coronary syndrome (ACS), as higher-risk patients are more likely to benefit from earlier and more aggressive treatment strategies.1 The Global Registry of Acute Coronary Events (GRACE) risk score is a prognostic model which encompasses the full spectrum of patients with ACS and is recommended in international guidelines.2 3 There were several GRACE risk scores published in the early and mid-2000s, which estimate the risk of both mortality or the composite of mortality and myocardial infarction (MI). The original Granger
model predicted the risk of inhospital events only.4 The subsequent Eagle model5 estimated risk in ACS patients from discharge until 6 months postdischarge and, more recently, the Fox model6 estimates risk from admission-to-6-months after presentation. However, variation in the management of ACS between healthcare systems and evolving treatment strategies mean that validation of the models within the population in whom they are to be used is important. Furthermore, the GRACE equations were developed in unselected ACS admissions identified at hospital admission; but in clinical practice it is common to apply the equations in the cohort of patients admitted to a coronary care unit (CCU) with ACS. However, ACS patients who either do not survive to be admitted to CCU, or are not clinically eligible for admission to a subspecialty team, may have had a burden of comorbidity not fully captured by the GRACE equation variables. For example, malignancy, severe airways disease and sepsis are not included. If that is the case, the GRACE equations may overestimate risk in those admitted to CCUs. In New Zealand, all patients have a National Health Identifier (NHI) number allowing linkage of our hospital ACS registry to national hospitalisation and mortality records to identify postdischarge mortality and rehospitalisation for MI. We used this linkage between the ACS registry and national records to evaluate the performance of the admission-to-6-month and discharge-to-6-month GRACE scores in a contemporary cohort of patients admitted with ACS to two New Zealand coronary care units.
METHODS Consecutive patients ≥18 years old admitted to cardiology services with suspected ACS at Middlemore Hospital (catchment 500 000) between August 2007 and October 2011, and Waikato Hospital (catchment 700 000) between January 2008 and February 2010, had data recorded during their index admission by trained clinical staff using the Acute PREDICTweb-based electronic database.
Data collection Acute PREDICT collects admission and discharge dates, ACS risk stratification, diagnostic and inhospital investigation, management and outcome data.7 Data quality is supported by definition fields within the electronic form and 3-monthly scheduled audits of data quality.8 From 2013, this registry was rebranded as the All New Zealand ACS
A, et al. Heart 2014. 2014;0:1–7. doi:10.1136/heartjnl-2014-306062 1 Copyright Article author (or their Lin employer) Produced by BMJ Publishing Group Ltd (& BCS) under licence.
Downloaded from heart.bmj.com on October 18, 2014 - Published by group.bmj.com
Coronary artery disease Quality Improvement (ANZACS-QI) registry, and is the electronic backbone of the national ANZACS-QI programme.
Data and definitions Only patients with a confirmed diagnosis of non-ST-segment elevation ACS (NSTEACS, comprising both unstable angina and non-ST elevation MI (NSTEMI)) or ST-segment elevation MI (STEMI) were included. MI was defined according to the contemporary universal definition.9 The following GRACE admission-to-6-month equation variables were collected using standard definitions—patient age, admission systolic blood pressure, admission heart rate, Killip score, admission creatinine level, cardiac arrest on admission, ST segment deviation on initial ECG (ST elevation >2 mm in V1–V3 or >1 mm in other leads present in at least two contiguous leads, or ST depression ≥0.5 mm) and positive initial cardiac biomarkers. The GRACE discharge-to-6 months predictors overlap the admission-to6-month predictors, but also include in-hospital percutaneous coronary intervention (PCI), past MI and past congestive heart failure (CHF), but not cardiac arrest. Prior MI and history of CHF were not collected in the Acute PREDICT registry. These events prior to the index admission were identified from the national hospitalisation datasets using the relevant ICD-10 codes ( prior MI ICD10 I210-I214, I219-I221, I228, I229 and I252: Prior CHF I110, I130, I132, I500, I501, I509, J81).
Outcomes All New Zealanders have a unique NHI number. We used an encrypted version of the NHI to anonymously link inhospital Acute PREDICT patient records to subsequent outcomes captured in national public hospitalisation and mortality datasets. The encryption and linkage methodology is described elsewhere.7 Recurrent MI, or death during the index admission, was captured in the Acute PREDICT record. Recurrent MI postdischarge was identified if there was a readmission after the index admission with a primary or secondary diagnosis of MI (ICD-10 codes I210-I214, I219-I221, I228, I229). Post-discharge deaths were identified using the national mortality data set. Linkage of risk to outcomes has been approved by the National Multi Region Ethics Committee (MEC/07/19/EXP).
models was evaluated by comparing the predicted with the observed outcomes in each tertile of predicted risk. Model discrimination was then assessed by calculating the area under the receiver operating characteristic curve, otherwise known as the c-index. Additionally, the ability of the Fox admission-to6-month mortality equation to stratify risk in patients with STEMI was compared with those with NSTEACS. The ability of the Fox admission-to-6-month mortality equation to predict longer-term outcomes was evaluated by estimating Kaplan-Maier survival in the ESC defined low, intermediate and high-risk subgroups. Data was analysed using SAS statistical package, V.9.2 (SAS Institute, Cary, North Carolina, USA).
RESULTS The cohort comprised consecutive New Zealand residents ≥18 years (n=3743) admitted with an ACS between 2007 and 2011 (2531 to Middlemore Hospital, 1211 to Waikato Hospital). Acute PREDICT records were completed in 99% of all patients admitted to the service. All patients had at least 6 months of potential follow-up data available through the linkage to national datasets. The demographics, diagnoses, risk factors, investigations and management according to risk group are shown in table 1. Europeans were more likely to be in the higher-risk category than Maori, Pacific or Indians (31% vs 18%, 20% and 16%, respectively). High-risk patients were also more likely to be older, male and have a history of cerebrovascular disease, prior MI or heart failure. High-risk patients were less likely to undergo revascularisation with PCI or coronary artery bypass surgery than lower-risk patients. By 6 months after admission, 160 (4.3%) patients had died (11 (0.9%) low-risk category, 32 (2.2%) intermediate-risk category, and 117 (11.6%) high-risk category), and 429 (11.5%) patients had either died or had a MI (77 (6.0%) low-risk category, 154 (10.6%) intermediate-risk category, and 198 (19.6%) high-risk category). Figure 1 shows survival to 3 years postadmission (mean follow-up time 2.59 years) for patients in low, intermediate and high-risk groups. The curves continue to diverge beyond 6 months.
Statistical analysis
GRACE equation calibration and discrimination
To describe the baseline characteristics of the population, three risk categories were established using the cutoff points set out in the European Society of Cardiology (ESC) guidelines.2 Using the admission-to-6-months mortality equation, the three categories were: low-risk, GRACE score
Heart Online First, published on September 4, 2014 as 10.1136/heartjnl-2014-306062 Coronary artery disease
ORIGINAL ARTICLE
Performance of the GRACE scores in a New Zealand acute coronary syndrome cohort Aaron Lin,1 Gerry Devlin,2 Mildred Lee,1 Andrew J Kerr1,3 1
Department of Cardiology, Middlemore Hospital, Auckland, New Zealand 2 Department of Cardiology, Waikato Hospital, Hamilton, New Zealand 3 Section of Epidemiology and Biostatistics, University of Auckland, Auckland, New Zealand Correspondence to Dr Andrew Kerr, Department of Cardiology, Middlemore Hospital, Otahuhu, Auckland 93311, New Zealand; [email protected] Received 16 April 2014 Revised 22 June 2014 Accepted 31 July 2014
ABSTRACT Background Risk stratification after acute coronary syndrome (ACS) event is recommended to guide intensity and timing of investigation and management. The Global Registry of Acute Coronary Events (GRACE) investigators have published several scores for predicting patient risk both at hospital admission and discharge. Objective To evaluate the performance of the admission-to-6-month and discharge-to-6-month GRACE scores for predicting myocardial infarction (MI) and mortality in a contemporary cohort of patients admitted with ACS. Methods The cohort comprised 3743 consecutive patients admitted to cardiology services in two large New Zealand hospitals with an ACS between 2007 and 2011. Risk score data was collected in an electronic registry and linked anonymously to national hospitalisation and mortality records. Results Between admission and 6 months, 160 patients died and another 269 were rehospitalised with an MI. The GRACE admission-to-6-month total mortality and mortality/MI scores both overestimated event rates approximately twofold. The discharge-to-6-month mortality equation was better calibrated. Global discrimination was very good for both admission-to-6month and discharge-to-6-month mortality scores (c=0.805 and c=0.795, respectively) and moderately good for the corresponding mortality/MI equations (c=0.652 and c=0.624, respectively). Conclusions In a contemporary ACS cohort, the GRACE discharge-to-6-month mortality score has very good discrimination and accurately predicts mortality rates, whereas the admission-to-6-month equation, despite good discrimination, overestimated risk. Recalibration or more dynamic modelling of inhospital risk which includes variables such as time from admission to risk assessment are needed to support use of ACS risk assessment inhospital.
INTRODUCTION
To cite: Lin A, Devlin G, Lee M, et al. Heart Published Online First: [please include Day Month Year] doi:10.1136/heartjnl2014-306062
Early risk stratification plays a pivotal role in the management of acute coronary syndrome (ACS), as higher-risk patients are more likely to benefit from earlier and more aggressive treatment strategies.1 The Global Registry of Acute Coronary Events (GRACE) risk score is a prognostic model which encompasses the full spectrum of patients with ACS and is recommended in international guidelines.2 3 There were several GRACE risk scores published in the early and mid-2000s, which estimate the risk of both mortality or the composite of mortality and myocardial infarction (MI). The original Granger
model predicted the risk of inhospital events only.4 The subsequent Eagle model5 estimated risk in ACS patients from discharge until 6 months postdischarge and, more recently, the Fox model6 estimates risk from admission-to-6-months after presentation. However, variation in the management of ACS between healthcare systems and evolving treatment strategies mean that validation of the models within the population in whom they are to be used is important. Furthermore, the GRACE equations were developed in unselected ACS admissions identified at hospital admission; but in clinical practice it is common to apply the equations in the cohort of patients admitted to a coronary care unit (CCU) with ACS. However, ACS patients who either do not survive to be admitted to CCU, or are not clinically eligible for admission to a subspecialty team, may have had a burden of comorbidity not fully captured by the GRACE equation variables. For example, malignancy, severe airways disease and sepsis are not included. If that is the case, the GRACE equations may overestimate risk in those admitted to CCUs. In New Zealand, all patients have a National Health Identifier (NHI) number allowing linkage of our hospital ACS registry to national hospitalisation and mortality records to identify postdischarge mortality and rehospitalisation for MI. We used this linkage between the ACS registry and national records to evaluate the performance of the admission-to-6-month and discharge-to-6-month GRACE scores in a contemporary cohort of patients admitted with ACS to two New Zealand coronary care units.
METHODS Consecutive patients ≥18 years old admitted to cardiology services with suspected ACS at Middlemore Hospital (catchment 500 000) between August 2007 and October 2011, and Waikato Hospital (catchment 700 000) between January 2008 and February 2010, had data recorded during their index admission by trained clinical staff using the Acute PREDICTweb-based electronic database.
Data collection Acute PREDICT collects admission and discharge dates, ACS risk stratification, diagnostic and inhospital investigation, management and outcome data.7 Data quality is supported by definition fields within the electronic form and 3-monthly scheduled audits of data quality.8 From 2013, this registry was rebranded as the All New Zealand ACS
A, et al. Heart 2014. 2014;0:1–7. doi:10.1136/heartjnl-2014-306062 1 Copyright Article author (or their Lin employer) Produced by BMJ Publishing Group Ltd (& BCS) under licence.
Downloaded from heart.bmj.com on October 18, 2014 - Published by group.bmj.com
Coronary artery disease Quality Improvement (ANZACS-QI) registry, and is the electronic backbone of the national ANZACS-QI programme.
Data and definitions Only patients with a confirmed diagnosis of non-ST-segment elevation ACS (NSTEACS, comprising both unstable angina and non-ST elevation MI (NSTEMI)) or ST-segment elevation MI (STEMI) were included. MI was defined according to the contemporary universal definition.9 The following GRACE admission-to-6-month equation variables were collected using standard definitions—patient age, admission systolic blood pressure, admission heart rate, Killip score, admission creatinine level, cardiac arrest on admission, ST segment deviation on initial ECG (ST elevation >2 mm in V1–V3 or >1 mm in other leads present in at least two contiguous leads, or ST depression ≥0.5 mm) and positive initial cardiac biomarkers. The GRACE discharge-to-6 months predictors overlap the admission-to6-month predictors, but also include in-hospital percutaneous coronary intervention (PCI), past MI and past congestive heart failure (CHF), but not cardiac arrest. Prior MI and history of CHF were not collected in the Acute PREDICT registry. These events prior to the index admission were identified from the national hospitalisation datasets using the relevant ICD-10 codes ( prior MI ICD10 I210-I214, I219-I221, I228, I229 and I252: Prior CHF I110, I130, I132, I500, I501, I509, J81).
Outcomes All New Zealanders have a unique NHI number. We used an encrypted version of the NHI to anonymously link inhospital Acute PREDICT patient records to subsequent outcomes captured in national public hospitalisation and mortality datasets. The encryption and linkage methodology is described elsewhere.7 Recurrent MI, or death during the index admission, was captured in the Acute PREDICT record. Recurrent MI postdischarge was identified if there was a readmission after the index admission with a primary or secondary diagnosis of MI (ICD-10 codes I210-I214, I219-I221, I228, I229). Post-discharge deaths were identified using the national mortality data set. Linkage of risk to outcomes has been approved by the National Multi Region Ethics Committee (MEC/07/19/EXP).
models was evaluated by comparing the predicted with the observed outcomes in each tertile of predicted risk. Model discrimination was then assessed by calculating the area under the receiver operating characteristic curve, otherwise known as the c-index. Additionally, the ability of the Fox admission-to6-month mortality equation to stratify risk in patients with STEMI was compared with those with NSTEACS. The ability of the Fox admission-to-6-month mortality equation to predict longer-term outcomes was evaluated by estimating Kaplan-Maier survival in the ESC defined low, intermediate and high-risk subgroups. Data was analysed using SAS statistical package, V.9.2 (SAS Institute, Cary, North Carolina, USA).
RESULTS The cohort comprised consecutive New Zealand residents ≥18 years (n=3743) admitted with an ACS between 2007 and 2011 (2531 to Middlemore Hospital, 1211 to Waikato Hospital). Acute PREDICT records were completed in 99% of all patients admitted to the service. All patients had at least 6 months of potential follow-up data available through the linkage to national datasets. The demographics, diagnoses, risk factors, investigations and management according to risk group are shown in table 1. Europeans were more likely to be in the higher-risk category than Maori, Pacific or Indians (31% vs 18%, 20% and 16%, respectively). High-risk patients were also more likely to be older, male and have a history of cerebrovascular disease, prior MI or heart failure. High-risk patients were less likely to undergo revascularisation with PCI or coronary artery bypass surgery than lower-risk patients. By 6 months after admission, 160 (4.3%) patients had died (11 (0.9%) low-risk category, 32 (2.2%) intermediate-risk category, and 117 (11.6%) high-risk category), and 429 (11.5%) patients had either died or had a MI (77 (6.0%) low-risk category, 154 (10.6%) intermediate-risk category, and 198 (19.6%) high-risk category). Figure 1 shows survival to 3 years postadmission (mean follow-up time 2.59 years) for patients in low, intermediate and high-risk groups. The curves continue to diverge beyond 6 months.
Statistical analysis
GRACE equation calibration and discrimination
To describe the baseline characteristics of the population, three risk categories were established using the cutoff points set out in the European Society of Cardiology (ESC) guidelines.2 Using the admission-to-6-months mortality equation, the three categories were: low-risk, GRACE score
