Emergency. 2017; 5 (1): e4 O RIGINAL R ESEARCH
Comparison of APACHE II and SAPS II Scoring Systems in Prediction of Critically Ill Patient’s Outcome Hamed Aminiahidashti1 , Farzad Bozorgi1 , Seyyed Hosein Montazer1 , Majid Baboli1∗ , Abolfazl Firouzian2 1. Department of Emergency Medicine, Faculty of Medicine, Mazandaran University of Medical Sciences, Sari, Iran. 2. Department of Anesthesiology, Faculty of Medicine, Mazandaran University of Medical Sciences, Sari, Iran.
Received: December 2015; Accepted: January 2016; Published online: 8 January 2017
Abstract:
Introduction: Using physiologic scoring systems for identifying high-risk patients for mortalityhas been consideredrecently. This study was designed to evaluate the values of Acute Physiology and Chronic Health Evaluation II (APACHE II)and SimplifiedAcute Physiologic Score (SAPS II) models in prediction of 1-month mortality of critically ill patients. Methods: The present prospective cross sectional study was performed on critically ill patientspresented to emergency department during 6 months.Data required for calculation of the scores were gathered and performance of the models in prediction of 1-month mortality were assessed using STATA software 11.0. Results: 82 critically ill patients with the mean age of 53.45 ± 20.37 years were included (65.9% male). Their mortality rate was 48%. Mean SAPS II (p < 0.0001) and APACHE II (p = 0.0007) scores were significantly higher in dead patients. Area under the ROC curve of SAPS II and APACHE II for prediction of mortality were 0.75 (95% CI: 0.64-0.86) and 0.72 (95% CI: 0.60-0.83), respectively (p = 0.24). The slope and intercept of SAPS II were 1.02 and 0.04, respectively. In addition, these values were 0.92 and 0.09 for APACHE II, respectively. Conclusion: The findings of the present study showed that APACHE II and SAPS II had similar value in predicting 1-month mortality of patients. Discriminatory powers of the mentioned models were acceptable but their calibration had some amount of lack of fit, which reveals that APACHE II and SAPS II are partially perfect.
Keywords: APACHE; patient outcome assessment; critical illness; validation studies [publication type]; emergency service, hospital © Copyright (2017) Shahid Beheshti University of Medical Sciences Cite this article as: Aminiahidashti H, Bozorgi F, Montazer S H, Baboli M, Firouzian A. Comparison of APACHE II and SAPS IIScoring Systems in Prediction of Critically III Patient’s Outcome. Emergency. 2017; 5 (1): e4.
1. Introduction Triage of high-risk patients in emergency department (ED) and focusedand carefulmanagement of themmight result in a drop in their mortality rate (1-4). A scoring model with high screening performance characteristics can provide considerable advantages for health systems. These advantages include prediction of patient outcome, evaluating the efficiency of treatments used, efficient pre- and in-hospital triage, and quality improvement of treatment measures and preventive plans (6). In addition, scoring systems are able to convert the severity of an illness into a number, which results in a common understanding between physicians for taking
∗ Corresponding Author:
Majid Baboli; Imam Khomeini Hospital, Amir Mazandarani Boulevard, Sari, Mazandaran, Iran. Tel:+989113540546, Email: [email protected].
measures and developing quality control plans regarding patient care. Researchers have long attempted to design various scoring systems for this purpose. They have modified these systems to increase their efficiency, accuracy, and validity. Despite significant advances in these systems, unfortunately these models have had some deficiencies and limitations(5). These limitations include complicated calculations for some models, their high number of variables, and unevaluated validity in various clinical conditions. Therefore, research in this field is ongoing and new models are introduced each year. Using physiologic scoring systems for identifying high-risk patients for deathhas been especially considered in recent years. To date, some physiologic scoring systems have been invented and introduced. One of the first physiologic scoring systems is A cute Physiology and Chronic Health Evaluation II (APACHE II), introduced by Knaus et al. in 1985. This model is calculated based on 12 physiologic criteria, age, and previous condition of the patient. Existing
This open-access article distributed under the terms of the Creative Commons Attribution NonCommercial 3.0 License (CC BY-NC 3.0). Downloaded from: www.jemerg.com
H. Aminiahidashti et al.
studies have revealed the close relation of this score with inhospital and 1-month mortalityin critically ill patients (7, 8). Simplified Acute Physiologic Score (SAPS II) is among other scoring models in this field, proposed by Le Gall et al. This model consists of 17 variables including 12 physiologic factors, age, type of admission, and 3 variables regarding underlying diseases (9). Predictive value of this model has been confirmed in different clinical conditions (10-12). These 2 models have been compared in different studies that have yielded somehow contradicting results (13-15). Therefore, the present study was designed aiming to evaluate and compare the values of APACHE II and SAPS II models in prediction of 1-month mortality of critically ill patients presented to emergency department (ED).
2
Figure 1: Mean score± standard error of APACHE II and SAPS II in alive and dead patients (p < 0.001).
2. Methods 2.1. Study design and settings The present prospective cross sectional study was performed on critically ill patients admitted to Imam Khomeini Hospital, Sari, Iran, during February to June 2015 and assessedthe accuracy of APACHE II and SAPS II in prediction of in hospital mortality. Ethics committee of Mazandaran University of Medical Sciences approved the protocol of the study. Informed consent was taken from patients. The researchers adhered to principles of Helsinki Deceleration.
2.2. Participants Critically ill patients were diagnosed based on appearance of patients, neurological assessment, respiratory status, cardiovascular assessment at time of admission to ED (Panel 1) (16) and were enrolled using convenience sampling. Participants lost to follow-up were excluded. Age, gender, diagnosis impression, underlying diseases, vital signs, Glasgow Coma Scale (GCS), urinary output, need for ventilator, andlength ofintensive care unit (ICU) and hospital stay of all participantswere gathered using a pre-designed checklist. Moreover, laboratory data including cell blood count (CBC), hematocrit, sodium, potassium, creatinine, bilirubin, and arterial blood gas analysis (pH, bicarbonate level, and oxygen and carbon dioxide pressure) were measured and recorded. APACHE II and SAPS II scores were calculated during the first 24 hours after admission based on detailed method of calculations presented in previous studies (7, 17). 30-day mortality rate was assessed using patient’s medical records and calling them by the phone. Finally, patients were classified as alive and dead.
2.3. Statistical analysis The number of samples was calculated to be 82 patient’s based on a 50% prevalence of mortality in critically ill patients (18-20), considering a confidence interval (CI) of 95%
Figure 2: Receiver operating characteristic (ROC) curve of SAPS II and APACHE II in mortality prediction (p = 0.24).
(α = 0.05), and a power of 80% (β = 0.2). STATA software version 11.0 was used for data analysis. Qualitative variables are presented as frequency and percentage and quantitative factors are presented as mean and standard deviation. MannWhitney U testand Fisher’s exact testwere used for comparisons. Validations of the models were assessed using discriminatory powerestimation, calibration of predictive models, or a combination of the two. The discriminatory power was evaluated through calculating area under the receiver operating characteristic (ROC) curve (AUC) with 95% CI. General calibration of the model was also evaluated through drawing a calibration plot. In this plot, the perfect calibration is the reference line with an intercept of zero and a slope of 1. The overall performance was eventually assessed via Brier score in order to evaluate predictive accuracy and reliability of the model. P value < 0.05 was considered statistically significant.
3. Results 82 critically ill patients with the mean age of 53.45 ± 20.37 yearswere included (65.9% male). There were no cases of loss
This open-access article distributed under the terms of the Creative Commons Attribution NonCommercial 3.0 License (CC BY-NC 3.0). Downloaded from: www.jemerg.com
3
Emergency. 2017; 5 (1): e4
Panel 1: Diagnostic criteria of critically ill patients Appearance Neurological Respiratory Cardiovascular Gray skin Unresponsive Silent chest PR< 50 Blue skin Eyes open to pain only RR < 8 or > 30 PR > 150 Mottled skin Fitting Agonal respiration SPB< 60 RR: respiratory rateper minute;PR: pulse rate per minute, SBP: systolic blood pressure (mmHg).
Table 1: Baseline characteristics of participants based on their outcome
Factor Alive Age (year) 45.90±20.78 Gender Male 29 (69.05) Female 13 (39.95) Reason of hospitalization Medical 24 (57.14) Surgical (emergent) 13 (30.95) Surgical (Elective) 5 (11.90) Underlying disease None 31 (73.81) Acute renal failure 0 (0.0) Carcinoma 2 (4.76) Metastasis 2 (4.76) Systemic weakness 3 (7.14) Other 4 (9.52) Reason of ICU admission Cardiovascular 0 (0.0) Infection 12 (28.57) Respiratory 5 (11.90) Neurologic 5 (11.90) Multiple trauma 10 (23.81) Head trauma 8 (19.05) Other 2 (4.76) Length of ICU stay (day) 3.38 ± 3.01 Data are presented as mean ± standard deviation or number (%).
Death 61.38±16.84 25 (62.50) 15 (37.50) 36 (90.0) 4 (10.0) 0 (0.0)
P 53.45±20.38
Total 0.0006
54 (65.58) 28 (34.15)
0.53
60 (73.17) 0.002 0.24 17 (20.73) 5 (5.10)
17 (42.50) 1 (2.50) 8 (20.0) 4 (10.0) 5 (12.50) 5 (12.50)
48 (58.54) 1 (1.22) 10 (12.20) 6 (7.32) 8 (9.76) 9 (10.98)
0.06
2 (5.0) 21 (52.50) 7 (17.50) 3 (7.50) 2 (5.0) 1 (2.50) 4 (10.0) 6.10 ± 3.83
2 (2.44) 33 (40.24) 12 (14.63) 8 (9.76) 12 (14.63) 9 (10.98) 6 (7.32) 4.77 ± 3.70
0.007
0.0003
Table 2: Overall performances of SAPS II and APACHE II Model SAPS II APACHE II
Brier Score 0.201 0.213
Sanders resolution 0.182 0.193
to follow-up. The most common cause of hospitalization was non-surgical (73.17%). Mean length of hospital stay was 4.78 ± 3.69 days and mortality rate was 48% (40 patients). Table 1 shows the baseline characteristics of patients. Age (p = 0.0006), reason of hospitalization (p = 0.002), and reason of ICU admission (p = 0.007) correlated with mortality. Mean SAPS II and APACHE II scores were 42.85 ± 19.67 and 19.69 ± 8.91, respectively. Mean SAPS II (p < 0.0001) and APACHE II (p = 0.0007) scores were significantly higher in dead pa-
Reliability 0.019 0.024
Goodness of fit (%) 86.22 84.51
tients (Figure 1). AUC of SAPS II and APACHE II for prediction of mortality were 0.75 (95% CI: 0.64-0.86) and 0.72 (95% CI: 0.60-0.83), respectively (p = 0.24) (Figure 2). Calibration plots of these two scoring systems were presented in figure 3. The slope and intercept of SAPS II were 1.02 and 0.04, respectively. In addition, these values were 0.92 and 0.09 for APACHE II, respectively. Overall performances of SAPS II and APACHE II are presented in table 2. Brier score of SAPS II and APACHE II were 0.201 and 0.213, respectively. In addition, re-
This open-access article distributed under the terms of the Creative Commons Attribution NonCommercial 3.0 License (CC BY-NC 3.0). Downloaded from: www.jemerg.com
H. Aminiahidashti et al.
liability of 0.019 and 0.024 for SAPS II and APACHE II shows goodness of fit of them in prediction of mortality.
4. Discussion Results of the present study showed that APACHE II and SAPS II models have similar value in prediction of 1-month mortality of the patients. Calibration of the 2 models had some amount of lack of fit. The two models showed partial adherence to the reference line, which indicates that the models are partially perfect in prediction of mortality. Discriminatory power was acceptable for both models. In comparison with the results of the present study, Alizadeh et al. have expressed that APACHE II has higher value in prediction of mortality and disability resulting from intoxication compared to SAPSII (13). Similar findings have been reported by TaghaviGilani et al. (21). However, by paying close attention to the TaghaviGilaniet al. article, we can see that AUC is 0.83 for APACHE II model and 0.78 for SAPS II; this difference does not seem statistically different. Haddadi et al. also revealed the value of these models in patient mortality prediction (11). In contrast, Sungurtekin et al. showed higher value for SAPS II model compared to APACHE II (22). These differences might be due to variations instudypopulation and sample size, duration offollow-up and participant selection criteria. Although the discriminatory powers of both APACHE II and SAPS II models were in an acceptable range, findings show some amount of lack of fit. Therefore, calibration of the mentioned models is not completely perfect. In line with the present study, Beck et al. also displayed the external validation of the mentioned models with a similar pattern but its calibration was imperfect (23). In another study, Khwannimit and Greater also expressed that AUC for APACHE II model in prediction of critically ill patient’s mortality is 0.79, yet the calibration of this model is reported to be poor (24). This might be mainly due to disease etiology and data gathering method not being homogenous (9, 24). Recent studies have shown that data gathering errors have been common, especially regarding patients with high or low APACHEII and GCS scores, and this affects the predictiverole of the mentioned models (25). However, in the present study we tried to minimize data gathering errors by training the resident before initiation of sampling. Possibility of selection bias in this study should not be overlooked since the study was single centric and participant selection was done using convenience sampling. Other limitations of this study include etiology of participant admission not being homogenous. This affected model calibration and led to detection of some amount of lack of fit in the 2 studied models.
4
5. Conclusion The findings of the present study showed that APACHE II and SAPS II had similar value in predicting 1-month mortality of patients. Discriminatory powers of the mentioned models were acceptable but their calibration had some amount of lack of fit, which reveals that APACHE II and SAPS II are partially perfect.
6. Appendix 6.1. Acknowledgements The authors wish to thank all the staff of the emergency department of Imam Khomeini Hospital, Sari, Iran.
6.2. Author contribution All authors passed four criteria for authorship contribution based on recommendations of the International Committee of Medical Journal Editors.
6.3. Conflict of interest None.
6.4. Funding support None.
References 1. Mommsen P, Zeckey C, Andruszkow H, Weidemann J, Fromke C, Puljic P, et al. Comparison of different thoracic trauma scoring systems in regards to prediction of posttraumatic complications and outcome in blunt chest trauma. Journal of surgical research. 2012;176(1):239-47. 2. Shojaee M, Faridaalaee G, Yousefifard M, Yaseri M, Arhami Dolatabadi A, Sabzghabaei A, et al. New scoring system for intra-abdominal injury diagnosis after blunt trauma. Chinese journal of traumatology. 2014;17(1):1924. 3. Forouzanfar MM, Safari S, Niazazari M, Baratloo A, Hashemi B, Hatamabadi HR, et al. Clinical decision rule to prevent unnecessary chest X-ray in patients with blunt multiple traumas. Emergency Medicine Australasia. 2014;26(6):561-6. 4. Hashemi B, Amanat M, Baratloo A, Forouzanfar MM, Rahmati F, Motamedi M, et al. Validation of CRASH Model in Prediction of 14-day Mortality and 6-month Unfavorable Outcome of Head Trauma Patients. EMERGENCY-An Academic Emergency Medicine Journal. 2016;4:[In press]. 5. Staff T, Eken T, Wik L, Roislien J, Sovik S. Physiologic, demographic and mechanistic factors predicting New In-
This open-access article distributed under the terms of the Creative Commons Attribution NonCommercial 3.0 License (CC BY-NC 3.0). Downloaded from: www.jemerg.com
5
Emergency. 2017; 5 (1): e4
Figure 3: Calibration plots of SAPS II (A) and APACHE II (B).
6.
7.
8.
9.
10.
11.
12.
13.
jury Severity Score (NISS) in motor vehicle accident victims. Injury. 2014;45(1):9-15. Chawda M, Hildebrand F, Pape HC, Giannoudis PV. Predicting outcome after multiple trauma: which scoring system Injury. 2004;35(4):347-58. Knaus WA, Draper EA, Wagner DP, Zimmerman JE. APACHE II: a severity of disease classification system. Critical care medicine. 1985;13(10):818-29. Giamarellos-Bourboulis EJ, Norrby-Teglund A, Mylona V, Savva A, Tsangaris I, Dimopoulou I, et al. Risk assessment in sepsis: a new prognostication rule by APACHE II score and serum soluble urokinase plasminogen activator receptor. Critical care (London, England). 2012;16(4):R149. Le Gall JR, Lemeshow S, Saulnier F. A new Simplified Acute Physiology Score (SAPS II) based on a European/North American multicenter study. Jama. 1993;270(24):2957-63. Czorlich P, Sauvigny T, Ricklefs F, Kluge S, Vettorazzi E, Regelsberger J, et al. The simplified acute physiology score II to predict hospital mortality in aneurysmal subarachnoid hemorrhage. Acta neurochirurgica. 2015;157(12):2051-9. Haddadi A, Ledmani M, Gainier M, Hubert H, De Micheaux PL. Comparing the APACHE II, SOFA, LOD, and SAPS II scores in patients who have developed a nosocomial infection. Bangladesh Critical Care Journal. 2014;2(1):4-9. Adamzik M, Langemeier T, Frey UH, Gorlinger K, Saner F, Eggebrecht H, et al. Comparison of thrombelastometry with simplified acute physiology score II and sequential organ failure assessment scores for the prediction of 30day survival: a cohort study. Shock. 2011;35(4):339-42. Alizadeh AM, Hassanian-Moghaddam H, Shadnia S, Za-
14.
15.
16.
17.
18.
19.
mani N, Mehrpour O. Simplified Acute Physiology Score II/Acute Physiology and Chronic Health Evaluation II and Prediction of the Mortality and Later Development of Complications in Poisoned Patients Admitted to Intensive Care Unit. Basic, Clinical Pharmacology, Toxicology. 2014;115(3):297-300. Brinkman S, Bakhshi-Raiez F, Abu-Hanna A, de Jonge E, Bosman RJ, Peelen L, et al. External validation of Acute Physiology and Chronic Health Evaluation IV in Dutch intensive care units and comparison with Acute Physiology and Chronic Health Evaluation II and Simplified Acute Physiology Score II. Journal of Critical Care. 2011;26(1):105.e11-.e18. Seak C-J, Ng C-J, Yen DH-T, Wong Y-C, Hsu K-H, Seak JC-Y, et al. Performance assessment of the Simplified Acute Physiology Score II, the Acute Physiology and Chronic Health Evaluation II score, and the Sequential Organ Failure Assessment score in predicting the outcomes of adult patients with hepatic portal venous gas in the ED. The American Journal of Emergency Medicine. 2014;32(12):1481-4. Frost P, Wise MP. Recognition and early management of the critically ill ward patient. British Journal of Hospital Medicine (2005). 2007;68(Sup10):M180-M3. Le Gall J-R, Lemeshow S, Saulnier F. A new simplified acute physiology score (SAPS II) based on a European/North American multicenter study. Jama. 1993;270(24):2957-63. Arabi Y, Haddad S, Goraj R, Al-Shimemeri A, Al-Malik S. Assessment of performance of four mortality prediction systems in a Saudi Arabian intensive care unit. Critical care. 2002;6(2):166-74. Faruq MO, Mahmud MR, Begum T, Ahsan AA, Fatema K,
This open-access article distributed under the terms of the Creative Commons Attribution NonCommercial 3.0 License (CC BY-NC 3.0). Downloaded from: www.jemerg.com
H. Aminiahidashti et al.
Ahmed F, et al. A comparison of severity systems APACHE II and SAPS II in critically ill patients. Bangladesh Critical Care Journal. 2013;1(1):27-32. 20. Asadzandi M, Taghizade K, Tadrisi S, Ebadi A. Estimation of the mortality rate using the APACHE II standard disease severity scoring system in intensive care unit patients. Iranian Journal of Critical Care Nursing (IJCCN). 2012;4(4):209-14. 21. Gilani MT, Razavi M, Azad AM. A comparison of Simplified Acute Physiology Score II, Acute Physiology and Chronic Health Evaluation II and Acute Physiology and Chronic Health Evaluation III scoring system in predicting mortality and length of stay at surgical intensive care unit. Nigerian Medical Journal : Journal of the Nigeria Medical Association. 2014;55(2):144-7. 22. Sungurtekin H, Gurses E, Balci C. Evaluation of several clinical scoring tools in organophosphate poi-
6
soned patients. Clinical toxicology (Philadelphia, Pa). 2006;44(2):121-6. 23. Beck DH, Smith GB, Pappachan JV, Millar B. External validation of the SAPS II, APACHE II and APACHE III prognostic models in South England: a multicentre study. Intensive care medicine. 2003;29(2):249-56. 24. Khwannimit B, Geater A. A comparison of APACHE II and SAPS II scoring systems in predicting hospital mortality in Thai adult intensive care units. Journal of the Medical Association of Thailand = Chotmaihet thangphaet. 2007;90(4):643-52. 25. HOLT AW, BURY LK, BERSTEN AD, SKOWRONSKI GA, VEDIG AE. Prospective evaluation of residents and nurses as severity score data collectors. Critical care medicine. 1992;20(12):1688-91.
This open-access article distributed under the terms of the Creative Commons Attribution NonCommercial 3.0 License (CC BY-NC 3.0). Downloaded from: www.jemerg.com
Comparison of APACHE II and SAPS II Scoring Systems in Prediction of Critically Ill Patient’s Outcome Hamed Aminiahidashti1 , Farzad Bozorgi1 , Seyyed Hosein Montazer1 , Majid Baboli1∗ , Abolfazl Firouzian2 1. Department of Emergency Medicine, Faculty of Medicine, Mazandaran University of Medical Sciences, Sari, Iran. 2. Department of Anesthesiology, Faculty of Medicine, Mazandaran University of Medical Sciences, Sari, Iran.
Received: December 2015; Accepted: January 2016; Published online: 8 January 2017
Abstract:
Introduction: Using physiologic scoring systems for identifying high-risk patients for mortalityhas been consideredrecently. This study was designed to evaluate the values of Acute Physiology and Chronic Health Evaluation II (APACHE II)and SimplifiedAcute Physiologic Score (SAPS II) models in prediction of 1-month mortality of critically ill patients. Methods: The present prospective cross sectional study was performed on critically ill patientspresented to emergency department during 6 months.Data required for calculation of the scores were gathered and performance of the models in prediction of 1-month mortality were assessed using STATA software 11.0. Results: 82 critically ill patients with the mean age of 53.45 ± 20.37 years were included (65.9% male). Their mortality rate was 48%. Mean SAPS II (p < 0.0001) and APACHE II (p = 0.0007) scores were significantly higher in dead patients. Area under the ROC curve of SAPS II and APACHE II for prediction of mortality were 0.75 (95% CI: 0.64-0.86) and 0.72 (95% CI: 0.60-0.83), respectively (p = 0.24). The slope and intercept of SAPS II were 1.02 and 0.04, respectively. In addition, these values were 0.92 and 0.09 for APACHE II, respectively. Conclusion: The findings of the present study showed that APACHE II and SAPS II had similar value in predicting 1-month mortality of patients. Discriminatory powers of the mentioned models were acceptable but their calibration had some amount of lack of fit, which reveals that APACHE II and SAPS II are partially perfect.
Keywords: APACHE; patient outcome assessment; critical illness; validation studies [publication type]; emergency service, hospital © Copyright (2017) Shahid Beheshti University of Medical Sciences Cite this article as: Aminiahidashti H, Bozorgi F, Montazer S H, Baboli M, Firouzian A. Comparison of APACHE II and SAPS IIScoring Systems in Prediction of Critically III Patient’s Outcome. Emergency. 2017; 5 (1): e4.
1. Introduction Triage of high-risk patients in emergency department (ED) and focusedand carefulmanagement of themmight result in a drop in their mortality rate (1-4). A scoring model with high screening performance characteristics can provide considerable advantages for health systems. These advantages include prediction of patient outcome, evaluating the efficiency of treatments used, efficient pre- and in-hospital triage, and quality improvement of treatment measures and preventive plans (6). In addition, scoring systems are able to convert the severity of an illness into a number, which results in a common understanding between physicians for taking
∗ Corresponding Author:
Majid Baboli; Imam Khomeini Hospital, Amir Mazandarani Boulevard, Sari, Mazandaran, Iran. Tel:+989113540546, Email: [email protected].
measures and developing quality control plans regarding patient care. Researchers have long attempted to design various scoring systems for this purpose. They have modified these systems to increase their efficiency, accuracy, and validity. Despite significant advances in these systems, unfortunately these models have had some deficiencies and limitations(5). These limitations include complicated calculations for some models, their high number of variables, and unevaluated validity in various clinical conditions. Therefore, research in this field is ongoing and new models are introduced each year. Using physiologic scoring systems for identifying high-risk patients for deathhas been especially considered in recent years. To date, some physiologic scoring systems have been invented and introduced. One of the first physiologic scoring systems is A cute Physiology and Chronic Health Evaluation II (APACHE II), introduced by Knaus et al. in 1985. This model is calculated based on 12 physiologic criteria, age, and previous condition of the patient. Existing
This open-access article distributed under the terms of the Creative Commons Attribution NonCommercial 3.0 License (CC BY-NC 3.0). Downloaded from: www.jemerg.com
H. Aminiahidashti et al.
studies have revealed the close relation of this score with inhospital and 1-month mortalityin critically ill patients (7, 8). Simplified Acute Physiologic Score (SAPS II) is among other scoring models in this field, proposed by Le Gall et al. This model consists of 17 variables including 12 physiologic factors, age, type of admission, and 3 variables regarding underlying diseases (9). Predictive value of this model has been confirmed in different clinical conditions (10-12). These 2 models have been compared in different studies that have yielded somehow contradicting results (13-15). Therefore, the present study was designed aiming to evaluate and compare the values of APACHE II and SAPS II models in prediction of 1-month mortality of critically ill patients presented to emergency department (ED).
2
Figure 1: Mean score± standard error of APACHE II and SAPS II in alive and dead patients (p < 0.001).
2. Methods 2.1. Study design and settings The present prospective cross sectional study was performed on critically ill patients admitted to Imam Khomeini Hospital, Sari, Iran, during February to June 2015 and assessedthe accuracy of APACHE II and SAPS II in prediction of in hospital mortality. Ethics committee of Mazandaran University of Medical Sciences approved the protocol of the study. Informed consent was taken from patients. The researchers adhered to principles of Helsinki Deceleration.
2.2. Participants Critically ill patients were diagnosed based on appearance of patients, neurological assessment, respiratory status, cardiovascular assessment at time of admission to ED (Panel 1) (16) and were enrolled using convenience sampling. Participants lost to follow-up were excluded. Age, gender, diagnosis impression, underlying diseases, vital signs, Glasgow Coma Scale (GCS), urinary output, need for ventilator, andlength ofintensive care unit (ICU) and hospital stay of all participantswere gathered using a pre-designed checklist. Moreover, laboratory data including cell blood count (CBC), hematocrit, sodium, potassium, creatinine, bilirubin, and arterial blood gas analysis (pH, bicarbonate level, and oxygen and carbon dioxide pressure) were measured and recorded. APACHE II and SAPS II scores were calculated during the first 24 hours after admission based on detailed method of calculations presented in previous studies (7, 17). 30-day mortality rate was assessed using patient’s medical records and calling them by the phone. Finally, patients were classified as alive and dead.
2.3. Statistical analysis The number of samples was calculated to be 82 patient’s based on a 50% prevalence of mortality in critically ill patients (18-20), considering a confidence interval (CI) of 95%
Figure 2: Receiver operating characteristic (ROC) curve of SAPS II and APACHE II in mortality prediction (p = 0.24).
(α = 0.05), and a power of 80% (β = 0.2). STATA software version 11.0 was used for data analysis. Qualitative variables are presented as frequency and percentage and quantitative factors are presented as mean and standard deviation. MannWhitney U testand Fisher’s exact testwere used for comparisons. Validations of the models were assessed using discriminatory powerestimation, calibration of predictive models, or a combination of the two. The discriminatory power was evaluated through calculating area under the receiver operating characteristic (ROC) curve (AUC) with 95% CI. General calibration of the model was also evaluated through drawing a calibration plot. In this plot, the perfect calibration is the reference line with an intercept of zero and a slope of 1. The overall performance was eventually assessed via Brier score in order to evaluate predictive accuracy and reliability of the model. P value < 0.05 was considered statistically significant.
3. Results 82 critically ill patients with the mean age of 53.45 ± 20.37 yearswere included (65.9% male). There were no cases of loss
This open-access article distributed under the terms of the Creative Commons Attribution NonCommercial 3.0 License (CC BY-NC 3.0). Downloaded from: www.jemerg.com
3
Emergency. 2017; 5 (1): e4
Panel 1: Diagnostic criteria of critically ill patients Appearance Neurological Respiratory Cardiovascular Gray skin Unresponsive Silent chest PR< 50 Blue skin Eyes open to pain only RR < 8 or > 30 PR > 150 Mottled skin Fitting Agonal respiration SPB< 60 RR: respiratory rateper minute;PR: pulse rate per minute, SBP: systolic blood pressure (mmHg).
Table 1: Baseline characteristics of participants based on their outcome
Factor Alive Age (year) 45.90±20.78 Gender Male 29 (69.05) Female 13 (39.95) Reason of hospitalization Medical 24 (57.14) Surgical (emergent) 13 (30.95) Surgical (Elective) 5 (11.90) Underlying disease None 31 (73.81) Acute renal failure 0 (0.0) Carcinoma 2 (4.76) Metastasis 2 (4.76) Systemic weakness 3 (7.14) Other 4 (9.52) Reason of ICU admission Cardiovascular 0 (0.0) Infection 12 (28.57) Respiratory 5 (11.90) Neurologic 5 (11.90) Multiple trauma 10 (23.81) Head trauma 8 (19.05) Other 2 (4.76) Length of ICU stay (day) 3.38 ± 3.01 Data are presented as mean ± standard deviation or number (%).
Death 61.38±16.84 25 (62.50) 15 (37.50) 36 (90.0) 4 (10.0) 0 (0.0)
P 53.45±20.38
Total 0.0006
54 (65.58) 28 (34.15)
0.53
60 (73.17) 0.002 0.24 17 (20.73) 5 (5.10)
17 (42.50) 1 (2.50) 8 (20.0) 4 (10.0) 5 (12.50) 5 (12.50)
48 (58.54) 1 (1.22) 10 (12.20) 6 (7.32) 8 (9.76) 9 (10.98)
0.06
2 (5.0) 21 (52.50) 7 (17.50) 3 (7.50) 2 (5.0) 1 (2.50) 4 (10.0) 6.10 ± 3.83
2 (2.44) 33 (40.24) 12 (14.63) 8 (9.76) 12 (14.63) 9 (10.98) 6 (7.32) 4.77 ± 3.70
0.007
0.0003
Table 2: Overall performances of SAPS II and APACHE II Model SAPS II APACHE II
Brier Score 0.201 0.213
Sanders resolution 0.182 0.193
to follow-up. The most common cause of hospitalization was non-surgical (73.17%). Mean length of hospital stay was 4.78 ± 3.69 days and mortality rate was 48% (40 patients). Table 1 shows the baseline characteristics of patients. Age (p = 0.0006), reason of hospitalization (p = 0.002), and reason of ICU admission (p = 0.007) correlated with mortality. Mean SAPS II and APACHE II scores were 42.85 ± 19.67 and 19.69 ± 8.91, respectively. Mean SAPS II (p < 0.0001) and APACHE II (p = 0.0007) scores were significantly higher in dead pa-
Reliability 0.019 0.024
Goodness of fit (%) 86.22 84.51
tients (Figure 1). AUC of SAPS II and APACHE II for prediction of mortality were 0.75 (95% CI: 0.64-0.86) and 0.72 (95% CI: 0.60-0.83), respectively (p = 0.24) (Figure 2). Calibration plots of these two scoring systems were presented in figure 3. The slope and intercept of SAPS II were 1.02 and 0.04, respectively. In addition, these values were 0.92 and 0.09 for APACHE II, respectively. Overall performances of SAPS II and APACHE II are presented in table 2. Brier score of SAPS II and APACHE II were 0.201 and 0.213, respectively. In addition, re-
This open-access article distributed under the terms of the Creative Commons Attribution NonCommercial 3.0 License (CC BY-NC 3.0). Downloaded from: www.jemerg.com
H. Aminiahidashti et al.
liability of 0.019 and 0.024 for SAPS II and APACHE II shows goodness of fit of them in prediction of mortality.
4. Discussion Results of the present study showed that APACHE II and SAPS II models have similar value in prediction of 1-month mortality of the patients. Calibration of the 2 models had some amount of lack of fit. The two models showed partial adherence to the reference line, which indicates that the models are partially perfect in prediction of mortality. Discriminatory power was acceptable for both models. In comparison with the results of the present study, Alizadeh et al. have expressed that APACHE II has higher value in prediction of mortality and disability resulting from intoxication compared to SAPSII (13). Similar findings have been reported by TaghaviGilani et al. (21). However, by paying close attention to the TaghaviGilaniet al. article, we can see that AUC is 0.83 for APACHE II model and 0.78 for SAPS II; this difference does not seem statistically different. Haddadi et al. also revealed the value of these models in patient mortality prediction (11). In contrast, Sungurtekin et al. showed higher value for SAPS II model compared to APACHE II (22). These differences might be due to variations instudypopulation and sample size, duration offollow-up and participant selection criteria. Although the discriminatory powers of both APACHE II and SAPS II models were in an acceptable range, findings show some amount of lack of fit. Therefore, calibration of the mentioned models is not completely perfect. In line with the present study, Beck et al. also displayed the external validation of the mentioned models with a similar pattern but its calibration was imperfect (23). In another study, Khwannimit and Greater also expressed that AUC for APACHE II model in prediction of critically ill patient’s mortality is 0.79, yet the calibration of this model is reported to be poor (24). This might be mainly due to disease etiology and data gathering method not being homogenous (9, 24). Recent studies have shown that data gathering errors have been common, especially regarding patients with high or low APACHEII and GCS scores, and this affects the predictiverole of the mentioned models (25). However, in the present study we tried to minimize data gathering errors by training the resident before initiation of sampling. Possibility of selection bias in this study should not be overlooked since the study was single centric and participant selection was done using convenience sampling. Other limitations of this study include etiology of participant admission not being homogenous. This affected model calibration and led to detection of some amount of lack of fit in the 2 studied models.
4
5. Conclusion The findings of the present study showed that APACHE II and SAPS II had similar value in predicting 1-month mortality of patients. Discriminatory powers of the mentioned models were acceptable but their calibration had some amount of lack of fit, which reveals that APACHE II and SAPS II are partially perfect.
6. Appendix 6.1. Acknowledgements The authors wish to thank all the staff of the emergency department of Imam Khomeini Hospital, Sari, Iran.
6.2. Author contribution All authors passed four criteria for authorship contribution based on recommendations of the International Committee of Medical Journal Editors.
6.3. Conflict of interest None.
6.4. Funding support None.
References 1. Mommsen P, Zeckey C, Andruszkow H, Weidemann J, Fromke C, Puljic P, et al. Comparison of different thoracic trauma scoring systems in regards to prediction of posttraumatic complications and outcome in blunt chest trauma. Journal of surgical research. 2012;176(1):239-47. 2. Shojaee M, Faridaalaee G, Yousefifard M, Yaseri M, Arhami Dolatabadi A, Sabzghabaei A, et al. New scoring system for intra-abdominal injury diagnosis after blunt trauma. Chinese journal of traumatology. 2014;17(1):1924. 3. Forouzanfar MM, Safari S, Niazazari M, Baratloo A, Hashemi B, Hatamabadi HR, et al. Clinical decision rule to prevent unnecessary chest X-ray in patients with blunt multiple traumas. Emergency Medicine Australasia. 2014;26(6):561-6. 4. Hashemi B, Amanat M, Baratloo A, Forouzanfar MM, Rahmati F, Motamedi M, et al. Validation of CRASH Model in Prediction of 14-day Mortality and 6-month Unfavorable Outcome of Head Trauma Patients. EMERGENCY-An Academic Emergency Medicine Journal. 2016;4:[In press]. 5. Staff T, Eken T, Wik L, Roislien J, Sovik S. Physiologic, demographic and mechanistic factors predicting New In-
This open-access article distributed under the terms of the Creative Commons Attribution NonCommercial 3.0 License (CC BY-NC 3.0). Downloaded from: www.jemerg.com
5
Emergency. 2017; 5 (1): e4
Figure 3: Calibration plots of SAPS II (A) and APACHE II (B).
6.
7.
8.
9.
10.
11.
12.
13.
jury Severity Score (NISS) in motor vehicle accident victims. Injury. 2014;45(1):9-15. Chawda M, Hildebrand F, Pape HC, Giannoudis PV. Predicting outcome after multiple trauma: which scoring system Injury. 2004;35(4):347-58. Knaus WA, Draper EA, Wagner DP, Zimmerman JE. APACHE II: a severity of disease classification system. Critical care medicine. 1985;13(10):818-29. Giamarellos-Bourboulis EJ, Norrby-Teglund A, Mylona V, Savva A, Tsangaris I, Dimopoulou I, et al. Risk assessment in sepsis: a new prognostication rule by APACHE II score and serum soluble urokinase plasminogen activator receptor. Critical care (London, England). 2012;16(4):R149. Le Gall JR, Lemeshow S, Saulnier F. A new Simplified Acute Physiology Score (SAPS II) based on a European/North American multicenter study. Jama. 1993;270(24):2957-63. Czorlich P, Sauvigny T, Ricklefs F, Kluge S, Vettorazzi E, Regelsberger J, et al. The simplified acute physiology score II to predict hospital mortality in aneurysmal subarachnoid hemorrhage. Acta neurochirurgica. 2015;157(12):2051-9. Haddadi A, Ledmani M, Gainier M, Hubert H, De Micheaux PL. Comparing the APACHE II, SOFA, LOD, and SAPS II scores in patients who have developed a nosocomial infection. Bangladesh Critical Care Journal. 2014;2(1):4-9. Adamzik M, Langemeier T, Frey UH, Gorlinger K, Saner F, Eggebrecht H, et al. Comparison of thrombelastometry with simplified acute physiology score II and sequential organ failure assessment scores for the prediction of 30day survival: a cohort study. Shock. 2011;35(4):339-42. Alizadeh AM, Hassanian-Moghaddam H, Shadnia S, Za-
14.
15.
16.
17.
18.
19.
mani N, Mehrpour O. Simplified Acute Physiology Score II/Acute Physiology and Chronic Health Evaluation II and Prediction of the Mortality and Later Development of Complications in Poisoned Patients Admitted to Intensive Care Unit. Basic, Clinical Pharmacology, Toxicology. 2014;115(3):297-300. Brinkman S, Bakhshi-Raiez F, Abu-Hanna A, de Jonge E, Bosman RJ, Peelen L, et al. External validation of Acute Physiology and Chronic Health Evaluation IV in Dutch intensive care units and comparison with Acute Physiology and Chronic Health Evaluation II and Simplified Acute Physiology Score II. Journal of Critical Care. 2011;26(1):105.e11-.e18. Seak C-J, Ng C-J, Yen DH-T, Wong Y-C, Hsu K-H, Seak JC-Y, et al. Performance assessment of the Simplified Acute Physiology Score II, the Acute Physiology and Chronic Health Evaluation II score, and the Sequential Organ Failure Assessment score in predicting the outcomes of adult patients with hepatic portal venous gas in the ED. The American Journal of Emergency Medicine. 2014;32(12):1481-4. Frost P, Wise MP. Recognition and early management of the critically ill ward patient. British Journal of Hospital Medicine (2005). 2007;68(Sup10):M180-M3. Le Gall J-R, Lemeshow S, Saulnier F. A new simplified acute physiology score (SAPS II) based on a European/North American multicenter study. Jama. 1993;270(24):2957-63. Arabi Y, Haddad S, Goraj R, Al-Shimemeri A, Al-Malik S. Assessment of performance of four mortality prediction systems in a Saudi Arabian intensive care unit. Critical care. 2002;6(2):166-74. Faruq MO, Mahmud MR, Begum T, Ahsan AA, Fatema K,
This open-access article distributed under the terms of the Creative Commons Attribution NonCommercial 3.0 License (CC BY-NC 3.0). Downloaded from: www.jemerg.com
H. Aminiahidashti et al.
Ahmed F, et al. A comparison of severity systems APACHE II and SAPS II in critically ill patients. Bangladesh Critical Care Journal. 2013;1(1):27-32. 20. Asadzandi M, Taghizade K, Tadrisi S, Ebadi A. Estimation of the mortality rate using the APACHE II standard disease severity scoring system in intensive care unit patients. Iranian Journal of Critical Care Nursing (IJCCN). 2012;4(4):209-14. 21. Gilani MT, Razavi M, Azad AM. A comparison of Simplified Acute Physiology Score II, Acute Physiology and Chronic Health Evaluation II and Acute Physiology and Chronic Health Evaluation III scoring system in predicting mortality and length of stay at surgical intensive care unit. Nigerian Medical Journal : Journal of the Nigeria Medical Association. 2014;55(2):144-7. 22. Sungurtekin H, Gurses E, Balci C. Evaluation of several clinical scoring tools in organophosphate poi-
6
soned patients. Clinical toxicology (Philadelphia, Pa). 2006;44(2):121-6. 23. Beck DH, Smith GB, Pappachan JV, Millar B. External validation of the SAPS II, APACHE II and APACHE III prognostic models in South England: a multicentre study. Intensive care medicine. 2003;29(2):249-56. 24. Khwannimit B, Geater A. A comparison of APACHE II and SAPS II scoring systems in predicting hospital mortality in Thai adult intensive care units. Journal of the Medical Association of Thailand = Chotmaihet thangphaet. 2007;90(4):643-52. 25. HOLT AW, BURY LK, BERSTEN AD, SKOWRONSKI GA, VEDIG AE. Prospective evaluation of residents and nurses as severity score data collectors. Critical care medicine. 1992;20(12):1688-91.
This open-access article distributed under the terms of the Creative Commons Attribution NonCommercial 3.0 License (CC BY-NC 3.0). Downloaded from: www.jemerg.com
