ORIGINAL ARTICLE
Clinical Decision Rule to Identify Orbital Wall Fracture Among Children Retrospective Derivation and Validation Study So Hyun Paek, MD,* Jin Hee Jung, MD,† Young Ho Kwak, PhD,* Do Kyun Kim, PhD,* Jin Hee Lee, MD,‡ Jae Yun Jung, MD,* and Sohee Oh, PhD§
Objectives: Head and face injuries are leading causes of emergency department visits in children. There is yet no clinical decision rule on face CT such as pediatric head CT rules. The goal was to develop and validate a clinical decision rule for identifying orbital wall fractures in children with periorbital trauma in the emergency department. Methods: This was a retrospective derivation and validation study. Children younger than 18 years who underwent orbit CT after periorbital trauma were included between January 2011 and December 2013 in 3 emergency centers. Among 16 candidate clinical variables, 13 clinical signs and symptoms were selected as clinical predictors. For the fracture model, these clinical predictors were analyzed by 3-fold cross-validation. Diagnostic performance was assessed using the area under the receiver operating characteristic (AUROC) curve in both cohorts. Results: Four variables (orbital rim tenderness, periorbital ecchymosis, painful extraocular movement, and nausea/vomiting) had the best predictive model with the highest AUROC value. The AUROC values for fracture prediction were 0.793 (95% confidence interval, 0.741–0.844) and 0.809 (95% confidence interval, 0.742–0.877) in the derivation cohort and validation cohort, respectively. The sensitivity and negative predictive values were 96.4% and 93.4%, respectively, in the derivation cohort, and 97.8% and 98.1%, respectively, in the validation cohort. The sum of these scores ranged from 0 to 4. Patients with a sum of scores of 1 or higher showed significantly increased risk for fracture. Conclusions: The 4-variable predictive model can be useful for finding clinically important orbital wall fractures in children. Key Words: adolescent, decision support technique, orbital fractures (Pediatr Emer Care 2017;00: 00–00)
O
rbital wall fracture is a common result of facial injuries. It can be associated with enophthalmos, diplopia, and infraorbital nerve hyperesthesia.1 Facial computed tomography (CT) is the technique of choice to evaluate suspicious orbital wall injuries.2–4 According to national data in the United States, the rate of orbital fracture was only 12.5% in patients who underwent CT imaging after eye and face injuries in emergency departments (EDs).5 In addition to the low yield of CT in identifying orbit
From the *Department of Emergency Medicines, Seoul National University Hospital, Seoul; †Department of Emergency Medicines, SMG-SNU Boramae Medical Center, and Department of Preventive Medicine, Kangwon National University, School of Medicine, Chuncheon; ‡Department of Emergency Medicines, Seoul National University Bundang Hospital, Seongnam; and §Department of Biostatistics, SMG-SNU Boramae Medical Center, Seoul, Republic of Korea. Disclosure: The authors declare no conflict of interest. Reprints: Jin Hee Jung, MD, Department of Emergency Medicines, SMG-SNU Boramae Medical Center, 20 Boramae-ro 5-gil, Dongjak-gu, Seoul 07061, Korea (e‐mail: [email protected]). The abstract has been presented at seminars in the American College of Emergency Physicians in 2015 at Boston and the Korean Society of Emergency Medicine in 2015. Copyright © 2017 Wolters Kluwer Health, Inc. All rights reserved. ISSN: 0749-5161
Pediatric Emergency Care • Volume 00, Number 00, Month 2017
fractures, clinical decision rules such as the head CT rule have been developed and implemented to reduce radiation hazard.5–8 Although a derivation study of clinical risk score for adult orbital fracture has shown good performance, the result has not yet been validated in other hospital settings for the pediatric population.5,9 Because the need for 3-dimensional facial CT is increasing to have more accurate evaluation for orbital fracture and surgery planning, significant radiation exposure might occur in the oropharynx, thyroid, and brain organs. The increase in CT scans nowadays is more pronounced in adolescents.10 Facial bone CT scan examinations increased the most compared with other types of CT scans performed in the EDs of Korean hospitals.11 To reduce unnecessary use of CT imaging to evaluate probable pediatric orbital wall fracture, a clinical decision rule can be useful. However, no such study has been performed for pediatric patients. Therefore, the objective of this study was to develop and validate a clinical decision rule to identify orbital wall fractures in children with orbital area trauma.
METHODS Study Design and Setting This study was a retrospective derivation and validation study in 3 urban EDs. The study was performed for a period of 3 years from January 2011 to December 2013. Derivation study was performed using data from 2 hospitals (Seoul National University Hospital and Seoul National University Bundang Hospital). The 2 hospitals are tertiary teaching hospitals with 70,000 and 85,000 annual ED visits, respectively. Validation study was performed in Seoul Metropolitan Government Seoul National University Medical Center, a public and teaching hospital with approximately 50,000 ED visits per year. This study was approved by the institutional review board of SMG-SNU Boramae Medical Center (IRB no. 16-2015-92).
Selection of Participants Participants were children aged 18 years or younger with orbital area trauma who underwent CT imaging of the orbits, or facial bones, in the ED. Patients were excluded from the study if they had a diagnosis besides orbital area trauma or if they were younger than 5 years (Fig. 1). Orbital area trauma was defined when a diagnosis with an International statistical classification of diseases and related health problems 10th revision (S00.1, S00.2, S01.1, S02.1, S02.3, S02.7, S02.8, S05.x) was made. Orbital wall fractures were confirmed by CT scan in consultation with an ophthalmologist.
Measurements Two researchers reviewed the electronic medical records (EMR) from the 3 hospitals. Demographic and clinical data (age, sex, mechanism of injury, diagnosis of fracture, performed www.pec-online.com
Copyright © 2017 Wolters Kluwer Health, Inc. Unauthorized reproduction of this article is prohibited.
1
Pediatric Emergency Care • Volume 00, Number 00, Month 2017
Paek et al
FIGURE 1. Flow diagram of the study process.
operation) were analyzed. In addition, 16 clinically important signs and symptoms were obtained from the electronic charts of orbital wall fracture cases. Clinically important signs were decided through 2 steps. In the first step, 15 clinically important signs and symptoms were extracted from the previous adult study.9 Because nausea and vomiting were found to be common and important symptoms of white-eyed blowout fractures in children based on the literature, nausea or vomiting was added as a clinical predictor.12,13 These 16 variables were evaluated by 2 researchers and coded in preformed case report form. In the second step, 3 variables (trismus, jaw locking, and orbital rim stepoff signs) were excluded because no participants had these variables in EMR. Finally, 13 clinical signs and symptoms were selected as clinical predictors.
Data Analysis Demographics of participants were compared in both cohorts. Univariate analyses were conducted to identify which variables among 16 predictors (age, sex, mechanism of injury, and 13 clinical signs and symptoms) were significantly different between the fracture group and no fracture group in derivation cohorts. Variables with a P value of less than 0.2 were chosen for model building. To select the best combination of variables for predicting orbital wall fracture, 3-fold cross-validation (CV) was performed. Cross-validation is a statistical method for estimating and evaluating the performance of models (each variable combination) to avoid overfitting problem. In 3-fold CV, the original data set was randomly partitioned into 3 subsets. Of the 3 subsets, 1 subset was retained as the internal validation data for testing the model whereas the remaining subsets were used as training data. The CV process was repeated 3 times with each of the 3 subsets used exactly once as validation data. To reduce the possibility of poor estimates because of chance divisions of data set, the 3-fold CV was repeated 1000 times. To select the best predictor combination, all possible models were fit in the training set. The predictability of each model was then evaluated in the test set. The predictability of each model was assessed by the average-test area under the
2
www.pec-online.com
receiver operating characteristic (AUROC) curve. To choose the best model for fracture, AUROCs were compared by the Delong test. Using the finally selected model, the predicted probability for orbital wall fracture was estimated according to the number of combinations of variables. An optimal cutoff point for fracture was calculated based on maximum Youden index (J = sensitivity + specificity-1). Sensitivity, specificity, positive predictive value, and negative predicted value (NPV) of the clinical decision rule for orbital fracture were computed for both the derivation cohort and validation cohort. All statistical analyses were conducted using IBM SPSS Statistics (Version 20, IBM Corp, Armonk, NY) and R version 3.1.0 (http://www.r-project.org). Demographic and clinical characteristics between derivation cohort and validation cohort were examined using a 2- t sample test, χ2 test, or Fisher exact test. A P value of 0.05 was considered statistically significant. Sample size was estimated using the “rule of ten” based on findings drawn from empiric simulation with logistic regression.14 We estimated that a total of 120 patients with orbital fractures were required to avoid overfitting the multivariable model. Because derivation cohort hospital EDs treated approximately 40 patients with orbital fracture annually, we expected to collect sufficient samples during the 3-year study period.
RESULTS Characteristics of Study Subjects and 2 Cohorts Of a total of 40,129 injured children and adolescents assessed for eligibility, 18,748 patients (46.7%) had head and facial trauma in the ED. The number of eligible patients who had orbital area trauma was 930 (5%), of which 544 (58.4%) underwent CT scans. There were 240 patients in the deviation cohort and 269 patients in the validation cohort (Fig. 1). Clinical characteristics of the study population were compared between the derivation cohort and the validation cohort (Table 1). There were statistically significant differences in sex, mechanism of injury, rate of fracture, and operation © 2017 Wolters Kluwer Health, Inc. All rights reserved.
Copyright © 2017 Wolters Kluwer Health, Inc. Unauthorized reproduction of this article is prohibited.
Pediatric Emergency Care • Volume 00, Number 00, Month 2017
TABLE 1. Demographics of Participants (aged 5 to 18 years) in Derivation and Validation Cohorts Derivation Validation Cohort N = 240 Cohort N = 269
Characteristics Age, y, mean (SD) Sex, male, N (%) Study site, N (%) Hospital A SNUH Hospital B BRM Hospital C SNUBH Mechanism of injury, N (%) Fall down Slip down Collision Assault Sports injury Traffic accident Fracture, n (%) Operation, n (%)
13.7 ± 3.3 210 (87.5)
14.2 ± 3.5 205 (76.2)
P 0.097
Clinical Decision Rule to Identify Orbital Wall Fracture Among Children Retrospective Derivation and Validation Study So Hyun Paek, MD,* Jin Hee Jung, MD,† Young Ho Kwak, PhD,* Do Kyun Kim, PhD,* Jin Hee Lee, MD,‡ Jae Yun Jung, MD,* and Sohee Oh, PhD§
Objectives: Head and face injuries are leading causes of emergency department visits in children. There is yet no clinical decision rule on face CT such as pediatric head CT rules. The goal was to develop and validate a clinical decision rule for identifying orbital wall fractures in children with periorbital trauma in the emergency department. Methods: This was a retrospective derivation and validation study. Children younger than 18 years who underwent orbit CT after periorbital trauma were included between January 2011 and December 2013 in 3 emergency centers. Among 16 candidate clinical variables, 13 clinical signs and symptoms were selected as clinical predictors. For the fracture model, these clinical predictors were analyzed by 3-fold cross-validation. Diagnostic performance was assessed using the area under the receiver operating characteristic (AUROC) curve in both cohorts. Results: Four variables (orbital rim tenderness, periorbital ecchymosis, painful extraocular movement, and nausea/vomiting) had the best predictive model with the highest AUROC value. The AUROC values for fracture prediction were 0.793 (95% confidence interval, 0.741–0.844) and 0.809 (95% confidence interval, 0.742–0.877) in the derivation cohort and validation cohort, respectively. The sensitivity and negative predictive values were 96.4% and 93.4%, respectively, in the derivation cohort, and 97.8% and 98.1%, respectively, in the validation cohort. The sum of these scores ranged from 0 to 4. Patients with a sum of scores of 1 or higher showed significantly increased risk for fracture. Conclusions: The 4-variable predictive model can be useful for finding clinically important orbital wall fractures in children. Key Words: adolescent, decision support technique, orbital fractures (Pediatr Emer Care 2017;00: 00–00)
O
rbital wall fracture is a common result of facial injuries. It can be associated with enophthalmos, diplopia, and infraorbital nerve hyperesthesia.1 Facial computed tomography (CT) is the technique of choice to evaluate suspicious orbital wall injuries.2–4 According to national data in the United States, the rate of orbital fracture was only 12.5% in patients who underwent CT imaging after eye and face injuries in emergency departments (EDs).5 In addition to the low yield of CT in identifying orbit
From the *Department of Emergency Medicines, Seoul National University Hospital, Seoul; †Department of Emergency Medicines, SMG-SNU Boramae Medical Center, and Department of Preventive Medicine, Kangwon National University, School of Medicine, Chuncheon; ‡Department of Emergency Medicines, Seoul National University Bundang Hospital, Seongnam; and §Department of Biostatistics, SMG-SNU Boramae Medical Center, Seoul, Republic of Korea. Disclosure: The authors declare no conflict of interest. Reprints: Jin Hee Jung, MD, Department of Emergency Medicines, SMG-SNU Boramae Medical Center, 20 Boramae-ro 5-gil, Dongjak-gu, Seoul 07061, Korea (e‐mail: [email protected]). The abstract has been presented at seminars in the American College of Emergency Physicians in 2015 at Boston and the Korean Society of Emergency Medicine in 2015. Copyright © 2017 Wolters Kluwer Health, Inc. All rights reserved. ISSN: 0749-5161
Pediatric Emergency Care • Volume 00, Number 00, Month 2017
fractures, clinical decision rules such as the head CT rule have been developed and implemented to reduce radiation hazard.5–8 Although a derivation study of clinical risk score for adult orbital fracture has shown good performance, the result has not yet been validated in other hospital settings for the pediatric population.5,9 Because the need for 3-dimensional facial CT is increasing to have more accurate evaluation for orbital fracture and surgery planning, significant radiation exposure might occur in the oropharynx, thyroid, and brain organs. The increase in CT scans nowadays is more pronounced in adolescents.10 Facial bone CT scan examinations increased the most compared with other types of CT scans performed in the EDs of Korean hospitals.11 To reduce unnecessary use of CT imaging to evaluate probable pediatric orbital wall fracture, a clinical decision rule can be useful. However, no such study has been performed for pediatric patients. Therefore, the objective of this study was to develop and validate a clinical decision rule to identify orbital wall fractures in children with orbital area trauma.
METHODS Study Design and Setting This study was a retrospective derivation and validation study in 3 urban EDs. The study was performed for a period of 3 years from January 2011 to December 2013. Derivation study was performed using data from 2 hospitals (Seoul National University Hospital and Seoul National University Bundang Hospital). The 2 hospitals are tertiary teaching hospitals with 70,000 and 85,000 annual ED visits, respectively. Validation study was performed in Seoul Metropolitan Government Seoul National University Medical Center, a public and teaching hospital with approximately 50,000 ED visits per year. This study was approved by the institutional review board of SMG-SNU Boramae Medical Center (IRB no. 16-2015-92).
Selection of Participants Participants were children aged 18 years or younger with orbital area trauma who underwent CT imaging of the orbits, or facial bones, in the ED. Patients were excluded from the study if they had a diagnosis besides orbital area trauma or if they were younger than 5 years (Fig. 1). Orbital area trauma was defined when a diagnosis with an International statistical classification of diseases and related health problems 10th revision (S00.1, S00.2, S01.1, S02.1, S02.3, S02.7, S02.8, S05.x) was made. Orbital wall fractures were confirmed by CT scan in consultation with an ophthalmologist.
Measurements Two researchers reviewed the electronic medical records (EMR) from the 3 hospitals. Demographic and clinical data (age, sex, mechanism of injury, diagnosis of fracture, performed www.pec-online.com
Copyright © 2017 Wolters Kluwer Health, Inc. Unauthorized reproduction of this article is prohibited.
1
Pediatric Emergency Care • Volume 00, Number 00, Month 2017
Paek et al
FIGURE 1. Flow diagram of the study process.
operation) were analyzed. In addition, 16 clinically important signs and symptoms were obtained from the electronic charts of orbital wall fracture cases. Clinically important signs were decided through 2 steps. In the first step, 15 clinically important signs and symptoms were extracted from the previous adult study.9 Because nausea and vomiting were found to be common and important symptoms of white-eyed blowout fractures in children based on the literature, nausea or vomiting was added as a clinical predictor.12,13 These 16 variables were evaluated by 2 researchers and coded in preformed case report form. In the second step, 3 variables (trismus, jaw locking, and orbital rim stepoff signs) were excluded because no participants had these variables in EMR. Finally, 13 clinical signs and symptoms were selected as clinical predictors.
Data Analysis Demographics of participants were compared in both cohorts. Univariate analyses were conducted to identify which variables among 16 predictors (age, sex, mechanism of injury, and 13 clinical signs and symptoms) were significantly different between the fracture group and no fracture group in derivation cohorts. Variables with a P value of less than 0.2 were chosen for model building. To select the best combination of variables for predicting orbital wall fracture, 3-fold cross-validation (CV) was performed. Cross-validation is a statistical method for estimating and evaluating the performance of models (each variable combination) to avoid overfitting problem. In 3-fold CV, the original data set was randomly partitioned into 3 subsets. Of the 3 subsets, 1 subset was retained as the internal validation data for testing the model whereas the remaining subsets were used as training data. The CV process was repeated 3 times with each of the 3 subsets used exactly once as validation data. To reduce the possibility of poor estimates because of chance divisions of data set, the 3-fold CV was repeated 1000 times. To select the best predictor combination, all possible models were fit in the training set. The predictability of each model was then evaluated in the test set. The predictability of each model was assessed by the average-test area under the
2
www.pec-online.com
receiver operating characteristic (AUROC) curve. To choose the best model for fracture, AUROCs were compared by the Delong test. Using the finally selected model, the predicted probability for orbital wall fracture was estimated according to the number of combinations of variables. An optimal cutoff point for fracture was calculated based on maximum Youden index (J = sensitivity + specificity-1). Sensitivity, specificity, positive predictive value, and negative predicted value (NPV) of the clinical decision rule for orbital fracture were computed for both the derivation cohort and validation cohort. All statistical analyses were conducted using IBM SPSS Statistics (Version 20, IBM Corp, Armonk, NY) and R version 3.1.0 (http://www.r-project.org). Demographic and clinical characteristics between derivation cohort and validation cohort were examined using a 2- t sample test, χ2 test, or Fisher exact test. A P value of 0.05 was considered statistically significant. Sample size was estimated using the “rule of ten” based on findings drawn from empiric simulation with logistic regression.14 We estimated that a total of 120 patients with orbital fractures were required to avoid overfitting the multivariable model. Because derivation cohort hospital EDs treated approximately 40 patients with orbital fracture annually, we expected to collect sufficient samples during the 3-year study period.
RESULTS Characteristics of Study Subjects and 2 Cohorts Of a total of 40,129 injured children and adolescents assessed for eligibility, 18,748 patients (46.7%) had head and facial trauma in the ED. The number of eligible patients who had orbital area trauma was 930 (5%), of which 544 (58.4%) underwent CT scans. There were 240 patients in the deviation cohort and 269 patients in the validation cohort (Fig. 1). Clinical characteristics of the study population were compared between the derivation cohort and the validation cohort (Table 1). There were statistically significant differences in sex, mechanism of injury, rate of fracture, and operation © 2017 Wolters Kluwer Health, Inc. All rights reserved.
Copyright © 2017 Wolters Kluwer Health, Inc. Unauthorized reproduction of this article is prohibited.
Pediatric Emergency Care • Volume 00, Number 00, Month 2017
TABLE 1. Demographics of Participants (aged 5 to 18 years) in Derivation and Validation Cohorts Derivation Validation Cohort N = 240 Cohort N = 269
Characteristics Age, y, mean (SD) Sex, male, N (%) Study site, N (%) Hospital A SNUH Hospital B BRM Hospital C SNUBH Mechanism of injury, N (%) Fall down Slip down Collision Assault Sports injury Traffic accident Fracture, n (%) Operation, n (%)
13.7 ± 3.3 210 (87.5)
14.2 ± 3.5 205 (76.2)
P 0.097
