IT IN THE ED Associate Editors: Marvin B. Harper, MD, Jeffrey M. Hoffman, MD, Joseph J. Zorc, MD, MSCE
Electronic Prescription Writing Errors in the Pediatric Emergency Department Courtney E. Nelson, MD*† and Steven M. Selbst, MD, FAAP, FACEP*† Key Words: electronic health record, medication error, medical education, residents (Pediatr Emer Care 2015;31: 368–372)
M
edication errors in the pediatric setting are common. The error rates range from 4% to 30% of all medication orders.1 These errors include the wrong dose, route, frequency, and medication choice, and can lead to an adverse drug event (ADE) in up to 16% of pediatric patients.2 The ADEs can require emergency medical care and hospital admission.3 The Food and Drug Administration reported that ADEs (injuries, disabilities, and death) tripled between 1998 and 2005.4 Medication errors occur in the emergency department as well. In 1 study at a pediatric tertiary care hospital, medication errors occurred in 10% of orders placed in the emergency department.5 Medication errors among ambulatory prescriptions written in the pediatric emergency department (PED) are also common, ranging from 4.3% to 59%.6,7 In these studies, the majority of errors are due to incorrectly writing the prescription, including minor omissions or incomplete directions, and not due to incorrect dosing. Physicians with numerous different training backgrounds, many of which are not primarily pediatric-based, work in academic PEDs. Some studies show that trainees have a higher error rate than attendings and that level of training is directly related to error rate.5 In terms of resident specialty, research shows that emergency medicine residents need further training in pediatric medication dosing.8 The electronic health record (EHR) and computerized physician order entry (CPOE) have the potential to improve prescription error rates. The EHR and CPOE software helps to prevent medication errors by connecting to either an online formulary, which provides recommended dosing and lists the side effects of a medication, or with a formal clinical decision support system (CDSS). The CDSS links the formulary to the patient's chart and will provide alerts, for example, if drug dose is in excess for age or weight or if there is an allergy contraindication to a medication.9,10 The Institute of Medicine recently endorsed CPOE software as a necessity to reduce medication error rates.11 A recent review found that the introduction of CPOE in the emergency department decreased medication error rates.12 More specifically, a second study found that electronic prescribing systems reduced error in a PED.13 The role of specialty and level of training on electronic prescription errors, however, has yet to be addressed in the literature. The purpose of this study is to evaluate the frequency, type, and severity of electronic prescription errors written by residents in From the *Jefferson Medical College, Philadelphia PA; and †Nemours/AI duPont Hospital for Children, Wilmington, DE. Disclosure: The authors declare no conflict of interest. Reprints: Courtney E. Nelson, MD, Department of Pediatrics, AI duPont Hospital for Children, 1600 Rockland Road, Wilmington, DE 19803 (e‐mail: [email protected]). Copyright © 2015 Wolters Kluwer Health, Inc. All rights reserved. ISSN: 0749-5161
368
www.pec-online.com
an academic PED and to determine if there is a relationship with error rate and resident training level or specialty. We hypothesize that with CPOE, error rates will be uniform among resident level of training and specialty.
METHODS We conducted a retrospective chart review of a random selection of electronic prescriptions written in an academic, suburban, tertiary care PED. The department has an annual census of 51,000 patients. Pediatric emergency medicine board-certified physicians, resident physicians training in several specialties, physician assistants, and urgent care pediatric board-certified physicians staff the emergency department. The department uses EPIC (Epic, Verona, WI) software as its EHR, CDSS, and CPOE. The Epic CDSS consists of radio buttons with the most common doses of each medication, as well as a location to free text the desired dose. In addition, with ambulatory prescriptions only, one has the option to free text the entire prescription. There is a hyperlink to the online formulary Lexicomp (Wolters Kluwer Health, Philadelphia, PA) for each medication ordered. When a dose is selected, the dose is calculated based on the electronically documented patient weight. Medications cannot be dispensed without a documented weight. Doses above or below the recommended dose trigger an alert window that may be closed and the order placed without further warning. On the contrary, all drug interactions and pregnancy category D and X drugs generate an alert window that can only be circumvented when a reason is selected. At the time of this study, each electronically written prescription was printed and attached to the discharge paperwork. The attending was not required to review each prescription. The hospital's institutional review board approved this study. We initially selected all medication prescriptions written electronically by residents between July 1, 2011, and June 30, 2012. Prescriptions ordered by attending physicians, fellows, and physician assistants were excluded. Narcotic prescriptions were excluded because these are generally reviewed by the attending physician in the emergency department and would not accurately represent resident prescription writing. Likewise, prescriptions for devices were also excluded. Finally, if a patient was admitted to the hospital, his or her prescription was excluded from the study as it was assumed he or she would not fill the prescription and may acquire a new prescription prior to discharge. We selected every sixtieth prescription for a goal of 350 patient prescriptions based on an a priori power analysis, which is described in more detail later in the text. From each prescription and electronic medical record, we extracted the patient medical record number, visit date, prescription date, patient allergies, weight, and age, medication name, dose, strength, route, quantity, and refills. In addition, the physician's name, year of training, and specialty were documented. Physician year of training was labeled as first, second, or third year resident. We considered third, fourth, and fifth year residents all “third year” or “senior residents” because only the combined residency programs extend to 4th and 5th years of training. Pediatric Emergency Care • Volume 31, Number 5, May 2015
Copyright © 2015 Wolters Kluwer Health, Inc. All rights reserved.
Pediatric Emergency Care • Volume 31, Number 5, May 2015
Electronic Prescription Writing Errors
TABLE 1. Error Categories, Adapted From a Prior Publication6 Error Categories Major omission Minor omission
Dose/directions Quantity Incomplete directions Allergy conflict
Missing drug name, strength (only if more than one available), or quantity to dispense (unless can calculate from signa). Major omission would require the pharmacist to contact the prescriber. Missing date, directions for use (dose, frequency, and route), physician signature, or refills not specified. Minor omission may be filled by the pharmacist based on professional judgment and/or additional information gathered from the parent or prescriber. Dosage or direction (including duration) that differ substantially from normal standards. Dosage form unavailable, inappropriate, or in excess of manufacture or literature recommendations and those inadequate to achieve desired therapeutic effect. Unclear quantity, such as 1 bottle or tube when more than one is available (over-the-counter medications not included). Omissions as above. Signa lacks specific directions such as indications for prn usage, solely states “as directed,” or no duration of therapy for non-routine medication An error if allergy status not documented on chart or a medication prescribed to which the patient has a listed allergy without documentation explaining the variance.
Both authors independently reviewed each prescription and corresponding medical record for errors. Errors were classified into a specific category and severity based on previously published classifications (Tables 1 and 2).6 The authors then reviewed emergency department discharge instructions for clarification of errors. If an error of omission or incomplete direction was rectified in the discharge instructions, that error was excluded from the data set. When there was more than 1 error, both were included in the analysis. Medication dose and directions were crossreferenced using Lexicomp. An incorrect dose was determined as 20% above or below recommended dosing, as used in similar studies.5 Patients were considered eligible for adult dosage at age 17 or at a weight of 70 kg. Medication dosing was considered correct despite reference values when it correlated directly with consultant recommendations. Likewise, a dose more than 20% below the standard was considered correct if it was administered during the emergency department visit and was documented as efficacious. If the volume of a medication dispensed was less than that needed for a treatment course, it was considered an error. However, if the volume dispensed was in excess, it was not an error if the written discharge instructions clarified the dose and course of treatment. Both authors met throughout the study period to discuss error classification and when there were discrepancies, a consensus was reached. The authors first conducted an a priori power analysis to determine the appropriate sample size for our study. Our primary outcome was prescription error as a dichotomous variable with
respect to level of training. Because pediatric and emergency medicine residency spans 3 years, we assumed degrees of freedom of 2. To detect a medium effect size (w = 0.30) with a type I error rate of 0.05 with a power of 0.80, we would need to enroll a total sample size of 108.14 We used descriptive statistics to analyze error type and severity with respect to level of training and specialty. We analyzed the data using Pearson χ2 test and Fisher exact test when expected outcomes were less than five. A P value of less than 0.05 was defined as significant.
RESULTS During the study period, residents in the emergency department wrote 23,640 prescriptions electronically. We selected every 60th prescription leaving 394 prescriptions. Of these 350 met our inclusion criteria. The most common type of medication prescribed was an antibiotic (Table 3). The prescribers included 141 different residents from 7 different specialties. Pediatric residents wrote 49% of the prescriptions, and nonpediatric residents wrote 51%. Among nonpediatric residents, emergency medicine residents wrote the most prescriptions. First year residents wrote the most prescriptions and second year residents the least. Sixteen percent of electronic prescriptions had at least 1 error and 4 had more than 1 error. Fourteen errors were corrected in the discharge instructions, leaving 46 errors (Table 3). Errors were categorized as shown in Figure 1. Incomplete directions and dose/direction errors were the most common error types; both
TABLE 2. Severity Categories; Adapted From a Prior Publication6 Severity Categories Severe
Medication with a low therapeutic index written for more than 10 times the normal dose, or the potential result of a pharmacological effect or serum concentration associated with severe or fatal toxic reactions. The drug had the potential to produce a severe life-threatening reaction in the patient (ie anaphylaxis). Dose too low for a lifesaving drug or one used for a severe illness. Serious Medication with a low therapeutic index, 4 to 10 times the normal dose, or potential for a serious toxic reaction. Dose too low for a serious illness. Wrong medication or route prescribed with potential for serious reaction or inadequate treatment of a serious illness. Significant Medication with a low therapeutic index, 1.5 to 4 times the normal dose. Dose of any medication more than 5 times the normal dose, or any degree greater than normal with potential for an adverse effect related to the dose. Dose or duration inadequate for therapeutic effect. Wrong medication for a non-severe illness or wrong route with potential for adverse/side effects or treatment failure. Duplicate therapy with potential of additive toxic reactions Problem Duplicate or without potential for increased side effects. Lacks specific drug, dose, strength, formulation, route, or frequency. Wrong route without potential for toxic reaction or treatment failure. Dose 5 times normal without potential for toxic reaction. Prescription unlikely to be filled due to nature of or missing information. Insignificant Prescriptions assigned an error by the described categories that do not meet the above criteria or are likely to be filled and without the potential for significant side effects or treatment failure.
© 2015 Wolters Kluwer Health, Inc. All rights reserved.
Copyright © 2015 Wolters Kluwer Health, Inc. All rights reserved.
www.pec-online.com
369
Pediatric Emergency Care • Volume 31, Number 5, May 2015
Nelson and Selbst
represented 37% of prescription errors. Among incomplete directions, physicians most often forgot to list the duration of therapy. The least common error type was a minor omission, and most of these errors involved failure to list a route of administration. There were no major omissions or allergy conflicts. Error severity was classified as severe, serious, significant, problematic, or insignificant based on previously published classifications (Fig. 2).6 In terms of severity, the majority of errors were insignificant (61%). Seventeen percent of errors were problematic, and another 17% of errors were significant. The majority of problematic errors were due to minor omissions of route and most significant errors were due to incorrect dose or directions for a medication. There were 2 serious errors, both of which occurred when a resident prescribed and dispensed only one dose of albuterol to a child who presented in respiratory distress. When error rate was analyzed using residency level of training, there was no statistically significant difference in error between residents within each level of training and all residents (P = 0.42) (Table 4). Similarly, clinically significant errors or TABLE 3. Prescription Characteristics Prescriptions Total Random sample Excluded Included Errors Corrected in d/c instructions Remaining prescriptions with error Level of training of prescriber First year Second year Third year* Subspecialty of prescriber Pediatric resident Subspecialty resident Emergency medicine Family medicine Medicine-pediatrics Emergency medicine/internal medicine Emergency medicine/family medicine Physical medicine and rehabilitation Class of drug prescribed Antibiotic Laxative Analgesic Steroid Albuterol Antiemetic Antihistamine Antacid Antiepileptic Benzodiazepine Epipen Topical steroid β-blocker Miscellaneous *Includes third, fourth, and fifth year residents.
370
www.pec-online.com
23,640 394 44 350 60 (16%) 14 42 (12%) n (%) (n = 350) 134 (38) 101 (29) 115 (33) 172 (49) 178 (51) 90 (26) 43 (12) 14 (4) 11 (3) 10 (3) 10 (3) 137 (39) 44 (13) 38 (11) 36 (10) 29 (8) 12 (3) 10 (3) 10 (3) 9 (3) 6 (2) 4 (1) 2 (0.6) 1 (0.3) 12 (3)
those with the potential to have a side effect for the patient, were also not statistically significantly different based on level of training. However, when pediatric residents were analyzed separately, the error rate decreased by over two-thirds between first and second year of training. There was no statistically significant difference in error rate based on level of training among nonpediatric residents. Finally, in terms of resident specialty, pediatric residents had the lowest prescription error rate (Table 5). Nonpediatric residents were nearly 2 times more likely to have a prescription error (95% confidence interval, 0.92–3.86; P value = 0.06) but this difference was not statistically significant. Emergency medicine residents had a statistically significant higher error rate compared to pediatric residents (21% vs. 8.7%; P = 0.009).
DISCUSSION Despite the EHR and CPOE with CDSS, prescription errors are still found. And while most errors are insignificant, problematic and serious errors do occur. Prescriptions written using CPOE have a relatively low error rate and the types of errors seen are different compared to errors using handwritten prescriptions. Our study is similar to a study completed at the same institution 13 years prior with a similar group of residents, which analyzed handwritten prescriptions.6 At that time, 59% of prescriptions contained an error and the most common errors were minor omission, and incomplete directions (Fig. 1). Our study had an error rate that was two thirds that of the previous study looking at handwritten prescriptions. Likewise, we found that minor omissions are now the least common error type, which is similar to a prior study looking at error rates using CPOE in an academic PED.15 The relative paucity of minor omissions is understandable given that the CPOE prompts the prescriber to enter all necessary components of a prescription, thus eliminating most minor errors. However, we continue to see incomplete directions because of the free text option for prescribing, which may hinder patient safety. Dose and direction errors were the second most common error reported after the introduction of the CPOE. This is surprising given that our CPOE is connected to a CDSS with dosing and direction recommendations and alerts when doses are inappropriate. A decrease in minor omissions and increase in dosing errors was also seen in an adult intensive care unit (ICU) setting after the introduction of the CPOE.10 In this study, there was no CDSS but there was access to an online formulary. The persistence of dosing and direction errors suggests that
FIGURE 1. Type of prescription errors prescribed using CPOE described as an absolute number of prescription errors that fell into each category. © 2015 Wolters Kluwer Health, Inc. All rights reserved.
Copyright © 2015 Wolters Kluwer Health, Inc. All rights reserved.
Pediatric Emergency Care • Volume 31, Number 5, May 2015
FIGURE 2. Level of prescription error severity using CPOE described as an absolute number of prescription errors that fell into each category of severity.
residents are not effectively using the resources connected to the CPOE and may be bypassing alerts. The use of CPOE does not entirely prevent clinically significant errors. Compared to the previous study at our institution, the absolute number of problematic errors for prescriptions written using CPOE decreased by 80% but the proportion of errors that were problematic was unchanged. Furthermore, the absolute number of significant errors is unchanged and the proportion is over 4 times higher.6 The majority of significant errors were due to incorrect dosing, which as stated above, suggests residents may not be utilizing the CDSS as intended. A recent meta-analysis looking at the clinical outcomes of CPOE in 12 different intensive care units found similar findings: a reduction in total error rate but no change in morbidity or mortality.16 One ICU study showed an increase in morbidity after CPOE implementation, and it was attributed to the implementation process itself.17 On the contrary, a separate study found a reduction in mortality after CPOE implementation when they used a heavily modified CPOE that best matched their prior paper documentation.18 A reduction in significant errors with the use of CPOE and CDSS is dependent on both the careful implementation and effective customization of the software. The CPOE with CDSS has the potential to erase discrepancies in pediatric knowledge related to specialty and level of training but its success depends on one's familiarity with the system. At our institution, pediatric residents spend 7 months of the year using EPIC software, whereas nonpediatric residents who rotate through our department may spend only 2 months using EPIC. Our study found no statistically significant difference in error related to specialty; however, the error rate among nonpediatric residents was nearly double that of pediatric residents. This finding is less than that seen in a previous study at our institution with handwritten prescriptions where nonpediatric residents were 4 times
Electronic Prescription Writing Errors
as likely to make a prescription error compared to pediatric residents.6 It is expected that the nonpediatric residents would have a higher error rate given their limited exposure to pediatric dosing. However, it is possible the CPOE and CDSS tempered some of this difference. Likewise, among pediatric residents, error rates using handwritten prescriptions have been shown to decrease with increasing seniority and presumed knowledge.5 However, in our study, first year residents had a higher error rate, but there was no statistically significant difference in error between the 3 levels of training. Thus, the lack of statistical significance may be related to safe guards in the CPOE and CDSS. The relative decrease in error between first and second year of training may be related to the process of mastering CPOE and CDSS. Familiarity with CPOE and CDSS affects the difference in error rates between specialties as well. Emergency medicine residents were the only group to have a statistically significant increase in error rate compared to pediatric residents. This finding is consistent with previous studies that suggest emergency medicine residents need more training in pediatric dosing.8 However, in our study, this may not be the case. Our emergency medicine residents come from multiple different programs that do not all use EPIC unlike our other nonpediatric residents who have exposure to EPIC in either their inpatient rotations or home program. It is possible that CPOE and CDSS can overcome medical misinformation but not system unfamiliarity. Finally, in addition to possibly mitigating the difference in error between those with varying degrees of medical knowledge, CPOE and CDSS may help decrease errors among senior residents that are attributed to inattentiveness. In the prior study at our institution with handwritten prescriptions, third year residents were statistically more likely to make an error than their younger colleagues.6 This was attributed to a higher workload and more hastily written prescriptions. Comparable studies also found higher error rates among senior residents using handwritten prescriptions.7 After the introduction of CPOE, we no longer found this discrepancy and a similar study found no difference in error rate based on level of training when CPOE was used.15 There were still errors due to incomplete prescriptions and omissions, but the absolute number of errors is less, and this may be attributed to the safeguards of CPOE and CDSS. There are limitations to this study. First, the study was conducted in 1 tertiary care center and limited to 1 calendar year. Therefore the data is a reflection of a subset of prescribers and resident educators. However, despite being 1 tertiary care center, there are over 100 different prescribers from 7 different specialties. Additionally, the resident prescribers previously attended medical school in over 20 different states and therefore are a broad sample of the US residency population. Finally, our CPOE and CDSS are
TABLE 4. Error Rates Based On Level of Training Resident Level of Training PGY 1 PGY 2 PGY 3 P
Total Errors, %
Pediatric Resident Error, %
Nonpediatric* Resident Error, %
Total Clinically Significant Error, %
20/134 (14.9)† 10/101 (9.9) 12/115 (10.4) 0.42
10/69 (14.5) 2/46 (4.3) 3/57 (5.3) 0.09
10/65 (15.4) 8/55 (14.5) 9/58 (15.5) 0.99
7/134 (5.2) 5/101 (5.0) 2/115 (1.7) 0.32
*Includes family medicine, emergency medicine, emergency medicine/internal medicine, family medicine/internal medicine, medicine/pediatrics, and physical medicine and rehabilitation. † Number of prescriptions with an error divided by total number of prescriptions written electronically by all residents in each level of training, pediatric residents at each level of training, and nonpediatric residents at each level of training, respectively. The percentage of prescriptions with error is listed in parentheses.
© 2015 Wolters Kluwer Health, Inc. All rights reserved.
Copyright © 2015 Wolters Kluwer Health, Inc. All rights reserved.
www.pec-online.com
371
Pediatric Emergency Care • Volume 31, Number 5, May 2015
Nelson and Selbst
REFERENCES
TABLE 5. Error Rates Based Specialty Resident Specialty Pediatric Nonpediatric† P
Total Errors (%)
Total Clinically Significant Error
15/172 (8.7)* 27/178 (15.2) 0.06
4/172 (2.3) 10/178 (5.6) 0.17
*Number of prescriptions with an error divided by total number of prescriptions written electronically by all residents in each category. The percentage of prescriptions with error is listed in parentheses. † Includes family medicine, emergency medicine, emergency medicine/ internal medicine, family medicine/internal medicine, medicine/pediatrics, and physical medicine and rehabilitation.
tailored to our hospital system and are not representative of all systems. However, our study reinforces the importance of reviewing one's CPOE and CDSS as a means to further modify that individual system and improve patient care. Second, we used a previously designed error classification system, which may over estimate error rates. Failure to include an end date for a prescription was classified as an incomplete direction error regardless of whether enough pills were prescribed to complete the therapy. Likewise, any prescription without a route was considered an error despite the fact that most medications could only be given by mouth. Although this possibly overestimates error, it provides concrete examples of necessary improvements for CPOE and resident training. In addition, we presumed that residents wrote prescriptions independently but it is unknown whether there was supervision by senior residents, fellows, or attendings, which may have caught electronic prescription errors before they were signed and thus underestimated error. We excluded narcotic prescriptions because at our institution the attending physician has to sign these separately and presumably would have reviewed the prescription thereby correcting any error a resident may have made. It is possible that removing narcotic prescriptions underestimated error rates because narcotics are one of the most common types of prescriptions to have error.7 Finally, the authors of this study independently reviewed each prescription and classified error based on the predefined criteria. The authors met frequently to discuss prescription error classifications and scrutinized each prescription to minimize interobserver differences, but interrater reliability was not formally assessed.
CONCLUSIONS Despite the introduction of CPOE and CDSS, prescription errors continue to occur and include clinically significant errors. There was no statistically significant discrepancy in error based on level of training and only emergency medicine residents had a statistically higher error rate compared to pediatric residents. Further research in prescription error after the introduction of CPOE is needed to help highlight opportunities for improved software implementation and modification with the ultimate goal of better patient safety.
372
www.pec-online.com
1. Miller MR, Robinson KA, Lubomski LH, et al. Medication errors in paediatric care: a systematic review of epidemiology and an evaluation of evidence supporting reduction strategy recommendations. Qual Saf Health Care. 2007;16:116–126. 2. Kaushal R, Goldmann DA, Keohane CA, et al. Adverse drug events in pediatric outpatients. Ambul Pediatr. 2007;7:383–389. 3. Budnitz DS, Pollock DA, Weidenbach KN, et al. National surveillance of emergency department visits for outpatient adverse drug events. JAMA. 2006;296:1858–1866. 4. Moore TJ, Cohen MR, Furberg CD. Serious adverse drug events reported to the Food and Drug Administration, 1998–2005. Arch Intern Med. 2007; 167:1752–1759. 5. Kozer E, Scolnik D, Macpherson A, et al. Variables associated with medication errors in the pediatric emergency department. Pediatrics. 2002; 110:737–742. 6. Taylor BL, Selbst SM, Shah AEC. Prescription writing errors in the pediatric emergency department. Pediatr Emerg Care. 2005;21:822–827. 7. Rinke ML, Moon M, Clark JS, et al. Prescribing errors in a pediatric emergency department. Pediatr Emerg Care. 2008;24:1–8. 8. Nelson LS, Gordon PE, Simmons MD, et al. The benefit of houseofficer education on proper medication dose calculation and ordering. Acad Emerg Med. 2000;7:1311–1316. 9. Cusack CM. Electronic health records and electronic prescribing: promise and pitfalls. Obstet Gynecol Clin North Am. 2008;35:63–79. 10. Shulman R, Singer M, Goldstone J, et al. Medication errors: a prospective cohort study of hand-written and computerized physician order entry in the intensive care unit. Crit Care. 2005;9:R516–R521. 11. Institute of Medicine, Committee on Identifying and Preventing Medication Errors. In: Aspden P, ed. Preventing Medication Errors. Washington, DC: National Academies Press; 2007. 12. Georgiou A, Prgomet M, Paoloni R, et al. The effect of computerized provider order entry systems on clinical care and work processes in emergency departments: a systematic review of the quantitative literature. Ann Emerg Med. 2013;61:644–653. 13. Sard BE, Walsh KE, Doros G, et al. Retrospective evaluation of a computerized physician order entry adaptation to prevent prescribing errors in a pediatric emergency department. Pediatrics. 2008;122:782–787. 14. Faul F, Erdfelder E, Lang A-G, et al. G*Power 3: a flexible statistical power analysis for the social, behavioral, and biomedical sciences. Behav Res Methods. 2007;39:175–191. 15. Pacheco GS, Viscusi C, Hays DP, et al. The effects of resident level of training on the rate of pediatric prescription errors in an academic emergency department. J Emerg Med. 2012;43:e343–e348. 16. Van Roose F, Maat B, Rademaker CMA, et al. The effect of computerized physician order entry on medication prescription errors and clinical outcome in pediatric and intensive care: a systematic review. Pediatrics. 2009;123:1184–1190. 17. Han YY, Carcillo JA, Venkataraman ST, et al. Unexpected increased mortality after implementation of a commercially sold computerized physician order entry system. Pediatrics. 2005;116:1506–1512. 18. Longhurst, et al. Decrease in hospital-wide mortality rate after implementation of a commercially sold computerized physician order entry system. Pediatrics. 2010;126:e1–e8.
© 2015 Wolters Kluwer Health, Inc. All rights reserved.
Copyright © 2015 Wolters Kluwer Health, Inc. All rights reserved.
Electronic Prescription Writing Errors in the Pediatric Emergency Department Courtney E. Nelson, MD*† and Steven M. Selbst, MD, FAAP, FACEP*† Key Words: electronic health record, medication error, medical education, residents (Pediatr Emer Care 2015;31: 368–372)
M
edication errors in the pediatric setting are common. The error rates range from 4% to 30% of all medication orders.1 These errors include the wrong dose, route, frequency, and medication choice, and can lead to an adverse drug event (ADE) in up to 16% of pediatric patients.2 The ADEs can require emergency medical care and hospital admission.3 The Food and Drug Administration reported that ADEs (injuries, disabilities, and death) tripled between 1998 and 2005.4 Medication errors occur in the emergency department as well. In 1 study at a pediatric tertiary care hospital, medication errors occurred in 10% of orders placed in the emergency department.5 Medication errors among ambulatory prescriptions written in the pediatric emergency department (PED) are also common, ranging from 4.3% to 59%.6,7 In these studies, the majority of errors are due to incorrectly writing the prescription, including minor omissions or incomplete directions, and not due to incorrect dosing. Physicians with numerous different training backgrounds, many of which are not primarily pediatric-based, work in academic PEDs. Some studies show that trainees have a higher error rate than attendings and that level of training is directly related to error rate.5 In terms of resident specialty, research shows that emergency medicine residents need further training in pediatric medication dosing.8 The electronic health record (EHR) and computerized physician order entry (CPOE) have the potential to improve prescription error rates. The EHR and CPOE software helps to prevent medication errors by connecting to either an online formulary, which provides recommended dosing and lists the side effects of a medication, or with a formal clinical decision support system (CDSS). The CDSS links the formulary to the patient's chart and will provide alerts, for example, if drug dose is in excess for age or weight or if there is an allergy contraindication to a medication.9,10 The Institute of Medicine recently endorsed CPOE software as a necessity to reduce medication error rates.11 A recent review found that the introduction of CPOE in the emergency department decreased medication error rates.12 More specifically, a second study found that electronic prescribing systems reduced error in a PED.13 The role of specialty and level of training on electronic prescription errors, however, has yet to be addressed in the literature. The purpose of this study is to evaluate the frequency, type, and severity of electronic prescription errors written by residents in From the *Jefferson Medical College, Philadelphia PA; and †Nemours/AI duPont Hospital for Children, Wilmington, DE. Disclosure: The authors declare no conflict of interest. Reprints: Courtney E. Nelson, MD, Department of Pediatrics, AI duPont Hospital for Children, 1600 Rockland Road, Wilmington, DE 19803 (e‐mail: [email protected]). Copyright © 2015 Wolters Kluwer Health, Inc. All rights reserved. ISSN: 0749-5161
368
www.pec-online.com
an academic PED and to determine if there is a relationship with error rate and resident training level or specialty. We hypothesize that with CPOE, error rates will be uniform among resident level of training and specialty.
METHODS We conducted a retrospective chart review of a random selection of electronic prescriptions written in an academic, suburban, tertiary care PED. The department has an annual census of 51,000 patients. Pediatric emergency medicine board-certified physicians, resident physicians training in several specialties, physician assistants, and urgent care pediatric board-certified physicians staff the emergency department. The department uses EPIC (Epic, Verona, WI) software as its EHR, CDSS, and CPOE. The Epic CDSS consists of radio buttons with the most common doses of each medication, as well as a location to free text the desired dose. In addition, with ambulatory prescriptions only, one has the option to free text the entire prescription. There is a hyperlink to the online formulary Lexicomp (Wolters Kluwer Health, Philadelphia, PA) for each medication ordered. When a dose is selected, the dose is calculated based on the electronically documented patient weight. Medications cannot be dispensed without a documented weight. Doses above or below the recommended dose trigger an alert window that may be closed and the order placed without further warning. On the contrary, all drug interactions and pregnancy category D and X drugs generate an alert window that can only be circumvented when a reason is selected. At the time of this study, each electronically written prescription was printed and attached to the discharge paperwork. The attending was not required to review each prescription. The hospital's institutional review board approved this study. We initially selected all medication prescriptions written electronically by residents between July 1, 2011, and June 30, 2012. Prescriptions ordered by attending physicians, fellows, and physician assistants were excluded. Narcotic prescriptions were excluded because these are generally reviewed by the attending physician in the emergency department and would not accurately represent resident prescription writing. Likewise, prescriptions for devices were also excluded. Finally, if a patient was admitted to the hospital, his or her prescription was excluded from the study as it was assumed he or she would not fill the prescription and may acquire a new prescription prior to discharge. We selected every sixtieth prescription for a goal of 350 patient prescriptions based on an a priori power analysis, which is described in more detail later in the text. From each prescription and electronic medical record, we extracted the patient medical record number, visit date, prescription date, patient allergies, weight, and age, medication name, dose, strength, route, quantity, and refills. In addition, the physician's name, year of training, and specialty were documented. Physician year of training was labeled as first, second, or third year resident. We considered third, fourth, and fifth year residents all “third year” or “senior residents” because only the combined residency programs extend to 4th and 5th years of training. Pediatric Emergency Care • Volume 31, Number 5, May 2015
Copyright © 2015 Wolters Kluwer Health, Inc. All rights reserved.
Pediatric Emergency Care • Volume 31, Number 5, May 2015
Electronic Prescription Writing Errors
TABLE 1. Error Categories, Adapted From a Prior Publication6 Error Categories Major omission Minor omission
Dose/directions Quantity Incomplete directions Allergy conflict
Missing drug name, strength (only if more than one available), or quantity to dispense (unless can calculate from signa). Major omission would require the pharmacist to contact the prescriber. Missing date, directions for use (dose, frequency, and route), physician signature, or refills not specified. Minor omission may be filled by the pharmacist based on professional judgment and/or additional information gathered from the parent or prescriber. Dosage or direction (including duration) that differ substantially from normal standards. Dosage form unavailable, inappropriate, or in excess of manufacture or literature recommendations and those inadequate to achieve desired therapeutic effect. Unclear quantity, such as 1 bottle or tube when more than one is available (over-the-counter medications not included). Omissions as above. Signa lacks specific directions such as indications for prn usage, solely states “as directed,” or no duration of therapy for non-routine medication An error if allergy status not documented on chart or a medication prescribed to which the patient has a listed allergy without documentation explaining the variance.
Both authors independently reviewed each prescription and corresponding medical record for errors. Errors were classified into a specific category and severity based on previously published classifications (Tables 1 and 2).6 The authors then reviewed emergency department discharge instructions for clarification of errors. If an error of omission or incomplete direction was rectified in the discharge instructions, that error was excluded from the data set. When there was more than 1 error, both were included in the analysis. Medication dose and directions were crossreferenced using Lexicomp. An incorrect dose was determined as 20% above or below recommended dosing, as used in similar studies.5 Patients were considered eligible for adult dosage at age 17 or at a weight of 70 kg. Medication dosing was considered correct despite reference values when it correlated directly with consultant recommendations. Likewise, a dose more than 20% below the standard was considered correct if it was administered during the emergency department visit and was documented as efficacious. If the volume of a medication dispensed was less than that needed for a treatment course, it was considered an error. However, if the volume dispensed was in excess, it was not an error if the written discharge instructions clarified the dose and course of treatment. Both authors met throughout the study period to discuss error classification and when there were discrepancies, a consensus was reached. The authors first conducted an a priori power analysis to determine the appropriate sample size for our study. Our primary outcome was prescription error as a dichotomous variable with
respect to level of training. Because pediatric and emergency medicine residency spans 3 years, we assumed degrees of freedom of 2. To detect a medium effect size (w = 0.30) with a type I error rate of 0.05 with a power of 0.80, we would need to enroll a total sample size of 108.14 We used descriptive statistics to analyze error type and severity with respect to level of training and specialty. We analyzed the data using Pearson χ2 test and Fisher exact test when expected outcomes were less than five. A P value of less than 0.05 was defined as significant.
RESULTS During the study period, residents in the emergency department wrote 23,640 prescriptions electronically. We selected every 60th prescription leaving 394 prescriptions. Of these 350 met our inclusion criteria. The most common type of medication prescribed was an antibiotic (Table 3). The prescribers included 141 different residents from 7 different specialties. Pediatric residents wrote 49% of the prescriptions, and nonpediatric residents wrote 51%. Among nonpediatric residents, emergency medicine residents wrote the most prescriptions. First year residents wrote the most prescriptions and second year residents the least. Sixteen percent of electronic prescriptions had at least 1 error and 4 had more than 1 error. Fourteen errors were corrected in the discharge instructions, leaving 46 errors (Table 3). Errors were categorized as shown in Figure 1. Incomplete directions and dose/direction errors were the most common error types; both
TABLE 2. Severity Categories; Adapted From a Prior Publication6 Severity Categories Severe
Medication with a low therapeutic index written for more than 10 times the normal dose, or the potential result of a pharmacological effect or serum concentration associated with severe or fatal toxic reactions. The drug had the potential to produce a severe life-threatening reaction in the patient (ie anaphylaxis). Dose too low for a lifesaving drug or one used for a severe illness. Serious Medication with a low therapeutic index, 4 to 10 times the normal dose, or potential for a serious toxic reaction. Dose too low for a serious illness. Wrong medication or route prescribed with potential for serious reaction or inadequate treatment of a serious illness. Significant Medication with a low therapeutic index, 1.5 to 4 times the normal dose. Dose of any medication more than 5 times the normal dose, or any degree greater than normal with potential for an adverse effect related to the dose. Dose or duration inadequate for therapeutic effect. Wrong medication for a non-severe illness or wrong route with potential for adverse/side effects or treatment failure. Duplicate therapy with potential of additive toxic reactions Problem Duplicate or without potential for increased side effects. Lacks specific drug, dose, strength, formulation, route, or frequency. Wrong route without potential for toxic reaction or treatment failure. Dose 5 times normal without potential for toxic reaction. Prescription unlikely to be filled due to nature of or missing information. Insignificant Prescriptions assigned an error by the described categories that do not meet the above criteria or are likely to be filled and without the potential for significant side effects or treatment failure.
© 2015 Wolters Kluwer Health, Inc. All rights reserved.
Copyright © 2015 Wolters Kluwer Health, Inc. All rights reserved.
www.pec-online.com
369
Pediatric Emergency Care • Volume 31, Number 5, May 2015
Nelson and Selbst
represented 37% of prescription errors. Among incomplete directions, physicians most often forgot to list the duration of therapy. The least common error type was a minor omission, and most of these errors involved failure to list a route of administration. There were no major omissions or allergy conflicts. Error severity was classified as severe, serious, significant, problematic, or insignificant based on previously published classifications (Fig. 2).6 In terms of severity, the majority of errors were insignificant (61%). Seventeen percent of errors were problematic, and another 17% of errors were significant. The majority of problematic errors were due to minor omissions of route and most significant errors were due to incorrect dose or directions for a medication. There were 2 serious errors, both of which occurred when a resident prescribed and dispensed only one dose of albuterol to a child who presented in respiratory distress. When error rate was analyzed using residency level of training, there was no statistically significant difference in error between residents within each level of training and all residents (P = 0.42) (Table 4). Similarly, clinically significant errors or TABLE 3. Prescription Characteristics Prescriptions Total Random sample Excluded Included Errors Corrected in d/c instructions Remaining prescriptions with error Level of training of prescriber First year Second year Third year* Subspecialty of prescriber Pediatric resident Subspecialty resident Emergency medicine Family medicine Medicine-pediatrics Emergency medicine/internal medicine Emergency medicine/family medicine Physical medicine and rehabilitation Class of drug prescribed Antibiotic Laxative Analgesic Steroid Albuterol Antiemetic Antihistamine Antacid Antiepileptic Benzodiazepine Epipen Topical steroid β-blocker Miscellaneous *Includes third, fourth, and fifth year residents.
370
www.pec-online.com
23,640 394 44 350 60 (16%) 14 42 (12%) n (%) (n = 350) 134 (38) 101 (29) 115 (33) 172 (49) 178 (51) 90 (26) 43 (12) 14 (4) 11 (3) 10 (3) 10 (3) 137 (39) 44 (13) 38 (11) 36 (10) 29 (8) 12 (3) 10 (3) 10 (3) 9 (3) 6 (2) 4 (1) 2 (0.6) 1 (0.3) 12 (3)
those with the potential to have a side effect for the patient, were also not statistically significantly different based on level of training. However, when pediatric residents were analyzed separately, the error rate decreased by over two-thirds between first and second year of training. There was no statistically significant difference in error rate based on level of training among nonpediatric residents. Finally, in terms of resident specialty, pediatric residents had the lowest prescription error rate (Table 5). Nonpediatric residents were nearly 2 times more likely to have a prescription error (95% confidence interval, 0.92–3.86; P value = 0.06) but this difference was not statistically significant. Emergency medicine residents had a statistically significant higher error rate compared to pediatric residents (21% vs. 8.7%; P = 0.009).
DISCUSSION Despite the EHR and CPOE with CDSS, prescription errors are still found. And while most errors are insignificant, problematic and serious errors do occur. Prescriptions written using CPOE have a relatively low error rate and the types of errors seen are different compared to errors using handwritten prescriptions. Our study is similar to a study completed at the same institution 13 years prior with a similar group of residents, which analyzed handwritten prescriptions.6 At that time, 59% of prescriptions contained an error and the most common errors were minor omission, and incomplete directions (Fig. 1). Our study had an error rate that was two thirds that of the previous study looking at handwritten prescriptions. Likewise, we found that minor omissions are now the least common error type, which is similar to a prior study looking at error rates using CPOE in an academic PED.15 The relative paucity of minor omissions is understandable given that the CPOE prompts the prescriber to enter all necessary components of a prescription, thus eliminating most minor errors. However, we continue to see incomplete directions because of the free text option for prescribing, which may hinder patient safety. Dose and direction errors were the second most common error reported after the introduction of the CPOE. This is surprising given that our CPOE is connected to a CDSS with dosing and direction recommendations and alerts when doses are inappropriate. A decrease in minor omissions and increase in dosing errors was also seen in an adult intensive care unit (ICU) setting after the introduction of the CPOE.10 In this study, there was no CDSS but there was access to an online formulary. The persistence of dosing and direction errors suggests that
FIGURE 1. Type of prescription errors prescribed using CPOE described as an absolute number of prescription errors that fell into each category. © 2015 Wolters Kluwer Health, Inc. All rights reserved.
Copyright © 2015 Wolters Kluwer Health, Inc. All rights reserved.
Pediatric Emergency Care • Volume 31, Number 5, May 2015
FIGURE 2. Level of prescription error severity using CPOE described as an absolute number of prescription errors that fell into each category of severity.
residents are not effectively using the resources connected to the CPOE and may be bypassing alerts. The use of CPOE does not entirely prevent clinically significant errors. Compared to the previous study at our institution, the absolute number of problematic errors for prescriptions written using CPOE decreased by 80% but the proportion of errors that were problematic was unchanged. Furthermore, the absolute number of significant errors is unchanged and the proportion is over 4 times higher.6 The majority of significant errors were due to incorrect dosing, which as stated above, suggests residents may not be utilizing the CDSS as intended. A recent meta-analysis looking at the clinical outcomes of CPOE in 12 different intensive care units found similar findings: a reduction in total error rate but no change in morbidity or mortality.16 One ICU study showed an increase in morbidity after CPOE implementation, and it was attributed to the implementation process itself.17 On the contrary, a separate study found a reduction in mortality after CPOE implementation when they used a heavily modified CPOE that best matched their prior paper documentation.18 A reduction in significant errors with the use of CPOE and CDSS is dependent on both the careful implementation and effective customization of the software. The CPOE with CDSS has the potential to erase discrepancies in pediatric knowledge related to specialty and level of training but its success depends on one's familiarity with the system. At our institution, pediatric residents spend 7 months of the year using EPIC software, whereas nonpediatric residents who rotate through our department may spend only 2 months using EPIC. Our study found no statistically significant difference in error related to specialty; however, the error rate among nonpediatric residents was nearly double that of pediatric residents. This finding is less than that seen in a previous study at our institution with handwritten prescriptions where nonpediatric residents were 4 times
Electronic Prescription Writing Errors
as likely to make a prescription error compared to pediatric residents.6 It is expected that the nonpediatric residents would have a higher error rate given their limited exposure to pediatric dosing. However, it is possible the CPOE and CDSS tempered some of this difference. Likewise, among pediatric residents, error rates using handwritten prescriptions have been shown to decrease with increasing seniority and presumed knowledge.5 However, in our study, first year residents had a higher error rate, but there was no statistically significant difference in error between the 3 levels of training. Thus, the lack of statistical significance may be related to safe guards in the CPOE and CDSS. The relative decrease in error between first and second year of training may be related to the process of mastering CPOE and CDSS. Familiarity with CPOE and CDSS affects the difference in error rates between specialties as well. Emergency medicine residents were the only group to have a statistically significant increase in error rate compared to pediatric residents. This finding is consistent with previous studies that suggest emergency medicine residents need more training in pediatric dosing.8 However, in our study, this may not be the case. Our emergency medicine residents come from multiple different programs that do not all use EPIC unlike our other nonpediatric residents who have exposure to EPIC in either their inpatient rotations or home program. It is possible that CPOE and CDSS can overcome medical misinformation but not system unfamiliarity. Finally, in addition to possibly mitigating the difference in error between those with varying degrees of medical knowledge, CPOE and CDSS may help decrease errors among senior residents that are attributed to inattentiveness. In the prior study at our institution with handwritten prescriptions, third year residents were statistically more likely to make an error than their younger colleagues.6 This was attributed to a higher workload and more hastily written prescriptions. Comparable studies also found higher error rates among senior residents using handwritten prescriptions.7 After the introduction of CPOE, we no longer found this discrepancy and a similar study found no difference in error rate based on level of training when CPOE was used.15 There were still errors due to incomplete prescriptions and omissions, but the absolute number of errors is less, and this may be attributed to the safeguards of CPOE and CDSS. There are limitations to this study. First, the study was conducted in 1 tertiary care center and limited to 1 calendar year. Therefore the data is a reflection of a subset of prescribers and resident educators. However, despite being 1 tertiary care center, there are over 100 different prescribers from 7 different specialties. Additionally, the resident prescribers previously attended medical school in over 20 different states and therefore are a broad sample of the US residency population. Finally, our CPOE and CDSS are
TABLE 4. Error Rates Based On Level of Training Resident Level of Training PGY 1 PGY 2 PGY 3 P
Total Errors, %
Pediatric Resident Error, %
Nonpediatric* Resident Error, %
Total Clinically Significant Error, %
20/134 (14.9)† 10/101 (9.9) 12/115 (10.4) 0.42
10/69 (14.5) 2/46 (4.3) 3/57 (5.3) 0.09
10/65 (15.4) 8/55 (14.5) 9/58 (15.5) 0.99
7/134 (5.2) 5/101 (5.0) 2/115 (1.7) 0.32
*Includes family medicine, emergency medicine, emergency medicine/internal medicine, family medicine/internal medicine, medicine/pediatrics, and physical medicine and rehabilitation. † Number of prescriptions with an error divided by total number of prescriptions written electronically by all residents in each level of training, pediatric residents at each level of training, and nonpediatric residents at each level of training, respectively. The percentage of prescriptions with error is listed in parentheses.
© 2015 Wolters Kluwer Health, Inc. All rights reserved.
Copyright © 2015 Wolters Kluwer Health, Inc. All rights reserved.
www.pec-online.com
371
Pediatric Emergency Care • Volume 31, Number 5, May 2015
Nelson and Selbst
REFERENCES
TABLE 5. Error Rates Based Specialty Resident Specialty Pediatric Nonpediatric† P
Total Errors (%)
Total Clinically Significant Error
15/172 (8.7)* 27/178 (15.2) 0.06
4/172 (2.3) 10/178 (5.6) 0.17
*Number of prescriptions with an error divided by total number of prescriptions written electronically by all residents in each category. The percentage of prescriptions with error is listed in parentheses. † Includes family medicine, emergency medicine, emergency medicine/ internal medicine, family medicine/internal medicine, medicine/pediatrics, and physical medicine and rehabilitation.
tailored to our hospital system and are not representative of all systems. However, our study reinforces the importance of reviewing one's CPOE and CDSS as a means to further modify that individual system and improve patient care. Second, we used a previously designed error classification system, which may over estimate error rates. Failure to include an end date for a prescription was classified as an incomplete direction error regardless of whether enough pills were prescribed to complete the therapy. Likewise, any prescription without a route was considered an error despite the fact that most medications could only be given by mouth. Although this possibly overestimates error, it provides concrete examples of necessary improvements for CPOE and resident training. In addition, we presumed that residents wrote prescriptions independently but it is unknown whether there was supervision by senior residents, fellows, or attendings, which may have caught electronic prescription errors before they were signed and thus underestimated error. We excluded narcotic prescriptions because at our institution the attending physician has to sign these separately and presumably would have reviewed the prescription thereby correcting any error a resident may have made. It is possible that removing narcotic prescriptions underestimated error rates because narcotics are one of the most common types of prescriptions to have error.7 Finally, the authors of this study independently reviewed each prescription and classified error based on the predefined criteria. The authors met frequently to discuss prescription error classifications and scrutinized each prescription to minimize interobserver differences, but interrater reliability was not formally assessed.
CONCLUSIONS Despite the introduction of CPOE and CDSS, prescription errors continue to occur and include clinically significant errors. There was no statistically significant discrepancy in error based on level of training and only emergency medicine residents had a statistically higher error rate compared to pediatric residents. Further research in prescription error after the introduction of CPOE is needed to help highlight opportunities for improved software implementation and modification with the ultimate goal of better patient safety.
372
www.pec-online.com
1. Miller MR, Robinson KA, Lubomski LH, et al. Medication errors in paediatric care: a systematic review of epidemiology and an evaluation of evidence supporting reduction strategy recommendations. Qual Saf Health Care. 2007;16:116–126. 2. Kaushal R, Goldmann DA, Keohane CA, et al. Adverse drug events in pediatric outpatients. Ambul Pediatr. 2007;7:383–389. 3. Budnitz DS, Pollock DA, Weidenbach KN, et al. National surveillance of emergency department visits for outpatient adverse drug events. JAMA. 2006;296:1858–1866. 4. Moore TJ, Cohen MR, Furberg CD. Serious adverse drug events reported to the Food and Drug Administration, 1998–2005. Arch Intern Med. 2007; 167:1752–1759. 5. Kozer E, Scolnik D, Macpherson A, et al. Variables associated with medication errors in the pediatric emergency department. Pediatrics. 2002; 110:737–742. 6. Taylor BL, Selbst SM, Shah AEC. Prescription writing errors in the pediatric emergency department. Pediatr Emerg Care. 2005;21:822–827. 7. Rinke ML, Moon M, Clark JS, et al. Prescribing errors in a pediatric emergency department. Pediatr Emerg Care. 2008;24:1–8. 8. Nelson LS, Gordon PE, Simmons MD, et al. The benefit of houseofficer education on proper medication dose calculation and ordering. Acad Emerg Med. 2000;7:1311–1316. 9. Cusack CM. Electronic health records and electronic prescribing: promise and pitfalls. Obstet Gynecol Clin North Am. 2008;35:63–79. 10. Shulman R, Singer M, Goldstone J, et al. Medication errors: a prospective cohort study of hand-written and computerized physician order entry in the intensive care unit. Crit Care. 2005;9:R516–R521. 11. Institute of Medicine, Committee on Identifying and Preventing Medication Errors. In: Aspden P, ed. Preventing Medication Errors. Washington, DC: National Academies Press; 2007. 12. Georgiou A, Prgomet M, Paoloni R, et al. The effect of computerized provider order entry systems on clinical care and work processes in emergency departments: a systematic review of the quantitative literature. Ann Emerg Med. 2013;61:644–653. 13. Sard BE, Walsh KE, Doros G, et al. Retrospective evaluation of a computerized physician order entry adaptation to prevent prescribing errors in a pediatric emergency department. Pediatrics. 2008;122:782–787. 14. Faul F, Erdfelder E, Lang A-G, et al. G*Power 3: a flexible statistical power analysis for the social, behavioral, and biomedical sciences. Behav Res Methods. 2007;39:175–191. 15. Pacheco GS, Viscusi C, Hays DP, et al. The effects of resident level of training on the rate of pediatric prescription errors in an academic emergency department. J Emerg Med. 2012;43:e343–e348. 16. Van Roose F, Maat B, Rademaker CMA, et al. The effect of computerized physician order entry on medication prescription errors and clinical outcome in pediatric and intensive care: a systematic review. Pediatrics. 2009;123:1184–1190. 17. Han YY, Carcillo JA, Venkataraman ST, et al. Unexpected increased mortality after implementation of a commercially sold computerized physician order entry system. Pediatrics. 2005;116:1506–1512. 18. Longhurst, et al. Decrease in hospital-wide mortality rate after implementation of a commercially sold computerized physician order entry system. Pediatrics. 2010;126:e1–e8.
© 2015 Wolters Kluwer Health, Inc. All rights reserved.
Copyright © 2015 Wolters Kluwer Health, Inc. All rights reserved.
