NEWS & VIEWS Do electronic alerts for AKI improve outcomes? Matthew T. James and Amit X. Garg Refers to Wilson, F. P. et al. Automated, electronic alerts for acute kidney injury: a single-blind, parallel-group, randomised controlled trial. Lancet http://dx.doi.org/10.1016/S0140‑6736(15)60266‑5
Automated, real-time alert systems have the potential to improve recognition and management of acute kidney injury (AKI). A recent patient-level, randomized controlled trial, however, demonstrated no effect of such a system on AKI progression, receipt of dialysis or death. These findings inform the future implementation and testing of AKI alerts. Acute kidney injury (AKI) affects 10% of hospitalized patients and is associated with an increased risk of death, longer hospital stays and high health-care costs, but is often unrecognized or poorly managed.1 Automated systems that alert physicians to patients with AKI have the potential to improve care.2,3 Such systems can use consensus definitions of AKI based on changes in serum creatinine levels and leverage new hospital-based linkages between electronic medical records and laboratory reporting systems. Although great enthusiasm exists for AKI alert systems, few trials have assessed their efficacy. Wilson et al. recently completed a single-centre, patient-level randomized controlled trial to investigate the effects of automated electronic AKI alerts versus usual care on the outcomes of AKI severity, receipt of dialysis and death.4 To our knowledge this study, which included 2,393 patients with AKI at the hospital of the University of Pennsylvania, USA, is the first major randomized trial of an AKI alert system. Patients were assigned to receive usual care or AKI alerts that were initiated based on the Kidney Disease: Improving Global Outcomes (KDIGO) AKI criteria (an increase in serum creatinine level ≥26 μmol/l [0.3 mg/dl] in the previous 48 h or ≥50% in the previous 7 days).5 The covering care provider (intern, resident or nurse practitioner) and the unit pharmacist were alerted to the presence of AKI via a page sent to their mobile phones. Alerts were sent within 1 h of the creatinine result that identified AKI and were sent only once per patient.
The primary outcome of the trial was a ranked composite score of the relative maximum change in creatinine concentration, dialysis or death within 7 days after assignment to the AKI or usual care group. Randomization was stratified by medical versus surgical admission and by intensive care unit (ICU) admission; 30% of patients were in the ICU at the time of randomization and 42% were on a surgical ward. No significant differences were reported in the primary composite outcome or the outcomes of receipt of dialysis or death in the alert and usual care groups. In the surgical subgroup, alerts versus usual care resulted in more nephrology consultations (12% versus 5%) and a greater use of dialysis (6% versus 3%). The trial had several methodological strengths. Allocation to study group was performed by the hospital electronic medical record system, thereby maintaining allocation concealment. Moreover a page was sent to the care provider for 97% of patients who generated an AKI alert; no AKI alerting occurred outside the trial. Although care providers and patients could not be blinded owing to the nature of the intervention, the study outcomes were objective and the outcome assessors were masked to allocation. The analysis appropriately followed an intention-to-treat approach. As acknowledged by the researchers, the trial design did have some important limitations. For example, many care providers could have treated patients in both study groups, potentially resulting in an inadvertent improvement in AKI recognition and treatment in the usual care group. The risk
NATURE REVIEWS | NEPHROLOGY
of such contamination of the intervention is unavoidable in a patient-level randomized trial of a complex intervention of this nature. The use of clustered randomized trials in which hospitals, wards or groups of care providers undergo randomization, rather than individual patients, would minimize this risk. Notably the median change in serum creatinine levels 7 days after randomization was 0.0% in the alert group and 0.6% in the usual care group, illustrating that the majority of participants did not progress to a more severe stage of AKI during this period. This observation suggests that AKI alerts that adhere to the KDIGO serum-creatinine-based definition of AKI might be too sensitive and, therefore, vulnerable to ‘alert fatigue’ (in that care providers become less likely to respond to an alert the more often they are exposed to it, even when alerts are limited to one per patient). The development of alerts that incorporate AKI severity staging or are based on refined prognostication for progressive forms of AKI might address this limitation. Although studies of alerts and decisionsupport systems have generally shown that they can alter clinician decision making and actions, data on whether these effects translate into improved patient outcomes are limited.6 Wilson et al. found that AKI alerts
19:52
!
AKI alert
NPG
ACUTE KIDNEY INJURY
VOLUME 11 | JUNE 2015 © 2015 Macmillan Publishers Limited. All rights reserved
NEWS & VIEWS had no impact on processes of care for AKI (including exposure to nephrotoxic medications and administration of fluid boluses). Given this absence of any detectable change in provider behaviour, the finding that AKI alerts did not alter clinical outcomes is not surprising. The alert prompted providers to visit a website for more information on AKI, but no data are provided on whether traffic to this website increased as a result. Several features that have been shown to enhance the effectiveness of clinical decision support were not included in the AKI alerting intervention, including delivery of decision support as part of clinician workflow at the time and location of decision making, and provision of management recommendations.6–8 Building such elements into realtime alert and decision-support systems for AKI might be the best approach to impact patient care and outcomes. Two previous observational studies demonstrated that electronic AKI alerts can alter process-based outcomes. Colpaert et al. implemented an AKI alert accompanied by an educational course in AKI in the ICU of a tertiary care hospital in Belgium.9 Providers were notified of evolving AKI (worsening Risk, Injury, Failure, Loss of function, End-stage renal disease class) through a real-time digital cordless telephone system. Using a pre-implementation versus post- implementation comparison, the researchers showed that patients in the alert group were more likely to receive treatment for AKI (mainly fluid therapy) within 60 min of onset (29%) than those in the pre-implementation group (8%). In a study by McCoy et al., providers were alerted through a computerized order entry system and on printed rounding reports when a patient had a rise in their serum creatinine concentration >44 μmol/l
JUNE 2015 | VOLUME 11
(0.5 mg/dl) over 48 h.10 The AKI alert was linked to recommendations within the medication ordering system that prompted providers to discontinue or modify the dose of any of 122 medications that were nephrotoxic or required dose adjustment in the presence of reduced kidney function. In their time-series analysis, the investigators showed a significant increase in the rate of medication modification or discontinuation within 24 h of AKI associated with implementation of the alert (from 35 events per 100 medication exposures in the pre-intervention period to 53 events per 100 medication exposures in the post-intervention period). Taken together, these observational studies suggest that real-time AKI alerts might increase the frequency and timeliness of treatment. Wilson et al. are to be congratulated for performing the first randomized controlled trial of an automated electronic alert for AKI.4 Their study exemplifies the challenges of implementing and evaluating complex interventions designed to influence provider behaviour. Although the trial showed no impact of an automated AKI alert on care and outcomes in a single centre, the generalizability of this finding remains uncertain and further research using refined approaches is warranted. Lessons from this trial will inform how future AKI alerts and related computerized decision-support initiatives should be implemented and tested. Division of Nephrology, Departments of Medicine and Community Health Sciences, University of Calgary, Foothills Medical Centre, 1403 29th Street NW, Calgary, AB T2N 2T9, Canada, (M.T.J.). Western University, London Health Sciences Centre, Room ELL‑101, Westminster Tower, 800 Commissioners Road East, London, ON N6A 4G5, Canada, (A.X.G.). Correspondence to: M.T.J. [email protected]
doi:10.1038/nrneph.2015.55 Published online 21 April 2015 Competing interests The authors declare no competing interests. 1.
National Confidential Enquiry into Patient Outcome and Death. Adding insult to injury: a review of the care of patients who died in hospital with a primary diagnosis of acute kidney injury (acute renal failure). NCEPOD [online], http://www.ncepod.org.uk/ 2009report1/Downloads/AKI_summary.pdf (2009). 2. Porter, C. J. et al. A real-time electronic alert to improve detection of acute kidney injury in a large teaching hospital. Nephrol. Dial. Transplant. 29, 1888–1893 (2014). 3. Selby, N. M. et al. Use of electronic results reporting to diagnose and monitor AKI in hospitalized patients. Clin. J. Am. Soc. Nephrol. 7, 533–540 (2012). 4. Wilson, F. P. et al. Automated, electronic alerts for acute kidney injury: a single-blind, parallelgroup, randomised controlled trial. Lancet http://dx.doi.org/10.1016/S01406736(15)60266-5. 5. Kidney Disease: Improving Global Outcomes (KDIGO) Acute Kidney Injury Work Group. KDIGO clinical practice guideline for acute kidney injury. Kidney Int. 2, 19–36 (2012). 6. Garg, A. X. et al. Effects of computerized clinical decision support systems on practitioner performance and patient outcomes: a systematic review. JAMA 293, 1223–1238 (2005). 7. Kawamoto, K., Houlihan, C. A., Balas, E. A. & Lobach, D. F. Improving clinical practice using clinical decision support systems: a systematic review of trials to identify features critical to success. BMJ 330, 765 (2005). 8. Roshanov, P. S. et al. Features of effective computerised clinical decision support systems: meta-regression of 162 randomised trials. BMJ 346, f657 (2013). 9. Colpaert, K. et al. Impact of real-time electronic alerting of acute kidney injury on therapeutic intervention and progression of RIFLE class. Crit. Care Med. 40, 1164–1170 (2012). 10. McCoy, A. B. et al. A computerized provider order entry intervention for medication safety during acute kidney injury: a quality improvement report. Am. J. Kidney Dis. 56, 832–841 (2010).
www.nature.com/nrneph © 2015 Macmillan Publishers Limited. All rights reserved
Automated, real-time alert systems have the potential to improve recognition and management of acute kidney injury (AKI). A recent patient-level, randomized controlled trial, however, demonstrated no effect of such a system on AKI progression, receipt of dialysis or death. These findings inform the future implementation and testing of AKI alerts. Acute kidney injury (AKI) affects 10% of hospitalized patients and is associated with an increased risk of death, longer hospital stays and high health-care costs, but is often unrecognized or poorly managed.1 Automated systems that alert physicians to patients with AKI have the potential to improve care.2,3 Such systems can use consensus definitions of AKI based on changes in serum creatinine levels and leverage new hospital-based linkages between electronic medical records and laboratory reporting systems. Although great enthusiasm exists for AKI alert systems, few trials have assessed their efficacy. Wilson et al. recently completed a single-centre, patient-level randomized controlled trial to investigate the effects of automated electronic AKI alerts versus usual care on the outcomes of AKI severity, receipt of dialysis and death.4 To our knowledge this study, which included 2,393 patients with AKI at the hospital of the University of Pennsylvania, USA, is the first major randomized trial of an AKI alert system. Patients were assigned to receive usual care or AKI alerts that were initiated based on the Kidney Disease: Improving Global Outcomes (KDIGO) AKI criteria (an increase in serum creatinine level ≥26 μmol/l [0.3 mg/dl] in the previous 48 h or ≥50% in the previous 7 days).5 The covering care provider (intern, resident or nurse practitioner) and the unit pharmacist were alerted to the presence of AKI via a page sent to their mobile phones. Alerts were sent within 1 h of the creatinine result that identified AKI and were sent only once per patient.
The primary outcome of the trial was a ranked composite score of the relative maximum change in creatinine concentration, dialysis or death within 7 days after assignment to the AKI or usual care group. Randomization was stratified by medical versus surgical admission and by intensive care unit (ICU) admission; 30% of patients were in the ICU at the time of randomization and 42% were on a surgical ward. No significant differences were reported in the primary composite outcome or the outcomes of receipt of dialysis or death in the alert and usual care groups. In the surgical subgroup, alerts versus usual care resulted in more nephrology consultations (12% versus 5%) and a greater use of dialysis (6% versus 3%). The trial had several methodological strengths. Allocation to study group was performed by the hospital electronic medical record system, thereby maintaining allocation concealment. Moreover a page was sent to the care provider for 97% of patients who generated an AKI alert; no AKI alerting occurred outside the trial. Although care providers and patients could not be blinded owing to the nature of the intervention, the study outcomes were objective and the outcome assessors were masked to allocation. The analysis appropriately followed an intention-to-treat approach. As acknowledged by the researchers, the trial design did have some important limitations. For example, many care providers could have treated patients in both study groups, potentially resulting in an inadvertent improvement in AKI recognition and treatment in the usual care group. The risk
NATURE REVIEWS | NEPHROLOGY
of such contamination of the intervention is unavoidable in a patient-level randomized trial of a complex intervention of this nature. The use of clustered randomized trials in which hospitals, wards or groups of care providers undergo randomization, rather than individual patients, would minimize this risk. Notably the median change in serum creatinine levels 7 days after randomization was 0.0% in the alert group and 0.6% in the usual care group, illustrating that the majority of participants did not progress to a more severe stage of AKI during this period. This observation suggests that AKI alerts that adhere to the KDIGO serum-creatinine-based definition of AKI might be too sensitive and, therefore, vulnerable to ‘alert fatigue’ (in that care providers become less likely to respond to an alert the more often they are exposed to it, even when alerts are limited to one per patient). The development of alerts that incorporate AKI severity staging or are based on refined prognostication for progressive forms of AKI might address this limitation. Although studies of alerts and decisionsupport systems have generally shown that they can alter clinician decision making and actions, data on whether these effects translate into improved patient outcomes are limited.6 Wilson et al. found that AKI alerts
19:52
!
AKI alert
NPG
ACUTE KIDNEY INJURY
VOLUME 11 | JUNE 2015 © 2015 Macmillan Publishers Limited. All rights reserved
NEWS & VIEWS had no impact on processes of care for AKI (including exposure to nephrotoxic medications and administration of fluid boluses). Given this absence of any detectable change in provider behaviour, the finding that AKI alerts did not alter clinical outcomes is not surprising. The alert prompted providers to visit a website for more information on AKI, but no data are provided on whether traffic to this website increased as a result. Several features that have been shown to enhance the effectiveness of clinical decision support were not included in the AKI alerting intervention, including delivery of decision support as part of clinician workflow at the time and location of decision making, and provision of management recommendations.6–8 Building such elements into realtime alert and decision-support systems for AKI might be the best approach to impact patient care and outcomes. Two previous observational studies demonstrated that electronic AKI alerts can alter process-based outcomes. Colpaert et al. implemented an AKI alert accompanied by an educational course in AKI in the ICU of a tertiary care hospital in Belgium.9 Providers were notified of evolving AKI (worsening Risk, Injury, Failure, Loss of function, End-stage renal disease class) through a real-time digital cordless telephone system. Using a pre-implementation versus post- implementation comparison, the researchers showed that patients in the alert group were more likely to receive treatment for AKI (mainly fluid therapy) within 60 min of onset (29%) than those in the pre-implementation group (8%). In a study by McCoy et al., providers were alerted through a computerized order entry system and on printed rounding reports when a patient had a rise in their serum creatinine concentration >44 μmol/l
JUNE 2015 | VOLUME 11
(0.5 mg/dl) over 48 h.10 The AKI alert was linked to recommendations within the medication ordering system that prompted providers to discontinue or modify the dose of any of 122 medications that were nephrotoxic or required dose adjustment in the presence of reduced kidney function. In their time-series analysis, the investigators showed a significant increase in the rate of medication modification or discontinuation within 24 h of AKI associated with implementation of the alert (from 35 events per 100 medication exposures in the pre-intervention period to 53 events per 100 medication exposures in the post-intervention period). Taken together, these observational studies suggest that real-time AKI alerts might increase the frequency and timeliness of treatment. Wilson et al. are to be congratulated for performing the first randomized controlled trial of an automated electronic alert for AKI.4 Their study exemplifies the challenges of implementing and evaluating complex interventions designed to influence provider behaviour. Although the trial showed no impact of an automated AKI alert on care and outcomes in a single centre, the generalizability of this finding remains uncertain and further research using refined approaches is warranted. Lessons from this trial will inform how future AKI alerts and related computerized decision-support initiatives should be implemented and tested. Division of Nephrology, Departments of Medicine and Community Health Sciences, University of Calgary, Foothills Medical Centre, 1403 29th Street NW, Calgary, AB T2N 2T9, Canada, (M.T.J.). Western University, London Health Sciences Centre, Room ELL‑101, Westminster Tower, 800 Commissioners Road East, London, ON N6A 4G5, Canada, (A.X.G.). Correspondence to: M.T.J. [email protected]
doi:10.1038/nrneph.2015.55 Published online 21 April 2015 Competing interests The authors declare no competing interests. 1.
National Confidential Enquiry into Patient Outcome and Death. Adding insult to injury: a review of the care of patients who died in hospital with a primary diagnosis of acute kidney injury (acute renal failure). NCEPOD [online], http://www.ncepod.org.uk/ 2009report1/Downloads/AKI_summary.pdf (2009). 2. Porter, C. J. et al. A real-time electronic alert to improve detection of acute kidney injury in a large teaching hospital. Nephrol. Dial. Transplant. 29, 1888–1893 (2014). 3. Selby, N. M. et al. Use of electronic results reporting to diagnose and monitor AKI in hospitalized patients. Clin. J. Am. Soc. Nephrol. 7, 533–540 (2012). 4. Wilson, F. P. et al. Automated, electronic alerts for acute kidney injury: a single-blind, parallelgroup, randomised controlled trial. Lancet http://dx.doi.org/10.1016/S01406736(15)60266-5. 5. Kidney Disease: Improving Global Outcomes (KDIGO) Acute Kidney Injury Work Group. KDIGO clinical practice guideline for acute kidney injury. Kidney Int. 2, 19–36 (2012). 6. Garg, A. X. et al. Effects of computerized clinical decision support systems on practitioner performance and patient outcomes: a systematic review. JAMA 293, 1223–1238 (2005). 7. Kawamoto, K., Houlihan, C. A., Balas, E. A. & Lobach, D. F. Improving clinical practice using clinical decision support systems: a systematic review of trials to identify features critical to success. BMJ 330, 765 (2005). 8. Roshanov, P. S. et al. Features of effective computerised clinical decision support systems: meta-regression of 162 randomised trials. BMJ 346, f657 (2013). 9. Colpaert, K. et al. Impact of real-time electronic alerting of acute kidney injury on therapeutic intervention and progression of RIFLE class. Crit. Care Med. 40, 1164–1170 (2012). 10. McCoy, A. B. et al. A computerized provider order entry intervention for medication safety during acute kidney injury: a quality improvement report. Am. J. Kidney Dis. 56, 832–841 (2010).
www.nature.com/nrneph © 2015 Macmillan Publishers Limited. All rights reserved
