EDITORIAL
What Do We Know About the Safe Surgery Checklist Now? Alex B. Haynes, MD, MPH,∗ † William R. Berry, MD, MPH,†‡ and Atul A. Gawande, MD, MPH‡§
I
n 2009, when The New England Journal of Medicine published the 8-city trial of the World Health Organization’s Safe Surgery Checklist, the idea that a formal system of planning and communication could significantly improve patient outcomes was outside the surgical mainstream. The pre-post study found substantial reductions in complications and deaths, with the improvements significant in both higher and lower income settings.1 However, the study had notable weaknesses. There was no control group, nor was the intervention applied on all units in the study hospitals. The program had not been demonstrated to be replicable at a large scale, and the sustainability of the effect had not been proved. A series of subsequent studies with strong control groups have since replicated the substantial value of using a systematic checklist-based approach to surgical team planning and communication. For instance, van Klei et al2 demonstrated in more than 25,000 patients that the odds ratio of 30-day mortality was 0.44 for patients whose teams completed the checklist compared with 1.16 for those whose teams did not. Beneficial findings were also demonstrated at a large scale: the SURPASS trial showed that a comprehensive surgical checklist approach reduced mortality rates by 47% in 7 Dutch hospitals compared with controls,3 and Neily et al4 demonstrated using a step-wedge methodology that introduction of a checklist-driven system of briefing and debriefing in 74 Veterans Administration (VA) hospitals through a concerted team training program led to an 18% reduction in mortality. As a result, several countries have introduced a safe surgery checklist program nationally. For instance, after several years of flat surgical inpatient mortality, NHS Scotland carried out a concerted nationwide implementation program that it credited with making major reductions in inpatient mortality that meant more than 9000 fewer lives lost.5 Recently, however, Urbach et al6 examined the effects of a law mandating surgical checklist use in Ontario hospitals and found only a modest mortality reduction that did not reach statistical significance. Given these conflicting data, a remaining concern about the validity of even the prior controlled studies was that they had not been randomized. The study by Haugen and colleagues7 in the current issue of Annals of Surgery therefore makes a major contribution by reporting the results of the first cluster randomized trial of a surgical safety checklist, in this case employing random allocation of dates of implementation of the intervention among different units within 2 hospitals in Norway. Analysis of outcomes focused on a cohort of patients with a significant rate of morbidity and in-hospital mortality (19.9% and 1.6%, respectively) and followed a step-wedge methodology. The study confirmed that use of the checklist resulted in substantial improvement in outcomes—with a relative risk reduction in major complications of 0.42 (95% confidence interval, 0.33–0.50). The number needed to treat to prevent morbidity was just 12. Deaths fell from 1.6% to 1.0%, which is potentially substantial, but the study was not powered to detect if this was a significant difference (P = 0.151). Importantly, the authors also confirmed a dose effect by assessing the actual use of the checklist through direct observation: they found an even larger reduction in major complications when teams completed all portions of the checklist—a finding consistent with that of van Klei et al.2 Such findings confirm that research on the effectiveness of quality tools such as checklists must include assessment of use, as no quality tool, particularly one that relies upon team communication and interaction, can be expected to make a difference if it is not actually used in any meaningful sense. This work also affirms emerging evidence that institution of an effective safe surgery checklist program at a large scale requires a deliberate implementation process and at least some form of monitoring and learning with frontline teams.8 Haugen and colleagues7,9 are to be complimented not only for the thoroughness of the process but also for the details on the implementation efforts shared in this article and in prior publications. They detail several precepts of their implementation that we believe may be key to success in team-based interventions. First, they modified the checklist to fit with local context. Although modification must be undertaken in a thoughtful manner, this is key to ensuring that the instrument fits the local workflow and circumstances. Second, after modification,
From the ∗ Department of Surgery, Massachusetts General Hospital, Boston, MA; †Ariadne Labs: A joint center of the Harvard School of Public Health and Brigham and Women’s Hospital, Boston, MA; ‡Department of Health Policy and Management, Harvard School of Public Health, Boston, MA; and §Department of Surgery, Brigham and Women’s Hospital, Boston, MA. Disclosure: The authors declare no conflicts of interest. Reprints: Alex B. Haynes, MD, MPH, Department of Surgery, Massachusetts General Hospital, 55 Fruit St, Yawkey 7B, Boston, MA 02114. E-mail: [email protected]. C 2015 Wolters Kluwer Health, Inc. All rights reserved. Copyright ISSN: 0003-4932/15/26105-0829 DOI: 10.1097/SLA.0000000000001144
Annals of Surgery r Volume 261, Number 5, May 2015
www.annalsofsurgery.com | 829
Copyright © 2015 Wolters Kluwer Health, Inc. Unauthorized reproduction of this article is prohibited.
Annals of Surgery r Volume 261, Number 5, May 2015
Haynes et al
the hospitals piloted the checklist at a small scale, with only a few teams, and then used the experience to make further modifications. We believe this produces both learning and a few “champions” or “super-users” who are essential to success in later phases. Third, once the checklist had been finalized, the implementation team introduced it more widely through dedicated training sessions and distribution of education materials involving all disciplines. Even a seemingly simple and intuitive checklist involves changes in practices that clinicians have to understand and become comfortable with. Finally, and equally importantly, the hospitals’ implementation group regularly observed use of the checklist during cases day-to-day and both provided to and received feedback from individual clinicians, allowing opportunities for further education and reinforcement. The rollout program by the VA and by the NHS Scotland followed similar precepts. Scotland facilitated a gradual process of developing local implementation teams that included surgeons, made local checklist modifications, carried out small-scale piloting, and led dedicated training for all disciplines involved in surgical care.5 In the VA’s program, implementation teams obtained local feedback for checklist modification and shut down nonemergency surgery for a full day of training, including practice with team communication skills and the checklist, at each facility.4 In each quarter after this training, postoperative mortality declined significantly compared with nonparticipating facilities where there was no decline. In contrast, Ontario’s government mandate was not accompanied by a sustained and systematic implementation program with these kinds of features and resulted in little measurable change in outcomes, at least in a short-term evaluation.6 In sum, Haugen and colleagues have made an invaluable contribution. Using a cluster randomized control methodology, a carefully structured implementation program, and measurement of the actual use of the checklist, they conducted the most rigorous study of surgical safety checklists to date. They confirmed substantial patient benefit from taking this kind of formalized checklist approach
830 | www.annalsofsurgery.com
to team planning and communication in the operating room, demonstrating marked reductions in complications and possibly mortality. Not insignificantly, they detailed the elements of an effective implementation of a hospital-wide quality improvement program, specifics of which are often overlooked in manuscripts in favor of statistical methodology. Their work should be an example for surgical teams everywhere and a model for publication of quality improvement trials.
REFERENCES 1. Haynes AB, Weiser TG, Berry WR, et al. A surgical safety checklist to reduce morbidity and mortality in a global patient population. N Engl J Med. 2009;350:491–499. 2. van Klei WA, Hoff RG, van Aarnhem EE, et al. Effects of the introduction of the WHO “Surgical Safety Checklist” on in-hospital mortality: a cohort study. Ann Surg. 2012;255:44–49. 3. de Vries EN, Prins HA, Crolla RM, et al. Effect of a comprehensive surgical safety system on patient outcomes. N Engl J Med. 2010;363:1928–1937. 4. Neily J, Mills PD, Young-Xu Y, et al. Association between implementation of a medical team training program and surgical mortality. JAMA. 2010;304:1693– 1700. 5. Scotland National Health Service. The healthcare quality strategy for the NHS Scotland. Report at the International Society for Quality in Healthcare, 24 October 2013. Available at: http://www.isqua.org/docs/ edinburgh-powerpoint-presentations-2013/1545-afternoon-plenary-jason-leitch -pentland-mon-scotlands-quality-journey.pdf?sfvrsn=2. Accessed December 15, 2014. 6. Urbach DR, Govindarajan A, Saskin R, et al. Introduction of surgical safety checklists in Ontario, Canada. N Engl J Med. 2014;370:1029–1038. 7. Haugen AS, Softeland E, Almeland SK, et al. Effect of the World Health Organization checklist on patient outcomes: a stepped wedge cluster randomized controlled trial. Ann Surg. 2015;261:821–828. 8. Conley DM, Singer SJ, Edmondson L, et al. Effective surgical safety checklist implementation. J Am Coll Surg. 2011;212:873–879. 9. Haugen AS, Softeland E, Eide GE, et al. Impact of the World Health Organization’s Surgical Safety Checklist on safety culture in the operating theatre: a controlled intervention study. Br J Anaesth. 2013;110:807–815.
C 2015 Wolters Kluwer Health, Inc. All rights reserved.
Copyright © 2015 Wolters Kluwer Health, Inc. Unauthorized reproduction of this article is prohibited.
What Do We Know About the Safe Surgery Checklist Now? Alex B. Haynes, MD, MPH,∗ † William R. Berry, MD, MPH,†‡ and Atul A. Gawande, MD, MPH‡§
I
n 2009, when The New England Journal of Medicine published the 8-city trial of the World Health Organization’s Safe Surgery Checklist, the idea that a formal system of planning and communication could significantly improve patient outcomes was outside the surgical mainstream. The pre-post study found substantial reductions in complications and deaths, with the improvements significant in both higher and lower income settings.1 However, the study had notable weaknesses. There was no control group, nor was the intervention applied on all units in the study hospitals. The program had not been demonstrated to be replicable at a large scale, and the sustainability of the effect had not been proved. A series of subsequent studies with strong control groups have since replicated the substantial value of using a systematic checklist-based approach to surgical team planning and communication. For instance, van Klei et al2 demonstrated in more than 25,000 patients that the odds ratio of 30-day mortality was 0.44 for patients whose teams completed the checklist compared with 1.16 for those whose teams did not. Beneficial findings were also demonstrated at a large scale: the SURPASS trial showed that a comprehensive surgical checklist approach reduced mortality rates by 47% in 7 Dutch hospitals compared with controls,3 and Neily et al4 demonstrated using a step-wedge methodology that introduction of a checklist-driven system of briefing and debriefing in 74 Veterans Administration (VA) hospitals through a concerted team training program led to an 18% reduction in mortality. As a result, several countries have introduced a safe surgery checklist program nationally. For instance, after several years of flat surgical inpatient mortality, NHS Scotland carried out a concerted nationwide implementation program that it credited with making major reductions in inpatient mortality that meant more than 9000 fewer lives lost.5 Recently, however, Urbach et al6 examined the effects of a law mandating surgical checklist use in Ontario hospitals and found only a modest mortality reduction that did not reach statistical significance. Given these conflicting data, a remaining concern about the validity of even the prior controlled studies was that they had not been randomized. The study by Haugen and colleagues7 in the current issue of Annals of Surgery therefore makes a major contribution by reporting the results of the first cluster randomized trial of a surgical safety checklist, in this case employing random allocation of dates of implementation of the intervention among different units within 2 hospitals in Norway. Analysis of outcomes focused on a cohort of patients with a significant rate of morbidity and in-hospital mortality (19.9% and 1.6%, respectively) and followed a step-wedge methodology. The study confirmed that use of the checklist resulted in substantial improvement in outcomes—with a relative risk reduction in major complications of 0.42 (95% confidence interval, 0.33–0.50). The number needed to treat to prevent morbidity was just 12. Deaths fell from 1.6% to 1.0%, which is potentially substantial, but the study was not powered to detect if this was a significant difference (P = 0.151). Importantly, the authors also confirmed a dose effect by assessing the actual use of the checklist through direct observation: they found an even larger reduction in major complications when teams completed all portions of the checklist—a finding consistent with that of van Klei et al.2 Such findings confirm that research on the effectiveness of quality tools such as checklists must include assessment of use, as no quality tool, particularly one that relies upon team communication and interaction, can be expected to make a difference if it is not actually used in any meaningful sense. This work also affirms emerging evidence that institution of an effective safe surgery checklist program at a large scale requires a deliberate implementation process and at least some form of monitoring and learning with frontline teams.8 Haugen and colleagues7,9 are to be complimented not only for the thoroughness of the process but also for the details on the implementation efforts shared in this article and in prior publications. They detail several precepts of their implementation that we believe may be key to success in team-based interventions. First, they modified the checklist to fit with local context. Although modification must be undertaken in a thoughtful manner, this is key to ensuring that the instrument fits the local workflow and circumstances. Second, after modification,
From the ∗ Department of Surgery, Massachusetts General Hospital, Boston, MA; †Ariadne Labs: A joint center of the Harvard School of Public Health and Brigham and Women’s Hospital, Boston, MA; ‡Department of Health Policy and Management, Harvard School of Public Health, Boston, MA; and §Department of Surgery, Brigham and Women’s Hospital, Boston, MA. Disclosure: The authors declare no conflicts of interest. Reprints: Alex B. Haynes, MD, MPH, Department of Surgery, Massachusetts General Hospital, 55 Fruit St, Yawkey 7B, Boston, MA 02114. E-mail: [email protected]. C 2015 Wolters Kluwer Health, Inc. All rights reserved. Copyright ISSN: 0003-4932/15/26105-0829 DOI: 10.1097/SLA.0000000000001144
Annals of Surgery r Volume 261, Number 5, May 2015
www.annalsofsurgery.com | 829
Copyright © 2015 Wolters Kluwer Health, Inc. Unauthorized reproduction of this article is prohibited.
Annals of Surgery r Volume 261, Number 5, May 2015
Haynes et al
the hospitals piloted the checklist at a small scale, with only a few teams, and then used the experience to make further modifications. We believe this produces both learning and a few “champions” or “super-users” who are essential to success in later phases. Third, once the checklist had been finalized, the implementation team introduced it more widely through dedicated training sessions and distribution of education materials involving all disciplines. Even a seemingly simple and intuitive checklist involves changes in practices that clinicians have to understand and become comfortable with. Finally, and equally importantly, the hospitals’ implementation group regularly observed use of the checklist during cases day-to-day and both provided to and received feedback from individual clinicians, allowing opportunities for further education and reinforcement. The rollout program by the VA and by the NHS Scotland followed similar precepts. Scotland facilitated a gradual process of developing local implementation teams that included surgeons, made local checklist modifications, carried out small-scale piloting, and led dedicated training for all disciplines involved in surgical care.5 In the VA’s program, implementation teams obtained local feedback for checklist modification and shut down nonemergency surgery for a full day of training, including practice with team communication skills and the checklist, at each facility.4 In each quarter after this training, postoperative mortality declined significantly compared with nonparticipating facilities where there was no decline. In contrast, Ontario’s government mandate was not accompanied by a sustained and systematic implementation program with these kinds of features and resulted in little measurable change in outcomes, at least in a short-term evaluation.6 In sum, Haugen and colleagues have made an invaluable contribution. Using a cluster randomized control methodology, a carefully structured implementation program, and measurement of the actual use of the checklist, they conducted the most rigorous study of surgical safety checklists to date. They confirmed substantial patient benefit from taking this kind of formalized checklist approach
830 | www.annalsofsurgery.com
to team planning and communication in the operating room, demonstrating marked reductions in complications and possibly mortality. Not insignificantly, they detailed the elements of an effective implementation of a hospital-wide quality improvement program, specifics of which are often overlooked in manuscripts in favor of statistical methodology. Their work should be an example for surgical teams everywhere and a model for publication of quality improvement trials.
REFERENCES 1. Haynes AB, Weiser TG, Berry WR, et al. A surgical safety checklist to reduce morbidity and mortality in a global patient population. N Engl J Med. 2009;350:491–499. 2. van Klei WA, Hoff RG, van Aarnhem EE, et al. Effects of the introduction of the WHO “Surgical Safety Checklist” on in-hospital mortality: a cohort study. Ann Surg. 2012;255:44–49. 3. de Vries EN, Prins HA, Crolla RM, et al. Effect of a comprehensive surgical safety system on patient outcomes. N Engl J Med. 2010;363:1928–1937. 4. Neily J, Mills PD, Young-Xu Y, et al. Association between implementation of a medical team training program and surgical mortality. JAMA. 2010;304:1693– 1700. 5. Scotland National Health Service. The healthcare quality strategy for the NHS Scotland. Report at the International Society for Quality in Healthcare, 24 October 2013. Available at: http://www.isqua.org/docs/ edinburgh-powerpoint-presentations-2013/1545-afternoon-plenary-jason-leitch -pentland-mon-scotlands-quality-journey.pdf?sfvrsn=2. Accessed December 15, 2014. 6. Urbach DR, Govindarajan A, Saskin R, et al. Introduction of surgical safety checklists in Ontario, Canada. N Engl J Med. 2014;370:1029–1038. 7. Haugen AS, Softeland E, Almeland SK, et al. Effect of the World Health Organization checklist on patient outcomes: a stepped wedge cluster randomized controlled trial. Ann Surg. 2015;261:821–828. 8. Conley DM, Singer SJ, Edmondson L, et al. Effective surgical safety checklist implementation. J Am Coll Surg. 2011;212:873–879. 9. Haugen AS, Softeland E, Eide GE, et al. Impact of the World Health Organization’s Surgical Safety Checklist on safety culture in the operating theatre: a controlled intervention study. Br J Anaesth. 2013;110:807–815.
C 2015 Wolters Kluwer Health, Inc. All rights reserved.
Copyright © 2015 Wolters Kluwer Health, Inc. Unauthorized reproduction of this article is prohibited.
