Randomized clinical trial
Randomized clinical trial of the impact of surgical ward-care checklists on postoperative care in a simulated environment P. H. Pucher1 , R. Aggarwal1,2 , M. Qurashi1 , P. Singh1 and A. Darzi1 1
Department of Surgery and Cancer, St Mary’s Hospital, Imperial College London, London, UK, and 2 Department of Surgery, Hospital of the University of Pennsylvania, Perelman School of Medicine, Philadelphia, Pennsylvania, USA Correspondence to: Dr P. H. Pucher, Department of Surgery and Cancer, 10th Floor QEQM Building, St Mary’s Hospital, Praed Street, London W2 1NY, UK (e-mail: [email protected])
Background: Complications are a common and accepted risk of surgery. Failure to optimize the
management of patients who suffer postoperative morbidity may result in poorer surgical outcomes. This study aimed to evaluate a checklist-based tool to improve and standardize care of postoperative complications. Methods: Surgical trainees conducted baseline ward rounds of three patients with common postoperative complications in a high-fidelity simulated ward environment. Subjects were randomized to intervention or control groups, and final ward rounds were conducted with or without the aid of checklists for management of postoperative complications. Adherence to critical care processes was assessed, in addition to technical (Surgical Ward-care Assessment Tool, SWAT) and non-technical (Ward NOn-TECHnical Skills (W-NOTECHS) scale) performance. Subjects completed a feedback questionnaire regarding their perception of the checklists. Results: Twenty trainees completed 120 patient assessments. All intervention group subjects opted to use the checklists, resulting in significantly fewer critical errors compared with controls (median (i.q.r.) 0 (0–0) versus 60 (40–73) per cent; P < 0⋅001). The intervention group demonstrated improved patient management (SWAT-M) (P < 0⋅001) and non-technical skills (P = 0⋅043) between baseline and final ward rounds, whereas controls did not (P = 0⋅571 and P = 0⋅809 respectively). A small learning effect was seen with improvement in patient assessment (SWAT-A) in both groups (P < 0⋅001). Intervention group subjects found checklists easy and effective to use, and would want them used for their own care if they were to experience postoperative complications. Conclusion: Checklist use resulted in significantly improved standardization, evidence-based management of postoperative complications, and quality of ward rounds. Simulation-based piloting aided appropriate use of checklists and staff engagement. Checklists represent a low-cost intervention to reduce rates of failure to rescue and to improve patient care. Paper accepted 20 August 2014 Published online 28 October 2014 in Wiley Online Library (www.bjs.co.uk). DOI: 10.1002/bjs.9654
Introduction
Complications are frequent occurrences after surgery, with reported morbidity rates following major gastrointestinal procedures of 30–50 per cent1 – 4 . The most frequently reported postoperative complications, such as pneumonia or wound infection, are common to patients undergoing gastrointestinal surgery, irrespective of the specific procedure4 – 6 . Despite their relatively common nature, published evidence7,8 suggests a high degree of variability in the management of postoperative complications. Although several associated structural factors, such as nurse staffing © 2014 BJS Society Ltd Published by John Wiley & Sons Ltd
levels, have been identified9 – 11 , these fail to account for the majority of variability seen in surgical outcomes. Process-level failures, at the point of interaction between the patient and clinician, have been investigated less thoroughly. Postoperative patient assessment and the surgical ward round are subject to pressures of time, resource limitation and the prioritization of other clinical activities12 . Coupled with natural tendencies for human error, the ward-based phase of surgical care is rife with potential for adverse events13 . Checklists are seen as one of the most effective means to combat medical error14,15 . As mental BJS 2014; 101: 1666–1673
Surgical checklists and postoperative care
prompts and physical reminders, checklists serve to standardize process-led care. They have been adopted widely in recent years, with their use becoming the accepted standard of care in many areas of surgical care16 – 18 . Evidence also suggests that checklists may act beyond the included processes, to improve safety culture as a whole, through wider individual involvement and lessening traditional clinical hierarchies19,20 . At the same time, the wider uptake of checklists has served to highlight some of the potential pitfalls that may be encountered when attempting to change established process and clinical culture21 . The clinical effectiveness of checklists is dependent on more than the content of the checklist alone. Without effective staff involvement and training, implementation of such interventions may be, at best, ineffective22 or, at worst, a source of staff frustration and discord23 . To address this issue, interventions have been shown to benefit from initial trialling in simulated environments to assess feasibility and clinical effectiveness, before full clinical implementation24 . Such an approach benefits from the known advantages of simulation, with controlled and reliable scenarios within a dedicated assessment space and elimination of potential risks to patients. Patients who suffer complications in the postoperative phase are the group most at risk of adverse events13,25 and vulnerable to variations in their care7 . In the management of postoperative morbidity, failures to adhere to principles of best practice place unwell patients at risk of avoidable harm and poorer outcomes (unpublished work, Pucher et al.). Adherence to care process-oriented guidelines, such as expert body recommendations26 or antimicrobial stewardship policies27 , may contribute significantly to standardization of care and optimization of patient management. The aim of this study was to assess the clinical effectiveness of checklists in the management of common postoperative complications within a high-fidelity simulated ward environment. Methods
General surgical registrars were recruited from a single academic hospital. Registrar-level trainees (with a minimum of 2 completed years of specialist training in general surgery) were selected, to reflect a level at which trainees can be expected to assume responsibility for routine patient assessment during the ward round in typical practice in the UK28 . A sample size calculation was conducted, assuming a study powered to 90 per cent and an α level of 0⋅05. As this represented an exploratory study, with no available previous data for identical outcomes, published rates of © 2014 BJS Society Ltd Published by John Wiley & Sons Ltd
1667
surgical ward round error were used25 , with the presumed effect size based on data from a previous study24 of surgical process checklists in the operating theatre. This suggested a necessary sample size of only three subjects in each group. Given the surrogate nature of data used, this was increased to ten to maximize data collection and increase the validity of results, based on experience from previous ward simulation-based trials29 . Simulated ward rounds took place within a three-bed simulated ward. This was a fully furnished, high-fidelity simulation environment, complete with patient monitoring equipment and access to simulated patient records, radiology and pathology results (Fig. 1). The environment has been validated and described in detail previously29,30 . Professional medical actors were employed to simulate patients and exhibit realistic clinical signs. An integrated audiovisual system recorded all activity digitally, with realistic background ‘white noise’ recorded from a real clinical environment. Trainees were exposed to a series of simulated scenarios in which patients experienced one of the following postoperative complications: pneumonia, anastomotic leak, wound infection, urinary tract infection, postoperative bleeding and intra-abdominal sepsis. These complications were identified as the most common types following major gastrointestinal surgery, with subsequent development of checklists through expert input, best available evidence and end-user input (unpublished work, Pucher et al.). All trainees conducted a baseline ward round of three patients within the simulated ward, according to personal practice (Fig. 2). Ward rounds were supported by a nurse and a junior doctor trained to provide a standardized level of information relating to patients as part of the simulation, if prompted or asked by the trainee. Trainees were blinded to study design and assessment endpoints. To create a realistic sense of time pressure, trainees were told they were required to finish each ward round within 30 min, based on previous data from similar scenarios29 . Patient scenarios assigned to each trainee’s ward round were determined through a computer-generated randomization process, with all six scenarios being assessed by each trainee over the course of the study. Following the baseline ward round assessment, each trainee was randomized (using a further computergenerated randomization sequence) to either a control or an intervention group. Trainees in the intervention group received a didactic session, with written and verbal instruction, on use of the previously developed checklists for management of surgical complications (unpublished work, Pucher et al.), then completed their final ward round, for which the checklists were made available for use (Appendix www.bjs.co.uk
BJS 2014; 101: 1666–1673
1668
Fig. 1
P. H. Pucher, R. Aggarwal, M. Qurashi, P. Singh and A. Darzi
High-fidelity simulated ward environment Trainees recruited n = 20
Baseline ward round assessment (3 patients per ward round)
Randomized to control n = 10
Randomized to intervention n = 10
Checklist training n = 10
Second simulated ward round (3 patients per ward round) n = 10
Second simulated ward round (3 patients per ward round) n = 10
Feedback questionnaire n = 10 Fig. 2
CONSORT flow diagram for the study
S1, supporting information). Trainees were informed that use of the checklists was not compulsory, and need be used only if the trainee was comfortable with them or felt they would help improve care given. Trainees in the control group conducted a final ward round according to their own personal practice, thus acting as controls for any potential learning effects resulting from repeated exposure to the simulator. Baseline and final ward rounds were conducted in separate sessions on different days. The primary endpoint for the study was the rate of failure to execute critical management steps for each © 2014 BJS Society Ltd Published by John Wiley & Sons Ltd
postoperative complication. Each checklist involved five to eight management processes, such as the ordering of blood cultures and administration of antibiotics for sepsis, or arranging a blood transfusion and preparing a patient for return to the operating room in case of postoperative haemorrhage (unpublished work, Pucher et al.). Completion or omission of each checklist item was recorded in a binary manner through direct observation. Secondary endpoints included technical and non-technical performance. Technical performance was assessed using the Surgical Ward care Assessment Tool (SWAT), a checklist-based, validated tool to assess thoroughness of patient assessment (SWAT-A) and management (SWAT-M)25,29 . Non-technical performance was assessed using the Ward NOn-TECHnical Skills (W-NOTECHS) scale, assessing five behavioural domains on a Likert scale, resulting in a final score of 5–2529,31 . Prescribing errors were recorded. Each ward round was recorded digitally using two integrated ceiling cameras, with 20 per cent of ward rounds rated by a second blinded rater to assess potential for observer bias. Inter-rater reliability was assessed using Cronbach’s α. Trainees in the intervention group were surveyed regarding their perceptions of the checklists, with questionnaire responses graded from 1 (strongly disagree) to 5 (strongly agree) on a Likert scale.
Statistical analysis Demographic and ward round performance data for failure rate, SWAT, W-NOTECHS and time taken was compared www.bjs.co.uk
BJS 2014; 101: 1666–1673
Surgical checklists and postoperative care
1669
between groups, in addition to within-group comparison of baseline and final ward rounds. Non-parametric tests were used, with Mann–Whitney U and Wilcoxon signed-rank tests for intergroup and intragroup comparisons respectively. The χ2 test was used to compare categorical data. Secondary post hoc analyses were performed to assess potentially confounding factors. To investigate differences in management of different complications, failure rates were stratified by scenario and differences in means assessed using ANOVA. A linear regression model was used to adjust for trainee factors (sex and years of experience) when interpreting the effect of trial group (control versus intervention) on failure rates. Questionnaire responses were collated and reported as mean(s.d.). Responses given as a 4 or 5 on a Likert scale were considered positive. Data analysis was performed using SPSS® version 21 (IBM, Armonk, New York, USA), with P < 0⋅050 considered statistically significant. All results are reported as median (i.q.r.) unless indicated otherwise.
Results
80
80
a
P
9:1 3 (2–4)
7:3 3 (2–4)
0⋅264† 0⋅796‡
60
40
20
20
0
Intervention (n = 10)
Of all ward rounds, 20 per cent were assessed by a second blinded assessor, with excellent inter-rater reliability (Cronbach α = 0⋅893). Considering the rates of failure to complete critical processes in the management of postoperative complications, there was no difference between control and intervention groups for baseline assessment (58 (43–75) versus 67 (50–67) per cent respectively; P = 0⋅988) (Fig. 3). For the final ward round, process failure rates were significantly different, with near-elimination of errors in the intervention group (0 (0–0) per cent versus 60 (40–73) per cent for the control group; P < 0⋅001). All trainees in the intervention group elected to use the checklists where these were made available, for the final ward round. There were no statistically significant differences between groups at baseline for SWAT-A, SWAT-M or WNOTECHS metrics (Table 2). In the final ward round, the intervention group scored significantly higher than the control group in patient management (SWAT-M: control 0⋅63 (0⋅50–0⋅75) versus intervention 0⋅83 (0⋅63–0⋅88); P = 0⋅005), although not for patient assessment (SWAT-A, P = 0⋅094) or non-technical skill (W-NOTECHS, 100
40
Control (n = 10)
*Values are median (i.q.r.). †χ2 test; ‡Mann–Whitney U test.
100
60
Demographics of subjects and controls
Sex ratio (M : F) Length of surgical training (years)*
Failure rate (%)
Failure rate (%)
Twenty trainees were recruited, resulting in 120 patient assessments in the final analysis. Each trainee completed all six patient scenarios, randomized equally to control and intervention groups. All intervention group trainees elected to use the checklists in their final ward rounds. There were no differences in sex or experience levels of participants between groups (Table 1).
Table 1
Control
Baseline assessment
0
Intervention
b
Control (no checklist)
Intervention (with checklist)
Final assessment
Failure to adhere to critical processes in the management of postoperative complications. Box plot comparisons of performance between control and intervention groups for a baseline and b final ward rounds. Thick bar denotes median value, box denotes i.q.r. and whiskers show the range; small circles indicate outliers. a P = 0⋅988, b P < 0⋅001 (Mann–Whitney U test)
Fig. 3
© 2014 BJS Society Ltd Published by John Wiley & Sons Ltd
www.bjs.co.uk
BJS 2014; 101: 1666–1673
1670
Table 2
P. H. Pucher, R. Aggarwal, M. Qurashi, P. Singh and A. Darzi
Simulated ward round performance metrics Baseline assessment
SWAT-A Intragroup P† SWAT-M Intragroup P† W-NOTECHS Intragroup P† Ward round time (min)* Intragroup P† Prescription errors Intragroup P†
Final assessment
Control (n = 30)
Intervention (n = 30)
P‡
0⋅62 (0⋅55–0⋅81)
0⋅73 (0⋅64–0⋅81)
0⋅436
0⋅63 (0⋅50–0⋅75)
0⋅56 (0⋅46–0⋅67)
0⋅350
19⋅5 (17–22)
17⋅0 (16–20)
0⋅190
26⋅5(5⋅1)
26⋅8(5⋅1)
0⋅900
10
22
0⋅019
Control (no checklist) (n = 30)
Intervention (checklist) (n = 30)
0⋅74 (0⋅64–0⋅82) < 0⋅001 0⋅63 (0⋅50–0⋅75) 0⋅571 19⋅0 (18⋅5–21) 0⋅809 23⋅3(4⋅2) 0⋅349 15 0⋅298
0⋅77 (0⋅73–0⋅91) < 0⋅001 0⋅83 (0⋅63–0⋅88) < 0⋅001 20⋅5 (18⋅5–23) 0⋅043 22⋅2(5⋅7) 0⋅111 5 0⋅020
P‡ 0⋅094 0⋅005 0⋅218 0⋅730 0⋅043
Values are median (i.q.r.), except *mean(s.d.). SWAT(-A/M), Surgical Ward-care Assessment Tool (-Assessment/Management); W-NOTECHS, Ward NOn-TECHnical Skills scale. †Final versus baseline assessment (Wilcoxon signed-rank test); ‡Mann–Whitney U test.
Baseline ward round performance stratified by clinical scenario
Table 3
Responses to use of checklists for management of postoperative complications
Table 4
Failure rate (%) Questionnaire statement Pneumonia Wound infection Anastomotic leak Sepsis Urinary tract infection Postoperative haemorrhage
65(26) 58(13) 61(13) 61(13) 70(33) 68(18)
Values are mean(s.d.). P = 0⋅796 (ANOVA).
P = 0⋅218). There was no significant difference in the time taken for each ward round between groups or between ward rounds. In terms of intragroup comparisons, a small but significant improvement was seen in both groups for SWAT-A (both P < 0⋅001). Regarding patient management (SWAT-M), improvement was seen only in the intervention group (P < 0⋅001). Similarly, total W-NOTECHS scores improved only in the intervention group (P = 0⋅043; control group, P = 0⋅809). There were no significant differences in median values for individual W-NOTECHS domain scores. Although the intervention group had a higher number of prescription errors recorded in the baseline round (22 versus 10 in the control group; P = 0⋅019), the use of checklists resulted in a significant reduction in the final round (5 versus 15 errors respectively; P = 0⋅043) (Table 2). There were no differences in performance between scenarios (ANOVA for baseline ward round, P = 0⋅796) (Table 3). Controlling for trainee experience and sex through inclusion as independent variables in a regression model had no effect on results, with trial group allocation the only significant predictor of process failure rates (odds ratio 0⋅54, 95 per cent c.i. 0⋅46 to 0⋅62; P < 0⋅001). © 2014 BJS Society Ltd Published by John Wiley & Sons Ltd
The checklists were easy to use The checklists cover clinical conditions relevant to my practice I found this checklist applicable to my practice The checklists may improve management of complications I would want this checklist to be used for me if I experienced a postoperative complication I would consider using the checklists in my routine practice
Response score*
Positive response (%)
4⋅7 (4–5) 4⋅8 (4–5)
100 100
4⋅1 (3–5)
80
4⋅4 (4–5)
100
4⋅3 (4–5)
100
4⋅1 (3–5)
80
*Values are mean (range). Responses were received from ten of ten subjects in the intervention group.
Questionnaire responses All intervention group study participants returned completed questionnaires. Responses were positive, with all believing that checklists were easy to use and would improve clinical practice (Table 4). All participants indicated that they would want the checklists used if they themselves were to experience a postoperative complication. Discussion
Despite their common nature, wide variations in the management of postoperative complications exist across different centres2,8,32 – 35 . Checklists are being adopted increasingly in routine surgical care36 , but no such clinical aids have been available for the non-routine care of patients suffering postoperative complications. The results of the present study demonstrate not only the significant variability that exists in the management www.bjs.co.uk
BJS 2014; 101: 1666–1673
Surgical checklists and postoperative care
1671
of common complications, but how error and variation in clinical care may be eliminated essentially through a single intervention. The introduction of checklists to guide patient management, once the diagnosis of a postoperative complication had been made and a decision taken to treat, led to the standardization of treatment in line with evidence-based guidelines, and reduction of median error rates from 60 to 0 per cent. Despite their simple nature, or perhaps because of it, implementing checklists in the manner of this study has the potential to reduce the variability of ward-based care25 and may act significantly to reduce failure-to-rescue rates and improve outcomes. The experimental, simulated context in which this study was conducted represents a potential confounder in the interpretation of results, although the overwhelmingly positive effect is in agreement with previously published data16,24 . The introduction of checklist-based protocols for antiseptic central venous catheter placement eliminated catheter-associated sepsis over the course of 3 months16 . A simulation-based trial assessing the management of operating room crises reported a reduction in error rates of 75 per cent when checklists were used to guide the operating room team’s response to rare events such as asystole, malignant hyperthermia or unstable arrhythmia24 . Patient management without the aid of checklists placed patients at risk of critical errors such as the failure to prescribe antibiotics for a patient with an anastomotic leak, insufficient intravenous access in a patient suffering from haemorrhage, or failure to obtain blood cultures in a patient with sepsis and pyrexia. Checklist use led to near elimination of these errors, standardizing care without compromising clinicians’ decision-making or patient care. It is important to note that the checklists were introduced as aids to guide care only, with clinicians free to use them (or not) as they felt appropriate, dependent upon the suspected diagnosis and intended treatment. Although they represented an additional piece of paperwork, the checklists were well received, with all reporting that they were easy to use, and improved their practice. Checklist use did not prolong the ward rounds, with no significant difference in time taken between groups. Perhaps most significantly, all trainees reported that they would want checklists used in their own care, were they to require care for postoperative complications themselves. Introduction of the checklist also led to improvements in non-technical skill. The ability of checklists to flatten clinical hierarchies and improve communication has been seen previously in improved safety attitudes19 and staff questionnaire responses37 , but this is the first time that the effect of a checklist on non-technical performance has been observed directly and measured in a clinical context.
The positive feedback seen in this study represents a crucial aspect of checklist implementation, illustrating the benefit of simulation-based trialling of such an intervention before introduction into clinical practice. Centres have reported significant difficulties associated with attempts to change established clinical process21 – 23 . Negative feedback captured in the simulation can therefore be addressed before clinical implementation. This study, intended as an exploratory study of the effectiveness of checklists for postoperative care, must be considered in the context of its limitations. Only the performance of surgical trainees was assessed. Other health professionals, including nursing staff and other grades of medical staff, were not included. As registrars are the group of surgical trainees commonly responsible for routine patient review, and most likely to make the first diagnosis of a postoperative complication and initiate treatment, this is unlikely to represent a major confounder. Interaction with other members of the clinical team, although important, would be secondary to the clinician’s diagnosis and decision to treat. This was a single-centre study of a simulation-based trial. Localization and adaptation to local protocols and practices is a critical component of successful checklist implementation18,38 , and adapted checklists would be required for individual centres. The study did not evaluate the longevity or retention of changes to practice, which may experience degradation of checklist use over time. Other groups implementing checklist-type interventions have variably reported both deterioration39 and improvement16 of performance over time. Analogous to the acquisition of technical skill through repeated training40 , it may be that repeated training sessions should be considered for such non-technical interventions. Further longitudinal assessments of postoperative checklists would be required. Finally, it must be recognized that the management of postoperative complications is a two-stage process, requiring the correct diagnosis first to be made before appropriate management can be instituted. Checklists address the latter, but not the former7,8 . Although management of a given condition is amenable to checklist-based care, based upon best evidence and expert guidelines, diagnostic ability is dependent on a clinician’s own ability and practice. To improve patient assessment and the recognition of complications, broader interventions, such as educational curricula to improve trainees’ conduct of ward rounds in general, can be effective41,42 . This study has demonstrated the viability of checklists in the management of postoperative complications. Future work should include a clinical implementation study. As seen in this and other checklist studies17,24 , steps to ensure staff participation and ‘buy-in’ are essential to success with
© 2014 BJS Society Ltd Published by John Wiley & Sons Ltd
www.bjs.co.uk
BJS 2014; 101: 1666–1673
1672
the involvement of local ‘champions’, regular audit and performance feedback, as well as simulation-based trialling and staff training to ensure appropriate use. Development of checklists for further complications, such as pulmonary embolism or deep vein thrombosis, might also be considered. Considering the common nature of complications, improving the care of patients with postoperative morbidity has perhaps the greatest potential for impact in reducing failure-to-rescue rates; the use of appropriate checklists may standardize treatment and improve outcomes. Disclosure
The authors declare no conflict of interest. References 1 Ito H, Are C, Gonen M, D’Angelica M, Dematteo RP, Kemeny NE et al. Effect of postoperative morbidity on long-term survival after hepatic resection for metastatic colorectal cancer. Ann Surg 2008; 247: 994–1002. 2 Ghaferi AA, Birkmeyer JD, Dimick JB. Hospital volume and failure to rescue with high-risk surgery. Med Care 2011; 49: 1076–1081. 3 Hii MW, Smithers BM, Gotley DC, Thomas JM, Thomson I, Martin I et al. Impact of postoperative morbidity on long-term survival after oesophagectomy. Br J Surg 2013; 100: 95–104. 4 Merkow RP, Bilimoria KY, McCarter MD, Phillips JD, Decamp MM, Sherman KL et al. Short-term outcomes after esophagectomy at 164 American College of Surgeons National Surgical Quality Improvement Program Hospitals: effect of operative approach and hospital-level variation. Arch Surg 2012; 147: 1009–1016. 5 Cone MM, Herzig DO, Diggs BS, Rea JD, Hardiman KM, Lu KC. Effect of surgical approach on 30-day mortality and morbidity after elective colectomy: a NSQIP study. J Gastrointest Surg 2012; 16: 1212–1217. 6 Molena D, Mungo B, Stem M, Feinberg RL, Lidor AO. Outcomes of esophagectomy for esophageal achalasia in the United States. J Gastrointest Surg 2014; 18: 310–317. 7 Ghaferi AA, Birkmeyer JD, Dimick JB. Variation in hospital mortality associated with inpatient surgery. N Engl J Med 2009; 361: 1368–1375. 8 Ghaferi AA, Birkmeyer JD, Dimick JB. Complications, failure to rescue, and mortality with major inpatient surgery in medicare patients. Ann Surg 2009; 250: 1029–1034. 9 Aiken LH, Clarke SP, Sloane DM, Sochalski J, Silber JH. Hospital nurse staffing and patient mortality, nurse burnout, and job dissatisfaction. JAMA 2002; 288: 1987–1993. 10 Friese CR, Earle CC, Silber JH, Aiken LH. Hospital characteristics, clinical severity, and outcomes for surgical oncology patients. Surgery 2010; 147: 602–609.
© 2014 BJS Society Ltd Published by John Wiley & Sons Ltd
P. H. Pucher, R. Aggarwal, M. Qurashi, P. Singh and A. Darzi
11 Ghaferi AA, Osborne NH, Birkmeyer JD, Dimick JB. Hospital characteristics associated with failure to rescue from complications after pancreatectomy. J Am Coll Surg 2010; 211: 325–330. 12 Creamer GL, Dahl A, Perumal D, Tan G, Koea JB. Anatomy of the ward round: the time spent in different activities. ANZ J Surg 2010; 80: 930–932. 13 Vincent C, Neale G, Woloshynowych M. Adverse events in British hospitals: preliminary retrospective record review. BMJ 2001; 322: 517–519. 14 Reason J. Human error: models and management. BMJ 2000; 320: 768–770. 15 Reason J. Combating omission errors through task analysis and good reminders. Qual Saf Health Care 2002; 11: 40–44. 16 Pronovost P, Needham D, Berenholtz S, Sinopoli D, Chu H, Cosgrove S et al. An intervention to decrease catheter-related bloodstream infections in the ICU. N Engl J Med 2006; 355: 2725–2732. 17 Haynes AB, Weiser TG, Berry WR, Lipsitz SR, Breizat AH, Dellinger EP et al.; Safe Surgery Saves Lives Study Group. A surgical safety checklist to reduce morbidity and mortality in a global population. N Engl J Med 2009; 360: 491–499. 18 de Vries EN, Prins HA, Crolla RM, den Outer AJ, van Andel G, van Helden SH et al.; SURPASS Collaborative Group. Effect of a comprehensive surgical safety system on patient outcomes. N Engl J Med 2010; 363: 1928–1937. 19 Haynes AB, Weiser TG, Berry WR, Lipsitz SR, Breizat AH, Dellinger EP et al.; Safe Surgery Saves Lives Study Group. Changes in safety attitude and relationship to decreased postoperative morbidity and mortality following implementation of a checklist-based surgical safety intervention. BMJ Qual Saf 2011; 20: 102–107. 20 Russ S, Rout S, Sevdalis N, Moorthy K, Darzi A, Vincent C. Do safety checklists improve teamwork and communication in the operating room? A systematic review. Ann Surg 2013; 258: 856–871. 21 Borchard A, Schwappach DL, Barbir A, Bezzola P. A systematic review of the effectiveness, compliance, and critical factors for implementation of safety checklists in surgery. Ann Surg 2012; 256: 925–933. 22 Spence J, Goodwin B, Enns C, Dean H. Student-observed surgical safety practices across an urban regional health authority. BMJ Qual Saf 2011; 20: 580–586. 23 Conley DM, Singer SJ, Edmondson L, Berry WR, Gawande AA. Effective surgical safety checklist implementation. J Am Coll Surg 2011; 212: 873–879. 24 Arriaga AF, Bader AM, Wong JM, Lipsitz SR, Berry WR, Ziewacz JE et al. Simulation-based trial of surgical-crisis checklists. N Engl J Med 2013; 368: 246–253. 25 Pucher PH, Aggarwal R, Darzi A. Surgical ward round quality and impact on variable patient outcomes. Ann Surg 2014; 259: 222–226. 26 Hooton TM, Bradley SF, Cardenas DD, Colgan R, Geerlings SE, Rice JC et al.; Infectious Diseases Society of America. Diagnosis, prevention, and treatment of catheter-associated urinary tract infection in adults: 2009
www.bjs.co.uk
BJS 2014; 101: 1666–1673
Surgical checklists and postoperative care
27
28
29
30 31
32
33
34
35
International Clinical Practice Guidelines from the Infectious Diseases Society of America. Clin Infect Dis 2010; 50: 625–663. Evans RS, Pestotnik SL, Classen DC, Clemmer TP, Weaver LK, Orme JF Jr et al. A computer-assisted management program for antibiotics and other antiinfective agents. N Engl J Med 1998; 338: 232–238. Intercollegiate Surgical Curriculum Programme. General Surgery Specialty Syllabus; 2013. http://www.iscp.ac.uk [accessed 1 April 2014]. Pucher PH, Aggarwal R, Srisatkunam T, Darzi A. Validation of the simulated ward environment for assessment of ward-based surgical care. Ann Surg 2014; 259: 215–221. Pucher PH, Darzi A, Aggarwal R. Simulation for ward processes of surgical care. Am J Surg 2013; 206: 96–102. Steinemann S, Berg B, Ditullio A, Skinner A, Terada K, Anzelon K et al. Assessing teamwork in the trauma bay: introduction of a modified ‘NOTECHS’ scale for trauma. Am J Surg 2012; 203: 69–75. Glance LG, Dick AW, Meredith JW, Mukamel DB. Variation in hospital complication rates and failure-to-rescue for trauma patients. Ann Surg 2011; 253: 811–816. McHugh MD, Kelly LA, Smith HL, Wu ES, Vanak JM, Aiken LH. Lower mortality in magnet hospitals. Med Care 2013; 51: 382–388. Almoudaris AM, Mamidanna R, Bottle A, Aylin P, Vincent C, Faiz O et al. Failure to rescue patients after reintervention in gastroesophageal cancer surgery in England. JAMA Surg 2013; 148: 272–276. Henneman D, Snijders HS, Fiocco M, van Leersum NJ, Kolfschoten NE, Wiggers T et al. Hospital variation in
1673
36
37
38
39
40
41
42
failure to rescue after colorectal cancer surgery: results of the Dutch Surgical Colorectal Audit. Ann Surg Oncol 2013; 20: 2117–2123. Adamina M, Kehlet H, Tomlinson GA, Senagore AJ, Delaney CP. Enhanced recovery pathways optimize health outcomes and resource utilization: a meta-analysis of randomized controlled trials in colorectal surgery. Surgery 2011; 149: 830–840. Blanco M, Clarke JR, Martindell D. Wrong site surgery near misses and actual occurrences. AORN J 2009; 90: 215–218, 221–212. Sparks EA, Wehbe-Janek H, Johnson RL, Smythe WR, Papaconstantinou HT. Surgical Safety Checklist compliance: a job done poorly! J Am Coll Surg 2013; 217: 867–873.e1–e3. Vats A, Vincent CA, Nagpal K, Davies RW, Darzi A, Moorthy K. Practical challenges of introducing WHO surgical checklist: UK pilot experience. BMJ 2010; 340: b5433. Crochet P, Aggarwal R, Dubb SS, Ziprin P, Rajaretnam N, Grantcharov T et al. Deliberate practice on a virtual reality laparoscopic simulator enhances the quality of surgical technical skills. Ann Surg 2011; 253: 1216–1222. Pucher PH, Aggarwal R, Singh P, Srisatkunam T, Twaij A, Darzi A. Ward simulation to improve surgical ward round performance: a randomised controlled trial of a simulation-based curriculum. Ann Surg 2014; [Epub ahead of print]. Pucher PH, Darzi A, Aggarwal R. Development of an evidence-based curriculum for training of ward-based surgical care. Am J Surg 2014; 207: 213–217.
Supporting information
Additional supporting information may be found in the online version of this article: Appendix S1 Example of the postoperative surgical checklist for hospital care (Word document)
© 2014 BJS Society Ltd Published by John Wiley & Sons Ltd
www.bjs.co.uk
BJS 2014; 101: 1666–1673
Randomized clinical trial of the impact of surgical ward-care checklists on postoperative care in a simulated environment P. H. Pucher1 , R. Aggarwal1,2 , M. Qurashi1 , P. Singh1 and A. Darzi1 1
Department of Surgery and Cancer, St Mary’s Hospital, Imperial College London, London, UK, and 2 Department of Surgery, Hospital of the University of Pennsylvania, Perelman School of Medicine, Philadelphia, Pennsylvania, USA Correspondence to: Dr P. H. Pucher, Department of Surgery and Cancer, 10th Floor QEQM Building, St Mary’s Hospital, Praed Street, London W2 1NY, UK (e-mail: [email protected])
Background: Complications are a common and accepted risk of surgery. Failure to optimize the
management of patients who suffer postoperative morbidity may result in poorer surgical outcomes. This study aimed to evaluate a checklist-based tool to improve and standardize care of postoperative complications. Methods: Surgical trainees conducted baseline ward rounds of three patients with common postoperative complications in a high-fidelity simulated ward environment. Subjects were randomized to intervention or control groups, and final ward rounds were conducted with or without the aid of checklists for management of postoperative complications. Adherence to critical care processes was assessed, in addition to technical (Surgical Ward-care Assessment Tool, SWAT) and non-technical (Ward NOn-TECHnical Skills (W-NOTECHS) scale) performance. Subjects completed a feedback questionnaire regarding their perception of the checklists. Results: Twenty trainees completed 120 patient assessments. All intervention group subjects opted to use the checklists, resulting in significantly fewer critical errors compared with controls (median (i.q.r.) 0 (0–0) versus 60 (40–73) per cent; P < 0⋅001). The intervention group demonstrated improved patient management (SWAT-M) (P < 0⋅001) and non-technical skills (P = 0⋅043) between baseline and final ward rounds, whereas controls did not (P = 0⋅571 and P = 0⋅809 respectively). A small learning effect was seen with improvement in patient assessment (SWAT-A) in both groups (P < 0⋅001). Intervention group subjects found checklists easy and effective to use, and would want them used for their own care if they were to experience postoperative complications. Conclusion: Checklist use resulted in significantly improved standardization, evidence-based management of postoperative complications, and quality of ward rounds. Simulation-based piloting aided appropriate use of checklists and staff engagement. Checklists represent a low-cost intervention to reduce rates of failure to rescue and to improve patient care. Paper accepted 20 August 2014 Published online 28 October 2014 in Wiley Online Library (www.bjs.co.uk). DOI: 10.1002/bjs.9654
Introduction
Complications are frequent occurrences after surgery, with reported morbidity rates following major gastrointestinal procedures of 30–50 per cent1 – 4 . The most frequently reported postoperative complications, such as pneumonia or wound infection, are common to patients undergoing gastrointestinal surgery, irrespective of the specific procedure4 – 6 . Despite their relatively common nature, published evidence7,8 suggests a high degree of variability in the management of postoperative complications. Although several associated structural factors, such as nurse staffing © 2014 BJS Society Ltd Published by John Wiley & Sons Ltd
levels, have been identified9 – 11 , these fail to account for the majority of variability seen in surgical outcomes. Process-level failures, at the point of interaction between the patient and clinician, have been investigated less thoroughly. Postoperative patient assessment and the surgical ward round are subject to pressures of time, resource limitation and the prioritization of other clinical activities12 . Coupled with natural tendencies for human error, the ward-based phase of surgical care is rife with potential for adverse events13 . Checklists are seen as one of the most effective means to combat medical error14,15 . As mental BJS 2014; 101: 1666–1673
Surgical checklists and postoperative care
prompts and physical reminders, checklists serve to standardize process-led care. They have been adopted widely in recent years, with their use becoming the accepted standard of care in many areas of surgical care16 – 18 . Evidence also suggests that checklists may act beyond the included processes, to improve safety culture as a whole, through wider individual involvement and lessening traditional clinical hierarchies19,20 . At the same time, the wider uptake of checklists has served to highlight some of the potential pitfalls that may be encountered when attempting to change established process and clinical culture21 . The clinical effectiveness of checklists is dependent on more than the content of the checklist alone. Without effective staff involvement and training, implementation of such interventions may be, at best, ineffective22 or, at worst, a source of staff frustration and discord23 . To address this issue, interventions have been shown to benefit from initial trialling in simulated environments to assess feasibility and clinical effectiveness, before full clinical implementation24 . Such an approach benefits from the known advantages of simulation, with controlled and reliable scenarios within a dedicated assessment space and elimination of potential risks to patients. Patients who suffer complications in the postoperative phase are the group most at risk of adverse events13,25 and vulnerable to variations in their care7 . In the management of postoperative morbidity, failures to adhere to principles of best practice place unwell patients at risk of avoidable harm and poorer outcomes (unpublished work, Pucher et al.). Adherence to care process-oriented guidelines, such as expert body recommendations26 or antimicrobial stewardship policies27 , may contribute significantly to standardization of care and optimization of patient management. The aim of this study was to assess the clinical effectiveness of checklists in the management of common postoperative complications within a high-fidelity simulated ward environment. Methods
General surgical registrars were recruited from a single academic hospital. Registrar-level trainees (with a minimum of 2 completed years of specialist training in general surgery) were selected, to reflect a level at which trainees can be expected to assume responsibility for routine patient assessment during the ward round in typical practice in the UK28 . A sample size calculation was conducted, assuming a study powered to 90 per cent and an α level of 0⋅05. As this represented an exploratory study, with no available previous data for identical outcomes, published rates of © 2014 BJS Society Ltd Published by John Wiley & Sons Ltd
1667
surgical ward round error were used25 , with the presumed effect size based on data from a previous study24 of surgical process checklists in the operating theatre. This suggested a necessary sample size of only three subjects in each group. Given the surrogate nature of data used, this was increased to ten to maximize data collection and increase the validity of results, based on experience from previous ward simulation-based trials29 . Simulated ward rounds took place within a three-bed simulated ward. This was a fully furnished, high-fidelity simulation environment, complete with patient monitoring equipment and access to simulated patient records, radiology and pathology results (Fig. 1). The environment has been validated and described in detail previously29,30 . Professional medical actors were employed to simulate patients and exhibit realistic clinical signs. An integrated audiovisual system recorded all activity digitally, with realistic background ‘white noise’ recorded from a real clinical environment. Trainees were exposed to a series of simulated scenarios in which patients experienced one of the following postoperative complications: pneumonia, anastomotic leak, wound infection, urinary tract infection, postoperative bleeding and intra-abdominal sepsis. These complications were identified as the most common types following major gastrointestinal surgery, with subsequent development of checklists through expert input, best available evidence and end-user input (unpublished work, Pucher et al.). All trainees conducted a baseline ward round of three patients within the simulated ward, according to personal practice (Fig. 2). Ward rounds were supported by a nurse and a junior doctor trained to provide a standardized level of information relating to patients as part of the simulation, if prompted or asked by the trainee. Trainees were blinded to study design and assessment endpoints. To create a realistic sense of time pressure, trainees were told they were required to finish each ward round within 30 min, based on previous data from similar scenarios29 . Patient scenarios assigned to each trainee’s ward round were determined through a computer-generated randomization process, with all six scenarios being assessed by each trainee over the course of the study. Following the baseline ward round assessment, each trainee was randomized (using a further computergenerated randomization sequence) to either a control or an intervention group. Trainees in the intervention group received a didactic session, with written and verbal instruction, on use of the previously developed checklists for management of surgical complications (unpublished work, Pucher et al.), then completed their final ward round, for which the checklists were made available for use (Appendix www.bjs.co.uk
BJS 2014; 101: 1666–1673
1668
Fig. 1
P. H. Pucher, R. Aggarwal, M. Qurashi, P. Singh and A. Darzi
High-fidelity simulated ward environment Trainees recruited n = 20
Baseline ward round assessment (3 patients per ward round)
Randomized to control n = 10
Randomized to intervention n = 10
Checklist training n = 10
Second simulated ward round (3 patients per ward round) n = 10
Second simulated ward round (3 patients per ward round) n = 10
Feedback questionnaire n = 10 Fig. 2
CONSORT flow diagram for the study
S1, supporting information). Trainees were informed that use of the checklists was not compulsory, and need be used only if the trainee was comfortable with them or felt they would help improve care given. Trainees in the control group conducted a final ward round according to their own personal practice, thus acting as controls for any potential learning effects resulting from repeated exposure to the simulator. Baseline and final ward rounds were conducted in separate sessions on different days. The primary endpoint for the study was the rate of failure to execute critical management steps for each © 2014 BJS Society Ltd Published by John Wiley & Sons Ltd
postoperative complication. Each checklist involved five to eight management processes, such as the ordering of blood cultures and administration of antibiotics for sepsis, or arranging a blood transfusion and preparing a patient for return to the operating room in case of postoperative haemorrhage (unpublished work, Pucher et al.). Completion or omission of each checklist item was recorded in a binary manner through direct observation. Secondary endpoints included technical and non-technical performance. Technical performance was assessed using the Surgical Ward care Assessment Tool (SWAT), a checklist-based, validated tool to assess thoroughness of patient assessment (SWAT-A) and management (SWAT-M)25,29 . Non-technical performance was assessed using the Ward NOn-TECHnical Skills (W-NOTECHS) scale, assessing five behavioural domains on a Likert scale, resulting in a final score of 5–2529,31 . Prescribing errors were recorded. Each ward round was recorded digitally using two integrated ceiling cameras, with 20 per cent of ward rounds rated by a second blinded rater to assess potential for observer bias. Inter-rater reliability was assessed using Cronbach’s α. Trainees in the intervention group were surveyed regarding their perceptions of the checklists, with questionnaire responses graded from 1 (strongly disagree) to 5 (strongly agree) on a Likert scale.
Statistical analysis Demographic and ward round performance data for failure rate, SWAT, W-NOTECHS and time taken was compared www.bjs.co.uk
BJS 2014; 101: 1666–1673
Surgical checklists and postoperative care
1669
between groups, in addition to within-group comparison of baseline and final ward rounds. Non-parametric tests were used, with Mann–Whitney U and Wilcoxon signed-rank tests for intergroup and intragroup comparisons respectively. The χ2 test was used to compare categorical data. Secondary post hoc analyses were performed to assess potentially confounding factors. To investigate differences in management of different complications, failure rates were stratified by scenario and differences in means assessed using ANOVA. A linear regression model was used to adjust for trainee factors (sex and years of experience) when interpreting the effect of trial group (control versus intervention) on failure rates. Questionnaire responses were collated and reported as mean(s.d.). Responses given as a 4 or 5 on a Likert scale were considered positive. Data analysis was performed using SPSS® version 21 (IBM, Armonk, New York, USA), with P < 0⋅050 considered statistically significant. All results are reported as median (i.q.r.) unless indicated otherwise.
Results
80
80
a
P
9:1 3 (2–4)
7:3 3 (2–4)
0⋅264† 0⋅796‡
60
40
20
20
0
Intervention (n = 10)
Of all ward rounds, 20 per cent were assessed by a second blinded assessor, with excellent inter-rater reliability (Cronbach α = 0⋅893). Considering the rates of failure to complete critical processes in the management of postoperative complications, there was no difference between control and intervention groups for baseline assessment (58 (43–75) versus 67 (50–67) per cent respectively; P = 0⋅988) (Fig. 3). For the final ward round, process failure rates were significantly different, with near-elimination of errors in the intervention group (0 (0–0) per cent versus 60 (40–73) per cent for the control group; P < 0⋅001). All trainees in the intervention group elected to use the checklists where these were made available, for the final ward round. There were no statistically significant differences between groups at baseline for SWAT-A, SWAT-M or WNOTECHS metrics (Table 2). In the final ward round, the intervention group scored significantly higher than the control group in patient management (SWAT-M: control 0⋅63 (0⋅50–0⋅75) versus intervention 0⋅83 (0⋅63–0⋅88); P = 0⋅005), although not for patient assessment (SWAT-A, P = 0⋅094) or non-technical skill (W-NOTECHS, 100
40
Control (n = 10)
*Values are median (i.q.r.). †χ2 test; ‡Mann–Whitney U test.
100
60
Demographics of subjects and controls
Sex ratio (M : F) Length of surgical training (years)*
Failure rate (%)
Failure rate (%)
Twenty trainees were recruited, resulting in 120 patient assessments in the final analysis. Each trainee completed all six patient scenarios, randomized equally to control and intervention groups. All intervention group trainees elected to use the checklists in their final ward rounds. There were no differences in sex or experience levels of participants between groups (Table 1).
Table 1
Control
Baseline assessment
0
Intervention
b
Control (no checklist)
Intervention (with checklist)
Final assessment
Failure to adhere to critical processes in the management of postoperative complications. Box plot comparisons of performance between control and intervention groups for a baseline and b final ward rounds. Thick bar denotes median value, box denotes i.q.r. and whiskers show the range; small circles indicate outliers. a P = 0⋅988, b P < 0⋅001 (Mann–Whitney U test)
Fig. 3
© 2014 BJS Society Ltd Published by John Wiley & Sons Ltd
www.bjs.co.uk
BJS 2014; 101: 1666–1673
1670
Table 2
P. H. Pucher, R. Aggarwal, M. Qurashi, P. Singh and A. Darzi
Simulated ward round performance metrics Baseline assessment
SWAT-A Intragroup P† SWAT-M Intragroup P† W-NOTECHS Intragroup P† Ward round time (min)* Intragroup P† Prescription errors Intragroup P†
Final assessment
Control (n = 30)
Intervention (n = 30)
P‡
0⋅62 (0⋅55–0⋅81)
0⋅73 (0⋅64–0⋅81)
0⋅436
0⋅63 (0⋅50–0⋅75)
0⋅56 (0⋅46–0⋅67)
0⋅350
19⋅5 (17–22)
17⋅0 (16–20)
0⋅190
26⋅5(5⋅1)
26⋅8(5⋅1)
0⋅900
10
22
0⋅019
Control (no checklist) (n = 30)
Intervention (checklist) (n = 30)
0⋅74 (0⋅64–0⋅82) < 0⋅001 0⋅63 (0⋅50–0⋅75) 0⋅571 19⋅0 (18⋅5–21) 0⋅809 23⋅3(4⋅2) 0⋅349 15 0⋅298
0⋅77 (0⋅73–0⋅91) < 0⋅001 0⋅83 (0⋅63–0⋅88) < 0⋅001 20⋅5 (18⋅5–23) 0⋅043 22⋅2(5⋅7) 0⋅111 5 0⋅020
P‡ 0⋅094 0⋅005 0⋅218 0⋅730 0⋅043
Values are median (i.q.r.), except *mean(s.d.). SWAT(-A/M), Surgical Ward-care Assessment Tool (-Assessment/Management); W-NOTECHS, Ward NOn-TECHnical Skills scale. †Final versus baseline assessment (Wilcoxon signed-rank test); ‡Mann–Whitney U test.
Baseline ward round performance stratified by clinical scenario
Table 3
Responses to use of checklists for management of postoperative complications
Table 4
Failure rate (%) Questionnaire statement Pneumonia Wound infection Anastomotic leak Sepsis Urinary tract infection Postoperative haemorrhage
65(26) 58(13) 61(13) 61(13) 70(33) 68(18)
Values are mean(s.d.). P = 0⋅796 (ANOVA).
P = 0⋅218). There was no significant difference in the time taken for each ward round between groups or between ward rounds. In terms of intragroup comparisons, a small but significant improvement was seen in both groups for SWAT-A (both P < 0⋅001). Regarding patient management (SWAT-M), improvement was seen only in the intervention group (P < 0⋅001). Similarly, total W-NOTECHS scores improved only in the intervention group (P = 0⋅043; control group, P = 0⋅809). There were no significant differences in median values for individual W-NOTECHS domain scores. Although the intervention group had a higher number of prescription errors recorded in the baseline round (22 versus 10 in the control group; P = 0⋅019), the use of checklists resulted in a significant reduction in the final round (5 versus 15 errors respectively; P = 0⋅043) (Table 2). There were no differences in performance between scenarios (ANOVA for baseline ward round, P = 0⋅796) (Table 3). Controlling for trainee experience and sex through inclusion as independent variables in a regression model had no effect on results, with trial group allocation the only significant predictor of process failure rates (odds ratio 0⋅54, 95 per cent c.i. 0⋅46 to 0⋅62; P < 0⋅001). © 2014 BJS Society Ltd Published by John Wiley & Sons Ltd
The checklists were easy to use The checklists cover clinical conditions relevant to my practice I found this checklist applicable to my practice The checklists may improve management of complications I would want this checklist to be used for me if I experienced a postoperative complication I would consider using the checklists in my routine practice
Response score*
Positive response (%)
4⋅7 (4–5) 4⋅8 (4–5)
100 100
4⋅1 (3–5)
80
4⋅4 (4–5)
100
4⋅3 (4–5)
100
4⋅1 (3–5)
80
*Values are mean (range). Responses were received from ten of ten subjects in the intervention group.
Questionnaire responses All intervention group study participants returned completed questionnaires. Responses were positive, with all believing that checklists were easy to use and would improve clinical practice (Table 4). All participants indicated that they would want the checklists used if they themselves were to experience a postoperative complication. Discussion
Despite their common nature, wide variations in the management of postoperative complications exist across different centres2,8,32 – 35 . Checklists are being adopted increasingly in routine surgical care36 , but no such clinical aids have been available for the non-routine care of patients suffering postoperative complications. The results of the present study demonstrate not only the significant variability that exists in the management www.bjs.co.uk
BJS 2014; 101: 1666–1673
Surgical checklists and postoperative care
1671
of common complications, but how error and variation in clinical care may be eliminated essentially through a single intervention. The introduction of checklists to guide patient management, once the diagnosis of a postoperative complication had been made and a decision taken to treat, led to the standardization of treatment in line with evidence-based guidelines, and reduction of median error rates from 60 to 0 per cent. Despite their simple nature, or perhaps because of it, implementing checklists in the manner of this study has the potential to reduce the variability of ward-based care25 and may act significantly to reduce failure-to-rescue rates and improve outcomes. The experimental, simulated context in which this study was conducted represents a potential confounder in the interpretation of results, although the overwhelmingly positive effect is in agreement with previously published data16,24 . The introduction of checklist-based protocols for antiseptic central venous catheter placement eliminated catheter-associated sepsis over the course of 3 months16 . A simulation-based trial assessing the management of operating room crises reported a reduction in error rates of 75 per cent when checklists were used to guide the operating room team’s response to rare events such as asystole, malignant hyperthermia or unstable arrhythmia24 . Patient management without the aid of checklists placed patients at risk of critical errors such as the failure to prescribe antibiotics for a patient with an anastomotic leak, insufficient intravenous access in a patient suffering from haemorrhage, or failure to obtain blood cultures in a patient with sepsis and pyrexia. Checklist use led to near elimination of these errors, standardizing care without compromising clinicians’ decision-making or patient care. It is important to note that the checklists were introduced as aids to guide care only, with clinicians free to use them (or not) as they felt appropriate, dependent upon the suspected diagnosis and intended treatment. Although they represented an additional piece of paperwork, the checklists were well received, with all reporting that they were easy to use, and improved their practice. Checklist use did not prolong the ward rounds, with no significant difference in time taken between groups. Perhaps most significantly, all trainees reported that they would want checklists used in their own care, were they to require care for postoperative complications themselves. Introduction of the checklist also led to improvements in non-technical skill. The ability of checklists to flatten clinical hierarchies and improve communication has been seen previously in improved safety attitudes19 and staff questionnaire responses37 , but this is the first time that the effect of a checklist on non-technical performance has been observed directly and measured in a clinical context.
The positive feedback seen in this study represents a crucial aspect of checklist implementation, illustrating the benefit of simulation-based trialling of such an intervention before introduction into clinical practice. Centres have reported significant difficulties associated with attempts to change established clinical process21 – 23 . Negative feedback captured in the simulation can therefore be addressed before clinical implementation. This study, intended as an exploratory study of the effectiveness of checklists for postoperative care, must be considered in the context of its limitations. Only the performance of surgical trainees was assessed. Other health professionals, including nursing staff and other grades of medical staff, were not included. As registrars are the group of surgical trainees commonly responsible for routine patient review, and most likely to make the first diagnosis of a postoperative complication and initiate treatment, this is unlikely to represent a major confounder. Interaction with other members of the clinical team, although important, would be secondary to the clinician’s diagnosis and decision to treat. This was a single-centre study of a simulation-based trial. Localization and adaptation to local protocols and practices is a critical component of successful checklist implementation18,38 , and adapted checklists would be required for individual centres. The study did not evaluate the longevity or retention of changes to practice, which may experience degradation of checklist use over time. Other groups implementing checklist-type interventions have variably reported both deterioration39 and improvement16 of performance over time. Analogous to the acquisition of technical skill through repeated training40 , it may be that repeated training sessions should be considered for such non-technical interventions. Further longitudinal assessments of postoperative checklists would be required. Finally, it must be recognized that the management of postoperative complications is a two-stage process, requiring the correct diagnosis first to be made before appropriate management can be instituted. Checklists address the latter, but not the former7,8 . Although management of a given condition is amenable to checklist-based care, based upon best evidence and expert guidelines, diagnostic ability is dependent on a clinician’s own ability and practice. To improve patient assessment and the recognition of complications, broader interventions, such as educational curricula to improve trainees’ conduct of ward rounds in general, can be effective41,42 . This study has demonstrated the viability of checklists in the management of postoperative complications. Future work should include a clinical implementation study. As seen in this and other checklist studies17,24 , steps to ensure staff participation and ‘buy-in’ are essential to success with
© 2014 BJS Society Ltd Published by John Wiley & Sons Ltd
www.bjs.co.uk
BJS 2014; 101: 1666–1673
1672
the involvement of local ‘champions’, regular audit and performance feedback, as well as simulation-based trialling and staff training to ensure appropriate use. Development of checklists for further complications, such as pulmonary embolism or deep vein thrombosis, might also be considered. Considering the common nature of complications, improving the care of patients with postoperative morbidity has perhaps the greatest potential for impact in reducing failure-to-rescue rates; the use of appropriate checklists may standardize treatment and improve outcomes. Disclosure
The authors declare no conflict of interest. References 1 Ito H, Are C, Gonen M, D’Angelica M, Dematteo RP, Kemeny NE et al. Effect of postoperative morbidity on long-term survival after hepatic resection for metastatic colorectal cancer. Ann Surg 2008; 247: 994–1002. 2 Ghaferi AA, Birkmeyer JD, Dimick JB. Hospital volume and failure to rescue with high-risk surgery. Med Care 2011; 49: 1076–1081. 3 Hii MW, Smithers BM, Gotley DC, Thomas JM, Thomson I, Martin I et al. Impact of postoperative morbidity on long-term survival after oesophagectomy. Br J Surg 2013; 100: 95–104. 4 Merkow RP, Bilimoria KY, McCarter MD, Phillips JD, Decamp MM, Sherman KL et al. Short-term outcomes after esophagectomy at 164 American College of Surgeons National Surgical Quality Improvement Program Hospitals: effect of operative approach and hospital-level variation. Arch Surg 2012; 147: 1009–1016. 5 Cone MM, Herzig DO, Diggs BS, Rea JD, Hardiman KM, Lu KC. Effect of surgical approach on 30-day mortality and morbidity after elective colectomy: a NSQIP study. J Gastrointest Surg 2012; 16: 1212–1217. 6 Molena D, Mungo B, Stem M, Feinberg RL, Lidor AO. Outcomes of esophagectomy for esophageal achalasia in the United States. J Gastrointest Surg 2014; 18: 310–317. 7 Ghaferi AA, Birkmeyer JD, Dimick JB. Variation in hospital mortality associated with inpatient surgery. N Engl J Med 2009; 361: 1368–1375. 8 Ghaferi AA, Birkmeyer JD, Dimick JB. Complications, failure to rescue, and mortality with major inpatient surgery in medicare patients. Ann Surg 2009; 250: 1029–1034. 9 Aiken LH, Clarke SP, Sloane DM, Sochalski J, Silber JH. Hospital nurse staffing and patient mortality, nurse burnout, and job dissatisfaction. JAMA 2002; 288: 1987–1993. 10 Friese CR, Earle CC, Silber JH, Aiken LH. Hospital characteristics, clinical severity, and outcomes for surgical oncology patients. Surgery 2010; 147: 602–609.
© 2014 BJS Society Ltd Published by John Wiley & Sons Ltd
P. H. Pucher, R. Aggarwal, M. Qurashi, P. Singh and A. Darzi
11 Ghaferi AA, Osborne NH, Birkmeyer JD, Dimick JB. Hospital characteristics associated with failure to rescue from complications after pancreatectomy. J Am Coll Surg 2010; 211: 325–330. 12 Creamer GL, Dahl A, Perumal D, Tan G, Koea JB. Anatomy of the ward round: the time spent in different activities. ANZ J Surg 2010; 80: 930–932. 13 Vincent C, Neale G, Woloshynowych M. Adverse events in British hospitals: preliminary retrospective record review. BMJ 2001; 322: 517–519. 14 Reason J. Human error: models and management. BMJ 2000; 320: 768–770. 15 Reason J. Combating omission errors through task analysis and good reminders. Qual Saf Health Care 2002; 11: 40–44. 16 Pronovost P, Needham D, Berenholtz S, Sinopoli D, Chu H, Cosgrove S et al. An intervention to decrease catheter-related bloodstream infections in the ICU. N Engl J Med 2006; 355: 2725–2732. 17 Haynes AB, Weiser TG, Berry WR, Lipsitz SR, Breizat AH, Dellinger EP et al.; Safe Surgery Saves Lives Study Group. A surgical safety checklist to reduce morbidity and mortality in a global population. N Engl J Med 2009; 360: 491–499. 18 de Vries EN, Prins HA, Crolla RM, den Outer AJ, van Andel G, van Helden SH et al.; SURPASS Collaborative Group. Effect of a comprehensive surgical safety system on patient outcomes. N Engl J Med 2010; 363: 1928–1937. 19 Haynes AB, Weiser TG, Berry WR, Lipsitz SR, Breizat AH, Dellinger EP et al.; Safe Surgery Saves Lives Study Group. Changes in safety attitude and relationship to decreased postoperative morbidity and mortality following implementation of a checklist-based surgical safety intervention. BMJ Qual Saf 2011; 20: 102–107. 20 Russ S, Rout S, Sevdalis N, Moorthy K, Darzi A, Vincent C. Do safety checklists improve teamwork and communication in the operating room? A systematic review. Ann Surg 2013; 258: 856–871. 21 Borchard A, Schwappach DL, Barbir A, Bezzola P. A systematic review of the effectiveness, compliance, and critical factors for implementation of safety checklists in surgery. Ann Surg 2012; 256: 925–933. 22 Spence J, Goodwin B, Enns C, Dean H. Student-observed surgical safety practices across an urban regional health authority. BMJ Qual Saf 2011; 20: 580–586. 23 Conley DM, Singer SJ, Edmondson L, Berry WR, Gawande AA. Effective surgical safety checklist implementation. J Am Coll Surg 2011; 212: 873–879. 24 Arriaga AF, Bader AM, Wong JM, Lipsitz SR, Berry WR, Ziewacz JE et al. Simulation-based trial of surgical-crisis checklists. N Engl J Med 2013; 368: 246–253. 25 Pucher PH, Aggarwal R, Darzi A. Surgical ward round quality and impact on variable patient outcomes. Ann Surg 2014; 259: 222–226. 26 Hooton TM, Bradley SF, Cardenas DD, Colgan R, Geerlings SE, Rice JC et al.; Infectious Diseases Society of America. Diagnosis, prevention, and treatment of catheter-associated urinary tract infection in adults: 2009
www.bjs.co.uk
BJS 2014; 101: 1666–1673
Surgical checklists and postoperative care
27
28
29
30 31
32
33
34
35
International Clinical Practice Guidelines from the Infectious Diseases Society of America. Clin Infect Dis 2010; 50: 625–663. Evans RS, Pestotnik SL, Classen DC, Clemmer TP, Weaver LK, Orme JF Jr et al. A computer-assisted management program for antibiotics and other antiinfective agents. N Engl J Med 1998; 338: 232–238. Intercollegiate Surgical Curriculum Programme. General Surgery Specialty Syllabus; 2013. http://www.iscp.ac.uk [accessed 1 April 2014]. Pucher PH, Aggarwal R, Srisatkunam T, Darzi A. Validation of the simulated ward environment for assessment of ward-based surgical care. Ann Surg 2014; 259: 215–221. Pucher PH, Darzi A, Aggarwal R. Simulation for ward processes of surgical care. Am J Surg 2013; 206: 96–102. Steinemann S, Berg B, Ditullio A, Skinner A, Terada K, Anzelon K et al. Assessing teamwork in the trauma bay: introduction of a modified ‘NOTECHS’ scale for trauma. Am J Surg 2012; 203: 69–75. Glance LG, Dick AW, Meredith JW, Mukamel DB. Variation in hospital complication rates and failure-to-rescue for trauma patients. Ann Surg 2011; 253: 811–816. McHugh MD, Kelly LA, Smith HL, Wu ES, Vanak JM, Aiken LH. Lower mortality in magnet hospitals. Med Care 2013; 51: 382–388. Almoudaris AM, Mamidanna R, Bottle A, Aylin P, Vincent C, Faiz O et al. Failure to rescue patients after reintervention in gastroesophageal cancer surgery in England. JAMA Surg 2013; 148: 272–276. Henneman D, Snijders HS, Fiocco M, van Leersum NJ, Kolfschoten NE, Wiggers T et al. Hospital variation in
1673
36
37
38
39
40
41
42
failure to rescue after colorectal cancer surgery: results of the Dutch Surgical Colorectal Audit. Ann Surg Oncol 2013; 20: 2117–2123. Adamina M, Kehlet H, Tomlinson GA, Senagore AJ, Delaney CP. Enhanced recovery pathways optimize health outcomes and resource utilization: a meta-analysis of randomized controlled trials in colorectal surgery. Surgery 2011; 149: 830–840. Blanco M, Clarke JR, Martindell D. Wrong site surgery near misses and actual occurrences. AORN J 2009; 90: 215–218, 221–212. Sparks EA, Wehbe-Janek H, Johnson RL, Smythe WR, Papaconstantinou HT. Surgical Safety Checklist compliance: a job done poorly! J Am Coll Surg 2013; 217: 867–873.e1–e3. Vats A, Vincent CA, Nagpal K, Davies RW, Darzi A, Moorthy K. Practical challenges of introducing WHO surgical checklist: UK pilot experience. BMJ 2010; 340: b5433. Crochet P, Aggarwal R, Dubb SS, Ziprin P, Rajaretnam N, Grantcharov T et al. Deliberate practice on a virtual reality laparoscopic simulator enhances the quality of surgical technical skills. Ann Surg 2011; 253: 1216–1222. Pucher PH, Aggarwal R, Singh P, Srisatkunam T, Twaij A, Darzi A. Ward simulation to improve surgical ward round performance: a randomised controlled trial of a simulation-based curriculum. Ann Surg 2014; [Epub ahead of print]. Pucher PH, Darzi A, Aggarwal R. Development of an evidence-based curriculum for training of ward-based surgical care. Am J Surg 2014; 207: 213–217.
Supporting information
Additional supporting information may be found in the online version of this article: Appendix S1 Example of the postoperative surgical checklist for hospital care (Word document)
© 2014 BJS Society Ltd Published by John Wiley & Sons Ltd
www.bjs.co.uk
BJS 2014; 101: 1666–1673
