Society for Pediatric Anesthesia Section Editor: James DiNardo
National Pediatric Anesthesia Safety Quality Improvement Program in the United States C. Dean Kurth, MD,* Don Tyler, MD,† Eugenie Heitmiller, MD, FAAP,‡ Steven R. Tosone, MD,§ Lynn Martin, MD, MBA,║ and Jayant K. Deshpande, MD, MPH¶ BACKGROUND: As pediatric anesthesia has become safer over the years, it is difficult to quantify these safety advances at any 1 institution. Safety analytics (SA) and quality improvement (QI) are used to study and achieve high levels of safety in nonhealth care industries. We describe the development of a multiinstitutional program in the United States, known as Wake-Up Safe (WUS), to determine the rate of serious adverse events (SAE) in pediatric anesthesia and to apply SA and QI in the pediatric anesthesia departments to decrease the SAE rate. METHODS: QI was used to design and implement WUS in 2008. The key drivers in the design were an organizational structure; an information system for the SAE; SA to characterize the SAE; QI to imbed high-reliability care; communications to disseminate the learnings; and engaged leadership in each department. Interventions for the key drivers, included Participation Agreements, Patient Safety Organization designation, IRB approval, Data Management Co., membership fee, SAE standard templates, SA and QI workshops, and department leadership meetings. RESULTS: WUS has 19 institutions, 39 member anesthesiologists, 734 SAE, and 736,365 anesthetics as of March, 2013. The initial members joined at year 1, and initial SAE were recorded by year 2. The SAE rate is 1.4 per 1000 anesthetics. Of SAE, respiratory was most common, followed by cardiac arrest, care escalation, and cardiovascular, collectively 76% of SAE. In care escalation, medication errors and equipment dysfunction were 89%. Of member anesthesiologists, 70% were trained in SA and QI by March 2013; virtually, none had SA and QI expertise before joining WUS. CONCLUSION: WUS documented the incidence and types of SAE nationally in pediatric anesthesiology. Education and application of QI and SA in anesthesia departments are key strategies to improve perioperative safety by WUS. (Anesth Analg 2014;119:112–21)
Q
uality improvement (QI) science as applied to safety in anesthesia has evolved considerably over the past 30 years. Cooper et al.’s1,2 pioneering work beginning in the 1970s on analysis of critical incidents led to a systematic approach to analyzing errors, learning from them, and then modifying anesthetic technique through education. This methodology is now largely incorporated in anesthesia practice. The American Society of Anesthesiologists Closed Claims Study (ASA-CCS) is another landmark in adverse From the *Department of Anesthesiology, Cincinnati Children’s Hospital Medical Center, University of Cincinnati College of Medicine, Cincinnati, Ohio; †Department of Anesthesiology and Critical Care Medicine, Children’s Hospital of Philadelphia, University of Pennsylvania School of Medicine, Philadelphia, Pennsylvania; ‡Department of Anesthesiology and Critical Care Medicine, Johns Hopkins University School of Medicine, The Johns Hopkins Hospital, Baltimore, Maryland; §Department of Anesthesiology and Pediatrics, Emory University School of Medicine, Atlanta, Georgia; ║Department of Anesthesiology & Pain Medicine, Seattle Children’s Hospital, University of Washington School of Medicine, Seattle, Washington; and ¶Department of Anesthesiology, Arkansas Children’s Hospital, University of Arkansas for Medical Sciences, Little Rock, Arkansas. Accepted for publication October 8, 2013. Funding: None. The authors declare no conflicts of interest. Reprints will not be available from the authors. Address correspondence to C. Dean Kurth, MD, Department of Anesthesiology, Cincinnati Children’s Hospital Medical Center, University of Cincinnati College of Medicine, 3333 Burnet Ave., Cincinnati, OH 45229. Address e-mail to [email protected]. Copyright © 2014 International Anesthesia Research Society DOI: 10.1213/ANE.0000000000000040
112 www.anesthesia-analgesia.org
event analysis.3–8 In this ongoing study, investigators access malpractice cases that have been settled or adjudicated and then enlist anesthesiologists to analyze the anesthetic technique to uncover errors that may have led to an untoward outcome. The findings from this effort also changed the way anesthesiology is currently practiced, as anesthesiologists learn from publication of the studies and adapt the learnings into their practice. The Perioperative Cardiac Arrest (POCA) project, a voluntary registry of pediatric cardiac arrests, initiated in 1997, identified common causes of cardiac arrest in children.7,9,10 Several important observations from this registry changed pediatric anesthesia practice, again through dissemination of the findings and education, which contributed to the demise of halothane, widespread use of sevoflurane in its place, and a decrease in cardiac arrest in pediatric anesthesia. As pediatric anesthesia has become safer over the years, it is difficult to quantify these safety advances at any 1 institution. The rate of critical events in pediatric anesthesia is low; therefore, the number of events occurring at a single institution precludes systematic study. More data are needed to provide greater precision concerning incidence, causation(s), statistical analysis, and process control to mitigate events.7,11–18 Outside the field of medicine, the science of safety analytics (SA) and QI have developed considerably during the past 30 years, particularly in the aviation, manufacturing, and nuclear industries.19–21 While some of this expertise has been deployed to medicine, it remains limited at any July 2014 • Volume 119 • Number 1
single institution or anesthesia department. As anesthesia and perioperative care have become more system oriented through team-based care using technology and informatics, SA and QI have become increasingly germane. Today, critical events require a systems view and an individual practitioner view to consider multiple factors, many external to the anesthesiologist. The Society for Pediatric Anesthesia (SPA) represents anesthesiologists and allied health care professionals who are engaged in the practice of pediatric anesthesia. The mission of the SPA is to continually advance the safety and quality of anesthetic care, perioperative management, and alleviation of pain in children. Given the evolution of system-based care in pediatric anesthesia, SPA supported the development of a national program to apply SA and QI to better enable departments to decrease the incidence of critical events in children undergoing anesthesia. We describe this program, known as “Wake-Up Safe” (WUS), and how it developed and is currently being executed using QI methods. QI methods contain 4 components: appreciation of a system, understanding variation, action learning, and change management.21,22 A system is an interdependent group working together toward a common purpose. QI defines the parts of the system to identify the linkages and get them to interact effectively. Understanding variation is embodied in statistical process control, a form of statistics found in manufacturing that differs from medical statistics, exemplified by run charts and special and common-cause variation. In action learning, QI follows the scientific method of aim, hypothesis, study design, and experimentation, appearing in the form of Key Driver Diagrams as smart aim, key drivers, and interventions. Change management embodies the disciplines of leadership, organizational behavior, and psychology to unify a system, communicate variation, conduct action learning, and thereby improve quality. Because QI differs somewhat from research, a different format was developed for QI manuscripts, known as the Standards for Quality Improvement Reporting Excellence (SQUIRE).23,24 In the present work, the SQUIRE format is used to describe the WUS program.
METHODS Setting
WUS was designed and implemented using QI methods during meetings of an executive team, the executive board, and full board of the organization. The executive team was a pilot group that was disbanded after the executive board and full board were established. The executive board consists of 4 elected individuals from the full board. The full board consists of 1 pediatric anesthesiologist from each institution in WUS. These groups met in person or via teleconference to create and implement the WUS QI project.
Ethical Issues
The executive team judged it prudent to obtain IRB approval at each institution because many journals require this to publish studies using registry data. We therefore developed a common IRB protocol not requiring informed consent that could be used by all institutions. The data submitted to the registry do not contain information that could be used to identify the patient or the institution (“de-identified”
July 2014 • Volume 119 • Number 1
data) and are stored in a secure database at Axio Research (Seattle, WA), a contract research organization. Data and medical records are stored in a secure location according to the quality assurance policy of the site.
Planning the Program and Study of Interventions
In early 2007, several SPA members began to discuss programs to improve quality and safety in pediatric anesthesia, which eventually focused on safety. An executive team was created to lead the project, which officially launched on January 1, 2008. The executive team comprises pediatric anesthesia chiefs from several children’s hospitals and board members from SPA and the Anesthesia Patient Safety Foundation. The executive team designed the Key Driver Diagram for the WUS QI project, which displays the strategic map for process improvement and contains the following components: Smart Aim, Key Drivers, and Interventions (Fig. 1). The Smart Aim expresses the goal in a specific, measurable, actionable, relevant, and time-bound sentence. The global aim is to eliminate all preventable harm to children undergoing anesthesia. WUS focuses on a component of harm, specifically serious adverse events (SAE), defined as untoward occurrences during anesthesia care or within 24 hours of the end anesthesia care that result in life-threatening injury, requiring unplanned hospitalization or prolongation of hospitalization, result in disability, incapacity, or death. In WUS, the Smart Aim is specific for SAE, is measured to decrease SAE by 10%, has actions that can be taken to decrease SAE incidence that include those listed in the interventions, include SAE that are relevant to pediatric anesthesia, and is time-bound to achieve the goal in 5 years (2010–2015). The Key Drivers to achieve the Smart Aim are the critical components of the QI project. For WUS, there are 6 key drivers: (1) An organizational structure with the right people and the money to fund it; (2) An information system to collect and disseminate data about the SAE; (3) A common analytical methodology to characterize the SAE; (4) QI capability to establish high-performing teams and imbed high-reliability interventions of care; (5) Communications to disseminate the learning about the events and successful interventions; and (6) Leadership support at each institution. Key Driver 1 (Organizational Structure) The executive team decided to become a “special interest group” within the SPA to use its infrastructure of legal advice, accounting, and general management. The executive team developed bylaws to delineate the governance and requirements to become a Patient Safety Organization (PSO), sanctioned by the Federal Government Agency for Healthcare Research and Quality. The bylaws disbanded the executive team and created a full board composed of 1 representative from each institution, an executive board composed of president, vice president, secretary, and treasurer elected for a 1-year term by the full board, and an appointed executive administrator. At each institution, there are 1 to 4 anesthesiologists participating in analyzing SAE, and 1 or 2 support personnel who assist with entering SAE information in the database and providing denominator data. Of the member anesthesiologists, one serves on the full board, and all are expected to learn SA and QI and
www.anesthesia-analgesia.org
113
Quality Improvement-Driven Safety Program
INTERVENTIONS
KEY DRIVERS People, structure and finance to form organization Develop system to collect information about events SMART AIM Reduce preventable anesthesia related serious events 10% by 2017 GLOBAL AIM Eliminate preventable harm in children undergoing anesthesia
Safety science analytics to understand etiology of events Quality Improvement to embed capability to drive safe practices Spread knowledge to institutions to adopt safe practices Institutional leadership support to fund initiatives and reward safe practices
People 1. Recruit executive team 2. Recruit key participants to achieve a breakeven membership of 30 Structure 1. Develop Bylaws & Participation Agreement 2. Become Patient Safety Organization 3. Obtain IRB approval Finance 1. Develop funding & financial controls 1. Develop definitions and questions about events 3. Collect demographic data to understand frequency of events 4. Hire data management company 5. Develop system to get reports of data 1. Educate participating institutions 2. Workshop on Root Cause Methodology 1. Educate participating institutions 2. Members to complete IHI course 3. Workshop to develop QI program 1. Newsletter to members informing them of SAE learnings 2. Author manuscripts from database, QI and SA projects 3. National presentations from database, QI and SA projects 4. Develop peer visitation program 1. Engage department chairs, hospital CMOs & CEOs 2. Communications to PALC, CHA, ASA, NSQUIP
Figure 1. Key driver diagram for wake-up safe quality improvement program. The smart aim is the goal for the program, which is part of a global aim for the Society for Pediatric Anesthesia. The key drivers are the components that are required for the program to be fully functioning to achieve the smart aim. The interventions are the activities that are performed to make the key drivers fully operational. Interventions in the green boxes indicate those that are completed; those in the white boxes are ongoing and not yet completed.
able to lead projects. The chief of anesthesia of the institution selects the member anesthesiologists and the representative to the full board. A participation agreement was signed between SPA and the institutions. The agreement mainly clarified issues of intellectual property and identification of the institutions and the potential impact on reputation and litigation of reported cases. The agreement specified that the institutions’ identities would be protected. A membership business model was selected to finance the organization in which the institutions contributed money to join (initiation fee) and to remain a member (annual dues). The program requires 30 institutions paying annual dues to support the program’s infrastructure and requires the institutions to collectively administer 400,000 anesthetics per year to conduct safety-oriented QI projects with statistical significance. The SPA and Anesthesia Patient Safety Foundation helped to fund the initial startup. The benefits of membership included the reports about the SAE, workshops and networking to learn SA and QI to drive high-reliability methods into local practice, and scholarship to present and publish the learnings. Interventions to recruit institutions and anesthesiologists included an executive team espousing the benefits of WUS to departmental and hospital leadership, executive team personally contacting pediatric anesthesiologists known to have interests in SA and QI, executive team personally speaking with the department chiefs at the 6 largest freestanding children’s hospitals, and the executive administrator providing expertise to potential institutions to get the participation agreement and IRB document approved.
114 www.anesthesia-analgesia.org
Key Driver 2 (Information System) The system to collect information about the events was designed to be easy to input the information into, to query the system to report isolated, unusual, or high-impact events, to import into statistical software to describe and analyze the events, and to keep the information confidential and secure. A Web-based information system fit these criteria. Because the institutions and the SPA did not have this expertise, a request for proposal was distributed and 6 contract research organizations that were well versed in health care databases responded. After reviewing the responses, 3 were interviewed and site visited. Axio Research (Seattle, WA) was selected, and a contract was signed. The full board and Axio Research developed the database, Web interface, and online reporting tool. Considerable time was spent categorizing, naming, and defining the SAE (Table 1). The full board decided to view SAE from the patient perspective and not just the anesthesiologist view, to shift our thinking broadly, as perioperative pediatric physicians. Accordingly, SAE were categorized as death, life threatening, disability/incapacity, or unplanned/ prolonged hospitalization. For each SAE category, a series of questions was constructed using templates to describe it by severity, primary cause, and contributing factors, and whether the SAE was the result of the anesthesia, surgery, or patient condition. The templates and questions followed the common formats for PSO. Several pilots were conducted using fabricated cases to investigate concordance among member anesthesiologists; template questions were repeatedly revised until concordance was achieved. At the institution, 3 anesthesiologists
anesthesia & analgesia
Table 1. Categories of Events and Their Definitions Serious adverse event Death Life threatening
Category Death Airway event
Life threatening
Cardiac arrest
Disability/incapacity Disability/incapacity
Awareness anesthesia MSJC injury
Disability/incapacity Disability/incapacity Disability/incapacity Hospitalization Hospitalization or life threatening Hospitalization
Nervous system injury Visual system injury Wrong surgery Acute lung injury Cardiovascular event Care escalation
Definition Death during anesthesia or within 24 h end anesthesia Severe hypoxemia or hypercapnia receiving emergent intubation and medications during anesthesia or within 24 h after anesthesia Bradycardia or hypotension receiving cardiac compressions during anesthesia or within 24 h end anesthesia Patient memory of events during general anesthesia Injury to muscle, bone, joint, skin, or soft tissue from anesthesia care limiting function receiving medical or surgical treatment Injury to brain, spinal cord, or peripheral nerves unrelated to surgery limiting function Injury to eye or visual pathway unrelated to surgery lasting beyond 24 h after anesthesia Start of surgery or anesthesia care on body part that was not consented for procedure New onset impaired gas exchange with parenchyma signs that escalates care Unanticipated cardiovascular support during anesthesia or within 24 h end anesthesia that escalates care Fire, malignant hyperthermia, medication, or equipment error during anesthesia care that escalates care
SAE = serious adverse event (untoward occurrences during anesthesia care or within 24 hours of end anesthesia care, that result in death, are life threatening, require unplanned or prolongation of hospitalization, or result in disability or incapacity); MSJC = musculoskeletal, joint, cutaneous.
not involved in the case analyzed the case using the template questions. The information entered in the database was the consensus of the analyzing anesthesiologists. Institutions ascertained the numerator through voluntary quality assurance reports by anesthesiologists and nurses, other physicians, pharmacists, and patient families to the hospital (hospital safety reports, legal reports, electronic drug auditstrigger tools, family complaints). Interventions to obtain the SAE cases and total anesthetic cases in the database included meetings between Axio Research and institution information technology support personnel to import total anesthetic cases using billing databases, teaching member anesthesiologists at board meetings how to import the SAE cases in the database, having the executive administrator personally speaking with member anesthesiologists to conduct the analyses and import the SAE cases in a timely manner, member anesthesiologists encouraging their colleagues to report cases, generating SAE reports from the database to institutions to provide feedback, and leadership in the department encouraging or incentivizing anesthesiologists to report SAE cases, such as withholding bonuses for failure to report and/ or public praise for reporting. The use of incentive remained at the discretion of the department leadership and was not standardized across institutions. Key Driver 3 (Analytical Methodology) SA aims to determine the primary and secondary causes for the SAE. SA is a rigorous, detailed, and established science, as exemplified by transportation industry and federal agencies, involving “root” cause, common cause, contributing factors, sequence of events, and systems analyses. Based on this analysis of the information, strategies can be developed, and processes can be applied to mitigate the risk of future events. Recently, several hospitals have begun to learn and apply SA to determine causes of serious safety and “never” events. The executive team initially envisioned each anesthesia department to have this capability; however, it soon became apparent that many did not. Interventions to train the member anesthesiologists in SA occurred through the members attending a WUS-run workshop facilitated by an expert and members attending external workshops and courses that were approved by the full board.
July 2014 • Volume 119 • Number 1
Key Driver 4 (QI Capability) Knowing who is at risk of an event and why the event occurs does not reliably prevent future events; in other words, information and education are not high-reliability processes. QI uses high-reliability processes such as checklists, standardization, technologies, and automation that have become the cornerstone to ultrasafe industries. As with SA, the executive team envisioned each anesthesia department to have QI capability and likewise discovered that few had it. Interventions to train the member anesthesiologists consisted of WUS purchasing a subscription to online courses developed by the Institute for Healthcare Improvement for members to complete the courses, members attending a WUS-run workshop on QI facilitated by an expert, and members attending external workshops and courses that were approved by the full board. Key Driver 5 (Communications) This concerns spread of knowledge and adoption of successful processes from one institution to another. Adoption of best practice from anesthesia provider to provider within an institution takes months, and from anesthesia provider to provider at different institutions takes years. For example, hyperkalemic cardiac arrest associated with a large volume blood transfusion remains a problem 15 years after it was reported by POCA. This illustrates the slow spread and adoption of processes to mitigate hyperkalemia by those caring for the bleeding patient and those managing the supply of blood products. Interventions to hasten spread and adoption include a peer visitation program similar to what the nuclear industry has used successfully, an electronic newsletter to institutions describing events reported to the database similar to what the air force uses to inform pilots of near misses, presentations at national conferences, and publications of events from the database and QI and SA projects in books, journals, and society newsletters. Key Driver 6 (Leadership) The final key driver pertains to leadership support by the institution. For WUS to be successful, the member anesthesiologists require nonclinical time and financial support to participate in the organization. In addition, the WUS QI
www.anesthesia-analgesia.org
115
Quality Improvement-Driven Safety Program
projects obviously require institutional and anesthesia chief support for the projects to be successful. Finally, the annual membership fee requires the department chief’s endorsement. Interventions consisted of WUS being a standing agenda item at the twice annual Pediatric Anesthesia Leadership Council, the organization comprises the pediatric anesthesia chairs, directors, and chiefs from the children’s hospitals and university hospitals in the United States, to bring important matters of WUS to their attention.
Methods of Evaluation
A specific metric was associated with each key driver. For key driver 1, the goal is to increase membership to the target of 30 institutions and to have at each institution 1 member anesthesiologist per 10,000 to 15,000 anesthetics. Thus, the metrics are number of institutions and member anesthesiologists. An institution was considered a member when a participation agreement was signed, the IRB was approved, and the initiation dues were paid. An anesthesiologist was considered a member when enrolled by the executive administrator. The executive administrator reported the number of institutions and member anesthesiologists against the goal every 3 months. For key driver 2, the goal is complete reporting of SAE’s and the total number of anesthetics by the institutions. Accordingly, the metric is completed reports for the number of SAE, the types of SAE, and the total number of anesthetics in the database. For key drivers 3 and 4, the goals are to establish expertise in SA and QI in the anesthesia departments that would be indicated by 90% of member anesthesiologists completing SA and QI training. The metrics are the percentages of member anesthesiologists who had completed SA and QI training. The goal for key driver 5 is dissemination of knowledge about SAE, SA, and QI that would be indicated by 10% of WUS institutions each year authoring manuscripts or presenting at national meetings about the information in the database or results of QI or SA projects. Accordingly, the metric is number of presentations and publications per year by the WUS institutions using the database and training. There was no duplicate counting for coauthorship of manuscripts.
Analysis
Statistical process control run charts were used to measure the impact of the interventions over time on the metrics for the key drivers. Run charts can distinguish common-cause and special-cause variation and can evaluate the effectiveness of the interventions. An observed variation is considered statistically significant in run charts when ≥6 consecutive points are increasing or decreasing, or ≥8 consecutive points are either above or below the mean centerline.21 The preintervention time was defined from the official launch of WUS (January 1, 2008) until the first intervention for the key driver was begun.
RESULTS
Figure 1 demonstrates the key drivers for the WUS program. Of the 6 key drivers, only the system to collect information about the events (key driver 2) is complete. The membership continues to implement the interventions to complete the other key drivers. Figure 2 displays the number of institutions and member anesthesiologists in WUS since start up, representing the sum of interventions to complete key driver 1. It took 10 months before any institution joined; thereafter, a steady increase in membership occurred. Member anesthesiologists in an institution were often added incrementally after the institution joined, and thus, the number of member anesthesiologists could increase even though the number of institutions did not change. In March 2013, WUS has 19 institutions and 39 member anesthesiologists, representing 65% of intended membership. The recruitment of institutions was stopped after year 2 because key drivers 1 and 2 were insufficiently developed to support more members. After 30 months, recruitment was restarted because the structure for key driver 1 and all of key driver 2 were completed. Figure 3 depicts the number of SAE and anesthetics reported to the database since inception. SAE and anesthetic reports began in July 2010, approximately 30 months after start up and 20 months after the first institution joined, and have continued to increase. Since January 2011, the average number of SAE reported into the database is 28 per month.
Figure 2. Run chart for number of institutions and member anesthesiologists since inception of the program.
116 www.anesthesia-analgesia.org
anesthesia & analgesia
The average number of anesthetics per institution is 19,800 per year. During the past 24 months, the membership collectively conducted 240,000 anesthetics per year, with approximately 1.4 SAE per 1000 anesthetics. Of the reported SAE, the most common was respiratory events, followed by cardiac arrest, care escalation, and cardiovascular events, which collectively accounted for 76% of events (Fig 4). Visual system injury, “wrong surgery,” and awareness under anesthesia were rare, representing 1.1%, 0.4%, and 0.3 % of SAE, respectively. Respiratory events usually arose from complete airway obstruction (pharyngeal obstruction, laryngospasm, bronchospasm) that was successfully treated before cardiac arrest ensued. Cardiovascular events originated from a variety of causes but were treated successfully to prevent cardiac arrest. Cardiac arrest usually arose from respiratory or cardiovascular events. In the
category “care escalation,” medication errors accounted for 65%, equipment dysfunction 24%, blood reactions 9%, malignant hyperthermia 1%, and operating room fire 1% of SAEs. In care escalation, the patient did not experience long-term or transient physical harm; however, significant health care resources were devoted to prevent this from occurring. The expended resources included unplanned admission to the hospital or intensive care unit along with administration of additional medications, consumption of supplies, and health care provider time. Figure 5 illustrates the percentage of member anesthesiologists completing the QI and SA training since start up. Of the member anesthesiologists, very few had QI or SA expertise at start up. Workshops were conducted in November 2011 and March 2013 that trained 24 and 30 member anesthesiologists in QI and SA, respectively, that
Figure 3. Run chart for cumulative number of serious adverse events and cumulative number of anesthetics reported to the database since launch of the program.
Figure 4. Pareto diagram showing the cumulative frequency of the various categories of serious adverse events (SAE). Numbers in the bars represent the number of SAE in the category. The number in the boxes represents the cumulative percentage moving from most frequent to least frequent category of SAE. Total number of SAE is 740.
July 2014 • Volume 119 • Number 1
www.anesthesia-analgesia.org
117
Quality Improvement-Driven Safety Program
Figure 5. Run chart for the percentage of member anesthesiologists completing the safety analytics and quality improvement training since inception of the program. Goal is 90%.
abruptly increased the percentage of members completing the training at those time points. The addition of member anesthesiologists without training after the QI workshop decreased the percentage with QI training after November 2011. Among institutions, the number of member anesthesiologists ranged from 1 to 5 based on the number of anesthetics administered annually at the institution (1 member anesthesiologist per 10,000–15,000 anesthetics at the institution). As of March 2013, 70% of the member anesthesiologists were trained in SA and QI, below our goal of 90%, with 33% to 100% and 0% to 100% of member anesthesiologists at each institution completing the SA and QI training, respectively. Member anesthesiologists delivered oral presentations at the SPA meetings and authored manuscripts during 2011, 2012, and 2013 that were 14%, 13%, and 11% of member anesthesiologists, respectively, at the goal (10%). Before 2011, there were no presentations or manuscripts.
DISCUSSION
Over the years, many anesthesia organizations created subsidiaries to improve safety in clinical care. Perhaps, the better known include the ASA-CCS, POCA, the ASA Anesthesia Quality Institute (AQI), the Australian and New Zealand College of Anaesthetists Mortality Working Group (ANZCA-MWG), and the Royal College of Anaesthetists Safe Anaesthesia Liaison Group. SPA-WUS is another subsidiary organization dedicated to improving safety, although somewhat distinct in the approach and focus, applying SA and QI to empower anesthesia departments with these tools to decrease SAE specifically in children undergoing anesthesia. The present report illustrates this approach using QI to plan and implement this in subsidiary organizations and to monitor and communicate progress on its goals to the membership. WUS chose a somewhat different definition for safety events than the other anesthesia safety organizations.
118 www.anesthesia-analgesia.org
WUS chose to focus on SAE during anesthesia or within 24 hours after anesthesia, which include several categories of untoward occurrences that result in death, disability, are life threatening, or require hospitalization. ANZCA-MWG and POCA focused on death and cardiac arrest, respectively, a fraction of SAE that patients experience. AQI tracks safety as well as anesthesia processes and quality metrics from an anesthesiologist’s perspective. ASA-CCS and Royal College of Anaesthetists Safe Anaesthesia Liaison Group review legal cases and administrative reports that constitute a fraction of safety events involving patient harm. WUS chose a patient and perioperative medicine perspective for the definition that encompassed more types of events than the other organizations. In addition, the time was extended to 24 hours after anesthesia, and the events were not limited to those only involving anesthesia factors or errors attributed to anesthesia care. In fact, many SAE experienced by patients are not attributed to 1 provider or specialty nor is the etiology always apparent or known. By taking a patient-oriented definition, WUS hopes to breakdown the specialty silos to safety and identify research opportunities involving multidisciplinary collaboration to prevent events without a currently known etiology. Getting an accurate number for SAE and anesthetics administered to calculate an incidence of SAE was important to assess progress toward our smart aim to decrease SAE per anesthetic by 10% in 5 years. Despite considerable time and effort to capture SAE, the incidence of SAE does contain some uncertainty. Several sources account for this uncertainty. In the denominator, the number of anesthetics seems an unlikely cause of uncertainty, because the number originates from billing data that are audited by accountants in the departments and hospitals. In contrast, the number of SAE captured is less certain for several reasons. Institutions determine the numerator through voluntary reports by anesthesiologists (quality assurance) and reports by nurses, other
anesthesia & analgesia
physicians, pharmacists, and patient families to the hospital (hospital safety reports, legal reports, electronic drug audits- trigger tools, family complaints). Potential errors in the numerator include not capturing a report (event was not voluntarily reported or not detected by hospital reporting systems) and not sending a report to WUS by the institution (an event was captured at the institution but analyzed incorrectly as not a SAE). Not sending a report is no doubt negligible compared with not capturing a report because the former is analyzed by 3 anesthesiologists at the institution. It is unlikely that not capturing a report was substantial, because SAE have high visibility in the clinical environment, are relatively uncommon, and detection is multifaceted. Nevertheless, the true incidence of SAE is likely more than that reported by WUS. Yet this uncertainty does not negate the ability of WUS to assess progress toward its goal, because the goal is a percent decrease from the incidence in 2011 to 2013. As long as the method of determining incidence remains stable, WUS can assess its progress. The incidence of SAE in WUS, 1.4 per 1000 anesthetics, is in the range of events reported by others for pediatric anesthesia during the past decade.11,13–15,18,22 Among these studies, differences in the definitions of events and types of procedures and anesthetics that the patient population underwent partly explain the variation in incidence.11,13–15,18,22 Tay et al.14 and Cravero et al.13 observed incidences of 29 per 1000 anesthetics and 57 per 1000 anesthetics, respectively, while Murat et al.11 reported an incidence of events of 31 per 1000 anesthetics in the operating room and 48 per 1000 anesthetics in the postanesthesia care unit. These studies included events of less severity than WUS that would account for a higher incidence than in WUS. Marcus25 and Couloures et al.18 noted incidences for major complications of 1 per 1000 anesthetics and 0.9 per 1000 anesthetics, respectively, in radiology, whereas Bennet et al.15 observed an incidence of 93 per 1000 anesthetics in the cardiac catheterization laboratory. The lower incidence in radiology and higher incidences in cardiac catheterization compared with that of WUS might be explained by populations at different risk for complications. In 2 large studies, cardiac arrest during anesthesia occurred at 0.16 to 0.87 per 1000 anesthetics, similar to that in WUS at 0.3 per 1000 anesthetics.11,16 From the ANZA database, van der Griend et al.17 reported mortality within 24 hours after anesthesia at 1.3 per 1000 anesthetics, more than WUS at 0.1 per 1000 anesthetics. Considering the variations in definitions and populations of children undergoing surgery and anesthesia, the similarity of incidences in WUS and other studies suggests the methodology used by WUS works reasonably well. Among the types of SAE in WUS, respiratory events were most common, followed by cardiac arrest, care escalation, and cardiovascular events, collectively accounting for 76% of events. Our finding of respiratory events as the most common types in pediatric anesthesia concurs with other studies.11,14,26,27 Care escalation was a category of untoward occurrences that required unplanned hospitalization or prolongation of hospitalization and did not primarily fit into another category of organ system harm (e.g., respiratory, cardiovascular). In care escalation, the patient did not experience severe temporary or permanent injury although significant additional health care resources
July 2014 • Volume 119 • Number 1
were devoted to prevent this from occurring. Medication errors and equipment malfunction accounted for nearly 90% of care escalation events; in our SA, defects in anesthetic processes played dominant roles in these SAE, unlike the other categories in which patient and surgical factors played large roles. Other studies also report equipment, and medication events constitute a similar or higher percentage of total safety events in pediatric anesthesia than in WUS.7,9,11,25 From a strategic viewpoint, QI projects for equipment or medication processes are good initial projects for WUS because anesthesia factors play a large role in care escalation category. WUS has similarities and differences from the other anesthesia safety organizations. All organizations receive inputs of information and materials that are used to deliver outputs to meet their mission. An input in WUS that is similar to POCA, AQI, and ANZCA is the voluntary reporting of events by many departments across a country into an electronic information system. ANZCA and WUS also have in common the use of SA, a formal structure and process to categorize and learn about events. Outputs in WUS that are similar to these organizations include lectures, newsletters, conferences, and original manuscripts about the events to educate anesthesiologists so that they may alter their practice accordingly. There are several differences between WUS and the other organizations. First, WUS inputs and outputs remain undeveloped compared with the other organizations, as the number of events in the database, presentations, and publications are few by comparison. Second, WUS focuses on pediatric anesthesia and is embedding SA and QI in each anesthesia department by training member anesthesiologists to drive anesthesia practice, not relying solely on education of the individual practitioner to improve safety. This ambitious program required 3 years of planning to form a functioning organization. One year passed from idea to an organization on paper and nearly 1 more year before the first institution joined. Institution concerns about reputation, confidentiality, and liability prevented many from initially joining the organization that dissipated when WUS became a PSO. The Patient Safety and Quality Improvement Act of 2005 authorized the creation of PSOs to improve the quality and safety of U.S. health care delivery. The PSO establishes a framework by which hospitals, doctors, and other health care providers may collect and analyze information and prevents the use of this information in criminal, civil, administrative, or disciplinary proceedings. These strong protections enable the WUS institutions and member anesthesiologists to engage in detailed discussions about the causes of SAE without the fear of liability. Consequently, 10 institutions joined within 1 year of PSO designation. At year 3, WUS stopped recruiting to remain at 10 institutions. The executive board recognized the interventions to accomplish key driver 2 were becoming increasingly difficult as group size became larger. Scheduling meetings to discuss definitions and visit the data management companies were quite challenging with 10 institutions. When key driver 2 was completed, recruitment was resumed to add new institutions at 3 to 4 per year until the goal. It has been faster and easier to get the input side of WUS operational than the output side. Nearly 3 years were
www.anesthesia-analgesia.org
119
Quality Improvement-Driven Safety Program
required to establish the inputs of structure, governance, and event definitions culminating in the electronic registry, whereas it has taken 4 years to get the outputs of education and research productive to goal, and we have yet to establish high-reliability units in each institution. To establish high-reliability units, it will be necessary to train a critical mass of anesthesiologists in QI and SA at the institutions, and WUS remains well below the goal in this regard. Very few pediatric anesthesiologists possess expertise in QI or SA because it is not part of anesthesia residency or fellowship. It has been particularly difficult to get to goal for QI and SA expertise because as institutions joined WUS, the number of untrained member anesthesiologists increased, and in turn, the percentage of trained member anesthesiologists decreased. Looking ahead, QI and SA training will move front and center to the WUS Key Driver Diagram because both are requisite to imbed high reliability in the anesthesia departments. Safety requires processes with high reliability or, conversely, with a low defect rate. Reliability is expressed in log units of the defect rate: 1 defect in 10 (10–1) is level 1, 1 defect in 100 (10–2) is level 2, etc. Safe systems display reliability at level 5 or higher.19–21 Education in and of itself of a process yields level 1 reliability; thus, educating anesthesia providers through newsletters, conferences, and lectures when the incidence of events is well
National Pediatric Anesthesia Safety Quality Improvement Program in the United States C. Dean Kurth, MD,* Don Tyler, MD,† Eugenie Heitmiller, MD, FAAP,‡ Steven R. Tosone, MD,§ Lynn Martin, MD, MBA,║ and Jayant K. Deshpande, MD, MPH¶ BACKGROUND: As pediatric anesthesia has become safer over the years, it is difficult to quantify these safety advances at any 1 institution. Safety analytics (SA) and quality improvement (QI) are used to study and achieve high levels of safety in nonhealth care industries. We describe the development of a multiinstitutional program in the United States, known as Wake-Up Safe (WUS), to determine the rate of serious adverse events (SAE) in pediatric anesthesia and to apply SA and QI in the pediatric anesthesia departments to decrease the SAE rate. METHODS: QI was used to design and implement WUS in 2008. The key drivers in the design were an organizational structure; an information system for the SAE; SA to characterize the SAE; QI to imbed high-reliability care; communications to disseminate the learnings; and engaged leadership in each department. Interventions for the key drivers, included Participation Agreements, Patient Safety Organization designation, IRB approval, Data Management Co., membership fee, SAE standard templates, SA and QI workshops, and department leadership meetings. RESULTS: WUS has 19 institutions, 39 member anesthesiologists, 734 SAE, and 736,365 anesthetics as of March, 2013. The initial members joined at year 1, and initial SAE were recorded by year 2. The SAE rate is 1.4 per 1000 anesthetics. Of SAE, respiratory was most common, followed by cardiac arrest, care escalation, and cardiovascular, collectively 76% of SAE. In care escalation, medication errors and equipment dysfunction were 89%. Of member anesthesiologists, 70% were trained in SA and QI by March 2013; virtually, none had SA and QI expertise before joining WUS. CONCLUSION: WUS documented the incidence and types of SAE nationally in pediatric anesthesiology. Education and application of QI and SA in anesthesia departments are key strategies to improve perioperative safety by WUS. (Anesth Analg 2014;119:112–21)
Q
uality improvement (QI) science as applied to safety in anesthesia has evolved considerably over the past 30 years. Cooper et al.’s1,2 pioneering work beginning in the 1970s on analysis of critical incidents led to a systematic approach to analyzing errors, learning from them, and then modifying anesthetic technique through education. This methodology is now largely incorporated in anesthesia practice. The American Society of Anesthesiologists Closed Claims Study (ASA-CCS) is another landmark in adverse From the *Department of Anesthesiology, Cincinnati Children’s Hospital Medical Center, University of Cincinnati College of Medicine, Cincinnati, Ohio; †Department of Anesthesiology and Critical Care Medicine, Children’s Hospital of Philadelphia, University of Pennsylvania School of Medicine, Philadelphia, Pennsylvania; ‡Department of Anesthesiology and Critical Care Medicine, Johns Hopkins University School of Medicine, The Johns Hopkins Hospital, Baltimore, Maryland; §Department of Anesthesiology and Pediatrics, Emory University School of Medicine, Atlanta, Georgia; ║Department of Anesthesiology & Pain Medicine, Seattle Children’s Hospital, University of Washington School of Medicine, Seattle, Washington; and ¶Department of Anesthesiology, Arkansas Children’s Hospital, University of Arkansas for Medical Sciences, Little Rock, Arkansas. Accepted for publication October 8, 2013. Funding: None. The authors declare no conflicts of interest. Reprints will not be available from the authors. Address correspondence to C. Dean Kurth, MD, Department of Anesthesiology, Cincinnati Children’s Hospital Medical Center, University of Cincinnati College of Medicine, 3333 Burnet Ave., Cincinnati, OH 45229. Address e-mail to [email protected]. Copyright © 2014 International Anesthesia Research Society DOI: 10.1213/ANE.0000000000000040
112 www.anesthesia-analgesia.org
event analysis.3–8 In this ongoing study, investigators access malpractice cases that have been settled or adjudicated and then enlist anesthesiologists to analyze the anesthetic technique to uncover errors that may have led to an untoward outcome. The findings from this effort also changed the way anesthesiology is currently practiced, as anesthesiologists learn from publication of the studies and adapt the learnings into their practice. The Perioperative Cardiac Arrest (POCA) project, a voluntary registry of pediatric cardiac arrests, initiated in 1997, identified common causes of cardiac arrest in children.7,9,10 Several important observations from this registry changed pediatric anesthesia practice, again through dissemination of the findings and education, which contributed to the demise of halothane, widespread use of sevoflurane in its place, and a decrease in cardiac arrest in pediatric anesthesia. As pediatric anesthesia has become safer over the years, it is difficult to quantify these safety advances at any 1 institution. The rate of critical events in pediatric anesthesia is low; therefore, the number of events occurring at a single institution precludes systematic study. More data are needed to provide greater precision concerning incidence, causation(s), statistical analysis, and process control to mitigate events.7,11–18 Outside the field of medicine, the science of safety analytics (SA) and QI have developed considerably during the past 30 years, particularly in the aviation, manufacturing, and nuclear industries.19–21 While some of this expertise has been deployed to medicine, it remains limited at any July 2014 • Volume 119 • Number 1
single institution or anesthesia department. As anesthesia and perioperative care have become more system oriented through team-based care using technology and informatics, SA and QI have become increasingly germane. Today, critical events require a systems view and an individual practitioner view to consider multiple factors, many external to the anesthesiologist. The Society for Pediatric Anesthesia (SPA) represents anesthesiologists and allied health care professionals who are engaged in the practice of pediatric anesthesia. The mission of the SPA is to continually advance the safety and quality of anesthetic care, perioperative management, and alleviation of pain in children. Given the evolution of system-based care in pediatric anesthesia, SPA supported the development of a national program to apply SA and QI to better enable departments to decrease the incidence of critical events in children undergoing anesthesia. We describe this program, known as “Wake-Up Safe” (WUS), and how it developed and is currently being executed using QI methods. QI methods contain 4 components: appreciation of a system, understanding variation, action learning, and change management.21,22 A system is an interdependent group working together toward a common purpose. QI defines the parts of the system to identify the linkages and get them to interact effectively. Understanding variation is embodied in statistical process control, a form of statistics found in manufacturing that differs from medical statistics, exemplified by run charts and special and common-cause variation. In action learning, QI follows the scientific method of aim, hypothesis, study design, and experimentation, appearing in the form of Key Driver Diagrams as smart aim, key drivers, and interventions. Change management embodies the disciplines of leadership, organizational behavior, and psychology to unify a system, communicate variation, conduct action learning, and thereby improve quality. Because QI differs somewhat from research, a different format was developed for QI manuscripts, known as the Standards for Quality Improvement Reporting Excellence (SQUIRE).23,24 In the present work, the SQUIRE format is used to describe the WUS program.
METHODS Setting
WUS was designed and implemented using QI methods during meetings of an executive team, the executive board, and full board of the organization. The executive team was a pilot group that was disbanded after the executive board and full board were established. The executive board consists of 4 elected individuals from the full board. The full board consists of 1 pediatric anesthesiologist from each institution in WUS. These groups met in person or via teleconference to create and implement the WUS QI project.
Ethical Issues
The executive team judged it prudent to obtain IRB approval at each institution because many journals require this to publish studies using registry data. We therefore developed a common IRB protocol not requiring informed consent that could be used by all institutions. The data submitted to the registry do not contain information that could be used to identify the patient or the institution (“de-identified”
July 2014 • Volume 119 • Number 1
data) and are stored in a secure database at Axio Research (Seattle, WA), a contract research organization. Data and medical records are stored in a secure location according to the quality assurance policy of the site.
Planning the Program and Study of Interventions
In early 2007, several SPA members began to discuss programs to improve quality and safety in pediatric anesthesia, which eventually focused on safety. An executive team was created to lead the project, which officially launched on January 1, 2008. The executive team comprises pediatric anesthesia chiefs from several children’s hospitals and board members from SPA and the Anesthesia Patient Safety Foundation. The executive team designed the Key Driver Diagram for the WUS QI project, which displays the strategic map for process improvement and contains the following components: Smart Aim, Key Drivers, and Interventions (Fig. 1). The Smart Aim expresses the goal in a specific, measurable, actionable, relevant, and time-bound sentence. The global aim is to eliminate all preventable harm to children undergoing anesthesia. WUS focuses on a component of harm, specifically serious adverse events (SAE), defined as untoward occurrences during anesthesia care or within 24 hours of the end anesthesia care that result in life-threatening injury, requiring unplanned hospitalization or prolongation of hospitalization, result in disability, incapacity, or death. In WUS, the Smart Aim is specific for SAE, is measured to decrease SAE by 10%, has actions that can be taken to decrease SAE incidence that include those listed in the interventions, include SAE that are relevant to pediatric anesthesia, and is time-bound to achieve the goal in 5 years (2010–2015). The Key Drivers to achieve the Smart Aim are the critical components of the QI project. For WUS, there are 6 key drivers: (1) An organizational structure with the right people and the money to fund it; (2) An information system to collect and disseminate data about the SAE; (3) A common analytical methodology to characterize the SAE; (4) QI capability to establish high-performing teams and imbed high-reliability interventions of care; (5) Communications to disseminate the learning about the events and successful interventions; and (6) Leadership support at each institution. Key Driver 1 (Organizational Structure) The executive team decided to become a “special interest group” within the SPA to use its infrastructure of legal advice, accounting, and general management. The executive team developed bylaws to delineate the governance and requirements to become a Patient Safety Organization (PSO), sanctioned by the Federal Government Agency for Healthcare Research and Quality. The bylaws disbanded the executive team and created a full board composed of 1 representative from each institution, an executive board composed of president, vice president, secretary, and treasurer elected for a 1-year term by the full board, and an appointed executive administrator. At each institution, there are 1 to 4 anesthesiologists participating in analyzing SAE, and 1 or 2 support personnel who assist with entering SAE information in the database and providing denominator data. Of the member anesthesiologists, one serves on the full board, and all are expected to learn SA and QI and
www.anesthesia-analgesia.org
113
Quality Improvement-Driven Safety Program
INTERVENTIONS
KEY DRIVERS People, structure and finance to form organization Develop system to collect information about events SMART AIM Reduce preventable anesthesia related serious events 10% by 2017 GLOBAL AIM Eliminate preventable harm in children undergoing anesthesia
Safety science analytics to understand etiology of events Quality Improvement to embed capability to drive safe practices Spread knowledge to institutions to adopt safe practices Institutional leadership support to fund initiatives and reward safe practices
People 1. Recruit executive team 2. Recruit key participants to achieve a breakeven membership of 30 Structure 1. Develop Bylaws & Participation Agreement 2. Become Patient Safety Organization 3. Obtain IRB approval Finance 1. Develop funding & financial controls 1. Develop definitions and questions about events 3. Collect demographic data to understand frequency of events 4. Hire data management company 5. Develop system to get reports of data 1. Educate participating institutions 2. Workshop on Root Cause Methodology 1. Educate participating institutions 2. Members to complete IHI course 3. Workshop to develop QI program 1. Newsletter to members informing them of SAE learnings 2. Author manuscripts from database, QI and SA projects 3. National presentations from database, QI and SA projects 4. Develop peer visitation program 1. Engage department chairs, hospital CMOs & CEOs 2. Communications to PALC, CHA, ASA, NSQUIP
Figure 1. Key driver diagram for wake-up safe quality improvement program. The smart aim is the goal for the program, which is part of a global aim for the Society for Pediatric Anesthesia. The key drivers are the components that are required for the program to be fully functioning to achieve the smart aim. The interventions are the activities that are performed to make the key drivers fully operational. Interventions in the green boxes indicate those that are completed; those in the white boxes are ongoing and not yet completed.
able to lead projects. The chief of anesthesia of the institution selects the member anesthesiologists and the representative to the full board. A participation agreement was signed between SPA and the institutions. The agreement mainly clarified issues of intellectual property and identification of the institutions and the potential impact on reputation and litigation of reported cases. The agreement specified that the institutions’ identities would be protected. A membership business model was selected to finance the organization in which the institutions contributed money to join (initiation fee) and to remain a member (annual dues). The program requires 30 institutions paying annual dues to support the program’s infrastructure and requires the institutions to collectively administer 400,000 anesthetics per year to conduct safety-oriented QI projects with statistical significance. The SPA and Anesthesia Patient Safety Foundation helped to fund the initial startup. The benefits of membership included the reports about the SAE, workshops and networking to learn SA and QI to drive high-reliability methods into local practice, and scholarship to present and publish the learnings. Interventions to recruit institutions and anesthesiologists included an executive team espousing the benefits of WUS to departmental and hospital leadership, executive team personally contacting pediatric anesthesiologists known to have interests in SA and QI, executive team personally speaking with the department chiefs at the 6 largest freestanding children’s hospitals, and the executive administrator providing expertise to potential institutions to get the participation agreement and IRB document approved.
114 www.anesthesia-analgesia.org
Key Driver 2 (Information System) The system to collect information about the events was designed to be easy to input the information into, to query the system to report isolated, unusual, or high-impact events, to import into statistical software to describe and analyze the events, and to keep the information confidential and secure. A Web-based information system fit these criteria. Because the institutions and the SPA did not have this expertise, a request for proposal was distributed and 6 contract research organizations that were well versed in health care databases responded. After reviewing the responses, 3 were interviewed and site visited. Axio Research (Seattle, WA) was selected, and a contract was signed. The full board and Axio Research developed the database, Web interface, and online reporting tool. Considerable time was spent categorizing, naming, and defining the SAE (Table 1). The full board decided to view SAE from the patient perspective and not just the anesthesiologist view, to shift our thinking broadly, as perioperative pediatric physicians. Accordingly, SAE were categorized as death, life threatening, disability/incapacity, or unplanned/ prolonged hospitalization. For each SAE category, a series of questions was constructed using templates to describe it by severity, primary cause, and contributing factors, and whether the SAE was the result of the anesthesia, surgery, or patient condition. The templates and questions followed the common formats for PSO. Several pilots were conducted using fabricated cases to investigate concordance among member anesthesiologists; template questions were repeatedly revised until concordance was achieved. At the institution, 3 anesthesiologists
anesthesia & analgesia
Table 1. Categories of Events and Their Definitions Serious adverse event Death Life threatening
Category Death Airway event
Life threatening
Cardiac arrest
Disability/incapacity Disability/incapacity
Awareness anesthesia MSJC injury
Disability/incapacity Disability/incapacity Disability/incapacity Hospitalization Hospitalization or life threatening Hospitalization
Nervous system injury Visual system injury Wrong surgery Acute lung injury Cardiovascular event Care escalation
Definition Death during anesthesia or within 24 h end anesthesia Severe hypoxemia or hypercapnia receiving emergent intubation and medications during anesthesia or within 24 h after anesthesia Bradycardia or hypotension receiving cardiac compressions during anesthesia or within 24 h end anesthesia Patient memory of events during general anesthesia Injury to muscle, bone, joint, skin, or soft tissue from anesthesia care limiting function receiving medical or surgical treatment Injury to brain, spinal cord, or peripheral nerves unrelated to surgery limiting function Injury to eye or visual pathway unrelated to surgery lasting beyond 24 h after anesthesia Start of surgery or anesthesia care on body part that was not consented for procedure New onset impaired gas exchange with parenchyma signs that escalates care Unanticipated cardiovascular support during anesthesia or within 24 h end anesthesia that escalates care Fire, malignant hyperthermia, medication, or equipment error during anesthesia care that escalates care
SAE = serious adverse event (untoward occurrences during anesthesia care or within 24 hours of end anesthesia care, that result in death, are life threatening, require unplanned or prolongation of hospitalization, or result in disability or incapacity); MSJC = musculoskeletal, joint, cutaneous.
not involved in the case analyzed the case using the template questions. The information entered in the database was the consensus of the analyzing anesthesiologists. Institutions ascertained the numerator through voluntary quality assurance reports by anesthesiologists and nurses, other physicians, pharmacists, and patient families to the hospital (hospital safety reports, legal reports, electronic drug auditstrigger tools, family complaints). Interventions to obtain the SAE cases and total anesthetic cases in the database included meetings between Axio Research and institution information technology support personnel to import total anesthetic cases using billing databases, teaching member anesthesiologists at board meetings how to import the SAE cases in the database, having the executive administrator personally speaking with member anesthesiologists to conduct the analyses and import the SAE cases in a timely manner, member anesthesiologists encouraging their colleagues to report cases, generating SAE reports from the database to institutions to provide feedback, and leadership in the department encouraging or incentivizing anesthesiologists to report SAE cases, such as withholding bonuses for failure to report and/ or public praise for reporting. The use of incentive remained at the discretion of the department leadership and was not standardized across institutions. Key Driver 3 (Analytical Methodology) SA aims to determine the primary and secondary causes for the SAE. SA is a rigorous, detailed, and established science, as exemplified by transportation industry and federal agencies, involving “root” cause, common cause, contributing factors, sequence of events, and systems analyses. Based on this analysis of the information, strategies can be developed, and processes can be applied to mitigate the risk of future events. Recently, several hospitals have begun to learn and apply SA to determine causes of serious safety and “never” events. The executive team initially envisioned each anesthesia department to have this capability; however, it soon became apparent that many did not. Interventions to train the member anesthesiologists in SA occurred through the members attending a WUS-run workshop facilitated by an expert and members attending external workshops and courses that were approved by the full board.
July 2014 • Volume 119 • Number 1
Key Driver 4 (QI Capability) Knowing who is at risk of an event and why the event occurs does not reliably prevent future events; in other words, information and education are not high-reliability processes. QI uses high-reliability processes such as checklists, standardization, technologies, and automation that have become the cornerstone to ultrasafe industries. As with SA, the executive team envisioned each anesthesia department to have QI capability and likewise discovered that few had it. Interventions to train the member anesthesiologists consisted of WUS purchasing a subscription to online courses developed by the Institute for Healthcare Improvement for members to complete the courses, members attending a WUS-run workshop on QI facilitated by an expert, and members attending external workshops and courses that were approved by the full board. Key Driver 5 (Communications) This concerns spread of knowledge and adoption of successful processes from one institution to another. Adoption of best practice from anesthesia provider to provider within an institution takes months, and from anesthesia provider to provider at different institutions takes years. For example, hyperkalemic cardiac arrest associated with a large volume blood transfusion remains a problem 15 years after it was reported by POCA. This illustrates the slow spread and adoption of processes to mitigate hyperkalemia by those caring for the bleeding patient and those managing the supply of blood products. Interventions to hasten spread and adoption include a peer visitation program similar to what the nuclear industry has used successfully, an electronic newsletter to institutions describing events reported to the database similar to what the air force uses to inform pilots of near misses, presentations at national conferences, and publications of events from the database and QI and SA projects in books, journals, and society newsletters. Key Driver 6 (Leadership) The final key driver pertains to leadership support by the institution. For WUS to be successful, the member anesthesiologists require nonclinical time and financial support to participate in the organization. In addition, the WUS QI
www.anesthesia-analgesia.org
115
Quality Improvement-Driven Safety Program
projects obviously require institutional and anesthesia chief support for the projects to be successful. Finally, the annual membership fee requires the department chief’s endorsement. Interventions consisted of WUS being a standing agenda item at the twice annual Pediatric Anesthesia Leadership Council, the organization comprises the pediatric anesthesia chairs, directors, and chiefs from the children’s hospitals and university hospitals in the United States, to bring important matters of WUS to their attention.
Methods of Evaluation
A specific metric was associated with each key driver. For key driver 1, the goal is to increase membership to the target of 30 institutions and to have at each institution 1 member anesthesiologist per 10,000 to 15,000 anesthetics. Thus, the metrics are number of institutions and member anesthesiologists. An institution was considered a member when a participation agreement was signed, the IRB was approved, and the initiation dues were paid. An anesthesiologist was considered a member when enrolled by the executive administrator. The executive administrator reported the number of institutions and member anesthesiologists against the goal every 3 months. For key driver 2, the goal is complete reporting of SAE’s and the total number of anesthetics by the institutions. Accordingly, the metric is completed reports for the number of SAE, the types of SAE, and the total number of anesthetics in the database. For key drivers 3 and 4, the goals are to establish expertise in SA and QI in the anesthesia departments that would be indicated by 90% of member anesthesiologists completing SA and QI training. The metrics are the percentages of member anesthesiologists who had completed SA and QI training. The goal for key driver 5 is dissemination of knowledge about SAE, SA, and QI that would be indicated by 10% of WUS institutions each year authoring manuscripts or presenting at national meetings about the information in the database or results of QI or SA projects. Accordingly, the metric is number of presentations and publications per year by the WUS institutions using the database and training. There was no duplicate counting for coauthorship of manuscripts.
Analysis
Statistical process control run charts were used to measure the impact of the interventions over time on the metrics for the key drivers. Run charts can distinguish common-cause and special-cause variation and can evaluate the effectiveness of the interventions. An observed variation is considered statistically significant in run charts when ≥6 consecutive points are increasing or decreasing, or ≥8 consecutive points are either above or below the mean centerline.21 The preintervention time was defined from the official launch of WUS (January 1, 2008) until the first intervention for the key driver was begun.
RESULTS
Figure 1 demonstrates the key drivers for the WUS program. Of the 6 key drivers, only the system to collect information about the events (key driver 2) is complete. The membership continues to implement the interventions to complete the other key drivers. Figure 2 displays the number of institutions and member anesthesiologists in WUS since start up, representing the sum of interventions to complete key driver 1. It took 10 months before any institution joined; thereafter, a steady increase in membership occurred. Member anesthesiologists in an institution were often added incrementally after the institution joined, and thus, the number of member anesthesiologists could increase even though the number of institutions did not change. In March 2013, WUS has 19 institutions and 39 member anesthesiologists, representing 65% of intended membership. The recruitment of institutions was stopped after year 2 because key drivers 1 and 2 were insufficiently developed to support more members. After 30 months, recruitment was restarted because the structure for key driver 1 and all of key driver 2 were completed. Figure 3 depicts the number of SAE and anesthetics reported to the database since inception. SAE and anesthetic reports began in July 2010, approximately 30 months after start up and 20 months after the first institution joined, and have continued to increase. Since January 2011, the average number of SAE reported into the database is 28 per month.
Figure 2. Run chart for number of institutions and member anesthesiologists since inception of the program.
116 www.anesthesia-analgesia.org
anesthesia & analgesia
The average number of anesthetics per institution is 19,800 per year. During the past 24 months, the membership collectively conducted 240,000 anesthetics per year, with approximately 1.4 SAE per 1000 anesthetics. Of the reported SAE, the most common was respiratory events, followed by cardiac arrest, care escalation, and cardiovascular events, which collectively accounted for 76% of events (Fig 4). Visual system injury, “wrong surgery,” and awareness under anesthesia were rare, representing 1.1%, 0.4%, and 0.3 % of SAE, respectively. Respiratory events usually arose from complete airway obstruction (pharyngeal obstruction, laryngospasm, bronchospasm) that was successfully treated before cardiac arrest ensued. Cardiovascular events originated from a variety of causes but were treated successfully to prevent cardiac arrest. Cardiac arrest usually arose from respiratory or cardiovascular events. In the
category “care escalation,” medication errors accounted for 65%, equipment dysfunction 24%, blood reactions 9%, malignant hyperthermia 1%, and operating room fire 1% of SAEs. In care escalation, the patient did not experience long-term or transient physical harm; however, significant health care resources were devoted to prevent this from occurring. The expended resources included unplanned admission to the hospital or intensive care unit along with administration of additional medications, consumption of supplies, and health care provider time. Figure 5 illustrates the percentage of member anesthesiologists completing the QI and SA training since start up. Of the member anesthesiologists, very few had QI or SA expertise at start up. Workshops were conducted in November 2011 and March 2013 that trained 24 and 30 member anesthesiologists in QI and SA, respectively, that
Figure 3. Run chart for cumulative number of serious adverse events and cumulative number of anesthetics reported to the database since launch of the program.
Figure 4. Pareto diagram showing the cumulative frequency of the various categories of serious adverse events (SAE). Numbers in the bars represent the number of SAE in the category. The number in the boxes represents the cumulative percentage moving from most frequent to least frequent category of SAE. Total number of SAE is 740.
July 2014 • Volume 119 • Number 1
www.anesthesia-analgesia.org
117
Quality Improvement-Driven Safety Program
Figure 5. Run chart for the percentage of member anesthesiologists completing the safety analytics and quality improvement training since inception of the program. Goal is 90%.
abruptly increased the percentage of members completing the training at those time points. The addition of member anesthesiologists without training after the QI workshop decreased the percentage with QI training after November 2011. Among institutions, the number of member anesthesiologists ranged from 1 to 5 based on the number of anesthetics administered annually at the institution (1 member anesthesiologist per 10,000–15,000 anesthetics at the institution). As of March 2013, 70% of the member anesthesiologists were trained in SA and QI, below our goal of 90%, with 33% to 100% and 0% to 100% of member anesthesiologists at each institution completing the SA and QI training, respectively. Member anesthesiologists delivered oral presentations at the SPA meetings and authored manuscripts during 2011, 2012, and 2013 that were 14%, 13%, and 11% of member anesthesiologists, respectively, at the goal (10%). Before 2011, there were no presentations or manuscripts.
DISCUSSION
Over the years, many anesthesia organizations created subsidiaries to improve safety in clinical care. Perhaps, the better known include the ASA-CCS, POCA, the ASA Anesthesia Quality Institute (AQI), the Australian and New Zealand College of Anaesthetists Mortality Working Group (ANZCA-MWG), and the Royal College of Anaesthetists Safe Anaesthesia Liaison Group. SPA-WUS is another subsidiary organization dedicated to improving safety, although somewhat distinct in the approach and focus, applying SA and QI to empower anesthesia departments with these tools to decrease SAE specifically in children undergoing anesthesia. The present report illustrates this approach using QI to plan and implement this in subsidiary organizations and to monitor and communicate progress on its goals to the membership. WUS chose a somewhat different definition for safety events than the other anesthesia safety organizations.
118 www.anesthesia-analgesia.org
WUS chose to focus on SAE during anesthesia or within 24 hours after anesthesia, which include several categories of untoward occurrences that result in death, disability, are life threatening, or require hospitalization. ANZCA-MWG and POCA focused on death and cardiac arrest, respectively, a fraction of SAE that patients experience. AQI tracks safety as well as anesthesia processes and quality metrics from an anesthesiologist’s perspective. ASA-CCS and Royal College of Anaesthetists Safe Anaesthesia Liaison Group review legal cases and administrative reports that constitute a fraction of safety events involving patient harm. WUS chose a patient and perioperative medicine perspective for the definition that encompassed more types of events than the other organizations. In addition, the time was extended to 24 hours after anesthesia, and the events were not limited to those only involving anesthesia factors or errors attributed to anesthesia care. In fact, many SAE experienced by patients are not attributed to 1 provider or specialty nor is the etiology always apparent or known. By taking a patient-oriented definition, WUS hopes to breakdown the specialty silos to safety and identify research opportunities involving multidisciplinary collaboration to prevent events without a currently known etiology. Getting an accurate number for SAE and anesthetics administered to calculate an incidence of SAE was important to assess progress toward our smart aim to decrease SAE per anesthetic by 10% in 5 years. Despite considerable time and effort to capture SAE, the incidence of SAE does contain some uncertainty. Several sources account for this uncertainty. In the denominator, the number of anesthetics seems an unlikely cause of uncertainty, because the number originates from billing data that are audited by accountants in the departments and hospitals. In contrast, the number of SAE captured is less certain for several reasons. Institutions determine the numerator through voluntary reports by anesthesiologists (quality assurance) and reports by nurses, other
anesthesia & analgesia
physicians, pharmacists, and patient families to the hospital (hospital safety reports, legal reports, electronic drug audits- trigger tools, family complaints). Potential errors in the numerator include not capturing a report (event was not voluntarily reported or not detected by hospital reporting systems) and not sending a report to WUS by the institution (an event was captured at the institution but analyzed incorrectly as not a SAE). Not sending a report is no doubt negligible compared with not capturing a report because the former is analyzed by 3 anesthesiologists at the institution. It is unlikely that not capturing a report was substantial, because SAE have high visibility in the clinical environment, are relatively uncommon, and detection is multifaceted. Nevertheless, the true incidence of SAE is likely more than that reported by WUS. Yet this uncertainty does not negate the ability of WUS to assess progress toward its goal, because the goal is a percent decrease from the incidence in 2011 to 2013. As long as the method of determining incidence remains stable, WUS can assess its progress. The incidence of SAE in WUS, 1.4 per 1000 anesthetics, is in the range of events reported by others for pediatric anesthesia during the past decade.11,13–15,18,22 Among these studies, differences in the definitions of events and types of procedures and anesthetics that the patient population underwent partly explain the variation in incidence.11,13–15,18,22 Tay et al.14 and Cravero et al.13 observed incidences of 29 per 1000 anesthetics and 57 per 1000 anesthetics, respectively, while Murat et al.11 reported an incidence of events of 31 per 1000 anesthetics in the operating room and 48 per 1000 anesthetics in the postanesthesia care unit. These studies included events of less severity than WUS that would account for a higher incidence than in WUS. Marcus25 and Couloures et al.18 noted incidences for major complications of 1 per 1000 anesthetics and 0.9 per 1000 anesthetics, respectively, in radiology, whereas Bennet et al.15 observed an incidence of 93 per 1000 anesthetics in the cardiac catheterization laboratory. The lower incidence in radiology and higher incidences in cardiac catheterization compared with that of WUS might be explained by populations at different risk for complications. In 2 large studies, cardiac arrest during anesthesia occurred at 0.16 to 0.87 per 1000 anesthetics, similar to that in WUS at 0.3 per 1000 anesthetics.11,16 From the ANZA database, van der Griend et al.17 reported mortality within 24 hours after anesthesia at 1.3 per 1000 anesthetics, more than WUS at 0.1 per 1000 anesthetics. Considering the variations in definitions and populations of children undergoing surgery and anesthesia, the similarity of incidences in WUS and other studies suggests the methodology used by WUS works reasonably well. Among the types of SAE in WUS, respiratory events were most common, followed by cardiac arrest, care escalation, and cardiovascular events, collectively accounting for 76% of events. Our finding of respiratory events as the most common types in pediatric anesthesia concurs with other studies.11,14,26,27 Care escalation was a category of untoward occurrences that required unplanned hospitalization or prolongation of hospitalization and did not primarily fit into another category of organ system harm (e.g., respiratory, cardiovascular). In care escalation, the patient did not experience severe temporary or permanent injury although significant additional health care resources
July 2014 • Volume 119 • Number 1
were devoted to prevent this from occurring. Medication errors and equipment malfunction accounted for nearly 90% of care escalation events; in our SA, defects in anesthetic processes played dominant roles in these SAE, unlike the other categories in which patient and surgical factors played large roles. Other studies also report equipment, and medication events constitute a similar or higher percentage of total safety events in pediatric anesthesia than in WUS.7,9,11,25 From a strategic viewpoint, QI projects for equipment or medication processes are good initial projects for WUS because anesthesia factors play a large role in care escalation category. WUS has similarities and differences from the other anesthesia safety organizations. All organizations receive inputs of information and materials that are used to deliver outputs to meet their mission. An input in WUS that is similar to POCA, AQI, and ANZCA is the voluntary reporting of events by many departments across a country into an electronic information system. ANZCA and WUS also have in common the use of SA, a formal structure and process to categorize and learn about events. Outputs in WUS that are similar to these organizations include lectures, newsletters, conferences, and original manuscripts about the events to educate anesthesiologists so that they may alter their practice accordingly. There are several differences between WUS and the other organizations. First, WUS inputs and outputs remain undeveloped compared with the other organizations, as the number of events in the database, presentations, and publications are few by comparison. Second, WUS focuses on pediatric anesthesia and is embedding SA and QI in each anesthesia department by training member anesthesiologists to drive anesthesia practice, not relying solely on education of the individual practitioner to improve safety. This ambitious program required 3 years of planning to form a functioning organization. One year passed from idea to an organization on paper and nearly 1 more year before the first institution joined. Institution concerns about reputation, confidentiality, and liability prevented many from initially joining the organization that dissipated when WUS became a PSO. The Patient Safety and Quality Improvement Act of 2005 authorized the creation of PSOs to improve the quality and safety of U.S. health care delivery. The PSO establishes a framework by which hospitals, doctors, and other health care providers may collect and analyze information and prevents the use of this information in criminal, civil, administrative, or disciplinary proceedings. These strong protections enable the WUS institutions and member anesthesiologists to engage in detailed discussions about the causes of SAE without the fear of liability. Consequently, 10 institutions joined within 1 year of PSO designation. At year 3, WUS stopped recruiting to remain at 10 institutions. The executive board recognized the interventions to accomplish key driver 2 were becoming increasingly difficult as group size became larger. Scheduling meetings to discuss definitions and visit the data management companies were quite challenging with 10 institutions. When key driver 2 was completed, recruitment was resumed to add new institutions at 3 to 4 per year until the goal. It has been faster and easier to get the input side of WUS operational than the output side. Nearly 3 years were
www.anesthesia-analgesia.org
119
Quality Improvement-Driven Safety Program
required to establish the inputs of structure, governance, and event definitions culminating in the electronic registry, whereas it has taken 4 years to get the outputs of education and research productive to goal, and we have yet to establish high-reliability units in each institution. To establish high-reliability units, it will be necessary to train a critical mass of anesthesiologists in QI and SA at the institutions, and WUS remains well below the goal in this regard. Very few pediatric anesthesiologists possess expertise in QI or SA because it is not part of anesthesia residency or fellowship. It has been particularly difficult to get to goal for QI and SA expertise because as institutions joined WUS, the number of untrained member anesthesiologists increased, and in turn, the percentage of trained member anesthesiologists decreased. Looking ahead, QI and SA training will move front and center to the WUS Key Driver Diagram because both are requisite to imbed high reliability in the anesthesia departments. Safety requires processes with high reliability or, conversely, with a low defect rate. Reliability is expressed in log units of the defect rate: 1 defect in 10 (10–1) is level 1, 1 defect in 100 (10–2) is level 2, etc. Safe systems display reliability at level 5 or higher.19–21 Education in and of itself of a process yields level 1 reliability; thus, educating anesthesia providers through newsletters, conferences, and lectures when the incidence of events is well
