International Journal for Quality in Health Care 2014; Volume 26, Number 5: pp. 553–560 Advance Access Publication: 30 July 2014
10.1093/intqhc/mzu072
Does adding an appended oncology module to the Global Trigger Tool increase its value? THEA OTTO MATTSSON1,2, JANNE LEHMANN KNUDSEN3, KIM BRIXEN2,4 AND JØRN HERRSTEDT1,2 1
Department of Oncology, Odense University Hospital, Odense, Denmark, 2Department of Clinical Research, University of Southern Denmark, Odense, Denmark, 3Danish Cancer Society, Copenhagen, Denmark, and 4Department of Endocrinology, Odense University Hospital, Odense, Denmark Address reprint requests to: Thea Otto Mattsson, Department of Oncology, Odense University Hospital, Sdr. Boulevard 29, 5000 Odense C, Denmark. Tel:+45 66113333; Fax: +45 66127203; Email: [email protected] Accepted for publication 11 June 2014
Abstract Objective. To determine any additional value in the evaluation of safety levels by adding an appended oncology module to the Institute for Healthcare Improvement’s Global Trigger Tool (GTT).
Setting. The Department of Clinical Oncology at a Danish University Hospital (1000 beds). Participants. All inpatients admitted to the hospital in 2010, n = 3692, biweekly sample of 10 admission charts resulting in a double review of 240 charts. Main Outcome Measures. Total number of identified adverse events (AEs), distribution of identified AEs in the harm categories of the National Coordinating Council for Medication Error Reporting and Prevention (NCC MERP), AEs per 100 admissions and AEs per 1000 admission days. Results. No significant (95% confidence interval) difference was found between review teams using the general GTT versus the general GTT plus the appended oncology module on the total number of identified AEs, AEs per 100 admissions, AEs per 1000 admission days or in the overall distribution of identified AEs in the five NCC MERP harm categories. Conclusions. The study showed that adding the appended oncology module to the GTT did not increase its value regarding the evaluation of safety levels. This finding could be due to the measurement error of the GTT. Further studies evaluating the measurement properties and the specific additional modules to the general GTT are needed. Keywords: adverse events, quality measurement, patient safety, oncology
Introduction Clinical oncology is an area with complex and high-risk radiation and medical treatments. Keeping patients safe in the course of treatment is crucial to the outcome of the treatment. As new treatments and protocols are rapidly introduced, new safety hazards evolve and development of new preventive measures is called for. To ensure progress of these complex and combined efforts, there is a need for continuous measurement of patient safety. Using the trigger-based manual medical record review method, the Institute for Healthcare Improvement (IHI) developed the Global Trigger Tool (GTT) [1], with the aim of providing a practical tool to detect and monitor adverse events
(AEs) across the entire spectrum of inpatient care. The IHI describes the GTT as a ‘valid, reliable methodology for harm detection in individual hospitals, which can be used to track rates of harm over time and thereby as a measurement tool when evaluating patient safety within an organization’ [1–3]. The sensitivity and specificity of the GTT is very high compared with other methods for the detection of AEs using the manual chart review approach [4–6]. However, no method provides complete detection of AEs and different methods identify different AEs [5–8]. Therefore, the true sensitivity of all methods, including the GTT, remains unknown. The GTT is currently used in several countries, including Denmark, to measure progress in patient safety campaigns [9–14]. As part of the ‘1000 lives’ campaign in Wales, Velindre Cancer Centre
International Journal for Quality in Health Care vol. 26 no. 5 © The Author 2014. Published by Oxford University Press in association with the International Society for Quality in Health Care; all rights reserved
553
Downloaded from by guest on November 14, 2015
Design. Comparison of two independent retrospective chart reviews: one review team using the general GTT method and one using the general GTT method plus the appended oncology module on the same inpatient charts.
Mattsson et al.
developed an oncology-specific addition to the GTT, with the aim of achieving a more specific evaluation of their patient safety enhancement initiatives [10]. Despite the widespread use of the GTT and the increasing number of additional modules, published data on the utility and measurement properties of the GTT remain scarce. To our knowledge, no studies have evaluated the additional value of adding cancer-specific trigger tool modules to the general GTT module. The overall aim of this study was, therefore, to evaluate the utility of the GTT and the additional value of the oncology module as well as the measurement properties of the GTT in a population of Danish oncology inpatients. The data on the general measurement properties of the GTT have been reported in a prior publication [15]. This paper focuses on evaluating the additional value of the appended oncology module in a specialized cancer unit.
of the IHI GTT consisting of 55 triggers and the Velindre appended oncology module consisting of 17 triggers [10] were used (Fig. 1). According to the methodology, a brief review of the medical chart, lasting no >20 min, for the presence of triggers was conducted by two primary reviewers. On identification of an AE, the level of harm was assessed using the National Coordinating Council for Medication Error Reporting and Prevention (NCC MERP) categories E through I, ranging from ‘temporary harm to the patient requiring intervention’ to ‘harm contributed to death’ [16]. A secondary review was conducted by a physician on charts containing one or more events, confirming or dismissing the occurrence and category of the event. Data were collected on the number of AEs and severity of the events, as well as length of the stay in days, as recommended [1]. Selection and training of reviewers
Methods Setting
Study population All oncology patients admitted to the department from 1 January to 31 December 2010, aged >18 years and with a hospital stay >24 h were eligible for inclusion in the study. From this population, using discharge records, a random sample of 10 charts ( plus five extra charts) every 2 weeks were selected for review. Incomplete or unavailable charts were excluded; in such cases, extra charts were used in selected order. The evaluation process The IHI GTT defines an AE as any unintended physical injury resulting from or contributed to by medical care [1]. The GTT measure AEs based on a retrospective medical chart review for the presence of ‘triggers’. Triggers are sentinel conditions believed to be likely associated with the occurrence of an AE. Identification of a trigger leads to further study of the chart to determine whether an AE occurred. By applying the trigger approach to a biweekly sample of medical charts, the GTT generates harm rates over time, thus providing a continuous measure of the safety level within an organization. A translated version
554
Study set-up Following training, primary reviewers (nurses) were divided into two groups of four, one group using the GTT and the other using the GTT plus the oncology module (GTT + O). All charts were reviewed in sets of 10 from each period, but in random order by both groups. The two groups were further divided into teams of two. Each team of two nurses would review half of the charts, but both nurses within the team would review the same charts. In that way, a brief review of each sampled chart was conducted by two groups of nurses using two different methods (Fig. 2). Each review was conducted independently by one of the two nurses in a team. After each individual chart review, the two nurses would present their results to each other and reach agreement on length of stay, number of AEs and level of harm. In addition, we asked the nurses to provide a brief narrative description of the identified and categorized AEs as recommended in the Danish translation of the GTT [17]. These results were then presented to the secondary reviewer (the physician), who would study the chart and confirm or dismiss the presence of any AE including category. The general triggers were assessed prior to assessing the oncologyspecific triggers by the GTT + O team. All steps of the review process were documented before any discussion of the results. The nurses were also instructed not to discuss any aspects of the study with other teams during the study period.
Downloaded from by guest on November 14, 2015
This study was performed at the Department of Clinical Oncology, Odense University Hospital, Odense, Denmark. Odense University Hospital is the largest university hospital in the Nordic countries with ∼1.100 beds. The department of oncology is one of the five highly specialized oncology departments in Denmark. It provides both ambulatory and inpatient care free of charge (tax-funded) to all adult patients with cancer, within the Region of Southern Denmark. The department offers both standard and experimental medical and radiation treatment to patients diagnosed with all types of solid tumors except sarcoma. The department had 87 363 outpatient visits in 2010 and ∼2800 new patients were referred. The inpatient care units had a total of 11 522 admission days in connection with 3709 individual admissions.
Eight primary reviewers were selected among research assistant nurses employed at the Department of Oncology, Odense University Hospital. All the nurses had several years of experience (median 11 years) with reviewing patient records within the institution as part of their job of working with cancer treatment protocols. One physician within clinical oncology was selected as secondary reviewer. All reviewers attended training in accordance with the GTT methodology. Training comprised a full 2-day course and was provided by experienced staff members from the Centre for Quality, Region of Southern Denmark. Training included orientation and review of standardized training records and site-specific medical records using both general and oncology-specific triggers.
Adding an appended oncology module to the GTT • Quality improvement methods
Downloaded from by guest on November 14, 2015
Figure 1 List of triggers from the general Global Trigger Tool and the appended oncology module. Outcome measures The primary outcome measures were the number of AEs per 100 admissions, number of AEs per 1000 admission days and distribution of assigned harm in the five NCC MERP harm categories. Secondary measures were the nurse reviewer’s narrative descriptions following the identified AEs (Table 1). Pre- and post-study data analysis Pre-study power calculation was done using a two-sided test for difference of proportions in paired observations. We found that with an expected rate of ∼20 AEs per 100 admissions, the study had a power of 0.80 to detect approximately an absolute difference of 8% between the GTT and the GTT + O team, provided 240 charts were included in each group. We calculated the number of charts with at least one identified AE, as well as rates of AEs per 100 admissions and per 1000 admission days in accordance with the IHI
guidelines [1, 18] to permit comparison between the two teams and with previously published reports. Data are shown as medians [95% confidence interval (CI)]. A test for differences between teams in safety levels as well as distribution in the NCC MERP harm categories was calculated using the McNemar’s χ 2 test for comparison of proportions in paired data. Descriptive statistics were used to describe and compare the narrative descriptions of the identified AEs between the teams. P-values of
10.1093/intqhc/mzu072
Does adding an appended oncology module to the Global Trigger Tool increase its value? THEA OTTO MATTSSON1,2, JANNE LEHMANN KNUDSEN3, KIM BRIXEN2,4 AND JØRN HERRSTEDT1,2 1
Department of Oncology, Odense University Hospital, Odense, Denmark, 2Department of Clinical Research, University of Southern Denmark, Odense, Denmark, 3Danish Cancer Society, Copenhagen, Denmark, and 4Department of Endocrinology, Odense University Hospital, Odense, Denmark Address reprint requests to: Thea Otto Mattsson, Department of Oncology, Odense University Hospital, Sdr. Boulevard 29, 5000 Odense C, Denmark. Tel:+45 66113333; Fax: +45 66127203; Email: [email protected] Accepted for publication 11 June 2014
Abstract Objective. To determine any additional value in the evaluation of safety levels by adding an appended oncology module to the Institute for Healthcare Improvement’s Global Trigger Tool (GTT).
Setting. The Department of Clinical Oncology at a Danish University Hospital (1000 beds). Participants. All inpatients admitted to the hospital in 2010, n = 3692, biweekly sample of 10 admission charts resulting in a double review of 240 charts. Main Outcome Measures. Total number of identified adverse events (AEs), distribution of identified AEs in the harm categories of the National Coordinating Council for Medication Error Reporting and Prevention (NCC MERP), AEs per 100 admissions and AEs per 1000 admission days. Results. No significant (95% confidence interval) difference was found between review teams using the general GTT versus the general GTT plus the appended oncology module on the total number of identified AEs, AEs per 100 admissions, AEs per 1000 admission days or in the overall distribution of identified AEs in the five NCC MERP harm categories. Conclusions. The study showed that adding the appended oncology module to the GTT did not increase its value regarding the evaluation of safety levels. This finding could be due to the measurement error of the GTT. Further studies evaluating the measurement properties and the specific additional modules to the general GTT are needed. Keywords: adverse events, quality measurement, patient safety, oncology
Introduction Clinical oncology is an area with complex and high-risk radiation and medical treatments. Keeping patients safe in the course of treatment is crucial to the outcome of the treatment. As new treatments and protocols are rapidly introduced, new safety hazards evolve and development of new preventive measures is called for. To ensure progress of these complex and combined efforts, there is a need for continuous measurement of patient safety. Using the trigger-based manual medical record review method, the Institute for Healthcare Improvement (IHI) developed the Global Trigger Tool (GTT) [1], with the aim of providing a practical tool to detect and monitor adverse events
(AEs) across the entire spectrum of inpatient care. The IHI describes the GTT as a ‘valid, reliable methodology for harm detection in individual hospitals, which can be used to track rates of harm over time and thereby as a measurement tool when evaluating patient safety within an organization’ [1–3]. The sensitivity and specificity of the GTT is very high compared with other methods for the detection of AEs using the manual chart review approach [4–6]. However, no method provides complete detection of AEs and different methods identify different AEs [5–8]. Therefore, the true sensitivity of all methods, including the GTT, remains unknown. The GTT is currently used in several countries, including Denmark, to measure progress in patient safety campaigns [9–14]. As part of the ‘1000 lives’ campaign in Wales, Velindre Cancer Centre
International Journal for Quality in Health Care vol. 26 no. 5 © The Author 2014. Published by Oxford University Press in association with the International Society for Quality in Health Care; all rights reserved
553
Downloaded from by guest on November 14, 2015
Design. Comparison of two independent retrospective chart reviews: one review team using the general GTT method and one using the general GTT method plus the appended oncology module on the same inpatient charts.
Mattsson et al.
developed an oncology-specific addition to the GTT, with the aim of achieving a more specific evaluation of their patient safety enhancement initiatives [10]. Despite the widespread use of the GTT and the increasing number of additional modules, published data on the utility and measurement properties of the GTT remain scarce. To our knowledge, no studies have evaluated the additional value of adding cancer-specific trigger tool modules to the general GTT module. The overall aim of this study was, therefore, to evaluate the utility of the GTT and the additional value of the oncology module as well as the measurement properties of the GTT in a population of Danish oncology inpatients. The data on the general measurement properties of the GTT have been reported in a prior publication [15]. This paper focuses on evaluating the additional value of the appended oncology module in a specialized cancer unit.
of the IHI GTT consisting of 55 triggers and the Velindre appended oncology module consisting of 17 triggers [10] were used (Fig. 1). According to the methodology, a brief review of the medical chart, lasting no >20 min, for the presence of triggers was conducted by two primary reviewers. On identification of an AE, the level of harm was assessed using the National Coordinating Council for Medication Error Reporting and Prevention (NCC MERP) categories E through I, ranging from ‘temporary harm to the patient requiring intervention’ to ‘harm contributed to death’ [16]. A secondary review was conducted by a physician on charts containing one or more events, confirming or dismissing the occurrence and category of the event. Data were collected on the number of AEs and severity of the events, as well as length of the stay in days, as recommended [1]. Selection and training of reviewers
Methods Setting
Study population All oncology patients admitted to the department from 1 January to 31 December 2010, aged >18 years and with a hospital stay >24 h were eligible for inclusion in the study. From this population, using discharge records, a random sample of 10 charts ( plus five extra charts) every 2 weeks were selected for review. Incomplete or unavailable charts were excluded; in such cases, extra charts were used in selected order. The evaluation process The IHI GTT defines an AE as any unintended physical injury resulting from or contributed to by medical care [1]. The GTT measure AEs based on a retrospective medical chart review for the presence of ‘triggers’. Triggers are sentinel conditions believed to be likely associated with the occurrence of an AE. Identification of a trigger leads to further study of the chart to determine whether an AE occurred. By applying the trigger approach to a biweekly sample of medical charts, the GTT generates harm rates over time, thus providing a continuous measure of the safety level within an organization. A translated version
554
Study set-up Following training, primary reviewers (nurses) were divided into two groups of four, one group using the GTT and the other using the GTT plus the oncology module (GTT + O). All charts were reviewed in sets of 10 from each period, but in random order by both groups. The two groups were further divided into teams of two. Each team of two nurses would review half of the charts, but both nurses within the team would review the same charts. In that way, a brief review of each sampled chart was conducted by two groups of nurses using two different methods (Fig. 2). Each review was conducted independently by one of the two nurses in a team. After each individual chart review, the two nurses would present their results to each other and reach agreement on length of stay, number of AEs and level of harm. In addition, we asked the nurses to provide a brief narrative description of the identified and categorized AEs as recommended in the Danish translation of the GTT [17]. These results were then presented to the secondary reviewer (the physician), who would study the chart and confirm or dismiss the presence of any AE including category. The general triggers were assessed prior to assessing the oncologyspecific triggers by the GTT + O team. All steps of the review process were documented before any discussion of the results. The nurses were also instructed not to discuss any aspects of the study with other teams during the study period.
Downloaded from by guest on November 14, 2015
This study was performed at the Department of Clinical Oncology, Odense University Hospital, Odense, Denmark. Odense University Hospital is the largest university hospital in the Nordic countries with ∼1.100 beds. The department of oncology is one of the five highly specialized oncology departments in Denmark. It provides both ambulatory and inpatient care free of charge (tax-funded) to all adult patients with cancer, within the Region of Southern Denmark. The department offers both standard and experimental medical and radiation treatment to patients diagnosed with all types of solid tumors except sarcoma. The department had 87 363 outpatient visits in 2010 and ∼2800 new patients were referred. The inpatient care units had a total of 11 522 admission days in connection with 3709 individual admissions.
Eight primary reviewers were selected among research assistant nurses employed at the Department of Oncology, Odense University Hospital. All the nurses had several years of experience (median 11 years) with reviewing patient records within the institution as part of their job of working with cancer treatment protocols. One physician within clinical oncology was selected as secondary reviewer. All reviewers attended training in accordance with the GTT methodology. Training comprised a full 2-day course and was provided by experienced staff members from the Centre for Quality, Region of Southern Denmark. Training included orientation and review of standardized training records and site-specific medical records using both general and oncology-specific triggers.
Adding an appended oncology module to the GTT • Quality improvement methods
Downloaded from by guest on November 14, 2015
Figure 1 List of triggers from the general Global Trigger Tool and the appended oncology module. Outcome measures The primary outcome measures were the number of AEs per 100 admissions, number of AEs per 1000 admission days and distribution of assigned harm in the five NCC MERP harm categories. Secondary measures were the nurse reviewer’s narrative descriptions following the identified AEs (Table 1). Pre- and post-study data analysis Pre-study power calculation was done using a two-sided test for difference of proportions in paired observations. We found that with an expected rate of ∼20 AEs per 100 admissions, the study had a power of 0.80 to detect approximately an absolute difference of 8% between the GTT and the GTT + O team, provided 240 charts were included in each group. We calculated the number of charts with at least one identified AE, as well as rates of AEs per 100 admissions and per 1000 admission days in accordance with the IHI
guidelines [1, 18] to permit comparison between the two teams and with previously published reports. Data are shown as medians [95% confidence interval (CI)]. A test for differences between teams in safety levels as well as distribution in the NCC MERP harm categories was calculated using the McNemar’s χ 2 test for comparison of proportions in paired data. Descriptive statistics were used to describe and compare the narrative descriptions of the identified AEs between the teams. P-values of
