Prevalence and Test Characteristics of National Health Safety Network Ventilator-Associated Events Craig M. Lilly, MD1,2,3,4; Karen E. Landry, BS5; Rahul N. Sood, MD1; Cheryl H. Dunnington, RN, MS5,6; Richard T. Ellison III, MD1,5,7; Peter H. Bagley, MD1,5; Stephen P. Baker, MScPH1,2,4,8,9,10; Shawn Cody, RN, MSN/MBA5,6; Richard S. Irwin, MD1,8; for the UMass Memorial Critical Care Operations Group

Objectives: The primary aim of the study was to measure the test characteristics of the National Health Safety Network ventilatorassociated event/ventilator-associated condition constructs for detecting ventilator-associated pneumonia. Its secondary aims were to report the clinical features of patients with National Health

Department of Medicine, University of Massachusetts Medical School, Worcester, MA. 2 Department of Anesthesiology and Surgery, University of Massachusetts Medical School, Worcester, MA. 3 Clinical and Population Health Research Program, University of Massachusetts Medical School, Worcester, MA. 4 Graduate School of Biomedical Sciences, University of Massachusetts Medical School, Worcester, MA. 5 Department of Critical Care Operations, UMass Memorial Health Care, Worcester, MA. 6 Department of Nursing, UMass Memorial Medical Center, Worcester, MA. 7 Department of Microbiology and Physiological Systems, University of Massachusetts Medical School, Worcester, MA. 8 Graduate School of Nursing Sciences, University of Massachusetts Medical School, Worcester, MA. 9 Department of Information Services, University of Massachusetts Medical School, Worcester, MA. 10 Department of Cell Biology, University of Massachusetts Medical School, Worcester, MA. Dr. Lilly, Ms. Landry, and Dr. Baker had full access to the data and take responsibility for its integrity and the accuracy of the analyses. Drs. Lilly, Ellison, and Bagley contributed to study concept and design. Dr. Lilly, Ms. Landry, Dr. Sood, Ms. Dunnington, Dr. Ellison, and Mr. Cody contributed to acquisition of data. Drs. Lilly, Baker, Irwin, and Bagley contributed to analysis and interpretation of the data. Supplemental digital content is available for this article. Direct URL citations appear in the printed text and are provided in the HTML and PDF versions of this article on the journal’s website (http://journals.lww.com/ccmjournal). The authors have disclosed that they do not have any potential conflicts of interest. Address requests for reprints to: Craig M. Lilly, MD, Department of Medicine, University of Massachusetts Medical School, UMass Memorial Medical Center, 281 Lincoln Street, Worcester, MA 01605. E-mail: craig.lilly@ umassmed.edu Copyright © 2014 by the Society of Critical Care Medicine and Lippincott Williams & Wilkins DOI: 10.1097/CCM.0000000000000396 1

Critical Care Medicine

Safety Network ventilator-associated event/ventilator-associated condition, measure costs of surveillance, and its susceptibility to manipulation. Design: Prospective cohort study. Setting: Two inpatient campuses of an academic medical center. Patients: Eight thousand four hundred eight mechanically ventilated adults discharged from an ICU. Interventions: None. Measurements and Main Results: The National Health Safety Network ventilator-associated event/ventilator-associated condition constructs detected less than a third of ventilator-associated pneumonia cases with a sensitivity of 0.325 and a positive predictive value of 0.07. Most National Health Safety Network ventilator-associated event/ventilator-associated condition cases (93%) did not have ventilator-associated pneumonia or other hospital-acquired complications; 71% met the definition for acute respiratory distress syndrome. Similarly, most patients with National Health Safety Network probable ventilator-associated pneumonia did not have ventilator-associated pneumonia because radiographic criteria were not met. National Health Safety Network ventilator-associated event/ ventilator-associated condition rates were reduced 93% by an unsophisticated manipulation of ventilator management protocols. Conclusions: The National Health Safety Network ventilator-associated event/ventilator-associated condition constructs failed to detect many patients who had ventilator-associated pneumonia, detected many cases that did not have a hospital complication, and were susceptible to manipulation. National Health Safety Network ventilator-associated event/ventilator-associated condition surveillance did not perform as well as ventilator-associated pneumonia surveillance and had several undesirable characteristics. (Crit Care Med 2014; 42:2019–2028) Key Words: mechanical ventilation; patient safety; quality measures; ventilator-associated event; ventilator-associated pneumonia

V

entilator-associated pneumonia (VAP) is an iatrogenic infection reported to have a 10% attributable mortality (1–3) that is estimated to affect more than 250,000 www.ccmjournal.org

2019

Lilly et al

Americans each year (4). There are increasing numbers of reports of effective prevention strategies (4, 5) for this complication of mechanical ventilation (5). The Centers for Disease Control and Prevention (CDC), Division of Healthcare Quality Promotion VAP Surveillance Working Group, a group that was established in collaboration with the CDC Prevention Epicenters, and the Critical Care Societies Collaborative (6) have proposed the National Health Safety Network (NHSN) ventilator-associated event (VAE)/ventilator-associated condition (VAC) constructs to improve upon the VAP system of reporting nosocomial pulmonary infections among mechanically ventilated adults. This working group initially created (6) and then modified (7) case definitions for NHSN VAE/VAC. In addition, patients with NHSN VAE/VAC can also be further classified as having an infection-related ventilator condition (IVAC) when antimicrobials are prescribed, or with NHSN possible VAP, or NHSN probable VAP when there is evidence of a pulmonary infection. Although the new array of VAE constructs are complicated, they were developed with the goal of creating more efficient and less subjective surveillance programs for identifying complications of mechanical ventilation (7, 8) than established methods of detecting VAP (9). The established case definition for VAP was also created by a CDC-endorsed working group. There are concerns regarding its low levels of accuracy (10), interobserver variability (11), costs of measurement, and its failure to identify some cases treated in clinical practice (12). On the other hand, the creation of the VAP definition has fostered the implementation of programs that can successfully prevent pulmonary infections in mechanically ventilated patients (13, 14). The aims of those who have put forth the NHSN VAE/VAC constructs are to achieve levels of objectivity, reliability, ability to detect complications, and resistance to manipulation that are sufficient for public reporting, valid comparison of interfacility results, and to justify differential rates of reimbursement based on derived performance measures (6). Knowledge of how the new constructs perform compared to established methods of VAP surveillance is fundamental to understanding the benefits and unintended consequences of replacing the established method of detecting pulmonary infections caused by mechanical ventilation. The primary aims of this study were to measure the ability of the NHSN VAE/VAC constructs including its VAC, IVAC, NHSN possible VAP, and NHSN probable VAP variants to detect patients with VAP (15) and to understand the clinical characteristics of VAC patients who do not have VAP. Its secondary aims were to compare the time required for VAP and NHSN VAE/VAC surveillance using electronic tools and to model how changes in clinician behavior impact rates of NHSN VAE/VAC detection.

METHODS VAP cases were prospectively identified using an established system designed to perform VAP surveillance in the closest practically achievable accord with the NHSN guidelines. The analytical plan was developed before the development and validation of an electronic tool for identifying patients who met the criteria of 2020

www.ccmjournal.org

the NHSN VAE/VAC constructs including its VAC, IVAC, NHSN possible VAP, and NHSN probable VAP variants according to the January 13, 2013 criteria (7). These tools identified cases from the prospectively collected dataset. The study was conducted using electronic medical records of patients from three medical (45 beds), two surgical (23 beds), one cardiovascular (16 bed), and one neurosciences (16 bed) ICU(s) from two inpatient campuses of a university-associated academic medical center located in central Massachusetts using previously described abstraction methods and validation procedures (16). The study was approved in advance by the University of Massachusetts Human Subjects Committee that waived the requirement for informed consent. Cases were selected from all 21,145 patients who were 18 years old or older and had an admission to an adult ICU that was of at least 10 minutes in duration, during the period from January 3, 2009, to April 30, 2012. Among these patients, there were 2,857 individuals who could have met the NHSN VAE/VAC definitions and 3,313 who could have met the VAP definition, among 8,408 who were managed with mechanical ventilation (Fig. 1). Mechanical ventilation, including changes in oxygen and positive end-expiratory pressure (PEEP) levels, was managed using established protocols with high reported rates of adherence (16) that were modeled on studies of improved outcomes for patients with acute respiratory distress syndrome (ARDS) (17, 18). NHSN VAE/VAC Case Selection NHSN VAE/VAC case selection was in strict accord with the January 2013 CDC guidance document (7). NHSN VAE/VAC cases were defined as having a baseline period of stability of two or more calendar days during which the daily minimum Fio2 or PEEP value was stable or decreased. “Baseline” levels of PEEP and Fio2 were defined as the lowest values recorded in the medical record during this period of stability or improvement. Patients were classified as having NHSN VAE/VAC when, immediately after a period of two or more calendar days of stability, the lowest daily Fio2 value recorded in the medical record was at least 0.20 greater than its baseline value or the lowest daily recorded PEEP value was at least 3 cm of water greater than its baseline value for two or more calendar days. Patients who met the NHSN VAE/VAC definition were classified as having an IVAC when, on or after 3 days of mechanical ventilation and within two calendar days before or after the onset of worsening oxygenation, the patient met both of the following criteria: 1) temperature greater than 38°C or a WBC count of greater than or equal to 12,000/μL or less than or equal to 4,000/μL and 2) a new antimicrobial agent was started and continued for four or more calendar days. Patients with IVAC were classified as having an NHSN possible VAP when their secretions from lungs, bronchi, or trachea contained greater than or equal to 25 neutrophils and less than or equal to 10 squamous epithelial cells per low power field or 2) a (qualitative, semiquantitative, or quantitative) culture of sputum, endotracheal aspirate, bronchoalveolar lavage, lung tissue, or protected specimen brushing was positive for a pathogen. These cases were classified as having an NHSN probable VAP when they had purulent respiratory secretions (as defined above) and one of the following: 1) September 2014 • Volume 42 • Number 9

Clinical Investigations

Figure 1. Selection of patients who were at risk for National Health Safety Network ventilator-associated event/ventilator-associated condition and for ventilator-associated pneumonia.

an endotracheal aspirate with greater than or equal to 105 CFU/ mL* of a pathogen; a culture of bronchoalveolar lavage or lung tissue with greater than or equal to 104 CFU/mL* of a pathogen; or a protected specimen brush culture with greater than or equal to 103 CFU/mL* of a pathogen (*or equivalent semiquantitative results) (7) or 2) one of the following with or without purulent respiratory secretions: pleural fluid culture with a pathogen, lung histopathology demonstrating pathogenic organisms, a positive test for Legionella species, a positive diagnostic test for influenza virus, respiratory syncytial virus, adenovirus, or parainfluenza virus from respiratory secretions (Fig. 2, top). VAP Case Selection VAP cases were selected in strict accord with the CDC/NHSN definitions (9) and were determined by the following threestage process. Each morning the daily chest radiographs of every adult ICU patient who had been mechanically ventilated for more than 48 hours were screened for new or progressive and persistent infiltrates, consolidation, or cavitation. The daily reviews were performed independently by a board-certified intensive care specialist who was not the physician of record of any of the patients. Screen positive cases that also had a temperature greater than 38°C or less than 36° or a WBC count of greater than or equal to 12,000/μL or less than or equal to 4,000/μL cells were evaluated by an infection control practitioner and those found to have purulent respiratory secretions (as defined above), signs and symptoms consistent with a lung infection (9), worsening gas exchange, or cultures, laboratory tests, or pathological specimens positive for respiratory pathogens were classified, according to the CDC definitions, as potentially having VAP (9). Finally, the defining variables were evaluated, and the presence of new or progressive and persistent infiltrates, cavitation, or consolidation was confirmed Critical Care Medicine

by an infection control officer who holds an active American Board of Internal Medicine (ABIM) certification in Infectious Diseases and two physicians with active ABIM certification in Pulmonary Medicine and classified as having VAP when all or a majority agreed (Fig. 2, bottom). The evaluation process focused on whether cases met the NHSN criteria for VAP rather on whether a clinical diagnosis of VAP was appropriate or not. Factors known to compromise gas exchange were identified for those with NHSN VAE/VAC. Cases were identified as being associated with ARDS when at the time of the VAE, they had acute hypoxemia in the setting of an ARDS defining event, bilateral opacities were present on a chest imaging study, and the Pao2/Fio2 ratio was less than 300 (19), and as having volume overload when they were in greater than or equal to 5-L positive fluid balance from the time of ICU admission to the onset of NHSN VAE/VAC. The underlying cause for ARDS was classified as related to the presenting illness or not. Compromised renal function, detected as acute kidney injury (AKI), was selected as a surrogate for failure to respond to therapies intended to restore or maintain perfusion. Cases were defined as being associated with AKI when the creatinine value at the time of the first worsening had increased greater than or equal to 0.3 mg/dL or a 1.5-fold or greater value than at the time of ICU admission (20). Cases were classified as other when they did not meet at least one of these three definitions. The times required for screening chest images for the presence or absence of new or progressive and persistent infiltrates; the time required to review respiratory secretion characteristics, gas exchange variables, the results of microbiological and pathological studies, as well as the time required to assess volume status, identify AKI, and for other clinical review were measured by chronometry of all or representative samples of determinations. www.ccmjournal.org

2021

Lilly et al

Halton test when numbers of observations in each cell were inadequate. Test and population characteristics were calculated arithmetically. VAP was used as the criterion standard because these cases have attributable mortality and can be prevented rather than because it is a gold standard test. Unadjusted differences between four mutually exclusive groups (VAC only, VAP only, both, and neither) were evaluated using the KruskalWallis analysis of variance by ranks (21). In the presence of a significant effect, pairwise comparisons of interest were evaluated using the Dunn multiple comparisons test upon ranks (22). Relative mortality risk was modeled by logistic regression. Likelihood ratio tests (23) were used to select Acute Physiology and Chronic Health Evaluation (APACHE) IV score and type of ICU as the only predictors with p values less than 0.05. Length of stay (LOS) and ventilator days were evaluated using Cox proportionate hazards models using LOS and ventilator days as the time factor stratifying on APACHE IV severity score categorized into low, medium, and high severity ranges (< 35, 35–50, > 50). The outcome was hospital discharge with hospital mortality as a censoring facFigure 2. Case detection and classification algorithms. NHSN = National Health Safety Network, tor for competing risk analyses. VAE = ventilator-associated event, VAC = ventilator-associated condition, VAP = ventilator-associated The p values reported were from pneumonia, PEEP = positive end-expiratory pressure, IVAC = infection-related ventilator condition, RSV = Wald tests of model variables. respiratory syncytial virus. In the presence of significant We constructed a model to test a hypothetical systematic effects, separate Kaplan-Meier product limit survival analyses change in clinician behavior intended to reduce the detection were performed with VAP and VAE/VAC as comparison factors of NHSN VAE/VAC. We compared the rates of NHSN VAE/ to obtain estimates of LOS and duration of ventilation. StratiVAC and IVAC in our population to those in which every other fication was again done on APACHE IV score severity categoday the PEEP was increased by 1 cm of water and the Fio2 was ries with time estimates made within strata and significance of increased by 0.1 (or to 1.0 when the value was > 0.9) above the differences within strata performed using the Tarone-Ware stahighest value for that day, maintained for 24 hours, and then tistic. Significance was set at the 0.05 level. Statistical analyses reset to the actual value at the start of the next day. Alternate- were performed using SigmaPlot version 11 and SPSS version 20 day Fio2 and PEEP values were not changed. (IBM, Armonk, NY) (24). Statistical Analyses Categorical variables were compared by chi-square analysis with appropriate degrees of freedom or by the Fisher Freeman 2022

www.ccmjournal.org

RESULTS The characteristics of the population of patients that met the NHSN VAE/VAC definitions were significantly different than September 2014 • Volume 42 • Number 9

Table 1.

Clinical Investigations

Patient Characteristics

Group

Number Age, mean (95% CI) Female (%) Acute Physiology and Chronic Health Evaluation IV score, mean (95% CI)

Mechanically Ventilated

NHSN NHSN Ventilator-Associated Infection-Related Event/Ventilator-Associated Ventilator Condition Condition

8,408b 61 (61.8–61.1) 3,357 (40) 75 (76–74.7)

387 58 (59.5–56.2) 130 (34)

344

VentilatorAssociated Pneumonia

pa

83

< 0.001

57 (58.7–55.8) 53 (56.6–49.7) < 0.001 114 (33)

29 (35%)

< 0.001

81 (83.8–77.9)

80 (83.4–77.2) 76 (80.9–70.1) < 0.001

1,313 (15.6)

82 (21.2)

71 (20.6)

7 (8.4)

 Trauma

669 (8.0)

44 (11.4)

39 (11.3)

23 (27.7)

 Coronary artery bypass grafting

664 (7.9)

2 (0.5)

2 (0.6)

1 (1.2)

 Sepsis

560 (6.7)

35 (9.0)

32 (9.3)

1 (1.2)

 Cardiac arrest

346 (4.1)

21 (5.4)

17 (4.9)

3 (3.6)

 Intracranial hemorrhage/hematoma

198 (2.4)

14 (3.6)

11 (3.2)

5 (6.0)

 Cerebrovascular accident

187 (2.2)

8 (2.1)

5 (1.5)

2 (2.4)

 Cancer

182 (2.2)

11 (2.8)

11 (3.2)

3 (3.6)

 Acute myocardial infarction

166 (2.0)

1 (0.3)

0 (0)

1 (1.2)

 Seizures

154 (1.8)

9 (2.3)

8 (2.3)

0 (0.0)

 Congestive heart failure

153 (1.8)

10 (2.6)

10 (2.9)

1 (1.2)

 Coma

131 (1.6)

3 (0.8)

3 (0.9)

2 (2.4)

 Subarachnoid hemorrhage

90 (1.1)

6 (1.6)

6 (1.7)

9 (10.8)

 Pancreatitis

87 (1.0)

8 (2.1)

6 (1.7)

3 (3.6)

 Acute hepatic failure

66 (0.8)

6 (1.6)

6 (1.7)

0 (0.0)

 Asthma

25 (0.3)

2 (0.5)

1 (0.3)

0 (0.0)

 Medical

3,364 (40)

201 (52)

183 (53)

21 (25)

0.001

 Surgical

3,058 (36)

151 (39)

132 (39)

53 (64)

< 0.001

 Cardiac

1,986 (24)

35 (9)

29 (8)

9 (11)

0.009

Primary admission diagnosis (%)c  Respiratory failure

Type of ICU

NHSN = National Health Safety Network. a Significance comparing four mutually exclusive groups (ventilator-associated condition only, ventilator-associated pneumonia only, both, and neither); NS = not significant. b Two thousand eight hundred fifty-seven patients had mechanical ventilation of sufficient duration to meet the case definition for NHSN ventilator-associated event/ventilator-associated condition. c Includes the 10 most frequent diagnoses for each group. Data from all patients are included in this tabulation; statistical analyses excluded patients in more than one category as detailed in the Statistical Analyses section.

those that met the definition for VAP (Table 1). The incidence of NHSN VAE/VAC was 13.8 cases/1,000 days of mechanical ventilation, the prevalence of IVAC was 8.8 cases/1,000 ventilator days, and both were significantly higher than the prevalence of VAP, which was 2.96 cases/1,000 ventilator days, a VAP rate similar to the reported national rate (25). Patients with NHSN VAE/VAC, IVAC, and VAP were significantly younger than all ventilated patients and those with NHSN VAE/VAC and IVAC were significantly more likely to be male than all ventilated patients. VAP patients were similar with regard to acuity to all ventilated patients while those with NHSN VAE/ Critical Care Medicine

VAC and IVAC had significantly higher APACHE IV acuity scores. The NHSN VAE/VAC and IVAC populations were predominantly from medical ICUs while those with VAP were predominantly from surgical ICUs. The frequency of primary admission diagnoses also distinguished the NHSN VAE/VAC and IVAC groups from the VAP group. The frequencies for respiratory failure, trauma, and subarachnoid hemorrhage for the VAP group were divergent from those of the NHSN VAE/ VAC and IVAC groups (Table 1). The NHSN VAE/VAC construct had a sensitivity of 0.325 for detecting VAP cases using methods in which chest www.ccmjournal.org

2023

Lilly et al

radiographs were obtained for nearly every qualifying patient each day of mechanical ventilation (0.98 ± 0.08 [SD]; range [0.65–1] radiographs/day of mechanical ventilation). The sensitivity of the IVAC construct, NHSN possible VAP, and NHSN probable VAP constructs for detecting VAP were lower at 0.301, 0.217, and 0.048, respectively (Table 2). Among 79 VAP cases that did not meet the NHSN definition for probable VAP, 56 did not have a period of stability followed by worsening oxygenation in the time frame required in the NHSN VAC definition, 21 had a pulmonary pathogen isolated outside of the 2 day before or after the VAE event window, and two did not have antibacterials prescribed within this time frame (Fig. 2). Conversely, most of the 58 NHSN probable VAP patients who were not classified as having VAP did not meet radiographic criteria; 20 patients had abnormalities that preceded the onset of mechanical ventilation; 13 had infiltrates that were not present on more than one daily radiograph; 11 did not have infiltrates; 12 immunocompetent patients did not meet temperature or leukocyte count criteria; and two cases were excluded by a 2 to 1 reviewer majority. There were many more cases of NHSN VAE/VAC than VAP; for every VAP case, there were 4.7 cases of NHSN VAE/VAC. Sensitivity analyses using alternative methods for defining VAP cases including using all 164 cases identified by the infection control practitioners or only the 84 cases that would have been identified by the infection control officer or

the 79 or 83 cases that would have been identified by the others and those using only VAP cases from which lower airway pathogens were isolated also calculated similar low sensitivity and PPV and high specificity and NPV values. The NHSN VAE/VAC and IVAC groups had significantly higher crude mortality rates than all mechanically ventilated patients and those with VAP. After adjustment for acuity and type of ICU, the differences among the VAP and NHSN VAE/VAC and IVAC groups were no longer statistically significant. Unadjusted analyses of hospital LOS and the duration of mechanical ventilation revealed that the VAP group had longer durations than the NHSN VAE/VAC and IVAC groups (Table 3). Overall, there was no significant interaction between VAP and VAC for either LOS or duration of mechanical ventilation; hence, the VAP and VAC effects were independent, that is, there is no statistical evidence to exclude this group when evaluating VAP or VAC. Competing risk analyses that adjusted for APACHE IV score, type of ICU, and included hospital mortality as a censoring factor (to account for the fact that mechanical ventilation or hospitalization ended when a patient died) demonstrated that hospital LOS was significantly longer for VAP patients (65.3 d [95% CI, 42.3–88.3 d]; p = 0.006) than for VAC group patients (41.4 d [95% CI, 33.1–49.6 d]). The duration of mechanical ventilation was also significantly longer for patients with VAP (44.3 d [95% CI, 17.9–70.6 d]; p = 0.02) than those with

Table 2. Test Characteristics of the National Health Safety Network Ventilator-Associated Event/Ventilator-Associated Condition Constructs for the Diagnosis of Ventilator-Associated Pneumonia Group

VAP Cases

Cases Without VAP

Total Cases

Test Characteristics

NHSN VAE/VAC cases

27

360

387

PPV = 0.070

Cases without NHSN VAE/VAC

56

7,965

8,021

NPV = 0.993

Total cases

83

8,325

8,408

Sensitivity = 0.325; specificity = 0.957 IVAC cases

25

319

344

PPV = 0.073

Cases without IVAC

58

8,006

8,064

NPV = 0.993

Total cases

83

8,325

8,408

Sensitivity = 0.301; specificity = 0.962 NHSN possible VAP

18

225

243

PPV = 0.074

Without possible VAP

65

8,100

8,165

NPV = 0.992

Total cases

83

8,325

8,408

Sensitivity = 0.217; specificity = 0.973 NHSN probable VAP

4

58

62

PPV = 0.065

Without probable VAP

79

8,267

8,346

NPV = 0.991

Total cases

83

8,325

8,408

Sensitivity = 0.048; specificity = 0.993 VAP = ventilator-associated pneumonia, NHSN VAE/VAC = National Health Safety Network ventilator-associated event/ventilator-associated condition, PPV = positive predictive value, NPV = negative predictive value, IVAC = infection-related ventilator condition.

2024

www.ccmjournal.org

September 2014 • Volume 42 • Number 9

Clinical Investigations

Table 3.

Outcomes by Group

Characteristic

Mechanically Ventilated

National Health Safety Network Ventilator-Associated Event/VentilatorAssociated Condition

Infection-Related Ventilator Condition

VentilatorAssociated Pneumonia

Actual hospital mortality (%)

1,921 (23.8)

158 (42.0)a

143 (42.7)a

23 (28.4)

Predicted hospital mortality (sd)

2,231 (0.26)

125 (0.25)

111 (0.25)

23 (0.21)

1.26

1.29

0.98

O/E hospital mortality ratio In-hospital mortality, odds ratio (95% CI)b

0.86 Reference

1.84 (0.95–3.6)

1.32 (0.66–2.6)

1.03 (0.61–1.7)

Actual ventilator days (95% CI)

4.8 (4.98–4.62)

14.8 (16.6–13.2)a

14.5 (15.8–13.1)a

17.6 (20.5–14.8)a

Predicted ventilator days (95% CI)c

4.0 (4.01–3.98)

4.7 (4.82–4.53)

4.7 (4.84–4.54)

4.5 (4.81–4.21)

O/E ventilator days (95% CI)

1.1 (1.17–1.09)

3.3 (3.68–3.01)

3.3 (3.63–2.93)

4.2 (4.93–3.4)

Actual hospital LOS (95% CI)

15.1 (15.2–15.1)

25.3 (26.9–23.6)a

25.1 (26.9–23.3)a

31.1 (35.2–27.1)e

Predicted hospital LOS (95% CI)d

12.3 (12.4–12.2)

13.7 (14.3–13.2)

13.7 (14.2–13.1)

13.7 (14.5–12.8)

1.3 (1.36–1.28)

2.0 (2.2–1.89)

2.0 (2.22–1.86)

2.45 (2.83–2.08)

O/E hospital LOS (95% CI)

O/E = observed/expected, LOS = length of stay. a p < 0.001 compared to mechanically ventilated patients without ventilator-associated pneumonia (VAP) or ventilator-associated condition (VAC). b Adjusted for Acute Physiology and Chronic Health Evaluation IV score and type of ICU. c Among 5,804 mechanically ventilated, 281 VAC, 249 infection-related ventilator condition (IVAC), and 57 VAP patients had valid predictions. d Among 8,408 mechanically ventilated, 374 VAC, 333 IVAC, and 81 VAP patients had valid predictions. Data from all patients are included in this tabulation; statistical analyses excluded patients in more than one category as detailed in the Statistical Analyses section. e p < 0.05 compared to those without VAP in the National Health Safety Network ventilator-associated event/ventilator-associated condition and IVAC groups.

VAC (21.7 d [95% CI, 17.0–26.5 d]). Stratified analyses demonstrated significant differences only for the highest acuity group of patients. An etiology for the gas exchange abnormalities was identified for 86% of the cases that met the NHSN VAE/VAC case definition. Only a small minority (7%) were associated with VAP. ARDS was nearly always related to the presenting illness (96%) and was the most frequently identified cause of NHSN VAE/VAC being associated with 282 or 73% of NHSN VAE/VAC cases. Microbiological evidence of a pulmonary infection was present for 39 of 282 or 14% of these patients. AKI was associated with 27% cases, 14% had other causes such as pulmonary hemorrhage or embolism, or inflammatory lung diseases, and volume overload was identified in 7% (Table 4). VAP surveillance required 1,152 person hours for 5,448 episodes or 12.6 minutes for each episode of mechanical ventilation and NHSN VAE/VAC surveillance required 621 hours for 2,857 episodes or 12.4 minutes for each episode as detailed in the online supplement (Supplemental Digital Content 1, http://links.lww.com/CCM/A964). When we modeled the effects of an algorithmic and easily achievable manipulation of respiratory care protocols in which PEEP was increased by 1 cm and the Fio2 was increased by 0.1 above the previous day’s value on alternate days, the number of NHSN VAE/VAC cases that were detected was reduced because the definition of a stable baseline period was not met. In this example, NHSN VAE/VAC and IVAC case detection both decreased by 93% (28 of 387 NHSN VAE/VAC cases were detected; and 25 of 244 IVAC cases were detected). Critical Care Medicine

Table 4. Risk Factors for Respiratory Failure

of Those Meeting the National Health Safety Network Ventilator-Associated Event/ Ventilator-Associated Condition Definition n (%)

Condition

National Health Safety Network ventilator-associated event/ventilatorassociated condition Ventilator-associated pneumonia

387 (100)

27 (7.0)

Risk factors for respiratory failure  ARDS

181 (46.8)

 Acute kidney injury and ARDS

77 (19.9)

 Acute kidney injury

20 (5.2)

 ARDS and volume overload

18 (4.7)

 Acute kidney injury, ARDS, and volume overload

6 (1.6)

 Volume overload

2 (0.5)

 Acute kidney injury and volume overload

2 (0.5)

 Other

54 (14.0)

ARDS = acute respiratory distress syndrome.

DISCUSSION The main finding of this study was that the NHSN VAE/VAC construct had low sensitivity for identifying VAP. Seventy-nine of 83 patients who met the VAP definition and had VAP with www.ccmjournal.org

2025

Lilly et al

high levels of clinical certainty were not detected as having NHSN probable VAP mainly because they did not meet the requirements for stable baseline mechanical ventilator settings or threshold levels of worsening gas exchange (Table 2). Another explanation for the low sensitivity of the NHSN possible VAP construct (18 of 83 VAP; sensitivity of 0.22) and the NHSN probable VAP construct (four of 83 cases; sensitivity of 0.05) is that the VAE/VAE constructs do not use the most sensitive variable for pathologically diagnosed VAP (26, 27), that is, the chest radiograph. The sensitivity and ability to judge anatomic location, time of onset, and progression are among the reasons that the state of the art of diagnosing life-threatening chest infections includes chest imaging studies. One key consideration for calculating and interpreting the test characteristics is that either VAP or NHSN probable VAP must be selected as the criterion standard. This is difficult because the established definition for VAP also has serious flaws. VAP was selected as the criterion standard because it is an established clinically accepted and publicly reported entity that is widely regarded as a complication of hospitalization. Despite poor overall specificity and accuracy, the Johannson criteria (28) upon which the NHSN VAP definition is based has sensitivity for pathologically diagnosed pneumonia of 64–75% (10, 26, 29), VAP is reported to have attributable mortality of 8% (1–3), and effective methods for VAP prevention are known. Analyses that used NHSN probable VAP as the criterion standard also produced concerning findings, we found that VAP also had low sensitivity for NHSN probable VAP. Fifty-eight of 62 patients with NHSN probable VAP were not detected as having VAP because of radiographic evidence that their infiltrates were present before mechanical ventilation and therefore were not the result of it. Measuring these test characteristics allows us to compare how this new surveillance paradigm compares to the established VAP paradigm using the criteria published by the CDC Division of Healthcare Quality Promotion VAP Surveillance Working Group (6). The most important criteria for comparing a public health program designed to improve upon one known to be able to lower attributable mortality is the ability of the programs to achieve that aim. The electiveness of VAP and NHSN VAE/VAC surveillance programs for reducing preventable mortality can be compared. Using conservative estimates of the prevalence VAP of 250,000 cases annually in the United States (4), rates of effectiveness for VAP prevention of 80% (30, 31), and an attributable mortality rate of 8% (1–3). A VAP-focused program would prevent 200,000 infections and 16,000 deaths per year; an HNSN VAE/VAC program that detected one third of VAP cases would detect 82,500 cases and prevent only 6,600 deaths. In this example, 9,400 more Americans would survive their hospitalization each year using achievable VAP-focused strategies than the proposed NHSN VAE/VAC strategy. In this example, little benefit was ascribed to identifying VAC/VAE patients because most of this clinically heterogeneous group of patients did not have a complication of hospitalization for which prevention has been shown to be effective (Table 4). It is clear, however, that some of these cases could be prevented or their detection evaded. Using the IVAC, 2026

www.ccmjournal.org

NHSN possible VAP, and NHSN probable VAP constructs rather than VAP would result in similar proportions of missed opportunities for prevention of mortality (1) attributable to this infectious complication of mechanical ventilation. In accord with the program evaluation criteria identified by the working group, we compared the accuracy of the alternative programs for identifying complications of mechanical ventilation. In addition to failing to detect many patients who with reasonable clinical certainty had VAP, most of the patients who met the NHSN VAE/VAC definition did not have evidence of any hospital-acquired complication (Table 4). More than 70% patients who met the NHSN VAE/VAC definition, as a consequence of having their ventilator settings increased, met the case definition for ARDS. The ARDS defining illness was nearly always their presenting illness rather than being caused by hospitalization or mechanical ventilation. Most of these patients did not have VAP or evidence of lower airway infection. Most NHSN VAE/VAC ARDS patients (88%) had lower airway cultures and only 14% had any microbiological evidence of a pulmonary infection. VAP surveillance more accurately detected complications of hospitalization and is strikingly more efficient than NHSN VAE/VAC surveillance for reducing mortality attributable to VAP. NHSN VAE/VAC surveillance had lower total cost than VAP surveillance, and less collaboration among disciplines was required than the collaborative multidisciplinary team approach for detecting VAP used for this study. The costs of NHSN VAE/VAC surveillance is higher than that suggested by previous reports because of the substantial number of patients detected and the time required for determining the cause(s) of the hypoxemic event. In addition, NHSN VAE/VAC surveillance screens a smaller population than VAP surveillance; accordingly, the costs of measurement per case screened were equivalent. Other criteria identified by the working group included reliability and resistance to manipulation. We investigated concerns that automated systems for VAE surveillance that use the January 2013 definitions (7) are susceptible to manipulation. When we modeled the effects of simple algorithmic changes to respiratory therapy protocols and assessed the ability of an automated system to detect NHSN VAE/VAC, we found that 364 of 387 (93%) NHSN VAE/VAC cases escaped detection. In this example, cases escaped detection because they did not meet the requirement for a stable or improving baseline period. Susceptibility to this type of manipulation appears to have been increased by changes to the September 2011 definitions (6) that were published in January 2013 (7). Models that explored the effects of varying the frequency of patients with conditions that were mistaken for VAP calculated that VAP rates could vary approximately five-fold (32), in our model that varied clinician behavior NHSN VAE/VAC rates changed approximately 10-fold. Measurements of rates of complications by the NHSN VAE/VAC method were not more reliable by the VAP method. This study has important limitations. These include that it was retrospective and was not performed on a random sample of patients from the United States adult ICUs. In addition, the September 2014 • Volume 42 • Number 9

Clinical Investigations

two inpatient campuses of the academic medical center had well-defined and supported procedures for preventing VAP that include nearly all nondevice-related best practices (14, 33). The ratio of VAP to NHSN VAE/VAC and total costs of VAP surveillance may be higher for ICUs with less robust preventive measures and lower for systems that obtain radiographs less frequently daily for mechanically ventilated adults. The findings of this study indicate that VAP surveillance programs performed better than NHSN VAE/VAC surveillance programs according to the criteria put forth by the CDC, Division of Healthcare Quality Promotion VAP Surveillance Working Group. A VAP surveillance program was better able to detect hospital complications, reduce attributable mortality, was less susceptible to manipulation, and was able to measure rates of complications more reliably than the NHSN VAE/VAC surveillance program. The finding that unsophisticated and easily achievable changes in clinician behavior can nearly eliminate the detection of NHSN VAE/VAC cases demonstrates that it lacks the validity required for a publicly reported measure. The finding that most of the individuals who have NHSN VAE/ VAC did not have evidence of a hospital-acquired complication also makes the NHSN VAE/VAC constructs unsuitable for comparative measures, especially those tied to financial incentives. The findings of this study suggest that patients are better served by VAP than NHSN VAE/VAC surveillance programs.

ACKNOWLEDGMENTS The authors recognize the efforts of Susan Bradbury, Rose Erlichman, Gail Frigoletto, Deborah Ann Mack, Jennifer MacPherson, and Zita Melvin who collected and organized clinical information for this study. UMass Memorial Critical Care Operations Group: J. Matthias Walz, MD, Nicholas A. Smyrnios, MD, Stephen O. Heard, MD, Timothy Emhoff, MD, Peter H. Bagley, MD, Michelle M. Fernald, MS, RN, Debra Lynn Svec, RN, Nami Heui Kim, MD, Cheryl H. Dunnington, MS, RN, Nancy Simon, MS, RN, M. Elizabeth Colo, MS, RN, Bruce J. Simon, MD, Karen Shea, MS, RN, Wiley R. Hall, MD, Robert Spicer, RN, Craig Smith, MD, Melinda Darrigo, PhD, Linda Josephson, MS, RN, Khaldoun Faris, MD, Paulo Oliveira, MD, Donald Bellerive, RRT, Luanne Hills, RRT, Karen Landry, Stanley Tam, MD, Victoria Diamond, MBA, Michelle L. O’Rourke, MSN, DNP, Elizabeth A. Rekowski, BSN, Brian S. Smith, PharmD, Dinesh Yogaratnam, PharmD, and Maichi Tran, PharmD.

REFERENCES

1. Agrafiotis M, Siempos II, Ntaidou TK, et al: Attributable mortality of ventilator-associated pneumonia: A meta-analysis. Int J Tuberc Lung Dis 2011; 15:1154–1163 2. Nguile-Makao M, Zahar JR, Français A, et al: Attributable mortality of ventilator-associated pneumonia: Respective impact of main characteristics at ICU admission and VAP onset using conditional logistic regression and multi-state models. Intensive Care Med 2010; 36:781–789 3. Schumacher M, Wangler M, Wolkewitz M, et al: Attributable mortality due to nosocomial infections. A simple and useful application of multistate models. Methods Inf Med 2007; 46:595–600 4. Koenig SM, Truwit JD: Ventilator-associated pneumonia: Diagnosis, treatment, and prevention. Clin Microbiol Rev 2006; 19:637–657

Critical Care Medicine

5. Sinuff T, Muscedere J, Cook DJ, et al; Canadian Critical Care Trials Group: Implementation of clinical practice guidelines for ventilatorassociated pneumonia: A multicenter prospective study. Crit Care Med 2013; 41:15–23 6. Burns S, Hammer B, Hess D, et al: Improving Surveillance for Ventilator-Associated Events in Adults. Atlanta, GA, VAP Surveillance Working Group Centers for Disease Control and Prevention, 2012 7. Prevention CfDCa: Ventilator Associated Pneumonia (VAP) Events. Washington, DC, Centers for Disease Control and Prevention, 2013, pp 1–55 8. Klompas M, Khan Y, Kleinman K, et al; CDC Prevention Epicenters Program: Multicenter evaluation of a novel surveillance paradigm for complications of mechanical ventilation. PLoS One 2011; 6:e18062 9. Horan TC, Andrus M, Dudeck MA: CDC/NHSN surveillance definition of health care-associated infection and criteria for specific types of infections in the acute care setting. Am J Infect Control 2008; 36:309–332 10. Tejerina E, Esteban A, Fernández-Segoviano P, et al: Accuracy of clinical definitions of ventilator-associated pneumonia: Comparison with autopsy findings. J Crit Care 2010; 25:62–68 11. Klompas M: Interobserver variability in ventilator-associated pneumonia surveillance. Am J Infect Control 2010; 38:237–239 12. Grgurich PE, Hudcova J, Lei Y, et al: Diagnosis of ventilator-associated pneumonia: Controversies and working toward a gold standard. Curr Opin Infect Dis 2013; 26:140–150 13. Kollef MH: The prevention of ventilator-associated pneumonia. N Engl J Med 1999; 340:627–634 14. Ibrahim EH, Ward S, Sherman G, et al: Experience with a clinical guideline for the treatment of ventilator-associated pneumonia. Crit Care Med 2001; 29:1109–1115 15. Tablan OC, Anderson LJ, Besser R, et al: Guidelines for Preventing Health-Care-Associated Pneumonia, 2003. Recommendations of CDC and the Healthcare Infection Control Practices Advisory Committee [pt. II-III]. Atlanta, GA, U.S. Department of Health & Human Services, Centers for Disease Control and Prevention, 2004 16. Lilly CM, Cody S, Zhao H, et al; University of Massachusetts Memorial Critical Care Operations Group: Hospital mortality, length of stay, and preventable complications among critically ill patients before and after tele-ICU reengineering of critical care processes. JAMA 2011; 305:2175–2183 17. Brower RG, Lanken PN, MacIntyre N, et al; National Heart, Lung, and Blood Institute ARDS Clinical Trials Network: Higher versus lower positive end-expiratory pressures in patients with the acute respiratory distress syndrome. N Engl J Med 2004; 351:327–336 18. Ventilation with lower tidal volumes as compared with traditional tidal volumes for acute lung injury and the acute respiratory distress syndrome. The Acute Respiratory Distress Syndrome Network. N Engl J Med 2000; 342:1301–1308 19. Ferguson ND, Fan E, Camporota L, et al: The Berlin definition of ARDS: An expanded rationale, justification, and supplementary material. Intensive Care Med 2012; 38:1573–1582 20. Mehta RL, Kellum JA, Shah SV, et al; Acute Kidney Injury Network: Acute Kidney Injury Network: Report of an initiative to improve outcomes in acute kidney injury. Crit Care 2007; 11:R31 21. Kruskal WH: WWA: Use of ranks in one-criterion variance analysis. J Am Stat Assoc 1952; 47:583–621 22. Daniel WW: Applied Nonparametric Statistics. Boston, Houghton Mifflin, 1978 23. Hosmer DW, Lemeshow S: Applied Logistic Regression. New York, Wiley, 1989 24. International Business Machines: IBM SPSS Statistics. New York, IBM Corporation, 2011 25. Dudeck MA, Horan TC, Peterson KD, et al: National Healthcare Safety Network (NHSN) Report, Data Summary for 2010, Device-associated Module. Washington, DC, Unites States Government, 2010 26. Fàbregas N, Ewig S, Torres A, et al: Clinical diagnosis of ventilator associated pneumonia revisited: Comparative validation using immediate post-mortem lung biopsies. Thorax 1999; 54:867–873 27. Torres A, el-Ebiary M, Padró L, et al: Validation of different techniques for the diagnosis of ventilator-associated pneumonia. Comparison www.ccmjournal.org

2027

Lilly et al with immediate postmortem pulmonary biopsy. Am J Respir Crit Care Med 1994; 149:324–331 28. Johanson WG Jr, Pierce AK, Sanford JP, et al: Nosocomial respiratory infections with gram-negative bacilli. The significance of colonization of the respiratory tract. Ann Intern Med 1972; 77:701–706 29. Andrews CP, Coalson JJ, Smith JD, et al: Diagnosis of nosocomial bacterial pneumonia in acute, diffuse lung injury. Chest 1981; 80:254–258 30. Bouadma L, Wolff M, Lucet JC: Ventilator-associated pneumonia and its prevention. Curr Opin Infect Dis 2012; 25:395–404

2028

www.ccmjournal.org

31. Ramirez P, Bassi GL, Torres A: Measures to prevent nosocomial infections during mechanical ventilation. Curr Opin Crit Care 2012; 18:86–92 32. Klompas M, Kulldorff M, Platt R: Risk of misleading ventilator-associated pneumonia rates with use of standard clinical and microbiological criteria. Clin Infect Dis 2008; 46:1443–1446 33. Kollef MH: Prevention of ventilator-associated pneumonia or ventilator-associated complications: A worthy, yet challenging, goal. Crit Care Med 2012; 40:271–277

September 2014 • Volume 42 • Number 9

Prevalence and test characteristics of national health safety network ventilator-associated events.

The primary aim of the study was to measure the test characteristics of the National Health Safety Network ventilator-associated event/ventilator-asso...
660KB Sizes 0 Downloads 3 Views