Correspondence
λ Se + (1 − λ )Sp
where λ (between 0 and 1) is the weight that study investigators attributed to the sensitivity. A λ of 0·5 attributes equal weights to sensitivity and specificity, equivalent to maximisation of the Youden index. Our method calculates a summary receiver operating characteristic (SROC) curve based on the assumption that all investigators have optimised their cutoffs with a common weighting parameter—λ— which is estimated. To apply this model to the given data, we used the R (version R-2.15.3) package meta-analysis of diagnostic accuracy (version 0.5.4)3 and reproduced figure 3 from Wacker and colleagues’ study, showing the bivariate estimate of the average pair of sensitivity and specificity (figure). The red SROC curve (following Rutter and Gatsonis’ approach4) corresponds to that shown in figure 3. Our method produced the green curve shown in the figure. As expected, that curve runs below the Rutter-Gatsonis curve, showing the ability of our approach to account for the best cutoff selection and to avoid bias caused by an overoptimistic SROC curve, by contrast with the standard approach. The estimate of λ was 0·491, which is remarkably close to 0·5. This finding supports the hypothesis that investigators tended to select the cutoffs that maximised the Youden index. www.thelancet.com/infection Vol 13 December 2013
As for the role of procalcitonin as a potential diagnostic marker for sepsis, its ability to distinguish between patients with sepsis and those with a systemic inflammatory response syndrome of non-infectious origin might be even lower than that derived from the meta-analysis done by Wacker and colleagues.1 Our method suggests a pooled sensitivity of 0·72 (rather than 0·77) and a pooled specificity of 0·73 (rather than 0·79)—values that have already been questioned to be sufficiently large to justify the use of procalcitonin routinely in clinical practice.5 GR was funded by the German Research Foundation (DFG; grant number RU 1747/1-1). We declare that we have no conflicts of interest.
*Gerta Rücker, Martin Schumacher [email protected] Institute of Medical Biometry and Medical Statistics, University Medical Center Freiburg, Freiburg 79104, Germany 1
2
3
4
5
Wacker C, Prkno A, Brunkhorst FM, Schlattmann P. Procalcitonin as a diagnostic marker for sepsis: a systematic review and meta-analysis. Lancet Infect Dis 2013; 13: 426–35. Rücker G, Schumacher M. Summary ROC curve based on the weighted Youden index for selecting an optimal cutpoint in meta-analysis of diagnostic accuracy. Stat Med 2010; 29: 3069–78. R Development Core Team. R: a language and environment for statistical computing. Vienna: R Foundation for Statistical Computing, 2012. Rutter CM, Gatsonis CA. A hierarchical regression approach to meta-analysis of diagnostic test accuracy evaluations. Stat Med 2001; 20: 2865–84. Afshari A, Harbarth S. Procalcitonin as diagnostic biomarker of sepsis. Lancet Infect Dis 2013; 13: 382–84.
We read with interest the article by Christina Wacker and colleagues1 about the use of procalcitonin as a sepsis biomarker. Contrary to a previous metaanalysis,2 the investigators conclude that procalcitonin usefully differentiates sepsis from a systemic inflammatory response syndrome (SIRS) arising from non-infectious causes. Although several studies that used a sensitive assay were included, the sensitivity and specificity of procalcitonin to diagnose sepsis were only marginally higher than reported previously.
1·0
0·8
0·6 Sensitivity
for diagnostic accuracy”.1 As such, the investigators of the primary studies seemed to select cutoffs such that they maximised the sum of sensitivity and specificity. We want to call the reader’s attention to an approach for meta-analysis of data from studies of diagnostic test accuracy that we published previously to account for this occurrence. 2 We modelled the assumption that the investigators of the primary studies selected their cutoff such that it maximised a weighted sum of sensitivity (Se) and specificity (Sp):
0·4
Summary operating point 95% confidence region 95% prediction region SROC curve* SROC curve accounting for selection†
0·2
0 0
0·2
0·4
0·6
0·8
1·0
False-positive rate
Figure: Results of Wacker and colleagues’ procalcitonin meta-analysis1 SROC=summary receiver operating characteristic. *Following Rutter and Gatsonis‘ approach.4 †Weighted sum, λ=0·491.
Similar to D-Dimer testing in venous thromboembolism, procalcitonin is a valuable marker to rule out bacterial infection or sepsis in patients with a low pretest probability of disease.3 However, in individuals with a moderate to high suspicion of sepsis, we believe that one procalcitonin measurement and one cutoff for all patients, irrespective of their underlying disease (eg, medical vs surgical or trauma), should not be used to guide initial treatment decisions because of its insufficient ability to differentiate SIRS from sepsis in critically ill patients.4 The strength of procalcitonin lies in its favourable kinetics compared with traditional markers like C-reactive protein. Consistent evidence supports the use of serial procalcitonin measurements for early discontinuation of antibiotics in patients with pneumonia and in those admitted to intensive-care units (ICU). 3 Additionally, several studies have shown that a rise in procalcitonin can precede infection by 24 h, and that the change between procalcitonin measurements on day –1 and day 0 might be a better marker
For the R statistical programme see http://www.R-project.org/
1013
Correspondence
See Online for appendix
of sepsis than is one procalcitonin measurement on the day of fever.5 This finding suggests the need to investigate an approach that uses procalcitonin kinetics (eg, two measurements within 8–12 h for newly admitted patients) and differential cutoffs to distinguish SIRS from sepsis. Stored blood samples from day 1 are frequently available for procalcitonin testing in inpatients and ICU patients with new onset fever. Once patients have started on antibiotics, intensivists are frequently reluctant to stop antibiotics unless cultures are negative at 48 h. Serial procalcitonin testing within 24 h could reduce unnecessary use of antibiotics and selective pressure for multiresistant pathogens. We have suggested an approach to the use of procalcitonin in ICU patients with suspected sepsis (appendix). Procalcitonin might truly be a very useful marker if used as one test in the right context for ruling out sepsis, or by use of its short half-life to rationalise antibiotics. However, a one-size-fits-all approach (as tested in a recent study4) will only generate substantial costs with no benefits for patients. We declare that we have no conflicts of interest.
Michael Osthoff, *Damon P Eisen [email protected] Victorian Infectious Diseases Service, Royal Melbourne Hospital, VIC 3050, Australia (MO, DPE); and Department of Medicine, Royal Melbourne Hospital, University of Melbourne, VIC, Australia (MO, DPE) 1
2
3
4
1014
Wacker C, Prkno A, Brunkhorst FM, Schlattmann P. Procalcitonin as a diagnostic marker for sepsis: a systematic review and meta-analysis. Lancet Infect Dis 2013; 13: 426–35. Tang BM, Eslick GD, Craig JC, McLean AS. Accuracy of procalcitonin for sepsis diagnosis in critically ill patients: systematic review and meta-analysis. Lancet Infect Dis 2007; 7: 210–17. Schuetz P, Briel M, Christ-Crain M, et al. Procalcitonin to guide initiation and duration of antibiotic treatment in acute respiratory infections: an individual patient data metaanalysis. Clin Infect Dis 2012; 55: 651–62. Layios N, Lambermont B, Canivet JL, et al. Procalcitonin usefulness for the initiation of antibiotic treatment in intensive care unit patients. Crit Care Med 2012; 40: 2304–09.
5
Charles PE, Kus E, Aho S, Prin S, Doise JM, Olsson NO, et al. Serum procalcitonin for the early recognition of nosocomial infection in the critically ill patients: a preliminary report. BMC Infect Dis 2009; 9: 49.
Attributable mortality of ventilator-associated pneumonia Wilhemina Melsen and colleagues1 estimated the attributable mortality of ventilator-associated pneumonia from individual patient data derived from randomised prevention studies. On the basis of the search terms “ventilator-associated pneumonia” and “randomisation” the investigators identified 45 trials, of which 24 were included. They state that the included studies are reliable representatives of all studies of prevention of ventilator-associated pneumonia. However, the analysis included only six decontamination studies. This point is remarkable, because a meta-analysis2 of 36 randomised trials concluded that treatment with a combination of topical and systemic antibiotics reduces the incidence of respiratory tract infections from 40% to 19%, and overall mortality from 30% to 24%, with four patients needed-totreat to prevent one infection, and 18 to prevent one death. From these data suggesting that a 53% reduction in respiratory tract infections leads to a 6% reduction in mortality, the attributable mortality of respiratory infections is 11%, which is similar to the 13% calculated by Melsen and colleagues. Furthermore, Melsen and colleagues state that mortality after ventilator-associated pneumonia is mainly caused by prolonged exposure to risk of death due to increased length of stay in an intensive-care unit (ICU). However, this assumption contradicts previous findings. In a case-control study, Klompas and colleagues3 elegantly
showed that ventilator-associated pneumonia significantly prolonged both median duration of mechanical ventilation and median length of stay in an ICU by 6 days. However, hospital mortality did not differ significantly. Indeed, mortality is not proportional to length of ICU stay—severely ill patients dying early during their ICU admission have a shorter length of stay than do those recovering after prolonged mechanical ventilation. In a retrospective analysis of 543 mechanically ventilated patients, Hayashi and colleagues4 also reported significantly prolonged mechanical ventilation and length of stay in an ICU without an increase in mortality in patients with ventilatorassociated complications; however, they did note that patients with ventilator-associated complications more often had chronic obstructive pulmonary disease and acute kidney injury than did those with no complications. In patients with ventilator-associated pneumonia, underlying illness and complicating organ failure could have a greater effect on mortality than does length of ICU stay. I declare that I have no conflicts of interest.
Anne-Cornélie JM de Pont [email protected] Adult Intensive Care Unit, Academic Medical Centre, University of Amsterdam, Amsterdam 1105 AZ, Netherlands 1
2
3
4
Melsen WG, Rovers MM, Groenwold RH, et al. Attributable mortality of ventilatorassociated pneumonia: a meta-analysis of individual patient data from randomised prevention studies. Lancet Infect Dis 2013; 13: 665–71. D’Amico R, Pifferi S, Torri V, et al. Antibiotic prophylaxis to reduce respiratory tract infections and mortality in adults receiving intensive care. Cochrane Database Syst Rev 2009; 4: CD000022. Klompas M, Khan Y, Kleinman K, et al. Multicenter evaluation of a novel surveillance paradigm for complications of mechanical ventilation. PLoS One 2011; 6: e18062. Hayashi Y, Morisawa K, Klompas M, et al. Toward improved surveillance: the impact of ventilator-associated complications on length of stay and antibiotics use in patients in intensive care units. Clin Infect Dis 2013; 56: 471-77.
www.thelancet.com/infection Vol 13 December 2013
λ Se + (1 − λ )Sp
where λ (between 0 and 1) is the weight that study investigators attributed to the sensitivity. A λ of 0·5 attributes equal weights to sensitivity and specificity, equivalent to maximisation of the Youden index. Our method calculates a summary receiver operating characteristic (SROC) curve based on the assumption that all investigators have optimised their cutoffs with a common weighting parameter—λ— which is estimated. To apply this model to the given data, we used the R (version R-2.15.3) package meta-analysis of diagnostic accuracy (version 0.5.4)3 and reproduced figure 3 from Wacker and colleagues’ study, showing the bivariate estimate of the average pair of sensitivity and specificity (figure). The red SROC curve (following Rutter and Gatsonis’ approach4) corresponds to that shown in figure 3. Our method produced the green curve shown in the figure. As expected, that curve runs below the Rutter-Gatsonis curve, showing the ability of our approach to account for the best cutoff selection and to avoid bias caused by an overoptimistic SROC curve, by contrast with the standard approach. The estimate of λ was 0·491, which is remarkably close to 0·5. This finding supports the hypothesis that investigators tended to select the cutoffs that maximised the Youden index. www.thelancet.com/infection Vol 13 December 2013
As for the role of procalcitonin as a potential diagnostic marker for sepsis, its ability to distinguish between patients with sepsis and those with a systemic inflammatory response syndrome of non-infectious origin might be even lower than that derived from the meta-analysis done by Wacker and colleagues.1 Our method suggests a pooled sensitivity of 0·72 (rather than 0·77) and a pooled specificity of 0·73 (rather than 0·79)—values that have already been questioned to be sufficiently large to justify the use of procalcitonin routinely in clinical practice.5 GR was funded by the German Research Foundation (DFG; grant number RU 1747/1-1). We declare that we have no conflicts of interest.
*Gerta Rücker, Martin Schumacher [email protected] Institute of Medical Biometry and Medical Statistics, University Medical Center Freiburg, Freiburg 79104, Germany 1
2
3
4
5
Wacker C, Prkno A, Brunkhorst FM, Schlattmann P. Procalcitonin as a diagnostic marker for sepsis: a systematic review and meta-analysis. Lancet Infect Dis 2013; 13: 426–35. Rücker G, Schumacher M. Summary ROC curve based on the weighted Youden index for selecting an optimal cutpoint in meta-analysis of diagnostic accuracy. Stat Med 2010; 29: 3069–78. R Development Core Team. R: a language and environment for statistical computing. Vienna: R Foundation for Statistical Computing, 2012. Rutter CM, Gatsonis CA. A hierarchical regression approach to meta-analysis of diagnostic test accuracy evaluations. Stat Med 2001; 20: 2865–84. Afshari A, Harbarth S. Procalcitonin as diagnostic biomarker of sepsis. Lancet Infect Dis 2013; 13: 382–84.
We read with interest the article by Christina Wacker and colleagues1 about the use of procalcitonin as a sepsis biomarker. Contrary to a previous metaanalysis,2 the investigators conclude that procalcitonin usefully differentiates sepsis from a systemic inflammatory response syndrome (SIRS) arising from non-infectious causes. Although several studies that used a sensitive assay were included, the sensitivity and specificity of procalcitonin to diagnose sepsis were only marginally higher than reported previously.
1·0
0·8
0·6 Sensitivity
for diagnostic accuracy”.1 As such, the investigators of the primary studies seemed to select cutoffs such that they maximised the sum of sensitivity and specificity. We want to call the reader’s attention to an approach for meta-analysis of data from studies of diagnostic test accuracy that we published previously to account for this occurrence. 2 We modelled the assumption that the investigators of the primary studies selected their cutoff such that it maximised a weighted sum of sensitivity (Se) and specificity (Sp):
0·4
Summary operating point 95% confidence region 95% prediction region SROC curve* SROC curve accounting for selection†
0·2
0 0
0·2
0·4
0·6
0·8
1·0
False-positive rate
Figure: Results of Wacker and colleagues’ procalcitonin meta-analysis1 SROC=summary receiver operating characteristic. *Following Rutter and Gatsonis‘ approach.4 †Weighted sum, λ=0·491.
Similar to D-Dimer testing in venous thromboembolism, procalcitonin is a valuable marker to rule out bacterial infection or sepsis in patients with a low pretest probability of disease.3 However, in individuals with a moderate to high suspicion of sepsis, we believe that one procalcitonin measurement and one cutoff for all patients, irrespective of their underlying disease (eg, medical vs surgical or trauma), should not be used to guide initial treatment decisions because of its insufficient ability to differentiate SIRS from sepsis in critically ill patients.4 The strength of procalcitonin lies in its favourable kinetics compared with traditional markers like C-reactive protein. Consistent evidence supports the use of serial procalcitonin measurements for early discontinuation of antibiotics in patients with pneumonia and in those admitted to intensive-care units (ICU). 3 Additionally, several studies have shown that a rise in procalcitonin can precede infection by 24 h, and that the change between procalcitonin measurements on day –1 and day 0 might be a better marker
For the R statistical programme see http://www.R-project.org/
1013
Correspondence
See Online for appendix
of sepsis than is one procalcitonin measurement on the day of fever.5 This finding suggests the need to investigate an approach that uses procalcitonin kinetics (eg, two measurements within 8–12 h for newly admitted patients) and differential cutoffs to distinguish SIRS from sepsis. Stored blood samples from day 1 are frequently available for procalcitonin testing in inpatients and ICU patients with new onset fever. Once patients have started on antibiotics, intensivists are frequently reluctant to stop antibiotics unless cultures are negative at 48 h. Serial procalcitonin testing within 24 h could reduce unnecessary use of antibiotics and selective pressure for multiresistant pathogens. We have suggested an approach to the use of procalcitonin in ICU patients with suspected sepsis (appendix). Procalcitonin might truly be a very useful marker if used as one test in the right context for ruling out sepsis, or by use of its short half-life to rationalise antibiotics. However, a one-size-fits-all approach (as tested in a recent study4) will only generate substantial costs with no benefits for patients. We declare that we have no conflicts of interest.
Michael Osthoff, *Damon P Eisen [email protected] Victorian Infectious Diseases Service, Royal Melbourne Hospital, VIC 3050, Australia (MO, DPE); and Department of Medicine, Royal Melbourne Hospital, University of Melbourne, VIC, Australia (MO, DPE) 1
2
3
4
1014
Wacker C, Prkno A, Brunkhorst FM, Schlattmann P. Procalcitonin as a diagnostic marker for sepsis: a systematic review and meta-analysis. Lancet Infect Dis 2013; 13: 426–35. Tang BM, Eslick GD, Craig JC, McLean AS. Accuracy of procalcitonin for sepsis diagnosis in critically ill patients: systematic review and meta-analysis. Lancet Infect Dis 2007; 7: 210–17. Schuetz P, Briel M, Christ-Crain M, et al. Procalcitonin to guide initiation and duration of antibiotic treatment in acute respiratory infections: an individual patient data metaanalysis. Clin Infect Dis 2012; 55: 651–62. Layios N, Lambermont B, Canivet JL, et al. Procalcitonin usefulness for the initiation of antibiotic treatment in intensive care unit patients. Crit Care Med 2012; 40: 2304–09.
5
Charles PE, Kus E, Aho S, Prin S, Doise JM, Olsson NO, et al. Serum procalcitonin for the early recognition of nosocomial infection in the critically ill patients: a preliminary report. BMC Infect Dis 2009; 9: 49.
Attributable mortality of ventilator-associated pneumonia Wilhemina Melsen and colleagues1 estimated the attributable mortality of ventilator-associated pneumonia from individual patient data derived from randomised prevention studies. On the basis of the search terms “ventilator-associated pneumonia” and “randomisation” the investigators identified 45 trials, of which 24 were included. They state that the included studies are reliable representatives of all studies of prevention of ventilator-associated pneumonia. However, the analysis included only six decontamination studies. This point is remarkable, because a meta-analysis2 of 36 randomised trials concluded that treatment with a combination of topical and systemic antibiotics reduces the incidence of respiratory tract infections from 40% to 19%, and overall mortality from 30% to 24%, with four patients needed-totreat to prevent one infection, and 18 to prevent one death. From these data suggesting that a 53% reduction in respiratory tract infections leads to a 6% reduction in mortality, the attributable mortality of respiratory infections is 11%, which is similar to the 13% calculated by Melsen and colleagues. Furthermore, Melsen and colleagues state that mortality after ventilator-associated pneumonia is mainly caused by prolonged exposure to risk of death due to increased length of stay in an intensive-care unit (ICU). However, this assumption contradicts previous findings. In a case-control study, Klompas and colleagues3 elegantly
showed that ventilator-associated pneumonia significantly prolonged both median duration of mechanical ventilation and median length of stay in an ICU by 6 days. However, hospital mortality did not differ significantly. Indeed, mortality is not proportional to length of ICU stay—severely ill patients dying early during their ICU admission have a shorter length of stay than do those recovering after prolonged mechanical ventilation. In a retrospective analysis of 543 mechanically ventilated patients, Hayashi and colleagues4 also reported significantly prolonged mechanical ventilation and length of stay in an ICU without an increase in mortality in patients with ventilatorassociated complications; however, they did note that patients with ventilator-associated complications more often had chronic obstructive pulmonary disease and acute kidney injury than did those with no complications. In patients with ventilator-associated pneumonia, underlying illness and complicating organ failure could have a greater effect on mortality than does length of ICU stay. I declare that I have no conflicts of interest.
Anne-Cornélie JM de Pont [email protected] Adult Intensive Care Unit, Academic Medical Centre, University of Amsterdam, Amsterdam 1105 AZ, Netherlands 1
2
3
4
Melsen WG, Rovers MM, Groenwold RH, et al. Attributable mortality of ventilatorassociated pneumonia: a meta-analysis of individual patient data from randomised prevention studies. Lancet Infect Dis 2013; 13: 665–71. D’Amico R, Pifferi S, Torri V, et al. Antibiotic prophylaxis to reduce respiratory tract infections and mortality in adults receiving intensive care. Cochrane Database Syst Rev 2009; 4: CD000022. Klompas M, Khan Y, Kleinman K, et al. Multicenter evaluation of a novel surveillance paradigm for complications of mechanical ventilation. PLoS One 2011; 6: e18062. Hayashi Y, Morisawa K, Klompas M, et al. Toward improved surveillance: the impact of ventilator-associated complications on length of stay and antibiotics use in patients in intensive care units. Clin Infect Dis 2013; 56: 471-77.
www.thelancet.com/infection Vol 13 December 2013
