CORRESPONDENCE
Confounding explains ‘deaths avoided’ Although it is clear that in the UK, standardised mortality rates have decreased dramatically over the past three decades,1 it may not be immediately apparent that this secular trend may substantially confound relatively short-term studies. Based on retrospectively obtained, uncontrolled data, Schmidt et al2 assert that introduction of an electronic physiological surveillance system reduced excess deaths. If, however, we control for the secular decrease in mortality rate, then this apparent benefit almost vanishes, as is shown in table 1, where I have applied the relevant decrease in standardised mortality rate for England1 to their Portsmouth data. The final row of table 1 contains the number of ‘deaths avoided’ after application of secular improvements to the number of expected deaths; the row above contains the numbers imputed by Schmidt et al. In the online supplementary appendix A, I provide details of my calculations. Most of the apparent improvement is shown to represent secular change. In these data, the small residual number of ‘deaths avoided’ deserves further examination, as it may still represent a valid signal. From control chart analysis of adjusted data, the authors identify a transition point in mid-to-late 2008, and attribute this change to ‘surveillance for the whole patient journey’ (extension to a single stroke ward). The residual difference is, however, apparent in the period from 2005 to 2008, before widespread deployment of their surveillance system. A further confounder may more than explain the residual difference. Table 1
The study death rates are derived from Dr Foster, and exclude palliative care-related deaths. It is now known that in the UK, the reporting of deaths as ‘palliative’ increased from about 5% to about 15% over the period of the study.3 Recent data from Portsmouth suggest a rate of 12%.3 4 An increase in the number of deaths related to palliative care would result in a corresponding, apparent decrease in study deaths. Of note is that at the second hospital studied (University Hospital Coventry), owing to the shorter interval the secular effect of changes in standardised mortality is less marked, the residual difference is larger and current palliative care rates are lower (6.5%),3 4 suggesting that part of the apparent improvement here may not be an artefact. The profound influence of such confounders on this study is a caution for designers and reviewers of all future, similar studies, particularly those that rely on uncontrolled retrospective data. It would seem wise to ensure (at the very least) that study design incorporates the examination of ‘control institutions’ where the intervention of interest has not been applied. It may even be possible to ‘rescue’ the results of existing studies like that of Schmidt et al by adjustment using appropriate controls. Johan Michael van Schalkwyk
Correspondence to Dr Johan Michael van Schalkwyk, Auckland City Hospital—Anaesthesia and Perioperative Medicine, 2 Park Road Grafton, Auckland 1023, New Zealand; [email protected]
online (http://dx.doi.org/10.1136/bmjqs2014-003748).
To cite van Schalkwyk JM. BMJ Qual Saf 2015;24:175. Received 3 November 2014 Accepted 4 November 2014
▸ http://dx.doi.org/10.1136/bmjqs-2014-003845 BMJ Qual Saf 2015;24:175. doi:10.1136/bmjqs-2014-003748
REFERENCES 1 Office for National Statistics. Mortality in the United Kingdom, 2010. Office for National Statistics 20 January 2012. http://www.ons.gov.uk/ons/rel/ mortality-ageing/mortality-in-theunited-kingdom/mortality-in-the-unitedkingdom–2010/mortality-in-the-uk2010.html (accessed 2 Nov 2014). 2 Schmidt PE, Meredith P, Prytherch DR, et al. Impact of introducing an electronic physiological surveillance system on hospital mortality. BMJ Qual Saf 2015;24:10–20. 3 “dr foster intelligence”. Palliative care. Dr Foster Intelligence 2014:11–12. Portsmouth code is “RHU”, UHC/ Warwickshire code is “RKB”. http:// www.drfoster.com/wp-content/uploads/ 2014/08/Palliative-report-final-260314. pdf (accessed 2 Nov 2014). 4 “dr foster intelligence”. My Hospital Guide: HG2013-data.zip (downloadable ZIP file) Dr Foster Intelligence 2013. http://myhospitalguide.drfosterintelligence. co.uk/#/mortality (accessed 2 Nov 2014).
Competing interests None. Provenance and peer review Not commissioned; internally peer reviewed. ▸ Additional material is published online only. To view please visit the journal
Secular adjustment for reported deaths (Portsmouth)
Deaths Expected deaths Secular adjustment factor ‘Deaths avoided’ Secular ‘deaths avoided’
2004
2005
2006
2007
2008
2009
2010
2168
2172 2216 0.98 44 52
2055 2242 0.95 187 123
2053 2277 0.92 224 175
2132 2368 0.92 236 200
1996 2359 0.87 363 302
1904 2301 0.85 397 335
1.00
BMJ Qual Saf February 2015 Vol 24 No 2
175
Confounding explains ‘deaths avoided’ Although it is clear that in the UK, standardised mortality rates have decreased dramatically over the past three decades,1 it may not be immediately apparent that this secular trend may substantially confound relatively short-term studies. Based on retrospectively obtained, uncontrolled data, Schmidt et al2 assert that introduction of an electronic physiological surveillance system reduced excess deaths. If, however, we control for the secular decrease in mortality rate, then this apparent benefit almost vanishes, as is shown in table 1, where I have applied the relevant decrease in standardised mortality rate for England1 to their Portsmouth data. The final row of table 1 contains the number of ‘deaths avoided’ after application of secular improvements to the number of expected deaths; the row above contains the numbers imputed by Schmidt et al. In the online supplementary appendix A, I provide details of my calculations. Most of the apparent improvement is shown to represent secular change. In these data, the small residual number of ‘deaths avoided’ deserves further examination, as it may still represent a valid signal. From control chart analysis of adjusted data, the authors identify a transition point in mid-to-late 2008, and attribute this change to ‘surveillance for the whole patient journey’ (extension to a single stroke ward). The residual difference is, however, apparent in the period from 2005 to 2008, before widespread deployment of their surveillance system. A further confounder may more than explain the residual difference. Table 1
The study death rates are derived from Dr Foster, and exclude palliative care-related deaths. It is now known that in the UK, the reporting of deaths as ‘palliative’ increased from about 5% to about 15% over the period of the study.3 Recent data from Portsmouth suggest a rate of 12%.3 4 An increase in the number of deaths related to palliative care would result in a corresponding, apparent decrease in study deaths. Of note is that at the second hospital studied (University Hospital Coventry), owing to the shorter interval the secular effect of changes in standardised mortality is less marked, the residual difference is larger and current palliative care rates are lower (6.5%),3 4 suggesting that part of the apparent improvement here may not be an artefact. The profound influence of such confounders on this study is a caution for designers and reviewers of all future, similar studies, particularly those that rely on uncontrolled retrospective data. It would seem wise to ensure (at the very least) that study design incorporates the examination of ‘control institutions’ where the intervention of interest has not been applied. It may even be possible to ‘rescue’ the results of existing studies like that of Schmidt et al by adjustment using appropriate controls. Johan Michael van Schalkwyk
Correspondence to Dr Johan Michael van Schalkwyk, Auckland City Hospital—Anaesthesia and Perioperative Medicine, 2 Park Road Grafton, Auckland 1023, New Zealand; [email protected]
online (http://dx.doi.org/10.1136/bmjqs2014-003748).
To cite van Schalkwyk JM. BMJ Qual Saf 2015;24:175. Received 3 November 2014 Accepted 4 November 2014
▸ http://dx.doi.org/10.1136/bmjqs-2014-003845 BMJ Qual Saf 2015;24:175. doi:10.1136/bmjqs-2014-003748
REFERENCES 1 Office for National Statistics. Mortality in the United Kingdom, 2010. Office for National Statistics 20 January 2012. http://www.ons.gov.uk/ons/rel/ mortality-ageing/mortality-in-theunited-kingdom/mortality-in-the-unitedkingdom–2010/mortality-in-the-uk2010.html (accessed 2 Nov 2014). 2 Schmidt PE, Meredith P, Prytherch DR, et al. Impact of introducing an electronic physiological surveillance system on hospital mortality. BMJ Qual Saf 2015;24:10–20. 3 “dr foster intelligence”. Palliative care. Dr Foster Intelligence 2014:11–12. Portsmouth code is “RHU”, UHC/ Warwickshire code is “RKB”. http:// www.drfoster.com/wp-content/uploads/ 2014/08/Palliative-report-final-260314. pdf (accessed 2 Nov 2014). 4 “dr foster intelligence”. My Hospital Guide: HG2013-data.zip (downloadable ZIP file) Dr Foster Intelligence 2013. http://myhospitalguide.drfosterintelligence. co.uk/#/mortality (accessed 2 Nov 2014).
Competing interests None. Provenance and peer review Not commissioned; internally peer reviewed. ▸ Additional material is published online only. To view please visit the journal
Secular adjustment for reported deaths (Portsmouth)
Deaths Expected deaths Secular adjustment factor ‘Deaths avoided’ Secular ‘deaths avoided’
2004
2005
2006
2007
2008
2009
2010
2168
2172 2216 0.98 44 52
2055 2242 0.95 187 123
2053 2277 0.92 224 175
2132 2368 0.92 236 200
1996 2359 0.87 363 302
1904 2301 0.85 397 335
1.00
BMJ Qual Saf February 2015 Vol 24 No 2
175
