Downloaded from http://qualitysafety.bmj.com/ on July 12, 2015 - Published by group.bmj.com
Correspondence
Impact of introducing an electronic physiological surveillance system on hospital mortality Shaw et al1 are correct in pointing out that the use of a single year’s mortality data as a baseline comparator could introduce bias to our findings. They and Van Schalkwyk2 are also right to highlight the national reduction in hospital mortality rates over the past decade. However, we did not rely solely upon the overall mortality 176
reduction to come to the conclusion that there was an association between the timing of the introduction of an electronic physiological surveillance system (EPSS) and the reduced mortality at the two study hospitals.3 Both Shaw et al and Van Schalkwyk appear to ignore the significance of the main findings of our study. The reductions in annual observed deaths are concentrated in specific years (Portsmouth 2006 and 2009, years 2 and 5 after baseline; Coventry 2009, year 3 after baseline).3 Also, the monthly p-charts reveal very significant, abrupt and sustained changes in observed deaths in both hospitals within the space of a few months of the full rollout at each site. Finally, these reductions in mortality are associated with substantially decreased month-to-month variation in deaths. As we described in our paper,3 and as Van Schalkwyk observes, the signal is clearer at Coventry in 2009 because a prior statistically significant change in hospital deaths had already occurred at Portsmouth in 2006, coincident with the initial deployment of EPSS on the Acute Medicine Unit. This lends support to our view that the majority of avoided deaths over the 7 years occurred in the periods following the introduction of EPSS. However, we were careful to point out that not all avoided deaths in this period were necessarily the effect of EPSS. Van Schalkwyk uses long-run standardised (age-adjusted) mortality rates (SMR) to calculate a ‘secular adjustment factor’, which he then uses to adjust observed in-hospital deaths. However, population SMRs represent the risk at a particular age that a proportion of the population will die. They have been declining because of a huge range of factors including lifestyle and technological changes even if healthcare delivery improvements are excluded. In contrast, hospital deaths are drawn from a different sample of the population, namely those who, despite increased longevity, have reached a point where they need to be hospitalised for an BMJ Qual Saf February 2015 Vol 24 No 2
Downloaded from http://qualitysafety.bmj.com/ on July 12, 2015 - Published by group.bmj.com
Correspondence acute illness, usually then with more comorbidities. While Van Schalkwyk is correct to point out the potential confounding effect that varying the number of admissions coded with palliative care can have on hospital standardised mortality ratio (HSMR) calculation, it is important to point out that we used the observed deaths in the HSMR 80 data set, not HSMR. The only way a change in ‘palliative care’ deaths could have contributed to the change in observed deaths that we saw would have been for there to be a sudden change in practice in 2006 or 2009 such that a large proportion of patients requiring palliative care died at home or in a hospice rather than in a hospital. There is no evidence to support this. We agree with Shaw et al that well-intentioned interventions in healthcare occasionally produce unintended harm. Arguably for EPSS to have had an effect, earlier or more frequent escalation to junior doctors was anticipated. We have no evidence to support the view that EPSS led to either an inappropriate increase in workload or a reallocation of junior doctor time and priorities. The introduction of EPSS did not lead to a change in junior doctor staffing. Shaw et al’s view that a multicentre randomised controlled trial (RCT) would assist in identifying whether EPSS was the true reason for the fall in mortality fails to recognise the complexity, required duration and expense of implementing such a trial. In addition, RCTs may not
BMJ Qual Saf February 2015 Vol 24 No 2
necessarily be the best research method for complex interventions such as ours.4–6 Perhaps the best that can be hoped for is a much larger study of the type that we conducted, but comprising many hospitals using a de facto multicentre stepped wedge approach. Finally, two of the authors of the letter by Shaw et al1 appear to have undeclared conflicts of interest pertinent to their interest in our research.7 8 Paul E Schmidt,1 Paul Meredith,2 David R Prytherch,2,3 Duncan Watson,4 Valerie Watson,5 Roger M Killen,6 Peter Greengross,6,7 Mohammed A Mohammed,8 Gary B Smith9 1 Medical Assessment Unit, Portsmouth Hospitals NHS Trust, Portsmouth, Hampshire, UK 2 TEAMS Centre, Portsmouth Hospitals NHS Trust, Portsmouth, Hampshire, UK 3 School of Computing, University of Portsmouth, Portsmouth, Hampshire, UK 4 Intensive Care Medicine and Anaesthesia, University Hospitals, Coventry and Warwickshire NHS Trust, Coventry, UK 5 Critical Care Outreach, University Hospitals Coventry and Warwickshire NHS Trust, Coventry, UK 6 The Learning Clinic, London, UK 7 Department of Primary Care and Public Health, Imperial College Healthcare NHS Trust, London, UK 8 Quality & Effectiveness, School of Health Studies, University of Bradford, Bradford, UK 9 School of Health & Social Care, University of Bournemouth, Bournemouth, UK
Correspondence to Dr Paul Schmidt, Medical Assessment Unit, Portsmouth Hospitals NHS Trust, C Level, Queen Alexandra Hospital, Cosham, Portsmouth, Hampshire PO6 3LY, UK; [email protected]
Competing interests None. Provenance and peer review Not commissioned; internally peer reviewed.
To cite Schmidt PE, Meredith P, Prytherch DR, et al. BMJ Qual Saf 2015;24:176–177. Accepted 5 December 2014
▸ http://dx.doi.org/10.1136/bmjqs-2014-003748 ▸ http://dx.doi.org/10.1136/bmjqs-2014-003804 BMJ Qual Saf 2015;24:176–177. doi:10.1136/bmjqs-2014-003845
REFERENCES 1 Shaw D, Blakey J, Rylance J. Measuring the true impact of an electronic physiological surveillance system (EPSS). BMJ Qual Saf 2015;24:175–6. 2 Van Schalkwyk J. Confounding explains ‘deaths avoided’. BMJ Qual Saf 2015; 24:175. 3 Schmidt PE, Meredith P, Prytherch DR, et al. Impact of introducing an electronic physiological surveillance system on hospital mortality. BMJ Qual Saf 2015;24:10–20. 4 Shojania KG. Conventional evaluations of improvement interventions: more trials or just more tribulations? BMJ Qual Saf 2013;22:881–4. 5 Hillman K, Chen J, May E. Complex intensive care unit interventions. Crit Care Med 2009;37:S102–6. 6 DeVita M, Bellomo R. The case of rapid response systems: are randomized clinical trials the right methodology to evaluate systems of care? Crit Care Med 2007;35:1413–14. 7 http://www.lstmliverpool.ac.uk/research/ departments/staff-profiles/john-blakey (accessed 28 Nov 2014). 8 http://nuhrise.org/2012/06/nuh-hn-teamwins-top-award-from-the-bmj/ (accessed 28 Nov 2014).
177
Downloaded from http://qualitysafety.bmj.com/ on July 12, 2015 - Published by group.bmj.com
Impact of introducing an electronic physiological surveillance system on hospital mortality Paul E Schmidt, Paul Meredith, David R Prytherch, Duncan Watson, Valerie Watson, Roger M Killen, Peter Greengross, Mohammed A Mohammed and Gary B Smith BMJ Qual Saf 2015 24: 176-177
doi: 10.1136/bmjqs-2014-003845 Updated information and services can be found at: http://qualitysafety.bmj.com/content/24/2/176
These include:
References Email alerting service
This article cites 3 articles, 1 of which you can access for free at: http://qualitysafety.bmj.com/content/24/2/176#BIBL Receive free email alerts when new articles cite this article. Sign up in the box at the top right corner of the online article.
Notes
To request permissions go to: http://group.bmj.com/group/rights-licensing/permissions To order reprints go to: http://journals.bmj.com/cgi/reprintform To subscribe to BMJ go to: http://group.bmj.com/subscribe/
Correspondence
Impact of introducing an electronic physiological surveillance system on hospital mortality Shaw et al1 are correct in pointing out that the use of a single year’s mortality data as a baseline comparator could introduce bias to our findings. They and Van Schalkwyk2 are also right to highlight the national reduction in hospital mortality rates over the past decade. However, we did not rely solely upon the overall mortality 176
reduction to come to the conclusion that there was an association between the timing of the introduction of an electronic physiological surveillance system (EPSS) and the reduced mortality at the two study hospitals.3 Both Shaw et al and Van Schalkwyk appear to ignore the significance of the main findings of our study. The reductions in annual observed deaths are concentrated in specific years (Portsmouth 2006 and 2009, years 2 and 5 after baseline; Coventry 2009, year 3 after baseline).3 Also, the monthly p-charts reveal very significant, abrupt and sustained changes in observed deaths in both hospitals within the space of a few months of the full rollout at each site. Finally, these reductions in mortality are associated with substantially decreased month-to-month variation in deaths. As we described in our paper,3 and as Van Schalkwyk observes, the signal is clearer at Coventry in 2009 because a prior statistically significant change in hospital deaths had already occurred at Portsmouth in 2006, coincident with the initial deployment of EPSS on the Acute Medicine Unit. This lends support to our view that the majority of avoided deaths over the 7 years occurred in the periods following the introduction of EPSS. However, we were careful to point out that not all avoided deaths in this period were necessarily the effect of EPSS. Van Schalkwyk uses long-run standardised (age-adjusted) mortality rates (SMR) to calculate a ‘secular adjustment factor’, which he then uses to adjust observed in-hospital deaths. However, population SMRs represent the risk at a particular age that a proportion of the population will die. They have been declining because of a huge range of factors including lifestyle and technological changes even if healthcare delivery improvements are excluded. In contrast, hospital deaths are drawn from a different sample of the population, namely those who, despite increased longevity, have reached a point where they need to be hospitalised for an BMJ Qual Saf February 2015 Vol 24 No 2
Downloaded from http://qualitysafety.bmj.com/ on July 12, 2015 - Published by group.bmj.com
Correspondence acute illness, usually then with more comorbidities. While Van Schalkwyk is correct to point out the potential confounding effect that varying the number of admissions coded with palliative care can have on hospital standardised mortality ratio (HSMR) calculation, it is important to point out that we used the observed deaths in the HSMR 80 data set, not HSMR. The only way a change in ‘palliative care’ deaths could have contributed to the change in observed deaths that we saw would have been for there to be a sudden change in practice in 2006 or 2009 such that a large proportion of patients requiring palliative care died at home or in a hospice rather than in a hospital. There is no evidence to support this. We agree with Shaw et al that well-intentioned interventions in healthcare occasionally produce unintended harm. Arguably for EPSS to have had an effect, earlier or more frequent escalation to junior doctors was anticipated. We have no evidence to support the view that EPSS led to either an inappropriate increase in workload or a reallocation of junior doctor time and priorities. The introduction of EPSS did not lead to a change in junior doctor staffing. Shaw et al’s view that a multicentre randomised controlled trial (RCT) would assist in identifying whether EPSS was the true reason for the fall in mortality fails to recognise the complexity, required duration and expense of implementing such a trial. In addition, RCTs may not
BMJ Qual Saf February 2015 Vol 24 No 2
necessarily be the best research method for complex interventions such as ours.4–6 Perhaps the best that can be hoped for is a much larger study of the type that we conducted, but comprising many hospitals using a de facto multicentre stepped wedge approach. Finally, two of the authors of the letter by Shaw et al1 appear to have undeclared conflicts of interest pertinent to their interest in our research.7 8 Paul E Schmidt,1 Paul Meredith,2 David R Prytherch,2,3 Duncan Watson,4 Valerie Watson,5 Roger M Killen,6 Peter Greengross,6,7 Mohammed A Mohammed,8 Gary B Smith9 1 Medical Assessment Unit, Portsmouth Hospitals NHS Trust, Portsmouth, Hampshire, UK 2 TEAMS Centre, Portsmouth Hospitals NHS Trust, Portsmouth, Hampshire, UK 3 School of Computing, University of Portsmouth, Portsmouth, Hampshire, UK 4 Intensive Care Medicine and Anaesthesia, University Hospitals, Coventry and Warwickshire NHS Trust, Coventry, UK 5 Critical Care Outreach, University Hospitals Coventry and Warwickshire NHS Trust, Coventry, UK 6 The Learning Clinic, London, UK 7 Department of Primary Care and Public Health, Imperial College Healthcare NHS Trust, London, UK 8 Quality & Effectiveness, School of Health Studies, University of Bradford, Bradford, UK 9 School of Health & Social Care, University of Bournemouth, Bournemouth, UK
Correspondence to Dr Paul Schmidt, Medical Assessment Unit, Portsmouth Hospitals NHS Trust, C Level, Queen Alexandra Hospital, Cosham, Portsmouth, Hampshire PO6 3LY, UK; [email protected]
Competing interests None. Provenance and peer review Not commissioned; internally peer reviewed.
To cite Schmidt PE, Meredith P, Prytherch DR, et al. BMJ Qual Saf 2015;24:176–177. Accepted 5 December 2014
▸ http://dx.doi.org/10.1136/bmjqs-2014-003748 ▸ http://dx.doi.org/10.1136/bmjqs-2014-003804 BMJ Qual Saf 2015;24:176–177. doi:10.1136/bmjqs-2014-003845
REFERENCES 1 Shaw D, Blakey J, Rylance J. Measuring the true impact of an electronic physiological surveillance system (EPSS). BMJ Qual Saf 2015;24:175–6. 2 Van Schalkwyk J. Confounding explains ‘deaths avoided’. BMJ Qual Saf 2015; 24:175. 3 Schmidt PE, Meredith P, Prytherch DR, et al. Impact of introducing an electronic physiological surveillance system on hospital mortality. BMJ Qual Saf 2015;24:10–20. 4 Shojania KG. Conventional evaluations of improvement interventions: more trials or just more tribulations? BMJ Qual Saf 2013;22:881–4. 5 Hillman K, Chen J, May E. Complex intensive care unit interventions. Crit Care Med 2009;37:S102–6. 6 DeVita M, Bellomo R. The case of rapid response systems: are randomized clinical trials the right methodology to evaluate systems of care? Crit Care Med 2007;35:1413–14. 7 http://www.lstmliverpool.ac.uk/research/ departments/staff-profiles/john-blakey (accessed 28 Nov 2014). 8 http://nuhrise.org/2012/06/nuh-hn-teamwins-top-award-from-the-bmj/ (accessed 28 Nov 2014).
177
Downloaded from http://qualitysafety.bmj.com/ on July 12, 2015 - Published by group.bmj.com
Impact of introducing an electronic physiological surveillance system on hospital mortality Paul E Schmidt, Paul Meredith, David R Prytherch, Duncan Watson, Valerie Watson, Roger M Killen, Peter Greengross, Mohammed A Mohammed and Gary B Smith BMJ Qual Saf 2015 24: 176-177
doi: 10.1136/bmjqs-2014-003845 Updated information and services can be found at: http://qualitysafety.bmj.com/content/24/2/176
These include:
References Email alerting service
This article cites 3 articles, 1 of which you can access for free at: http://qualitysafety.bmj.com/content/24/2/176#BIBL Receive free email alerts when new articles cite this article. Sign up in the box at the top right corner of the online article.
Notes
To request permissions go to: http://group.bmj.com/group/rights-licensing/permissions To order reprints go to: http://journals.bmj.com/cgi/reprintform To subscribe to BMJ go to: http://group.bmj.com/subscribe/
