Intern Emerg Med DOI 10.1007/s11739-015-1219-3
EM - ORIGINAL
Use of a risk nomogram to predict emergency department reattendance in older people after discharge: a validation study Glenn Arendts • Christopher Etherton-Beer • Roslyn Jones • Kate Bullow Ellen MacDonald • Sandra Dumas • Daniel Parker • Marani Hutton • Sally Burrows • Simon G. A. Brown • Osvaldo P. Almeida
•
Received: 31 December 2014 / Accepted: 15 February 2015 Ó SIMI 2015
Abstract In older people, revisit to the emergency department (ED) in the short period after discharge is not entirely avoidable, but in a proportion of cases is preventable, and should ideally be minimised. We have previously derived a risk probability nomogram to predict the likelihood of revisit. In this study, we sought to validate the nomogram for use as a general risk stratification tool for use in older people being discharged from ED. We conducted a prospective cohort study, applying the nomogram to consecutive community dwelling discharged patients aged 65 and over. Patients were followed up at 28 days post-discharge to determine whether there had been any unplanned ED revisit in that period. We cross tabulated predicted risk versus revisit rates. In 1143 study subjects, we find the odds of revisit increases progressively with increasing strata of predicted risk, culminating in an OR of G. Arendts C. Etherton-Beer S. G. A. Brown O. P. Almeida University of Western Australia, Crawley, Australia G. Arendts (&) E. MacDonald S. G. A. Brown Centre for Clinical Research in Emergency Medicine, Harry Perkins Institute, MRF Building, Royal Perth Hospital, Perth, Australia e-mail: [email protected] C. Etherton-Beer O. P. Almeida Western Australian Centre for Health and Ageing, Harry Perkins Institute, Perth, Australia R. Jones K. Bullow S. Dumas M. Hutton South Metropolitan Health Service, Perth, Australia D. Parker North Metropolitan Health Service, Perth, Australia S. Burrows Medical Research Foundation, Royal Perth Hospital, Perth, Australia
9.7 (95 % CI 4.7–19.9) in the highest risk group. The 28-day revisit rates across strata range from 16 % through 65 %, with the difference between strata being statistically highly significant (p \ 0.001). The area under the ROC curve is 0.65. We conclude that the risk nomogram can classify older people discharged from ED into risk strata, and has modest overall predictive value. Keywords Risk assessment Discharge planning Emergency department Reattendance Introduction In elderly patients, a ‘risk averse’ approach to discharge may result in prolonged admissions and exposure to welldocumented risks of hospitalisation [1] for no benefit. For this reason, as well as cost containment, clinicians may be inclined to discharge elderly patients from the Emergency Department (ED) wherever possible. However, about onefifth of adults aged 65 years or over who are discharged home from the ED revisit within a month [2]. Revisit sometimes reflects shortfalls in the initial ED episode of care: inadequate diagnosis or management of the acute problem, lack of recognition of geriatric syndromes contributing to the initial visit, or inappropriate transitional care arrangements after discharge [3]. However, in people with significant medical comorbidities, it is unrealistic to expect a zero rate of revisit after an ED episode of care, and at least half of early revisits of older patients appear unrelated to the first presentation [4]. ED staff might benefit from having an objective tool to quantify, at the point of intended discharge, the risk of revisit for elderly patients. This could either reinforce the clinical opinion of ED staff that revisit risk is low, or alert them that the risk may be higher than they believe.
123
Intern Emerg Med
We have previously reported data on a simple risk assessment nomogram that predicted revisit to the ED within 28 days of discharge [5]. Independent predictors included age, polypharmacy, cognitive impairment and depression. In this study, we aimed to prospectively validate the use of the nomogram for stratifying patients into different risk groups for revisit.
Methods Over a nine-month period commencing in November 2012, we conducted a prospective cohort study of older adults attending a single large urban ED with an annual census of 82,790 visits. We recruited men and women aged 65 years or over who had been medically assessed in the ED and designated for discharge back to the community. We excluded from participation people who were (1) living in a residential aged care facility, (2) unable to consent due to advanced cognitive impairment, or (3) receiving palliative care with a life expectancy of less than six months. Research nurses recruiting patients were not rostered on duty overnight, thus enrollments were restricted to patients presenting between 0700 and 2100 h, seven days a week. Our hospital Human Research Ethics Committee approved the study (RPH EC 2012/114). The nomogram is illustrated in Fig. 1. Risk factors are assigned the appropriate number of corresponding points displayed in the nomogram (top line). The points allocated for each risk factor are then summed to obtain total points for that patient, which is used to calculate the probability of having no unplanned revisit during the 28 days after discharge (bottom line). In the study and in clinical practice, we used an electronic spreadsheet that calculated the probability of revisit based on risk data entered into the spreadsheet, avoiding the need for the paper-based nomogram as represented in Fig. 1. The definitions of the nomogram components used to determine if they were present or absent are shown in Table 1. Research nurses trained for the study recruited patients and used patient interview in conjunction with the medical record and hospital electronic patient tracking systems to determine whether each of these factors were present. The predicted probability of revisit (1-probability of no reattendance) for each patient along with their demographic details was saved electronically. The primary outcome was any visit to an ED (not just the ED in which the study was conducted) within 28 days of discharge, excluding planned reviews. Where it was unclear if the revisit was planned on the electronic tracking system, the medical record for the patient was reviewed. The health movements of participants were monitored
123
using electronic data methods linking the individual’s unique identifier to any hospital-based episode of care, including ED visit [6]. Participants with more than one visit within 28 days were counted once only. An a priori sample size calculation conducted for the study found that for a binary response variable (revisit on a continuous, normally distributed variable (nomogramderived estimated risk,) a sample size of 2435 would have 80 % power at a 0.05 significance level to detect a change in probability of revisit being positive from 0.02 for the mean of risk to 0.03 when risk is increased to one standard deviation above the mean. This change corresponds to an odds ratio of 1.5. Because of possible incomplete assessments, and the likelihood that patients initially designated for possible discharge were subsequently admitted, we inflated the sample size by 1/3 and planned to conduct our analysis once a minimum of 3247 patients had been assessed. We used SPSS v21 (IBM, New York, USA) for data management and analysis. Descriptive statistics were employed to report the demographic and clinical characteristics of the sample, and the estimated risk of revisit, as determined by the nomogram, was cross tabulated against actual reattendance rates using Pearson’s v2 statistic, the Cochran Armitage test for trend and a binary logistic regression analysis. A receiver operating characteristic (ROC) curve was plotted to determine the area under the curve of revisit predicted in the study sample.
Results In the study hours, 9663 patients aged 65 and over visited the ED and of these, 3315 were flagged for possible discharge after initial medical assessment. Our study population comprised the 1143 patients from this group that had no exclusion criteria; consented to be involved in the study; had a completed screen; and were ultimately discharged. The study flow chart is shown in Fig. 2. The study sample was 55 % female with a median age of 79 years (IQR 72–85). Many patients had nonspecific and uncertain diagnoses at discharge such as chest pain for investigation, abdominal pain of unknown cause, dizziness or dyspnoea. The leading individual discrete discharge diagnoses were angina, fall with no injury, constipation, urinary tract infection, and respiratory tract infection. The predicted risk of revisit cross tabulated against revisit rates is shown in Table 2. It can be seen that each stratum of the nomogram is associated with an escalating rate of revisit (Pearson’s v2 = 81.8, p \ 0.001, Cochran Armitage p \ 0.001). Table 3 shows the results of a binary logistic regression model used to calculate the odds of revisit of each risk strata referent to the lowest risk group.
Intern Emerg Med 0
10
0
1
20
30
40
50
60
70
80
90
100
Points Prior number attendances 2
3
4
5
6
7
8
9
10
11
Age (years) Yes
65
70
75
80 90
105
Male No
Yes
Polypharmacy No
0
5
4
SIS cognition score 6
1
2
3
Yes
Malignancy No
Yes
CCT Intervention No
Yes
Depression Hx No
Total Points 0
20
40
60
80
0.99
0.98
100
120
140
160
180
200
220
240
260
280
Probability of no re-presentation 0.95
0.9
0.8 0.7
0.5
0.3
0.1
Fig. 1 Nomogram
Table 1 Nomogram components Component
Definition
Count of prior visits in last 12 months
Data from ED electronic information verified with patient interview exclude planned ED revisit
Age in years
Patient identified
Male gender
Patient identified
Polypharmacy
Data from medical record verified with patient interview. 6 or more prescribed or over-the-counter medications that are therapeutic and taken on a regular basis, excluding topical therapies
SIS (six item screener)
Patient interview. Score from 0 to 6 based on 3 min recall of 3 objects (1 point each) and orientation to year, month and day of week (1 point each)
Malignancy
Data from medical record verified with patient interview. A history in the past 10 years of malignant tumour requiring active surgical or medical treatment. Includes haematological malignancy, excludes benign tumours and non-melanoma skin cancer
Depression
Data from medical record verified with patient interview. A history of depression, patient self reporting of significant depressive symptoms (e.g. on Geriatric Depression Scale) or use of antidepressant medication for mood/anxiety problem
Allied health (care coordination team = CCT) intervention
Data from allied health record. Any intervention initiated by allied health on this visit from an intervention list
The ROC for the nomogram is shown in Fig. 3, with an area under the curve of 0.65.
Discussion In this study we have shown, using a simple nomogram, that it is possible to group patients into strata of revisit risk
through the use of a nomogram that weighs the factors associated with revisit attendance, and numerically projects the risk. This can be seen as a clinical advance on previous means by which ED staff identify those at risk of discharge, which are usually divided into clinical reasoning; the outsourcing of risk assessment to geriatric specialists within the ED; or the use of risk assessment tools that dichotomise patients into high or low risk.
123
Intern Emerg Med Fig. 2 Study flow
Table 2 Reattendance risk versus reattendance rates Nomogram predicted risk strata (%)
Number of patients
Number (%) with 28-day revisits
1
279
45 (16.1 %)
2
327
78 (23.9 %)
5
231
65 (28.1 %)
10
107
40 (37.4 %)
20
104
40 (38.5 %)
30
33
17 (51.5 %)
50
22
14 (63.6 %)
[50
40
26 (65.0 %)
1143
325 (28.4 %)
All
Table 3 Odds of revisit relative to lowest risk group
123
Pearson’s v2 = 81.8, p \ 0.001
Nomogram predicted risk strata (%)
Odds ratio (95 % CI)
p value
1
Referent
2
1.6 (1.1–2.4)
5
2.0 (1.3–3.1)
0.001
10
3.1 (1.9–5.1)
\0.001
0.019
20
3.3 (2.0–5.4)
\0.001
30 50
5.5 (2.6–11.7) 9.1 (3.6–23.0)
\0.001 \0.001
[50
9.7 (4.7–19.9)
\0.001
Intern Emerg Med
Fig. 3 ROC curve
Clinical reasoning is the bedrock of ED risk assessment, relying on the experience and expertise of ED staff to identify high-risk patients. However, ED staff must juggle multiple completing priorities and are rarely able to devote extended periods of time without interruption to a single process, therefore, having to focus on one high-priority problem at a time—usually the presenting problem in relative isolation. Our experience is that complex risk assessments require an hour or more of uninterrupted careful assessment, thus deficient ED assessments are unsurprising [7]. The ED environment presents a high risk of misdiagnosis and underestimation of functional impairment of patients [8–10]. The use of geriatric teams within ED, and the various compositions of these teams, is described by a number of authors [11–13]. Fundamentally, these teams are provided with the necessary time, specific training and quality control systems to be able to provide a high-quality risk assessment. Observational studies have found that such teams identify a substantial number of at-risk discharges by uncovering issues not recognised by ED physicians [14]. However, a problem with them is that additional resources are required to staff them, and they can only rarely be provided for the 24 h a day that an ED has to provide care. It is possible that structured tools could improve accuracy of assessment by ED staff under the time constraints imposed by the ED environment. A large number of risk assessment tools for this purpose have been developed, summarised in a recent study [15]. The most commonly used, the ISAR and TRST, dichotomise patients into high and low discharge risk based on composite outcomes postdischarge [16, 17].
Our approach is novel and improves on these methods as it quantitatively stratifies discharge risk, which provides additional new information and potentially allows clinicians to tailor post-discharge intervention to predicted risk. At present there is no certainty around what to do clinically with patients that the nomogram identifies as very high risk. Besides cautious reconsideration of the decision to discharge and recognising the high likelihood of ‘failed’ discharge, there is no clear evidence base to recommend practice change for this group. We are currently using the nomogram as a research tool, to select populations in the high-risk strata for a clinical trial (see ACTRN12612000798864 at http:// www.anzctr.org.au), in the same way other scoring systems are used to select patients for cancer or sepsis trials [18]. This offers a benefit over risk tools that dichotomise risk, which result in a high false-positive rate and potentially dilute the impact of high-intensity interventions in the most at-risk group [15]. Use of our nomogram, however, does pose potential problems. It requires busy clinicians to input multiple fields to obtain the risk projection, yet in keeping with all published tools that predict reattendance, the area under the ROC for our tool is modest and reaffirms that there is no single ‘‘cut-off,’’ but rather a continuum of risk. No comparable tool has an area under the curve better than the 0.65 we found with our nomogram. For example, Salvi et al. [19] found an AUC for ISAR and TRST of 0.63 and 0.61, respectively, in a study of over 2000 Italian patients. Factors that result in revisit are many and complex, rendering it extremely challenging to predict, and the development of all tools such as ours strikes a balance between parsimony versus comprehensiveness. [20]. The % risk predicted by the nomogram is based on using the entire discharged population as the denominator, hence the actual revisit rates (where the denominator is the number within that stratum of risk,) are substantially higher than the predicted risk. What the nomogram clearly demonstrates, however, is escalating risk with each increasing stratum. Nevertheless, it is possible that some systematic factor relevant to revisit is not being measured by the nomogram, despite the fact that over 40 variables were used in its derivation. We postulate that one such factor may be the sum of comorbid and frailty burden, since in the derivation study we analysed each illness independently. We plan future research to see if the use of indices to measure overall comorbidity burden will further improve the predictive value of the nomogram. The overall revisit rate in this study of 28 % is substantially higher than that quoted in other comparable research [2, 21]. This raises the possibility that the study sample here is unrepresentative of the broader discharged ED population. It is possible that because a reasonable amount of time is required to identify and screen
123
Intern Emerg Med
potentially eligible patients, those with very minor illness and injury were selected out of the study sample, as their time in ED was too brief to be enrolled. It is also possible that although medical staff were blinded to the nomogram results, discharge decision making behaviour changed because the ED staff knew the patients were being screened for the study. A number of limitations exist in this study. Most notably, the country from which it was derived and validated are the same, and factors peculiar to the ED and health systems in different countries may influence the nomogram performance. The nomogram only weights ‘risk’ factors not ‘protective’ factors such as primary care availability and access to subsidised medicines that will vary between different settings or for different individuals within the same setting. Although our results are highly significant, a much larger number of patients than expected were excluded from our study, especially a number of eligible patients had no or incomplete screen. Because the nomogram was only derived from discharged patients, it could not be applied to the large cohort that was admitted. Further work is required to determine if this nomogram is equally applicable to patients admitted to hospital from ED. Patients presenting overnight were also excluded, but this feature was in common with the derivation study. The strength of this study is that we have robustly validated the nomogram in a prospective trial in a large study population. The nomogram, derived from a population of over 1400 older people and prospectively validated in a separate population of over 1100, can now be considered ready for use in clinical practice as a risk stratification tool. Unlike other commonly used tools that have been derived using composite outcomes, our tool purely projects revisit risk—in practice it can provide a simple statement to the clinician along the lines of ‘‘This individual has an X % risk of revisit in the next 28 days. This is in the Y quartile of risk’’. However, it is important to recognise the limitations of the nomogram as a prediction instrument. In conclusion, our risk nomogram stratifies older people discharged from the ED into revisit risk groups, and has modest overall predictive value. Acknowledgments This research was funded by a grant from the State health Research Advisory Council of Western Australia. Conflict of interest Authors RJ, KB, SD and MH are employees of the Western Australian State Government that funded the research through the State Health Research Advisory Council. All other authors declare they have no conflicts of interest. Statement of human and animal rights All procedures performed in studies involving human participants were in accordance with the ethical standards of the institutional and/or national research committee and with the 1964 Helsinki declaration and its later amendments or comparable ethical standards.
123
Informed consent Informed consent was obtained from all individual participants included in the study.
References 1. Covinsky KE, Palmer RM, Fortinsky RH, Counsell SR, Stewart AL, Kresevic D, Burant CJ, Landefeld CS (2003) Loss of independence in activities of daily living in older adults hospitalized with medical illnesses: increased vulnerability with age. J Am Geriatr Soc 51(4):451–458 2. McCusker J, Roberge D, Vadeboncoeur A, Verdon J (2009) Safety of discharge of seniors from the emergency department to the community. Healthc Q 12 Spec No Patient:24–32 3. Arendts G, Lowthian J (2013) Demography is destiny: an agenda for geriatric emergency medicine in Australasia. Emerg Med Australas 25(3):271–278 4. Arendts G (2010) Predicting outcome following allied health intervention and discharge of the elderly patient from the emergency department. In: 2010 International Conference on Emergency Medicine (ICEM) Selected Abstracts. Acad Emerg Med 17(11):1258–1262 5. Arendts G, Fitzhardinge S, Pronk K, Hutton M, Nagree Y, Donaldson M (2013) Derivation of a nomogram to estimate probability of revisit in at-risk older adults discharged from the emergency department. Intern Emerg Med 8(3):249–254 6. Holman CDAJ, Bass AJ, Rosman DL, Smith MB, Semmens JB, Glasson EJ, Brook EL, Trutwein B, Rouse IL, Watson CR et al (2008) A decade of data linkage in Western Australia: strategic design, applications and benefits of the WA data linkage system. Aust Health Rev 32(4):766–777 7. Salvi F, Morichi V, Grilli A, Giorgi R, De Tommaso G, DessiFulgheri P (2007) The elderly in the emergency department: a critical review of problems and solutions. Intern Emerg Med 2(4):292–301 8. Khan SA, Miskelly FG, Platt JS, Bhattachryya BK (1996) Missed diagnoses among elderly patients discharged from an accident and emergency department. J Accid Emerg Med 13(4):256–257 9. Rodriguez-Molinero A, Lopez-Dieguez M, Tabuenca AI, de la Cruz JJ, Banegas JR (2006) Functional assessment of older patients in the emergency department: comparison between standard instruments, medical records and physicians’ perceptions. BMC Geriatr 6:13 10. Rodriguez-Molinero A, Lopez-Dieguez M, Tabuenca AI, de la Cruz JJ, Banegas JR (2010) Physicians’ impression on the elders’ functionality influences decision making for emergency care. Am J Emerg Med 28(7):757–765 11. Brazil K, Bolton C, Ulrichsen D, Knott C (1998) Substituting home care for hospitalization: the role of a quick response service for the elderly. J Community Health 23(1):29–43 12. Arendts G, Fitzhardinghe S, Pronk K, Donaldson M, Hutton M, Nagree Y (2012) The impact of early emergency department allied health intervention on admission rates in older people: a non-randomized clinical study. BMC Geriatr 12:8 13. Moss JE, Flower CL, Houghton LM, Moss DL, Nielsen DA, Taylor DM (2002) A multidisciplinary Care Coordination Team improves emergency department discharge planning practice. Med J Aust 177(8):435–439 14. Ngian VJJ, Ong BS, O’Rourke F, Nguyen HV, Chan DKY (2008) Review of a rapid geriatric medical assessment model based in emergency department. Age Ageing 37(6):696–699 15. Moons P, De Ridder K, Geyskens K, Sabbe M, Braes T, Flamaing J, Milisen K (2007) Screening for risk of readmission of patients aged 65 years and above after discharge from the emergency
Intern Emerg Med department: predictive value of four instruments. Eur J Emerg Med 14(6):315–323 16. McCusker J, Bellavance F, Cardin S, Trepanier S, Verdon J, Ardman O (1999) Detection of older people at increased risk of adverse health outcomes after an emergency visit: the ISAR screening tool. J Am Geriatr Soc 47(10):1229–1237 17. Meldon SW, Mion LC, Palmer RM, Drew BL, Connor JT, Lewicki LJ, Bass DM, Emerman CL (2003) A brief risk-stratification tool to predict repeat emergency department visits and hospitalizations in older patients discharged from the emergency department. Acad Emerg Med 10(3):224–232 18. Marshall JC (2014) The PIRO (predisposition, insult, response, organ dysfunction) model: toward a staging system for acute illness. Virulence 5(1):27–35
19. Salvi F, Morichi V, Lorenzetti B, Rossi L, Spazzafumo L, Luzi R, De Tommaso G, Lattanzio F (2012) Risk stratification of older patients in the emergency department: comparison between the Identification of Seniors at Risk and Triage Risk Screening Tool. Rejuvenation Res 15(3):288–294 20. LaMantia MA, Platts-Mills TF, Biese K, Khandelwal C, Forbach C, Cairns CB, Busby-Whitehead J, Kizer JS (2010) Predicting hospital admission and returns to the emergency department for elderly patients. Acad Emerg Med 17(3):252–259 21. Arendts G, Fitzhardinge S, Pronk K, Hutton M (2013) Outcomes in older patients requiring comprehensive allied health care prior to discharge from the emergency department. Emerg Med Australas 25(2):127–131
123
EM - ORIGINAL
Use of a risk nomogram to predict emergency department reattendance in older people after discharge: a validation study Glenn Arendts • Christopher Etherton-Beer • Roslyn Jones • Kate Bullow Ellen MacDonald • Sandra Dumas • Daniel Parker • Marani Hutton • Sally Burrows • Simon G. A. Brown • Osvaldo P. Almeida
•
Received: 31 December 2014 / Accepted: 15 February 2015 Ó SIMI 2015
Abstract In older people, revisit to the emergency department (ED) in the short period after discharge is not entirely avoidable, but in a proportion of cases is preventable, and should ideally be minimised. We have previously derived a risk probability nomogram to predict the likelihood of revisit. In this study, we sought to validate the nomogram for use as a general risk stratification tool for use in older people being discharged from ED. We conducted a prospective cohort study, applying the nomogram to consecutive community dwelling discharged patients aged 65 and over. Patients were followed up at 28 days post-discharge to determine whether there had been any unplanned ED revisit in that period. We cross tabulated predicted risk versus revisit rates. In 1143 study subjects, we find the odds of revisit increases progressively with increasing strata of predicted risk, culminating in an OR of G. Arendts C. Etherton-Beer S. G. A. Brown O. P. Almeida University of Western Australia, Crawley, Australia G. Arendts (&) E. MacDonald S. G. A. Brown Centre for Clinical Research in Emergency Medicine, Harry Perkins Institute, MRF Building, Royal Perth Hospital, Perth, Australia e-mail: [email protected] C. Etherton-Beer O. P. Almeida Western Australian Centre for Health and Ageing, Harry Perkins Institute, Perth, Australia R. Jones K. Bullow S. Dumas M. Hutton South Metropolitan Health Service, Perth, Australia D. Parker North Metropolitan Health Service, Perth, Australia S. Burrows Medical Research Foundation, Royal Perth Hospital, Perth, Australia
9.7 (95 % CI 4.7–19.9) in the highest risk group. The 28-day revisit rates across strata range from 16 % through 65 %, with the difference between strata being statistically highly significant (p \ 0.001). The area under the ROC curve is 0.65. We conclude that the risk nomogram can classify older people discharged from ED into risk strata, and has modest overall predictive value. Keywords Risk assessment Discharge planning Emergency department Reattendance Introduction In elderly patients, a ‘risk averse’ approach to discharge may result in prolonged admissions and exposure to welldocumented risks of hospitalisation [1] for no benefit. For this reason, as well as cost containment, clinicians may be inclined to discharge elderly patients from the Emergency Department (ED) wherever possible. However, about onefifth of adults aged 65 years or over who are discharged home from the ED revisit within a month [2]. Revisit sometimes reflects shortfalls in the initial ED episode of care: inadequate diagnosis or management of the acute problem, lack of recognition of geriatric syndromes contributing to the initial visit, or inappropriate transitional care arrangements after discharge [3]. However, in people with significant medical comorbidities, it is unrealistic to expect a zero rate of revisit after an ED episode of care, and at least half of early revisits of older patients appear unrelated to the first presentation [4]. ED staff might benefit from having an objective tool to quantify, at the point of intended discharge, the risk of revisit for elderly patients. This could either reinforce the clinical opinion of ED staff that revisit risk is low, or alert them that the risk may be higher than they believe.
123
Intern Emerg Med
We have previously reported data on a simple risk assessment nomogram that predicted revisit to the ED within 28 days of discharge [5]. Independent predictors included age, polypharmacy, cognitive impairment and depression. In this study, we aimed to prospectively validate the use of the nomogram for stratifying patients into different risk groups for revisit.
Methods Over a nine-month period commencing in November 2012, we conducted a prospective cohort study of older adults attending a single large urban ED with an annual census of 82,790 visits. We recruited men and women aged 65 years or over who had been medically assessed in the ED and designated for discharge back to the community. We excluded from participation people who were (1) living in a residential aged care facility, (2) unable to consent due to advanced cognitive impairment, or (3) receiving palliative care with a life expectancy of less than six months. Research nurses recruiting patients were not rostered on duty overnight, thus enrollments were restricted to patients presenting between 0700 and 2100 h, seven days a week. Our hospital Human Research Ethics Committee approved the study (RPH EC 2012/114). The nomogram is illustrated in Fig. 1. Risk factors are assigned the appropriate number of corresponding points displayed in the nomogram (top line). The points allocated for each risk factor are then summed to obtain total points for that patient, which is used to calculate the probability of having no unplanned revisit during the 28 days after discharge (bottom line). In the study and in clinical practice, we used an electronic spreadsheet that calculated the probability of revisit based on risk data entered into the spreadsheet, avoiding the need for the paper-based nomogram as represented in Fig. 1. The definitions of the nomogram components used to determine if they were present or absent are shown in Table 1. Research nurses trained for the study recruited patients and used patient interview in conjunction with the medical record and hospital electronic patient tracking systems to determine whether each of these factors were present. The predicted probability of revisit (1-probability of no reattendance) for each patient along with their demographic details was saved electronically. The primary outcome was any visit to an ED (not just the ED in which the study was conducted) within 28 days of discharge, excluding planned reviews. Where it was unclear if the revisit was planned on the electronic tracking system, the medical record for the patient was reviewed. The health movements of participants were monitored
123
using electronic data methods linking the individual’s unique identifier to any hospital-based episode of care, including ED visit [6]. Participants with more than one visit within 28 days were counted once only. An a priori sample size calculation conducted for the study found that for a binary response variable (revisit on a continuous, normally distributed variable (nomogramderived estimated risk,) a sample size of 2435 would have 80 % power at a 0.05 significance level to detect a change in probability of revisit being positive from 0.02 for the mean of risk to 0.03 when risk is increased to one standard deviation above the mean. This change corresponds to an odds ratio of 1.5. Because of possible incomplete assessments, and the likelihood that patients initially designated for possible discharge were subsequently admitted, we inflated the sample size by 1/3 and planned to conduct our analysis once a minimum of 3247 patients had been assessed. We used SPSS v21 (IBM, New York, USA) for data management and analysis. Descriptive statistics were employed to report the demographic and clinical characteristics of the sample, and the estimated risk of revisit, as determined by the nomogram, was cross tabulated against actual reattendance rates using Pearson’s v2 statistic, the Cochran Armitage test for trend and a binary logistic regression analysis. A receiver operating characteristic (ROC) curve was plotted to determine the area under the curve of revisit predicted in the study sample.
Results In the study hours, 9663 patients aged 65 and over visited the ED and of these, 3315 were flagged for possible discharge after initial medical assessment. Our study population comprised the 1143 patients from this group that had no exclusion criteria; consented to be involved in the study; had a completed screen; and were ultimately discharged. The study flow chart is shown in Fig. 2. The study sample was 55 % female with a median age of 79 years (IQR 72–85). Many patients had nonspecific and uncertain diagnoses at discharge such as chest pain for investigation, abdominal pain of unknown cause, dizziness or dyspnoea. The leading individual discrete discharge diagnoses were angina, fall with no injury, constipation, urinary tract infection, and respiratory tract infection. The predicted risk of revisit cross tabulated against revisit rates is shown in Table 2. It can be seen that each stratum of the nomogram is associated with an escalating rate of revisit (Pearson’s v2 = 81.8, p \ 0.001, Cochran Armitage p \ 0.001). Table 3 shows the results of a binary logistic regression model used to calculate the odds of revisit of each risk strata referent to the lowest risk group.
Intern Emerg Med 0
10
0
1
20
30
40
50
60
70
80
90
100
Points Prior number attendances 2
3
4
5
6
7
8
9
10
11
Age (years) Yes
65
70
75
80 90
105
Male No
Yes
Polypharmacy No
0
5
4
SIS cognition score 6
1
2
3
Yes
Malignancy No
Yes
CCT Intervention No
Yes
Depression Hx No
Total Points 0
20
40
60
80
0.99
0.98
100
120
140
160
180
200
220
240
260
280
Probability of no re-presentation 0.95
0.9
0.8 0.7
0.5
0.3
0.1
Fig. 1 Nomogram
Table 1 Nomogram components Component
Definition
Count of prior visits in last 12 months
Data from ED electronic information verified with patient interview exclude planned ED revisit
Age in years
Patient identified
Male gender
Patient identified
Polypharmacy
Data from medical record verified with patient interview. 6 or more prescribed or over-the-counter medications that are therapeutic and taken on a regular basis, excluding topical therapies
SIS (six item screener)
Patient interview. Score from 0 to 6 based on 3 min recall of 3 objects (1 point each) and orientation to year, month and day of week (1 point each)
Malignancy
Data from medical record verified with patient interview. A history in the past 10 years of malignant tumour requiring active surgical or medical treatment. Includes haematological malignancy, excludes benign tumours and non-melanoma skin cancer
Depression
Data from medical record verified with patient interview. A history of depression, patient self reporting of significant depressive symptoms (e.g. on Geriatric Depression Scale) or use of antidepressant medication for mood/anxiety problem
Allied health (care coordination team = CCT) intervention
Data from allied health record. Any intervention initiated by allied health on this visit from an intervention list
The ROC for the nomogram is shown in Fig. 3, with an area under the curve of 0.65.
Discussion In this study we have shown, using a simple nomogram, that it is possible to group patients into strata of revisit risk
through the use of a nomogram that weighs the factors associated with revisit attendance, and numerically projects the risk. This can be seen as a clinical advance on previous means by which ED staff identify those at risk of discharge, which are usually divided into clinical reasoning; the outsourcing of risk assessment to geriatric specialists within the ED; or the use of risk assessment tools that dichotomise patients into high or low risk.
123
Intern Emerg Med Fig. 2 Study flow
Table 2 Reattendance risk versus reattendance rates Nomogram predicted risk strata (%)
Number of patients
Number (%) with 28-day revisits
1
279
45 (16.1 %)
2
327
78 (23.9 %)
5
231
65 (28.1 %)
10
107
40 (37.4 %)
20
104
40 (38.5 %)
30
33
17 (51.5 %)
50
22
14 (63.6 %)
[50
40
26 (65.0 %)
1143
325 (28.4 %)
All
Table 3 Odds of revisit relative to lowest risk group
123
Pearson’s v2 = 81.8, p \ 0.001
Nomogram predicted risk strata (%)
Odds ratio (95 % CI)
p value
1
Referent
2
1.6 (1.1–2.4)
5
2.0 (1.3–3.1)
0.001
10
3.1 (1.9–5.1)
\0.001
0.019
20
3.3 (2.0–5.4)
\0.001
30 50
5.5 (2.6–11.7) 9.1 (3.6–23.0)
\0.001 \0.001
[50
9.7 (4.7–19.9)
\0.001
Intern Emerg Med
Fig. 3 ROC curve
Clinical reasoning is the bedrock of ED risk assessment, relying on the experience and expertise of ED staff to identify high-risk patients. However, ED staff must juggle multiple completing priorities and are rarely able to devote extended periods of time without interruption to a single process, therefore, having to focus on one high-priority problem at a time—usually the presenting problem in relative isolation. Our experience is that complex risk assessments require an hour or more of uninterrupted careful assessment, thus deficient ED assessments are unsurprising [7]. The ED environment presents a high risk of misdiagnosis and underestimation of functional impairment of patients [8–10]. The use of geriatric teams within ED, and the various compositions of these teams, is described by a number of authors [11–13]. Fundamentally, these teams are provided with the necessary time, specific training and quality control systems to be able to provide a high-quality risk assessment. Observational studies have found that such teams identify a substantial number of at-risk discharges by uncovering issues not recognised by ED physicians [14]. However, a problem with them is that additional resources are required to staff them, and they can only rarely be provided for the 24 h a day that an ED has to provide care. It is possible that structured tools could improve accuracy of assessment by ED staff under the time constraints imposed by the ED environment. A large number of risk assessment tools for this purpose have been developed, summarised in a recent study [15]. The most commonly used, the ISAR and TRST, dichotomise patients into high and low discharge risk based on composite outcomes postdischarge [16, 17].
Our approach is novel and improves on these methods as it quantitatively stratifies discharge risk, which provides additional new information and potentially allows clinicians to tailor post-discharge intervention to predicted risk. At present there is no certainty around what to do clinically with patients that the nomogram identifies as very high risk. Besides cautious reconsideration of the decision to discharge and recognising the high likelihood of ‘failed’ discharge, there is no clear evidence base to recommend practice change for this group. We are currently using the nomogram as a research tool, to select populations in the high-risk strata for a clinical trial (see ACTRN12612000798864 at http:// www.anzctr.org.au), in the same way other scoring systems are used to select patients for cancer or sepsis trials [18]. This offers a benefit over risk tools that dichotomise risk, which result in a high false-positive rate and potentially dilute the impact of high-intensity interventions in the most at-risk group [15]. Use of our nomogram, however, does pose potential problems. It requires busy clinicians to input multiple fields to obtain the risk projection, yet in keeping with all published tools that predict reattendance, the area under the ROC for our tool is modest and reaffirms that there is no single ‘‘cut-off,’’ but rather a continuum of risk. No comparable tool has an area under the curve better than the 0.65 we found with our nomogram. For example, Salvi et al. [19] found an AUC for ISAR and TRST of 0.63 and 0.61, respectively, in a study of over 2000 Italian patients. Factors that result in revisit are many and complex, rendering it extremely challenging to predict, and the development of all tools such as ours strikes a balance between parsimony versus comprehensiveness. [20]. The % risk predicted by the nomogram is based on using the entire discharged population as the denominator, hence the actual revisit rates (where the denominator is the number within that stratum of risk,) are substantially higher than the predicted risk. What the nomogram clearly demonstrates, however, is escalating risk with each increasing stratum. Nevertheless, it is possible that some systematic factor relevant to revisit is not being measured by the nomogram, despite the fact that over 40 variables were used in its derivation. We postulate that one such factor may be the sum of comorbid and frailty burden, since in the derivation study we analysed each illness independently. We plan future research to see if the use of indices to measure overall comorbidity burden will further improve the predictive value of the nomogram. The overall revisit rate in this study of 28 % is substantially higher than that quoted in other comparable research [2, 21]. This raises the possibility that the study sample here is unrepresentative of the broader discharged ED population. It is possible that because a reasonable amount of time is required to identify and screen
123
Intern Emerg Med
potentially eligible patients, those with very minor illness and injury were selected out of the study sample, as their time in ED was too brief to be enrolled. It is also possible that although medical staff were blinded to the nomogram results, discharge decision making behaviour changed because the ED staff knew the patients were being screened for the study. A number of limitations exist in this study. Most notably, the country from which it was derived and validated are the same, and factors peculiar to the ED and health systems in different countries may influence the nomogram performance. The nomogram only weights ‘risk’ factors not ‘protective’ factors such as primary care availability and access to subsidised medicines that will vary between different settings or for different individuals within the same setting. Although our results are highly significant, a much larger number of patients than expected were excluded from our study, especially a number of eligible patients had no or incomplete screen. Because the nomogram was only derived from discharged patients, it could not be applied to the large cohort that was admitted. Further work is required to determine if this nomogram is equally applicable to patients admitted to hospital from ED. Patients presenting overnight were also excluded, but this feature was in common with the derivation study. The strength of this study is that we have robustly validated the nomogram in a prospective trial in a large study population. The nomogram, derived from a population of over 1400 older people and prospectively validated in a separate population of over 1100, can now be considered ready for use in clinical practice as a risk stratification tool. Unlike other commonly used tools that have been derived using composite outcomes, our tool purely projects revisit risk—in practice it can provide a simple statement to the clinician along the lines of ‘‘This individual has an X % risk of revisit in the next 28 days. This is in the Y quartile of risk’’. However, it is important to recognise the limitations of the nomogram as a prediction instrument. In conclusion, our risk nomogram stratifies older people discharged from the ED into revisit risk groups, and has modest overall predictive value. Acknowledgments This research was funded by a grant from the State health Research Advisory Council of Western Australia. Conflict of interest Authors RJ, KB, SD and MH are employees of the Western Australian State Government that funded the research through the State Health Research Advisory Council. All other authors declare they have no conflicts of interest. Statement of human and animal rights All procedures performed in studies involving human participants were in accordance with the ethical standards of the institutional and/or national research committee and with the 1964 Helsinki declaration and its later amendments or comparable ethical standards.
123
Informed consent Informed consent was obtained from all individual participants included in the study.
References 1. Covinsky KE, Palmer RM, Fortinsky RH, Counsell SR, Stewart AL, Kresevic D, Burant CJ, Landefeld CS (2003) Loss of independence in activities of daily living in older adults hospitalized with medical illnesses: increased vulnerability with age. J Am Geriatr Soc 51(4):451–458 2. McCusker J, Roberge D, Vadeboncoeur A, Verdon J (2009) Safety of discharge of seniors from the emergency department to the community. Healthc Q 12 Spec No Patient:24–32 3. Arendts G, Lowthian J (2013) Demography is destiny: an agenda for geriatric emergency medicine in Australasia. Emerg Med Australas 25(3):271–278 4. Arendts G (2010) Predicting outcome following allied health intervention and discharge of the elderly patient from the emergency department. In: 2010 International Conference on Emergency Medicine (ICEM) Selected Abstracts. Acad Emerg Med 17(11):1258–1262 5. Arendts G, Fitzhardinge S, Pronk K, Hutton M, Nagree Y, Donaldson M (2013) Derivation of a nomogram to estimate probability of revisit in at-risk older adults discharged from the emergency department. Intern Emerg Med 8(3):249–254 6. Holman CDAJ, Bass AJ, Rosman DL, Smith MB, Semmens JB, Glasson EJ, Brook EL, Trutwein B, Rouse IL, Watson CR et al (2008) A decade of data linkage in Western Australia: strategic design, applications and benefits of the WA data linkage system. Aust Health Rev 32(4):766–777 7. Salvi F, Morichi V, Grilli A, Giorgi R, De Tommaso G, DessiFulgheri P (2007) The elderly in the emergency department: a critical review of problems and solutions. Intern Emerg Med 2(4):292–301 8. Khan SA, Miskelly FG, Platt JS, Bhattachryya BK (1996) Missed diagnoses among elderly patients discharged from an accident and emergency department. J Accid Emerg Med 13(4):256–257 9. Rodriguez-Molinero A, Lopez-Dieguez M, Tabuenca AI, de la Cruz JJ, Banegas JR (2006) Functional assessment of older patients in the emergency department: comparison between standard instruments, medical records and physicians’ perceptions. BMC Geriatr 6:13 10. Rodriguez-Molinero A, Lopez-Dieguez M, Tabuenca AI, de la Cruz JJ, Banegas JR (2010) Physicians’ impression on the elders’ functionality influences decision making for emergency care. Am J Emerg Med 28(7):757–765 11. Brazil K, Bolton C, Ulrichsen D, Knott C (1998) Substituting home care for hospitalization: the role of a quick response service for the elderly. J Community Health 23(1):29–43 12. Arendts G, Fitzhardinghe S, Pronk K, Donaldson M, Hutton M, Nagree Y (2012) The impact of early emergency department allied health intervention on admission rates in older people: a non-randomized clinical study. BMC Geriatr 12:8 13. Moss JE, Flower CL, Houghton LM, Moss DL, Nielsen DA, Taylor DM (2002) A multidisciplinary Care Coordination Team improves emergency department discharge planning practice. Med J Aust 177(8):435–439 14. Ngian VJJ, Ong BS, O’Rourke F, Nguyen HV, Chan DKY (2008) Review of a rapid geriatric medical assessment model based in emergency department. Age Ageing 37(6):696–699 15. Moons P, De Ridder K, Geyskens K, Sabbe M, Braes T, Flamaing J, Milisen K (2007) Screening for risk of readmission of patients aged 65 years and above after discharge from the emergency
Intern Emerg Med department: predictive value of four instruments. Eur J Emerg Med 14(6):315–323 16. McCusker J, Bellavance F, Cardin S, Trepanier S, Verdon J, Ardman O (1999) Detection of older people at increased risk of adverse health outcomes after an emergency visit: the ISAR screening tool. J Am Geriatr Soc 47(10):1229–1237 17. Meldon SW, Mion LC, Palmer RM, Drew BL, Connor JT, Lewicki LJ, Bass DM, Emerman CL (2003) A brief risk-stratification tool to predict repeat emergency department visits and hospitalizations in older patients discharged from the emergency department. Acad Emerg Med 10(3):224–232 18. Marshall JC (2014) The PIRO (predisposition, insult, response, organ dysfunction) model: toward a staging system for acute illness. Virulence 5(1):27–35
19. Salvi F, Morichi V, Lorenzetti B, Rossi L, Spazzafumo L, Luzi R, De Tommaso G, Lattanzio F (2012) Risk stratification of older patients in the emergency department: comparison between the Identification of Seniors at Risk and Triage Risk Screening Tool. Rejuvenation Res 15(3):288–294 20. LaMantia MA, Platts-Mills TF, Biese K, Khandelwal C, Forbach C, Cairns CB, Busby-Whitehead J, Kizer JS (2010) Predicting hospital admission and returns to the emergency department for elderly patients. Acad Emerg Med 17(3):252–259 21. Arendts G, Fitzhardinge S, Pronk K, Hutton M (2013) Outcomes in older patients requiring comprehensive allied health care prior to discharge from the emergency department. Emerg Med Australas 25(2):127–131
123
