LWW/JNCQ
JNCQ-D-13-00084
July 28, 2014
15:54
J Nurs Care Qual Vol. 29, No. 4, pp. 336–344 c 2014 Wolters Kluwer Health | Lippincott Williams & Wilkins Copyright
Bias Reduction in Calculation of Inpatient Fall Rates Kera F. Weiserbs, MHS, PhD; Barry J. Hahn, MD, FACEP Inpatient falls are the most common adverse hospital events. Despite the recognized importance of reducing inpatient falls, tracking and reporting methods are inconsistent. Moreover, recommended methods and statistical tests for comparing rates are complicated. This article demonstrates how to calculate fall rates using 3 common methods, summarizes the advantages and disadvantages of each method, and recommends best practices. Key words: inpatient falls, biases, health care evaluation mechanisms, nursing evaluation research, statistics and numerical data, quality improvement
R
EDUCING INPATIENT FALLS is important: they are the most common, preventable adverse hospital event, increasing morbidity1 and cost.2 Fortunately, inpatient falls are recognized as an important performance measurement by organizations dedicated to improving patient safety, including The Joint Commission,3 the National Quality Forum,4 and the Centers for Medicare & Medicaid Services.5 However, despite the recognized importance of reducing inpatient falls, tracking and reporting proven quality improvement methods for root-cause assessment and program assessment6,7 are applied inconsistently to inpatient fall-prevention programs. This problem is compounded by the
Author Affiliations: Former, Department of Academic Affairs (Dr Weiserbs) and Emergency Medicine (Dr Hahn), Staten Island University Hospital, Staten Island, New York. Dr Weiserbs is now a freelance consultant. The authors declare no conflict of interest. Supplemental digital content is available for this article. Direct URL citations appear in the printed text and are provided in the HTML and PDF versions of this article on the journal’s Web site (www.jncqjournal.com). Correspondence: Kera F. Weiserbs, MHS, PhD, Statistical Consultant 445 Argyle Rd, Brooklyn, NY 11218 ([email protected]). Accepted for Publication: March 2, 2014 Published online before print: April 15, 2014 DOI: 10.1097/NCQ.0000000000000063
lack of standardized methods that limit intraand inter-institutional comparisons due to the different biases (deviations from the actual events) associated with each method.8 A summary of these biases and the advantages associated with each method is shown in Supplemental Digital Content, Table 1 (available at: http://links.lww.com/JNCQ/A84). This article has 3 goals. The first goal is to demonstrate how to calculate inpatient fall rates using 3 methods and to summarize the advantages and disadvantages of each method. These methods include new admissions; the midnight census (the number of hospital beds occupied at midnight); and the person-time method (the sum of each inpatient’s time at risk for a fall; eg, the patient’s length of stay). The second goal is to present relative improvement rates (RIRs), a statistic for comparing rates. The third goal is to recommend best practices for the calculation and comparison of inpatient fall rates. METHODS Data sources This report uses data from a large tertiary care teaching hospital in New York City. The data set compares fall rates measures, pre- and post-implementation of a fallprevention initiative (FPI) that began in March 2006. This intervention and the associated performance improvements are described in
336
Copyright © 2014 Lippincott Williams & Wilkins. Unauthorized reproduction of this article is prohibited.
LWW/JNCQ
JNCQ-D-13-00084
July 28, 2014
15:54
Bias Reduction in Calculation of Inpatient Fall Rates another article.9 In addition to the FPI data, sample cases are used to illustrate how each statistic is calculated. Measures of population at risk The most common methods for assessing fall risk include counts and fall rates (the incidence of falls during an observation interval).8,10,11 Counts of inpatient falls are easy to quantify, provide numerator data for fall rates, and help determine resource allocation for fall prevention. However, counts do not estimate program efficacy, or adjust for hospital size or inpatient’s length of stay. Inpatient fall rates (falls in an observation interval per population at risk for falling in an observation interval) are a much better measure of program efficacy than counts, because rates adjust for hospital size and patient’s length of stay. This rate measures incidence, the frequency of new events (eg, falls) in a population (eg, inpatients at a specific hospital) during a given observation interval (eg, July) (new cases of disease or new events in an observation interval per population at risk for a disease or an event in an observational interval).12 Incidence rates use standardized denominators, for example, inpatient falls per 1000 inpatient-days or colon cases per 100 000 person-years, to facilitate discussing rare disease (or event) rates. For example, although 0.38 falls per inpatientday and 380 falls per 1000 inpatient-days are equivalent mathematically, a fall rate of 0.38 falls per day is not meaningful; a person cannot have a 0.38 falls, because falls are dichotomous events (either occur or do not occur). In contrast, a rate of 380 falls per 1000 days (380 falls in 1000 days) is meaningful. Standardized denominators are calculated by multiplying the numerator and denominator by a power of 10, such as 1 000 000 1000 10 000 100 000 , , , or . 1000 10 000 100 000 1 000 000 Unit choice depends on the rarity of the event or historical precedence. Inpatient falls are commonly reported per 1000 inpatientdays (abbreviated as 1000 days).
337
New admissions and midnight census New admissions and the midnight census are the most common measures of inpatient fall risk. Although these statistics are readily available, they are not the best measures of inpatient fall risk. These statistics are collected for other purposes. New admissions measure inpatient flow and the midnight census measures service needs, which are required for Medicare Part A billing. New admissions adjust for differences in hospital size (new admissions are a function of hospital size), but do not access inpatient’s length of stay, a risk factor for inpatient falls.13,14 The midnight census estimates hospital size and approximates inpatient length of stay (days at risk for a fall); however, the midnight census truncates length of stay to the number of nights that a patient stayed in a hospital and therefore underestimates length of stay. The National Database of Nursing Quality Indicators (NDNQI) recognizes the midnight census as an adequate, albeit a suboptimal, estimate of length of stay, because it underestimates length of stay.15 Person-time The NDNQI recommends person-time as the most accurate length of stay estimate.15 The person-time approach, although uncommon in the nursing literature, is the criterion standard for evaluating a rare event (eg, inpatient fall rates). Limited methodological documentation in the quality improvement literature, and limited statistical training in nursing programs, has curtailed mainstream use of this approach. This section summarizes the assumptions used in the person-time approach and presents 2 methods for calculating person-time: a simpler method, which overestimates length of stay, and a more accurate, albeit more complicated, method. Person-time measures length of stay. It is the sum of all the times that everyone in the sample or population is at risk for an event. For example, if 3 people are admitted to a hospital for 2, 3, and 10 days, respectively, their person-time at risk for a fall equals 15 days.8,10 This method assumes (1) constant
Copyright © 2014 Lippincott Williams & Wilkins. Unauthorized reproduction of this article is prohibited.
LWW/JNCQ
338
JNCQ-D-13-00084
July 28, 2014
15:54
JOURNAL OF NURSING CARE QUALITY/OCTOBER–DECEMBER 2014
event risk within an observation interval (eg, risk of falling during a month is uniform); (2) lack of secular trends (trends over time) within an interval (eg, falls do not increase at the end of the month); (3) withdrawals (participants who leave a study) do not differ from those who continue under observation (this usually refers to clinical trials); and (4) withdrawals occur uniformly within an observation interval.16 Violation of the first, second, and fourth assumptions is correctable by shortening the observation interval, which will stabilize risk within an interval. The third assumption, similar risk among withdrawals and participants, is usually violated because discharged patients are very different from patients who remain in the hospital. However, this assumption is less critical than the other assumptions. In addition to validation of these assumptions, person-time analyses require projectspecific inclusion and exclusion criteria to minimize bias. This paragraph summarizes and explains the rationale for inclusion and exclusion criteria used in the FPI analysis.9 Short-term patients, admitted for less than 24 hours, and long-term inpatients, with hospital stay more than 60 days due to low fall rates, are excluded from the analysis. At our hospital, short-stay patients are common and have very low inpatient fall rates (0.07 per 1000 inpatient-days). In addition, long-stay patients, although uncommon (0.16% of all inpatient admissions), are excluded from the analysis; this frequently immobile group would contribute many days to the population at risk for a fall, but have very low fall risk. Including inpatients with low fall risk inflates the population at risk for a fall and artificially deflates risk. Transfer patients (patients discharged from one unit and admitted to another unit on the same day) are assigned a length of stay beginning from their first admit date until their final discharge date. In addition to the general assumptions associated with inpatient falls, the simpler persontime method assumes: (1) patients stay in the hospital for 24-hour intervals, and (2) falls and discharges occur at the end of a 24-hour inter-
val. These assumptions overestimate time at risk, because admissions, discharges, and falls occur throughout the day. However, these assumptions facilitate the calculations. The more accurate person-time method uses exact and/or estimated admission, discharge, and fall times. This method includes 3 assumptions. First, each 24-hour inpatient stay (without a fall or discharge) contributes 1 day to the patient’s fall risk. Second, a fall contributes a 1/ day of risk to both the pre- and post-fall risk 2 intervals. Third, a discharge contributes a 1/2 day of risk. The person-time approach also requires guidelines for including and excluding unique patients, such as patients admitted and discharged in different observation intervals. In this analysis, patients admitted prior to the study or discharged after the study are included in the study; however, the analysis is limited to the observation intervals. Similarly, patients with lengths of stay that overlap 2 intervals are also included in the study. These patients contribute a fall risk to their length of stay in each interval. (Note: Multiple criteria are possible. A valid alternative criterion might exclude patients with lengths of stay that overlap intervals. The latter simplifies analysis and minimizes bias if fall risk differs in sequential observation periods.) However, falls among patients excluded from the denominator must also be excluded from the numerator; otherwise, fall rates will violate the definition of fall rates (falls in an observation interval per population at risk for falling in an observation interval). The NDNQI guidelines do not address inclusion and exclusion criteria for patients that cross observation intervals. This can be problematic since the NDNQI recommends 1-month observation intervals that would undoubtedly include many patients that cross observation intervals. Analysis of sample data Overview Table 1 uses 3 sample patients to demonstrate how to calculate fall risk and fall rates using new admissions, the midnight
Copyright © 2014 Lippincott Williams & Wilkins. Unauthorized reproduction of this article is prohibited.
Day 2
Day 3
Day 4
3.0
1
3 8.0 375.0
4.0
1
2 3.0 666.7
2.0
1.0
1
3 10.0 300.0
4.0
4.0
Entire sample
New admissions
3 9 333.3
3.5
4.0
1.5
2 5.5 363.6
2.5
1.5
1.5
At risk for 1st Fall (R1 )
1 2.0 500.0
1.0
1.0
0
0 1.5 0.0
0
1.5
0
At risk for At risk for 2nd Fall (R2 ) 3rd Fall (R3 )
b Assumes
a Excludes
short-stay admission (length of stay
JNCQ-D-13-00084
July 28, 2014
15:54
J Nurs Care Qual Vol. 29, No. 4, pp. 336–344 c 2014 Wolters Kluwer Health | Lippincott Williams & Wilkins Copyright
Bias Reduction in Calculation of Inpatient Fall Rates Kera F. Weiserbs, MHS, PhD; Barry J. Hahn, MD, FACEP Inpatient falls are the most common adverse hospital events. Despite the recognized importance of reducing inpatient falls, tracking and reporting methods are inconsistent. Moreover, recommended methods and statistical tests for comparing rates are complicated. This article demonstrates how to calculate fall rates using 3 common methods, summarizes the advantages and disadvantages of each method, and recommends best practices. Key words: inpatient falls, biases, health care evaluation mechanisms, nursing evaluation research, statistics and numerical data, quality improvement
R
EDUCING INPATIENT FALLS is important: they are the most common, preventable adverse hospital event, increasing morbidity1 and cost.2 Fortunately, inpatient falls are recognized as an important performance measurement by organizations dedicated to improving patient safety, including The Joint Commission,3 the National Quality Forum,4 and the Centers for Medicare & Medicaid Services.5 However, despite the recognized importance of reducing inpatient falls, tracking and reporting proven quality improvement methods for root-cause assessment and program assessment6,7 are applied inconsistently to inpatient fall-prevention programs. This problem is compounded by the
Author Affiliations: Former, Department of Academic Affairs (Dr Weiserbs) and Emergency Medicine (Dr Hahn), Staten Island University Hospital, Staten Island, New York. Dr Weiserbs is now a freelance consultant. The authors declare no conflict of interest. Supplemental digital content is available for this article. Direct URL citations appear in the printed text and are provided in the HTML and PDF versions of this article on the journal’s Web site (www.jncqjournal.com). Correspondence: Kera F. Weiserbs, MHS, PhD, Statistical Consultant 445 Argyle Rd, Brooklyn, NY 11218 ([email protected]). Accepted for Publication: March 2, 2014 Published online before print: April 15, 2014 DOI: 10.1097/NCQ.0000000000000063
lack of standardized methods that limit intraand inter-institutional comparisons due to the different biases (deviations from the actual events) associated with each method.8 A summary of these biases and the advantages associated with each method is shown in Supplemental Digital Content, Table 1 (available at: http://links.lww.com/JNCQ/A84). This article has 3 goals. The first goal is to demonstrate how to calculate inpatient fall rates using 3 methods and to summarize the advantages and disadvantages of each method. These methods include new admissions; the midnight census (the number of hospital beds occupied at midnight); and the person-time method (the sum of each inpatient’s time at risk for a fall; eg, the patient’s length of stay). The second goal is to present relative improvement rates (RIRs), a statistic for comparing rates. The third goal is to recommend best practices for the calculation and comparison of inpatient fall rates. METHODS Data sources This report uses data from a large tertiary care teaching hospital in New York City. The data set compares fall rates measures, pre- and post-implementation of a fallprevention initiative (FPI) that began in March 2006. This intervention and the associated performance improvements are described in
336
Copyright © 2014 Lippincott Williams & Wilkins. Unauthorized reproduction of this article is prohibited.
LWW/JNCQ
JNCQ-D-13-00084
July 28, 2014
15:54
Bias Reduction in Calculation of Inpatient Fall Rates another article.9 In addition to the FPI data, sample cases are used to illustrate how each statistic is calculated. Measures of population at risk The most common methods for assessing fall risk include counts and fall rates (the incidence of falls during an observation interval).8,10,11 Counts of inpatient falls are easy to quantify, provide numerator data for fall rates, and help determine resource allocation for fall prevention. However, counts do not estimate program efficacy, or adjust for hospital size or inpatient’s length of stay. Inpatient fall rates (falls in an observation interval per population at risk for falling in an observation interval) are a much better measure of program efficacy than counts, because rates adjust for hospital size and patient’s length of stay. This rate measures incidence, the frequency of new events (eg, falls) in a population (eg, inpatients at a specific hospital) during a given observation interval (eg, July) (new cases of disease or new events in an observation interval per population at risk for a disease or an event in an observational interval).12 Incidence rates use standardized denominators, for example, inpatient falls per 1000 inpatient-days or colon cases per 100 000 person-years, to facilitate discussing rare disease (or event) rates. For example, although 0.38 falls per inpatientday and 380 falls per 1000 inpatient-days are equivalent mathematically, a fall rate of 0.38 falls per day is not meaningful; a person cannot have a 0.38 falls, because falls are dichotomous events (either occur or do not occur). In contrast, a rate of 380 falls per 1000 days (380 falls in 1000 days) is meaningful. Standardized denominators are calculated by multiplying the numerator and denominator by a power of 10, such as 1 000 000 1000 10 000 100 000 , , , or . 1000 10 000 100 000 1 000 000 Unit choice depends on the rarity of the event or historical precedence. Inpatient falls are commonly reported per 1000 inpatientdays (abbreviated as 1000 days).
337
New admissions and midnight census New admissions and the midnight census are the most common measures of inpatient fall risk. Although these statistics are readily available, they are not the best measures of inpatient fall risk. These statistics are collected for other purposes. New admissions measure inpatient flow and the midnight census measures service needs, which are required for Medicare Part A billing. New admissions adjust for differences in hospital size (new admissions are a function of hospital size), but do not access inpatient’s length of stay, a risk factor for inpatient falls.13,14 The midnight census estimates hospital size and approximates inpatient length of stay (days at risk for a fall); however, the midnight census truncates length of stay to the number of nights that a patient stayed in a hospital and therefore underestimates length of stay. The National Database of Nursing Quality Indicators (NDNQI) recognizes the midnight census as an adequate, albeit a suboptimal, estimate of length of stay, because it underestimates length of stay.15 Person-time The NDNQI recommends person-time as the most accurate length of stay estimate.15 The person-time approach, although uncommon in the nursing literature, is the criterion standard for evaluating a rare event (eg, inpatient fall rates). Limited methodological documentation in the quality improvement literature, and limited statistical training in nursing programs, has curtailed mainstream use of this approach. This section summarizes the assumptions used in the person-time approach and presents 2 methods for calculating person-time: a simpler method, which overestimates length of stay, and a more accurate, albeit more complicated, method. Person-time measures length of stay. It is the sum of all the times that everyone in the sample or population is at risk for an event. For example, if 3 people are admitted to a hospital for 2, 3, and 10 days, respectively, their person-time at risk for a fall equals 15 days.8,10 This method assumes (1) constant
Copyright © 2014 Lippincott Williams & Wilkins. Unauthorized reproduction of this article is prohibited.
LWW/JNCQ
338
JNCQ-D-13-00084
July 28, 2014
15:54
JOURNAL OF NURSING CARE QUALITY/OCTOBER–DECEMBER 2014
event risk within an observation interval (eg, risk of falling during a month is uniform); (2) lack of secular trends (trends over time) within an interval (eg, falls do not increase at the end of the month); (3) withdrawals (participants who leave a study) do not differ from those who continue under observation (this usually refers to clinical trials); and (4) withdrawals occur uniformly within an observation interval.16 Violation of the first, second, and fourth assumptions is correctable by shortening the observation interval, which will stabilize risk within an interval. The third assumption, similar risk among withdrawals and participants, is usually violated because discharged patients are very different from patients who remain in the hospital. However, this assumption is less critical than the other assumptions. In addition to validation of these assumptions, person-time analyses require projectspecific inclusion and exclusion criteria to minimize bias. This paragraph summarizes and explains the rationale for inclusion and exclusion criteria used in the FPI analysis.9 Short-term patients, admitted for less than 24 hours, and long-term inpatients, with hospital stay more than 60 days due to low fall rates, are excluded from the analysis. At our hospital, short-stay patients are common and have very low inpatient fall rates (0.07 per 1000 inpatient-days). In addition, long-stay patients, although uncommon (0.16% of all inpatient admissions), are excluded from the analysis; this frequently immobile group would contribute many days to the population at risk for a fall, but have very low fall risk. Including inpatients with low fall risk inflates the population at risk for a fall and artificially deflates risk. Transfer patients (patients discharged from one unit and admitted to another unit on the same day) are assigned a length of stay beginning from their first admit date until their final discharge date. In addition to the general assumptions associated with inpatient falls, the simpler persontime method assumes: (1) patients stay in the hospital for 24-hour intervals, and (2) falls and discharges occur at the end of a 24-hour inter-
val. These assumptions overestimate time at risk, because admissions, discharges, and falls occur throughout the day. However, these assumptions facilitate the calculations. The more accurate person-time method uses exact and/or estimated admission, discharge, and fall times. This method includes 3 assumptions. First, each 24-hour inpatient stay (without a fall or discharge) contributes 1 day to the patient’s fall risk. Second, a fall contributes a 1/ day of risk to both the pre- and post-fall risk 2 intervals. Third, a discharge contributes a 1/2 day of risk. The person-time approach also requires guidelines for including and excluding unique patients, such as patients admitted and discharged in different observation intervals. In this analysis, patients admitted prior to the study or discharged after the study are included in the study; however, the analysis is limited to the observation intervals. Similarly, patients with lengths of stay that overlap 2 intervals are also included in the study. These patients contribute a fall risk to their length of stay in each interval. (Note: Multiple criteria are possible. A valid alternative criterion might exclude patients with lengths of stay that overlap intervals. The latter simplifies analysis and minimizes bias if fall risk differs in sequential observation periods.) However, falls among patients excluded from the denominator must also be excluded from the numerator; otherwise, fall rates will violate the definition of fall rates (falls in an observation interval per population at risk for falling in an observation interval). The NDNQI guidelines do not address inclusion and exclusion criteria for patients that cross observation intervals. This can be problematic since the NDNQI recommends 1-month observation intervals that would undoubtedly include many patients that cross observation intervals. Analysis of sample data Overview Table 1 uses 3 sample patients to demonstrate how to calculate fall risk and fall rates using new admissions, the midnight
Copyright © 2014 Lippincott Williams & Wilkins. Unauthorized reproduction of this article is prohibited.
Day 2
Day 3
Day 4
3.0
1
3 8.0 375.0
4.0
1
2 3.0 666.7
2.0
1.0
1
3 10.0 300.0
4.0
4.0
Entire sample
New admissions
3 9 333.3
3.5
4.0
1.5
2 5.5 363.6
2.5
1.5
1.5
At risk for 1st Fall (R1 )
1 2.0 500.0
1.0
1.0
0
0 1.5 0.0
0
1.5
0
At risk for At risk for 2nd Fall (R2 ) 3rd Fall (R3 )
b Assumes
a Excludes
short-stay admission (length of stay
