ORIGINAL ARTICLE
Evaluation of AHRQ’s On-Time Pressure Ulcer Prevention Program A Facilitator-assisted Clinical Decision Support Intervention for Nursing Homes Lauren E. W. Olsho, PhD,* William D. Spector, PhD,w Christianna S. Williams, PhD,* William Rhodes, PhD,* Rebecca V. Fink, MPH,* Rhona Limcangco, PhD, MPharm,z and Donna Hurd, MSN*
Background: Pressure ulcers present serious health and economic consequences for nursing home residents. The Agency for Healthcare Research & Quality, in partnership with the New York State Department of Health, implemented the pressure ulcer module of On-Time Quality Improvement for Long Term Care (On-Time), a clinical decision support intervention to reduce pressure ulcer incidence rates. Objective: To evaluate the effectiveness of the On-Time program in reducing the rate of in-house–acquired pressure ulcers among nursing home residents. Research Design and Subjects: We employed an interrupted timeseries design to identify impacts of 4 core On-Time program components on resident pressure ulcer incidence in 12 New York State nursing homes implementing the intervention (n = 3463 residents). The sample was purposively selected to include nursing homes with high baseline prevalence and incidence of pressure ulcers and high motivation to reduce pressure ulcers. Differential timing and sequencing of 4 core On-Time components across intervention nursing homes and units enabled estimation of separate impacts for each component. Inclusion of a nonequivalent comparison group of 13 nursing homes not implementing On-Time (n = 2698 residents) accounts for potential mean-reversion bias. Impacts were estimated via a random-effects Poisson model including resident-level and facility-level covariates.
From the *Abt Associates Inc., US Health Division, Cambridge, MA; wAgency for Healthcare Research & Quality, US Department of Health & Human Services; and zSocial & Scientific Systems Inc., Rockville, MD. The findings and conclusions in this manuscript are those of the authors and do not necessarily represent the official position of AHRQ or DHHS. Supported by the Agency for Healthcare Research & Quality (AHRQ), Department of Health & Human Services (DHHS), under contract #HHSA290200600011i. This study was reviewed and approved by the Abt Associates Institutional Review Board. The authors declare no conflict of interest. Reprints: Lauren E. W. Olsho, PhD, Abt Associates Inc., 55 Wheeler Street, Cambridge, MA 02138. E-mail: [email protected]. Copyright r 2013 by Lippincott Williams & Wilkins ISSN: 0025-7079/14/5203-0258
258 | www.lww-medicalcare.com
Results: We find a large and statistically significant reduction in pressure ulcer incidence associated with the joint implementation of 4 core On-Time components (incidence rate ratio = 0.409; P = 0.035). Impacts vary with implementation of specific component combinations. Conclusions: On-Time implementation is associated with sizable reductions in pressure ulcer incidence. Key Words: pressure ulcers, quality improvement, clinical decision support, interrupted time series (Med Care 2014;52: 258–266)
P
ressure ulcers remain a serious problem in nursing homes despite regulatory and market approaches to encourage prevention and treatment.1 Pressure ulcers cause pain, disfigurement, and increased infection risk, and are associated with longer hospital stays and increased morbidity and mortality.2 Median annual nursing home pressure ulcer prevalence was 7.5% in 2009,3 with associated costs of $3.3 billion annually.4 In the 1990s the Agency for Health Care Policy and Research (AHCPR, now AHRQ—the Agency for Healthcare Research & Quality) issued pressure ulcer prevention and treatment guidelines that have served as a model for more recent guidelines.5,6 The Centers for Medicare and Medicaid Services subsequently developed nursing home quality measures for pressure ulcers, now incorporated into a quality reporting system for all certified nursing homes in the United States.7 Nursing home quality regulations currently include a complex set of Centers for Medicare and Medicaid Services– issued federal standards with additional standards in some States, coupled with compliance monitoring inspections, State and federal sanctions (citations, financial penalties, and administrative actions), and complaint investigations. A complementary approach to quality monitoring and sanctions is to use health information technology (HIT) to improve care at the point of delivery via clinical decision supports (CDS). Nursing home staff often lacks timely or comprehensive information about changing resident risk profiles, limiting ability to intervene early to reduce pressure Medical Care
Volume 52, Number 3, March 2014
Medical Care
Volume 52, Number 3, March 2014
ulcer formation risk. HIT can provide easily accessible CDS reports summarizing real-time information from multiple sources to facilitate clinical intervention. Evidence is growing that CDS can improve clinical outcomes8–11; however, success relies on consistent staff use. CDS is more likely to be adopted with top management support, leader advocacy and active involvement, adaptability to local contexts, timely delivery of impact information to staff, and high-quality, team-based training.12,13 Factors related to successful implementation of CDS interventions include administrative and nursing staff buy-in and support, a consistent process for integrating risk reports into ongoing workflow, and a staff “champion” to keep efforts focused.14 Use of a facilitator to lead implementation and provide training also improves chances of success; primary care interventions using facilitators are 3 times more likely to be adopted.15 Higher intensity and length of facilitator involvement are associated with larger intervention effects and higher likelihood that an intervention is sustained.16–19
ON-TIME QUALITY IMPROVEMENT FOR LONG-TERM CARE Building on this literature, AHRQ developed the pressure ulcer module for “On-Time Quality Improvement for Long Term Care” (On-Time) via contract to Institute for Clinical Outcomes Research and Health Management Strategies, with additional grant funds from the California Health Care Foundation.20 On-Time is a CDS intervention aimed at nursing homes with leadership and staff support for quality improvements, a team-based approach to care, and leadership commitment. On-Time uses risk reports embedded in HIT systems to identify recent changes in risks, with guided facilitation to support integration of these reports into frontline practices. In addition to the pressure ulcer module, OnTime also includes modules to reduce falls and to prevent avoidable hospitalizations. In implementing On-Time pressure ulcer prevention, nursing homes use HIT to generate weekly CDS reports from certified nursing assistant (CNA) and nursing documentation. Reports profile resident pressure ulcer risk factors that are likely to change, such as nutritional status, incontinence, and recent pressure ulcer history. On-Time facilitators work with CNAs and front-line staff via biweekly conference calls to redesign workflow and improve processes to integrate On-Time reports into day-to-day practice. (The integrated reports and associated process improvements will be referred to hereafter as “On-Time components.”) Implementation of On-Time components is expected to produce more timely risk information, improve communication across disciplines, strengthen care plans and preventive practices, and ultimately reduce pressure ulcer incidence rates. More specifically, 5 On-Time CDS reports are populated from daily CNA documentation enhanced with standardized data elements and electronic data capture. A documentation completeness report is used to improve accuracy of CNA documentation and to monitor completeness r
2013 Lippincott Williams & Wilkins
On-Time Pressure Ulcer Prevention Evaluation
of needed elements for other CDS reports, but is not itself intended to directly influence pressure ulcer outcomes. The enhanced CNA documentation is used to populate 4 weekly core reports identifying residents at high risk for pressure ulcer formation, enabling monitoring of weekly changes in risk. These core reports include a nutrition report (monitors weight changes and decreased meal intake), a weight summary report (reports trended weight information over 180 d), a trigger summary report (monitors weight, urinary and bowel incontinence, and Foley catheter use at resident and unit levels), and a priority report (identifies changes in meal intake, weight loss, incontinence, behavior problems, and new/worsening pressure ulcers). During the intervention period, units in 8 of 12 intervention nursing homes additionally used an optional behavior report to test whether behavior changes could be documented electronically from CNA observations. Although the behavior report was not focused on pressure ulcer prevention, it was implemented over the same period as On-Time, and was also therefore included in analyses as a potential confounder. Implementation specifics are tailored to each nursing home’s needs. After implementation of the completeness report to assure data integrity, there is no uniform required sequence or timeframe for implementing other On-Time components. Minimum expectation for full On-Time implementation is initial use of the completeness report, followed by successful implementation of at least 3 intervention components employing 3 of the 4 core reports; nursing homes may implement additional components beyond this minimum standard if desired.21 Facilitators engage nursing home implementation teams in biweekly conference calls to select appropriate components for implementation until full implementation has been successfully achieved, followed by monthly conference calls for 12 subsequent months. Examples of team-based process improvements implemented in conjunction with On-Time reports as integrated intervention components include: using the nutrition and/or weight report in a briefing including CNAs, nurses, and dietary staff focused on improving nutrition status for high-risk residents, or using the trigger summary report in wound management meetings. To date, no formal evaluation of On-Time has been conducted, although an early pilot using simple pre-post comparisons found some reductions in pressure ulcer incidence.22 We investigate whether On-Time intervention components, individually and in combination, reduced pressure ulcer incidence in nursing homes.
METHODS Study Design We employed an interrupted time-series design to identify On-Time impacts on pressure ulcer incidence in 12 New York State nursing homes implementing the intervention. The unit of analysis was the nursing home resident. We collected monthly resident-level data on in-house– acquired pressure ulcers, starting in each intervention nursing home unit in the month the completeness report was first implemented, and continuing for 12 months after full www.lww-medicalcare.com |
259
Olsho et al
implementation. Sequence and timing of intervention component implementation varied considerably across nursing homes and units within nursing homes (Fig. 1). Within each unit, observation of a resident before and after an On-Time component was introduced allows us to observe whether pressure ulcer incidence decreased for that resident after implementation; averaging these changes in pressure ulcer incidence across residents in all study units implementing that component identifies our treatment effect. Variable timing of component implementation across units allows us to account for cyclical calendar-month effects, and reduces collinearity resulting from similar timeframes for individual component implementation within units. In addition, we analyze data from 13 New York State nursing homes that did not implement On-Time. We collected 12 months of resident-level data on in-house–acquired pressure ulcers in each of these comparison nursing homes, corresponding to the final 12 months of data collection in the intervention group. Inclusion of comparison nursing homes allows us to more confidently attribute observed declines in
Medical Care
Volume 52, Number 3, March 2014
pressure ulcer incidence in intervention nursing homes over the observation period to effects of the intervention rather than reversion to the mean.
Sample Recruitment and Selection The 2 groups of nursing homes were recruited to participate in the On-Time evaluation by the New York State Department of Health via a Request for Applications issued in 200723 and a Solicitation of Interest issued in 2010.24 Eligibility was limited to licensed nursing homes with pressure ulcer prevalence rates averaging at least 10% in the 2 most recent quarters, and no severe quality-of-care issues. In addition, only long-stay, nonrehabilitation, nonventilator nursing units with pressure ulcer incidence of at least 2% over the prior 6 months were eligible; the latter requirement was intended to minimize potential “floor effects.” The 2007 applicants were required to commit to implementing OnTime in all eligible long-stay nursing units; New York State Department of Health offered subsidies to partially offset
FIGURE 1. Data collection and On-Time implementation timeline, intervention, and comparison nursing homes. On-Time report components were implemented in varying timelines and sequences across implementation nursing homes, allowing separate identification of individual report impacts and calendar-month effects in our interrupted time-series analysis. Inclusion of comparison nursing homes allows us to account for possible reversion to the mean, of concern due to purposive selection of nursing homes with high pressure ulcer incidence and prevalence.
260 | www.lww-medicalcare.com
r
2013 Lippincott Williams & Wilkins
Medical Care
Volume 52, Number 3, March 2014
HIT costs, and AHRQ provided On-Time facilitators to support implementation. To avoid sample contamination, the 2010 applicants were required to demonstrate that they had not previously implemented key On-Time intervention components; however, to ensure similar levels of motivation across the 2 groups, these applicants were required to have participated in at least 1 other pressure ulcer reduction initiative. Both groups of applicants completed questionnaires assessing engagement of the management team (including director of nursing and/or assistant director of nursing); availability of a multidisciplinary implementation team including CNAs, nurses, dietitians, rehabilitation therapists or restorative nurses, and social work staff; stability of leadership and CNA staffing; available resources for staff educators; use of CNA mentors; experience with quality improvement and clinical data reports; and sophistication of existing wound management programs. Questionnaires were scored to assess capability for implementing On-Time; applications from the 13 highest-scoring eligible applicants in each group were accepted. One 2007 applicant agreed to implement On-Time but declined participation in the evaluation, yielding our final analytic sample of 12 nursing homes (with 42 participating nursing units) that implemented On-Time, and 13 nursing homes (with 33 participating units) that did not implement On-Time.
On-Time Pressure Ulcer Prevention Evaluation
data for all participating units after full implementation; however, 4 nursing homes submitted
Evaluation of AHRQ’s On-Time Pressure Ulcer Prevention Program A Facilitator-assisted Clinical Decision Support Intervention for Nursing Homes Lauren E. W. Olsho, PhD,* William D. Spector, PhD,w Christianna S. Williams, PhD,* William Rhodes, PhD,* Rebecca V. Fink, MPH,* Rhona Limcangco, PhD, MPharm,z and Donna Hurd, MSN*
Background: Pressure ulcers present serious health and economic consequences for nursing home residents. The Agency for Healthcare Research & Quality, in partnership with the New York State Department of Health, implemented the pressure ulcer module of On-Time Quality Improvement for Long Term Care (On-Time), a clinical decision support intervention to reduce pressure ulcer incidence rates. Objective: To evaluate the effectiveness of the On-Time program in reducing the rate of in-house–acquired pressure ulcers among nursing home residents. Research Design and Subjects: We employed an interrupted timeseries design to identify impacts of 4 core On-Time program components on resident pressure ulcer incidence in 12 New York State nursing homes implementing the intervention (n = 3463 residents). The sample was purposively selected to include nursing homes with high baseline prevalence and incidence of pressure ulcers and high motivation to reduce pressure ulcers. Differential timing and sequencing of 4 core On-Time components across intervention nursing homes and units enabled estimation of separate impacts for each component. Inclusion of a nonequivalent comparison group of 13 nursing homes not implementing On-Time (n = 2698 residents) accounts for potential mean-reversion bias. Impacts were estimated via a random-effects Poisson model including resident-level and facility-level covariates.
From the *Abt Associates Inc., US Health Division, Cambridge, MA; wAgency for Healthcare Research & Quality, US Department of Health & Human Services; and zSocial & Scientific Systems Inc., Rockville, MD. The findings and conclusions in this manuscript are those of the authors and do not necessarily represent the official position of AHRQ or DHHS. Supported by the Agency for Healthcare Research & Quality (AHRQ), Department of Health & Human Services (DHHS), under contract #HHSA290200600011i. This study was reviewed and approved by the Abt Associates Institutional Review Board. The authors declare no conflict of interest. Reprints: Lauren E. W. Olsho, PhD, Abt Associates Inc., 55 Wheeler Street, Cambridge, MA 02138. E-mail: [email protected]. Copyright r 2013 by Lippincott Williams & Wilkins ISSN: 0025-7079/14/5203-0258
258 | www.lww-medicalcare.com
Results: We find a large and statistically significant reduction in pressure ulcer incidence associated with the joint implementation of 4 core On-Time components (incidence rate ratio = 0.409; P = 0.035). Impacts vary with implementation of specific component combinations. Conclusions: On-Time implementation is associated with sizable reductions in pressure ulcer incidence. Key Words: pressure ulcers, quality improvement, clinical decision support, interrupted time series (Med Care 2014;52: 258–266)
P
ressure ulcers remain a serious problem in nursing homes despite regulatory and market approaches to encourage prevention and treatment.1 Pressure ulcers cause pain, disfigurement, and increased infection risk, and are associated with longer hospital stays and increased morbidity and mortality.2 Median annual nursing home pressure ulcer prevalence was 7.5% in 2009,3 with associated costs of $3.3 billion annually.4 In the 1990s the Agency for Health Care Policy and Research (AHCPR, now AHRQ—the Agency for Healthcare Research & Quality) issued pressure ulcer prevention and treatment guidelines that have served as a model for more recent guidelines.5,6 The Centers for Medicare and Medicaid Services subsequently developed nursing home quality measures for pressure ulcers, now incorporated into a quality reporting system for all certified nursing homes in the United States.7 Nursing home quality regulations currently include a complex set of Centers for Medicare and Medicaid Services– issued federal standards with additional standards in some States, coupled with compliance monitoring inspections, State and federal sanctions (citations, financial penalties, and administrative actions), and complaint investigations. A complementary approach to quality monitoring and sanctions is to use health information technology (HIT) to improve care at the point of delivery via clinical decision supports (CDS). Nursing home staff often lacks timely or comprehensive information about changing resident risk profiles, limiting ability to intervene early to reduce pressure Medical Care
Volume 52, Number 3, March 2014
Medical Care
Volume 52, Number 3, March 2014
ulcer formation risk. HIT can provide easily accessible CDS reports summarizing real-time information from multiple sources to facilitate clinical intervention. Evidence is growing that CDS can improve clinical outcomes8–11; however, success relies on consistent staff use. CDS is more likely to be adopted with top management support, leader advocacy and active involvement, adaptability to local contexts, timely delivery of impact information to staff, and high-quality, team-based training.12,13 Factors related to successful implementation of CDS interventions include administrative and nursing staff buy-in and support, a consistent process for integrating risk reports into ongoing workflow, and a staff “champion” to keep efforts focused.14 Use of a facilitator to lead implementation and provide training also improves chances of success; primary care interventions using facilitators are 3 times more likely to be adopted.15 Higher intensity and length of facilitator involvement are associated with larger intervention effects and higher likelihood that an intervention is sustained.16–19
ON-TIME QUALITY IMPROVEMENT FOR LONG-TERM CARE Building on this literature, AHRQ developed the pressure ulcer module for “On-Time Quality Improvement for Long Term Care” (On-Time) via contract to Institute for Clinical Outcomes Research and Health Management Strategies, with additional grant funds from the California Health Care Foundation.20 On-Time is a CDS intervention aimed at nursing homes with leadership and staff support for quality improvements, a team-based approach to care, and leadership commitment. On-Time uses risk reports embedded in HIT systems to identify recent changes in risks, with guided facilitation to support integration of these reports into frontline practices. In addition to the pressure ulcer module, OnTime also includes modules to reduce falls and to prevent avoidable hospitalizations. In implementing On-Time pressure ulcer prevention, nursing homes use HIT to generate weekly CDS reports from certified nursing assistant (CNA) and nursing documentation. Reports profile resident pressure ulcer risk factors that are likely to change, such as nutritional status, incontinence, and recent pressure ulcer history. On-Time facilitators work with CNAs and front-line staff via biweekly conference calls to redesign workflow and improve processes to integrate On-Time reports into day-to-day practice. (The integrated reports and associated process improvements will be referred to hereafter as “On-Time components.”) Implementation of On-Time components is expected to produce more timely risk information, improve communication across disciplines, strengthen care plans and preventive practices, and ultimately reduce pressure ulcer incidence rates. More specifically, 5 On-Time CDS reports are populated from daily CNA documentation enhanced with standardized data elements and electronic data capture. A documentation completeness report is used to improve accuracy of CNA documentation and to monitor completeness r
2013 Lippincott Williams & Wilkins
On-Time Pressure Ulcer Prevention Evaluation
of needed elements for other CDS reports, but is not itself intended to directly influence pressure ulcer outcomes. The enhanced CNA documentation is used to populate 4 weekly core reports identifying residents at high risk for pressure ulcer formation, enabling monitoring of weekly changes in risk. These core reports include a nutrition report (monitors weight changes and decreased meal intake), a weight summary report (reports trended weight information over 180 d), a trigger summary report (monitors weight, urinary and bowel incontinence, and Foley catheter use at resident and unit levels), and a priority report (identifies changes in meal intake, weight loss, incontinence, behavior problems, and new/worsening pressure ulcers). During the intervention period, units in 8 of 12 intervention nursing homes additionally used an optional behavior report to test whether behavior changes could be documented electronically from CNA observations. Although the behavior report was not focused on pressure ulcer prevention, it was implemented over the same period as On-Time, and was also therefore included in analyses as a potential confounder. Implementation specifics are tailored to each nursing home’s needs. After implementation of the completeness report to assure data integrity, there is no uniform required sequence or timeframe for implementing other On-Time components. Minimum expectation for full On-Time implementation is initial use of the completeness report, followed by successful implementation of at least 3 intervention components employing 3 of the 4 core reports; nursing homes may implement additional components beyond this minimum standard if desired.21 Facilitators engage nursing home implementation teams in biweekly conference calls to select appropriate components for implementation until full implementation has been successfully achieved, followed by monthly conference calls for 12 subsequent months. Examples of team-based process improvements implemented in conjunction with On-Time reports as integrated intervention components include: using the nutrition and/or weight report in a briefing including CNAs, nurses, and dietary staff focused on improving nutrition status for high-risk residents, or using the trigger summary report in wound management meetings. To date, no formal evaluation of On-Time has been conducted, although an early pilot using simple pre-post comparisons found some reductions in pressure ulcer incidence.22 We investigate whether On-Time intervention components, individually and in combination, reduced pressure ulcer incidence in nursing homes.
METHODS Study Design We employed an interrupted time-series design to identify On-Time impacts on pressure ulcer incidence in 12 New York State nursing homes implementing the intervention. The unit of analysis was the nursing home resident. We collected monthly resident-level data on in-house– acquired pressure ulcers, starting in each intervention nursing home unit in the month the completeness report was first implemented, and continuing for 12 months after full www.lww-medicalcare.com |
259
Olsho et al
implementation. Sequence and timing of intervention component implementation varied considerably across nursing homes and units within nursing homes (Fig. 1). Within each unit, observation of a resident before and after an On-Time component was introduced allows us to observe whether pressure ulcer incidence decreased for that resident after implementation; averaging these changes in pressure ulcer incidence across residents in all study units implementing that component identifies our treatment effect. Variable timing of component implementation across units allows us to account for cyclical calendar-month effects, and reduces collinearity resulting from similar timeframes for individual component implementation within units. In addition, we analyze data from 13 New York State nursing homes that did not implement On-Time. We collected 12 months of resident-level data on in-house–acquired pressure ulcers in each of these comparison nursing homes, corresponding to the final 12 months of data collection in the intervention group. Inclusion of comparison nursing homes allows us to more confidently attribute observed declines in
Medical Care
Volume 52, Number 3, March 2014
pressure ulcer incidence in intervention nursing homes over the observation period to effects of the intervention rather than reversion to the mean.
Sample Recruitment and Selection The 2 groups of nursing homes were recruited to participate in the On-Time evaluation by the New York State Department of Health via a Request for Applications issued in 200723 and a Solicitation of Interest issued in 2010.24 Eligibility was limited to licensed nursing homes with pressure ulcer prevalence rates averaging at least 10% in the 2 most recent quarters, and no severe quality-of-care issues. In addition, only long-stay, nonrehabilitation, nonventilator nursing units with pressure ulcer incidence of at least 2% over the prior 6 months were eligible; the latter requirement was intended to minimize potential “floor effects.” The 2007 applicants were required to commit to implementing OnTime in all eligible long-stay nursing units; New York State Department of Health offered subsidies to partially offset
FIGURE 1. Data collection and On-Time implementation timeline, intervention, and comparison nursing homes. On-Time report components were implemented in varying timelines and sequences across implementation nursing homes, allowing separate identification of individual report impacts and calendar-month effects in our interrupted time-series analysis. Inclusion of comparison nursing homes allows us to account for possible reversion to the mean, of concern due to purposive selection of nursing homes with high pressure ulcer incidence and prevalence.
260 | www.lww-medicalcare.com
r
2013 Lippincott Williams & Wilkins
Medical Care
Volume 52, Number 3, March 2014
HIT costs, and AHRQ provided On-Time facilitators to support implementation. To avoid sample contamination, the 2010 applicants were required to demonstrate that they had not previously implemented key On-Time intervention components; however, to ensure similar levels of motivation across the 2 groups, these applicants were required to have participated in at least 1 other pressure ulcer reduction initiative. Both groups of applicants completed questionnaires assessing engagement of the management team (including director of nursing and/or assistant director of nursing); availability of a multidisciplinary implementation team including CNAs, nurses, dietitians, rehabilitation therapists or restorative nurses, and social work staff; stability of leadership and CNA staffing; available resources for staff educators; use of CNA mentors; experience with quality improvement and clinical data reports; and sophistication of existing wound management programs. Questionnaires were scored to assess capability for implementing On-Time; applications from the 13 highest-scoring eligible applicants in each group were accepted. One 2007 applicant agreed to implement On-Time but declined participation in the evaluation, yielding our final analytic sample of 12 nursing homes (with 42 participating nursing units) that implemented On-Time, and 13 nursing homes (with 33 participating units) that did not implement On-Time.
On-Time Pressure Ulcer Prevention Evaluation
data for all participating units after full implementation; however, 4 nursing homes submitted
