bs_bs_banner
Journal of Evaluation in Clinical Practice ISSN 1365-2753
Exploratory analysis of the relationship between home health agency engagement in a national campaign and reduction in acute care hospitalization in US home care patients E. Eve Esslinger RN BSN MS,1 Charles P. Schade MD MPH,4 Cynthia K. Sun RN MSN,5 Ying Hua Sun MS,2 Jill Manna BA,3 Bethany Knowles Hall BS,6 Shanen Wright BA,7 Karen L. Hannah MBA8 and Janet R. Lynch PhD9 1 Lead Project Coordinator, 2Statistician, 3Corporate Director of Analytic Resources, West Virginia Medical Institute and Quality Insights, Harrisburg, PA, USA 4 Consultant, 5Project Coordinator, 6Communications Specialist, 7HHQI National Campaign Director, 8Epidemiologist, West Virginia Medical Institute and Quality Insights, Charleston, WV, USA 9 Corporate Science Officer, West Virginia Medical Institute and Quality Insights, Richmond, VA, USA
Keywords community networks, health professional, home care agency, patient admission, programme evaluation, quality improvement Correspondence Dr Janet R. Lynch West Virginia Medical Institute and Quality Insights 300 Arboretum Place Suite 310, Richmond VA 23236 USA E-mail: [email protected] Accepted for publication: 8 May 2014 doi:10.1111/jep.12198
Abstract Rationale, aims and objectives To determine whether US home health agencies that intensively engaged with the 2010 Home Health Quality Improvement National Campaign were more likely to reduce acute care hospitalization (ACH) rates than less engaged agencies. Method We included all Medicare-certified agencies that accessed Campaign resources in the first month of the Campaign and also responded to an online survey of resource utilization at month two. We used the survey data and item response theory to estimate a latent construct we called engagement with the campaign. ACH rates were calculated from the Centers for Medicare & Medicaid Services Outcome and Assessment Information Set for pre- and post-intervention periods (March–November 2009 and 2010, respectively). Results Staff from 1077 agencies accessed resources in the first month of the Campaign. Of these, 382 provided information about resource use and had 10 or more monthly discharges throughout the measurement periods. Dividing these agencies into quartiles based on engagement score, we found an association between engagement and reduction in ACH rates, P = 0.049 (χ2 for trend). Exploratory path analysis revealed the effect of engagement score on reduction in ACH rate to be partially mediated through reduction in average length of service rates. Conclusion We found evidence that early intensity of engagement with the Campaign, as measured through use of activities and resources, was positively associated with improvement. To continue the investigation of this relationship, future work in this and other campaigns should focus on further development of engagement measures.
Introduction Background Each year in the United States, more than 3 million homebound beneficiaries in need of skilled nursing care or therapy receive Medicare-paid home health services [1]. Since 2000, an increasing number of people, electing to live independent, noninstitutionalized lives, are receiving home health services as their physical capabilities diminish. Despite patients’ preferences to 664
remain in their homes whenever possible, more than one in four home health patient episodes result in hospitalization [1,2]. Several studies have identified improving transitions between the hospital and other clinical settings as associated with reductions in re-hospitalization [3–9]. Yet, opportunities for improvement remain in the home health setting. A dissemination strategy is needed to raise awareness of best practices and assist home health agencies in providing high-quality care. The Home Health Quality Improvement (HHQI) National Campaign, which began in January 2007 and continues through the present, is the Centers for
Journal of Evaluation in Clinical Practice 20 (2014) 664–670 © 2014 John Wiley & Sons, Ltd.
E.E. Esslinger et al.
Medicare & Medicaid Services (CMS) response to this need. A Quality Improvement Organization (QIO)-led initiative, the HHQI incorporated the efforts of 27 QIOs representing all regions of the country and one territory during the period of this study. QIOs are private organizations under contract to CMS to improve care for Medicare beneficiaries. We report here results from calendar years 2009 and 2010 related to reducing acute care hospitalization (ACH) rates among participating agencies. In previous research documenting results from the earlier years, we demonstrated an association between adoption of campaign methods and reduction in ACH rates, even while no association was apparent between agency sign-on (registration to participate in the Campaign) and improvement [10]. Along with describing the continuing agency response to the Campaign, we further investigate the relationship between use of campaign methods and reduction in ACH rates by examining the association between intensity of engagement with the HHQI campaign and ACH rates. We used the Standards for Quality Improvement Reporting Excellence [11] to plan, deploy and report our study results.
Local problem The Campaign was targeted to Medicare-certified home health agencies, with a focus on promoting a learning network – that is, a network of connected professionals who seek improvement through the structured exchange of information [12]. Collaboration among agencies and other stakeholders, including professional and trade associations, academic institutions and Medicare QIOs, was encouraged. Based on information we provided as part of the Campaign, as well as knowledge of their own circumstances, agencies identified local problems to be addressed through suitable interventions, many of which were products of the Campaign.
Intended improvement The Campaign’s aim was to achieve large-scale improvement in multiple care processes and patient outcomes. This article focuses on one of the primary goals, which was to decrease agency ACH rates.
Study question In this article, we examine the question of whether agencies that intensively engaged with the 2010 HHQI Campaign through educational activities and use of campaign resources were more likely to reduce ACH rates than less engaged agencies.
Methods Ethical issues As a quality improvement activity in which individual patients were not the subject of study, the project did not require nor undergo review by an institutional review board [13,14].
Setting US home health agencies certified by Medicare vary in ownership, size, location and services offered. Enrolment in the Campaign
© 2014 John Wiley & Sons, Ltd.
Home health agency campaign engagement
required an agency CMS Certification Number; however, other health care providers could also gain access to campaign resources.
Planning the interventions The Campaign’s multifaceted interventional approach was designed to recruit a large group of agencies, engage them and give them the tools they needed to improve care to Medicare beneficiaries. The Campaign used approaches generally found to be effective in health care quality improvement such as education of health care providers [15] and data audit or feedback [16]. In addition, the Campaign incorporated innovative approaches such as use of social networking and electronic communications. Best Practice Intervention Packages (BPIPs) were central to the Campaign educational strategy. The ACH BPIP was the main intervention to reduce ACH rates. This BPIP introduced the best practices related to hospitalization risk assessment and emergency care planning [17]. Additional BPIPs released over the course of the Campaign were on topics complementary to reducing hospitalizations. A description of each interventional approach, main factors contributing to selection, initial plans for implementation and evolution are available in supplementary materials from the authors.
Planning the study of the intervention We planned to measure the percent of Medicare-certified agencies registering for the Campaign; measure intensity of agency engagement early in the Campaign; examine improvement in ACH rates over time; and test the hypothesis that level of engagement is positively associated with reduction in ACH rates. We expected the Campaign to raise staff awareness about the need to reduce ACH among home health patients and to provide staff with the encouragement and tools to act upon this awareness. Because the tools were based on scientific research and expert opinion, we expected higher percentages of the more engaged agencies to be successful in improving ACH rates, compared with the less engaged agencies. The Campaign offered all agencies the opportunity to participate; we did not control exposure to the Campaign intervention. Therefore, the study design was necessarily observational. In this paper, we focus on a self-selected subgroup of participating agencies for which we had information on level of participation and outcome. We examine ACH rates pre- and post-intervention in relation to agencies’ level of engagement with the Campaign. Observational research poses numerous threats to internal validity, and the design limits our ability to infer causality. However, external validity is positively impacted by the inclusion of agencies with varying characteristics.
Methods of evaluation To measure participation and implementation, we used online survey software to collect information from agency staff who downloaded the complete ACH BPIP from the HHQI web site within the first 30 days of release. We surveyed one person per agency. When more than one person from an agency downloaded the ACH BPIP, we selected the individual who made the largest number of downloads of campaign materials during the first 30 days. We fielded the survey 2 months after the BPIP release, providing time for implementation. 665
Home health agency campaign engagement
The survey included questions about use of recommended best practices and campaign resources, as well as the timing of implementation. A pre-test with five volunteer agency staff found the instrument to possess face validity. (A full description of the survey is available from the authors.) Two targeted reminders following survey administration increased the response rate. After removing incomplete surveys, we constructed a composite measure that we interpreted as agency engagement in the Campaign. We used CMS-provided Outcome and Assessment Information Set (OASIS) data to measure agency-specific ACH rates, defined as the percent of home health episodes of care ending in an inpatient hospitalization. OASIS is a standardized dataset containing beneficiary assessment information used in clinical care, quality reporting and payment. Agencies collect and submit the data to CMS. Through the CMS OASIS Guidance Manual, agencies are instructed in how to verify data accuracy through clinical record and data entry audits [18]; however, it is difficult for CMS to validate these self-reported data. OASIS data have been reported to be reliable [19,20].
Analysis To assess pre–post-intervention changes, we calculated ACH rates for the baseline period of 1 March 2009 to 30 November 2009, and for the post-intervention period of 1 March 2010, through 30 November 2010. We selected the same months for pre- and post-measurement to avoid seasonality effects, as well as to avoid unpredictable differences due to year-to-year changes in influenza activity, which is known to impact ACH. We assessed improvement by calculating the difference in the two rates. The primary unit of analysis was the agency. We gathered information on agency characteristics, including location and ownership, from the CMS Health Customer Information System [21]. We classified agencies as urban or rural based on the urban–rural continuum code for their county of location, per the 2005 Health Resources and Services Administration Area Resource File [22]. We classified agencies as Southern or non-Southern based on state location as described in a previous paper [23]. (Researchers have shown significant differences in hospitalization rates between Southern and other states [24]). We linked survey data to agency characteristics based on the agency provider number and email address of the respondent. Ambiguities in respondent identification were possible because the survey software captured only the email address of the respondent, who could have more than one agency affiliation. We resolved the small number of ambiguous linkages by matching to all agencies associated with the same respondent. We verified that aggregated item response frequencies were similar if we randomly picked a single agency from the possibilities. We chose to include every agency that matched to a respondent rather than a single randomly selected agency because the outcome assessment was based on agency-level performance, which might have differed among the agencies matched to each individual respondent. OASIS records were linked by beneficiary claim number to build episodes of care. From the episode information, we extracted agency identification, discharge disposition and service dates. We calculated ACH rates and average length of service (ALOS) during the 9-month pre- and post-intervention periods for every agency 666
E.E. Esslinger et al.
having at least 10 discharges in every month. After calculating the difference in ACH rate from the pre-intervention to postintervention period, we classified an agency’s performance as ‘improved’ if the post-intervention ACH rate was less than the pre-intervention measure. For agencies having a respondent to the assessment questionnaire, item response theory (IRT) was used to obtain estimates of a latent construct we called engagement with the HHQI campaign. Using IRT, we created a score for each survey respondent that approximated a ‘level’ of engagement. IRT assumes that every respondent has some location on a continuous latent dimension or trait, called theta (θ). The probability of a positive response to a survey item is assumed to be a function of the trait that θ represents, in this study, engagement. IRT has been described as similar to clinical inference. Whereas clinical inference is used to make a diagnosis for a person based on observed symptoms and background knowledge, IRT is used to produce a trait estimate based on survey response patterns and a mathematical model [25]. There are a variety of different IRT mathematical models that can be used to explain how items influence response behaviour and how best to estimate θ. We chose the two-item logistic regression model, which is appropriate when the underlying trait is unidimensional and survey responses are binary [26]. To test our hypothesis that level of engagement is positively associated with improvement – that is, reduction – in ACH rate, we tabulated the percent of agencies improving by quartile of engagement score and calculated chi-square for trend. Because ALOS is known to influence ACH, as defined in this study, at the agency level [23], we also tabulated the percent of agencies that decreased their ALOS by quartile of engagement score. Engagement might reduce the ACH rate both directly and indirectly (through reducing ALOS). To explore this possibility, we used path analysis, a method of decomposing correlations, to examine potential linkages. For this analysis, we first tested the distribution of the engagement score θ for conformity to the normal distribution and discovered a significant (P = 0.048) excess number of cases in the upper and lower 2% of the distribution. These cases were situations where respondents indicated they had followed all of the campaign recommended practices (upper 2%) or none of them (lower 2%). We eliminated these cases as outliers prior to performing the path analysis.
Results There were 4135 registered participating agencies among the population of Medicare-certified agencies with 10 or more discharges per month, representing 54.1% of this group. Many were reached through the Campaign’s multi-pronged educational intervention strategy. All received OASIS data reports and electronic messages. Interventions evolved as the Campaign progressed based on our assessment of the needs of agencies. Changes included increased emphasis on improving care transitions between health care settings in accordance with CMS goals; enhancements in data feedback to drive improvement; and additions to the rewards programme to increase participation. None of these improvements affected the evaluation of engagement, which relied on measuring the uptake of specific activities and resources in the initial months of the campaign, including:
© 2014 John Wiley & Sons, Ltd.
E.E. Esslinger et al.
• use of a hospitalization risk assessment tool; • use of a patient emergency plan tool; • use of the ‘Call Me First’ poster; • use of the ‘Call the Nurse First’ poster; • revision of agency policies or protocols; • participation in campaign webinars; • use of Campaign podcasts or other information on the Campaign’s web site and other social networking sites; and • setting agency ACH goals. The ACH BPIP was posted on the HHQI web site on 26 January 2010. Within a month, 1085 agency staff downloaded the full BPIP at least once. Removing duplicate names reduced the number to 1021 individuals linked to 1077 agencies. On 26 March, we invited these individuals by email to respond to the online survey. Five hundred twenty-nine responses were received by 15 April, for a response rate of 51.8% of the ‘downloading’ individuals (49.1% of ‘downloading’ agencies). The largest group of respondent agencies were not-for-profit, whereas the largest group of non-respondent agencies were for-profit (P = 0.0008). Otherwise, respondent and non-respondent agencies were similar with almost 60% of both groups among the highest volume agencies, and nearly evenly distributed across large metropolitan, small metropolitan and rural locations. About a third of each group was agencies in Southern states. See Table 1. Agencies reported activities consistent with campaign recommendations as well as frequent use of campaign materials. About 84% participated in more than one activity, and more than half participated in three or more. The majority of agencies reported they began the recommended activities prior to downloading materials in the BPIP package; however, 30% reported initiating use of the hospitalization risk assessment or modifying an existing similar tool after downloading. Twenty-six percent similarly reported use of the emergency planning tool. Each of these tools was used by more than 60% of respondent agencies. Four hundred ninety-four agencies evaluated the usefulness of campaign resources, with a large percentage voicing the opinion that these materials either helped staff think about the issue (202, 40.9%), helped to change patient management to avoid ACH (206, 41.7%) or resulted in measurable reductions in ACH (62, 12.6%). Only about 5% reported no impact. Of the agencies with complete responses to the assessment questionnaire, 382 agencies, or 72.2% of all respondents, had 10 or more discharges per month. For these facilities, there was a statistically significant association between quartile of engagement scores and improvement in ACH rates, P = 0.049 (χ2 for trend). See Figure 1, where we also display a similar trend for ALOS (P = 0.024). In the exploratory path analysis, holding constant for 2009 rates of ACH and ALOS, we see that the total effect of engagement on reduction in ACH rate (i.e. 2009 rate – 2010 rate) was positive and statistically significant but partially mediated by reduction in ALOS (2009–2010 rate). Standardized regression coefficients showing the relative strengths of the paths between presumed exogenous and endogenous variables are shown in the diagram of Figure 2, with the total effect of engagement score on reduction in acute care hospitalization rate divided into direct and indirect effects in the table below. The indirect effect is the product of the two paths connecting engagement score to reduction in acute care hospitalization rate.
© 2014 John Wiley & Sons, Ltd.
Home health agency campaign engagement
Table 1 Home health agency characteristics of survey respondents and non-respondents Respondents % Non-respondents % (n) (n) P-value Ownership For profit
40.9 (217) 12.8 (68) 46.3 (246)
52.6 (267) 10.0 (51) 37.4 (190)
2.3 (12) 11.1 (59) 28.4 (151) 58.3 (310)
3.9 (20) 12.4 (63) 25.2 (128) 58.5 (297)
35.6 (187) Small metropolitan 35.0 (184) Rural 29.5 (155) Southern state Yes 28.9 (154) No 71.1 (378)
33.8 (169) 36.2 (181) 30.0 (150)
Government Not for profit Patient volume Quartile 1 Quartile 2 Quartile 3 Quartile 4 Location Large metropolitan
33.9 (172) 66.1 (336)
0.0008
0.2893
0.8361
0.0879
Total responses will not sum to the potential number of agencies (1077) because information was not available for all agencies through the Centers for Medicare & Medicaid Services’ Health Customer Information System (ownership, volume) or the Area Resource File (location).
Discussion Summary The 2010 HHQI Campaign in the United States was a collaboration of volunteer home health agencies and other stakeholders that worked together to improve the quality of home care services. The Campaign reached a large portion of agencies using multiple interventions to reduce ACH rates among Medicare beneficiaries. The multi-pronged approach relied on web-based communications, dissemination of best practices and data feedback reports. Among participants, the majority of agencies that downloaded the ACH BPIP within the first month after its release and responded to an assessment survey found the campaign resources useful. Intensity of engagement with the Campaign, as represented by engagement score, was related to improvement, or reduction in ACH rate; however, in the exploratory path analysis the relationship between engagement and reduction in ACH rate was shown to be partially mediated through change in ALOS. 667
Home health agency campaign engagement
E.E. Esslinger et al.
70
Percent of agencies
60 50 40
ACH rate decreased
30
in the Campaign [10], in this study, we only surveyed agencies that we knew had accessed campaign materials. As a result, we were able to quantify level of engagement among agencies that all met a minimum threshold, and we demonstrated an association between formal engagement and improvement – that is, reduction in ACH rate. In the path diagram, that relationship was shown to be partially mediated by a reduction in ALOS.
ALOS decreased
20
Limitations 10 0 1
2
3
4
Quartile of agency engagement score
Figure 1 Percent of home health agencies whose acute care hospitalization (ACH) rate decreased and percent whose average length of service (ALOS) rate decreased from the baseline period (1 March 2009–30 November 2009) to the re-measurement period (1 March 2010–30 November 2010), by quartile of campaign engagement score.
Relation to other evidence Unlike public health campaigns promoting healthy behaviours, many of which have demonstrated efficacy [27], broad-based quality improvement campaigns targeting health professionals have yet to unequivocally demonstrate effectiveness. An example is the debate surrounding the impact of the Institute for Healthcare Improvement’s 100 000 Lives Campaign. The 100 000 Lives Campaign likely improved practice in many hospitals and saved lives. How many lives and what proportion was attributable to the Campaign is in dispute [28,29]. 100 000 Lives promoted evidencebased interventions, but was subject to so many externalities that definitive outcome evaluation was probably impossible. This was also the case with the HHQI National Campaign [10]. While we did not evaluate the overall effectiveness of the 2010 HHQI National Campaign, we offered a description to provide a context for our measurement of engagement with the campaign. The campaign provided multifaceted interventions, which we know are more effective at improving quality of care than single interventions [30]. Evidence also supports many of the specific interventions employed in the campaign, for example, providing educational materials [15] and audit and feedback [16]. In addition, our study incorporated a communication strategy that used web-based technology and promoted a learning community. Consistent with other research, we demonstrated the success of mass media in reaching large audiences [31]. However, dissemination is only one aspect of incorporating evidence into practice [32]. Participants must employ the evidence, and learning networks are one method for encouraging this [12]. Because little is known about the effectiveness of such vehicles in encouraging adoption of successful practices [33], it is important to develop measures of engagement to further research in this area. Our research is an initial step in this direction. Findings with respect to intensity of engagement in the Campaign reinforce results from earlier work showing that level of participation is related to outcome. Whereas we previously surveyed a sample of all agencies and were able to identify projectrelated improvement in agencies that purported not to participate 668
Among the limitations of this exploratory research are selection bias, measurement bias, confounding and power. While we had a fairly high response to our assessment survey, respondents may differ from non-respondents in systematic ways. Findings are applicable only to the respondent population, which included predominantly high volume agencies. We know very little about low volume agencies. Respondents did not know their outcomes at the time of the survey, eliminating one threat of bias. Nevertheless, self-reported information is subject to potential bias. Also, our measure of engagement is related only to use of campaign activities and resources in the early months. We did not ask about participation in the numerous other opportunities offered throughout the Campaign. We have no way of knowing if patients discharged to locations other than the hospital were hospitalized shortly thereafter. Sample size was barely adequate for evaluation of the relationship between engagement and ACH. There were also issues related to missing data, which reduced the number of useable survey responses.
Interpretation Consistent with our previous work, this study demonstrated that it is possible to persuade about half of home health agencies in the United States to register for a programme of quality improvement, but that a smaller proportion (about 1/5 of those with at least 10 discharges per month) used the resources that were offered to them. This does not mean that the other agencies made no effort to improve quality. There was nothing to prevent agencies from using other approaches besides those we offered or from using campaign materials they obtained from peers. In the first campaign, a minority of non-participants actually adopted approaches the Campaign advocated [10]. Our results demonstrated that agencies used a variety of ways to learn about quality improvement. Downloading materials from the web site was the most popular, but webinars, chat sessions and newer social media also contributed. Survey respondents found the Campaign materials useful, and many reported using them as intended. However, respondents, who were generally among the best-performing agencies, most commonly indicated that they had implemented recommended activities before the Campaign began, potentially limiting the additional effect of the Campaign on improvement. This is perhaps not surprising given that this was the second campaign focused on reducing ACH rates. We realize that the effects of Campaign engagement on ACH may have been partially indirect, through reductions in ALOS in the more active agencies, which we modelled through path analysis. With CMS’ adoption in 2013 of a claims-based hospitalization measure that considers unplanned hospitalizations within a fixed
© 2014 John Wiley & Sons, Ltd.
E.E. Esslinger et al.
Home health agency campaign engagement
Acute care hospitalizaon rate 2009
0.580
0.077
Engagement score
0.098
Reducon in average length of service rate e
0.250
Reducon in acute care hospitalizaon rate
-0.373
Average length of service rate 2009
e
Effect of engagement score on change in ACH rate
Figure 2 Standardized path coefficients for the direct and indirect effect of engagement score on reduction in acute care hospitalization rate between the baseline period (1 March 2009–30 November 2009) and the re-measurement period (1 March 2010–30 November 2010).
T ota l
Di re c t
Indirect
Path coefficient
0.102
0.077
0.024
Standard error
0.049
0.047
0.014
t-value
2.095
1.641
1.766
P-value
0.036
0.101
0.077
window from start of care rather than at discharge, future research may be better able to measure the relationship free of associations with reductions in ALOS.
not necessarily reflect the views or policies of the Department of Health & Human Services, nor does mention of trade names, commercial products, or organizations imply endorsement by the US government. The authors assume full responsibility for the accuracy and completeness of the ideas presented.
Conclusions The 2010 HHQI National Campaign was a multi-interventional, multi-focus project designed to engage home health agencies in improving the care they provided to Medicare beneficiaries. Here, we examined only one part of the Campaign related to reducing ACH rates. We found evidence that early intensity of engagement with the Campaign, as measured through use of activities and resources, was positively associated with improvement. To continue the investigation of this relationship, future work in this and other campaigns should focus on further development of engagement measures. US campaigns related to ACH reduction will benefit from use of the CMS claims-based measure.
Acknowledgements The analyses upon which this publication is based were performed under Contract Modification WV0005 to Contract Number 5002008-WV9THC, titled ‘Utilization and Quality Control Peer Review Organization for the State of West Virginia,’ sponsored by the Centers for Medicare & Medicaid Services, Department of Health & Human Services. The content of this publication does
© 2014 John Wiley & Sons, Ltd.
References 1. Medicare Payment Advisory Commission (2012) Report to the congress: Medicare payment policy. Available at: http://www.medpac.gov (last accessed 8 August 2013). 2. Shaughnessy, P. W., Hittle, D. F., Crisler, K. S., et al. (2002) Improving patient outcomes of home health care: findings from two demonstration trials of outcome-based quality improvement. Journal of the American Geriatric Society, 50 (8), 1354–1364. 3. Coleman, E. A., Parry, C., Chalmers, S. & Min, S. J. (2006) The care transitions intervention: results of a randomized controlled trial. Archives of Internal Medicine, 166 (17), 1822–1828. 4. Naylor, M. D., Brooten, D., Campbell, R., Jacobsen, B. S., Mezey, M. D., Pauly, M. V. & Schwartz, J. S. (1999) Comprehensive discharge planning and home follow-up of hospitalized elders: a randomized clinical trial. The Journal of the American Medical Association, 281 (7), 613–620. 5. Naylor, M. D., Brooten, D. A., Campbell, R. L., Maislin, G., McCauley, K. M. & Schwartz, J. S. (2004) Transitional care of older adults hospitalized with heart failure: a randomized, controlled trial. Journal of the American Geriatric Society, 52 (5), 675– 684.
669
Home health agency campaign engagement
6. Boutwell, A. & Hwu, S. (2009) Effective Interventions to Reduce Rehospitalizations: A Survey of the Published Evidence. Cambridge, MA: Institute for Healthcare Improvement. Available at: www.IHI.org (last accessed 4 March 2014). 7. Parry, C., Coleman, E. A., Smith, J. D., Frank, J. & Kramer, A. M. (2003) The care transitions intervention: a patient-centered approach to ensuring effective transfers between sites of geriatric care. Home Health Care Services Quarterly, 22 (3), 1–17. 8. Sevin, C. E., Evdokimoff, M., Sobolewski, S., Taylor, J., Rutherford, P. & Coleman, E. A. (2013) How-to-Guide: Improving Transitions from the Hospital to Home Health Care to Reduce Avoidable Rehospitalization. Cambridge, MA: Institute for Healthcare Improvement. Available at: http://www.ihi.org (last accessed 18 October 2013). 9. Brock, J., Mitchell, J., Irby, K., Stevens, B., Archibald, T., Goroski, A. & Lynn, J. (2013) Association between quality improvement for care transitions in communities and rehospitalizations among Medicare beneficiaries. The Journal of the American Medical Association, 309 (4), 381–391. 10. Schade, C. P., Esslinger, E., Anderson, D., Sun, Y. & Knowles, B. (2009) Impact of a national campaign on hospital readmissions in home care patients. International Journal for Quality in Health Care, 21 (3), 176–182. 11. Ogrinc, G., Mooney, S. E., Estrada, C., et al. (2008) The SQUIRE (Standards for QUality Improvement Reporting Excellence) guidelines for quality improvement reporting: explanation and elaboration. Quality and Safety in Health Care, 17 (Suppl. 1), i13–i32. 12. McCannon, C. J. & Perla, R. J. (2009) Learning networks for sustainable, large-scale improvement. Joint Commission Journal on Quality and Patient Safety, 35 (5), 286–291. 13. United States Department of Health and Human Services (2009) Human subjects research (45 CFR 46). Available at: http://www .hhs.gov/ohrp/humansubjects/guidance/45cfr46.html (last accessed 4 March 2013). 14. United States Department of Health and Human Services (2011) Quality improvement activities – FAQs. Available at: http:// answers.hhs.gov/ohrp/categories/1569 (last accessed 4 March 2013). 15. Giguere, A., Légaré, F., Grimshaw, J., Turcotte, S., Fiander, M., Grudniewicz, A., Makosso-Kallyth, S., Wolf, F., Farmer, A. P. & Gagnon, M. P. (2012) Printed educational materials: effects on professional practice and health care outcomes. Cochrane Database of Systematic Reviews, (10), CD004398, doi: 10.1002/14651858. CD004398.pub3. Available at: http://www.thecochranelibrary.com (last accessed 28 February 2014). 16. Ivers, N., Jamtvedt, G., Flottorp, S., Young, J. M., Odgaard-Jensen, J., French, S. D., O’Brian, M. A., Johansen, M., Grimshaw, J. & Oxman, A. D. (2012) Audit and feedback: effects on professional practice and healthcare outcomes. Cochrane Database of Systematic Reviews, (6), CD000259, doi: 10.1002/14651858.CD000259.pub3. Available at: http://www.cochrane.org/cochrane-reviews/cochrane-databasesystematic-reviews-numbers (last accessed 4 March 2014). 17. Campaign Home Health Quality Improvement (2010) Best Practice Intervention Package: fundamentals of reducing acute care hospitalization. Available at: http://www.homehealthquality.org/Education/ Best-Practices.aspx (last accessed 18 April 2013). 18. Centers for Medicare & Medicaid Services (2013) OASIS user manuals. Available at: http://www.cms.gov/Medicare/QualityInitiatives-Patient-Assessment-Instruments/HomeHealthQualityInits/ HHQIOASISUserManual.html (last accessed 19, April 2013).
670
E.E. Esslinger et al.
19. Hittle, D. F., Shaughnessy, P. W., Crisler, K. S., Powell, M. C., Richard, A. A., Conway, K. S., Stearns, P. M. & Engle, K. (2003) A study of reliability and burden of home health assessment using OASIS. Home Health Care Services Quarterly, 22 (4), 43–63. 20. Madigan, E. A. & Fortinsky, R. H. (2004) Interrater reliability of the outcomes and assessment information set: results from the field. The Gerontologist, 44 (5), 689–692. 21. Centers for Medicare & Medicaid Services. (2010) Health Care Information System (HCIS) data file. [database on the Internet]. Available at: http://www.cms.gov/Research-Statistics-Data-and-Systems/Files -for-Order/NonIdentifiableDataFiles/HealthCareInformationSystem .html (last accessed 17 October 2013). 22. Bureau of Health Professions. (2005) Area resource file. [database on the Internet]. Available from: http://arf.hrsa.gov (last accessed 17 October 2013). 23. Schade, C. P. & Brehm, J. G. (2010) Improving the home health acute-care hospitalization quality measure. Health Services Research, 45 (3), 712–727. 24. Wray, N. P., DeBehnke, R. D., Ashton, C. M. & Dunn, J. K. (1988) Characteristics of the recurrently hospitalized adult. An information synthesis. Medical Care, 26 (11), 1046–1056. 25. Embretson, S. & Reise, S. (2000) Item Response Theory for Psychologists. Mahwah, NJ: Lawrence Earlbaum Associates, Inc. 26. Drasgow, F. & Hulin, C. L. (1990) Item response theory. In Handbook of Industrial and Organizational Psychology (eds M. D. Dunnette & L. M. Hough), pp. 577–636. Palo Alto, CA: Consulting Psychologists Press. 27. Community Preventive Services Task Force (2010) Health communication and social marketing: health communication campaigns that include mass media & health-related product distribution. Available at: http://www.thecommunityguide.org/healthcommunication/ RRcampaigns.html (last accessed 17 March 2013). 28. Wachter, R. M. & Pronovost, P. J. (2006) The 100 000 lives campaign: a scientific and policy review. Joint Commission Journal on Quality and Patient Safety, 32 (11), 621–627. 29. Ross, T. K. (2009) A second look at 100 000 lives campaign. Quality Management in Health Care, 18 (2), 120–125. 30. Grimshaw, J. M., Shirran, L., Thomas, R., Mowatt, G., Fraser, C., Bero, L., Grilli, R., Harvey, E., Oxman, A. & O’Brien, M. A. (2001) Changing provider behavior: an overview of systematic reviews of interventions. Medical Care, 39 (8 Suppl. 2), II2–II45. 31. Abroms, L. C. & Maibach, E. W. (2008) The effectiveness of mass communication to change public behavior. Annual Review of Public Health, 29, 219–234. 32. Nieva, V. F., Murphy, R., Ridley, N., Donaldson, N., Combes, J., Mitchell, P., Kovner, C., Hoy, E. & Carpenter, D. (2005) From science to service: a framework for the transfer of patient safety research into practice. In Advances in Patient Safety: From Research to Implementation, Vol. 2, Concepts and Methodology (eds K. Henriksen, J. B. Battles, E. S. Marks & D. I. Lewin), pp. 441–453. Rockville, MD: Agency for Healthcare Research and Quality. 33. Li, L. C., Grimshaw, J. M., Nielsen, C., Judd, M., Coyte, P. C. & Graham, I. D. (2009) Use of communities of practice in business and health care sectors: a systematic review. Implementation Science, 4, 27. doi: 10.1186/1748-5908-4-27. Available at: http://www .implementationscience.com (last accessed 28 February 2014).
© 2014 John Wiley & Sons, Ltd.
Journal of Evaluation in Clinical Practice ISSN 1365-2753
Exploratory analysis of the relationship between home health agency engagement in a national campaign and reduction in acute care hospitalization in US home care patients E. Eve Esslinger RN BSN MS,1 Charles P. Schade MD MPH,4 Cynthia K. Sun RN MSN,5 Ying Hua Sun MS,2 Jill Manna BA,3 Bethany Knowles Hall BS,6 Shanen Wright BA,7 Karen L. Hannah MBA8 and Janet R. Lynch PhD9 1 Lead Project Coordinator, 2Statistician, 3Corporate Director of Analytic Resources, West Virginia Medical Institute and Quality Insights, Harrisburg, PA, USA 4 Consultant, 5Project Coordinator, 6Communications Specialist, 7HHQI National Campaign Director, 8Epidemiologist, West Virginia Medical Institute and Quality Insights, Charleston, WV, USA 9 Corporate Science Officer, West Virginia Medical Institute and Quality Insights, Richmond, VA, USA
Keywords community networks, health professional, home care agency, patient admission, programme evaluation, quality improvement Correspondence Dr Janet R. Lynch West Virginia Medical Institute and Quality Insights 300 Arboretum Place Suite 310, Richmond VA 23236 USA E-mail: [email protected] Accepted for publication: 8 May 2014 doi:10.1111/jep.12198
Abstract Rationale, aims and objectives To determine whether US home health agencies that intensively engaged with the 2010 Home Health Quality Improvement National Campaign were more likely to reduce acute care hospitalization (ACH) rates than less engaged agencies. Method We included all Medicare-certified agencies that accessed Campaign resources in the first month of the Campaign and also responded to an online survey of resource utilization at month two. We used the survey data and item response theory to estimate a latent construct we called engagement with the campaign. ACH rates were calculated from the Centers for Medicare & Medicaid Services Outcome and Assessment Information Set for pre- and post-intervention periods (March–November 2009 and 2010, respectively). Results Staff from 1077 agencies accessed resources in the first month of the Campaign. Of these, 382 provided information about resource use and had 10 or more monthly discharges throughout the measurement periods. Dividing these agencies into quartiles based on engagement score, we found an association between engagement and reduction in ACH rates, P = 0.049 (χ2 for trend). Exploratory path analysis revealed the effect of engagement score on reduction in ACH rate to be partially mediated through reduction in average length of service rates. Conclusion We found evidence that early intensity of engagement with the Campaign, as measured through use of activities and resources, was positively associated with improvement. To continue the investigation of this relationship, future work in this and other campaigns should focus on further development of engagement measures.
Introduction Background Each year in the United States, more than 3 million homebound beneficiaries in need of skilled nursing care or therapy receive Medicare-paid home health services [1]. Since 2000, an increasing number of people, electing to live independent, noninstitutionalized lives, are receiving home health services as their physical capabilities diminish. Despite patients’ preferences to 664
remain in their homes whenever possible, more than one in four home health patient episodes result in hospitalization [1,2]. Several studies have identified improving transitions between the hospital and other clinical settings as associated with reductions in re-hospitalization [3–9]. Yet, opportunities for improvement remain in the home health setting. A dissemination strategy is needed to raise awareness of best practices and assist home health agencies in providing high-quality care. The Home Health Quality Improvement (HHQI) National Campaign, which began in January 2007 and continues through the present, is the Centers for
Journal of Evaluation in Clinical Practice 20 (2014) 664–670 © 2014 John Wiley & Sons, Ltd.
E.E. Esslinger et al.
Medicare & Medicaid Services (CMS) response to this need. A Quality Improvement Organization (QIO)-led initiative, the HHQI incorporated the efforts of 27 QIOs representing all regions of the country and one territory during the period of this study. QIOs are private organizations under contract to CMS to improve care for Medicare beneficiaries. We report here results from calendar years 2009 and 2010 related to reducing acute care hospitalization (ACH) rates among participating agencies. In previous research documenting results from the earlier years, we demonstrated an association between adoption of campaign methods and reduction in ACH rates, even while no association was apparent between agency sign-on (registration to participate in the Campaign) and improvement [10]. Along with describing the continuing agency response to the Campaign, we further investigate the relationship between use of campaign methods and reduction in ACH rates by examining the association between intensity of engagement with the HHQI campaign and ACH rates. We used the Standards for Quality Improvement Reporting Excellence [11] to plan, deploy and report our study results.
Local problem The Campaign was targeted to Medicare-certified home health agencies, with a focus on promoting a learning network – that is, a network of connected professionals who seek improvement through the structured exchange of information [12]. Collaboration among agencies and other stakeholders, including professional and trade associations, academic institutions and Medicare QIOs, was encouraged. Based on information we provided as part of the Campaign, as well as knowledge of their own circumstances, agencies identified local problems to be addressed through suitable interventions, many of which were products of the Campaign.
Intended improvement The Campaign’s aim was to achieve large-scale improvement in multiple care processes and patient outcomes. This article focuses on one of the primary goals, which was to decrease agency ACH rates.
Study question In this article, we examine the question of whether agencies that intensively engaged with the 2010 HHQI Campaign through educational activities and use of campaign resources were more likely to reduce ACH rates than less engaged agencies.
Methods Ethical issues As a quality improvement activity in which individual patients were not the subject of study, the project did not require nor undergo review by an institutional review board [13,14].
Setting US home health agencies certified by Medicare vary in ownership, size, location and services offered. Enrolment in the Campaign
© 2014 John Wiley & Sons, Ltd.
Home health agency campaign engagement
required an agency CMS Certification Number; however, other health care providers could also gain access to campaign resources.
Planning the interventions The Campaign’s multifaceted interventional approach was designed to recruit a large group of agencies, engage them and give them the tools they needed to improve care to Medicare beneficiaries. The Campaign used approaches generally found to be effective in health care quality improvement such as education of health care providers [15] and data audit or feedback [16]. In addition, the Campaign incorporated innovative approaches such as use of social networking and electronic communications. Best Practice Intervention Packages (BPIPs) were central to the Campaign educational strategy. The ACH BPIP was the main intervention to reduce ACH rates. This BPIP introduced the best practices related to hospitalization risk assessment and emergency care planning [17]. Additional BPIPs released over the course of the Campaign were on topics complementary to reducing hospitalizations. A description of each interventional approach, main factors contributing to selection, initial plans for implementation and evolution are available in supplementary materials from the authors.
Planning the study of the intervention We planned to measure the percent of Medicare-certified agencies registering for the Campaign; measure intensity of agency engagement early in the Campaign; examine improvement in ACH rates over time; and test the hypothesis that level of engagement is positively associated with reduction in ACH rates. We expected the Campaign to raise staff awareness about the need to reduce ACH among home health patients and to provide staff with the encouragement and tools to act upon this awareness. Because the tools were based on scientific research and expert opinion, we expected higher percentages of the more engaged agencies to be successful in improving ACH rates, compared with the less engaged agencies. The Campaign offered all agencies the opportunity to participate; we did not control exposure to the Campaign intervention. Therefore, the study design was necessarily observational. In this paper, we focus on a self-selected subgroup of participating agencies for which we had information on level of participation and outcome. We examine ACH rates pre- and post-intervention in relation to agencies’ level of engagement with the Campaign. Observational research poses numerous threats to internal validity, and the design limits our ability to infer causality. However, external validity is positively impacted by the inclusion of agencies with varying characteristics.
Methods of evaluation To measure participation and implementation, we used online survey software to collect information from agency staff who downloaded the complete ACH BPIP from the HHQI web site within the first 30 days of release. We surveyed one person per agency. When more than one person from an agency downloaded the ACH BPIP, we selected the individual who made the largest number of downloads of campaign materials during the first 30 days. We fielded the survey 2 months after the BPIP release, providing time for implementation. 665
Home health agency campaign engagement
The survey included questions about use of recommended best practices and campaign resources, as well as the timing of implementation. A pre-test with five volunteer agency staff found the instrument to possess face validity. (A full description of the survey is available from the authors.) Two targeted reminders following survey administration increased the response rate. After removing incomplete surveys, we constructed a composite measure that we interpreted as agency engagement in the Campaign. We used CMS-provided Outcome and Assessment Information Set (OASIS) data to measure agency-specific ACH rates, defined as the percent of home health episodes of care ending in an inpatient hospitalization. OASIS is a standardized dataset containing beneficiary assessment information used in clinical care, quality reporting and payment. Agencies collect and submit the data to CMS. Through the CMS OASIS Guidance Manual, agencies are instructed in how to verify data accuracy through clinical record and data entry audits [18]; however, it is difficult for CMS to validate these self-reported data. OASIS data have been reported to be reliable [19,20].
Analysis To assess pre–post-intervention changes, we calculated ACH rates for the baseline period of 1 March 2009 to 30 November 2009, and for the post-intervention period of 1 March 2010, through 30 November 2010. We selected the same months for pre- and post-measurement to avoid seasonality effects, as well as to avoid unpredictable differences due to year-to-year changes in influenza activity, which is known to impact ACH. We assessed improvement by calculating the difference in the two rates. The primary unit of analysis was the agency. We gathered information on agency characteristics, including location and ownership, from the CMS Health Customer Information System [21]. We classified agencies as urban or rural based on the urban–rural continuum code for their county of location, per the 2005 Health Resources and Services Administration Area Resource File [22]. We classified agencies as Southern or non-Southern based on state location as described in a previous paper [23]. (Researchers have shown significant differences in hospitalization rates between Southern and other states [24]). We linked survey data to agency characteristics based on the agency provider number and email address of the respondent. Ambiguities in respondent identification were possible because the survey software captured only the email address of the respondent, who could have more than one agency affiliation. We resolved the small number of ambiguous linkages by matching to all agencies associated with the same respondent. We verified that aggregated item response frequencies were similar if we randomly picked a single agency from the possibilities. We chose to include every agency that matched to a respondent rather than a single randomly selected agency because the outcome assessment was based on agency-level performance, which might have differed among the agencies matched to each individual respondent. OASIS records were linked by beneficiary claim number to build episodes of care. From the episode information, we extracted agency identification, discharge disposition and service dates. We calculated ACH rates and average length of service (ALOS) during the 9-month pre- and post-intervention periods for every agency 666
E.E. Esslinger et al.
having at least 10 discharges in every month. After calculating the difference in ACH rate from the pre-intervention to postintervention period, we classified an agency’s performance as ‘improved’ if the post-intervention ACH rate was less than the pre-intervention measure. For agencies having a respondent to the assessment questionnaire, item response theory (IRT) was used to obtain estimates of a latent construct we called engagement with the HHQI campaign. Using IRT, we created a score for each survey respondent that approximated a ‘level’ of engagement. IRT assumes that every respondent has some location on a continuous latent dimension or trait, called theta (θ). The probability of a positive response to a survey item is assumed to be a function of the trait that θ represents, in this study, engagement. IRT has been described as similar to clinical inference. Whereas clinical inference is used to make a diagnosis for a person based on observed symptoms and background knowledge, IRT is used to produce a trait estimate based on survey response patterns and a mathematical model [25]. There are a variety of different IRT mathematical models that can be used to explain how items influence response behaviour and how best to estimate θ. We chose the two-item logistic regression model, which is appropriate when the underlying trait is unidimensional and survey responses are binary [26]. To test our hypothesis that level of engagement is positively associated with improvement – that is, reduction – in ACH rate, we tabulated the percent of agencies improving by quartile of engagement score and calculated chi-square for trend. Because ALOS is known to influence ACH, as defined in this study, at the agency level [23], we also tabulated the percent of agencies that decreased their ALOS by quartile of engagement score. Engagement might reduce the ACH rate both directly and indirectly (through reducing ALOS). To explore this possibility, we used path analysis, a method of decomposing correlations, to examine potential linkages. For this analysis, we first tested the distribution of the engagement score θ for conformity to the normal distribution and discovered a significant (P = 0.048) excess number of cases in the upper and lower 2% of the distribution. These cases were situations where respondents indicated they had followed all of the campaign recommended practices (upper 2%) or none of them (lower 2%). We eliminated these cases as outliers prior to performing the path analysis.
Results There were 4135 registered participating agencies among the population of Medicare-certified agencies with 10 or more discharges per month, representing 54.1% of this group. Many were reached through the Campaign’s multi-pronged educational intervention strategy. All received OASIS data reports and electronic messages. Interventions evolved as the Campaign progressed based on our assessment of the needs of agencies. Changes included increased emphasis on improving care transitions between health care settings in accordance with CMS goals; enhancements in data feedback to drive improvement; and additions to the rewards programme to increase participation. None of these improvements affected the evaluation of engagement, which relied on measuring the uptake of specific activities and resources in the initial months of the campaign, including:
© 2014 John Wiley & Sons, Ltd.
E.E. Esslinger et al.
• use of a hospitalization risk assessment tool; • use of a patient emergency plan tool; • use of the ‘Call Me First’ poster; • use of the ‘Call the Nurse First’ poster; • revision of agency policies or protocols; • participation in campaign webinars; • use of Campaign podcasts or other information on the Campaign’s web site and other social networking sites; and • setting agency ACH goals. The ACH BPIP was posted on the HHQI web site on 26 January 2010. Within a month, 1085 agency staff downloaded the full BPIP at least once. Removing duplicate names reduced the number to 1021 individuals linked to 1077 agencies. On 26 March, we invited these individuals by email to respond to the online survey. Five hundred twenty-nine responses were received by 15 April, for a response rate of 51.8% of the ‘downloading’ individuals (49.1% of ‘downloading’ agencies). The largest group of respondent agencies were not-for-profit, whereas the largest group of non-respondent agencies were for-profit (P = 0.0008). Otherwise, respondent and non-respondent agencies were similar with almost 60% of both groups among the highest volume agencies, and nearly evenly distributed across large metropolitan, small metropolitan and rural locations. About a third of each group was agencies in Southern states. See Table 1. Agencies reported activities consistent with campaign recommendations as well as frequent use of campaign materials. About 84% participated in more than one activity, and more than half participated in three or more. The majority of agencies reported they began the recommended activities prior to downloading materials in the BPIP package; however, 30% reported initiating use of the hospitalization risk assessment or modifying an existing similar tool after downloading. Twenty-six percent similarly reported use of the emergency planning tool. Each of these tools was used by more than 60% of respondent agencies. Four hundred ninety-four agencies evaluated the usefulness of campaign resources, with a large percentage voicing the opinion that these materials either helped staff think about the issue (202, 40.9%), helped to change patient management to avoid ACH (206, 41.7%) or resulted in measurable reductions in ACH (62, 12.6%). Only about 5% reported no impact. Of the agencies with complete responses to the assessment questionnaire, 382 agencies, or 72.2% of all respondents, had 10 or more discharges per month. For these facilities, there was a statistically significant association between quartile of engagement scores and improvement in ACH rates, P = 0.049 (χ2 for trend). See Figure 1, where we also display a similar trend for ALOS (P = 0.024). In the exploratory path analysis, holding constant for 2009 rates of ACH and ALOS, we see that the total effect of engagement on reduction in ACH rate (i.e. 2009 rate – 2010 rate) was positive and statistically significant but partially mediated by reduction in ALOS (2009–2010 rate). Standardized regression coefficients showing the relative strengths of the paths between presumed exogenous and endogenous variables are shown in the diagram of Figure 2, with the total effect of engagement score on reduction in acute care hospitalization rate divided into direct and indirect effects in the table below. The indirect effect is the product of the two paths connecting engagement score to reduction in acute care hospitalization rate.
© 2014 John Wiley & Sons, Ltd.
Home health agency campaign engagement
Table 1 Home health agency characteristics of survey respondents and non-respondents Respondents % Non-respondents % (n) (n) P-value Ownership For profit
40.9 (217) 12.8 (68) 46.3 (246)
52.6 (267) 10.0 (51) 37.4 (190)
2.3 (12) 11.1 (59) 28.4 (151) 58.3 (310)
3.9 (20) 12.4 (63) 25.2 (128) 58.5 (297)
35.6 (187) Small metropolitan 35.0 (184) Rural 29.5 (155) Southern state Yes 28.9 (154) No 71.1 (378)
33.8 (169) 36.2 (181) 30.0 (150)
Government Not for profit Patient volume Quartile 1 Quartile 2 Quartile 3 Quartile 4 Location Large metropolitan
33.9 (172) 66.1 (336)
0.0008
0.2893
0.8361
0.0879
Total responses will not sum to the potential number of agencies (1077) because information was not available for all agencies through the Centers for Medicare & Medicaid Services’ Health Customer Information System (ownership, volume) or the Area Resource File (location).
Discussion Summary The 2010 HHQI Campaign in the United States was a collaboration of volunteer home health agencies and other stakeholders that worked together to improve the quality of home care services. The Campaign reached a large portion of agencies using multiple interventions to reduce ACH rates among Medicare beneficiaries. The multi-pronged approach relied on web-based communications, dissemination of best practices and data feedback reports. Among participants, the majority of agencies that downloaded the ACH BPIP within the first month after its release and responded to an assessment survey found the campaign resources useful. Intensity of engagement with the Campaign, as represented by engagement score, was related to improvement, or reduction in ACH rate; however, in the exploratory path analysis the relationship between engagement and reduction in ACH rate was shown to be partially mediated through change in ALOS. 667
Home health agency campaign engagement
E.E. Esslinger et al.
70
Percent of agencies
60 50 40
ACH rate decreased
30
in the Campaign [10], in this study, we only surveyed agencies that we knew had accessed campaign materials. As a result, we were able to quantify level of engagement among agencies that all met a minimum threshold, and we demonstrated an association between formal engagement and improvement – that is, reduction in ACH rate. In the path diagram, that relationship was shown to be partially mediated by a reduction in ALOS.
ALOS decreased
20
Limitations 10 0 1
2
3
4
Quartile of agency engagement score
Figure 1 Percent of home health agencies whose acute care hospitalization (ACH) rate decreased and percent whose average length of service (ALOS) rate decreased from the baseline period (1 March 2009–30 November 2009) to the re-measurement period (1 March 2010–30 November 2010), by quartile of campaign engagement score.
Relation to other evidence Unlike public health campaigns promoting healthy behaviours, many of which have demonstrated efficacy [27], broad-based quality improvement campaigns targeting health professionals have yet to unequivocally demonstrate effectiveness. An example is the debate surrounding the impact of the Institute for Healthcare Improvement’s 100 000 Lives Campaign. The 100 000 Lives Campaign likely improved practice in many hospitals and saved lives. How many lives and what proportion was attributable to the Campaign is in dispute [28,29]. 100 000 Lives promoted evidencebased interventions, but was subject to so many externalities that definitive outcome evaluation was probably impossible. This was also the case with the HHQI National Campaign [10]. While we did not evaluate the overall effectiveness of the 2010 HHQI National Campaign, we offered a description to provide a context for our measurement of engagement with the campaign. The campaign provided multifaceted interventions, which we know are more effective at improving quality of care than single interventions [30]. Evidence also supports many of the specific interventions employed in the campaign, for example, providing educational materials [15] and audit and feedback [16]. In addition, our study incorporated a communication strategy that used web-based technology and promoted a learning community. Consistent with other research, we demonstrated the success of mass media in reaching large audiences [31]. However, dissemination is only one aspect of incorporating evidence into practice [32]. Participants must employ the evidence, and learning networks are one method for encouraging this [12]. Because little is known about the effectiveness of such vehicles in encouraging adoption of successful practices [33], it is important to develop measures of engagement to further research in this area. Our research is an initial step in this direction. Findings with respect to intensity of engagement in the Campaign reinforce results from earlier work showing that level of participation is related to outcome. Whereas we previously surveyed a sample of all agencies and were able to identify projectrelated improvement in agencies that purported not to participate 668
Among the limitations of this exploratory research are selection bias, measurement bias, confounding and power. While we had a fairly high response to our assessment survey, respondents may differ from non-respondents in systematic ways. Findings are applicable only to the respondent population, which included predominantly high volume agencies. We know very little about low volume agencies. Respondents did not know their outcomes at the time of the survey, eliminating one threat of bias. Nevertheless, self-reported information is subject to potential bias. Also, our measure of engagement is related only to use of campaign activities and resources in the early months. We did not ask about participation in the numerous other opportunities offered throughout the Campaign. We have no way of knowing if patients discharged to locations other than the hospital were hospitalized shortly thereafter. Sample size was barely adequate for evaluation of the relationship between engagement and ACH. There were also issues related to missing data, which reduced the number of useable survey responses.
Interpretation Consistent with our previous work, this study demonstrated that it is possible to persuade about half of home health agencies in the United States to register for a programme of quality improvement, but that a smaller proportion (about 1/5 of those with at least 10 discharges per month) used the resources that were offered to them. This does not mean that the other agencies made no effort to improve quality. There was nothing to prevent agencies from using other approaches besides those we offered or from using campaign materials they obtained from peers. In the first campaign, a minority of non-participants actually adopted approaches the Campaign advocated [10]. Our results demonstrated that agencies used a variety of ways to learn about quality improvement. Downloading materials from the web site was the most popular, but webinars, chat sessions and newer social media also contributed. Survey respondents found the Campaign materials useful, and many reported using them as intended. However, respondents, who were generally among the best-performing agencies, most commonly indicated that they had implemented recommended activities before the Campaign began, potentially limiting the additional effect of the Campaign on improvement. This is perhaps not surprising given that this was the second campaign focused on reducing ACH rates. We realize that the effects of Campaign engagement on ACH may have been partially indirect, through reductions in ALOS in the more active agencies, which we modelled through path analysis. With CMS’ adoption in 2013 of a claims-based hospitalization measure that considers unplanned hospitalizations within a fixed
© 2014 John Wiley & Sons, Ltd.
E.E. Esslinger et al.
Home health agency campaign engagement
Acute care hospitalizaon rate 2009
0.580
0.077
Engagement score
0.098
Reducon in average length of service rate e
0.250
Reducon in acute care hospitalizaon rate
-0.373
Average length of service rate 2009
e
Effect of engagement score on change in ACH rate
Figure 2 Standardized path coefficients for the direct and indirect effect of engagement score on reduction in acute care hospitalization rate between the baseline period (1 March 2009–30 November 2009) and the re-measurement period (1 March 2010–30 November 2010).
T ota l
Di re c t
Indirect
Path coefficient
0.102
0.077
0.024
Standard error
0.049
0.047
0.014
t-value
2.095
1.641
1.766
P-value
0.036
0.101
0.077
window from start of care rather than at discharge, future research may be better able to measure the relationship free of associations with reductions in ALOS.
not necessarily reflect the views or policies of the Department of Health & Human Services, nor does mention of trade names, commercial products, or organizations imply endorsement by the US government. The authors assume full responsibility for the accuracy and completeness of the ideas presented.
Conclusions The 2010 HHQI National Campaign was a multi-interventional, multi-focus project designed to engage home health agencies in improving the care they provided to Medicare beneficiaries. Here, we examined only one part of the Campaign related to reducing ACH rates. We found evidence that early intensity of engagement with the Campaign, as measured through use of activities and resources, was positively associated with improvement. To continue the investigation of this relationship, future work in this and other campaigns should focus on further development of engagement measures. US campaigns related to ACH reduction will benefit from use of the CMS claims-based measure.
Acknowledgements The analyses upon which this publication is based were performed under Contract Modification WV0005 to Contract Number 5002008-WV9THC, titled ‘Utilization and Quality Control Peer Review Organization for the State of West Virginia,’ sponsored by the Centers for Medicare & Medicaid Services, Department of Health & Human Services. The content of this publication does
© 2014 John Wiley & Sons, Ltd.
References 1. Medicare Payment Advisory Commission (2012) Report to the congress: Medicare payment policy. Available at: http://www.medpac.gov (last accessed 8 August 2013). 2. Shaughnessy, P. W., Hittle, D. F., Crisler, K. S., et al. (2002) Improving patient outcomes of home health care: findings from two demonstration trials of outcome-based quality improvement. Journal of the American Geriatric Society, 50 (8), 1354–1364. 3. Coleman, E. A., Parry, C., Chalmers, S. & Min, S. J. (2006) The care transitions intervention: results of a randomized controlled trial. Archives of Internal Medicine, 166 (17), 1822–1828. 4. Naylor, M. D., Brooten, D., Campbell, R., Jacobsen, B. S., Mezey, M. D., Pauly, M. V. & Schwartz, J. S. (1999) Comprehensive discharge planning and home follow-up of hospitalized elders: a randomized clinical trial. The Journal of the American Medical Association, 281 (7), 613–620. 5. Naylor, M. D., Brooten, D. A., Campbell, R. L., Maislin, G., McCauley, K. M. & Schwartz, J. S. (2004) Transitional care of older adults hospitalized with heart failure: a randomized, controlled trial. Journal of the American Geriatric Society, 52 (5), 675– 684.
669
Home health agency campaign engagement
6. Boutwell, A. & Hwu, S. (2009) Effective Interventions to Reduce Rehospitalizations: A Survey of the Published Evidence. Cambridge, MA: Institute for Healthcare Improvement. Available at: www.IHI.org (last accessed 4 March 2014). 7. Parry, C., Coleman, E. A., Smith, J. D., Frank, J. & Kramer, A. M. (2003) The care transitions intervention: a patient-centered approach to ensuring effective transfers between sites of geriatric care. Home Health Care Services Quarterly, 22 (3), 1–17. 8. Sevin, C. E., Evdokimoff, M., Sobolewski, S., Taylor, J., Rutherford, P. & Coleman, E. A. (2013) How-to-Guide: Improving Transitions from the Hospital to Home Health Care to Reduce Avoidable Rehospitalization. Cambridge, MA: Institute for Healthcare Improvement. Available at: http://www.ihi.org (last accessed 18 October 2013). 9. Brock, J., Mitchell, J., Irby, K., Stevens, B., Archibald, T., Goroski, A. & Lynn, J. (2013) Association between quality improvement for care transitions in communities and rehospitalizations among Medicare beneficiaries. The Journal of the American Medical Association, 309 (4), 381–391. 10. Schade, C. P., Esslinger, E., Anderson, D., Sun, Y. & Knowles, B. (2009) Impact of a national campaign on hospital readmissions in home care patients. International Journal for Quality in Health Care, 21 (3), 176–182. 11. Ogrinc, G., Mooney, S. E., Estrada, C., et al. (2008) The SQUIRE (Standards for QUality Improvement Reporting Excellence) guidelines for quality improvement reporting: explanation and elaboration. Quality and Safety in Health Care, 17 (Suppl. 1), i13–i32. 12. McCannon, C. J. & Perla, R. J. (2009) Learning networks for sustainable, large-scale improvement. Joint Commission Journal on Quality and Patient Safety, 35 (5), 286–291. 13. United States Department of Health and Human Services (2009) Human subjects research (45 CFR 46). Available at: http://www .hhs.gov/ohrp/humansubjects/guidance/45cfr46.html (last accessed 4 March 2013). 14. United States Department of Health and Human Services (2011) Quality improvement activities – FAQs. Available at: http:// answers.hhs.gov/ohrp/categories/1569 (last accessed 4 March 2013). 15. Giguere, A., Légaré, F., Grimshaw, J., Turcotte, S., Fiander, M., Grudniewicz, A., Makosso-Kallyth, S., Wolf, F., Farmer, A. P. & Gagnon, M. P. (2012) Printed educational materials: effects on professional practice and health care outcomes. Cochrane Database of Systematic Reviews, (10), CD004398, doi: 10.1002/14651858. CD004398.pub3. Available at: http://www.thecochranelibrary.com (last accessed 28 February 2014). 16. Ivers, N., Jamtvedt, G., Flottorp, S., Young, J. M., Odgaard-Jensen, J., French, S. D., O’Brian, M. A., Johansen, M., Grimshaw, J. & Oxman, A. D. (2012) Audit and feedback: effects on professional practice and healthcare outcomes. Cochrane Database of Systematic Reviews, (6), CD000259, doi: 10.1002/14651858.CD000259.pub3. Available at: http://www.cochrane.org/cochrane-reviews/cochrane-databasesystematic-reviews-numbers (last accessed 4 March 2014). 17. Campaign Home Health Quality Improvement (2010) Best Practice Intervention Package: fundamentals of reducing acute care hospitalization. Available at: http://www.homehealthquality.org/Education/ Best-Practices.aspx (last accessed 18 April 2013). 18. Centers for Medicare & Medicaid Services (2013) OASIS user manuals. Available at: http://www.cms.gov/Medicare/QualityInitiatives-Patient-Assessment-Instruments/HomeHealthQualityInits/ HHQIOASISUserManual.html (last accessed 19, April 2013).
670
E.E. Esslinger et al.
19. Hittle, D. F., Shaughnessy, P. W., Crisler, K. S., Powell, M. C., Richard, A. A., Conway, K. S., Stearns, P. M. & Engle, K. (2003) A study of reliability and burden of home health assessment using OASIS. Home Health Care Services Quarterly, 22 (4), 43–63. 20. Madigan, E. A. & Fortinsky, R. H. (2004) Interrater reliability of the outcomes and assessment information set: results from the field. The Gerontologist, 44 (5), 689–692. 21. Centers for Medicare & Medicaid Services. (2010) Health Care Information System (HCIS) data file. [database on the Internet]. Available at: http://www.cms.gov/Research-Statistics-Data-and-Systems/Files -for-Order/NonIdentifiableDataFiles/HealthCareInformationSystem .html (last accessed 17 October 2013). 22. Bureau of Health Professions. (2005) Area resource file. [database on the Internet]. Available from: http://arf.hrsa.gov (last accessed 17 October 2013). 23. Schade, C. P. & Brehm, J. G. (2010) Improving the home health acute-care hospitalization quality measure. Health Services Research, 45 (3), 712–727. 24. Wray, N. P., DeBehnke, R. D., Ashton, C. M. & Dunn, J. K. (1988) Characteristics of the recurrently hospitalized adult. An information synthesis. Medical Care, 26 (11), 1046–1056. 25. Embretson, S. & Reise, S. (2000) Item Response Theory for Psychologists. Mahwah, NJ: Lawrence Earlbaum Associates, Inc. 26. Drasgow, F. & Hulin, C. L. (1990) Item response theory. In Handbook of Industrial and Organizational Psychology (eds M. D. Dunnette & L. M. Hough), pp. 577–636. Palo Alto, CA: Consulting Psychologists Press. 27. Community Preventive Services Task Force (2010) Health communication and social marketing: health communication campaigns that include mass media & health-related product distribution. Available at: http://www.thecommunityguide.org/healthcommunication/ RRcampaigns.html (last accessed 17 March 2013). 28. Wachter, R. M. & Pronovost, P. J. (2006) The 100 000 lives campaign: a scientific and policy review. Joint Commission Journal on Quality and Patient Safety, 32 (11), 621–627. 29. Ross, T. K. (2009) A second look at 100 000 lives campaign. Quality Management in Health Care, 18 (2), 120–125. 30. Grimshaw, J. M., Shirran, L., Thomas, R., Mowatt, G., Fraser, C., Bero, L., Grilli, R., Harvey, E., Oxman, A. & O’Brien, M. A. (2001) Changing provider behavior: an overview of systematic reviews of interventions. Medical Care, 39 (8 Suppl. 2), II2–II45. 31. Abroms, L. C. & Maibach, E. W. (2008) The effectiveness of mass communication to change public behavior. Annual Review of Public Health, 29, 219–234. 32. Nieva, V. F., Murphy, R., Ridley, N., Donaldson, N., Combes, J., Mitchell, P., Kovner, C., Hoy, E. & Carpenter, D. (2005) From science to service: a framework for the transfer of patient safety research into practice. In Advances in Patient Safety: From Research to Implementation, Vol. 2, Concepts and Methodology (eds K. Henriksen, J. B. Battles, E. S. Marks & D. I. Lewin), pp. 441–453. Rockville, MD: Agency for Healthcare Research and Quality. 33. Li, L. C., Grimshaw, J. M., Nielsen, C., Judd, M., Coyte, P. C. & Graham, I. D. (2009) Use of communities of practice in business and health care sectors: a systematic review. Implementation Science, 4, 27. doi: 10.1186/1748-5908-4-27. Available at: http://www .implementationscience.com (last accessed 28 February 2014).
© 2014 John Wiley & Sons, Ltd.
