Geriatric Nursing xx (2014) 1e7

Contents lists available at ScienceDirect

Geriatric Nursing journal homepage: www.gnjournal.com

Feature Article

Implementing toileting trials in nursing homes: Evaluation of a dissemination strategy Anna N. Rahman, PhD a, *, John F. Schnelle, PhD b, c, **, Dan Osterweil, MD d, e a

Gerontology Consultant, 519 Stassi Lane, Santa Monica, CA, USA Division of General Internal Medicine and Public Health, Center for Quality Aging, School of Medicine, Vanderbilt University Medical Center, Nashville, TN, USA c Geriatric Research, Education, and Clinical Center, VA Medical Center, Nashville, TN, USA d California Association of Long Term Care Medicine, Los Angeles, CA, USA e Division of Geriatrics, David Geffen School of Medicine, UCLA, Los Angeles, CA, USA b

a r t i c l e i n f o

a b s t r a c t

Article history: Received 6 March 2013 Received in revised form 4 March 2014 Accepted 10 March 2014 Available online xxx

Objectives: This study sought to improve incontinence care in nursing homes (NHs) by administering and evaluating a webinar course that provided extended instruction to help NHs implement toileting trials in accordance with recommended procedures. Of particular interest was: 1) whether the course design would prompt NH staffs to implement the recommended protocol, and 2) whether participants preferred this course design to other models. Design: The study collected descriptive evaluation data. Setting: The setting was comprised of seven NHs. Participants: Participants were staff members, typically nurses, from enrolled NHs who attended at least three of the six webinars that comprised the course. Measures: Data was collected using a course evaluation and implementation survey. Results: Staff in the participating nursing homes attended an average of 4.85 webinars, with an average of nine staff members attending each webinar (range: 3-20). Twelve of 16 responding participants said they preferred the webinar course to other course designs. All respondents said they would recommend the course and take a similar course again. All facilities submitted some evidence that staff providers had completed implementation assignments. Most facilities reported plans to sustain use of the recommended protocol. Conclusion: This study found limited evidence that the webinar course prompts NHs to implement a recommended toileting trial protocol and is preferred to other training program designs. Ó 2014 Mosby, Inc. All rights reserved.

Keywords: Nursing home Dissemination Incontinence care Translational research

Despite the best efforts of “many well-intentioned individuals” to resolve it1(p1810), incontinence remains a “costly, enduring, and vexing”2(p607) management problem in nursing homes (NHs), affecting an estimated 43% to half of all NH residents at an approximate cost of $5.3 billion annually.3 In an effort to improve incontinence care, the Minimum Data Set (MDS 3.0) assessment instrument now includes two items that specifically ask whether residents underwent a toileting trial and if so, what were the results.4,5 This dissemination study sought to improve incontinence care in NHs by administering and evaluating a webinar course designed * Corresponding author. 519 Stassi Lane, Santa Monica, CA 90402, USA. Tel.: þ1 513 258 4421. ** Corresponding author. Division of General Internal Medicine and Public Health, Center for Quality Aging, School of Medicine, Vanderbilt University, Medical Center, S-1121, Nashville, TN 37232-2400, USA. E-mail addresses: [email protected] (A.N. Rahman), john.schnelle@ vanderbilt.edu (J.F. Schnelle). 0197-4572/$ e see front matter Ó 2014 Mosby, Inc. All rights reserved. http://dx.doi.org/10.1016/j.gerinurse.2014.03.002

to help NHs implement toileting trials in accordance with recommended procedures. Using the Reach, Effectiveness, Adoption, Implementation, and Maintenance (RE-AIM) framework to evaluate this dissemination effort,6 we focused on the initial adoption results achieved during this nine-week course. Our primary evaluation goal was to assess whether the course prompted NH staff to adopt the recommended toileting trial protocol, as evidenced by their reports and documentation of new toileting trial assessments being completed for incontinent residents. Our secondary evaluation goal was to assess the course itself, including whether staff participants preferred this course to more traditional continuing education models. Evidence-based toileting trials Toileting trials are brief, typically three-day assessments during which NH providers attempt to assist incontinent residents to the

2

A.N. Rahman et al. / Geriatric Nursing xx (2014) 1e7

Dissemination researchers have found that some teaching strategies (e.g., one-day conferences or single-hour webinars) are too passive or time-limited to prompt real change in NHs while others (e.g., programs that employ outside consultants to visit NHs) are too labor-intensive or costly to readily replicate.14,15 Findings from dissemination research have also shown that giving sites the opportunity to try out a new intervention increases the likelihood it will be adopted and maintained.16 As a result, pilot-tests of new care practices are often recommended as an initial adoption strategy.17

instruction to NH staffs via a series of webinars or teleconferences (6e8 webinars/teleconferences) conducted over several weeks or months (10 weekse8 months). Each series featured field assignments that encouraged participants to implement recommended protocols during the course and get feedback on their progress. Participants in all three courses were encouraged to use the same assessment and implementation forms to complete field assignments (described below). Likewise, all three courses used the same, or very similar, evaluation surveys to assess participants’ preferences for training and each facility’s implementation results. The CAPTA course evaluated in this study was comprised of six webinars, each 40 min in length, held over nine weeks, with one to four weeks between webinars (see Table 1 for the curriculum). NHs accessed PowerPoint presentations and standardized assessment forms online from the webinar provider’s website. The audio portion was available via computer or phone (however, all participating NHs elected to access audio via the phone). At each participating NH, at least one staff memberdtypically a licensed nurse supervisor–agreed to attend all webinars, oversee implementation assignments, and act as the facility’s “champion” for the recommended toileting trial procedure. In email communications, we encouraged champions, at their discretion, to invite additional staff members, especially nurses and nurse aides involved in providing incontinence care, to attend the webinars. Administrators and licensed nurses received up to five continuing education (CEs) credit hours for their webinar participation. At the close of the first and third webinars, champions and their staffs were asked to jointly complete an implementation or field assignment before the next session, using a standardized assessment form(s) to guide the assignment (see Table 1). These assignments prompted NHs to: 1) interview a sample of incontinent residents to assess their preferences for incontinence care; 2) conduct a three-day toileting trial with these residents; 3) reinterview the residents following the trial; and 4) summarize the results from this procedure. We recommended that each facility implement these assignments with a sample of at least 10 incontinent residents, whom staff teams could select at their discretion. We discouraged teams, however, from selecting residents based on their cognitive or functional abilities, for our training emphasized that all incontinent residents should be considered candidates for toileting trials.5,8 About a week after each of these webinars, the principal investigator (P.I.) phoned each champion to remind her about the assignment and answer questions. Participants could also request assistance from the instructor or P.I. by phone or email at any time. Results from each assignment were discussed in the following

A dissemination strategy to promote adoption of toileting trials

Table 1 Course curriculum.

toilet every 2 h during the day. Best-practice trials also feature resident interviews both before and immediately following the three-day assessment to further identify resident preferences for care.7 A general rule is that residents who use the toilet appropriately on two-thirds or more of all toileting attempts during the trial are good candidates for continued toileting assistance.5 A best-practice toileting trial protocol, described in detail elsewhere,8 was first introduced in the 1990s, when Ouslander et al5 found that results from these assessments were the most valid predictor of which residents would have fewer incontinence episodes if provided with routine toileting assistance on a daily basis. Of note, toileting trial results also proved to be stronger predictors of continence improvement than residents’ cognitive status or functional ability, both of which are commonly and inappropriately used in NHs to target residents for toileting programs. Research shows that between 25% and 40% of incontinent residents will prove to be “responsive” during a toileting trial, with a reduction in their incontinence frequency from three to four episodes per day to one or less per day.5,9 Best practice guidelines call for “responsive” residents to continue to receive regular toileting assistance.10 Non-responders may benefit from a follow-up physician evaluation or a check-and-change program.10 Few studies have examined NHs’ adoption or implementation of toileting trials. Those that have suggest improvement is still needed in this common daily care area. One study found no evidence of toileting trial outcomes in medical charts for 426 NH residents.11 Another study reported that “bladder records” were “nonexistent” for a sample of 200 residents in 52 NHs.12 Presently, NHs receive little guidance to help them adopt toileting trials as part of their routine care practices. Federal survey guidelines, for instance, call for a comprehensive assessment of incontinent residents, but do not provide instructions or standardized assessment forms for conducting reliable toileting trials.13 Best practices for dissemination research

With this in mind, we administered a nine-week webinar course, titled the Choice and Preference Toileting Assistance course, or CAPTA course, to teach NH staffs how to conduct toileting trials. The nine-week course allowed participants sufficient time to implement each step of the recommended intervention with a sample of residents and receive feedback from a national expert on nursing home incontinence management (i.e., Dr. John F. Schnelle, a co-author of this paper) during the interactive webinars. CAPTA course design The CAPTA course was developed based on lessons learned from two earlier courses administered by the same research team and described in detail elsewhere.18,19 Drawing on active learning principles,20,21 all three courses offered incontinence management

Webinar 1, Week 1. How-to webinar: Assess resident preferences for incontinence care. Assignment: Interview at least 10 residents using the Resident Preference Interview Form and submit de-identified forms. Between session: Follow-up call to check on progress, provide reminders, answer questions. Webinar 2, Week 3. Webinar: Feedback, discussion of implementation assignment. Webinar 3, Week 4. How-to webinar: Conduct a 3-day toileting trial and analyze results. Assignment: Conduct toileting trials for at least 10 residents and record results on the Toileting Trial Form and then re-interview residents using the Resident Preference Interview Form. Analyze results using the Toileting Trial Results Form. Submit de-identified forms. Between session: Follow-up call Webinar 4, Week 8. Webinar: Feedback, discussion of implementation assignment. Webinar 5, Week 9: How-to webinar: Write-up care plans, survive the survey. Webinar 6, Week 11: How-to webinar: Monitor toileting assistance for continuous quality improvement.

A.N. Rahman et al. / Geriatric Nursing xx (2014) 1e7

webinar (i.e., the second and fourth webinars). The fifth and sixth webinars, which provided tips for writing up care plans and sustaining new incontinence care routines, did not feature field assignments. Three standardized forms needed to conduct the recommended toileting trial protocol were available from the course website (http:// caltcmcourses.wordpress.com): 1) a Resident Preference Interview form, for assessing residents’ care preferences both pre- and11 post-trial; 2) a Toileting Trial Form, for recording resident results from each attempt to use the toilet during a three-day trial; and 3) a Toileting Trial Results form, for summarizing the results from both the interview and toileting trial forms. These structured assessment forms were critical components of program adoption because they provided a standardized approach for NH staff to evaluate the appropriateness of a toileting program for individual residents. This course differed from the two earlier courses18,19 in three ways. First, it offered instruction over a shorter period: ten weeks in contrast to 6e8 months for the earlier courses. In the earlier courses, the goal to institutionalize comprehensive new incontinence care practices facility-wide during the months-long training period proved impractical.18,19 This shorter course aimed for a more short-term, attainable goal: A pilot test of a new assessment procedure, which could, ultimately, inform decisions about institution-wide implementation. Second, as described above, champions in this course submitted resident assessment forms that their staff completed for each implementation assignment. In the previous courses, summary results were abstracted from assessment forms and reported on a separate form, which was submitted at or near the end of the course. This reporting strategy resulted in a lot of missing data18,19 and prevented us from providing feedback on implementation results in real time. Finally, in the present course, the expert lecturer provided real-time feedback to the group in webinars during which he reviewed and commented on the assessment forms previously submitted by the participants. Thus, feedback was specific to actual implementation results. In the earlier courses, the course administrator (not the expert lecturer) called the champion at each participating facility between webinars to offer coaching assistance and answer questions. While feasible, this strategy was more time-consuming. Additionally, some champions in previous courses were difficult to reach by phone or, once reached, seemed reticent about asking questions or reporting specific implementation results; thus, any feedback offered was often in the absence of specific implementation outcomes. Study design This was a descriptive study using both quantitative and qualitative approaches to evaluate dissemination of the CAPTA Course. Sample Nursing homes in California were recruited by electronically disseminating a course announcement to 420 members of the California Association of Long Term Care Medicine, the state’s chapter of the American Medical Director’s Association. Announcements were also emailed to California’s Quality Improvement Organization and its for-profit NH membership association, for further dissemination to their constituents. Enrollment was rolling over two months, with an enrollment cap of 20 NHs. The enrollment limit was driven by this project’s budget constraints. Ten NHs registered. Enrolled facilities were considered to have completed the course if at least one staff member attended at least three of the six webinars. Two facilities, a for-profit and a non-profit, dropped out before the start of the course due to

3

staffing problems, and one was dropped due to insufficient attendance (this non-profit NH attended just the first webinar). Staff in seven NHs completed the course by attending at least half of the webinars. These facilities and their staff members who attended the webinars comprise the sample for this study. The final sample included four non-profit facilities, two forprofit facilities, and one government-owned facility.22 All but one participating NH had either a five- or a four-star government rating (range: 2e5 stars; CalQualityCare.org, accessed at the time of the study).22 Average bed size was 136 (range: 52e300 beds).22 Measures Using the RE-AIM framework,5 we conducted a multi-faceted course evaluation, with data collected primarily via two survey instruments: a course evaluation, completed by NH staff members who attended at least half of the webinars, and a pilot-test evaluation featuring items most appropriately answered at the facility level, which was completed by the champion at each participating NH site. Sixteen staff members in five nursing homes submitted course evaluations. Champions in six nursing homes submitted pilot-test evaluations. Table 2 presents assessment measures for each RE-AIM domain, with more detailed descriptions below. The research protocol was approved by the relevant institutional review board. Reach–course participation The webinar provider automatically recorded the phone number of all NHs that joined each webinar. Additionally, champions reported the number and roles of staff members who typically attended the webinars. Effectiveness of toileting trials Champions were asked in the pilot-test evaluation whether, in their opinion, the residents assessed during the course were drier than they were at the start of the course and whether more residents had individualized toileting plans as part of their overall care plan as a result of the facility’s participation in the course. Adoption of pilot-tests Champions were asked to submit de-identified resident assessment forms completed for the field assignments that followed teaching sessions 1 and 3. Champions also were asked in the pilot-test survey to report the number of residents who were: interviewed using the Resident Preference Interview form; received a three-day toileting trial; and were assessed using the Toileting Trial Results form. Implementation of the course and participant evaluations To assess the extent to which the course was implemented as we planned, we compared the original curriculum plan to the Table 2 Evaluation measures by RE-AIM domain. RE-AIM domain

Evaluation measure

Reach Effectiveness Adoption

Enrolled nursing homes’ webinar attendance Champion’s reports of resident outcomes Submitted resident assessment forms and champions’ reports of pilot-test results Comparison of the planned curriculum to the implemented curriculum; participants’ responses to satisfaction items on the course evaluation. Champions’ reports of toileting trial in-service training sessions at their facility and their facility’s plans for continuing the toileting trial protocol.

Implementation

Maintenance

4

A.N. Rahman et al. / Geriatric Nursing xx (2014) 1e7

implemented course curriculum (see Table 1 for the latter). Additionally, we assessed participants’ satisfaction, for one of our aims was to meet their expectations for the course. The course survey asked participants for their individual opinions regarding the best: 1) length for the webinars; 2) the number of webinars; and 3) and the interval between webinars. Participants also were asked: 1) whether they would participate in a similar course, and 2) whether they would recommend the course to colleagues. The final preference item asked participants to identify their preferred way of learning about toileting trials from a list of four options: a webinar curriculum like the one they had just completed; an online-only curriculum; an in-person, one-day training program offered offsite; and “other,” with a prompt to specify any other options. Participants also were invited to comment in writing on what they liked most and least about the course. In the pilot-test survey, champions were asked which of the four previously mentioned training formats they preferred for their staffs. Maintenance of toileting trials Champions reported in the pilot-test survey whether their facility conducted in-service training on each toileting trial form. They also identified whether the facility planned to continue using each form, expand its use, or discontinue its use. Course administration costs Research staff calculated actual and contributed costs to administer the course. Costs included contributed salary for the P.I., who recruited NHs, managed the course website, oversaw the evaluation, and provided customer service; a stipend for the lecturer (i.e., an hourly fee to present teleconferences with equal time to prepare); teleconferencing fees; a CE application fee; and miscellaneous costs for postage and office supplies. Analysis As appropriate to the data, frequencies, means, modes, medians, and ranges were calculated to report descriptive results. Openended comments on course evaluations were catalogued verbatim. Results Our primary evaluation goal was to assess whether the course prompted NH staff to adopt a best-practice toileting trial protocol, as evidenced by their reports and documentation of new toileting trial assessments being completed for incontinent residents. Our secondary evaluation goal was to evaluate the course itself, including whether staff participants preferred this course to more traditional continuing education models. We first report participation results, then the results pertaining to our primary and secondary goals. Other results concerning maintenance of the toileting trial protocol and course administration costs follow. Participation Staff in the seven participating NHs attended an average of 4.85 of the six webinars (range ¼ 4e6; mode ¼ 4; median ¼ 5). Six of seven champions reported on their staff’s webinar attendance. According to their reports, an average of nine staff members attended each webinar, with a wide range (3e20; median ¼ 5). Nurses, including DONs, MDS coordinators, and staff educators, typically attended; champions in two NHs reported that nurse aides regularly attended.

Pilot test results and evidence of improved incontinence Six champions (100% of respondents, with one missing) reported that their staff had completed all three toileting trial steps (i.e., interview, trial, and analysis) with at least some residents (range: 4e9 completed trials; mean ¼ 7 trials; mode ¼ 6 trials). As shown in Table 3, on average, champions reported interviewing nine residents (range: 6e10); conducting three-day trials with seven residents (range: 4e10); and analyzing results for seven residents (range: 4e10). All seven participating NHs, including the one for which the champion did not submit a pilot-test survey, submitted at least some implementation forms as evidence that the staff had completed these steps. Specifically, all seven NHs submitted pre-trial Resident Preference Interview Forms for a sample of residents; four (57.1% of participating NHs) submitted completed Toileting Trial Forms, and three (42.8% of participating NHs) submitted the Toileting Trial Results Form for assessed residents. Documented assessment results on these submitted forms showed compliance with the recommended toileting trial protocol, with one exception: One nursing home miscalculated appropriate toileting rates for assessed residents; as a result, further instruction on calculating this rate was provided in the fourth webinar. Three champions (50% of respondents) reported that, in their opinion, assessed residents were drier and that more incontinent residents had individualized toileting plans as a result of course participation (Table 3). Course expectations and evaluation The original curriculum plan called for seven webinars to be offered over 12 weeks. Webinars were to be offered every two weeks, with the exception of the last one, which was to be offered one week after the penultimate webinar. In practice, we conducted six webinars over ten weeks, with webinars spaced unequally over the weeks as shown in Table 1. This curriculum change was made at the request of participants, who wanted more time for some implementation assignments (e.g., the three day toileting trials) and less time between some sessions (e.g., they requested that webinars 3 and 5 be held a week after webinars 2 and 4, respectively, instead of two weeks later). The implemented course covered all topics originally planned. Fewer webinars were conducted because session three addressed two topics that we originally had planned to address in two webinars: how to conduct a toileting trial and analyze the results. Table 3 Champions’ implementation reports. Champions reporting:

N (total respondents ¼ 6)

Assessed residents are drier More residents have individualized care plans Facility conducted in-service on all 3 forms Facility conducted in-service on 2 forms Facility did not conduct any in-services Facility plans to continue/expand use of interview form Facility plans to continue/expand use of trial form Facility plans to continue/expand use of the results form Number of incontinent residents interviewed Number of incontinent residents with 3-day trial Number of incontinent residents with trial results analyzed

3 (50%) 3 (50%) 3 1 2 5

Mean/facility

(50%) (16.7%) (33.3%) (83.3%)

5 (83.3%) 5 (83.3%) 9 7 7

A.N. Rahman et al. / Geriatric Nursing xx (2014) 1e7

Twelve of 16 respondents who submitted course evaluations (75%) reported that the length of each webinar (40 min) was “just right”; three said the webinars were too short, and one skipped this item. All respondents reported that both the number and spacing of webinars was “just right.” All respondents also reported that they would both recommend the course and take a similar course again. Eighty-percent said they preferred the webinar course to either a conference held in a single location or a course completed entirely online. Fourteen participants (87.5%) reported open-ended comments about what they liked most about the course. Sample comments included the following: “The discussion and participation of the other facilities.”

5

positive resident outcomes (i.e., dryness, modified care plans) as a result of their staff’s participation in the course. These results are promising, but also indicate room for improvement. While it is not always possible to improve continence for all incontinent residents,5 it is possible to individualize care for all of them. It is possible that incontinence care would improve for all residents who need it if nursing homes continued to conduct toileting trials as recommended. A wealth of research shows that it takes considerable time, from months to years, to institutionalize improved healthcare practices and often more time for those improvements to affect downstream outcomes.23 Follow-up research is needed to assess this course’s longer-term effectiveness on resident outcomes.

“It is practical and realistic.” “Data gathering.” Only one respondent reported a “dislike:” “The math,” which was needed to calculate a resident’s responsiveness to toileting assistance. Two participants suggested course improvements, in both cases requesting more training time. Asked in the pilot-test survey which training format they preferred for their staffs, five of six responding champions (83.3%) said they preferred the webinar course; one champion preferred an off-site conference; no one elected an online course or “other” option (Table 3). Other results Maintenance As shown in Table 3, four champions (66% of respondents) reported that in-service programs on at least two, and more often on all three, of the recommended assessment forms were conducted in their NH during the course. Five champions (83.3% of respondents) said that their facility planned to continue or expand use of all three standardized assessment forms; one champion was uncertain of the facility’s plans. Course administration costs Direct costs to administer the course totaled $3,500, excluding the P.I.’s salary, which was contributed. The P.I. estimated that she spent 5e7 h per webinar on administration, website maintenance, and customer service, for an estimated additional cost to the project of $3500. Total projects costs are thus estimated at $7,000, or $1000 per participating NH. Discussion Incontinence care outcomes The primary aim of this dissemination training course was to prompt NHs to pilot-test toileting trials for at least a sample of incontinent NH residents as a means of improving incontinence care and resident outcomes. In this respect, the course was modestly successful. Training results indicated that each participating NH successfully completed toileting trials for at least some residents (average of 7, range 4e9 residents). Additionally, all NHs submitted completed resident preference interviews related to toileting; four submitted completed toileting trial forms; and three submitted completed results forms. With just one exception, documentation on these forms showed fidelity to the recommended protocol. While these pilot-test numbers are small, they indicate real implementation progress. Implementation of toileting trials appeared to be related to improved resident outcomes. Three of six champions reported

Course implementation: expectations and participant evaluations A premise of this study was that the CAPTA course’s design would facilitate implementation of best-practice toileting trials in participating nursing homes. With this in mind, and in contrast to its predecessor courses, the CAPTA course asked NHs to complete resident assessments within a shorter, but still feasible timeframe and to submit assessment forms as they were completed. As noted above, all NH champions reported that their staff completed toileting trials and all sites submitted at least some resident assessment forms. This represents an improvement over the previous courses, in which many but not all participating NHs (77%e86%) submitted some evidence of completed implementation assignments. This modest achievement in our view is reason enough to prefer this course’s design to the previous courses’ design. Asking NHs to submit resident assessment forms as NH staff completed these assignments allowed the course lecturer to give timely feedback based on actual results, which in turn may have provided some motivation for staff to complete subsequent assessments. This data-driven quality improvement process is not attainable when, as in the earlier courses, assignment results are summarized and reported weeks and sometimes months after the tasks are completed. It is notable that fewer NHs submitted fewer completed assessment forms as the course progressed. Several explanations may account for this trend: our a priori recommendation that NHs complete toileting trials with ten residents may have been too ambitious; NHs may have completed the assessments but champions did not submit the forms to us; or staff motivation to complete assessment tasks may have waned over time. One important limitation of this dissemination study is that we were unable to conduct onsite visits to independently verify completion of assessment tasks, but in light of our findings, this next step is recommended, for it could help shed new light on how NHs implement new interventions. To further facilitate completion of field assignment results, we modified the originally planned curriculum at the request of participants, as reported earlier. While we had not planned to modify the course schedule as we did, we had planned to elicit participants’ preferences as the course progressed and to make modifications as needed, as a strategy for improving outcomes. While we cannot be certain, this flexibility may have helped improve course results, for participant engagement in this course was stronger than it was in the earlier courses, as evidenced by the number of field assignment submissions. In other respects, the CAPTA courses performed much like the courses upon which it was modeled. As with those courses, satisfaction rates for the CAPTA course were high. All participants who evaluated the course reported that they would recommend it to colleagues and take a similar course again. Eighty percent said they preferred the webinar course to either a conference held in a single

6

A.N. Rahman et al. / Geriatric Nursing xx (2014) 1e7

location or a course completed entirely online. All but one responding champion echoed this training preference opinion. Additional course outcomes Course participation appeared to have sparked interest in institutionalizing evidence-based toileting trial assessments. Most champions (4 of 6) reported that their NH in-serviced staff on the toileting trial protocol, and all but one champion reported plans to continue or expand use of the protocol. These results are in keeping with findings reported for the previous courses.18,19 Also like its predecessor courses, this one was administered at low cost. By design, there were no travel, refreshment, site rental, or other fees that can make training conferences expensive to administer. Although total administration costs were lower for this course than for the earlier courses ($7000 vs. $26,510 and $17,464),18,19 enrollment also was lower (7 NHs vs. 14 and 34). As a result, per-facility costs for this course exceeded similar costs for one of the predecessor courses ($1000 vs. $514).19 Course costs per participating facility could be lowered by enrolling more NHs. Had the present course achieved capacity (at 20 NHs), per facility costs would have dropped considerably. Limitations Our findings cannot be generalized to all NHs, as the evaluated course enrolled a small sample of NHs in one state that volunteered to participate. Also, resident outcome and implementation findings must be viewed cautiously, for these results were self-reported by facility champions, even though de-identified assessment data also was shared with the project team to demonstrate implementation. Finally, our evaluation focused on initial adoption of toileting trials with a small group of incontinent residents, not long-term implementation and maintenance of the recommended toileting trial protocol facility-wide. A more extensive study, one that includes independent verification of implementation results, is needed to evaluate the long-term effectiveness of the course. However, these study limitations apply equally to most NH staff training efforts that are broadly distributed and not independently evaluated over time. Conclusion Notwithstanding these limitations, we believe that alternatives to more traditional educational programs are needed to prompt real quality improvement in NHs. The results of this study suggest that the evaluated course is a promising alternative to more traditional training formats, both in terms of its low cost and its immediate, short-term effectiveness. In our experience, evaluations of conferences aimed at healthcare professionals rarely include an evaluation of whether the information presented is actually translated into improved care practice or, indeed, any behavioral change. This course’s design allowed us to assess such change in nearly real time over the course’s ten-week duration. Results, while modest, showed that participants made progress in implementing the recommended protocol. Given this, we recommend that NH educator groups, including Quality Improvement Organizations and professional membership organizations, consider offeringdand continuously evaluatingdsimilarly designed courses. NH providers and residents stand to benefit from innovative training programs that are intentionally designed to speed implementation of recommended care practices.24 At the same time, course administrators stand to learn more about how NH staff actually implement evidence-based practices. With this in mind, educator groups should begin experimenting with new course designs, such

Table 4 Tips for designing and administering webinar/teleconference coaching courses.a Tailor course designs to the interventions advocated. In the toileting trial webinar course, for instance, we gave participants two weeks to complete resident interviews but four weeks to complete more time-intensive three-day toileting trials. In a different course that required nursing homes to in-service staff before new routines could be adopted, we allowed a month between webinars. Over-enroll nursing homes. With webinar and teleconference courses, expect one-third to a half of all registrants to drop out, often before they attend the first session.18,19,24 However, try not to enroll more participants than your team can manage. Based on past experience, we recommend enrollments under 35 nursing homes.18,19,24 Use user-friendly electronic communication tools. Some nursing home providers prefer to call-in to webinars by phone and to access PowerPoint presentations from their email (not online).19 Our advice: Keep it simple so that you do not inadvertently create technological barriers to participation. Treat participants like valued customers. Your job is to help NH providers do their jobs better. One small recommendation: Smile when you talk to participants on the phone, for it helps set a friendly tone. Strive to get to know participants. Keep notes of telephone calls and progress reports and use these to refresh your memory before upcoming phone calls or webinars. Email reminders about everything. including upcoming webinars, implementation assignments, due dates for evaluation forms, etc. Promote peer-sharing during webinars or teleconferences by calling on individual participants to recount their experiences. Peers can positively influence the adoption of new routines,16 but when participants cannot see each other, sharing may not occur unless the facilitator asks for it directly.25 Push assistance. In our courses, we always offer ongoing assistance by phone and email, and almost no one takes us up on these offers. If individualized assistance is needed to improve outcomes, don’t wait for participants to ask for help. Phone staff leaders and ask how the new practices are going and whether you can help them resolve any implementation problems. Find opportunities to offer positive feedback. When needed, offer critical feedback constructively. Continuously monitor and evaluate your program for the same reason that you want nursing homes to monitor and evaluate their programs: it is how we improve outcomes.26 This means you must collect evaluation data, but collect only what is absolutely necessary and then, if possible, only data that your participants need to do their jobs better (not yours).18 a These tips are based on findings from dissemination research and the authors’ experience administering six different webinar/teleconference courses, three of which have been described in detail elsewhere.18,19,24

as the webinar course evaluated here, that hold promise for improving outcomes without breaking the bank. To help spur innovation, Table 4 presents course design and administration tips that draw from our experience administering six such NH courses as well as lessons reported by other dissemination researchers. Acknowledgments We want to thank the nursing home staff that participated in this course and took time from their busy schedules to submit evaluation information. We learned as much from them as we hope they learned from us. This study was funded by a quality improvement award from the American Medical Directors Association. SCAN Health Plan provided CE credits for the course. Support for Dr. Rahman was provided by a grant from the National Institute on Aging (T32AG00037). References 1. Palmer MH, Johnson TM. Quality of incontinence management in U.S. nursing homes: a failing grade. J Am Geriatr Soc. 2003;51:1810e1812. 2. Lekan-Rutledge D. The new F-Tag 315. J Am Med Dir Assoc; 2006; Nov:607e610. 3. Hu TW, Wagner TH, Bentkover JD, et al. Costs of urinary incontinence and overactive bladder in the United States: a comparative study. Urology. 2004;63(3):461e465. 4. Centers for Medicare and Medicaid Services. Draft Minimum Data Set, Version 3.0 (MDS 3.0). Washington, DC: CMS; 2008.

A.N. Rahman et al. / Geriatric Nursing xx (2014) 1e7 5. Ouslander JG, Schnelle JF, Uman G, et al. Predictors of successful prompted voiding among incontinent nursing home residents. J Am Med Assoc. 1995;273(17):1366e1370. 6. RE-AIM.org. About Re-aim. http://www.re-aim.org/about_re-aim/index.html; Retrieved 02.01.14. 7. Schnelle JF. Treatment of urinary incontinence in nursing home patients by prompted voiding. J Am Geriatr Soc. 1990;38(3):356e360. 8. Vanderbilt Center for Quality Aging. Incontinence Management. https://www. mc.vanderbilt.edu/root/vumc.php?site¼cqa&doc¼43434; Retrieved 02.01.14. 9. Schnelle JF, Ouslander JG. Management of Incontinence in Long-term Care. Urinary Incontinence. St Louis: MO: Mosby-Year Book; 1996. 10. Ouslander JG. Quality improvement initiatives for urinary incontinence in nursing homes. J Am Med Dir Assoc. 2007;8(3 Suppl):S6eS11. 11. Schnelle JF, Cadogan MP, Yoshii J, et al. The minimum data set urinary incontinence quality indicators: do they reflect differences in care processes related to incontinence? Med Care. 2003;41(8):909e922. 12. Watson NM, Brink CA, Zimmer JG, et al. Use of the agency for health care policy and research urinary incontinence guideline in nursing homes. J Am Geriatr Soc. 2003;51:1779e1786. 13. Centers for Medicare and Medicaid Services. Revision of Appendix PPdSection 483.25(d)dUrinary Incontinence, Tags F315 and F316. Washington, DC: CMS; 2005. 14. Grimshaw JM, Shirran L, Thomas R, et al. Changing provider behavior: an overview of systematic reviews of interventions. Med Care. 2001;39(8 Suppl 2): II-2eII-45.

7

15. Rantz MJ, Zwygart-Stauffacher M, Hicks L, et al. Randomized multilevel intervention to improve outcomes of residents in nursing homes in need of improvement. J Am Med Dir Assoc. 2012;13(1):60e68. 16. Rogers E. Diffusion of Innovations. 5th ed. New York, NY: Free Press; 2003. 17. Senge P, Kleiner A, Roberts C, et al. The Dance of Change: The Challenges to Sustaining Momentum in Learning Organizations. New York: Doubleday; 1999. 18. Rahman A, Schnelle J, Yamashita T, et al. Distance learning: a strategy for improving incontinence care in nursing homes. Gerontol. 2010;50(1):121e132. 19. Rahman AN, Schnelle JF, Applebaum R, Lindabury K, Simmons S. Distance Coursework and coaching to improve nursing home incontinence care: the sequel. J Am Geriatr Soc. 2012;60(6):1157e1164. 20. Prince M. Does active learning work? A review of the research. J Eng Educ. 2004;93:223e231. 21. Michael J. Where’s the evidence that active learning works? Adv Physiol Educ. 2006;30:159e167. 22. California Healthcare Foundation. CalQualityCare.org: A Guide to Long-term Care in California. http://www.calqualitycare.org/; Retrieved on 23.01.12. 23. Lynn J, West J, Hausmann S, et al. Collaborative clinical quality improvement for pressure ulcers in nursing homes. J Am Geriatr Soc. 2007;55:1663e1669. 24. Rahman A, Applebaum R, Schnelle JF, et al. Translating research into practice in nursing homes: can we close the gap? Gerontologist. 2012;52(5):597e606. 25. Rahman AN, Simmons SF, Applebaum R, Lindabury K, Schnelle JF. The coach is in: improving nutritional care in nursing homes. Gerontologist. 2012;52(4):571e580. 26. Deming WE. Out of the Crisis. Cambridge, Massachusetts: Massachusetts Institute of Technology, Center for Advanced Engineering Study; 1986.