Behaviour Research and Therapy 53 (2014) 41e46
Contents lists available at ScienceDirect
Behaviour Research and Therapy journal homepage: www.elsevier.com/locate/brat
Cognitive Behavioral Therapy for insomnia with veterans: Evaluation of effectiveness and correlates of treatment outcomes Mickey Trockel a, b,1, Bradley E. Karlin c, d, *,1, C. Barr Taylor a, b, Rachel Manber a, b a
Sierra-Pacific Mental Illness Research, Education, and Clinical Center, Veterans Affairs Palo Alto Health Care System, USA Stanford University School of Medicine, USA c Mental Health Services, U.S. Department of Veterans Affairs Central Office, USA d Bloomberg School of Public Health, Johns Hopkins University, USA b
a r t i c l e i n f o
a b s t r a c t
Article history: Received 10 October 2013 Received in revised form 26 November 2013 Accepted 26 November 2013
This paper examines the effectiveness of Cognitive Behavioral Therapy for insomnia (CBT-I) in Veterans and the effects of two process measures on CBT-I outcomes: 1) therapist ratings of patient adherence and 2) patient ratings of therapeutic alliance. Data are from 316 therapists in the Department of Veterans Affairs CBT-I Training Program and 696 patients receiving CBT-I from therapists undergoing training. Mixed effects model results indicate Insomnia Severity Index scores decreased from 20.7 at baseline to 10.9 (d ¼ 2.3) during a typical course of CBT-I. Patients with highest tercile compared to those with lowest tercile adherence achieved, on average, 4.1 points greater reduction in ISI scores (d ¼ 0.95). The effect of therapeutic alliance on change in insomnia severity was not significant after adjusting for adherence to CBT-I. These results support the effectiveness and feasibility of large-scale training in and implementation of CBT-I and indicate that greater focus on patient adherence may lead to enhanced outcomes. The current findings suggest that CBT-I therapists and training programs place greater emphasis on attending to and increasing patient adherence. Published by Elsevier Ltd.
keywords: Cognitive Behavioral Therapy Insomnia Dissemination Veterans Adherence Alliance
Introduction National consensus statements in the United States and Great Britain recommend the use of Cognitive Behavioral Therapy (CBT) for the treatment of insomnia (“National Institutes of Health state of the science conference statement: Manifestations and management of chronic insomnia in adults June 13e15, 2005,” 2005; Wilson et al., 2010), based on a well-established body of efficacy research (Morin, Culbert, & Schwartz, 1994; Morin et al., 2009; Smith et al., 2002). CBT for insomnia (CBT-I) compared with standard pharmacotherapy for insomnia is equally effective in the short term and more effective in the long term (Jacobs, PaceSchott, Stickgold, & Otto, 2004). More recent evidence suggests that addition of hypnotic medication to CBT-I may improve outcomes in the short-term treatment of insomnia but that more optimal outcomes are achieved if the medication is discontinued
* Corresponding author. Mental Health Services (10P4M), U.S. Department of Veterans Affairs Central Office, 810 Vermont Avenue, NW, Washington, DC, 20420, USA. E-mail address: [email protected] (B.E. Karlin). 1 Co-first authors. 0005-7967/$ e see front matter Published by Elsevier Ltd. http://dx.doi.org/10.1016/j.brat.2013.11.006
and CBT-I alone is employed in the long term (Morin et al., 2009). Treatment of insomnia without hypnotic medication may be particularly advantageous in populations with high prevalence of disorders that may correlate with unfavorable treatment outcomes when sedative-hypnotic medications are prescribed, such as posttraumatic stress disorder (Van Minnen, Arntz, & Keijsers, 2002) or in populations with vulnerability to substance use problems, such as patients with bipolar disorder. There is some evidence of the effectiveness of CBT-I in routine sleep and non-sleep specialty practice settings. A recent report documented robust effectsdsimilar to those observed in controlled clinical trialsdamong Veterans receiving CBT-I from therapists participating in a competency-based CBT-I training program operated by the U.S. Department of Veterans Affairs (VA) health care system (Karlin, Trockel, Taylor, Gimeno, & Manber, 2013). The results of that paper, based on initial program evaluation data, suggested that non-sleep specialists can be trained to competency in CBT-I and that CBT-I can be successfully disseminated to routine practice settings and yield large improvements among real-world patients with often complex presentations. Less in known about the relationship between CBT-I outcomes and treatment process measures, particularly treatment adherence and therapeutic alliance. There is some evidence indicating that
42
M. Trockel et al. / Behaviour Research and Therapy 53 (2014) 41e46
patient adherence to CBT strategies predicts better insomnia treatment outcomes. For example, in the context of an intervention using time in bed restriction as a single component treatment, patients with greater consistency of time spent in bed per night and a more consistent arising time had better treatment outcomes in terms of time spent awake in the middle of the night and sleep efficiency (Riedel & Lichstein, 2001). In the context of group CBT-I, patients with higher therapist-rated adherence (rated at the end of treatment) had lower post-treatment sleep-related impairment (self-reported) and greater overall sleep quality (self-rated) but not significantly different outcomes on diary-based sleep parameters, such as sleep duration or efficiency (Vincent & Hameed, 2003). Results of another report indicated patients with higher selfreported adherence to the CBT-I strategies of “using bed only for sleep” and reducing time in bed (stimulus control and sleep restriction) had greater reductions in sleep latency and nighttime wakefulness (Harvey, Inglis, & Espie, 2002). A more recent investigation indicated greater adherence to CBT-I predicted better sleep outcomes among breast cancer survivors (Matthews, Schmiege, Cook, Berger, & Aloia, 2012). A large body of literature demonstrates that more robust therapeutic alliance predicts improved clinical outcome across a variety of psychotherapies (Castonguay, Constantino, & Holtforth, 2006; Horvath & Bedi, 2002), though there are very limited data on this relationship with CBT-I. One report examining the effects of therapeutic alliance on outcomes of group CBT-I indicated that patients who perceived their therapist as “critically confrontive” were more likely to drop out of treatment early and to be less satisfied with their treatment, if they stayed (Constantino et al., 2007). The same report also indicated that patients with low expectations of treatment (but not patients with high expectations) had better outcomes if they perceived their therapist as more “affiliative.” In contrast, when the therapeutic alliance was assessed with a nonvalidated single question it did not predict CBT-I outcome among breast cancer patients (Tremblay, Savard, & Ivers, 2009). Additional information on the relationship between outcomes and process measures, such as patient adherence and the therapeutic alliance, could help to inform CBT-I training efforts and provider practice in order to maximize outcomes. The purpose of the present paper is to examine whether patient adherence to CBT-I and the therapeutic alliance are significantly associated with patient outcomes. Specifically, the article aims to answer the following two key questions: 1) are high therapist ratings of patient adherence to CBT-I treatment components associated with greater reductions in patients’ insomnia severity? and 2) are high ratings of the therapeutic alliance by patients associated with greater reductions in patients’ insomnia severity? We also aim to supplement previously reported preliminary program evaluation results by using data from a substantially larger sample to examine the effectiveness of CBT-I provided by newly-trained therapists to Veterans in routine practice settings, and the extent to which therapists are able to attain competency and successfully complete training.
arm of VA e designed to promote the availability and fidelity of EBPs with Veterans (Karlin & Agarwal, 2013; Karlin & Cross, 2013). Details regarding the CBT-I Training Program and therapy protocol have been described elsewhere (Karlin et al., 2013; Manber et al., 2012). Briefly, the training begins with a 3-day CBTeI workshop. The workshop focuses on sleep regulation, theoretical foundation of etiology and treatment of insomnia, core CBTeI strategies, and implementation challenges. The workshop includes didactics, small- and large-group discussions, demonstration videos, and role-play skills practice with immediate feedback from expert CBTe I training consultants. Upon the completion of the workshop, clinicians begin treating patients with CBTeI and participate in 4 months of weekly 90-minutetelephone consultation sessions. The consultation sessions include groups of four consultees led by an expert CBTeI training consultant. Consultees are encouraged to treat at least 2 patients during the training. Consultation sessions include feedback from training consultants following review and rating of taped therapy sessions, role-play practice of specific CBTeI skills, and discussion of implementation challenges. CBTeI treatment The CBT-I therapy protocol consists of six sessions (an initial assessment and five treatment sessions), or fewer if a patient attains sufficient improvement and relapse prevention strategies can be implemented in fewer than six sessions. Treatment begins with a comprehensive sleep assessment and case conceptualization, which are used to guide selection and sequencing of CBTeI components. Treatment components include stimulus control, sleep restriction therapy, relaxation, and cognitive therapy. The final session focuses on maintenance of gains and relapse prevention. Clinicians incorporate forms and worksheets that were developed or adapted for the Training Program (Manber et al., 2012), including a measure of adherence discussed below. Clinician trainees Clinicians were eligible to participate in the CBTeI Training Program if they were licensed VA mental health staff, worked in settings where insomnia is a presenting issue and CBTeI can be implemented, and were able to attend the workshop and participate without interruption in the 4 months of weekly consultation. To successfully complete the Training Program, training participants must achieve a score 30 or higher on the CBT-I Competency Rating Scale (CRS), a standardized rating of audiotaped sessions we developed for the Training Program and have described elsewhere (Karlin et al., 2013). This is equivalent to an average per-item score of satisfactory. Program evaluation results on an earlier sample of training participants revealed that 92% of therapists achieved this criterion score and met other criteria for successful completion of training. There was also a significant increase in CBT-I-CRS competency ratings over the course of training (Karlin et al., 2013). Patients
Methods Training program description The goal of the VA CBT-I Training Program is to promote knowledge on the theoretical and applied components of CBT-I and to establish clinician competency to deliver CBTeI to Veterans with diverse presentations and comorbidities relatively common among Veteran patients. The CBT-I Training Program is part of a series of evidence-based psychotherapy training and dissemination initiatives within the Veterans Health Administration e the health care
Patients included Veterans meeting DSM-IV criteria for insomnia disorder (assessed by the treating clinicians) and presenting for treatment in a variety of mental health and primary care settings. Patients needed to agree to receive the treatment by therapists in training and consented to audio recording of sessions for training consultation purposes. Patients were excluded if they had severe daytime sleepiness or bipolar disorder. Patients with a history of drug abuse or dependence were required to have been substance free for at least 4 weeks. Patients with other mental health and medical comorbidities were not excluded, unless they
M. Trockel et al. / Behaviour Research and Therapy 53 (2014) 41e46
had uncontrolled medical or mental health conditions or were concurrently engaged in Prolonged Exposure Therapy (Foa, 2011) for PTSD (Manber et al., 2012). Measures Demographics Patients completed a demographic form at the first session including information regarding their age, gender, education level, race, and ethnicity. Insomnia severity Insomnia severity was measured with the Insomnia Severity Index (ISI), a validated measure of severity of insomnia that has been shown to reliably measure changes in insomnia severity (Morin, Belleville, Bélanger, & Ivers, 2011). Scores on the ISI may range from 0 to 28, with higher scores indicating greater insomnia severity. Recommended cutoffs on the ISI are as follows: 0e7: no clinically significant insomnia; 8e14: subthreshold insomnia; 15e 21: clinical insomnia, moderate; and 22e28: clinical insomnia, severe. Moderate improvement in insomnia severity is defined as a decrease in total ISI score of 8.4 points, and marked improvement is defined as a decrease of 9.9 points (Morin et al., 2011). The ISI was administered by clinicians before each session. Therapeutic alliance The Working Alliance InventoryeShort Revised (Hatcher & Gillaspy, 2006) was used to assess therapeutic alliance. The WAISR is a 12-item scale that measures three aspects of therapeutic alliance: goals (agreement on the goals of therapy), tasks (agreement on the tasks of therapy), and bond (the relational bond between client and therapist). The WAI-SR has demonstrated good psychometric properties in a variety of settings (Munder, Wilmers, Leonhart, Linster, & Barth, 2010; Perdrix, de Roten, Kolly, & Rossier, 2010). The WAI-SR was administered after sessions 1 and 3. Patient adherence to CBT-I Adherence was based on the Patient Adherence Form created for the VA CBT-I Training Program through an expert consensus process. At the end of each session, therapists rated the extent to which the patient had followed each of 6 different activities: going to bed only when sleepy, not going to bed before the prescribed item, getting out of bed at the prescribed time, limiting naps, getting out of when not able to sleep, and scheduling worry time when necessary. Each item was rated from 1 (no adherence) to 6 (complete adherence) or as NA (for methods that had not yet been introduced during the course of treatment). Chronbach’s alpha for the six-item scale was 0.84. Principle components analysis using session 2 data demonstrated that one “overall CBT-I adherence” component explained 57% of the variance in the set of six items and was the only component had an eigenvalue greater than 1.0. An overall adherence score was calculated on the averaged score on all six items for sessions 2e5. Scores were also divided into lowest, middle, and highest tercile. Data analysis Intent to treat analysis using mixed effects models (Raudenbush, 2002) was used to test all hypotheses. Analysis included available data from all 696 patients enrolled in CBT-I treatment provided by 316 therapists participating in the VA CBT-I Training Program. The initial mixed effects model was specified to test the effect of treatment on ISI scores during a typical course of six sessions (an assessment session and five treatment sessions) with random effects to account for the nested data structure of ISI scores across the
43
six sessions nested within individual patients and patients nested within therapists’ CBT-I practice during training consultation. Treatment time was coded as follows: session 1 ¼ 0; session 2 ¼ 0.2; session 3 ¼ 0.4; session 4 ¼ 0.6; session 5 ¼ 0.8 and session 6 ¼ 1.0. This treatment time variable was then squared for inclusion in a quadratic model specifying fixed effects for treatment time and treatment time squared to test the hypothesis that the rate of change in ISI scores decreases during the course of a typical six session course of CBT-I. In a subsequent model, fixed effects were added to test the effects on change in ISI scores of therapist rated patient adherence to CBT-I treatment components. Another model was specified to test the effects on change in ISI scores of the average therapeutic alliance score therapists receive from all available ratings provided by their cohort of patients. In addition, a model was specified to include fixed effects for both patient adherence and therapeutic alliance scores in the same mixed effects model, to test the hypothesis that each of these variables, after controlling for the other, accounts for variance in patients’ change in ISI scores during CBT-I treatment. A separate model was specified, using the sample of patients with available adherence scores, to test effects of therapeutic alliance scores on patient adherence scores. Cohen’s d effect sizes were estimated by dividing each relevant mixed model derived effect estimate by the baseline standard deviation of the dependent variable. Results Therapists Of 327 therapists who enrolled in CBT-I Training Program, 316 participated in the small-group consultation phase of training. Of these 316, 224 were women and 92 were men. The majorityd212 were psychologists, 73 were social workers, 18 were nurses, 11 were psychiatrists, and 2 did not indicate their professional discipline. Of the 316 therapists who began the small group consultation phase of training, 303 achieved all requirements for successful completion of CBT-I training. Patients During the consultation phase of their training, the 316 therapists provided CBT-I to 696 patients. Most (593) patients were men, 71 were women, and 32 did not indicate their gender. Patient age at initiation of treatment ranged from 22 to 85þ (Mean ¼ 52; SD ¼ 14). The majority of patients (477) identified themselves as white; 178 identified themselves as part of minority race, and 41 did not indicate their race. Six hundred one Veterans identified themselves as non-Hispanic; 57 identified themselves as Hispanic, and 38 did not indicate their ethnicity. In response to a question on highest education level achieved, 16 indicated less than high school education; 152 indicated high school education; 319 indicated some college; 95 were college graduates; 34 had attended some graduate school; 41 had a graduate degree, and 39 did not indicate their education level. Early in the course of treatment, 31 patients were deemed inappropriate for CBT-I by their therapist based on clinical eligibility guidelines (e.g., patients beginning Prolonged Exposure Therapy for PTSD). Another 19 Veterans initiated CBT-I too late to finish six sessions during the consultation phase of training. Of the remaining 646 patients, the majorityd432 (67%) e completed six or more sessions of CBT-I (n ¼ 656) or finished treatment (including a relapse prevention session) in fewer sessions due to symptom relief (n ¼ 76). Of the 214 patients who either did not complete treatment or were lost to follow-up, 157 dropped out of therapy or
44
M. Trockel et al. / Behaviour Research and Therapy 53 (2014) 41e46
could not attend sessions regularly; 36 were lost to follow-up because their therapist did not return requested forms, and 21 were lost to follow-up for unknown reasons. CBT-I treatment outcomes Mixed effects modeling intent to treat analysis estimates for treatment time and for treatment time squared were both statistically significant (see Table 1). The average total effect of treatment during a typical six session course of CBT-I is estimated by summing the effect of treatment time with the effect of treatment time squared. Therefore, the effect of treatment ¼ 12.3 þ 2.5 ¼ 9.8, indicating Veterans’ overall ISI score decreased from 20.7 at baseline to 10.9 (d ¼ 2.3) during a typical 6 session course of CBT-I treatment. The modest statistically significant effect of treatment time squared indicates the rate of change in ISI diminished modestly during the course of treatment (see Fig. 1). Among the patients who completed six sessions of CBT-I or finished early due to symptom relief, only one patient did not provide a final ISI score. Distributions of final ISI scores for the remaining 431 were: no clinically significant insomnia [ISI ¼ 0e7: n ¼ 150 (35%)], subthreshold insomnia [ISI ¼ 8e14: n ¼ 163 (38%)], moderate clinical insomnia [ISI ¼ 15e21: n ¼ 88 (20%)], severe clinical insomnia [ISI ¼ 22e28: n ¼ 30 (7%)]. Predictors of CBT-I treatment outcomes High therapist-rated patient adherence to CBT-I was associated with larger effects of treatment time on ISI and larger effects of treatment time squared on ISI (see Table 2). Considering both of these significant effects of adherence together, mixed effects model estimates indicate that, compared to patients with lowest tercile adherence ratings, patients who had high adherence ratings had, on average, 4.1 points greater ISI score reductions during CBT-I treatment (d ¼ 0.95). The difference in ISI score reductions during CBT-I treatment between patients with moderate adherence scores and patients with low adherence scores was not statistically significant. The interaction between therapists with highest tercile versus lowest tercile therapeutic alliance scores and treatment time was Table 1 Mixed effects model results: ISI scores across CBT-I treatment. Fixed effects:
Coefficient (SE)
SE
t (approximate df)
p-value
Intercept (average baseline score) Treatment time (Treatment time)2
21
0.18
114 (314)
Contents lists available at ScienceDirect
Behaviour Research and Therapy journal homepage: www.elsevier.com/locate/brat
Cognitive Behavioral Therapy for insomnia with veterans: Evaluation of effectiveness and correlates of treatment outcomes Mickey Trockel a, b,1, Bradley E. Karlin c, d, *,1, C. Barr Taylor a, b, Rachel Manber a, b a
Sierra-Pacific Mental Illness Research, Education, and Clinical Center, Veterans Affairs Palo Alto Health Care System, USA Stanford University School of Medicine, USA c Mental Health Services, U.S. Department of Veterans Affairs Central Office, USA d Bloomberg School of Public Health, Johns Hopkins University, USA b
a r t i c l e i n f o
a b s t r a c t
Article history: Received 10 October 2013 Received in revised form 26 November 2013 Accepted 26 November 2013
This paper examines the effectiveness of Cognitive Behavioral Therapy for insomnia (CBT-I) in Veterans and the effects of two process measures on CBT-I outcomes: 1) therapist ratings of patient adherence and 2) patient ratings of therapeutic alliance. Data are from 316 therapists in the Department of Veterans Affairs CBT-I Training Program and 696 patients receiving CBT-I from therapists undergoing training. Mixed effects model results indicate Insomnia Severity Index scores decreased from 20.7 at baseline to 10.9 (d ¼ 2.3) during a typical course of CBT-I. Patients with highest tercile compared to those with lowest tercile adherence achieved, on average, 4.1 points greater reduction in ISI scores (d ¼ 0.95). The effect of therapeutic alliance on change in insomnia severity was not significant after adjusting for adherence to CBT-I. These results support the effectiveness and feasibility of large-scale training in and implementation of CBT-I and indicate that greater focus on patient adherence may lead to enhanced outcomes. The current findings suggest that CBT-I therapists and training programs place greater emphasis on attending to and increasing patient adherence. Published by Elsevier Ltd.
keywords: Cognitive Behavioral Therapy Insomnia Dissemination Veterans Adherence Alliance
Introduction National consensus statements in the United States and Great Britain recommend the use of Cognitive Behavioral Therapy (CBT) for the treatment of insomnia (“National Institutes of Health state of the science conference statement: Manifestations and management of chronic insomnia in adults June 13e15, 2005,” 2005; Wilson et al., 2010), based on a well-established body of efficacy research (Morin, Culbert, & Schwartz, 1994; Morin et al., 2009; Smith et al., 2002). CBT for insomnia (CBT-I) compared with standard pharmacotherapy for insomnia is equally effective in the short term and more effective in the long term (Jacobs, PaceSchott, Stickgold, & Otto, 2004). More recent evidence suggests that addition of hypnotic medication to CBT-I may improve outcomes in the short-term treatment of insomnia but that more optimal outcomes are achieved if the medication is discontinued
* Corresponding author. Mental Health Services (10P4M), U.S. Department of Veterans Affairs Central Office, 810 Vermont Avenue, NW, Washington, DC, 20420, USA. E-mail address: [email protected] (B.E. Karlin). 1 Co-first authors. 0005-7967/$ e see front matter Published by Elsevier Ltd. http://dx.doi.org/10.1016/j.brat.2013.11.006
and CBT-I alone is employed in the long term (Morin et al., 2009). Treatment of insomnia without hypnotic medication may be particularly advantageous in populations with high prevalence of disorders that may correlate with unfavorable treatment outcomes when sedative-hypnotic medications are prescribed, such as posttraumatic stress disorder (Van Minnen, Arntz, & Keijsers, 2002) or in populations with vulnerability to substance use problems, such as patients with bipolar disorder. There is some evidence of the effectiveness of CBT-I in routine sleep and non-sleep specialty practice settings. A recent report documented robust effectsdsimilar to those observed in controlled clinical trialsdamong Veterans receiving CBT-I from therapists participating in a competency-based CBT-I training program operated by the U.S. Department of Veterans Affairs (VA) health care system (Karlin, Trockel, Taylor, Gimeno, & Manber, 2013). The results of that paper, based on initial program evaluation data, suggested that non-sleep specialists can be trained to competency in CBT-I and that CBT-I can be successfully disseminated to routine practice settings and yield large improvements among real-world patients with often complex presentations. Less in known about the relationship between CBT-I outcomes and treatment process measures, particularly treatment adherence and therapeutic alliance. There is some evidence indicating that
42
M. Trockel et al. / Behaviour Research and Therapy 53 (2014) 41e46
patient adherence to CBT strategies predicts better insomnia treatment outcomes. For example, in the context of an intervention using time in bed restriction as a single component treatment, patients with greater consistency of time spent in bed per night and a more consistent arising time had better treatment outcomes in terms of time spent awake in the middle of the night and sleep efficiency (Riedel & Lichstein, 2001). In the context of group CBT-I, patients with higher therapist-rated adherence (rated at the end of treatment) had lower post-treatment sleep-related impairment (self-reported) and greater overall sleep quality (self-rated) but not significantly different outcomes on diary-based sleep parameters, such as sleep duration or efficiency (Vincent & Hameed, 2003). Results of another report indicated patients with higher selfreported adherence to the CBT-I strategies of “using bed only for sleep” and reducing time in bed (stimulus control and sleep restriction) had greater reductions in sleep latency and nighttime wakefulness (Harvey, Inglis, & Espie, 2002). A more recent investigation indicated greater adherence to CBT-I predicted better sleep outcomes among breast cancer survivors (Matthews, Schmiege, Cook, Berger, & Aloia, 2012). A large body of literature demonstrates that more robust therapeutic alliance predicts improved clinical outcome across a variety of psychotherapies (Castonguay, Constantino, & Holtforth, 2006; Horvath & Bedi, 2002), though there are very limited data on this relationship with CBT-I. One report examining the effects of therapeutic alliance on outcomes of group CBT-I indicated that patients who perceived their therapist as “critically confrontive” were more likely to drop out of treatment early and to be less satisfied with their treatment, if they stayed (Constantino et al., 2007). The same report also indicated that patients with low expectations of treatment (but not patients with high expectations) had better outcomes if they perceived their therapist as more “affiliative.” In contrast, when the therapeutic alliance was assessed with a nonvalidated single question it did not predict CBT-I outcome among breast cancer patients (Tremblay, Savard, & Ivers, 2009). Additional information on the relationship between outcomes and process measures, such as patient adherence and the therapeutic alliance, could help to inform CBT-I training efforts and provider practice in order to maximize outcomes. The purpose of the present paper is to examine whether patient adherence to CBT-I and the therapeutic alliance are significantly associated with patient outcomes. Specifically, the article aims to answer the following two key questions: 1) are high therapist ratings of patient adherence to CBT-I treatment components associated with greater reductions in patients’ insomnia severity? and 2) are high ratings of the therapeutic alliance by patients associated with greater reductions in patients’ insomnia severity? We also aim to supplement previously reported preliminary program evaluation results by using data from a substantially larger sample to examine the effectiveness of CBT-I provided by newly-trained therapists to Veterans in routine practice settings, and the extent to which therapists are able to attain competency and successfully complete training.
arm of VA e designed to promote the availability and fidelity of EBPs with Veterans (Karlin & Agarwal, 2013; Karlin & Cross, 2013). Details regarding the CBT-I Training Program and therapy protocol have been described elsewhere (Karlin et al., 2013; Manber et al., 2012). Briefly, the training begins with a 3-day CBTeI workshop. The workshop focuses on sleep regulation, theoretical foundation of etiology and treatment of insomnia, core CBTeI strategies, and implementation challenges. The workshop includes didactics, small- and large-group discussions, demonstration videos, and role-play skills practice with immediate feedback from expert CBTe I training consultants. Upon the completion of the workshop, clinicians begin treating patients with CBTeI and participate in 4 months of weekly 90-minutetelephone consultation sessions. The consultation sessions include groups of four consultees led by an expert CBTeI training consultant. Consultees are encouraged to treat at least 2 patients during the training. Consultation sessions include feedback from training consultants following review and rating of taped therapy sessions, role-play practice of specific CBTeI skills, and discussion of implementation challenges. CBTeI treatment The CBT-I therapy protocol consists of six sessions (an initial assessment and five treatment sessions), or fewer if a patient attains sufficient improvement and relapse prevention strategies can be implemented in fewer than six sessions. Treatment begins with a comprehensive sleep assessment and case conceptualization, which are used to guide selection and sequencing of CBTeI components. Treatment components include stimulus control, sleep restriction therapy, relaxation, and cognitive therapy. The final session focuses on maintenance of gains and relapse prevention. Clinicians incorporate forms and worksheets that were developed or adapted for the Training Program (Manber et al., 2012), including a measure of adherence discussed below. Clinician trainees Clinicians were eligible to participate in the CBTeI Training Program if they were licensed VA mental health staff, worked in settings where insomnia is a presenting issue and CBTeI can be implemented, and were able to attend the workshop and participate without interruption in the 4 months of weekly consultation. To successfully complete the Training Program, training participants must achieve a score 30 or higher on the CBT-I Competency Rating Scale (CRS), a standardized rating of audiotaped sessions we developed for the Training Program and have described elsewhere (Karlin et al., 2013). This is equivalent to an average per-item score of satisfactory. Program evaluation results on an earlier sample of training participants revealed that 92% of therapists achieved this criterion score and met other criteria for successful completion of training. There was also a significant increase in CBT-I-CRS competency ratings over the course of training (Karlin et al., 2013). Patients
Methods Training program description The goal of the VA CBT-I Training Program is to promote knowledge on the theoretical and applied components of CBT-I and to establish clinician competency to deliver CBTeI to Veterans with diverse presentations and comorbidities relatively common among Veteran patients. The CBT-I Training Program is part of a series of evidence-based psychotherapy training and dissemination initiatives within the Veterans Health Administration e the health care
Patients included Veterans meeting DSM-IV criteria for insomnia disorder (assessed by the treating clinicians) and presenting for treatment in a variety of mental health and primary care settings. Patients needed to agree to receive the treatment by therapists in training and consented to audio recording of sessions for training consultation purposes. Patients were excluded if they had severe daytime sleepiness or bipolar disorder. Patients with a history of drug abuse or dependence were required to have been substance free for at least 4 weeks. Patients with other mental health and medical comorbidities were not excluded, unless they
M. Trockel et al. / Behaviour Research and Therapy 53 (2014) 41e46
had uncontrolled medical or mental health conditions or were concurrently engaged in Prolonged Exposure Therapy (Foa, 2011) for PTSD (Manber et al., 2012). Measures Demographics Patients completed a demographic form at the first session including information regarding their age, gender, education level, race, and ethnicity. Insomnia severity Insomnia severity was measured with the Insomnia Severity Index (ISI), a validated measure of severity of insomnia that has been shown to reliably measure changes in insomnia severity (Morin, Belleville, Bélanger, & Ivers, 2011). Scores on the ISI may range from 0 to 28, with higher scores indicating greater insomnia severity. Recommended cutoffs on the ISI are as follows: 0e7: no clinically significant insomnia; 8e14: subthreshold insomnia; 15e 21: clinical insomnia, moderate; and 22e28: clinical insomnia, severe. Moderate improvement in insomnia severity is defined as a decrease in total ISI score of 8.4 points, and marked improvement is defined as a decrease of 9.9 points (Morin et al., 2011). The ISI was administered by clinicians before each session. Therapeutic alliance The Working Alliance InventoryeShort Revised (Hatcher & Gillaspy, 2006) was used to assess therapeutic alliance. The WAISR is a 12-item scale that measures three aspects of therapeutic alliance: goals (agreement on the goals of therapy), tasks (agreement on the tasks of therapy), and bond (the relational bond between client and therapist). The WAI-SR has demonstrated good psychometric properties in a variety of settings (Munder, Wilmers, Leonhart, Linster, & Barth, 2010; Perdrix, de Roten, Kolly, & Rossier, 2010). The WAI-SR was administered after sessions 1 and 3. Patient adherence to CBT-I Adherence was based on the Patient Adherence Form created for the VA CBT-I Training Program through an expert consensus process. At the end of each session, therapists rated the extent to which the patient had followed each of 6 different activities: going to bed only when sleepy, not going to bed before the prescribed item, getting out of bed at the prescribed time, limiting naps, getting out of when not able to sleep, and scheduling worry time when necessary. Each item was rated from 1 (no adherence) to 6 (complete adherence) or as NA (for methods that had not yet been introduced during the course of treatment). Chronbach’s alpha for the six-item scale was 0.84. Principle components analysis using session 2 data demonstrated that one “overall CBT-I adherence” component explained 57% of the variance in the set of six items and was the only component had an eigenvalue greater than 1.0. An overall adherence score was calculated on the averaged score on all six items for sessions 2e5. Scores were also divided into lowest, middle, and highest tercile. Data analysis Intent to treat analysis using mixed effects models (Raudenbush, 2002) was used to test all hypotheses. Analysis included available data from all 696 patients enrolled in CBT-I treatment provided by 316 therapists participating in the VA CBT-I Training Program. The initial mixed effects model was specified to test the effect of treatment on ISI scores during a typical course of six sessions (an assessment session and five treatment sessions) with random effects to account for the nested data structure of ISI scores across the
43
six sessions nested within individual patients and patients nested within therapists’ CBT-I practice during training consultation. Treatment time was coded as follows: session 1 ¼ 0; session 2 ¼ 0.2; session 3 ¼ 0.4; session 4 ¼ 0.6; session 5 ¼ 0.8 and session 6 ¼ 1.0. This treatment time variable was then squared for inclusion in a quadratic model specifying fixed effects for treatment time and treatment time squared to test the hypothesis that the rate of change in ISI scores decreases during the course of a typical six session course of CBT-I. In a subsequent model, fixed effects were added to test the effects on change in ISI scores of therapist rated patient adherence to CBT-I treatment components. Another model was specified to test the effects on change in ISI scores of the average therapeutic alliance score therapists receive from all available ratings provided by their cohort of patients. In addition, a model was specified to include fixed effects for both patient adherence and therapeutic alliance scores in the same mixed effects model, to test the hypothesis that each of these variables, after controlling for the other, accounts for variance in patients’ change in ISI scores during CBT-I treatment. A separate model was specified, using the sample of patients with available adherence scores, to test effects of therapeutic alliance scores on patient adherence scores. Cohen’s d effect sizes were estimated by dividing each relevant mixed model derived effect estimate by the baseline standard deviation of the dependent variable. Results Therapists Of 327 therapists who enrolled in CBT-I Training Program, 316 participated in the small-group consultation phase of training. Of these 316, 224 were women and 92 were men. The majorityd212 were psychologists, 73 were social workers, 18 were nurses, 11 were psychiatrists, and 2 did not indicate their professional discipline. Of the 316 therapists who began the small group consultation phase of training, 303 achieved all requirements for successful completion of CBT-I training. Patients During the consultation phase of their training, the 316 therapists provided CBT-I to 696 patients. Most (593) patients were men, 71 were women, and 32 did not indicate their gender. Patient age at initiation of treatment ranged from 22 to 85þ (Mean ¼ 52; SD ¼ 14). The majority of patients (477) identified themselves as white; 178 identified themselves as part of minority race, and 41 did not indicate their race. Six hundred one Veterans identified themselves as non-Hispanic; 57 identified themselves as Hispanic, and 38 did not indicate their ethnicity. In response to a question on highest education level achieved, 16 indicated less than high school education; 152 indicated high school education; 319 indicated some college; 95 were college graduates; 34 had attended some graduate school; 41 had a graduate degree, and 39 did not indicate their education level. Early in the course of treatment, 31 patients were deemed inappropriate for CBT-I by their therapist based on clinical eligibility guidelines (e.g., patients beginning Prolonged Exposure Therapy for PTSD). Another 19 Veterans initiated CBT-I too late to finish six sessions during the consultation phase of training. Of the remaining 646 patients, the majorityd432 (67%) e completed six or more sessions of CBT-I (n ¼ 656) or finished treatment (including a relapse prevention session) in fewer sessions due to symptom relief (n ¼ 76). Of the 214 patients who either did not complete treatment or were lost to follow-up, 157 dropped out of therapy or
44
M. Trockel et al. / Behaviour Research and Therapy 53 (2014) 41e46
could not attend sessions regularly; 36 were lost to follow-up because their therapist did not return requested forms, and 21 were lost to follow-up for unknown reasons. CBT-I treatment outcomes Mixed effects modeling intent to treat analysis estimates for treatment time and for treatment time squared were both statistically significant (see Table 1). The average total effect of treatment during a typical six session course of CBT-I is estimated by summing the effect of treatment time with the effect of treatment time squared. Therefore, the effect of treatment ¼ 12.3 þ 2.5 ¼ 9.8, indicating Veterans’ overall ISI score decreased from 20.7 at baseline to 10.9 (d ¼ 2.3) during a typical 6 session course of CBT-I treatment. The modest statistically significant effect of treatment time squared indicates the rate of change in ISI diminished modestly during the course of treatment (see Fig. 1). Among the patients who completed six sessions of CBT-I or finished early due to symptom relief, only one patient did not provide a final ISI score. Distributions of final ISI scores for the remaining 431 were: no clinically significant insomnia [ISI ¼ 0e7: n ¼ 150 (35%)], subthreshold insomnia [ISI ¼ 8e14: n ¼ 163 (38%)], moderate clinical insomnia [ISI ¼ 15e21: n ¼ 88 (20%)], severe clinical insomnia [ISI ¼ 22e28: n ¼ 30 (7%)]. Predictors of CBT-I treatment outcomes High therapist-rated patient adherence to CBT-I was associated with larger effects of treatment time on ISI and larger effects of treatment time squared on ISI (see Table 2). Considering both of these significant effects of adherence together, mixed effects model estimates indicate that, compared to patients with lowest tercile adherence ratings, patients who had high adherence ratings had, on average, 4.1 points greater ISI score reductions during CBT-I treatment (d ¼ 0.95). The difference in ISI score reductions during CBT-I treatment between patients with moderate adherence scores and patients with low adherence scores was not statistically significant. The interaction between therapists with highest tercile versus lowest tercile therapeutic alliance scores and treatment time was Table 1 Mixed effects model results: ISI scores across CBT-I treatment. Fixed effects:
Coefficient (SE)
SE
t (approximate df)
p-value
Intercept (average baseline score) Treatment time (Treatment time)2
21
0.18
114 (314)
