Journal of Counseling Psychology 2014, Vol. 61, No. I, 15-23

© 2013 American Psychological Association 0022-0167/l4/$12.00 DOI: I0.1O37/a0034939

Client and Therapist Therapeutic Alliance, Session Evaluation, and Client Reliable Change: A Moderated Actor-Partner Interdependence Model Dennis M. Kivlighan Jr.

Cheri L. Marmarosh

University of Maryland

The George Washington University

Mark J. Hilsenroth Adelphi University Actor-partner interdependence modeling (APIM; Kashy & Kenny, 2000) was used to study the early therapeutic alliance in 74 clients being treated by 29 therapists to explore the relationship between the alliance and treatment progress, while prioritizing the dyadic nature of the alliance. The APM examines collaboration/ influence by modeling the impact of one dyad member's alliance ratings on the other member's session impact rating (partner effects). In terms of the alliance, the results revealed significant client-actor effects for client ratings of session depth and positivity as well as significant therapist-actor effects for therapist ratings of session smoothness and positivity. For client-rated alliance, there were also significant client-partner effects for therapist ratings of session depth. For clients who made a reliable change in treatment, an increase in client-reported alliance was related to therapist reporting more arousal in the 3rd session. For clients who did not make a reliable change in treatment, client-reported alliance was not related to therapist arousal. Limitations of the study and implications of the findings are discussed. Keywords: therapeutic alliance, session evaluation, actor-partner interdependence model

The therapeutic alliance is one of the most examined aspects of the therapeutic relationship. In meta-analyzing the results of over 30 years of research, Horvath, Del Re, Fliickiger, and Symonds (2011) found that the therapeutic alliance was a significant predictor of therapy outcome, accounting for approximately 8% of the variance in treatment outcomes. Theoretically, the therapeutic alliance should be differentially related to types of outcomes. In differentiating the therapeutic alliance and the real relationship, Gelso (2011) stated that the therapeutic alliance captured the working aspects of the therapist and client relationship, whereas the real relationship reflected the personal, nonworking aspect of this relationship. Therefore, the therapeutic alliance should show a stronger relationship to outcomes that reflect the work of therapy and a weaker relationship to outcomes that reflect the personal aspects of therapy. Researchers have not used this work versus personal distinction in their studies examining the relationship between the therapeutic alliance and outcome. Although one of the moderator analyses results in the Horvath et al. (2011) metaanalysis hints at this distinction. These authors found that the effect sizes for the relationship between therapeutic alliance and change

in depression and symptoms were .42 and .47, respectively, whereas the effect size for the relationship between therapeutic alliance and dropout was .18. However, only the depression effect size was signiflcantly different from the dropout effect size. Changes in depression and symptoms may reflect the working aspect of treatment, whereas dropout may reflect the personal relationship between the client and therapist. For example, Tryon and Kane (1995) found that therapist's ratings of how well clients related at intake were related to the client terminating unilaterally. Therefore, there may be an important theoretical and empirical distinction between outcomes that reflect the work of therapy and outcomes that reflect the personal relationship in therapy. As noted above, however, there are few differential relationships between the therapeutic alliance and type of outcome probably because outcome is usually measured at termination, which may be too distal to capture the subtleties of the therapeutic alliance-outcome relationship (e.g.. Hill & Lambert, 2004). Boswell, Castonguay, and Wasserman (2010) argued that looking at session outcome allows for a closer look at what comes between process (the therapeutic alliance in a session) and long-term outcome (the effects of a series of sessions or of a whole treatment). In line with this observation, Owen, Quirk, Hilsenroth, and Rodolfa (2012) examined how the therapeutic alliance related to session-level ratings of client intersession experience. They found that the alliance was significantly related to therapeutic work ("applying therapy") between sessions but was unrelated to the personal therapy relationship (imagining dialogues with the therapist) between sessions. In the present study, we build on the Owen et al. (2012) study by examining the relationship between the therapeutic alliance and session outcome.

This article was published Online First November 4, 2013. Dennis M. Kivlighan Jr., Department of Counseling and Personnel Services, University of Maryland; Cheri L. Marmarosh, Department of Psychology, The George Washington University; Mark J. Hilsenroth, The Demer Institute of Advanced Psychological Studies, Adelphi University. Correspondence conceming this article should be addressed to Dennis M. Kivlighan Jr., Department of Counseling and Personnel Services, 3214 Benjamin Building, University of Maryland, College Park, MD 20742. E-mail: [email protected] 15

16

KIVLIGHAN, MARMAROSH, AND HILSENROTH

One of the most widely used measures of session outcome is the Session Evaluation Questionnaire (SEQ; Stiles & Snow, 1984), which measures session outcome on the dimensions of Depth, Smoothness, Positivity, and Arousal. Several studies have examined the relationship between the Depth and/or Smoothness dimensions of session evaluation and the therapeutic alliance. Kivlighan (2007) found that counselors' and clients' ratings of the alliance were related to their own ratings of session depth and smoothness. Kim, Ng, and Ann (2009) only measured session depth and found a strong relationship between therapeutic alliance and session depth for Asian American clients. However, Gelso et al. (2005) found no relationship between client-rated therapeutic alliance and client-rated session depth and smoothness. Only one study (Mallinckrodt, 1993) examined the relationship between therapeutic alliance and all four dimensions of session outcome operationalized in the SEQ. In this complex study, alliance and session outcome were rated by both counselors and clients at different points in treatment. The one consistent flnding in this study was that early client ratings of the therapeutic alliance were significantly and positively related to client-rated session depth at early, mid, and late treatment. No other consistent relationships between therapeutic alliance (either client- or therapistrated) and session outcome (either client- or therapist-rated) emerged. There is some evidence, however, that specific SEQ scales may capture either the work or personal aspect of session outcome. In a large study examining the relationship between session evaluations and session impacts. Stiles et al. (1994) found that Depth and Arousal were more strongly related to workoriented impacts (Understanding and Problem Solving) and that Smoothness and Positivity were more strongly related to the relationship oriented impact (Relationship). Given these relationships we hypothesized that: Hypthothesis 1: Client and therapist ratings of the therapeutic alliance will be significantly and positively related to their ratings of session Depth and Arousal.

The Therapeutic Alliance: Collahoration and Mutual Influence There appears to be a consensus across theoreticians and researchers that collaboration, working together, and reciprocity are the core features of the therapeutic alliance (Hatcher & Barends, 2006; Horvath et al., 2011; Horvath & Symonds, 1991). Researchers can measure perceptions of collaboration and reciprocity or model collaboration and reciprocity statistically. This perception versus statistical distinction is similar to the objective versus subjective distinction made by researchers examining similarity in groups (e.g., Dunlop & Beauchamp, 2011). Most research on the therapeutic alliance involves perceptions of collaboration and reciprocity, relying on the wording of the therapeutic alliance inventory to get at the client's and therapist's perceptions of collaboration and reciprocity. Some items on therapeutic alliance inventories do assess perceptions of collaboration (e.g., "My therapist and I are working towards mutually agreed-upon goals"); however, other items assess individual perceptions (e.g., "I believe my therapist likes me"). Alternatively, collaboration and reciprocity can be assessed using the actor-partner interdependence model (APIM; Ledermann & Kenny, 2012).

The APIM has two types of effects: actor effects and partner effects. Actor effects describe the relationship between a participant's own predictor and her or his outcome (e.g., the therapist's therapeutic alliance and the therapist's session evaluation). Partner effects describe the relationship between a participant's own predictor and her or his partner's outcome (e.g., the therapist's therapeutic alliance and the client's session evaluation). The APIM also specifies two correlations: one describes the dependency between clients' and therapists' therapeutic alliance ratings and the other describes the dependency between clients' and therapists' session evaluation ratings. In the APIM, interpersonal influence is deflned as a signiflcant partner effect, whereas mutual influence is deflned as both partner effects being significant. Kivlighan (2007) argued that therapeutic alliance studies rarely statistically examine collaboration or mutual influence because the combined contributions of both therapists' and clients' therapeutic alliance ratings are not assessed in the same study. In the Horvath et al. (2011) meta-analysis, there were 112 independent effects for client-rated therapeutic alliance, but only 23 independent effects for therapist-rated therapeutic alliance. Horvath et al. did not report how many of the outcomes examined involved the clients' or the therapists' perspective. It is likely, however, that most of the studies involved only client ratings of outcome. Therefore, the therapeutic alliance-outcome literature appears to involve mostly an examination of the clients' perceptions of both the therapeutic alliance and treatment outcome (clientactor effects). Only two published studies used the APIM to examine the therapeutic relationship in individual counseling. Gelso et al. (2012) found that clients' ratings of the real relationship were related to their own rating of outcome (a client-actor effect) and to their therapist's ratings of outcome (a client-partner effect). In the study that is the most relevant for the present research, Kivlighan (2007) used the APIM to examine the relationship between clients' and therapists' ratings of the working alliance and their ratings of session Depth, a session's perceived power or value, and Smoothness, a session's comfort or pleasantness (Stiles & Snow 1984). He found that clients' and therapists' ratings of the therapeutic alliance were significantly related to their own ratings of session Depth and Smoothness (significant client- and therapist-actor effects). He also found that therapists' ratings of the therapeutic alliance were significantly related to clients' ratings of session Depth (a therapist-partner effect). One purpose of the present study was to replicate and extend Kivlighan's (2007) study in several important ways. We replicate the earlier study by using the APIM to once again examine the relationship between clients' and therapists' ratings of the working alliance and their ratings of session Depth and Smoothness. As noted above, the SEQ, however, includes two additional subscales: Arousal, the experience of excitement or vitality after a session, and Positivity, the experence of positive emotions after a session, but these subscales were not examined in Kivlighan (2007). Therefore, we extend the Kivlighan (2007) study by examining the Positivity and Arousal dimensions of session evaluation as rated by both clients and therapists. As noted above, the vast majority of alliance research has involved an examination of client-actor effects (i.e., the client's rating of alliance and the client's rating of either treatment or session outcome), yet very few of these studies have controlled for

CLIENT AND THERAPIST ALLIANCE

partner effects. Despite the lack of controlling for partner effects, we believe that the alliance studies and alliance theory support the existence of actor effects. On the basis of our analysis of workoriented versus personal-oriented outcomes, we made the following hypotheses: Hypothesis 2a: Clients' ratings of alliance will be significantly related to clients' ratings of session Depth and Arousal, when controlling for therapists' partner effects. Hypothesis 2b: Therapists' ratings of alliance will be significantly related to therapists' ratings of session Depth and Arousal, when controlling for clients' partner effects. Eugster and Wampold (1996) found that clients and therapists take different factors into account when making judgments about session outcome because of their differing role perspectives. Sullivan (1954) also described differences in the therapist's and client's roles in terms of the therapist's role as a participant observer. As a participant observer, the therapist observers the client's behavior and participates as a significant other in the counseling relationship. This participant observer role can be seen in Lent et al.'s (2006) suggestion that therapists' session evaluations are related to their own experiences of the counseling relationship and their inferences about their client's experiences of the counseling relationships. Therefore, as a participant observer, counselors' session evaluations should be related to their own and their client's perceptions of the alliance. The client's role, however, in counseling is to be a full participant in the counseling relationship. Therefore, clients tend to be more self-focused, using their perceptions of the alliance as evidence for deciding their evaluations of sessions. The Kivlighan (2007) study did not support this theoretical model, because he found partner effects only for the therapists. However, the Gelso et al. (2012) study did reveal significant client-partner effects. Despite these contradictory findings, we base our hypotî'eses concerning differences between clients and counselors in terms of the participant and observer roles, and we specifically hypothesized that: Hypothesis 3: Clients' ratings of alliance (client-partner effects) will be significantly related to therapists' session evaluations (greater Depth and Arousal), when controlling for clients' actor effects.

Therapeutic Alliance, Session Evaluations, and Outcome It is clear that therapeutic process differs when clients experience a more or less successful outcome (e.g., Gelso, Kivlighan, Wine, Jones, & Friedman, 1997). Therefore, it is important to examine how the actor-partner relationships described above vary as a function of the client's outcome status. We predict that the client-partner effect described in Hypothesis 3 will be stronger when the client has a better outcome. Specifically, we expected that client symptom change would moderate the relationship between clients' alliance ratings and therapists' session evaluations. This is because theory and research suggest that those therapists who are better at monitoring the state of their clients' alliances (as evidenced by a significant partner effect for client alliance ratings) will be more successful (Richards, 2011). Therefore, we made the final hypothesis:

17

Hypothesis 4: There will be significant client-partner effects of therapeutic alliance on session depth and arousal when clients report a reliable change and nonsignificant clientpartner effects when there is no reliable change. The APIM, depicted in Figure 1, contains two actor effects: (a) client alliance and session evaluation and (b) therapist alliance and session evaluation; two partner effects: (a) client alliance and therapist session evaluation and (b) therapist alliance and client session evaluation; and the effect for the reliable change index (RCI). The model also contains four interaction effects: (a) clientactor by RCI, (b) therapist-actor by RCI, (c) client-partner by RCI, and (d) therapist-partner by RCI. The APIM accounts for the nesting of clients and therapists within dyads by specifying a correlation between clients' and therapists' session evaluations.

Method Participants Clients. This study is an extension of Pésale, Hilsenroth, and Owen (2012). There were 74 clients receiving individual psychotherapy in the Psychodynamic Psychotherapy Treatment Team at a community outpatient psychological clinic and provided written informed consent for this research. For a complete description of the clients, see Pésale et al. (2012). Therapists. Twenty-nine (14 male, 15 female) third- or fourth-year doctoral students enrolled in an American Psychological Association-approved Clinical Psychology Ph.D. program conducted the psychological assessment and psychotherapy. One of these therapists was a racial/ethnic minority, and the rest were Caucasian. Five therapists saw one patient, eight therapists saw two patients, 11 therapists saw three clients, two therapists saw four clients, and one therapist saw five clients. See Pésale et al. (2012) for a description of these therapists. Treatment. Clients first received a psychological evaluation from a therapeutic model of assessment (see Hilsenroth, 2007, for

Oient Alliance

TTierapistAiriarKe

Oient Session Evaluation

Reliable Charge Imtex

aient AlliarceX RCI

TherapistSessiOTi EwJuation

Therapst Alliance XRQ

Figure 1. Actor-partner interdependence model for client and therapist alliance and client and therapist session evaluation and client reliable change. Reliable change index (RCI) is calculated from the Global Severity Index of the Brief Symptom Inventory. The intercorrelations among the five predictor variables are not depicted in the figure.

18

KIVLIGHAN, MARMAROSH, AND HILSENROTH

details). The clinician who carried out the psychological assessment also conducted the psychotherapy sessions. Psychotherapy consisted of once- or twice-weekly sessions of short-term psychodynamic psychotherapy (see Hilsenroth, 2007) and was openended. Blagys and Hilsenroth (2000) describe the key features of this treatment model.

Measures The SEQ (Stiles et al., 1994; Stiles & Snow, 1984). The SEQ is a measure of in-session psychotherapy process that consists of 24 bipolar adjective scales rated from 1 (e.g., weak) to 7 (e.g., powerful). It is separated into two sections, each consisting of 12 bipolar scales. Factor analysis revealed two evaluation indices— Depth and Smoothness—and two postsession mood indices— Positivity and Arousal (Stiles et al., 1994; Stiles & Snow, 1984). Previous research has revealed alphas ranging from .74 to .91 for these four SEQ subscales using a subset of the current participants (Ackerman, Hilsenroth, Baity, & Blagys, 2000). The validity ofthe SEQ has been demonstrated by significant correlations between SEQ scales and measures of therapist quality, session impact, and global session evaluations (Stiles et al., 1994). Combined Alliance Short Form-Patient Version (CASF-P; Hatcher & Barends, 1996). The CASF-P is a client-rated alliance measure consisting of 20 items rated on a 7-point scale where 1 = never, 2 = rarely, 3 = occasionally, 4 = sometimes, 5 = oflen, 6 = very often, and 7 = always. Hatcher and colleagues (Hatcher & Barends, 1996; Hatcher, Barends, Hansell, & Gutfreund, 1995) reported evidence on both the construct and incremental validity of this scale with regard to outcome. In addition, both Ackerman et al. (2000) and Clémence, Hilsenroth, Ackerman, Strassle, and Handler (2005) reported convergent validity data with related measures of psychotherapy process as well as criterion validity with regard to the prediction of treatment outcome using a sample of clients at the same university-based clinic as the clients in the present study, as well as a coefficient alpha of .91. Working Alliance Inventory-Therapist Version (WAI-T; Horvath & Greenberg, 1989). The WAI-T is a therapist-rated alliance measure. The Total scale score for the WAI-T used in this study was derived from a recent psychometric adaptation (Hatcher, 1999) using responses from two samples. The first was a national sample consisting of practicing therapists' ratings on one patient from their current practice iN = 251). The second was a clinical sample consisting of 63 therapists who completed ratings on 259 different patients. Previous research has revealed alphas ranging from .75 to .86 (Hatcher, 1999), and examining a subset of the current participants, coefficient alphas range from .74 to .91 (Clémence et al, 2005). Ratings on the WAI-T are reported on the same 7-point scale as described for the CASF-P, ranging from 1 (never) to 7 ialways). Brief Symptom Inventory (BSI; Derogatis, 1993). The BSI is a 53-item self-report inventory that assesses symptom distress in a number of different domains/problem areas using a Likert scale ranging from 0 {not at all) to 4 iextremely) and was collected at pre- and posttreatment. The psychometric properties, reliability, and validity of this measure, as well as a description of specific symptom subscale scores and a summary score, the Global Severity Index (GSI), are provided in the manual (Derogatis, 1993). The mean GSI for a normal population iN = 719, nonpatients) was

0.30 (5D = 0.31), and test-retest reliability using an outpatient sample was .90. The mean pretreatment GSI for the present sample was 1.06 iSD = 0.59). Reliable change (RC) was calculated using formulas described in Jacobsen and Truax (1991) and Jacobson, Roberts, Berns, and McGlinchy (1999). A client was classified as having made an RC if her or his RCI was greater than 1.96 (coded as 1) and as not having made an RC if her or his RCI was less than or equal to 1.96 (coded as 0).

Procedure Clients filled out the BSI prior to starting treatment. Client and therapist SEQ and therapeutic alliance ratings were collected from either the third or fourth treatment session. Clients were informed, both verbally and in writing, that their therapists would not have access to their responses on these session measures. The mean number of sessions in this sample was 31 iSD = 22), and the post-BSI was given at termination. Further details ofthe measures, methodology, and procedures used are described more fully elsewhere (Hilsenroth, 2007).

Data Analysis We used path analysis within a structural equation modeling framework to analyze the APIM (Kenny, Mannetti, Pierro, Livi, & Kashy, 2002). All variables were mean-centered before creating interaction terms.

Results The means and standard deviations for therapists' and clients' ratings of alliance, session Depth, session Smoothness, Positivity, and Arousal are displayed in Table 1. Clients' mean GSI posttreatment scores (0.67, SD = .62) were significantly smaller than their pretreatment scores (1.01, SD = .58), f(73) = -5.A5,p < .0001. Of the 74 clients, 47 (63.51%) did not make a reliable change (RCI < 1.96), and 27 (36.48%) did make a reliable change (RCI > 1.96). Because there was a good deal of variability in the number of treatment sessions, we examined the relationship between number of sessions completed and client reliable change. The point biserial correlation for number of treatment sessions and reliable change was .06 {p = .61). Therefore, reliable change was not confounded with sessions completed. Because therapists saw more than one client, we initially used a two-level hierarchical linear modeling (Raudenbush, Bryk, Cheong, & Congdon, 2005) analysis to test for therapist effects for Table 1 Means and Standard Deviations for Therapists' and Clients' Alliance Ratings and Session Evaluations Early in Treatment Therapists

Clients Variable

M

SD

M

SD

Alliance Depth Session Smoothness Positivity Arousal

6.15 5.51 4.86 4.92 3.97

0.63 0.87 1.12 1.08 1.08

5.75 5.02 4.65 5.23 4.48

0.44 0.911 1.00 0.92 1.13

CLIENT AND THERAPIST ALLIANCE the session evaluation variables. Therapists who only saw one client were excluded in these tests of therapist effects on client and therapist SEQ ratings. We conducted eight empty models to determine the variance in SEQ ratings accounted for by therapists. The variance estimates from these eight models were used to calculate intraclass correlation coefficients (ICCs). The ICCs for client session evaluation ratings were Depth (.06; p = .10), Smoothness (.10;p = .08), Positivity (.17; p = .02), and Arousal (.002; p = .42). The ICCs for therapist session evaluation ratings were Depth (.07; p = .09), Smoothness (.01; p = .21), Positivity (.06; p = .10), and Arousal (.05; p = .12). Because the therapist variance for only one of the SEQ ratings was significant, we did not model therapists in our main analyses. Data were analyzed using full information maximum likelihood estimation in EQS. Model fit statistics are not reported because the APIM model is saturated. Table 2 displays the results of the APIM analyses for alliance and session evaluation. There were significant client-actor effects for session Depth ib = .59, p < .05) and for Positivity ib = .47, p < .05). When clients had higher alliance ratings, they also had higher session depth and positivity ratings. The therapist-actor effects for session Smoothness ib = .76, p < .05) and session Positivity (¿ = .75, p < .05) were significant. When therapists had higher alliance ratings, they also had higher session Smoothness and Positivity ratings. None of the therapistpartner effects were significant. The client-partner effect for session Depth ib = .46, p < .05) was significant. When clients had higher alliance ratings, their therapists rated the sessions as deeper. There was a significant relationship between clients' early treatment Arousal ratings and client end-of-treatment reliable change ib = - .64, p < .05). When clients had higher arousal ratings early in treatment, they were less likely to subsequently show a reliable change. Clients' early treatment alliance ratings and client reliable change interacted to predict therapist early treatment arousal ib = 1.10, p < .05). The form of this interaction is displayed in Figure 2. The simple slope for clients who made a reliable change was 0.90 ip < .05). The simple slope for clients who did not make a reliable change was -0.04 (p > .05). As seen in the figure, when therapists had clients who did not make a reliable change at the end of treatment, there was no relationship between client alliance ratings and therapist arousal early in psychotherapy. When clients

19

Low Client Therapeutic Alliance High Client Therapeutic Alliance

Figure 2. Interaction of client alliance and client reliable change in predicting therapist arousal.

made a reliable change at the end of treatment, clients' alliance ratings and therapists' arousal ratings early in treatment also increased. The APIM variables accounted for 17% (p < .05) of the variance in clients' ratings of session Depth and 14% ip < .05) of the variance in therapists' ratings of session Depth. Clients' and therapists' ratings of session Depth were not significantly correlated (r = .14, p > .05). The APIM variables accounted for 10% ip < .05) of the variance in clients' ratings of session Smoothness and 15% ip < .05) of the variance in therapists' ratings of session Smoothness. Clients' and therapists' ratings of session Smoothness were not significantly correlated (r = -.04, p > .05). The APIM variables accounted for 13% ip < .05) of the variance in clients' ratings of Positivity and 26% ip < .05) of the variance in therapists' ratings of Positivity. Clients' and therapists' ratings of Positivity were not significantly correlated (/• = - . 0 1 , p > .05). The APIM variables accounted for 16% (p > .05) of the variance in clients' ratings of Arousal and 12% ip < .05) of the variance in therapists' ratings of Arousal. Clients' and therapists' ratings of Arousal were not significantly correlated (r = —.04, p > .05). In all four APIM analyses, clients' and therapists' ratings of the alliance were not significantly correlated (r = —.10, p > .05).

Table 2 Regression Coefficients, Standard Errors, and t Statistics for the Actor-Partner Interdependence Model for Client and Therapist Alliance and Client and Therapist Session Evaluation With Partner-Moderated Actor and Partner Effects Therapist alliance

Client alliance Variable Client Depth Therapist Depthi Client Smoothness Therapist Smoothness Client Positivity Therapist Positivity Client Arousal Therapist Arousal

b .59 .46 .34

.19 .47 .22 -.08 -.04

Reliable change index (RCI)

Client X RCI

Therapist X RCI

t

b

t

b

/

b

t

b

/

3.182.31* 1.35 .87 1.97* 1.19 -.32 -.17

.12 .33 .59 .76 .11 .75 .31 .16

.43 1.11 1.61 2.43* JO 2.70* .90 .43

.02 .04 .28 .35 .33 .23 -.64 .20

.12 .17 1.09 1.53 1.33 1.18 -2.65' .76

-.13 -.37 .09 -.64 .16 -32 .37 1.01

-.42 -1.09 .22 -1.76 .40 -1.02 .94 2.41*

.09 .37 -.93 -.27 .08 .53 .73 -.08

.21 1.00 -1.61 -.53 .14 1.22 1.34 -.14

Note. RCI = Global Severity Index of the Brief Symptom Inventory-reliable change index coded 0 for no reliable change attained and 1 for reliable change attained. Bold type = partner effects; regular type = actor effects. *p < .05.

KIVLIGHAN, MARMAROSH, AND HILSENROTH

20 Discussion Actor Effects

As hypothesized, clients' alliance ratings were related to their ratings of session Depth but unexpectedly also to their ratings of Positivity. Also contrary to our expectations, therapists' alHance ratings were related to their ratings of session Smoothness and Positivity. The findings for client Depth and for therapist Smoothness replicate results of Kivlighan (2007). However, he also found that therapists' alliance ratings were significantiy related to their ratings of Depth and that the clients' alliance ratings were related to their ratings of Smoothness. Across the two studies, clients appear to consistently associate the therapeutic alliance with the work-oriented outcome of session Depth. However, their therapist consistently associated the therapeutic alliance with the personaloriented outcome of session smoothness. These results confirm Eugster and Wampold's (1996) findings that clients and therapists take somewhat different factors into account when making judgments about the session. As far as we can tell, this was the ñrst study to examine actor effects for alliance and session Positivity and Arousal. Contrary to our hypothesis for both therapists and clients, their own ratings of alliance were related to their own ratings of Positivity but not to their own ratings of Arousal. It seems that both therapists and clients associate a strong therapeutic alliance with the experience of positive emotions after a session. However, neither party associates the therapeutic alliance with the experience of excitement or vitality after a session. There were, however, important moderated therapist-parmer effects that are discussed below. It is important to note that all of these significant actor effects were found when controlling for partner effects. Therefore, it appears that a participant's alliance rating is related in important ways to her or his evaluation of the session. The evaluations of sessions for clients and therapists have important convergences (therapeutic alliance and positivity) and divergences (client alliance with depth and therapist alliance with smoothness).

Partner Effects Our partner hypotheses were based on therapist and client role differences. As noted above, Sullivan (1954) described the therapist's role as that of a participant observer. As a participant observer, the therapist observers the client's behavior and participates as a signiflcant other in the counseling relationship. Therefore, therapists' session evaluations should be affected by their own and their clients' ratings of the alliance. The client's role in counseling, especially in early sessions, is, to the extent possible, to be a full participant in the counseling relationship. As participants, clients are typically attuned to only their own view of the alliance. Therefore, we hypothesized that there would be significant client-partner effects but not significant therapist-partner effects when examining work-oriented session outcomes (Depth and Arousal). We found one significant client-partner effect: Therapists' session Depth ratings were related to their clients' therapeutic alliance ratings. This is in contrast to Kivlighan (2007), who found that therapists' alliance ratings were related to their clients' ratings of session Depth. Our results are similar to Gelso et al. (2012), who

also found significant client-partner effects. In their study, therapists' evaluations of treatment outcome were related to their clients' ratings of the real relationship. It is hard to see why the Kivlighan (2007) study did not reveal significant client-partner effects for the alliance. All three studies used similar measures and similar client and therapist populations. It will be important for future research to examine the hypothesis conceming the effects of the therapist's role as a participant observer. Given the results of this study and of Gelso et al. (2012), it seems that therapists' are affected by their clients' view of the early therapy relationship (either the alliance or the real relationship). It is also interesting to note that there were no client-partner effects for session Smoothness, Arousal, and Positivity. Therefore, the client's perception of the alliance did not relate to her or his therapist's perceptions of the session's comfort or his or her positive emotions or experience of excitement or vitality after a session. We expect that for therapists. Depth may be related to the working aspect of psychodynamic treatment (i.e., exploratory and expressive), whereas Arousal, Positivity, and Smoothness may be related to comfort in treatment. If this is the case, then it seems that therapists pay attention to the client's alliance to gauge the work that can be or is being accomplished.

Moderation Effects We hypothesized that those therapists who were better at monitoring the state of their clients' alliances would be more successful (Richards, 2011); therefore, client-partner effects would be stronger when clients made a reliable change than when the clients did not make a reliable change. We found support for this hypothesis, but only in the case of Arousal. When clients did not make a reliable change in treatment, there was no significant relationship between the client's therapeutic alliance and the therapist's rating of Arousal early in treatment. The significant relationship between a client's therapeutic alliance and her or his therapist's Arousal only occurs when clients' make a reliable change across treatment. In examining the significant interaction, it is important to keep in mind that the therapeutic alliance and session evaluation ratings come from early ti-eatment, whereas the reliable change can only be calculated after the client terminates treatment, on average 27-28 weeks after the alliance and session evaluation ratings. For clients who will eventually make a reliable change, when the client rates the early session therapeutic alliance as stronger, the therapist ends that session feeling more arousal. For clients who will eventually not make a reliable change, their ratings of the therapeutic alliance are unrelated to their therapist's arousal. Therefore, it is a good prognostic indicator if in an early session, the therapist responds with invigoration or excitement to clients experiencing higher levels of the therapeutic alliance. And, as per the procedures of this study, clients were informed, both verbally and in writing, that their therapists would not have access to their responses on these session measures. So neither party had access to each other's scores on these process measures, and such sentiments were only discernable from the dyadic interactions during the session. Why would therapists' excitement in relationship to higher alliance ratings very early in treatment be related to clients' eventual improvement in treatment? Stiles et al. (1994) wondered whether the "energizing impact of psychotherapy" (p. 184) could be related to novice clinicians being able to apply what they had

CLIENT AND THERAPIST ALLIANCE learned with clients. It is certainly possible that when novice therapists perceived a stronger alliance, they became excited because they felt able to implement the techniques that they had learned. Pésale and Hilsenroth (2009) found that therapists' use of psychodynamic techniques was significantly related to clients' perceptions of Depth ratings early in the therapy process. In prior studies, therapists also rated more Depth when they used more psychodynamic or interpersonal techniques compared with when they used more cognitive-behavioral techniques (Stiles, Shapiro, & Firth-Cozen, 1988). In addition, dynamically oriented therapists may be more aroused by therapies that emphasize emotional engagement, exploration of underlying thoughts and needs, and the linkage of those internal experiences to the therapy relationship. Therefore, a series of events may be happening in early sessions for clients who make a reliable change. First, when their dynamically oriented therapists perceive that the client has a stronger therapeutic alliance, the therapist becomes aroused and excited because she or he sees this strengthened alliance as an opportunity to use dynamic and interpersonal techniques. Or, conversely, the therapist used these interventions with apparently good response on the part of the client, thus leading to an increase in therapist felt efficacy. The stronger alliance coupled with the presumed use of these techniques leads to the therapist seeing the session as deeper. Future research could test whether this sequence of events does happen in more successful treatments. Another possible explanation for the positive relationship between client therapeutic alliance and therapist Arousal in cases in which the client makes a reliable change is that therapists may be aroused by the emotional experience of being with clients who are doing the work of counseling. Therapists consider that their clients are working at a deep level when clients express unacceptable feelings (Greenberg & Safran, 1987). They would argue that it is the experiential learning, of being cared for and valued by the therapist during these moments of vulnerability that facihtates change. In essence, change in symptoms occurs when clients are being more emotionally vulnerable in the therapy. Relatedly, Fosha (2000) described the experience of "aliveness" and "realness" that is experienced when people are able to express these vulnerable affects in therapy. Fosha argued that therapists can rely on their own internal experience to determine whether their clients are doing the work in the treatment. She says when clients are less defensive and genuinely experiencing core emotions (i.e., exhibiting a strong alliance), the therapist feels as though therapy is moving or is alive (i.e., aroused), whereas defensiveness and lack of authenticity in the client engender boredom or the experience of going nowhere. The present study suggests that we need more research to tease apart what is happening in these early sessions of treatments and how these differences may impact therapists' and clients' perceptions of the alliance and Arousal. This is an important area of research because therapists come from a different vantage point and may feel invigorated by the therapy process, whereas clients may feel more somber, saddened, or unsettled after sessions. Clients may feel more introspective versus feelings as if they are "moving" or "energetic." Watson and Greenberg (1996) even found that some "change producing interventions are experienced as disorganizing and dissatisfying" (p. 272) by clients and that the initial reaction is often the result of confronting problems that can make one feel worse before feeling better. In this regard, however, it is important to note that reliable change was associated with

21

decreased client arousal. Clients who experienced a reliable change reported feeling less aroused after these early sessions. This fits with the notion of clients feeling more introspective.

Limitations There are several limitations to consider when drawing conclusions about the results. One of the most significant limitations is the reliance on a correlational design that inhibits us from making causal inferences about the direction of influence in the relationship between working alliance and session evaluations. We are not sure whether the alliance influences the session experiences or whether session experiences influence the alliance. In the future, collecting repeated measures of the alliance, session experience, and symptom change during the course of treatment would allow us to explore how these factors influence one another over time. In addition to the correlational design, we only focused on a relatively small sample of clients in psychodynamic treatment. Although the sample is sufficient for the statistical analysis used, we are not certain that our findings would generalize to another population of clients being treated with a different form of psychotherapy. None of the clients were randomly selected, and the majority of the clients were Caucasian females. The therapists were all graduate trainees, and we are not certain whether the results would replicate with a more seasoned group of clinicians. We tested for therapist nesting and found significant nesting for only clients' ratings of poiitivity. It is important to note, however, that the test for nesting was conducted on a fairly small sample, which may not have been large enough to detect what Kenny, Kashy, and Bolger (1998) called "consequential nonindependence." In other words, there may have been nesting effects in our data, but our tests were not powerful enough to detect it. Therefore, the results of the APIM should be viewed cautiously and hopefully replicated in larger samples. Finally, we presumed that therapists would be able to assess their clients' view of the alliance. However, we do not know how well our therapist could detect their client's perceptions of the therapeutic alliance. Another limitation is that alliance and session evaluation were measured only in an early session. It will be important for future research to examine how actor and partner effects may change across counseling sessions. For example, theory suggests that the client's observing capacity increases across treatment. This suggests that later in treatment, clients' session evaluations would berelatedto both their own and their therapists' alliance perceptions. Despite these limitations, the findings offer interesting directions for future research.

Conclusion The publication of five different meta-analyses is proof that researchers are interested in how the therapy alliance relates to treatment progress and outcome (Flückiger, Del Re, Wampold, Symonds, & Horvath, 2012; Horvath & Bedi, 2002; Horvath et al., 2011; Horvath & Symonds, 1991; Martin, Garske, & Davis, 2000). Unfortunately, the majority of our knowledge of the alliance comes from studies that focus only on the client's perspective. This is one of only a handful of studies that have used the APIM in a therapeutic context, and it is the first study to examine how treatment outcome would moderate these actor-partner relationships. When using APIM, we found that client's ratings of the

KIVLIGHAN, MARMAROSH, AND HILSENROTH

22

therapeutic alliance are related to their therapist's ratings of session Depth. We also found that there are significant positive relationships between the alliance and Arousal when clients make a reliable change in treatment. Our results encourage us to continue to study the alliance and to apply innovative methods, such as APIM, to understand and appreciate the complexity and interactive nature of the psychotherapy relationship. In addition, this was the fn-st study to examine the hypothesis that the therapeutic alliance would be differentially related to aspects of session outcome. As noted above, we found two significant partner effects (client alliance and therapist depth, client alliance-RCI interaction and therapist arousal), which both involved working aspects of session outcome. As argued above, these partner effects are especially important because they are a statistical way of identifying collaboration and influence. Therefore, these partner effects provide some evidence to support differential outcome effects. The concepts of work-related and relationship-related outcomes comes from Gelso's (2011) distinction that the working alliance is related to the work of counseling, whereas the real relationship is related to the personal relationship aspect of counseling. Future researchers could explore this conceptualization in more depth by simultaneously measuring the therapeutic alliance and the real relationship and relating them both to the different aspects of session outcome. In addition, future research may want to examine other aspects of session outcome like session impact, which more explicitly operationahzes work and relationship impacts (Stiles et al., 1994).

References Ackerman, S. J., Hilsenroth, M. J., Baity, M. R., & Blagys, M. D. (2000). Interaction of therapeutic process and alliance during psychological assessment. Journal of Personality Assessment, 75, 82-109. doi: 10.1207/S15327752JPA7501_7 Blagys, M., & Hilsenroth, M. (2000). Distinctive features of short-term psychodynamic-interpersonal psychotherapy; A review of the comparative psychotherapy process literature. Clinical Psychology: Science and Practice, 7, 167-188. doi:10.1093/clipsy.7.2.167 Boswell, J. F., Castonguay, L. G., & Wasserman, R. H. (2010). Effects of psychotherapy training and intervention use on session outcome. Journal of Consulting and Clinical Psychology, 78, 717-723. doi;10.1037/ a0020088 Clémence, A., Hilsenroth, M., Ackerman, S., Strassle, C , & Handler, L. (2005). Facets of the therapeutic alliance and perceived progress in psychotherapy: Relationship between patient and therapist perspectives. Clinical Psychology and Psychotherapy, 12, 443-454. doi:10.1002/cpp .467 Derogatis, L. (1993). Brief Symptom Inventory. Minneapolis, MN: National Computer Systems. Dunlop, W. L., & Beauchamp, M. R. (2011). Does similarity make a difference? Predicting cohesion and attendance behaviors within exercise group settings. Group Dynamics: Theory, Research, and Practice, 15, 258-266. doi:10.1037/a0023642 Eugster, S. L., & Wampold, B. E. (1996). Systematic effects of participant role on evaluation of the psychotherapy session. Journal of Consulting and Clinical Psychology, 64, 1020-1028. doi:10.1037/0022-006X.64.5 .1020 Fluckiger, C, Del Re, A. C , Wampold, B. E., Symonds, D., & Horvath, A. O. (2012). How central is the alliance in psychotherapy? A multilevel longitudinal meta-analysis. Journal of Counseling Psychology, 59, 1017. doi:10.1037/a0025749 Fosha, D. (2000). The transforming power of affect: A model of accelerated change. New York, NY: Basic Books.

Gelso, C. J. (2011). The real relationship in psychotherapy: The hidden foundation of change. Washington, DC: American Psychological Association. doi:10.1037/12349-000 Gelso, C. J., Kelley, F. A., Fuertes, J. N., Marmarosh, C , Holmes, S. E., Costa, C , & Hancock, G. R. (2005). Measuring the real relationship in psychotherapy: Initial validation ofthe therapist form. Journal of Counseling Psychology, 52, 640-649. doi:10.1037/0022-0167.52.4.640 Gelso, C. J., Kivlighan, D. M., Jr., Wine, B., Jones, A., & Friedman, S. C. (1997). Transference, insight, and the course of time-limited therapy. Journal of Counseling Psychology, 44, 209-217. doi: 10.1037/00220167.44.2.209 Gelso, C. J., Kivlighan, D. M., Jr., Busa-Knepp, J., Spiegel, E. B., Ain, S., Hummel, A. M., & Markin, R. D. (2012). The unfolding of the real relationship and the outcome of brief psychotherapy. Journal of Counseling Psychology, 59, 495-506. doi:10.1037/a0029838 Greenberg, L., & Safran, J. (1987). Emotion in psychotherapy: Affect, cognition and the process of change. New York, NY: Guilford Press. Hatcher, R. L. (1999). Therapists' view of treatment alliance and collaboration in therapy. Psychotherapy Research, 9, 405-423. Hatcher, R. L., & Barends, A. W. (1996). Patient's view ofthe alliance in psychotherapy: Exploratory factor analysis of three alliance measures. Journal of Consulting and Clinical Psychology, 64, 6, 1326-1336. doi: 10.1037/0022-006X.64.6.1326 Hatcher, R. L., & Barends, A. W. (2006). How a return to theory could help alliance research. Psychotherapy: Theory, Research, Practice, Training, 43, 292-299. doi:10.1037/0033-3204.43.3.292 Hatcher, R. L., Barends, A., Hansell, J., & Gutfreund, M. (1995). Patients' and therapists' shared and unique views of the therapeutic alliance: An investigation using confirmatory factor analysis in a nested design. Journal of Consulting and Clinical Psychology, 63, 636-643. doi: 10.1037/0022-006X.63.4.636 Hill, C. E., & Lambert, M. J. (2004). Methodological issues in studying psychotherapy process and outcomes. In M. J. Lambert (Ed.), Bergin and Garfield: Handbook ofpsychotherapy and behavior change (5th ed., pp. 84-135). New York, NY: Wiley. Hilsenroth, M. (2007). A programmatic study of short-term psychodynamic psychotherapy: Assessment, process, outcome, and training. Psychotherapy Research, 17, 31-45. doi:10.1080/10503300600953504 Horvath, A. O., & Bedi, R. P. (2002). The alliance. In J. C. Norcross (Ed.), Psychotherapy relationships that work: Therapist contributions and responsiveness to patient needs (pp. 37-69). New York, NY: Oxford University Press. Horvath, A. O., Del Re, A. C , Fluckiger, C , & Symonds, D. (2011). Alliance in individual psychotherapy. Psychotherapy, 48, 9-16. doi: 10.1037/a0022186 Horvath, A. O., & Greenberg, L. S. (1989). Development and validation of the Working Alliance Inventory. Journal of Counseling Psychology, 36, 223-233. doi:10.1037/0022-0167.36.2.223 Horvath, A. O., & Symonds, B. D. (1991). Relation between the working alliance and outcome in psychotherapy: A meta-analysis. Journal of Counseling Psychology, 38, 139-149. doi:10.1037/0022-0167.38.2.139 Jacobson, N. S., Roberts, L. J., Berns, S. B., & McGlinchy, J. B. (1999). Methods for denning and determining the clinical significance of treatment effects: Description, application and alternatives. Journal of Consulting and Clinical Psychology, 67, 300-307. doi:10.1037/0022-006X .67.3.300 Jacobson, N. S., & Truax, P. (1991). Clinical significance: A statistical approach to denning meaningful change in psychotherapy research. Journal of Consulting and Clinical Psychology, 59, 12-19. doi:10.1037/ 0022-006X.59.1.12 Kenny, D. A., Kashy, D. A., & Bolger, N. (1998). Data analysis in social psychology. In D. Gilbert, S. Fiske, & G. Lindzey (Eds.), Handbook of social psychology (4th ed.. Vol. 1, pp. 233-265). Boston, MA: McGrawHill.

CLIENT AND THERAPIST ALLIANCE Kenny, D. A., Mannetti, L., Pierro, A., Livi, S., & Kashy, D. A. (2002). The statistical analysis of data from small groups. Journal of Personality and Social Psychology, 83, 126-137. doi:10.1037/0022-3514.83.1.126 Kim, B. K., Ng, G. F., & Ann, A. J. (2009). Client adherence to Asian cultural values, common factors in counseling, and session outcome with Asian American clients at a university counseling center. Journal of Counseling and Development, 87, 131-142. doi:10.1002/j.1556-6678 .2009.tb00560.x Kivlighan, D. M., Jr. (2007). Where is the relationship in research on the alliance? Two methods for analyzing dyadic data. Journal of Counseling Psychology, 54, 423-433. doi: 10.1037/0022-0167.54.4.423 Ledermann, T., & Kenny, D. A. (2012). The common fate model for dyadic data: Variations of a theoretically important but underutilized model. Journal of Family Psychology, 26, 140-148. doi:10.1037/a0026624 Lent, R. W., Hoffman, M., Hill, C. E., Treistman, D., Mount, M., & Singley, D. (2006). Client-specific therapist self-efficacy in novice therapists: Relation to perceptions of session quality. Journal of Counseling Psychology, 53, 453-463. doi: 10.1037/0022-0167.53.4.453 Mallinckrodt, B. (1993). Session impact, working alliance, and treatment outcome of brief counseling. Journal of Counseling Psychology, 40, 25-32. doi:10.1037/0022-0167.40.1.25 Martin, D. J., Garske, J. P., & Davis, K. M. (2000). Relation of the therapeutic alliance with outcome and other variables: A meta-analytic review. Journal of Consulting and Clinical Psychology, 68, 438-450. doi: 10.1037/0022-006X.68.3.438 Owen, J., Quirk, K., Hilsenroth, M. J., & Rodolfa, E. (2012). Working through: In-session processes that promote between-session thoughts and activities. Journal of Counseling Psychology, 59, 161-167. doi: 10.1037/a0023616 Pésale, F. P., & Hilsenroth, M. J. (2009). Patient and therapist perspectives on session depth in relation to technique during psychodynamic psychotherapy. Psychotherapy: Theory, Research, Practice, Training, 46, 390396. doi:10.1037/a0016999

23

Pésale, F. P., Hilsenroth, M. J., & Owen, J. J. (2012). Patient early session experience and treatment outcome. Psychotherapy Research, 22, 417425. doi:10.1080/10503307.2012.662607 Raudenbush, S. W., Bryk, A. S., Cheong, Y. F., & Congdon, R. (2005). HLM6: Hierarchical linear and nonlinear modeling [Computer software manual]. Lincolnwood, IL: Scientific Software International. Richards, C. (2011). Alliance ruptures: Etiology and resolution. Counselling Psychology Review, 26, 56-62. Stiles, W. B., Reynolds, S., Hardy, G. E., Rees, A., Barkham, M., & Shapiro, D. A. (1994). Evaluation and description of psychotherapy sessions by clients using the Session Evaluation Questionnaire and the Session Impacts Scale. Journal of Counseling Psychology, 41, 175-185. doi:10.1037/0022-0167.41.2.175 Stiles, W. B., Shapiro, D. A., & Firth-Cozens, J. A. (1988). Verbal response mode use in contrasting psychotherapies: A within-subjects comparison. Journal of Consulting and Clinical Psychology, 56, 727733. doi:10.1037/0022-006X.56.5.727 Stiles, W. B., & Snow, J. S. (1984). Counseling session impact as viewed by novice counselors and their clients. Journal of Counseling Psychology, 31, 3-12. doi:10.1037/0022-0167.31.1.3 Sullivan, H. (1954). The psychiatric inten/iew. Oxford, England: W. W. Norton & Co. Tryon, G., & Kane, A. S. (1995). Client involvement, working alliance, and type of therapy termination. Psychotherapy Research, 5, 189-198. doi: 10.1080/10503309512331331306 Watson, J. C , & Greenberg, L. S. (1996). Pathways to change in the psychotherapy of depression: Relating process to session change and outcome. Psychotherapy: Theory, Research, Practice, Training, 33, 262-274. doi:10.1037/0033-3204.33.2.262

Received February 15, 2013 Revision received September 26, 2013 Accepted September 26, 2013 •

Copyright of Journal of Counseling Psychology is the property of American Psychological Association and its content may not be copied or emailed to multiple sites or posted to a listserv without the copyright holder's express written permission. However, users may print, download, or email articles for individual use.