Psychotherapy Research
ISSN: 1050-3307 (Print) 1468-4381 (Online) Journal homepage: http://www.tandfonline.com/loi/tpsr20
Multicultural competencies: What are we measuring? Joanna M. Drinane, Jesse Owen, Jill L. Adelson & Emil Rodolfa To cite this article: Joanna M. Drinane, Jesse Owen, Jill L. Adelson & Emil Rodolfa (2014): Multicultural competencies: What are we measuring?, Psychotherapy Research, DOI: 10.1080/10503307.2014.983581 To link to this article: http://dx.doi.org/10.1080/10503307.2014.983581
Published online: 08 Dec 2014.
Submit your article to this journal
Article views: 81
View related articles
View Crossmark data
Citing articles: 1 View citing articles
Full Terms & Conditions of access and use can be found at http://www.tandfonline.com/action/journalInformation?journalCode=tpsr20 Download by: [Deakin University Library]
Date: 06 November 2015, At: 00:39
Psychotherapy Research, 2014 http://dx.doi.org/10.1080/10503307.2014.983581
EMPIRICAL PAPER
Multicultural competencies: What are we measuring?
JOANNA M. DRINANE1, JESSE OWEN1, JILL L. ADELSON2, & EMIL RODOLFA3 1
Department of Counseling Psychology, University of Denver, Denver, CO, USA; 2Department of Educational and Counseling Psychology, University of Louisville, Louisville, KY, USA & 3Department of Psychology, Alliant International University, Davis, CA, USA
Downloaded by [Deakin University Library] at 00:39 06 November 2015
(Received 20 June 2014; revised 23 October 2014; accepted 27 October 2014)
Abstract The current study examined the validity of the client-rated version of the Cross-Cultural Counseling Inventory–Revised (CCCI-R). The first phase of this study involved a content validation of the CCCI-R by experts who had publications in the fields of multicultural competencies (MCCs) and psychotherapy research. Of the 20 items on the CCCI-R, 7 were rated as appropriate for client use. The second phase of this study utilized confirmatory factor analysis to examine construct validity by testing whether clients’ perceptions of their therapists’ MCCs (via the seven items validated by experts) were distinct from client-rated working alliance scores. Model fit statistics supported a theoretically based model in which MCCs were measured distinctly from working alliance, but where the two factors were related. Implications for theory and practice are discussed. Keywords: multicultural competencies; multicultural orientation; alliance; therapy
Cultural factors, such as race, ethnicity, gender, and sexual orientation, have long been thought to influence the therapeutic environment, and therapists have been consistently tasked with serving clients in a culturally competent manner. According to the American Psychological Association’s (APA) code of ethics (APA, 2010), psychologists must be aware of and demonstrate respect for clients’ cultural identities (e.g., race, ethnicity, sexual orientation, socioeconomic status, gender). This ethical and professional mandate, which underscores the important role that culture plays in therapy, was preceded by the APA’s adoption of a model of multicultural competencies (MCCs) to guide therapists through the cultural complexities that arise during the process of psychotherapy (APA, 2003). This widely accepted model, based upon the work of Sue, Arredondo, and McDavis (1992), asserts that psychotherapists must develop knowledge, skills, and awareness of culture in order to best serve their clients. More specifically, Sue and Sue (2008) assert that therapists need to have awareness and sensitivity to clients’ cultural heritage as
well as the awareness of therapist biases that may influence their clients. With regard to knowledge, therapist must be informed about systems that their clients operate in and barriers that they face as a result of their diverse backgrounds. Lastly, skills relate to therapists’ ability to communicate effectively with clients in styles that are appropriate according to culture (Sue & Sue, 2008). Stemming from this, researchers have investigated associations between client’s perceptions of their therapists’ MCCs and other therapeutic processes and outcomes. For instance, client-rated MCCs were positively associated with satisfaction with counseling and with therapy outcomes (e.g., Constantine, 2002; Fuertes & Brobst, 2002; Owen, Leach, Wampold, & Rodolfa, 2011), as well as with clients’ ratings of their therapists’ general competence, empathy, and credibility (Fuertes & Brobst, 2002; Fuertes et al., 2006; Kim, Li, & Liang, 2002). Additionally, clients’ ratings of their therapists’ MCCs were connected to key process variables, including working alliance and the real relationship (Fuertes et al., 2006; Owen, Tao,
Correspondence concerning this article should be addressed to Joanna M. Drinane, Department of Counseling Psychology, University of Denver, Denver, CO, USA. Email: [email protected] © 2014 Society for Psychotherapy Research
Downloaded by [Deakin University Library] at 00:39 06 November 2015
2
J. Drinane et al.
Leach, & Rodolfa, 2011). With regard to other cultural factors, clients’ ratings of therapists’ MCCs have also been positively correlated with cultural humility (Hook, Davis, Owen, Worthington, & Utsey, 2013) and negatively correlated with racial microaggressions (Constantine, 2007). Collectively, these studies highlight the importance of therapists’ MCCs as an integral part of the clients’ experiences in therapy. However, tools for measurement of this construct from a client’s vantage point are imperfect and have not undergone vigorous validation procedures. Accordingly, we will examine two pressing validity concerns, content and construct, within the MCCs literature. Currently, there are three primary means of acquiring ratings of therapists’ MCCs: (i) supervisors/observers, (ii) therapists, and (iii) clients. Supervisor/observer rating, while potentially useful as it connects to how therapists in training are evaluated, is resource intensive and is seldom used in the literature. Additionally, supervisor ratings are typically not standardized, which can result in measurement inconsistency. Therapist ratings of their own MCCs have been criticized because they often depict multicultural counseling self-efficacy, or confidence and anticipation about implementing this skill set, as opposed to counselors’ actual ability to perform multicultural counseling behaviors successfully (Constantine & Ladany, 2000). According to Fuertes et al. (2006), clients’ ratings of therapists’ MCCs were correlated at r = −.03 with therapists’ ratings of their own MCCs. This near null correlation suggests that clients’ and therapists’ perceptions of therapists’ MCCs are not congruent. Clients’ ratings of MCCs have been deemed an ideal way of reflecting clients’ perceptions of their therapists’ MCCs, because they fundamentally bring clients’ voice and experience into the process. Each of the studies described above, which examined clients’ ratings of therapists’ MCCs, used the Cross-Cultural Counseling Inventory-Revised measure (CCCI-R; LaFromboise, Coleman, & Hernandez, 1991). This scale was developed based on the tripartite model of MCCs (i.e., knowledge, skills, and awareness; Sue et al., 1992). The CCCI-R was not initially designed and validated for use by clients; rather, it was intended to be a supervisor/ observer-rated scale. Due to lack of another measure to obtain clients’ ratings, researchers made changes to voice, subject, and content of scale items in order to adapt them for client use. For example, as supervisorrated item, item 13 reads: “Communication is appropriate for the client.” This item was changed for clients to be: “My counselor sends messages that are appropriate to me based on my cultural heritage.” Another supervisor-rated item is: “Aware of professional
responsibilities.” The clients’ version reads, “My counselor is aware of the professional and ethical responsibilities of a counselor.” Changes like these can influence item meaning and message. To date, the client-rated version of the CCCI-R has missed some important steps in the validation process (e.g., content validation), yet it is one of the most widely used clientrated measures for therapists’ MCCs. Content validation is foundational to measurement development and to the ability to accurately reflect and draw inferences relating to an intended construct (Heppner, Wampold, & Kivlighan, 2007). As a first step in this study, we will examine the content validity of the client-rated version of the CCCI-R. Also to do with validity, another existing concern in the MCC literature has to do with whether or not MCCs are truly distinct from other general competencies or therapeutic processes. This relates to construct validity, as can be examined by how well we can discriminate between these two forms of competencies (cultural and general). In 1998, Coleman posed the question, “General and Multicultural Competency: Apples and Oranges?” when he found a high correlation between MCCs and general competencies. Similarly, high correlations have been observed between scores on the CCCI-R and scores on a number of measures of general therapeutic process variables, leaving questions about the unique nature of MCCs. In particular, consistently high correlations are demonstrated between clientrated MCCs (via the client-rated version of the CCCI-R which had not been fully validated) and client-rated working alliance (typically measured by the Working Alliance Inventory–Short Form Revised; Hatcher & Gillaspy, 2006). For example, Constantine (2007) found these two scales to be correlated at .70 (n = 40), and Owen, Tao, et al. (2011) and Fuertes et al. (2006) both found correlations of .73 (n = 276; n = 51, respectively). Although these correlations demonstrate the strong relationship between client-rated MCCs and working alliance, it is unclear if MCCs are accurately being measured and if they can be subsumed under the category of general therapeutic process competencies. The distinction between therapeutic alliance and MCCs is best tested from a measurement standpoint (e.g., construct validity), and to date, we are not aware of any studies that approached this question in this manner. Theory supports three measurement models of potential relationships that may exist between these two constructs. The first model (Distinct yet Related), displayed in Figure 1, is one in which the two factors (i.e., alliance and MCCs) are based on separate items but the overall factors are correlated. The second model (Distinct and Separate), displayed in Figure 2, is consistent with
Downloaded by [Deakin University Library] at 00:39 06 November 2015
Psychotherapy Research
3
Figure 1. Model 1: Distinct yet Related. Note: 1–12 = items from WAI-SR, 1–7 = items retained from CCCI-R.
the first model; however, there is no proposed correlation between the two factors. These models indicate that these constructs are defined differently (i.e., separate items for each factor). Although there is evidence in previous studies that MCCs and alliance are related (e.g., Constantine 2007; Fuertes et al., 2006), we tested Models 1 and 2 to estimate the association between alliance and MCCs after accounting for measurement considerations. The third (All for One), displayed in Figure 3, is based upon the high correlations between alliance and MCCs wherein these associations may reflect that these constructs are not separate. As such, all of the items for working alliance and multicultural competence load onto one global factor. As such, Models 1 and 2 propose that alliance and MCCs are distinct constructs (related or not), and Model 3 suggests that alliance and MCCs are not distinct constructs.
The Current Studies In Study 1, we will examine the content validity of the client-rated version of the CCCI-R based on expert feedback. In this process, we anticipate that we will screen out some items according to expert
Figure 2. Model 2: Distinct and Unrelated. Note: 1–12 = items from WAI-SR, 1–7 = items retained from CCCI-R.
feedback; however, unique from traditional measurement validation, we will not add new items. Rather, our focus is to better understand the validity of the CCCI-R and to determine which of its items can be used effectively in a client-rated capacity. Following this process, with the validated CCCI-R items, in Study 2, we will examine the factor structure of the remaining items and also test the distinctness of the following two constructs: clientrated MCCs and working alliance.
Study 1 Method Participants and procedures. Participants for the content validation conducted in Study 1 were recruited via email. Emails contained requests for participation, an introduction describing our motivation for conducting such a validation, and a link to the survey questions on the PsychData website. Those in receipt of an email were among a convenience sample and were preselected according to their
4
J. Drinane et al. and followed by a question to assess item quality: “How well does this item represent multicultural competence?” Experts rated each item using a Likert scale ranging from −3 (really bad item) to +3 (really good item). Following each item was also a space for experts to provide open-ended responses with commentary on the utility of each item for client use. In these responses, experts elaborated upon concerns they had about clients being able to understand the content of or even provide a meaningful score for each item.
Downloaded by [Deakin University Library] at 00:39 06 November 2015
Results and Discussion
Figure 3. Model 3: All for One. Note: 1–12 = items from WAI-SR, 1–7 = items retained from CCCI-R-7.
expert credentials. Such credentials involved having a doctoral-level degree, research expertise in psychotherapy, and therapist MCCs, and at least one (some with many more) peer reviewed publication in each of the following topic areas: MCCs and psychotherapy process/outcome research. Nineteen responses were obtained. The sample included 10 men and 9 women. Of the 19, 11 experts self-identified as some minority (e.g., racial ethnic minority [REM] or sexual minority). Due to confidentiality, we opted not to include more specific demographic information. Measures. The survey presented initial instructions with the aims of the project as well as a commonly utilized definition of MCCs focusing on therapists’ ability to implement knowledge, skills, and awareness regarding clients’ cultural heritage and social group identity/membership (Alvarez & Chen, 2008; APA, 2003; Sue, 1998; Sue & Torino, 2005). This framework for MCCs is consistent with the definition utilized in the development of the original items for the CCCI-R (LaFromboise et al., 1991). Next, the 20 CCCI-R items were displayed
Means and standard deviations for the quality ratings of the 20 items are presented in Table I. Items with higher means were rated as being better at targeting the underlying construct of therapists’ MCCs. Along with their item MCC quality ratings, experts offered written reactions to these items. These written reactions generally denoted issues regarding to the clarity, or lack thereof, of the concepts. The two lead authors evaluated open-ended responses to detect concerns by experts about item appropriateness. Responses that indicated that an item might be unclear resulted in items being flagged for further review by our research team of nine doctoral students in an APA accredited Counseling Psychology program. The team reached consensus regarding the severity of expert concerns and used this to eliminate items deemed ineffective at measuring client-rated MCCs. If experts indicated that a given item was appropriate for client use, it was then considered within the context of how well it depicted MCCs. Items with a MCCs score greater than zero were included as this indicated that these were rated as positive/good at capturing the meaning of MCCs. Items and their quality ratings are displayed in Table I. Fifteen items were rated above zero. From a quantitative standpoint, zero was selected as the cutoff due to the face validity associated with using a scale with negative and positive ends. To be comprehensive and also true to the process of content validity, we integrated the experts’ comments on each item. Clearly, the experts have more nuanced understanding of the construct that a single numerical rating could reveal. As such, we wanted to honor their voice in this content validation process. Using description provided from experts, we flagged a number of items that were indicated as not being appropriate for collecting data from clients (i.e., they were confusing or experts found them not reasonable for clients to answer). Adding their qualitative reactions allowed us to screen for items whose content related to the construct, but that should not have been used in this fashion. To be retained,
Psychotherapy Research
5
Table I. Cross-cultural counseling inventory-revised: content experts’ ratings.
Downloaded by [Deakin University Library] at 00:39 06 November 2015
Item (20) My counselor acknowledges and is comfortable with cultural differences (14) My counselor attempts to perceive my problems within the context of my cultural experience, values, and/or lifestyle (2) My counselor values and respects cultural differences (7) My counselor demonstrates knowledge about my culture (13) My counselor sends messages that are appropriate to me based on my cultural heritagea (6) My counselor understands the current socio-political system and its impact on mea (9) My counselor is aware of institutional barriers which might affect my circumstancesa (17) My counselor recognizes those limits determined by the cultural differences between usa (3) My counselor is aware of how his or her own values might affect me (1) My counselor is aware of his or her own cultural heritagea (5) My counselor is willing to suggest referral when cultural differences are extensivea (4) My counselor is comfortable with differences between us (16) My counselor is at ease talking with me (19) My counselor is aware of the professional and ethical responsibilities of a counselora (15) My counselor presents his or her own values to mea (8) My counselor has a clear understanding of counseling and the therapy process (12) My counselor is able to suggest institutional intervention skills that favor me (10) My counselor elicits a variety of verbal and non-verbal responses from me (18) My counselor appreciates that my social status is based on ethnic status (11) My counselor accurately sends and receives a variety of verbal and non-verbal messagesa
Mean
SD
2.69
0.60
2.69
0.60
2.38 2.31 2.13
0.72 0.79 1.12
2.06
1.24
2.00
0.82
1.80
1.32
1.63 1.50 1.44
1.02 1.75 1.15
1.38 1.13 0.56
1.31 2.13 2.06
0.25 –1.94 –1.93 –1.81 –1.77 –1.75
1.77 2.26 2.02 1.90 2.24 1.77
Note. Bolded items were retained for the CCCI-R-7. a Items were noted as being unclear by expert reviewers.
items had to go through this rigorous, multistep evaluation process. For example, item number 9 has a mean score of 2. This item is, “My counselor is aware of institutional barriers which might affect my circumstances.” In response, one rater noted, “Ok item, but might be hard for the client to rate unless the counselor explicitly discussed these in session.” In response to this feedback, this item was flagged and the panel determined that it should be excluded from an abbreviated scale. Therefore, any of the 15 items rated above zero, which did not receive expert feedback contraindicating its use will be retained and considered part of an abbreviated scale. To note, several reviewers commented that their ratings on the scale were disconnected from their reactions regarding the suitability of the item. As such, we weighted the totality of the responses. Expert ratings of the items of CCCI-R indicated that 7 of the 20 should be retained (referred to as the CCCI-R-7 for the remainder of the description of the analyses). The first is item 20, “My counselor acknowledges and is comfortable with cultural differences.” This assesses therapists’ ability to express acceptance and comfort of clients’ cultural heritage. The second is item 14, “My counselor attempts to perceive my problems within the context of my
cultural experience, values, and/or lifestyle.” This item targets therapists’ ability to integrate culture into conceptualization. The third is item 2, “My counselor values and respects cultural differences.” This item describes therapists’ expression of regard for clients’ culture. The fourth is item 7, “My counselor demonstrates knowledge about my culture.” This item assesses how informed therapists are about culture. The fifth item, item 3 states “My counselor is aware of how his or her own values might affect me.” This item targets therapists’ ability to introspect and be sensitive to imposing their own values on the client. Items six (number 4— “My counselor is comfortable with differences between us”) and seven (number 16— “My counselor is at ease talking with me”) both relate to therapists’ ability to appear genuinely comfortable during discussions with clients, specifically about cultural differences and otherwise. As these seven items combine to form a measure of clients’ ratings of therapist’s MCCs that is likely more appropriate than the entire CCCI-R, we can now turn to our second purpose, which is to determine whether client-rated MCCs are distinct from other psychotherapy process factors, the alliance.
6
J. Drinane et al. Study 2
Downloaded by [Deakin University Library] at 00:39 06 November 2015
Method Participants Clients. This sample included data from 279 clients compiled from two previous data sets (Owen, Tao, et al., 2011; Owen, Leach, et al., 2011). Although these data were compiled for previously conducted studies, the analysis performed in the present study is novel. Of this sample, 151 self-identified as white and 127 self-identified as an REM. Additionally, 74.9% identified as women, 23.7 identified as men, and 1.1% identified as transgender or other nonbinary gender. The average age was 24.44 years old (SD = 5.42, Median = 23). Clients’ data were collected from a university counseling center where students partook in brief psychotherapy (6–10 sessions). The median number of sessions was six. Therapists. In this study, 40 therapists treated the clients (M = 6.98 clients/therapist; range = 2–21). The therapists were predoctoral interns, postdoctoral fellows, staff psychologists, and staff psychotherapists from a large university counseling center located in the Western United States. Eleven of the psychotherapists self-identified as REM, and 29 self-identified as White. Information regarding the therapeutic approach of psychotherapists was not gathered. Measures The Cross-Cultural Counseling Inventory–Revised (CCCIR; LaFromboise et al., 1991). The CCCI-R was used to assess clients’ perceptions of their therapists’ MCCs. The CCCI-R has 20 items, which are rated on a 6point scale ranging from 1 (Strongly Disagree) to 6 (Strongly Agree). Higher scores indicate more perceived MCCs. However, for the purpose of this analysis, only the seven items retained from the content validation described above were included. These seven items (numbers 2, 3, 4, 7, 14, 16, and 20) were examined using exploratory factor analysis with principle axis extraction, with oblimin rotation. A Scree test (Cattell, 1966) and eigenvalues over one indicated that a onefactor solution should be retained. All of the items loaded onto one factor, with the factor loadings being high (all above .63). Additionally, the Cronbach’s alpha for the 7-item scale was .91. Working Alliance Inventory—Short Form Revised (WAI-SR; Hatcher & Gillaspy, 2006). The WAISR is a 12-item client-rated measure of working alliance. It assesses goals and tasks for psychotherapy and the client/psychotherapist relational bond. Items like “I believe [therapist] likes me” and “We agree on what is important for me to work on” are rated on a 1–7 scale, where 1 is “strongly disagree” and 7 is
“strongly agree.” Higher scores are indicative of a better working alliance. This scale is considered to be pantheoretic and captures three important facets of working alliance: (i) agreement on the tasks of therapy, (ii) agreement on the goals of therapy, and (iii) development of an affective bond. The reliability and validity have been demonstrated numerous times through comparison with other measures of working alliance and psychotherapy outcome (Hatcher & Gillaspy, 2006). The Cronbach’s alphas for the three subscales were: goals alpha = .91, tasks alpha = .93, and bond alpha = .93. Procedure As is outlined (Owen, Tao, et al., 2011; Owen, Leach, et al., 2011), participants were recruited from a large university counseling center located on the west coast in the United States. Clients were asked on their intake card(s) whether they would be willing to receive a survey about their therapy experience. Those who agreed were sent an email at the end of the academic quarter and were able to access the anonymous survey instruments online. If clients endorsed multiple individual therapists (or no therapist), they were excluded from the analyses. Additionally, clients who did not attend individual therapy (e.g., couple therapy, group therapy, or psychiatric appointments) were excluded from the sample. Clients initially completed an informed consent and then the measures. For purposes of this study, clients were specifically directed to rate their therapists on the WAI-SR and CCCI-R measures. The survey responses were collected anonymously (the participants’ email addresses were not linked to their responses), and all procedures were approved through the university’s institutional review board committee. Data Analysis Overview: Description of Hypothesized Models Three theoretically based models were defined to determine whether client-rated MCCs were conceptually distinct or not from their ratings of the alliance. Confirmatory factor analysis (CFA) was used to determine which of the three models might be the best representation of the relationship of the items on the CCCI-R-7 and the WAI-SR. Items for the working alliance were modeled from the three subscales of the WAI-SR (i.e., goals, tasks, and bonds). Indicators of MCCs included the seven items from the CCCI-R that were supported in Study 1. Model 1 (Distinct yet Related) correlated a global alliance factor with MCCs indicating their
Downloaded by [Deakin University Library] at 00:39 06 November 2015
Psychotherapy Research relatedness in the therapy process. The global alliance factor had three indicators (the alliance subscales: goals, tasks, and bonds), each predicting four alliance items and the MCCs factor had the seven indicators from the CCCI-R-7. This model suggests that from a measurement standpoint MCCs are distinct from alliance, but that the two constructs are related. Model 2 (Distinct and Separate) used the same variables; however, the global alliance factor was not correlated with the MCCs factor. This model suggests that MCCs are distinct from alliance at the measurement level and that there is no relationship between the factors. Model 3 (All for One) had the 12 WAI-SR items and the seven items from the CCCI-R-7 load onto one factor. This model suggests MCCs and alliance are best measured as one global construct. To evaluate model fit, we used the following statistics: confirmatory fit index (CFI), root mean square error of approximation (RMSEA), standardized root mean square residual (SRMR), and adjusted Bayesian information criterion (BIC). According to Kline (2011), model fit is considered to be good/acceptable when CFI is above .90, with greater values indicating better fit. RMSEA and SRMR values that are less than .08 are also indicative of acceptable model fit. Additionally, adjusted BIC values were provided as a means for model comparison, which do not rely on statistical significance tests. Although there are no clear cut-offs for how much lower the better fitting model should be, Raftery (1995) suggests that differences over 10 points is strong evidence for the better fitting model.
Results Model fit statistics for the three models are presented in Table II. The All for One model had poor overall model fit according to CFI, RMSEA, and SRMR statistics. The Distinct and Separate model showed better overall fit than the All for One model. However, for the Distinct and Separate model, RMSEA and SRMR were both above .08, which is Table II. Summary of model fit statistics. Model
CFI
RMSEA
SRMR
Adjusted BIC
Distinct yet Related Distinct and Unrelated All for One
.93 .92
.09 .09
.07 .20
12,172.13 12,218.88
.66
.20
.15
13,473.60
Note. Model 1, Distinct yet Related–– 3 alliance subscales, 1 global alliance, and 1 MCC (alliance and MCC correlated). Model 2, Distinct and Unrelated–– 3 alliance subscales, 1 global alliance, and 1 MCC (alliance and MCC uncorrelated). Model 3, All for One–– All items loading on one factor.
7
Table III. Unstandardized estimates for Model 1 (Distinct yet Related).
Factors and items
WAI WAI WAI WAI
Item Item Item Item
1 4 8 11
WAI WAI WAI WAI
Item Item Item Item
2 5 9 10
WAI WAI WAI WAI
Item Item Item Item
3 6 7 12
CCCI-R-7 CCCI-R-7 CCCI-R-7 CCCI-R-7 CCCI-R-7 CCCI-R-7 CCCI-R-7
Item Item Item Item Item Item Item
1 2 3 4 5 6 7
Estimate
Standard error
Goals 1.00 0.00 0.93 0.06 1.15 0.08 1.14 0.07 Tasks .91 0.05 .97 0.04 1.00 0.00 1.02 0.04 Bond 1.00 0.00 0.75 0.04 1.08 0.05 1.11 0.05 Cultural competence 1.00 0.00 0.85 0.07 0.95 0.06 1.08 0.07 1.22 0.08 0.95 0.05 1.07 0.05
95% Confidence interval lower, upper
1.00, 0.81, 0.99, 1.00,
1.00 1.05 1.31 1.28
0.81, 0.89, 1.00, 0.94,
1.01 1.05 1.00 1.10
1.00, 0.67, 0.98, 1.01,
1.00 0.83 1.18 1.21
1.00, 0.71, 0.83, 0.94, 1.06, 0.85, 0.97,
1.00 0.99 1.17 1.22 1.38 1.05 1.17
representative of poor model fit. The Distinct yet Related model had acceptable overall fit and was favored according to adjusted BIC and SRMR estimates. As the Distinct yet Related model demonstrated acceptable model fit and was superior to the other two models, we will proceed by interpreting it further. Distinct yet Related model supports the notion that the measurement of MCCs is distinct from the alliance, and further, that the two constructs are associated with one another (r = .43, p < .001). In other words, the way in which clients respond to questions on these topics is, in fact, different and should not be considered as representing one over-arching topic (as depicted by the All for One model). Further evidence is provided in Table III, which provides the unstandardized estimates for the Distinct yet Related model, and Figure 4 shows the standardized estimates for this model. We attempted to conduct a multiple group analysis of the models for REM and White clients; however, the models would not converge, likely due to a relatively small size for the number of estimated parameters. Additionally, a multilevel CFA was also attempted; however, the sample size at the therapist level was insufficient.
Downloaded by [Deakin University Library] at 00:39 06 November 2015
8
J. Drinane et al.
Figure 4. estimates.
Final model, Distinct and Related: standardized
Discussion Two main results emerged from these two studies. First, experts noted that the measurement of clientrated MCCs via the CCCI-R is better represented by only 7 of the 20 items. To our knowledge, little attention has been paid to the measurement properties of the client-rated version of the CCCI-R, despite it being the most often utilized client-rated MCCs measure (e.g., Constantine, 2007; Fuertes et al., 2006; Owen, Leach, et al., 2011). The 7-item version of the scale was supported via feedback from experts and has utility as a brief measure of therapists’ MCCs. These seven items integrate important elements of demonstrated knowledge, sensitivity toward and respect for cultural identity, as well as the ability to infuse clients’ cultural beliefs into treatment (Sue et al., 1998; Wampold, 2007). Moreover, these items capture that therapists are active in taking opportunities to infuse clients’ culture in the therapy process and that they are comfortable with cultural topics in therapy. Common issues among items that were not retained include: being beyond the scope of clients’ knowledge, being outside of clients’ conscious experience, being too general, or not being applicable to most
clients. For example, item 12 reads, “My counselor is able to suggest institutional intervention skills that favor me.” In this case, clients may not know about such interventions or this may not be applicable to them based on their circumstances. There is a need for more attention to measurement as there are very few measures of multicultural constructs in psychotherapy. Indeed, there are some new measures and approaches to examining therapists’ MCCs emerging (e.g., Hook et al., 2013; Imel et al., 2011). Second, from a measurement standpoint, clientrated MCCs are distinct from the working alliance, yet the two constructs are related. This theoretically based model is likely the most representative of the relationship between client-rated MCCs and working alliance, as can be determined by its superior fit over the other two models. Importantly, the poor fit of the All for One model indicates that the items for alliance and MCCs are not adequately represented as one global factor or one construct. These findings are also consistent with theoretical positions suggesting that MCCs have a unique role in the therapeutic process (Sue, 2003; Owen, Tao, et al., 2011). Clients’ perceptions of their therapists’ MCCs are different than how clients perceive their agreement on the goals for therapy, the agreement on the ways to reach those goals, and the emotional bond with their therapists. Clearly, the moderate to high correlation (r = .48) between client-rated MCCs and alliance illustrates that these constructs covary; moreover, theoretically they work in concert to enhance therapy outcomes (Hook et al., 2013; Owen, Tao, et al., 2011). Limitations and Future Research The strengths of our work should be interpreted in concert with the limitations. First, our sample size, while relatively large for a psychotherapy study, was limited in some important ways. For example, we attempted to run a multiple group analysis to determine if the Distinct but Related model functioned similarly among REM clients and White clients. Unfortunately, the model would not converge. Statistical power and the ability to estimate the model would be enhanced if we had more participants in each group. In addition, a larger sample size might have enabled us to assess the model fit for various racial and ethnic groups (e.g., African American) in concert with their intersecting of cultural identities (e.g., African American women). However, although not ideal, the acceptable model fit utilizing the overall sample is sufficient to inform future studies. Alternatively, the acceptable nature of the fit of the models tested suggests that other models that better
Downloaded by [Deakin University Library] at 00:39 06 November 2015
Psychotherapy Research represent these constructs may exist. Clearly, such models need to be considered via a theoretical frame; there could be some rationale for examining the overlap between the alliance-bond subscale and clients’ perceptions of MCCs (see Owen, 2013) or other related constructs, such as the real relationship (Gelso, 2009). The prospect of utilizing different models might also address issues regarding lack of fit of the multiple group analysis. For instance, there may be different models for REM versus White clients that could be useful to explore. Second, our findings must also be interpreted with knowledge of the abbreviated nature of the 7-item version of the CCCI-R. Although the seven items that were retained were deemed by experts to be good items and to be appropriate for client use, they may not encompass all that there is to know/measure about MCCs. We did not seek to generate more items, but instead chose to take the first step in refining measurement of this construct by trimming the measure that is considered to be the standard in the field. Therefore, there is a continued need for development of client-rated measures for assessing MCCs and for the comparison of such measures to other related therapeutic processes. Third, this study used cross-sectional retrospective data, which limited our ability to gather more detailed information regarding clients’ presenting problems, initial severity, and changes over the course of therapy. At this point, there are no known prospective studies of clients’ perceptions of their therapists’ MCCs. More work is needed in this area, and with refined measurement, the field will be better able to take strides toward this goal. Future studies should further investigate in more detail how therapists’ MCCs influence therapy outcomes. For example, Owen, Tao, et al. (2011) proposed that therapists’ MCCs influence the working alliance, which in turn affects therapy outcomes (i.e., mediation model). In this way, the current results support the connection between the alliance and therapists’ MCCs. However, it could be that therapists’ MCCs distinctly predict therapy outcomes over and above the role of the alliance. Testing these models will be important to understanding how therapists’ MCCs influence the therapeutic process. Implications Ultimately, both Study 1 and Study 2 have implications for how the field can understand MCCs. We hope that the results from Study 1, coupled with previous reviews of the limitations in the literature, serve as a call to the profession to be more mindful of measurement issues in the assessment of
9
culturally based measures in psychotherapy (see Worthington, Soth-McNett, & Moreno, 2007). Indeed, it should be sobering that the primary measure of client-rated MCCs has such content validity concerns. At the same time, we are encouraged that this study will provide some confidence that the seven items that were retained from the CCCI-R can provide a broad frame for the assessment of therapists’ MCCs. Study 2 may influence how we conceptualize MCCs as compared to other more general competencies (cf. Coleman, 1998). As MCCs are distinct from, yet related to working alliance, it is useful to disentangle therapists’ strength on alliance formation and their orientation toward their clients’ cultural identity. While connected, these skills are unique. Being clear on the specific therapeutic processes that are related to each construct will ultimately aid in this endeavor (also see Owen, 2013). For instance, it appears that how therapists are able to facilitate goals for treatment (part of the alliance) is unique from how they consider and integrate clients’ cultural heritage into the treatment process. In doing so, these distinct processes may contribute collectively to clients’ overall therapeutic outcomes. Consequently, we hope that our work will encourage more dialog and empirical work on therapists’ MCCs. References Alvarez, A. N., & Chen, G. A. (2008). Ruth as an Asian American: A multicultural, integrative perspective. In G. Corey (Ed.), Case approach to counseling and psychotherapy (pp. 298–306). Belmont, CA: Wadsworth. American Psychological Association. (2003). Guidelines on multicultural education, training, research, practice, and organizational change for psychologists. APA Council of Representatives. American Psychologist, 58, 377–402. American Psychological Association. (2010). Ethical principles of psychologists and code of conduct (Original published 2002, amended June 1, 2010). Retrieved from http://www.apa.org/ ethics/code/index.aspx Cattell, R. B. (1966). The data box: Its ordering of total resources in terms of possible relational systems. In R. B. Cattell (Ed.), Handbook of multivariate experimental psychology (pp. 67–128). Chicago, IL: Rand-McNally. Coleman, H. L. K. (1998). General and multicultural counseling competency: Apples and oranges? Journal of Multicultural Counseling and Development, 26(3), 147–156. doi:10.1002/j. 2161-1912.1998.tb00194.x Constantine, M. G. (2002). Predictors of satisfaction with counseling: Racial and ethnic minority clients’ attitudes toward counseling and ratings of their counselors’ general and multicultural counseling competence. Journal of Counseling Psychology, 49, 255–263. doi:10.1037/0022-0167.49.2.255 Constantine, M. G. (2007). Racial microaggressions against African American clients in cross-racial counseling relationships. Journal of Counseling Psychology, 54(1), 17–31. doi:10.1037/0022-0167. 54.1.17 Constantine, M. G., & Ladany, N. (2000). Self-report multicultural counseling competence scales: Their relation to social desirability attitudes and multicultural case conceptualization.
Downloaded by [Deakin University Library] at 00:39 06 November 2015
10
J. Drinane et al.
Journal of Counseling Psychology, 47, 155–164. doi:10.1037/ 0022-0167.47.2.155 Fuertes, J. N., & Brobst, K. (2002). Clients’ ratings of counselor multicultural competency. Cultural Diversity & Ethnic Minority Psychology, 8, 214–223. doi:10.1037/1099-9809.8.3.214 Fuertes, J. N., Stracuzzi, T. I., Bennett, J., Scheinholtz, J., Mislowack, A., Hersh, M., & Cheng, D. (2006). Therapist multicultural competency: A study of therapy dyads. Psychotherapy, 43, 480–490. doi:10.1037/0033-3204.43.4.480 Gelso, C. J. (2009). The real relationship in a postmodern world: Theoretical and empirical explorations. Psychotherapy Research, 19, 253–264. Hatcher, R. L., & Gillaspy, J. A. (2006). Development and validation of a revised short version of the working alliance inventory. Psychotherapy Research, 16(1), 12–25. doi:10.1080/ 10503300500352500 Heppner, P. P., Wampold, B. E., & Kivlighan, Jr., D. M. (2007). Research design in counseling (3rd ed.). Belmont, CA: Brooks Cole. Hook, J. N., Davis, D. E., Owen, J., Worthington, E. L., & Utsey, S. O. (2013). Cultural humility: Measuring openness to culturally diverse clients. Journal of Counseling Psychology, 60, 353–366. doi:10.1037/a0032595 Imel, Z. E., Baldwin, S. A., Atkins, D., Owen, J., Baardseth, T., & Wampold, B. E. (2011). Racial/ethnic disparities in therapist effectiveness: A conceptualization and initial study of cultural competence. Journal of Counseling Psychology, 58, 290–298. Kim, B. S. K., Li, L. C., & Liang, C. T. H. (2002). Effects of Asian American client adherence to Asian cultural values, session goal, and counselor emphasis of client expression on career counseling process. Journal of Counseling Psychology, 49, 342–354. doi:10.1037/0022-0167.49.3.342 Kline, R. B. (2011). Principles and practice of structural equation modeling (3rd ed.). New York, NY: Guilford Press. LaFromboise, T. D., Coleman, L. K., & Hernandez, A. (1991). Development and factor structure of the cross-cultural counseling inventory-revised. Professional Psychology: Research and Practice, 22, 380–388. doi:10.1037/0735-7028.22.5.380
Owen, J. (2013). Early career perspectives: Therapist effects, cultural dynamics, and other processes in psychotherapy. Psychotherapy, 50, 496–502. Owen, J., Leach, M., Wampold, B., & Rodolfa, E. (2011). Client and therapist variability in clients’ perceptions of their therapists’ multicultural competencies. Journal of Counseling Psychology, 58(1), 1–9. doi:10.1037/a0021496 Owen, J., Tao, K., Leach, M., & Rodolfa, E. (2011). Clients’ perceptions of their psychotherapists’ multicultural orientation. Psychotherapy, 48, 274–282. doi:10.1037/a0022065 Raftery, A. E. (1995). Bayesian model selection in social research (with discussion). Sociological Methodology, 25, 111–195. doi:10.2307/271063 Sue, D. W. (2003). Overcoming our racism: The journey to liberation. San Francisco, CA: Jossey-Bass. Sue, D. W., Arredondo, P., & McDavis, R. J. (1992). Multicultural counseling competencies and standards: A call to the profession. Journal of Counseling & Development, 70, 477–486. doi:10.1002/j.1556-6676.1992.tb01642.x Sue, D. W., Carter, R. T., Casas, J. M., Fouad, N. A., Ivey, A. E., Jensen, M., … Vazquez-Nutall, E. (Eds.). (1998). Multicultural counseling competencies: Individual and organizational development. Thousand Oaks, CA: Sage. Sue, D. W., & Torino, G. C. (2005). Racial-cultural competence: Awareness, knowledge, and skills. Handbook of RacialCultural Psychology and Counseling: Training and Practice, 2, 3–18. Sue, D. W., & Sue, D. (2008). Counseling the culturally diverse: Theory and practice (5th ed., p. 45). Hoboken, NJ: John Wiley & Sons. Sue, S. (1998). In search of cultural competence in psychotherapy and counseling. American Psychologist, 53, 440–448. doi:10.1037/0003-066X.53.4.440 Wampold, B. E. (2007). Psychotherapy: The humanistic (and effective) treatment. American Psychologist, 62, 857–873. Worthington, R. L., Soth-McNett, A. M., & Moreno, M. V. (2007). Multicultural counseling competencies research: A 20-year content analysis. Journal of Counseling Psychology, 54, 351–361. doi:10.1037/0022-0167.54.4.351
ISSN: 1050-3307 (Print) 1468-4381 (Online) Journal homepage: http://www.tandfonline.com/loi/tpsr20
Multicultural competencies: What are we measuring? Joanna M. Drinane, Jesse Owen, Jill L. Adelson & Emil Rodolfa To cite this article: Joanna M. Drinane, Jesse Owen, Jill L. Adelson & Emil Rodolfa (2014): Multicultural competencies: What are we measuring?, Psychotherapy Research, DOI: 10.1080/10503307.2014.983581 To link to this article: http://dx.doi.org/10.1080/10503307.2014.983581
Published online: 08 Dec 2014.
Submit your article to this journal
Article views: 81
View related articles
View Crossmark data
Citing articles: 1 View citing articles
Full Terms & Conditions of access and use can be found at http://www.tandfonline.com/action/journalInformation?journalCode=tpsr20 Download by: [Deakin University Library]
Date: 06 November 2015, At: 00:39
Psychotherapy Research, 2014 http://dx.doi.org/10.1080/10503307.2014.983581
EMPIRICAL PAPER
Multicultural competencies: What are we measuring?
JOANNA M. DRINANE1, JESSE OWEN1, JILL L. ADELSON2, & EMIL RODOLFA3 1
Department of Counseling Psychology, University of Denver, Denver, CO, USA; 2Department of Educational and Counseling Psychology, University of Louisville, Louisville, KY, USA & 3Department of Psychology, Alliant International University, Davis, CA, USA
Downloaded by [Deakin University Library] at 00:39 06 November 2015
(Received 20 June 2014; revised 23 October 2014; accepted 27 October 2014)
Abstract The current study examined the validity of the client-rated version of the Cross-Cultural Counseling Inventory–Revised (CCCI-R). The first phase of this study involved a content validation of the CCCI-R by experts who had publications in the fields of multicultural competencies (MCCs) and psychotherapy research. Of the 20 items on the CCCI-R, 7 were rated as appropriate for client use. The second phase of this study utilized confirmatory factor analysis to examine construct validity by testing whether clients’ perceptions of their therapists’ MCCs (via the seven items validated by experts) were distinct from client-rated working alliance scores. Model fit statistics supported a theoretically based model in which MCCs were measured distinctly from working alliance, but where the two factors were related. Implications for theory and practice are discussed. Keywords: multicultural competencies; multicultural orientation; alliance; therapy
Cultural factors, such as race, ethnicity, gender, and sexual orientation, have long been thought to influence the therapeutic environment, and therapists have been consistently tasked with serving clients in a culturally competent manner. According to the American Psychological Association’s (APA) code of ethics (APA, 2010), psychologists must be aware of and demonstrate respect for clients’ cultural identities (e.g., race, ethnicity, sexual orientation, socioeconomic status, gender). This ethical and professional mandate, which underscores the important role that culture plays in therapy, was preceded by the APA’s adoption of a model of multicultural competencies (MCCs) to guide therapists through the cultural complexities that arise during the process of psychotherapy (APA, 2003). This widely accepted model, based upon the work of Sue, Arredondo, and McDavis (1992), asserts that psychotherapists must develop knowledge, skills, and awareness of culture in order to best serve their clients. More specifically, Sue and Sue (2008) assert that therapists need to have awareness and sensitivity to clients’ cultural heritage as
well as the awareness of therapist biases that may influence their clients. With regard to knowledge, therapist must be informed about systems that their clients operate in and barriers that they face as a result of their diverse backgrounds. Lastly, skills relate to therapists’ ability to communicate effectively with clients in styles that are appropriate according to culture (Sue & Sue, 2008). Stemming from this, researchers have investigated associations between client’s perceptions of their therapists’ MCCs and other therapeutic processes and outcomes. For instance, client-rated MCCs were positively associated with satisfaction with counseling and with therapy outcomes (e.g., Constantine, 2002; Fuertes & Brobst, 2002; Owen, Leach, Wampold, & Rodolfa, 2011), as well as with clients’ ratings of their therapists’ general competence, empathy, and credibility (Fuertes & Brobst, 2002; Fuertes et al., 2006; Kim, Li, & Liang, 2002). Additionally, clients’ ratings of their therapists’ MCCs were connected to key process variables, including working alliance and the real relationship (Fuertes et al., 2006; Owen, Tao,
Correspondence concerning this article should be addressed to Joanna M. Drinane, Department of Counseling Psychology, University of Denver, Denver, CO, USA. Email: [email protected] © 2014 Society for Psychotherapy Research
Downloaded by [Deakin University Library] at 00:39 06 November 2015
2
J. Drinane et al.
Leach, & Rodolfa, 2011). With regard to other cultural factors, clients’ ratings of therapists’ MCCs have also been positively correlated with cultural humility (Hook, Davis, Owen, Worthington, & Utsey, 2013) and negatively correlated with racial microaggressions (Constantine, 2007). Collectively, these studies highlight the importance of therapists’ MCCs as an integral part of the clients’ experiences in therapy. However, tools for measurement of this construct from a client’s vantage point are imperfect and have not undergone vigorous validation procedures. Accordingly, we will examine two pressing validity concerns, content and construct, within the MCCs literature. Currently, there are three primary means of acquiring ratings of therapists’ MCCs: (i) supervisors/observers, (ii) therapists, and (iii) clients. Supervisor/observer rating, while potentially useful as it connects to how therapists in training are evaluated, is resource intensive and is seldom used in the literature. Additionally, supervisor ratings are typically not standardized, which can result in measurement inconsistency. Therapist ratings of their own MCCs have been criticized because they often depict multicultural counseling self-efficacy, or confidence and anticipation about implementing this skill set, as opposed to counselors’ actual ability to perform multicultural counseling behaviors successfully (Constantine & Ladany, 2000). According to Fuertes et al. (2006), clients’ ratings of therapists’ MCCs were correlated at r = −.03 with therapists’ ratings of their own MCCs. This near null correlation suggests that clients’ and therapists’ perceptions of therapists’ MCCs are not congruent. Clients’ ratings of MCCs have been deemed an ideal way of reflecting clients’ perceptions of their therapists’ MCCs, because they fundamentally bring clients’ voice and experience into the process. Each of the studies described above, which examined clients’ ratings of therapists’ MCCs, used the Cross-Cultural Counseling Inventory-Revised measure (CCCI-R; LaFromboise, Coleman, & Hernandez, 1991). This scale was developed based on the tripartite model of MCCs (i.e., knowledge, skills, and awareness; Sue et al., 1992). The CCCI-R was not initially designed and validated for use by clients; rather, it was intended to be a supervisor/ observer-rated scale. Due to lack of another measure to obtain clients’ ratings, researchers made changes to voice, subject, and content of scale items in order to adapt them for client use. For example, as supervisorrated item, item 13 reads: “Communication is appropriate for the client.” This item was changed for clients to be: “My counselor sends messages that are appropriate to me based on my cultural heritage.” Another supervisor-rated item is: “Aware of professional
responsibilities.” The clients’ version reads, “My counselor is aware of the professional and ethical responsibilities of a counselor.” Changes like these can influence item meaning and message. To date, the client-rated version of the CCCI-R has missed some important steps in the validation process (e.g., content validation), yet it is one of the most widely used clientrated measures for therapists’ MCCs. Content validation is foundational to measurement development and to the ability to accurately reflect and draw inferences relating to an intended construct (Heppner, Wampold, & Kivlighan, 2007). As a first step in this study, we will examine the content validity of the client-rated version of the CCCI-R. Also to do with validity, another existing concern in the MCC literature has to do with whether or not MCCs are truly distinct from other general competencies or therapeutic processes. This relates to construct validity, as can be examined by how well we can discriminate between these two forms of competencies (cultural and general). In 1998, Coleman posed the question, “General and Multicultural Competency: Apples and Oranges?” when he found a high correlation between MCCs and general competencies. Similarly, high correlations have been observed between scores on the CCCI-R and scores on a number of measures of general therapeutic process variables, leaving questions about the unique nature of MCCs. In particular, consistently high correlations are demonstrated between clientrated MCCs (via the client-rated version of the CCCI-R which had not been fully validated) and client-rated working alliance (typically measured by the Working Alliance Inventory–Short Form Revised; Hatcher & Gillaspy, 2006). For example, Constantine (2007) found these two scales to be correlated at .70 (n = 40), and Owen, Tao, et al. (2011) and Fuertes et al. (2006) both found correlations of .73 (n = 276; n = 51, respectively). Although these correlations demonstrate the strong relationship between client-rated MCCs and working alliance, it is unclear if MCCs are accurately being measured and if they can be subsumed under the category of general therapeutic process competencies. The distinction between therapeutic alliance and MCCs is best tested from a measurement standpoint (e.g., construct validity), and to date, we are not aware of any studies that approached this question in this manner. Theory supports three measurement models of potential relationships that may exist between these two constructs. The first model (Distinct yet Related), displayed in Figure 1, is one in which the two factors (i.e., alliance and MCCs) are based on separate items but the overall factors are correlated. The second model (Distinct and Separate), displayed in Figure 2, is consistent with
Downloaded by [Deakin University Library] at 00:39 06 November 2015
Psychotherapy Research
3
Figure 1. Model 1: Distinct yet Related. Note: 1–12 = items from WAI-SR, 1–7 = items retained from CCCI-R.
the first model; however, there is no proposed correlation between the two factors. These models indicate that these constructs are defined differently (i.e., separate items for each factor). Although there is evidence in previous studies that MCCs and alliance are related (e.g., Constantine 2007; Fuertes et al., 2006), we tested Models 1 and 2 to estimate the association between alliance and MCCs after accounting for measurement considerations. The third (All for One), displayed in Figure 3, is based upon the high correlations between alliance and MCCs wherein these associations may reflect that these constructs are not separate. As such, all of the items for working alliance and multicultural competence load onto one global factor. As such, Models 1 and 2 propose that alliance and MCCs are distinct constructs (related or not), and Model 3 suggests that alliance and MCCs are not distinct constructs.
The Current Studies In Study 1, we will examine the content validity of the client-rated version of the CCCI-R based on expert feedback. In this process, we anticipate that we will screen out some items according to expert
Figure 2. Model 2: Distinct and Unrelated. Note: 1–12 = items from WAI-SR, 1–7 = items retained from CCCI-R.
feedback; however, unique from traditional measurement validation, we will not add new items. Rather, our focus is to better understand the validity of the CCCI-R and to determine which of its items can be used effectively in a client-rated capacity. Following this process, with the validated CCCI-R items, in Study 2, we will examine the factor structure of the remaining items and also test the distinctness of the following two constructs: clientrated MCCs and working alliance.
Study 1 Method Participants and procedures. Participants for the content validation conducted in Study 1 were recruited via email. Emails contained requests for participation, an introduction describing our motivation for conducting such a validation, and a link to the survey questions on the PsychData website. Those in receipt of an email were among a convenience sample and were preselected according to their
4
J. Drinane et al. and followed by a question to assess item quality: “How well does this item represent multicultural competence?” Experts rated each item using a Likert scale ranging from −3 (really bad item) to +3 (really good item). Following each item was also a space for experts to provide open-ended responses with commentary on the utility of each item for client use. In these responses, experts elaborated upon concerns they had about clients being able to understand the content of or even provide a meaningful score for each item.
Downloaded by [Deakin University Library] at 00:39 06 November 2015
Results and Discussion
Figure 3. Model 3: All for One. Note: 1–12 = items from WAI-SR, 1–7 = items retained from CCCI-R-7.
expert credentials. Such credentials involved having a doctoral-level degree, research expertise in psychotherapy, and therapist MCCs, and at least one (some with many more) peer reviewed publication in each of the following topic areas: MCCs and psychotherapy process/outcome research. Nineteen responses were obtained. The sample included 10 men and 9 women. Of the 19, 11 experts self-identified as some minority (e.g., racial ethnic minority [REM] or sexual minority). Due to confidentiality, we opted not to include more specific demographic information. Measures. The survey presented initial instructions with the aims of the project as well as a commonly utilized definition of MCCs focusing on therapists’ ability to implement knowledge, skills, and awareness regarding clients’ cultural heritage and social group identity/membership (Alvarez & Chen, 2008; APA, 2003; Sue, 1998; Sue & Torino, 2005). This framework for MCCs is consistent with the definition utilized in the development of the original items for the CCCI-R (LaFromboise et al., 1991). Next, the 20 CCCI-R items were displayed
Means and standard deviations for the quality ratings of the 20 items are presented in Table I. Items with higher means were rated as being better at targeting the underlying construct of therapists’ MCCs. Along with their item MCC quality ratings, experts offered written reactions to these items. These written reactions generally denoted issues regarding to the clarity, or lack thereof, of the concepts. The two lead authors evaluated open-ended responses to detect concerns by experts about item appropriateness. Responses that indicated that an item might be unclear resulted in items being flagged for further review by our research team of nine doctoral students in an APA accredited Counseling Psychology program. The team reached consensus regarding the severity of expert concerns and used this to eliminate items deemed ineffective at measuring client-rated MCCs. If experts indicated that a given item was appropriate for client use, it was then considered within the context of how well it depicted MCCs. Items with a MCCs score greater than zero were included as this indicated that these were rated as positive/good at capturing the meaning of MCCs. Items and their quality ratings are displayed in Table I. Fifteen items were rated above zero. From a quantitative standpoint, zero was selected as the cutoff due to the face validity associated with using a scale with negative and positive ends. To be comprehensive and also true to the process of content validity, we integrated the experts’ comments on each item. Clearly, the experts have more nuanced understanding of the construct that a single numerical rating could reveal. As such, we wanted to honor their voice in this content validation process. Using description provided from experts, we flagged a number of items that were indicated as not being appropriate for collecting data from clients (i.e., they were confusing or experts found them not reasonable for clients to answer). Adding their qualitative reactions allowed us to screen for items whose content related to the construct, but that should not have been used in this fashion. To be retained,
Psychotherapy Research
5
Table I. Cross-cultural counseling inventory-revised: content experts’ ratings.
Downloaded by [Deakin University Library] at 00:39 06 November 2015
Item (20) My counselor acknowledges and is comfortable with cultural differences (14) My counselor attempts to perceive my problems within the context of my cultural experience, values, and/or lifestyle (2) My counselor values and respects cultural differences (7) My counselor demonstrates knowledge about my culture (13) My counselor sends messages that are appropriate to me based on my cultural heritagea (6) My counselor understands the current socio-political system and its impact on mea (9) My counselor is aware of institutional barriers which might affect my circumstancesa (17) My counselor recognizes those limits determined by the cultural differences between usa (3) My counselor is aware of how his or her own values might affect me (1) My counselor is aware of his or her own cultural heritagea (5) My counselor is willing to suggest referral when cultural differences are extensivea (4) My counselor is comfortable with differences between us (16) My counselor is at ease talking with me (19) My counselor is aware of the professional and ethical responsibilities of a counselora (15) My counselor presents his or her own values to mea (8) My counselor has a clear understanding of counseling and the therapy process (12) My counselor is able to suggest institutional intervention skills that favor me (10) My counselor elicits a variety of verbal and non-verbal responses from me (18) My counselor appreciates that my social status is based on ethnic status (11) My counselor accurately sends and receives a variety of verbal and non-verbal messagesa
Mean
SD
2.69
0.60
2.69
0.60
2.38 2.31 2.13
0.72 0.79 1.12
2.06
1.24
2.00
0.82
1.80
1.32
1.63 1.50 1.44
1.02 1.75 1.15
1.38 1.13 0.56
1.31 2.13 2.06
0.25 –1.94 –1.93 –1.81 –1.77 –1.75
1.77 2.26 2.02 1.90 2.24 1.77
Note. Bolded items were retained for the CCCI-R-7. a Items were noted as being unclear by expert reviewers.
items had to go through this rigorous, multistep evaluation process. For example, item number 9 has a mean score of 2. This item is, “My counselor is aware of institutional barriers which might affect my circumstances.” In response, one rater noted, “Ok item, but might be hard for the client to rate unless the counselor explicitly discussed these in session.” In response to this feedback, this item was flagged and the panel determined that it should be excluded from an abbreviated scale. Therefore, any of the 15 items rated above zero, which did not receive expert feedback contraindicating its use will be retained and considered part of an abbreviated scale. To note, several reviewers commented that their ratings on the scale were disconnected from their reactions regarding the suitability of the item. As such, we weighted the totality of the responses. Expert ratings of the items of CCCI-R indicated that 7 of the 20 should be retained (referred to as the CCCI-R-7 for the remainder of the description of the analyses). The first is item 20, “My counselor acknowledges and is comfortable with cultural differences.” This assesses therapists’ ability to express acceptance and comfort of clients’ cultural heritage. The second is item 14, “My counselor attempts to perceive my problems within the context of my
cultural experience, values, and/or lifestyle.” This item targets therapists’ ability to integrate culture into conceptualization. The third is item 2, “My counselor values and respects cultural differences.” This item describes therapists’ expression of regard for clients’ culture. The fourth is item 7, “My counselor demonstrates knowledge about my culture.” This item assesses how informed therapists are about culture. The fifth item, item 3 states “My counselor is aware of how his or her own values might affect me.” This item targets therapists’ ability to introspect and be sensitive to imposing their own values on the client. Items six (number 4— “My counselor is comfortable with differences between us”) and seven (number 16— “My counselor is at ease talking with me”) both relate to therapists’ ability to appear genuinely comfortable during discussions with clients, specifically about cultural differences and otherwise. As these seven items combine to form a measure of clients’ ratings of therapist’s MCCs that is likely more appropriate than the entire CCCI-R, we can now turn to our second purpose, which is to determine whether client-rated MCCs are distinct from other psychotherapy process factors, the alliance.
6
J. Drinane et al. Study 2
Downloaded by [Deakin University Library] at 00:39 06 November 2015
Method Participants Clients. This sample included data from 279 clients compiled from two previous data sets (Owen, Tao, et al., 2011; Owen, Leach, et al., 2011). Although these data were compiled for previously conducted studies, the analysis performed in the present study is novel. Of this sample, 151 self-identified as white and 127 self-identified as an REM. Additionally, 74.9% identified as women, 23.7 identified as men, and 1.1% identified as transgender or other nonbinary gender. The average age was 24.44 years old (SD = 5.42, Median = 23). Clients’ data were collected from a university counseling center where students partook in brief psychotherapy (6–10 sessions). The median number of sessions was six. Therapists. In this study, 40 therapists treated the clients (M = 6.98 clients/therapist; range = 2–21). The therapists were predoctoral interns, postdoctoral fellows, staff psychologists, and staff psychotherapists from a large university counseling center located in the Western United States. Eleven of the psychotherapists self-identified as REM, and 29 self-identified as White. Information regarding the therapeutic approach of psychotherapists was not gathered. Measures The Cross-Cultural Counseling Inventory–Revised (CCCIR; LaFromboise et al., 1991). The CCCI-R was used to assess clients’ perceptions of their therapists’ MCCs. The CCCI-R has 20 items, which are rated on a 6point scale ranging from 1 (Strongly Disagree) to 6 (Strongly Agree). Higher scores indicate more perceived MCCs. However, for the purpose of this analysis, only the seven items retained from the content validation described above were included. These seven items (numbers 2, 3, 4, 7, 14, 16, and 20) were examined using exploratory factor analysis with principle axis extraction, with oblimin rotation. A Scree test (Cattell, 1966) and eigenvalues over one indicated that a onefactor solution should be retained. All of the items loaded onto one factor, with the factor loadings being high (all above .63). Additionally, the Cronbach’s alpha for the 7-item scale was .91. Working Alliance Inventory—Short Form Revised (WAI-SR; Hatcher & Gillaspy, 2006). The WAISR is a 12-item client-rated measure of working alliance. It assesses goals and tasks for psychotherapy and the client/psychotherapist relational bond. Items like “I believe [therapist] likes me” and “We agree on what is important for me to work on” are rated on a 1–7 scale, where 1 is “strongly disagree” and 7 is
“strongly agree.” Higher scores are indicative of a better working alliance. This scale is considered to be pantheoretic and captures three important facets of working alliance: (i) agreement on the tasks of therapy, (ii) agreement on the goals of therapy, and (iii) development of an affective bond. The reliability and validity have been demonstrated numerous times through comparison with other measures of working alliance and psychotherapy outcome (Hatcher & Gillaspy, 2006). The Cronbach’s alphas for the three subscales were: goals alpha = .91, tasks alpha = .93, and bond alpha = .93. Procedure As is outlined (Owen, Tao, et al., 2011; Owen, Leach, et al., 2011), participants were recruited from a large university counseling center located on the west coast in the United States. Clients were asked on their intake card(s) whether they would be willing to receive a survey about their therapy experience. Those who agreed were sent an email at the end of the academic quarter and were able to access the anonymous survey instruments online. If clients endorsed multiple individual therapists (or no therapist), they were excluded from the analyses. Additionally, clients who did not attend individual therapy (e.g., couple therapy, group therapy, or psychiatric appointments) were excluded from the sample. Clients initially completed an informed consent and then the measures. For purposes of this study, clients were specifically directed to rate their therapists on the WAI-SR and CCCI-R measures. The survey responses were collected anonymously (the participants’ email addresses were not linked to their responses), and all procedures were approved through the university’s institutional review board committee. Data Analysis Overview: Description of Hypothesized Models Three theoretically based models were defined to determine whether client-rated MCCs were conceptually distinct or not from their ratings of the alliance. Confirmatory factor analysis (CFA) was used to determine which of the three models might be the best representation of the relationship of the items on the CCCI-R-7 and the WAI-SR. Items for the working alliance were modeled from the three subscales of the WAI-SR (i.e., goals, tasks, and bonds). Indicators of MCCs included the seven items from the CCCI-R that were supported in Study 1. Model 1 (Distinct yet Related) correlated a global alliance factor with MCCs indicating their
Downloaded by [Deakin University Library] at 00:39 06 November 2015
Psychotherapy Research relatedness in the therapy process. The global alliance factor had three indicators (the alliance subscales: goals, tasks, and bonds), each predicting four alliance items and the MCCs factor had the seven indicators from the CCCI-R-7. This model suggests that from a measurement standpoint MCCs are distinct from alliance, but that the two constructs are related. Model 2 (Distinct and Separate) used the same variables; however, the global alliance factor was not correlated with the MCCs factor. This model suggests that MCCs are distinct from alliance at the measurement level and that there is no relationship between the factors. Model 3 (All for One) had the 12 WAI-SR items and the seven items from the CCCI-R-7 load onto one factor. This model suggests MCCs and alliance are best measured as one global construct. To evaluate model fit, we used the following statistics: confirmatory fit index (CFI), root mean square error of approximation (RMSEA), standardized root mean square residual (SRMR), and adjusted Bayesian information criterion (BIC). According to Kline (2011), model fit is considered to be good/acceptable when CFI is above .90, with greater values indicating better fit. RMSEA and SRMR values that are less than .08 are also indicative of acceptable model fit. Additionally, adjusted BIC values were provided as a means for model comparison, which do not rely on statistical significance tests. Although there are no clear cut-offs for how much lower the better fitting model should be, Raftery (1995) suggests that differences over 10 points is strong evidence for the better fitting model.
Results Model fit statistics for the three models are presented in Table II. The All for One model had poor overall model fit according to CFI, RMSEA, and SRMR statistics. The Distinct and Separate model showed better overall fit than the All for One model. However, for the Distinct and Separate model, RMSEA and SRMR were both above .08, which is Table II. Summary of model fit statistics. Model
CFI
RMSEA
SRMR
Adjusted BIC
Distinct yet Related Distinct and Unrelated All for One
.93 .92
.09 .09
.07 .20
12,172.13 12,218.88
.66
.20
.15
13,473.60
Note. Model 1, Distinct yet Related–– 3 alliance subscales, 1 global alliance, and 1 MCC (alliance and MCC correlated). Model 2, Distinct and Unrelated–– 3 alliance subscales, 1 global alliance, and 1 MCC (alliance and MCC uncorrelated). Model 3, All for One–– All items loading on one factor.
7
Table III. Unstandardized estimates for Model 1 (Distinct yet Related).
Factors and items
WAI WAI WAI WAI
Item Item Item Item
1 4 8 11
WAI WAI WAI WAI
Item Item Item Item
2 5 9 10
WAI WAI WAI WAI
Item Item Item Item
3 6 7 12
CCCI-R-7 CCCI-R-7 CCCI-R-7 CCCI-R-7 CCCI-R-7 CCCI-R-7 CCCI-R-7
Item Item Item Item Item Item Item
1 2 3 4 5 6 7
Estimate
Standard error
Goals 1.00 0.00 0.93 0.06 1.15 0.08 1.14 0.07 Tasks .91 0.05 .97 0.04 1.00 0.00 1.02 0.04 Bond 1.00 0.00 0.75 0.04 1.08 0.05 1.11 0.05 Cultural competence 1.00 0.00 0.85 0.07 0.95 0.06 1.08 0.07 1.22 0.08 0.95 0.05 1.07 0.05
95% Confidence interval lower, upper
1.00, 0.81, 0.99, 1.00,
1.00 1.05 1.31 1.28
0.81, 0.89, 1.00, 0.94,
1.01 1.05 1.00 1.10
1.00, 0.67, 0.98, 1.01,
1.00 0.83 1.18 1.21
1.00, 0.71, 0.83, 0.94, 1.06, 0.85, 0.97,
1.00 0.99 1.17 1.22 1.38 1.05 1.17
representative of poor model fit. The Distinct yet Related model had acceptable overall fit and was favored according to adjusted BIC and SRMR estimates. As the Distinct yet Related model demonstrated acceptable model fit and was superior to the other two models, we will proceed by interpreting it further. Distinct yet Related model supports the notion that the measurement of MCCs is distinct from the alliance, and further, that the two constructs are associated with one another (r = .43, p < .001). In other words, the way in which clients respond to questions on these topics is, in fact, different and should not be considered as representing one over-arching topic (as depicted by the All for One model). Further evidence is provided in Table III, which provides the unstandardized estimates for the Distinct yet Related model, and Figure 4 shows the standardized estimates for this model. We attempted to conduct a multiple group analysis of the models for REM and White clients; however, the models would not converge, likely due to a relatively small size for the number of estimated parameters. Additionally, a multilevel CFA was also attempted; however, the sample size at the therapist level was insufficient.
Downloaded by [Deakin University Library] at 00:39 06 November 2015
8
J. Drinane et al.
Figure 4. estimates.
Final model, Distinct and Related: standardized
Discussion Two main results emerged from these two studies. First, experts noted that the measurement of clientrated MCCs via the CCCI-R is better represented by only 7 of the 20 items. To our knowledge, little attention has been paid to the measurement properties of the client-rated version of the CCCI-R, despite it being the most often utilized client-rated MCCs measure (e.g., Constantine, 2007; Fuertes et al., 2006; Owen, Leach, et al., 2011). The 7-item version of the scale was supported via feedback from experts and has utility as a brief measure of therapists’ MCCs. These seven items integrate important elements of demonstrated knowledge, sensitivity toward and respect for cultural identity, as well as the ability to infuse clients’ cultural beliefs into treatment (Sue et al., 1998; Wampold, 2007). Moreover, these items capture that therapists are active in taking opportunities to infuse clients’ culture in the therapy process and that they are comfortable with cultural topics in therapy. Common issues among items that were not retained include: being beyond the scope of clients’ knowledge, being outside of clients’ conscious experience, being too general, or not being applicable to most
clients. For example, item 12 reads, “My counselor is able to suggest institutional intervention skills that favor me.” In this case, clients may not know about such interventions or this may not be applicable to them based on their circumstances. There is a need for more attention to measurement as there are very few measures of multicultural constructs in psychotherapy. Indeed, there are some new measures and approaches to examining therapists’ MCCs emerging (e.g., Hook et al., 2013; Imel et al., 2011). Second, from a measurement standpoint, clientrated MCCs are distinct from the working alliance, yet the two constructs are related. This theoretically based model is likely the most representative of the relationship between client-rated MCCs and working alliance, as can be determined by its superior fit over the other two models. Importantly, the poor fit of the All for One model indicates that the items for alliance and MCCs are not adequately represented as one global factor or one construct. These findings are also consistent with theoretical positions suggesting that MCCs have a unique role in the therapeutic process (Sue, 2003; Owen, Tao, et al., 2011). Clients’ perceptions of their therapists’ MCCs are different than how clients perceive their agreement on the goals for therapy, the agreement on the ways to reach those goals, and the emotional bond with their therapists. Clearly, the moderate to high correlation (r = .48) between client-rated MCCs and alliance illustrates that these constructs covary; moreover, theoretically they work in concert to enhance therapy outcomes (Hook et al., 2013; Owen, Tao, et al., 2011). Limitations and Future Research The strengths of our work should be interpreted in concert with the limitations. First, our sample size, while relatively large for a psychotherapy study, was limited in some important ways. For example, we attempted to run a multiple group analysis to determine if the Distinct but Related model functioned similarly among REM clients and White clients. Unfortunately, the model would not converge. Statistical power and the ability to estimate the model would be enhanced if we had more participants in each group. In addition, a larger sample size might have enabled us to assess the model fit for various racial and ethnic groups (e.g., African American) in concert with their intersecting of cultural identities (e.g., African American women). However, although not ideal, the acceptable model fit utilizing the overall sample is sufficient to inform future studies. Alternatively, the acceptable nature of the fit of the models tested suggests that other models that better
Downloaded by [Deakin University Library] at 00:39 06 November 2015
Psychotherapy Research represent these constructs may exist. Clearly, such models need to be considered via a theoretical frame; there could be some rationale for examining the overlap between the alliance-bond subscale and clients’ perceptions of MCCs (see Owen, 2013) or other related constructs, such as the real relationship (Gelso, 2009). The prospect of utilizing different models might also address issues regarding lack of fit of the multiple group analysis. For instance, there may be different models for REM versus White clients that could be useful to explore. Second, our findings must also be interpreted with knowledge of the abbreviated nature of the 7-item version of the CCCI-R. Although the seven items that were retained were deemed by experts to be good items and to be appropriate for client use, they may not encompass all that there is to know/measure about MCCs. We did not seek to generate more items, but instead chose to take the first step in refining measurement of this construct by trimming the measure that is considered to be the standard in the field. Therefore, there is a continued need for development of client-rated measures for assessing MCCs and for the comparison of such measures to other related therapeutic processes. Third, this study used cross-sectional retrospective data, which limited our ability to gather more detailed information regarding clients’ presenting problems, initial severity, and changes over the course of therapy. At this point, there are no known prospective studies of clients’ perceptions of their therapists’ MCCs. More work is needed in this area, and with refined measurement, the field will be better able to take strides toward this goal. Future studies should further investigate in more detail how therapists’ MCCs influence therapy outcomes. For example, Owen, Tao, et al. (2011) proposed that therapists’ MCCs influence the working alliance, which in turn affects therapy outcomes (i.e., mediation model). In this way, the current results support the connection between the alliance and therapists’ MCCs. However, it could be that therapists’ MCCs distinctly predict therapy outcomes over and above the role of the alliance. Testing these models will be important to understanding how therapists’ MCCs influence the therapeutic process. Implications Ultimately, both Study 1 and Study 2 have implications for how the field can understand MCCs. We hope that the results from Study 1, coupled with previous reviews of the limitations in the literature, serve as a call to the profession to be more mindful of measurement issues in the assessment of
9
culturally based measures in psychotherapy (see Worthington, Soth-McNett, & Moreno, 2007). Indeed, it should be sobering that the primary measure of client-rated MCCs has such content validity concerns. At the same time, we are encouraged that this study will provide some confidence that the seven items that were retained from the CCCI-R can provide a broad frame for the assessment of therapists’ MCCs. Study 2 may influence how we conceptualize MCCs as compared to other more general competencies (cf. Coleman, 1998). As MCCs are distinct from, yet related to working alliance, it is useful to disentangle therapists’ strength on alliance formation and their orientation toward their clients’ cultural identity. While connected, these skills are unique. Being clear on the specific therapeutic processes that are related to each construct will ultimately aid in this endeavor (also see Owen, 2013). For instance, it appears that how therapists are able to facilitate goals for treatment (part of the alliance) is unique from how they consider and integrate clients’ cultural heritage into the treatment process. In doing so, these distinct processes may contribute collectively to clients’ overall therapeutic outcomes. Consequently, we hope that our work will encourage more dialog and empirical work on therapists’ MCCs. References Alvarez, A. N., & Chen, G. A. (2008). Ruth as an Asian American: A multicultural, integrative perspective. In G. Corey (Ed.), Case approach to counseling and psychotherapy (pp. 298–306). Belmont, CA: Wadsworth. American Psychological Association. (2003). Guidelines on multicultural education, training, research, practice, and organizational change for psychologists. APA Council of Representatives. American Psychologist, 58, 377–402. American Psychological Association. (2010). Ethical principles of psychologists and code of conduct (Original published 2002, amended June 1, 2010). Retrieved from http://www.apa.org/ ethics/code/index.aspx Cattell, R. B. (1966). The data box: Its ordering of total resources in terms of possible relational systems. In R. B. Cattell (Ed.), Handbook of multivariate experimental psychology (pp. 67–128). Chicago, IL: Rand-McNally. Coleman, H. L. K. (1998). General and multicultural counseling competency: Apples and oranges? Journal of Multicultural Counseling and Development, 26(3), 147–156. doi:10.1002/j. 2161-1912.1998.tb00194.x Constantine, M. G. (2002). Predictors of satisfaction with counseling: Racial and ethnic minority clients’ attitudes toward counseling and ratings of their counselors’ general and multicultural counseling competence. Journal of Counseling Psychology, 49, 255–263. doi:10.1037/0022-0167.49.2.255 Constantine, M. G. (2007). Racial microaggressions against African American clients in cross-racial counseling relationships. Journal of Counseling Psychology, 54(1), 17–31. doi:10.1037/0022-0167. 54.1.17 Constantine, M. G., & Ladany, N. (2000). Self-report multicultural counseling competence scales: Their relation to social desirability attitudes and multicultural case conceptualization.
Downloaded by [Deakin University Library] at 00:39 06 November 2015
10
J. Drinane et al.
Journal of Counseling Psychology, 47, 155–164. doi:10.1037/ 0022-0167.47.2.155 Fuertes, J. N., & Brobst, K. (2002). Clients’ ratings of counselor multicultural competency. Cultural Diversity & Ethnic Minority Psychology, 8, 214–223. doi:10.1037/1099-9809.8.3.214 Fuertes, J. N., Stracuzzi, T. I., Bennett, J., Scheinholtz, J., Mislowack, A., Hersh, M., & Cheng, D. (2006). Therapist multicultural competency: A study of therapy dyads. Psychotherapy, 43, 480–490. doi:10.1037/0033-3204.43.4.480 Gelso, C. J. (2009). The real relationship in a postmodern world: Theoretical and empirical explorations. Psychotherapy Research, 19, 253–264. Hatcher, R. L., & Gillaspy, J. A. (2006). Development and validation of a revised short version of the working alliance inventory. Psychotherapy Research, 16(1), 12–25. doi:10.1080/ 10503300500352500 Heppner, P. P., Wampold, B. E., & Kivlighan, Jr., D. M. (2007). Research design in counseling (3rd ed.). Belmont, CA: Brooks Cole. Hook, J. N., Davis, D. E., Owen, J., Worthington, E. L., & Utsey, S. O. (2013). Cultural humility: Measuring openness to culturally diverse clients. Journal of Counseling Psychology, 60, 353–366. doi:10.1037/a0032595 Imel, Z. E., Baldwin, S. A., Atkins, D., Owen, J., Baardseth, T., & Wampold, B. E. (2011). Racial/ethnic disparities in therapist effectiveness: A conceptualization and initial study of cultural competence. Journal of Counseling Psychology, 58, 290–298. Kim, B. S. K., Li, L. C., & Liang, C. T. H. (2002). Effects of Asian American client adherence to Asian cultural values, session goal, and counselor emphasis of client expression on career counseling process. Journal of Counseling Psychology, 49, 342–354. doi:10.1037/0022-0167.49.3.342 Kline, R. B. (2011). Principles and practice of structural equation modeling (3rd ed.). New York, NY: Guilford Press. LaFromboise, T. D., Coleman, L. K., & Hernandez, A. (1991). Development and factor structure of the cross-cultural counseling inventory-revised. Professional Psychology: Research and Practice, 22, 380–388. doi:10.1037/0735-7028.22.5.380
Owen, J. (2013). Early career perspectives: Therapist effects, cultural dynamics, and other processes in psychotherapy. Psychotherapy, 50, 496–502. Owen, J., Leach, M., Wampold, B., & Rodolfa, E. (2011). Client and therapist variability in clients’ perceptions of their therapists’ multicultural competencies. Journal of Counseling Psychology, 58(1), 1–9. doi:10.1037/a0021496 Owen, J., Tao, K., Leach, M., & Rodolfa, E. (2011). Clients’ perceptions of their psychotherapists’ multicultural orientation. Psychotherapy, 48, 274–282. doi:10.1037/a0022065 Raftery, A. E. (1995). Bayesian model selection in social research (with discussion). Sociological Methodology, 25, 111–195. doi:10.2307/271063 Sue, D. W. (2003). Overcoming our racism: The journey to liberation. San Francisco, CA: Jossey-Bass. Sue, D. W., Arredondo, P., & McDavis, R. J. (1992). Multicultural counseling competencies and standards: A call to the profession. Journal of Counseling & Development, 70, 477–486. doi:10.1002/j.1556-6676.1992.tb01642.x Sue, D. W., Carter, R. T., Casas, J. M., Fouad, N. A., Ivey, A. E., Jensen, M., … Vazquez-Nutall, E. (Eds.). (1998). Multicultural counseling competencies: Individual and organizational development. Thousand Oaks, CA: Sage. Sue, D. W., & Torino, G. C. (2005). Racial-cultural competence: Awareness, knowledge, and skills. Handbook of RacialCultural Psychology and Counseling: Training and Practice, 2, 3–18. Sue, D. W., & Sue, D. (2008). Counseling the culturally diverse: Theory and practice (5th ed., p. 45). Hoboken, NJ: John Wiley & Sons. Sue, S. (1998). In search of cultural competence in psychotherapy and counseling. American Psychologist, 53, 440–448. doi:10.1037/0003-066X.53.4.440 Wampold, B. E. (2007). Psychotherapy: The humanistic (and effective) treatment. American Psychologist, 62, 857–873. Worthington, R. L., Soth-McNett, A. M., & Moreno, M. V. (2007). Multicultural counseling competencies research: A 20-year content analysis. Journal of Counseling Psychology, 54, 351–361. doi:10.1037/0022-0167.54.4.351
