555884

research-article2014

ASMXXX10.1177/1073191114555884AssessmentShkalim

Article

Psychometric Evaluation of the MMPI-2/MMPI-2-RF Restructured Clinical Scales in an Israeli Sample

Assessment 1­–12 © The Author(s) 2014 Reprints and permissions: sagepub.com/journalsPermissions.nav DOI: 10.1177/1073191114555884 asm.sagepub.com

Eleanor Shkalim1

Abstract The current study cross-culturally evaluated the psychometric properties of the Minnesota Multiphasic Personality Inventory–2 (MMPI-2)/MMPI-2–Restructured Form Restructured Clinical (RC) Scales in psychiatric settings in Israel with a sample of 100 men and 133 women. Participants were administered the MMPI-2 and were rated by their therapists on a 188-item Patient Description Form. Results indicated that in most instances the RC Scales demonstrated equivalent or better internal consistencies and improved intercorrelation patterns relative to their clinical counterparts. Furthermore, external analyses revealed comparable or improved convergent validity (with the exceptions of Antisocial Behavior [RC4] and Ideas of Persecution [RC6] among men), and mostly greater discriminant validity. Overall, the findings indicate that consistent with previous findings, the RC Scales generally exhibit comparable to improved psychometric properties over the Clinical Scales. Implications of the results, limitations, and recommendations for future research are discussed. Keywords MMPI-2, MMPI-2-RF, Clinical Scales, RC Scales, personality assessment, clinical sample The Minnesota Multiphasic Personality Inventory–2 (MMPI-2; Butcher et al., 2001) has been the most widely used and researched self-report objective test for broadspectrum clinical assessment of psychopathology and personality characteristics (Camara, Nathan, & Puente, 2000). In 2008, a shorter revised version of the MMPI-2, the MMPI-2–Restructured Form (MMPI-2-RF; Ben-Porath & Tellegen, 2011; Tellegen & Ben-Porath, 2011) was released with the Restructured Clinical (RC) Scales (Tellegen et al., 2003) at its core. The RC Scales were designed to address conceptual overlap and content heterogeneity (Loevinger, 1972; Norman, 1972). A detailed description of the RC Scales development process is provided by Tellegen et al. (2003). The researchers’ approach involved factoring out a general common emotional distress factor, which they labeled Demoralization, and which they theoretically conceptualized as the MMPI-2 equivalent of the pleasantness–unpleasantness dimension in Watson and Tellegen’s (1985) mood model. This broad affect-laden dimension is the highest level of the affect hierarchy, and it reflects variations in general hedonic tone (Tellegen, Watson, & Clark, 1999). The RC Scales development process yielded a set of nine nonoverlapping scales whose labels reflect the core component assessed by each. No corresponding RC Scales were constructed for Scales 5 (Masculinity/Femininity) and 0 (Social Introversion), as they do not measure components

of psychopathology (Tellegan et al., 2003). The restructuring significantly changed the focus of RC3 (Cynicism), which represents the relatively narrow and reversed component of the original Scale 3 (Hysteria; Tellegen et al., 2003). Hence, the association between RC3 and Scale 3 is expected to be weak. In their monograph, Tellegen et al. (2003) initially validated the RC Scales and compared their features with those of their original counterparts in two outpatient and inpatient samples. The authors reported equal to improved internal consistencies as well as equivalent or superior convergent and discriminant validity patterns. Moreover, as expected, the RC Scales demonstrated less saturation with demoralization than the Clinical Scales. Extensive research appearing since the publication of the RC Scales has documented their improved psychometric properties over the Clinical Scales. Wallace and Liljequist (2005) evaluated convergence between the two sets of scales using an outpatient sample. The RC Scales were less intercorrelated, and most were strongly associated with their clinical counterparts. Mean RC Scales scores were 1

University of Haifa, Haifa, Israel

Corresponding Author: Eleanor Shkalim, Department of Psychology, Faculty of Social Sciences and Mathematics, University of Haifa, 199 Aba-Hushi Avenue, Mount Carmel, Haifa 3498838, Israel. Email: [email protected]

Downloaded from asm.sagepub.com at Kungl Tekniska Hogskolan / Royal Institute of Technology on July 12, 2015

2

Assessment 

significantly lower than the elevations on the Clinical Scales, reflecting the removal of nonspecific demoralization variance from the new measures. A number of studies, in different settings, have explored associations between RC Scales scores and various conceptually relevant external criteria. In these reports, the RC Scales exhibited comparable or improved internal consistency and reduced intercorrelations, and they were highly correlated with their corresponding Clinical Scales. Simms, Casillas, Clark, Watson, and Doebbeling (2005) analyzed psychology clinic outpatients and military veterans. They found that the new scales yielded clearer validity patterns in predicting scores on the Schedule for Nonadaptive and Adaptive Personality (Clark, 1993) and the Structured Clinical Interview for the Diagnostic and Statistical Manual of Mental Disorders, fourth edition (DSM-IV; American Psychiatric Association, 1994). Sellbom and Ben-Porath (2005) investigated how RC Scales were related to normal-range personality traits represented by the Multidimensional Personality Questionnaire (Tellegen & Waller, 2008) scales. As expected, Restructured Clinical Scales demoralization (RCd) was most strongly correlated with both Positive Emotionality (reversed) and Negative Emotionality, whereas RC2 (Low Positive Emotions) and RC7 (Dysfunctional Negative Emotions) correlated more distinctively with these higher level personality dimensions, respectively. Another study, conducted by Sellbom, Ben-Porath, and Graham (2006) in a college counseling setting, utilized the Client Description Form (Graham, Ben-Porath, & McNulty, 1999) and the Symptom Checklist 90–Revised Analogue (Derogatis, Rickels, & Rock, 1976) as collateral measures. In congruence with previous studies, the RC Scales were generally associated most highly with conceptually relevant criteria and mostly uncorrelated with nonrelevant criteria. Sellbom, Graham, and Schenk (2006) evaluated the RC Scales in a private practice sample, using the Multiaxial Diagnostic Inventory (Doverspike, 1990) as an extra-test measure. Results revealed that the RC Scales added incrementally to both the Clinical and the Content Scales in predicting clinical symptoms. Handel and Archer (2008) in a mental health inpatient setting applied selected chart variables and selected Symptom Checklist 90–Revised (Derogatis, 1983, 1994) items and clinician ratings on the Brief Psychiatric Rating Scale (Overall & Gorham, 1988). Their external correlational analyses revealed varying degrees of evidence of improvement in convergent and discriminant validity. In line with these findings, Sellbom, Ben-Porath, and Bagby (2008) extended the theoretical and empirical linkage between the RC Scales and dimensional models of personality, using the Revised NEO (Neuroticism–Extraversion–Openness) Personality Inventory (Costa & McCrae, 1992). Their results indicated that RC Scales conformed to hierarchical structures of psychopathology.

A special edition of the Journal of Personality Assessment (Meyer, 2006) was devoted to reviews and commentary on the RC Scales. In that issue, Nichols (2006) criticized the scales, arguing that they are redundant with other MMPI-2 scales, that RCd is a depressively biased marker of the MMPI-2 first factor, and that RC Scales do not sufficiently represent the complexity of the Clinical Scales. A critique was also presented by Rogers, Sewell, Harrison, and Jordan (2006) who reported that the RC Scales yield large proportions of within normal limits profiles in relation to the Clinical Scales. These concerns were addressed and discussed in the same issue by Tellegen et al. (2006), who provided evidence and new findings refuting each of these criticisms. Psychological tests developed and well established in the United States are often adapted to other cultures. In particular, English-language personality questionnaires are frequently translated into other languages and adapted for clinical evaluation in other countries. Cultures and populations vary in terms of language, norms, values, social structures and roles, and belief systems. Therefore, it is of particular importance to investigate these personality assessment instruments cross-culturally. Ben-Porath (1990) described three nonmutually exclusive goals for the adoption of existing personality measures across cultures. The first objective was to provide useful personality measures in the adopting culture. The second was to test the universality of personality constructs, and the third was to facilitate comparisons of typical personalities across cultures. The MMPI-2 has gained considerable cross-cultural attention and has been extensively applied worldwide (Butcher, 2004). This broad international use of the inventory stems from its empirically established clinical validity in a wide range of countries that are culturally different from the United States. In order to have broad cultural generalizability, the instrument’s international adaptation must be functionally equivalent to its original form (Cox, Weed, & Butcher, 2009) meaning that an essential condition for adopting a measure of personality is that it retains its psychometric features in the new culture. To date, only one study of the RC Scales’ psychometric characteristics, using a non-North American/non-Englishspeaking sample, has been published. That study was conducted in a European country, the Netherlands, by Van Der Heijden, Egger, and Derksen (2008). Two Dutch samples were used: a normative sample and an outpatient psychiatric sample. The results indicated that relative to their original counterparts, the RC Scales have comparable or better internal consistencies, lower scale-level intercorrelations and a clearer underlying structural pattern. The authors suggested that the U.S. validation studies on the RC Scales may be generalized to the Dutch-language version of the test. The current study was the first to explore the MMPI-2/ MMPI-2-RF RC Scales in Israel—a Middle Eastern country. Israeli society is composed of multiple ethnic subgroups and

Downloaded from asm.sagepub.com at Kungl Tekniska Hogskolan / Royal Institute of Technology on July 12, 2015

3

Shkalim is therefore a very heterogeneous cultural melting pot. Immigrants from Jewish communities were influenced by different local environments with distinct cultural contexts. This diversity is reflected through various, and sometimes clashing, social profiles, customs, manners, and religious traditions. Members of such communities have been immigrating to Israel over the past decades, since the establishment of the state, and they constitute the majority of its population. The Hebrew language is a substantial uniting factor in this unique social fabric. In this study The MMPI-2/MMPI2-RF RC Scales were evaluated with the Hebrew-language translation of the test (Almagor, Budescu, Montag, & Nevo, 1992). Its major objective was to further cross-culturally evaluate the psychometric properties of the RC Scales in order to assure equivalence in adapting them from their source culture to the target culture. More specifically, this research aimed to determine whether the findings reported by Tellegen et al. (2003) would be replicable with an Israeli clinical sample, using the Hebrew language. Based on previous psychometric evaluations of the RC Scales, it was hypothesized that they would demonstrate psychometric improvements relative to the Clinical Scales in this culture and language as well, and that the results would be comparable to those reported by the American studies.

Method Participants The initial sample included 270 (115 male and 155 female) inpatient and outpatient adult clients from psychiatric settings in Israel. The inpatients (82 male, 104 female) were from a state psychiatric hospital and from a psychiatry department at a state general hospital. The outpatients were from psychiatric day treatment units of these hospitals and from a community mental health center. All participants were receiving individual psychotherapeutic treatment, focused on short-term and evidence-based interventions. MMPI-2 profile invalidity was determined by meeting at least one of the following exclusionary criteria: Cannot say ≥ 18, TRIN > T = 80, F ≥ raw score 40, Fp ≥ raw score 8. After the elimination of 38 (14%) invalid protocols, the final sample was composed of 233 participants, 100 men (42.9%) and 133 women (57.1%). The majority of these individuals, 71% of men and 68.4% of women were inpatients, and the remainder outpatients. In terms of recruitment location, 39% of men and 31.6% of women were from the psychiatric hospital, 32% and 36.8% were from the general hospital, 23% and 24.1% were from the day treatment units, and 6% and 7.5% were from the community mental health center, respectively. Men had a mean age of 40.87 years (SD = 14.39, range = 18-74 years) whereas the mean age for women was 41.76 years (SD = 14.33, range = 18-77 years). The mean years of education were 12.86 (SD = 3.63)

for men and 12.44 (SD = 2.65) for women. All participants were native or equivalently fluent Hebrew speakers. The participants’ cultural/ethnic background (father’s origin) was self-reported as 43% of males and 27% of females American/European, 31% and 31% North African, 19% and 24% Israeli, whereas 5% and 9% other Middle Eastern, 1% and 12% Asian, 1% and 0% other origin, respectively. Regarding marital status, 28% of males and 39.1% of females were married at the time of assessment, with 48% and 30.1% single, 24% and 21.8% divorced/separated, 0% and 8.3% widowed, respectively. The most frequent diagnosis was major depression (58% and 63%, respectively), followed by suicidality (56% and 62%, respectively), generalized anxiety disorder (27% and 29%, respectively), and psychotic disorder (20% and 18.8%, respectively).

Measures The Minnesota Multiphasic Personality Inventory–2. The MMPI-2 (Butcher et al., 2001) is a 567-item true–false format personality and psychopathology inventory. The Hebrew-language version of the MMPI-2 (Almagor et al., 1992) was used. MMPI-2 translation into Hebrew followed a stringent procedure to assure test equivalence by the method of backtranslation (Butcher, 1982). Almagor and Nevo (1996) reported the following steps in the translation process: The English version of the test was initially translated to Hebrew by two bilingual psychologists. Next, the questionnaire items were back-translated to English by a different pair of bilingual psychologists. Comparisons were then made between the newly translated English version and the original, resulting in 64 items whose meaning had changed. These discrepant items were retranslated in exactly the same back-translation manner, employing different psychologists. Following this procedure, three discrepant items remained: 52, dealing with dissatisfaction of one’s life; 62, reflecting a wish to belong to the opposite sex, or for females—being satisfied with their gender; and 149, referring to somatic sensitivity. These items were translated not literally but in accordance to their meaning in Hebrew, since there are no linguistic equivalents in this language to the English terms and Hebrew differentiates between genders. The translated version was administered to a group of subjects who were asked to indicate whether the items were clear and easily understood. None of the items was rated by any of the subjects as inappropriate. An additional examination of the translation’s adequacy involved a study of bilingual subjects who completed the Hebrew and the English MMPI-2 in a balanced order, with a 2-week interval between administrations. The results of within-subject design analyses clearly indicated the congruence between the two administrations in terms of Clinical Scale and Content Scale raw scores. These findings led the authors to conclude that

Downloaded from asm.sagepub.com at Kungl Tekniska Hogskolan / Royal Institute of Technology on July 12, 2015

4

Assessment 

Table 1.  Internal Consistency Coefficients (Cronbach’s αs) of the Patient Description Form Scales by Gender. Scale Somatic Symptoms Depressed Pessimistic Insecure Achievement-Oriented Antisocial Aggressive Family problems Angry/Resentful Critical/Argumentative Suspicious Anxious Obsessive–Compulsive Agitated Psychotic Symptoms Introverted Passive–Submissive Histrionic Narcissistic

Men (n = 100)

Women (n = 133)

.84 .76 .70 .85 .77 .76 .78 .84 .86 .84 .76 .79 .65 .58 .82 .55 .73 .82 .81

.86 .77 .70 .89 .77 .74 .72 .82 .87 .85 .83 .66 .55 .59 .89 .63 .76 .90 .87

the translation process of the Hebrew-language version of the MMPI-2 yielded an instrument that appears adequate and comparable with the English version (Almagor & Nevo, 1996). The focus of the present study was on the MMPI-2 Clinical and RC Scales. Patient Description Form.  The Patient Description Form (PDF; Graham et al., 1999) is a 188-item list of patients’ personality and symptomatic characteristics commonly associated with MMPI/MMPI-2 correlates. Its items are rated by therapists on a 5-point Likert-type scale (ranging from 1 = not at all to 5 = very high), reflecting the degree to which the client possesses each of the characteristics. Using a combined rational/ statistical approach, Graham et al. (1999) developed 25 scales to assess the major content dimensions of the instrument. In this investigation, 16 of these scales, which Tellegen et al. (2003) determined as conceptually relevant to RC1/ Scale 1, RC2/Scale 2, RC4/Scale 4, RC6/Scale 6, RC7/Scale 7, and RC8/Scale 8, were used.1 Internal consistency coefficients (Cronbach’s alphas) of the PDF Scales in this study are provided in Table 1. Three of the 16 Scales (Obsessive–Compulsive; Agitated; Introverted) had alpha coefficients equal or lower than .65 for both genders, and therefore were not included in the analyses.

Procedure The research procedure was approved by the University of Haifa Ethics Committee for experiments on humans and by the Institutional Human Studies Review Board (Helsinki

Committee). Appropriate informed consent was obtained from all participants. In addition, participants were informed that nonparticipation would not affect their treatment and they were free to withdraw their participation at any time. The MMPI-2s and the PDFs were completed after the third psychotherapy session. The MMPI-2s were individually administered to patients in booklet form and the protocols were machine scored. Therapists were asked to rate their clients on the PDFs and were blind to MMPI-2 results until after they completed the forms.

Results Data Analyses First, the reliability of the Clinical Scales and the RC Scales was examined by calculating their internal consistency coefficients. Next, the internal validity of the RC Scales was evaluated through intercorrelations within and between the RC Scales and the Clinical Scales. Then, the RC Scales’ external validity was explored in predicting relevant extratest criteria, through zero-order correlations and multiple regression analyses. Uncorrected raw scale scores were used in all of the analyses.

Internal Consistency Table 2 includes internal consistency coefficients for the RC Scales and the Clinical Scales by gender. As seen in Table 2, internal consistencies across both genders ranged from .51 to .91 for the Clinical Scales and from .73 to .91 for the RC Scales. The RC Scales were equivalently to more internally consistent (M Cronbach’s α = .80 for both genders) than their clinical counterparts (M Cronbach’s α = .77 and .70 for men and women, respectively). In both genders Cronbach’s alphas for RC7 were slightly lower than their values for the corresponding Clinical Scale. The restructured versions of Scales 6 and 9 showed the greatest improvement over the Clinical Scales. Alpha coefficients for RC8 were substantially reduced relative to Cronbach’s alphas of Scale 8. These reductions are in line with the sensitivity of alpha coefficients to scale length differences (Henson, 2001) inasmuch as Scale 8 has 78 items whereas RC8 is much shorter and includes only 18 items. Accordingly, mean interitem correlations were calculated for both scales, which is a measure of reliability that is not affected by the scale length. The mean interitem correlations were higher for RC8 (.18 and .16 for men and women, respectively) than for Scale 8 (.10 and .09, respectively).

Internal Validity Analyses Intercorrelations between and within the RC Scales and the Clinical Scales are reported in Table 3, separately by

Downloaded from asm.sagepub.com at Kungl Tekniska Hogskolan / Royal Institute of Technology on July 12, 2015

5

Shkalim Table 2.  Internal Consistency Coefficients (Cronbach’s αs) for the Restructured Clinical (RC) Scales and Non–K-Corrected Clinical Scales by Gender. Men (n = 100) Scale name Demoralization Hypochondriasis/Somatic complaints Depression/Low Positive Emotions Hysteria/Cynicism Psychopathic deviate/Antisocial behavior Paranoia/Ideas of persecution Psychasthenia/Dysfunctional Negative Emotions Schizophrenia/Aberrant Experiences Hypomania/Hypomanic activation Mean

Women (n = 133)

Abbreviation

Clinical scales

RC scales

Clinical scales

RC scales

RCd 1/RC1 2/RC2 3/RC3 4/RC4 6/RC6 7/RC7 8/RC8 9/RC9 M

.85 .79 .77 .73 .67 .91 .90 .55 .77

.91 .84 .77 .75 .78 .75 .87 .79 .73 .80

.77 .73 .67 .70 .50 .90 .89 .51 .70

.91 .79 .76 .79 .79 .77 .86 .77 .78 .80

Table 3.  Intercorrelations Within and Between the Restructured Clinical (RC) Scales and Non–K-corrected Clinical Scales by Gender. Scale RCd RCd RC1 RC2 RC3 RC4 RC6 RC7 RC8 RC9 1 2 3 4 6 7 8 9

— .42* .58* .29* .25* .18* .70* .28* .17* .51* .67* .37* .62* .52* .87* .73* .13

RC1

RC2

.51* .69* — .37* .28* — .26* .05 .09 .07 .35* .07 .54* .32* .43* .03 .18* −.28* .93* .36* .41* .70* .56* .43* .28* .37* .48* .29* .58* .54* .65* .44* .26* −.27*

RC3

RC4

RC6

RC7

.22* .28* −.03 — .14 .43* .44* .33* .41* .25* −.01 –.14 .24* .16 .32* .44* .30*

.30* .25* .30* −.01 — .19* .31* .20* .42* .10 −.04 .03 .61* .30* .22* .37* .32*

.30* .45* .05 .40* .16 — .49* .68* .39* .28* .02 –.01 .32* .67* .36* .57* .50*

.66* .49* .32* .37* .27* .52* — .58* .50* .58* .37* .20* .54* .59* .84* .82* .47*

RC8

RC9

.40* .08 .44* .17 .14 −.25* .36* .47* .18 .12 .61* .41* .66* .39* — .45* .46* — .39* .15 .07 −29* .06 –.15 .33* .25* .59* .30* .50* .27* .69* .44* .61* .69*

1

2

3

.59* .94* .42* .26* .23* .43* .53* .45* .19* — .51* .65* .30* .45* .64* .66* .20*

.74* .54* .76* −.03 .21* .11 .42* .23* −.21* .64* — .59* .38* .36* .66* .45* −.21*

.48* .72* .49* −.11 .29* .12 .22* .17 −.11 .79* .69* — .32* .30* .40* .34* −.05

4 .69* .59* .57* .20* .57* .42* .56* .38* .16 .63* .63* .62* — .64* .57* .63* .30*

6

7

.46* .56* .34* .02 .30* .71* .48* .49* .22* .58* .44* .50* .62* — .61* .72* .39*

.88* .62* .58* .26* .29* .43* .81* .60* .22* .68* .73* .49* .69* .57* — .86* .29*

8

9

.78* .14 .67* .37* .57* −.24* .31* .39* .35* .34* .58* .48* .82* .45* .69* .55* .28* .70* .70* .34* .64* −.09 .47* .05 .73* .35* .67* .36* .89* .28* — .41* .47* —

Note. Intercorrelations for men (n = 100; rs > .18 are significant; p < .05) and women (n = 133; rs > .16 are significant; p < .05) samples are presented above and below the diagonal, respectively. Convergent correlations are underlined.

gender. The findings indicate that across both genders there were substantial correlations between each of the RC Scales and its corresponding Standard Scale, with the expected exception of RC3/Scale 3. RC1 and Scale 1 showed the highest degree of correlation, and RC3, RC4, and RC6 had the lowest degree of correlations with their corresponding Clinical Scales. Within-scale correlations among the RC Scales (M r = .32 for both men and women) were frequently lower than those of the Clinical Scales (M r = .54 and .46, for men and women, respectively). Of note in this regard is the decrease in the correlations of RC2 and RC7 Scales relative to the

correlations between their matching Clinical Scales. Correlations of RC2 and RC7 (measuring affect-related attributes) with RC4, RC6, and RC8 (measuring nonaffectrelated attributes) were markedly reduced as compared with the corresponding correlations of their original scales counterparts. Despite the general pattern of lowered correlations between the RC Scales, several intercorrelations were still substantial and are worth noting. RC9 produced a stronger relationship with RC7 than did Scales 9 and 7. Furthermore, the opposite-signed correlations of RC9 and RC2 were higher than the correlations between their corresponding

Downloaded from asm.sagepub.com at Kungl Tekniska Hogskolan / Royal Institute of Technology on July 12, 2015

6

Assessment 

Table 4.  Zero-Order Correlations of Restructured Clinical (RC) Scales and Non–K-Corrected Clinical Scales With the Patient Description Form (PDF) Scales for Men. PDF scales

RCd

RC1

Somatic symptoms Depressed Pessimistic Insecure Achievement-oriented Antisocial Aggressive Family problems Angry/resentful Critical/argumentative Suspicious Anxious Psychotic symptoms

.34* .34* .33* .25* .04 −.03 .05 .01 .10 .18* −.01 .21* −.26*

.40* .34* .38* .19* −.09 .03 .23* .21* .26* .23* .19* .26* −.17

RC2 .21* .20* .30* .30* −.04 −.21* −.10 −.06 −.07 .08 .05 .12 −.15

RC3 .02 .13 .13 .09 .08 −.02 .12 .17* .11 .17 −.06 .12 −.18

RC4 −.04 .06 .13 .11 −.20* .19* .11 .07 .14 .14 .06 .00 −.02

RC6

RC7

.06 .09 .13 .02 −.16 .05 .13 .03 .10 .13 .21* .15 .09

.15 .25 .22* .14 −.11 .04 .01 .03 .07 .06 −.02 .26* −.11

RC8 −.08 .06 .12 .00 −.04 −.07 −.06 −.09 −.11 −.08 .08 .09 .06

RC9 −.01 .05 .03 −.03 .03 .17 .12 .07 .16 .14 .09 .12 .04

1 .44* .37* .39* .17 −.09 .02 .26* .14 .23* .24* .18* .26* −.18*

2 .37* .30* .27* .16 −.01 −.12 .04 −.03 .02 .11 .08 .16 −.15

3 .34* .29* .31* .11 .00 −.04 .16 .07 .17 .18 .13 .20* .25*

4 .24* .26* .35* .21* −.03 .09 .20* .10 .23* .29* .14 .19* .−14

6 .18* .22* .27* .10 −.12 .04 .14 −.01 .16 .18* .30* .23* .04

7 .29* .33* .26* .18* −.03 −.02 .03 .03 .01 .13 .08 .30* .14

8

9

.23* −.06 .32* .08 .34 .14 .19* .00 −.12 −.05 −.01 .20* .00 .11 .03 .12 .09 .17 .16 .09 .15 .15 .25* .03 −.09 .11

Note. n = 100. Conceptually relevant correlations are underlined. *p < .05.

Clinical Scales. Furthermore, among women RC9 and RC4 showed increased correlation relative to the correlation between their clinical counterparts. As is also evident from Table 3, the magnitudes of correlations between RCd and the other RC Scales scores were often lower relative to those of RCd with their clinical counterparts. However, although RC2 and RC7 were less strongly correlated with RCd than were Scales 2 and 7, their relations with RCd were still high. Interestingly, among women there was an exception with regard to the correlation of RC9 and RCd, which was modestly greater than the correlation of Scale 9 with RCd, though the levels of both correlations were low. Another consequential finding worth mentioning here was the very strong association between RCd and Scale 7.

External Validity Analyses Zero-order correlations between the RC Scales and the Clinical Scales and conceptually relevant PDF scales are presented in Tables 4 and 5, for men and women, respectively. No PDF convergent criterion measure was available for RC3/Scale 3 and RC9/Scale 9, thus they were not included in these analyses. Examination of RCd’s profile of extra-test correlates revealed that elevated scores on RCd were most strongly associated with the PDF Depressed scale for both genders, and among men with Somatic Symptoms as well. For both genders, RCd often had higher or comparable correlations with PDF depression indicators (Depressed, Pessimistic, Insecure, and Achievement-Oriented) than did RC2 or Scale 2. Relative to RC7, RCd had slightly lower correlation with PDF Anxious scale for men and an equivalent degree of correlation with the same criterion for women. In comparison with Scale 1, RC1 exhibited comparable levels of convergent validity with PDF Somatic Symptoms

scale for both men and women. Relative to Scale 2, RC2 achieved similar correlations with PDF scales reflecting depression for women. For men, the findings revealed a mixed picture, as indicated by comparable convergent validity levels with PDF Pessimistic and Insecure scales, and on the other hand lower convergent validity of RC2 with PDF Depressed scale when compared with Scale 2. Neither RC2 nor Scale 2 was significantly correlated with the PDF Achievement-Oriented scale. Compared with the original Clinical Scales, RC4, RC6, and RC8 generally provided markedly improved convergence with relevant PDF criteria, but only for women. Among men, in comparison with their parent scales, RC4 and RC6 demonstrated reduced convergent validity with their respective PDF measures. One exception was RC4’s higher correlation with PDF Antisocial scale compared with that of its Clinical Scale counterpart. Furthermore, among men, neither RC8 nor Scale 8 showed convergent validity with PDF Psychotic Symptoms scale. Scale 7 and RC7 illustrated comparable correlations with PDF Anxious scale across both genders. A series of multiple regression analyses were conducted to compare the two sets of scales predictions of PDF criterion measures. Following Tellegen et al. (2003), each PDF scale served as the predicted variable, and the sets of RC Scales or Clinical Scales served as the predictors, and were entered in a stepwise method. The final steps of these analyses are summarized in Table 6. The findings indicate that, relative to the Clinical Scales, the RC Scales often produced comparable or better predictions of extra-test ratings. The largest differences between R2 values in favor of the RC Scales were for PDF Angry/Resentful (15%), followed by Antisocial (11.6%), and Family Problems (8.7%) among men, and Antisocial (9.3%), Critical/Argumentative (6%), and Psychotic Symptoms (4.9%) among women. In contrast, among men the greatest differences between R2 values in favor of the Clinical Scales occurred with Aggressive

Downloaded from asm.sagepub.com at Kungl Tekniska Hogskolan / Royal Institute of Technology on July 12, 2015

7

Shkalim Table 5.  Zero-Order Correlations of Restructured Clinical (RC) Scales and Non–K-Corrected Clinical Scales With the Patient Description Form (PDF) Scales for Women. PDF Scales

RCd

RC1

RC2

RC3

Somatic symptoms Depressed Pessimistic Insecure Achievement-oriented Antisocial Aggressive Family problems Angry/resentful Critical/argumentative Suspicious Anxious Psychotic symptoms

.24* .39* .29* .38* −.15 .11 .09 .13 .13 −.10 −.10 .15* −.22*

.37* .28* .16* .15* .00 .03 .11 .14 .17* .11 .00 .21* −.11

.15 .32* .29* .25* −.17* .10 −.06 .02 .04 −.13 −.17* .12 −.28*

.00 .03 .04 .00 −.03 .07 .08 .00 .05 .01 −.03 .06 .10

RC4 −.06 .21* .13 .29* .05 .43* .43* .25* .23* .24* .20* .24* .11

RC6 .05 .04 .03 −.04 −.14 .16* .12 .16* .22* .13 .24* .13 .31*

RC7

RC8

RC9

1

.17* .27* .18* .24* −.05 .16* .14 .22* .17* .06 .04 .19* −.04

.10 .03 −.01 .04 −.16* .12 .03 .11 .09 .05 .08 .13 .21*

−.03 .07 −.06 .10 .08 .38* .27* .12 .12 .19* .18* .13 .22*

2

.37* .31* .16* .19* −.01 .04 .10 .14 .15* .07 −.05 .20* −.14

.30* .36* .27* .25* −.16* −.12 −.13 .07 .09 −.11 −.20* .14 −.26*

3

4

.30* .32* .13 .15* −.01 .01 .06 .13 .14 .04 −.09 .16* −.18*

6

.12 .32* .24* .35* −.16* .27* .28* .29* .27* .12 .10 .31* −.02

7

.29* .25* .18* .17 −.16 .20* .17* .21* .27* .13 .15* .19 .08

.29* .35* .28* .35* −.17* .07 .01 .15* .10 −.06 −.08 .21* −.16*

8 .23* .32* .23* .31* −.16* .18* .13 .20* .20* .02 .05 .23* −.02

9 .02 .04 −.04 .04 −.07 .31* .13 .10 .07 .05 .14 .14 .27*

Note. n = 133. Conceptually relevant correlations are underlined. *p < .05.

Table 6.  Multiple Regressions for the Restructured Clinical (RC) Scales and Non–K-Corrected Clinical Scales Using Dependent Variables From the Patient Description Form. RC Scales Men (n = 100)

 

Clinical Scales Women (n = 133)

Criterion

Scale

β

R2

Scale

Somatic Symptoms

RC1 RC8 RCd RCd   RC1 RC2   RC4 RC2 RC4 RC1       RC1 RC8 RC1 RC8 RC9 RC1 RC8 RC9 RC6   RC7   RCd  

.44** −.37** .26* .34**

.16 .25 .30 .11

RC1

.38** .30**

.15 .09

−.20* −.28** .27* .23*

.04 .04 .11 .05

.31** −.23* .40** −.42** .29** .36*** −.35** .24 .21*

.04 .08 .06 .13 .20 .05 .10 .15 .04

.26*

.07

.26*

.07

Depressed Pessimistic Insecure Achievement-Oriented Antisocial Aggressive

Family Problems Angry/Resentful

Critical/Argumentative

Suspicious Anxious Psychotic Symptoms

Men (n = 100)

Women (n = 133)

R2

Scale

β

R2

Scale

.37**

.13

1 9

.52** −.24*

.19 .25

1

.37**

RCd

.39**

.15

1

.37**

.13

RCd RCd RC4 RC2 RC4 RC9 RC4

.29** .32** .20** −.17* .32** .22* .43**

.08 .14 .18 .03 .18 .22 .18

1 4

.39** .21*

.15 .04

no scale no scale

— —

— —

.25**

.06

.68** −.59** .44** −.37* —

.06 .12 .16 .20 —

.28** .22* .28** .22* .22* — .25** .18* .38** −.27**

RC4

1 8 4 3 no scale

2 4 7 4 7 no scale 9 4 4 2

4

.29**

RC4 RC6

.20* .18*

.05 .09

4

.23*

.05

4

.27**

RC4

.24**

.06

4

.29**

.08 no scale

RC6 RC2 RC4 RC1 RC6 RC2

.25** −.18* .21* .18* .32** −.29**

.06 .09 .05 .09 .09 .18

6

.30**

.09

7

.30**

.09

β

3

−.25*

.06

β

—

2 6 4

−.29** .25** .31**

9 7

.34** −.26**

Note. The β weights are those obtained when all three predictors are entered. The R2 values are those obtained at each step of the regression procedure. *p < .05. **p < .01.

Downloaded from asm.sagepub.com at Kungl Tekniska Hogskolan / Royal Institute of Technology on July 12, 2015

R2 .14     .13 .18 .08 .12 .16 — .10 .13 .08 .14     .08   .07     —     .04 .09 .09   .07 .13

8

Assessment 

(15%), followed by Suspicious (4.3%), and Depressed (2.2%). Among women, Depressed and Family Problems criteria were modestly better predicted by the Clinical Scales (than by the RC Scales), which accounted for the same additional variance (2.1%) in both cases. Concerning discriminant validity, across genders the majority of mean RC correlations with noncorresponding PDF measures, excluding RCd, were lower than those yielded by their matching Clinical Scales. Among men, RC1/ Scale 1 and RC2/Scale 2 exhibited identical mean discriminant validity coefficients (.22 and .12, respectively). RC6/ Scale 6 and RC7/Scale 7 achieved comparable values of mean discriminant validity (.10 vs. .14 and .10 vs. .13, respectively). RC4 and RC8 were markedly less correlated with the remaining discriminant measures when compared with their clinical counterparts (.10 vs. .20 and .07 vs. .16, respectively). Among women, RC1/Scale 1, RC4/ Scale 4, and RC7/Scale 7 produced equivalent mean discriminant validity coefficients (.12 vs. .13, .16 vs. .19., and .15 vs. .17, respectively). RC2, RC6, and RC8 demonstrated evidence of improved discriminant correlations relative to their corresponding scales (.11 vs. .16, .13 vs. .19, and .08 vs. .19, respectively).

Discussion The primary purpose of the present study was to cross-culturally assess the RC Scales’ psychometric functioning. The findings were mostly consistent with expectations and broadly replicated the U.S. RC Scales (Tellegen et al., 2003) results. The overall pattern of data suggests that Tellegen et al.’s (2003) challenging goal to address several weaknesses associated with the original Clinical Scales has been successfully achieved. Moreover, the current investigation provides further evidence that Tellegen et al.’s (2003) RC project findings can be replicated in a different language and a diverse culture, such as Israel’s. Cronbach’s alphas obtained for all RC Scales were adequate, and in most instances essentially equivalent or better than those of their clinical counterparts. These reliability findings were consistent across genders, and correspond with Tellegen et al.’s (2003) results in their clinical samples. They revealed that removing the excessive common demoralization variance from the Clinical Scales did not attenuate the homogeneity of the new set of scales, with the advantage of requiring fewer items. Internal consistency coefficients of the RC Scales in this study (M Cronbach’s α = .80 for both genders) were slightly lower than the coefficient levels reported in the U.S. monograph (e.g., in the Hennepin County Medical Center sample M Cronbach’s α = .87 and .86 for men and women, respectively, as reported in Tellegen et al., 2003). In general, the internal validity properties of the RC Scales were in line with those reported by Tellegen et al. (2003). Analyses of the between-scale correlations

indicated that the RC Scales expectedly resembled their Clinical Scales counterparts, showing the strongest associations with corresponding scales (excluding the previously noted RC3/Scale 3). Examination of within-scale correlations in the RC Scales exhibited reduction relative to the Clinical Scales, revealing that the former were more distinct from one another, as Tellegen et al. (2003) intended. In some instances, associations between RC Scales were greater than those between their matching Clinical Scales. One interesting finding in this regard was the appreciably higher correlations between RC9 and RC7 versus their matching Clinical Scales across both genders. This result is consistent with Tellegen et al.’s (2003) and Handel and Archer’s (2008) data. The most likely explanation for this pattern is that RC9 is the only RC Scale more highly correlated with RCd than its Clinical Scale counterpart. Another noteworthy finding for RC9 in both genders was its increased inverse correlation with RC2 in comparison with the parallel Clinical Scales correlation. This finding is consistent with these RC Scales’ core components, hypomanic activation, and low positive emotions. Greater degree of hypomania tends to be accompanied with increased positive emotionality—thus the stronger negative connection between the constructs. RC4 also showed increased correlation with RC9 relative to the correlation between their clinical counterparts, but only for women. In other words, in the current research, the removal of the demoralization component from Scale 4 yielded stronger correlation with RC9 among women, whereas among men it attenuated its correlation with RC9. Prior studies (Handel & Archer, 2008; Tellegen et al., 2003) found similar patterns of greater relations between RC4 and RC9 Scales than the corresponding relations, but for both men and women. Comorbidity between antisocial behavior and hypomanic activation may be responsible for this greater relation. Findings concerning RC4’s contrasting patterns among males and females in this sample will be discussed later. Correlations of RC Scales measuring affect-laden constructs (i.e., RC2 and RC7) with RC Scales measuring nonaffect-laden constructs (i.e., RC4, RC6, and RC8) showed clear reduction in contrast to similar correlations for their corresponding counterparts, and thus met expectations. These findings are congruent with earlier reports (Handel & Archer, 2008; Tellegen et al., 2003). Furthermore, as expected, most of the RC Scales were substantially less saturated with demoralization than the Clinical Scales, although not entirely demoralization-free. This is consistent with Tellegen et al.’s (2003) view that the remaining associations of RCd with the other scales may be an actual reflection of the co-occurrence between general distress and the psychopathology constructs assessed by the new set of scales. RCd was most highly related to RC2 and RC7, and thus it appeared to represent what its developers (Tellegen et al., 2003) sought to create; a marker of the

Downloaded from asm.sagepub.com at Kungl Tekniska Hogskolan / Royal Institute of Technology on July 12, 2015

9

Shkalim highest level of the affect hierarchy, the pleasantness– unpleasantness factor. The sole exception to the pattern of reduced saturation with demoralization was RC9 among women, which illustrated slightly higher correlation with RCd relative to Scale 9. The same was found in the U.S. clinical sample (Tellegen et al., 2003) and is likely an expression of the narrower content of RC9 which is more specifically focused on an affective state in comparison with its clinical counterpart. The results of high correlations between RCd and Scale 7, which approached .90 for both genders, is consistent with those of Tellegen et al. (2003) and Handel and Archer (2008). Such level of correlations suggests substantial overlap between the constructs that are evaluated by these scales. Nevertheless, Tellegen et al. (2006) demonstrated and discussed other evidence of significant discriminant validity for the two scales. Overall, the pattern of findings regarding the external validity characteristics of the RC Scales was largely consonant with the literature (Tellegen et al., 2003). In terms of convergent validity, the data generally provided evidence that when compared one-on-one with their clinical counterparts, the RC Scales had equivalent or stronger associations with designated external criterion measures. Furthermore, jointly the revised set of scales showed comparable or better predictive utility relative to the original set of scales. Two exceptions to these findings, for men only, involve the relative performance of RC4 and RC6 when compared with their Clinical Scales counterparts. In both cases, the revised scales exhibited weaker convergent validity than their clinical precursors. This was an opposite pattern than the one observed among women. These results substantially deviate from what was reported by Tellegen et al. (2003) and others. It is worth mentioning that the criterion measures for both scales displayed satisfactory internal consistencies (see Table 1), and thus they do not explain this pattern. It is possible that too low base rates of diagnosable antisocial personality disorder (4% of males, 2.3% of females) and psychotic symptoms (20% of males, 18.8% of females) have restricted evaluations of RC4, RC6, and even of RC8. This limitation calls for future replication with sufficient frequency of individuals with these disorders, to test these scales more thoroughly and adequately. Furthermore, with regard to RC4, the findings may be also due to cultural differences. Almagor and Koren’s (2001) cross-cultural factor analytic study on the Clinical Scales sheds some light on the present results. Their research illustrated different internal structures for Scale 4 in the Israeli and U.S. clinical samples. Two of the factors in both samples—Sense of Being Maltreated and Antisocial Behavior—tapped similar content areas. However, these corresponding factors correlated modestly. Each sample had an additional unique factor: In the Israeli sample, this was the first factor emerging in the factor analysis of Scale

Pd, which included items representing dissatisfaction with one’s life and feelings of guilt, and was thus termed Distress. In the U.S. sample it was the second factor, which included items denying feelings of shyness, and was thus called Denial of Shyness. Another U.S. factor, Denial of Happiness, had similar content to the Israeli Distress factor, but it was relatively minor, and was ultimately deleted because of low internal consistency. The researchers concluded that distress is a major component of Scale 4, which is more pronounced in the Israeli than in the U.S. sample. They also assessed the unique contribution of Harris–Lingoes subscales (1955, 1968) to the Israeli Distress factor, and found that it was best predicted by depression. The current findings seem consistent with the cultural specificity found in Almagor and Koren’s (2001) study, and they expand the understanding of the different role of demoralization/depression in antisocial behavior as a function of gender and culture. The present data might suggest a difference in Israel between genders in the structure of antisocial personality disorder. It appears that among females the main characteristic of this disorder is antisocial behavior, manifested in a pervasive pattern of poor behavioral controls, and that demoralization is not a prominent part of their experienced emotional repertoire. Among males, however, it seems that distress is an inherent feature of the antisocial personality disorder, which is very different from the traditional theoretical perception of viewing individuals with this syndrome as having impoverished moral sense and diminished capacity for remorse. Whether these findings represent actual culturally grounded dissimilarities will be determined by future studies, addressing antisocial behavior and RC4/Scale 4 in Israeli psychiatric patients. In addition to individuals diagnosed with antisocial personality disorder as participants, future research should include more comprehensive measures of antisocial behavior, for example, the Hare Psychopathy Checklist–Revised (Hare, 2003). At the diagnostic level, and for both genders, heightened scores on RCd were often most strongly related to depressive spectrum variables, yielding higher, or at least comparable, relations relative to those of RC2 or Scale 2 with the same measures. Moreover, among men RCd was equivalently linked with somatic symptoms and two salient depressive indicators (depressed and pessimistic). Likewise, in comparison with RCd, RC1 was equally or more highly correlated with the latter depressive indicators. These findings, in conjunction with those of the multiple regressions, suggest that male and female psychiatric patients may have different depressive syndromes. For women, the results are broadly in line with Tellegen et al.’s (2003) conclusion regarding the importance of the combined elevations on RCd and RC2 with respect to depression. For men, it appears that both RCd and RC1 are meaningful for predicting depression.

Downloaded from asm.sagepub.com at Kungl Tekniska Hogskolan / Royal Institute of Technology on July 12, 2015

10

Assessment 

RC1’s linkage to depression and demoralization only among psychiatric males in this study is quite unique, and may be a reflection of a cultural specificity. Since its establishment, the state of Israel has been exposed to a prolonged armed conflict, accompanied by constant military confrontation or terror. These circumstances constitute a substantial role in shaping the norms and values of Israeli society, which is known to be very macho. Through the socialization process, males in Israel primarily internalize the importance of behaviors traditionally considered masculine, meaning being tough, assertive, competitive, dominating, and self-controlled with regard to emotional expression (Nardy & Nardy, 1992). In this stereotypical perception, externalization of dysphoric feelings is commonly viewed as a feminine manifestation of psychological infirmity and vulnerability, and the need for emotional sharing and support tends to be considered socially unacceptable for men. In such a strict atmosphere and with constraints against revealing sensitive emotional states, somatic presentations may allow emotionally distressed males a more appropriate pathway or form of expressing their intrapsychic experience, without bearing the burden of being stigmatized as not “manly”. The association of somatic complaints to depression/ demoralization among males is of special importance in view of the fact that when all RC Scales were used, RC1 emerged as the best predictor of most antisocial indicators. Namely, the consequence of removing the demoralization factor from Scale 4 among men was that RC1 was the most definite predictor of antisocial behavior parameters. Taken together, these data support the aforementioned possible role of demoralization/depression as a core theme in antisocial behavior among psychiatric males in Israel. It would be reasonable to suggest that the widely accepted cultural stereotype of males as aggressive, tough-skinned, and nonemotional is much more prominent through males’ antisocial lens, and thus vetoes firmly any expressions of emotional distress and more pronouncedly leads to bypass somatic paths. Also, it might be that for antisocial males in Israel the emotional distress is one of the origins to the disorder. Finally, in terms of discriminant validity, the results revealed evidence of comparable to enhanced discriminant properties of the RC Scales over their clinical counterparts for both genders. These findings are consistent with previous reports (Handel & Archer, 2008; Tellegen et al., 2003) of the RCs improved discriminant validity, indicating they are clearer and more distinctive measures of the Clinical Scales’ core components. In summary, results of this study indicate that the Hebrew-language versions of the RC Scales generally exhibit comparable to improved psychometric properties over their Clinical Scales counterparts. The RC Scales demonstrate evidence of satisfactory internal coherence and promising internal validity. Within-scale RC correlations

are reduced relative to the Clinical scale–level intercorrelations, indicating that the restructured versions are more distinct from one another than the original ones. Furthermore, external analyses reveal comparable or improved convergent validity (with two exceptions among men), and mostly greater discriminant validity. The psychometric advantage of the revised set of scales is largely similar across genders. The successful replication of the U.S. RC findings in Israel, coupled with previously similar results in a Dutch study (Van Der Heijden et al., 2008) provides evidence that Tellegen et al.’s (2003) findings are applicable across different languages and cultures. These results contribute to the body of knowledge accumulated regarding the psychometric characteristics of the MMPI-2/MMPI-2-RF RC Scales relative to their corresponding MMPI-2 Clinical Scales. There are limitations to the present study that should be acknowledged. In addition to the aforementioned one regarding the features of the sample, another limitation is with respect to the available external conceptually relevant descriptors, which did not allow for examination of the correlates of RC3 and RC9. In follow-up studies it would be desirable to include appropriate extra-test independent criteria that are linked to the distinctive core constructs of these scales. It may also be beneficial to study the translated items with differential item functioning analysis, which may provide an indication of unexpected functioning of items in Hebrew Israeli versus English American clinical samples as well as help identify items whose meaning is different in the two cultural contexts. Finally, future research in this area should also replicate the current findings in other psychiatric settings and in other cultures. Acknowledgments I am grateful to my advisor Professor Moshe Almagor for his guidance and encouragement. I also wish to thank Professor Yossef S. Ben-Porath for his insightful comments and suggestions.

Author’s Note This article is based on my doctoral dissertation.

Declaration of Conflicting Interests The author declared no potential conflicts of interest with respect to the research, authorship, and/or publication of this article.

Funding The author received no financial support for the research, authorship, and/or publication of this article.

Note 1. Tellegen et al.’s (2003) PDF scales sorting: RC1/ Scale1—Somatic Symptoms; RC2/Scale 2—Depressed, Pessimistic, Insecure, and Achievement-Oriented; RC4/

Downloaded from asm.sagepub.com at Kungl Tekniska Hogskolan / Royal Institute of Technology on July 12, 2015

11

Shkalim Scale 4—Antisocial, Aggressive, Family Problems, Angry/Resentful, and Critical/Argumentative; RC6/Scale 6 —Suspicious; RC7/Scale 7—Anxious; RC8/Scale 8— Psychotic Symptoms.

References Almagor, M., Budescu, D., Montag, Z., & Nevo, B. (1992). Hebrew MMPI-2. Haifa, Israel: University of Haifa. Almagor, M., & Koren, D. (2001). The adequecy of the MMPI-2 Harris-Lingoes subscales: A cross-cultural factor analytic study of scales D, Hy, Pd, Pa, Sc, and Ma. Psychological Assessment, 13, 199-215. Almagor, M., & Nevo, B. (1996). The MMPI-2: Translation and first steps in its adaptation into Hebrew. In J. N. Butcher (Ed.), International adaptations of the MMPI-2 (pp. 487505). Minneapolis: University of Minnesota Press. American Psychiatric Association. (1994). Diagnostic and statistical manual of mental disorders, DSM-IV (4th ed.). Washington, DC: Author. Ben-Porath, Y. S. (1990). Cross-cultural assessment of personality: The case for replicatory factor analysis. In J. N. Butcher & C. D. Spielberger (Eds.), Recent advances in personality assessment (Vol. 8, pp. 27-48). Hillsdale, NJ: Lawrence Erlbaum. Ben-Porath, Y. S., & Tellegen, A. (2011). MMPI-2-RF (Minnesota Multiphasic Personality Inventory-2 Restructured Form): Manual for administration, scoring, and interpretation. Minneapolis: University of Minnesota Press. (Original work published 2008) Butcher, J. N. (1982). Cross-cultural research methods in clinical psychology. In P. C. Kendall & J. N. Butcher (Eds.), Handbook of research methods in clinical psychology (pp. 273-308). New York, NY: Wiley. Butcher, J. N. (2004). Personality assessment without borders: Adaptation of the MMPI-2 across cultures. Journal of Personality Assessment, 83, 90-104. Butcher, J. N., Graham, J. R., Ben-Porath, Y. S., Tellegen, A., Dahlstrom, W. G., & Kaemmer, B. (2001). Minnesota Multiphasic Personality Inventory-2 (MMPI-2): Manual for administration, scoring and interpretation (Rev. ed.). Minneapolis: University of Minnesota Press. Camara, W. J., Nathan, J. S., & Puente, A. E. (2000). Psychological test usage: Implications in professional psychology. Professional Psychology, 31, 141-154. Clark, L. A. (1993). Schedule for nonadaptive and adaptive personality: Manual for administration, scoring, and interpretation. Minneapolis: University of Minnesota Press. Costa, P. T., & McCrae, R. R. (1992). NEO PI-R and NEO-FFI: Professional manual. Odessa, FL: Psychological Assessment Resources. Cox, A. C., Weed, N. C., & Butcher, J. N. (2009). The MMPI2: History, interpretation, and clinical issues. In J. N. Butcher (Ed.), Oxford handbook of personality assessment (pp. 250-276). New York, NY: Oxford University Press. Derogatis, L. R. (1983). SCL-90-R: Administration, scoring, and procedures manual-II for the revised version and other instruments of the psychopathology rating scale series. Towson, MD: Clinical Psychometric Research.

Derogatis, L. R. (1994). SCL-90-R: Administration, scoring, and procedures manual (3rd ed.). Minneapolis, MN: NCS Pearson. Derogatis, L. R., Rickels, K., & Rock, A. (1976). The SCL-90 and the MMPI: A step in the validation of a new self-report scale. British Journal of Psychiatry, 128, 280-289. Doverspike, W. F. (1990). Multiaxial Diagnostic Inventory. Atlanta, GA: Professional Resource Press. Graham, J. R., Ben-Porath, Y. S., & McNulty, J. L. (1999). MMPI-2 correlates for outpatient mental health settings. Minneapolis: University of Minnesota Press. Handel, R. W., & Archer, R. P. (2008). An investigation of the psychometric properties of the MMPI-2 Restructured Clinical (RC) Scales with mental health inpatients. Journal of Personality Assessment, 90, 239-249. Hare, R. D. (2003). The Hare Psychopathy Checklist–Revised (2nd ed.). Toronto, Ontario, Canada: Multi-Health Systems. Harris, R., & Lingoes, J. (1955). Subscales for the Minnesota Multiphasic Personality Inventory (Mimeographed materials). San Francisco, CA: Langley Porter Clinic. Harris, R., & Lingoes, J. (1968). Subscales for the Minnesota Multiphasic Personality Inventory (Mimeographed materials). San Francisco, CA: Langley Porter Clinic. Henson, R. K. (2001). Understanding internal consistency reliability estimates: A conceptual primer on coefficient alpha. Measurment and Evaluation in Counseling and Development, 34, 177-189. Loevinger, J. (1972). Some limitations of objective personality tests. In J. N. Butcher (Ed.), Objective personality assessment: Changing perspectives (pp. 45-58). Oxford, England: Academic Press. Meyer, G. J. (2006). MMPI-2 Restructured Clinical Scales [Special issue]. Journal of Personality Assessment, 87, 119-120. Nardy, C., & Nardy, R. (1992). Men in change. Ben Shemen, Israel: Modan. Nichols, D. S. (2006). The trials of separating bath water from baby: A review and critique of the MMPI-2 Restructured Clinical (RC) Scales. Journal of Personality Assessment, 87, 121-138. Norman, W. (1972). Psychometric considerations for a revision of the MMPI. In J. N. Butcher (Ed.), Objective personality assessment: Changing perspectives (pp. 59-83). New York, NY: Academic Press. Overall, J. E., & Gorham, D. R. (1988). Introduction: The Brief Psychiatric Rating Scale (BPRS): Recent developments in ascertainment and scaling. Psychopharmacology Bulletin, 24, 97-98. Rogers, R., Swell, K. W., Harrison, K. S., & Jordan, M. J. (2006). The MMPI-2 Restructured Clinical Scales: A paradigmatic shift in scale development. Journal of Personality Assessment, 87, 139-147. Sellbom, M., & Ben-Porath, Y. S. (2005). Mapping the MMPI-2 Restructured Clinical (RC) Scales onto normal personality traits. Journal of Personality Assessment, 85, 179-187. Sellbom, M., Ben-Porath, Y. S., & Bagby, R. M. (2008). Personality and psychopathology: Mapping the MMPI-2 Restructured Clinical (RC) Scales onto the five factor model of personality. Journal of Personality Disorders, 22, 291-312.

Downloaded from asm.sagepub.com at Kungl Tekniska Hogskolan / Royal Institute of Technology on July 12, 2015

12

Assessment 

Sellbom, M., Ben-Porath, Y. S., & Graham, J. R. (2006). Correlates of the MMPI-2 Restructured Clinical (RC) Scales in a college counseling setting. Journal of Personality Assessment, 86, 89-99. Sellbom, M., Graham, J. R., & Schenk, P. W. (2006). Incremental validity of the MMPI-2 Restructured Clinical (RC) Scales in a private practice sample. Journal of Personality Assessment, 86, 196-205. Simms, L. J., Casillas, A., Clark, L. A., Watson, D., & Doebbeling, B. N. (2005). Psychometric elevation of the Restructured Clinical Scales of the MMPI-2. Psychological Assessment, 17, 28-43. Tellegen, A., & Ben-Porath, Y. S. (2011). MMPI-2-RF (Minnesota Multiphasic Personality Inventory-2 Restructured Form): Technical manual. Minneapolis: University of Minnesota Press. (Original work published 2008) Tellegen, A., Ben-Porath, Y. S., McNulty, J. L., Arbisi, P. A., Graham, J. R., & Kaemmer, B. (2003). MMPI-2 Restructured Clinical (RC) Scales: Development, validation, and interpretation. Minneapolis: University of Minnesota Press. Tellegen, A., Ben-Porath, Y. S., Sellbom, M., Arbisi, P. A., McNulty, J. L., & Graham, J. R. (2006). Further evidence on the validity of the MMPI-2 Restructured Clinical Scales:

Addressing questions raised by Rogers, Sewell, Harrison, and Jordan and Nichols. Journal of Personality Assessment, 87, 148-171. Tellegen, A., & Waller, N. G. (2008). Exploring personality through test construction: Development of the Multidimensional Personality Questionnaire. In G. J. Boyle, G. Matthews, & D. H. Saklofske (Eds.), Handbook of personality theory and testing: Personality measurement and assessment (Vol. 2, pp. 261-292). London, England: Sage. Tellegen, A., Watson, D., & Clark, L. A. (1999). On the dimensional and hierarchical structure of affect. Psychological Science, 10, 297-303. Van Der Heijden, P. T., Egger, J. I. M., & Derksen, J. J. L. (2010). Comparability of scores on the MMPI-2-RF scales generated with the MMPI-2 and MMPI-2-RF booklets. Journal of Personality Assessment, 92, 254-259. Wallace, A., & Liljequist, L. (2005). A comparison of the correlational structures and elevation patterns of the MMPI-2 Restructured Clinical (RC) and Clinical Scales. Assessment, 12, 290-294. Watson, D., & Tellegen, A. (1985). Toward a consensual structure of mood. Psychological Bulletin, 98, 219-235.

Downloaded from asm.sagepub.com at Kungl Tekniska Hogskolan / Royal Institute of Technology on July 12, 2015