Psychiatry and Clinical Neurosciences 2014; 68: 448–455
doi:10.1111/pcn.12151
Regular Article
Development of a Japanese version of the Reported and Intended Behaviour Scale: Reliability and validity Sosei Yamaguchi, PhD,1* Shinsuke Koike, MD, PhD,2 Kei-ichiro Watanabe, MD, PhD2 and Shuntaro Ando, MD, MScPH3,4 1
Department of Psychiatric Rehabilitation, Institute of Mental Health, National Center of Neurology and Psychiatry, 2Office for Mental Health Support, Division for Counseling and Support, 3Department of Neuropsychiatry, Graduate School of Medicine, The University of Tokyo, and 4Department of Psychiatry and Behavioral Sciences, Tokyo Metropolitan Institute of Medical Science, Tokyo, Japan
Aim: The Reported and Intended Behaviour Scale (RIBS) was developed in the UK to measure mental health-related behavior. The current study aimed to evaluate the applicability, and reliability of a Japanese version of the RIBS (RIBS-J) in a Japanese context, and further examine the construct validity of the RIBS-J. Methods: The sample included 224 undergraduate and postgraduate students at a Japanese university. Cronbach’s alpha was used to assess internal consistency. Pearson’s correlation coefficient was used to examine the divergent validity between the RIBS-J and the Mental Health Knowledge Schedule and the convergent validity between the second subscale of the RIBS-J and Japanese version of the Social Distance Scale. Confirmatory factor analysis assessed the goodness of model fit of the RIBS-J. We also examined test–retest reliability with another undergraduate sample (n = 29). Results: Most items exhibited no floor/ceiling effect. High internal consistency (α = 0.83) was reported.
HE STIGMA ATTACHED to mental health by the general public is a global concern, as it adversely influences all aspects of the lives of people with mental health problems, including income levels, housing, unemployment, the lower rate of marriage
T
*Correspondence: Sosei Yamaguchi, PhD, Department of Psychiatric Rehabilitation, Institute of Mental Health, National Center of Neurology and Psychiatry, 4-1-1 Ogawa-Higashi, Kodaira, Tokyo 187-8553, Japan. Email: [email protected] Received 21 October 2013; revised 14 December 2013; accepted 19 December 2013.
448
The second subscale of the RIBS-J, measuring intended behavior, correlated with the Mental Health Knowledge Schedule (r = 0.33, P < 0.001) and the Japanese version of the Social Distance Scale (r = −0.60, P < 0.001). In addition, confirmatory factor analysis found good model fit for the RIBS-J (χ2 = 41.001, d.f. = 19, P = 0.002, goodness-of-fit index = 0.956, adjusted goodness-of-fit index = 0.916, comparative fit index = 0.955, root mean square error of approximation = 0.072). Overall test–retest reliability (ρc) was 0.71.
Conclusion: The RIBS-J is an appropriate and psychometrically robust measure of behavior towards individuals with mental health problems in Japan. Further studies using a community sample could assess the generalizability of our findings. Key words: behavior, discrimination, psychometric properties, scale development, validity.
among this group, and smaller social networks.1 A conceptual framework of mental health-related stigma comprises the following three dimensions: ignorance (lack of knowledge), prejudice (negative attitudes and feelings), and discrimination (behavioral enactment of prejudice).2 Studies have consistently emphasized that the elimination of discrimination against the mentally ill is the most important aspect for combating mental health-related stigma.1,3 Nevertheless, recent systematic reviews of anti-stigma interventions have revealed that most studies seldom focused on others’ actual discriminatory behavior towards people with mental
© 2014 The Authors Psychiatry and Clinical Neurosciences © 2014 Japanese Society of Psychiatry and Neurology
Psychiatry and Clinical Neurosciences 2014; 68: 448–455
health problems as research outcomes.4–8 It is imperative to directly assess others’ discriminatory behavior (behavioral problems) in addition to knowledge and attitudes, as the improvements in proximal outcomes, such as people’s knowledge and attitudes, do not always guarantee their behavioral changes.9,10 Japanese studies in particular have rarely reported discrimination towards people with mental health problems, although Japanese adults and children both tend to hold more negative attitudes than individuals in other countries.11,12 This lack of research evidence on discrimination could be attributed to the absence of scales to accurately assess individuals’ behavior towards people with mental health problems. In fact, most of the scales that assess such stigma focus only on others’ knowledge of mental health issues and attitudes towards people with mental health problems.13 To the best of our knowledge, there is only one reliable scale, namely, the Reported and Intended Behaviour Scale (RIBS), which was designed to assess people’s past and present behaviors and behavioral intentions towards those with mental health problems, using two subscales comprising eight items in total.14 Overall, the RIBS was identified as having good feasibility (average time required for completion = 1 min) and good reliability as a result of the internal consistency of the second subscale (α = 0.85) and overall test–retest reliability (Lin’s concordance statistic: ρc = 0.75).14 Indeed, the RIBS has been utilized in several recent local and national studies on stigma, particularly in the UK.15–17 The RIBS has been mainly used in research conducted in the UK; therefore, little is known about its applicability in Japan or other countries. In addition, previous study has only examined one type of validity relating to the RIBS, namely, content validity.14 Therefore, this study aimed to develop a Japanese version of the RIBS (RBIS-J) and empirically test its applicability, reliability, (construct) validity, including convergent validity, divergent validity and goodness of model fit in a Japanese context.
METHODS Study 1 Participants Participants were undergraduate and graduate students of a university in Tokyo enrolled in several courses unrelated to mental health.
Validation of RIBS-J 449
Table 1 Participant characteristics n = 224 Sex School Major subjects
Mean age (SD)
Male Female Undergraduate Postgraduate Humanities† Science‡ Other 22.61 (2.47)
n
%
163 61 153 71 126 96 2
72.8 27.2 68.3 31.7 56.3 42.9 0.9
† Major subjects in the ‘Humanities’ include courses in art, linguistics, sociology, education, and psychology. ‡ Major subjects in ‘Science’ include courses in medicine, engineering, physics, mathematics, and chemistry.
The questionnaires were distributed to 235 students who attended a ‘stress management’ lecture. Participants were informed of the aim of the study, data collection procedures, and the implications of participation or non-participation in this study via the cover page of the questionnaire or verbal account from the second author (S.K.). Questionnaires were completed only by students who voluntarily consented to participate in the study. Informed consent was obtained from all participants (224 participants in total, response rate: 95.3%). Participants’ sociodemographic information is shown in Table 1. Approximately 70% of participants were male, with a mean age of 22.61 years (SD = 2.47). In addition, undergraduate students represented about 70% of the study sample. Around half of the participants were majoring in the humanities. Instrument The RIBS comprises two subscales containing eight items in total. The first subscale includes four statements relating to past or present contact with people with mental health problems. Each item receives a score of 1 for ‘yes’ answers and 0 for ‘no’ or ‘don’t know’ (score range of the subscale: 0–4). Higher scores indicate more past or present contact. The second subscale consists of four statements relating to participants’ future behavioral intentions when they come into contact with people with mental health problems. Responses to the items are rated as ‘agree strongly’, ‘agree slightly’, ‘neither agree nor
© 2014 The Authors Psychiatry and Clinical Neurosciences © 2014 Japanese Society of Psychiatry and Neurology
450 S. Yamaguchi et al.
Psychiatry and Clinical Neurosciences 2014; 68: 448–455
disagree’, ‘disagree slightly’, ‘disagree strongly’ and ‘don’t know’. For each item, a score of 5 was allocated for strong agreement, while a score of 1 was assigned to strong disagreement (score range of the subscale: 4–20). The response option ‘don’t know’, is coded as neutral (i.e., 3). Higher scores indicate more favorable behavioral intentions.
The SDSJ comprises five items measuring individuals’ desire for social distance from people with schizophrenia (e.g., ‘If a person with schizophrenia is a taxi driver, I would not want to take his/her taxi’); lower scores indicate a more positive attitude. This measure has shown good construct validity as shown by both exploratory and confirmatory factor analyses, good internal consistency (α = 0.82), and good test– retest reliability (r = 0.88) in Japanese settings.19 Confirmatory factor analysis was also performed to test the goodness of model fit of the RIBS-J in the Japanese context. All analyses were conducted with Stata version 12 and AMOS version 20 software.
Japanese translation We undertook the following three steps in order to translate the RIBS into Japanese. The original RIBS was translated into Japanese by the first author (S.Y.). Back-translation was conducted by a native English speaker from England, whose second language was Japanese. Finally, the back-translated version of the RIBS was confirmed and approved by the researchers who had originally developed the RIBS. Reliability/validity and statistical methods We examined several aspects relating to the reliability and validity of the RIBS-J. In terms of applicability, the floor and ceiling effects were assessed on the basis of the proportions of responses and means (SD) obtained for each item. T-tests and Pearson’s correlation coefficients were calculated to confirm whether scores on the RIBS-J differed according to sociodemographic characteristics. Similar to the UK study, Cronbach’s alpha (α) was calculated in order to determine the internal consistency of the second subscale of the RIBS-J, which measures intended behavior.14 Three types of construct validity were tested in this study. To test divergent and convergent validity, we calculated Pearson’s correlation coefficient (r) in order to determine the association between the RIBS-J and the Mental Health Knowledge Schedule (MAKS), and between the RIBS-J and the Japanese version of the Social Distance Scale (SDSJ). The MAKS comprises six items that assess knowledge of mental illness (e.g., ‘Medication can be an effective treatment for people with mental health problems’).18 Higher scores indicate greater knowledge of mental health problems. The MAKS was developed by the same research team that developed the RIBS. The content validity, internal consistency (α = 0.65), and overall test–retest reliability (ρc = 0.71) of the MAKS have been demonstrated from a previous study.18 The process of translation for the MAKS was identical to that of the RIBS-J.
Study 2 We recruited a second sample of 35 undergraduates from the same university by advertising on a university message board. Of these 35 participants, 29 agreed to complete the RIBS-J twice (at an interval of 1 month) to enable the evaluation of test–retest reliability. In this sample, 24 participants were male (82.8%) and the mean age was 19.62 (SD = 1.12). All the participants were undergraduate students. In accordance with Evans-Lacko et al., we used Lin’s concordance statistic to calculate test–retest reliability, not only for the first and second subscales of the RIBS-J, but also the entire RIBS-J scale.14 Lin’s concordance statistic (ρc) has often been employed in recent studies to examine test–retest reliability of measures assessing mental health-related stigma.14,15,20,21
Ethics approval This study was approved by the Research Ethics Committee at the University of Tokyo (approval number: 12-102).
RESULTS Study 1 Applicability (responses and means) Table 2 presents participants’ responses to the eight items of the RIBS-J, also indicating the distribution of responses according to all of the response options provided for each item. The non-response option constituted more than 75% of participants’ responses on the first subscale. Similarly, preference for ‘agree
© 2014 The Authors Psychiatry and Clinical Neurosciences © 2014 Japanese Society of Psychiatry and Neurology
Psychiatry and Clinical Neurosciences 2014; 68: 448–455
Validation of RIBS-J 451
Table 2 Distribution of participants’ responses Item no.
Statements in subscale 1: Reported behavior (n = 224)
RIBS-J 1
Are you currently living with, or have you ever lived with, someone with a mental health problem? RIBS-J 2 Are you currently working with, or have you ever worked with, someone with a mental health problem? RIBS-J 3 Do you currently have, or have you ever had, a neighbor with a mental health problem? RIBS-J 4 Do you currently have, or have you ever had, a close friend with a mental health problem? Total: Subscale 1†
Item no. RIBS-J 5
Statements in subscale 2: Intended behavior (n = 224)
In the future, I would be willing to live with someone with a mental health problem. RIBS-J 6 In the future, I would be willing to work with someone with a mental health problem. RIBS-J 7 In the future, I would be willing to live near someone with a mental health problem. RIBS-J 8 In the future, I would be willing to continue a relationship with a friend who has developed a mental health problem. Total: Subscale 2‡ Total: Subscales 1 and 2
n (%) Yes
Mean† (SD)
No
Don’t know
50 (22.3)
154 (68.8)
20 (8.94)
0.22 (0.42)
33 (14.7)
162 (72.3)
29 (12.9)
0.15 (0.36)
46 (20.5)
123 (54.9)
55 (24.6)
0.21 (0.40)
89 (39.7)
96 (43.3)
38 (17.0)
0.40 (0.49) 0.97 (1.07)
Agree strongly
Agree slightly
Neither agree nor disagree
Disagree slightly
Disagree strongly
Don’t know
Mean‡ (SD)
10 (4.5)
34 (15.2)
42 (18.8)
54 (24.1)
44 (19.6)
40 (17.9)
2.61 (1.10)
14 (6.3)
45 (20.1)
69 (30.8)
37 (16.5)
28 (12.5)
31 (13.8)
2.91 (1.06)
13 (5.8)
37 (16.5)
51 (22.8)
45 (20.1)
40 (17.9)
38 (17.0)
2.72 (1.11)
101 (45.1)
87 (38.8)
11 (4.9)
11 (4.9)
3 (1.3)
11 (4.9)
4.21 (0.91) 12.46 (3.42) 13.43 (3.83)
Calculation of mean scores: †‘Yes’ = 1, ‘No’ = 0, and ‘Don’t know’ = 0. ‡‘Agree strongly’ = 5, ‘Agree slightly’ = 4, ‘Neither agree nor disagree’ = 3 ‘Disagree slightly’ = 2, ‘Disagree strongly’ = 1, and ‘Don’t know’ = 3.
(disagree) strongly’ and ‘agree (disagree) slightly’ made up no more than 50% of participants’ total responses on three items of the second subscale. One item (RIBS-J 8: ‘In the future, I would be willing to continue a relationship with a friend who developed a mental health condition.’) had a slight ceiling effect (mean = 4.21, SD = 0.91). In addition, ‘don’t know’ as a response option made up no more than 30% for all of the items in both subscales. The mean scores on the first and second subscales of the RIBS-J were 0.97 (SD = 1.07) and 12.46 (SD = 3.42), respectively; the mean total score was 13.43 (SD = 3.83). The students’ age showed a slight, yet significant, correlation with the scores on the first subscale (r = 0.17, P = 0.011). However, no significant associations were found between scores on the RIBS-J (for both subscales) and other sociodemographic characteristics. Internal consistency and construct validity (convergent validity, divergent validity, and model fit) A Cronbach’s alpha (α) of 0.83 was obtained for the second subscale of the RIBS-J. The mean scores on the
MAKS and the SDSJ were 21.25 (SD = 2.93) and 6.90 (SD = 3.34), respectively. With regard to divergent validity, the score on the first subscale of the RIBS-J did not show a significant correlation with either the SDS-J score or the MAKS score. The MAKS score had a relatively weak correlation with the score on the second subscale of the RIBS-J (r = 0.33, P < 0.001). These indicated good divergent validity of the RIBS-J. Conversely, a moderate negative correlation was found between the score on the second subscale of the RIBS-J and the SDS-J score (r = −0.60, P < 0.001), indicating good convergent validity. The results of confirmatory factor analysis are presented in Table 3. The χ2-test for goodness-of-fit was significant (χ2 = 41.001, d.f. = 19, P = 0.002). However, the other three fitness indices were quite high (goodness-of-fit index [GFI] = 0.956, adjusted goodness-of-fit index [AGFI] = 0.916, comparative fit index [CFI] = 0.955), while one was less than 0.1 (RMSEA = 0.072). The standardized path coefficients for item 3 (RIBS-J 3: ‘Do you currently have, or have you ever had, a neighbor with a mental health problem?’) was relatively low (β = 0.36, P = 0.010). The standardized path coefficients of the remaining seven items were 0.44–0.87 (P < 0.001).
© 2014 The Authors Psychiatry and Clinical Neurosciences © 2014 Japanese Society of Psychiatry and Neurology
452 S. Yamaguchi et al.
Psychiatry and Clinical Neurosciences 2014; 68: 448–455
Table 3 Confirmatory factor analysis and model fit n = 224: χ2 = 41.001, d.f. = 19, (P = 0.002), GFI = 0.956, AGFI = 0.916, CFI = 0.955, RMSEA = 0.072 Correlation between subscale 1 (items 1–4) and subscale 2 (items 5–8) = 0.38, P < 0.001 Standardized path coefficients (stranded error) Subscale 1: Reported behavior
Item no.
Statements
RIBS-J 1
Are you currently living with, or have you ever lived with, someone with a mental health problem? Are you currently working with, or have you ever worked with, someone with a mental health problem? Do you currently have, or have you ever had, a neighbor with a mental health problem? Do you currently have, or have you ever had, a close friend with a mental health problem? In the future, I would be willing to live with someone with a mental health problem. In the future, I would be willing to work with someone with a mental health problem. In the future, I would be willing to live near someone with a mental health problem. In the future, I would be willing to continue a relationship with a friend who developed a mental health problem.
RIBS-J 2 RIBS-J 3 RIBS-J 4 RIBS-J 5 RIBS-J 6 RIBS-J 7 RIBS-J 8
Subscale 2: Intended behavior
0.44 (0.04)* 0.46 (0.04)* 0.36 (0.04)* 0.56 (0.05)* 0.81 (0.06)* 0.87 (0.06)* 0.85 (0.06)* 0.44 (0.06)*
*P < 0.01. AGFI, adjusted goodness-of-fit index; CFI, comparative fit index; GFI, goodness-of-fit index; RMSEA, root mean square error of approximation.
Study 2 Lin’s concordance statistic (ρc) for both the first and second subscales of the RIBS-J was 0.72 (SE = 0.09) and 0.70 (SE = 0.10), respectively. The entire scale was 0.71 (SE = 0.10), indicating the overall test– retest reliability of the RIBS-J.
DISCUSSION We tested the applicability and reliability of the RIBS-J in a Japanese setting, also evaluating several aspects relating to the validity of the RIBS-J. This study adds new evidence for the psychometric properties of the RIBS, particularly regarding convergent validity, divergent validity, and goodness of model fit.
Applicability in Japan Other than item 8 of the second subscale, the distribution of responses was not skewed for the other
items of the RIBS-J. A ceiling effect was observed for item 8 (mean = 4.21, SD = 0.91); however, EvansLacko et al. also observed a ceiling effect for this item (mean = 4.04, SD = 1.02) in a study conducted in the UK.14 In other words, the responses given by our sample may have been somewhat similar to those provided by the community sample used in the UK study. While the score on the RIBS-J showed no association with sex, major subjects, or academic grades (undergraduate/postgraduate), we found a weak correlation between age and scores on the first subscale of the RIBS-J, which addresses participants’ past and present experiences. Not surprisingly, older participants had more opportunities to have encountered various individuals, including people with mental health problems, in their lives, as opposed to younger participants. Therefore, we cannot assume that the weak correlation between age and scores on the first subscale of the RIBS-J completely nullifies the validity and applicability of the RIBS-J in Japan.
© 2014 The Authors Psychiatry and Clinical Neurosciences © 2014 Japanese Society of Psychiatry and Neurology
Psychiatry and Clinical Neurosciences 2014; 68: 448–455
Reliability The internal consistency of the second subscale of the RIBS-J was quite high, and satisfactory (α = 0.83) considering that, in general, the minimum threshold is 0.7.22 In addition, overall test–retest reliability for the RIBS-J (ρc = 0.71) was acceptable, when taking into account past studies aimed at developing scales measuring mental health-related stigma.14,18,20 In addition, it has been noted that the Cronbach’s alpha and Lin’s concordance statistic obtained in this study were similar to those reported for a study conducted in the UK (α = 0.85 and ρc = 0.75).14
Validity This study illustrates the originality of the RIBS-J in its assessment of behavior and behavioral intentions through the divergent validity test. Low correlation coefficients between a measure and other conceptually different measures are given as evidence of divergent validity.23 The lack of correlation between the first subscale of the RIBS-J and either the MAKS or the SDSJ means that the first subscale of the RIBS-J does not measure participants’ knowledge and attitudes, and thus indicates good divergent validity of the first subscale of the RIBS-J. The second subscale of the RIBS-J showed a weak correlation with the MAKS. In relation to this, past studies have also reported associations between participants’ knowledge of mental health issues and their behavioral intentions or social distance towards people with mental health problems.12,24 Therefore, it seems that the weak correlation, rather than strong or moderate correlation, observed between the second subscale of the RIBS-J and the MAKS further indicated the divergent validity of the RIBS-J. A significant correlation was found between the second subscale of the RIBS-J and the SDSJ. Strictly, the second subscale of the RIBS-J does not measure behavior, as such, but participants’ future intentions to be in close proximity to individuals with mental health problems in four given situations. The SDSJ measures participants’ attitudes or feelings, assuming participants’ desire for social distance from individuals with schizophrenia in five particular situations. In other words, the second subscale of the RIBS-J partially resembles a social distance scale. It is assumed that a relatively high correlation between the second subscale of the RIBS-J and the SDSJ ensures good convergent validity of the RIBS-J.
Validation of RIBS-J 453
On the other hand, the difference in the target conditions between RIBS-J (mental health problems) and SDSJ (schizophrenia) may affect the results. Generally, the general public tends to hold more stigmatizing attitudes towards people with schizophrenia than those with other psychiatric disorders, like depression, not only in Japan but also in Western countries.11,25,26 It is presumed that higher correlation coefficient would be found, if we employed other measures for people’s desire for social distance from people with mental health problems rather than schizophrenia. Overall, this study obtained good model fit for the RIBS-J. Although the χ2 for goodness of fit proved significant, when sample size is enlarged, the power to detect statistically significant differences also increases.27,28 In short, the fact that the χ2 yielded statistically significant results does not always indicate that the model is poor.28 Indeed, other indices of model fit for the RIBS-J in this study seemed to yield good model fit values; the minimum threshold of the GFI, AGFI, and CFI were more than 0.9, while the RMSEA was less than 0.1.28 The standardized path coefficients in seven of eight items were higher than 0.4 and all the standardized path coefficients were significant. The reason for the relatively low coefficient found for item 3 is unclear. However, it is assumed that this low coefficient may be due to the inherent differences in the constructs measured by item 3 and the other seven items. For instance, item 3 requires participants to provide answers about an unfamiliar person (a neighbor), whereas the other three items in the first subscale of the RIBS-J specifically relate to closer acquaintances, such as family members, friends, and colleagues. In Japanese culture, people often hesitate to mention distant acquaintances and their illnesses.11 Moreover, because communication with neighbors is relatively low in large cities, such as Tokyo, in Japan, people may not be well acquainted with their neighbors.29 As a result, the content of item 3 in relation to broader Japanese culture may have contributed to the relatively higher frequency (24.6%) of the ‘don’t know’ response compared to other items; in turn, this may have influenced the low coefficient that was obtained for this item.
Study limitations Some limitations of the study were identified. First, as opposed to a community sample, participants in this
© 2014 The Authors Psychiatry and Clinical Neurosciences © 2014 Japanese Society of Psychiatry and Neurology
454 S. Yamaguchi et al.
Psychiatry and Clinical Neurosciences 2014; 68: 448–455
study comprised university students who attended a lecture on ‘stress management’; thus, the study’s findings may not apply to the general public. Accordingly, the generalizability of the results could pose a challenge in the future. Second, we used the MAKS to test the divergent validity of the RIBS-J, because the same research team in the UK developed these two scales.14,18 However, the psychometrics of the MAKS in a Japanese setting is unclear. Therefore, we may not properly evaluate the divergent validity between RIBS-J and MAKS. The third limitation is the issue of social desirability. It cannot be denied that students may have reduced the extent of self-disclosure when completing the questionnaires due to social desirability distortion. To counter this, Evans-Lacko et al. conducted an online survey for their UK study to test the psychometric properties of the RIBS, as opposed to conducting interviews.14 This was because an online survey enhances participants’ perception of anonymity as compared to interviews, leading to a reduced tendency to give socially desirable responses.30 Although this study was not conducted online, its findings were similar to those obtained in the UK study, particularly with regard to the distribution of responses and reliability values. These similarities indicate that the RIBS-J may not be strongly affected by participants’ social desirability distortion. Despite this, we are not certain as to how social desirability influenced participants’ scores on the RIBS-J in this sample.
original Reported and Intended Behaviour Scale, for permitting us to develop a Japanese version of the scale. This study was supported by grants from the JSPS/MEXT (KAKENHI no. 25870143 to SK). All authors declare no conflicts of interest.
Conclusion We tested the psychometric properties of the RIBS-J, and found that most items did not have a floor or ceiling effect. Good internal consistency and reasonable test–retest reliability of the RIBS-J were obtained, similar to the UK study.14 Good construct validity, including convergent validity, divergent validity and good model fit, were also reported. Therefore, we consider that the RIBS-J is an appropriate and psychometrically robust scale for assessing behavior for mental health-related stigma in a Japanese university setting.
ACKNOWLEDGMENTS We are profoundly grateful to Professor Graham Thornicroft and Dr Sara Evans-Lacko of the Institute of Psychiatry, King’s College London, authors of the
REFERENCES 1. Thornicroft G. Shunned: Discrimination against People with Mental Illness. Oxford University Press, New York, 2006. 2. Thornicroft G, Rose D, Kassam A, Sartorius N. Stigma: Ignorance, prejudice or discrimination? Br. J. Psychiatry 2007; 190: 192–193. 3. Stuart H, Arboleda-Flórez J, Sartorius N. Paradigms Lost: Fighting Stigma and the Lessons Learned. Oxford University Press, New York, 2012. 4. Ando S, Clement S, Barley EA, Thornicroft G. The simulation of hallucinations to reduce the stigma of schizophrenia: A systematic review. Schizophr. Res. 2011; 133: 8–16. 5. Yamaguchi S, Wu S-I, Biswas M et al. Effects of short-term interventions to reduce mental health-related stigma in university or college students: A systematic review. J. Nerv. Ment. Dis. 2013; 201: 490–503. 6. Yamaguchi S, Mino Y, Uddin S. Strategies and future attempts to reduce stigmatization and increase awareness of mental health problems among young people: A narrative review of educational interventions. Psychiatry Clin. Neurosci. 2011; 65: 405–415. 7. Clement S, Lassman F, Barley E et al. Mass media interventions for reducing mental health-related stigma. Cochrane Database Syst. Rev. 2013. doi: 10.1002/14651858 .CD009453.pub2 8. Corrigan PW, Morris SB, Michaels PJ, Rafacz JD, Rusch N. Challenging the public stigma of mental illness: A metaanalysis of outcome studies. Psychiatr. Serv. 2012; 63: 963–973. 9. Booker A, Malcarne VL, Sadler GR. Evaluating outcomes of community-based cancer education interventions: A 10-year review of studies. J. Cancer Educ. 2013. doi: 10.1007/s13187-013-0578-6 10. Eisenberg D, Speer N, Hunt JB. Attitudes and beliefs about treatment among college students with untreated mental health problems. Psychiatr. Serv. 2012; 63: 711–713. 11. Ando S, Yamaguchi S, Aoki Y, Thornicroft G. Review of mental health-related stigma in Japan. Psychiatry Clin. Neurosci. 2013; 67: 471–482. 12. Yamaguchi S, Ling H, Kim K, Mino Y. Stigmatisation towards people with mental health problems in secondary school students: An international cross-sectional study between three cities in Japan, China and South-Korea. Int. J. Cult. Ment. Health 2013. doi: 10.1080/17542863.2013 .786108 13. Kassam A, Glozier N, Leese M, Henderson C, Thornicroft G. Development and responsiveness of a scale to measure
© 2014 The Authors Psychiatry and Clinical Neurosciences © 2014 Japanese Society of Psychiatry and Neurology
Psychiatry and Clinical Neurosciences 2014; 68: 448–455
Validation of RIBS-J 455
clinicians’ attitudes to people with mental illness (medical student version). Acta Psychiatr. Scand. 2010; 122: 153– 161. Evans-Lacko S, Rose D, Little K et al. Development and psychometric properties of the Reported and Intended Behaviour Scale (RIBS): A stigma-related behaviour measure. Epidemiol. Psychiatr. Sci. 2011; 20: 263–271. Evans-Lacko S, Henderson C, Thornicroft G. Public knowledge, attitudes and behaviour regarding people with mental illness in England 2009–2012. Br. J. Psychiatry Suppl. 2013; 202: s51–s57. Rusch N, Evans-Lacko SE, Henderson C, Flach C, Thornicroft G. Knowledge and attitudes as predictors of intentions to seek help for and disclose a mental illness. Psychiatr. Serv. 2011; 62: 675–678. Clement S, van Nieuwenhuizen A, Kassam A et al. Filmed v. live social contact interventions to reduce stigma: Randomised controlled trial. Br. J. Psychiatry 2012; 201: 57–64. Evans-Lacko S, Little K, Meltzer H et al. Development and psychometric properties of the mental health knowledge schedule. Can. J. Psychiatry 2010; 55: 157–165. Makita K. Development and reliability of the Japanese language version of Social Distance Scale (SDSJ). Jpn. Bull. Soc. Psychiat. 2006; 14: 231–241 (in Japanese). Clement S, Brohan E, Jeffery D, Henderson C, Hatch S, Thornicroft G. Development and psychometric properties the Barriers to Access to Care Evaluation scale (BACE) related to people with mental ill health. BMC Psychiatry 2012; 12: 36. Lin LIK. A concordance correlation coefficient to evaluate reproducibility. Biometrics 1989; 45: 255–268.
22. Santos JRA. Cronbach’s alpha: A tool for assessing the reliability of scales. J. Extension 1999; 37. [Cited 15 October 2013.] Available from URL: http://www.joe.org/ joe/1999april/tt3.php (last accessed 15 October 2013). 23. Prince M. Measurement in psychiatry. In: Prince M, Stewart R, Ford T, Hotopf M (eds). Practical Psychiatric Epidemiology. Oxford University Press, Oxford, 2003; 13–42. 24. Corrigan P, Markowitz FE, Watson A, Rowan D, Kubiak MA. An attribution model of public discrimination towards persons with mental illness. J. Health Soc. Behav. 2003; 44: 162–179. 25. Crisp AH, Gelder MG, Rix S, Meltzer HI, Rowlands OJ. Stigmatisation of people with mental illnesses. Br. J. Psychiatry 2000; 177: 4–7. 26. Angermeyer MC, Matschinger H. Public beliefs about schizophrenia and depression: Similarities and differences. Soc. Psychiatry Psychiatr. Epidemiol. 2003; 38: 526–534. 27. Cole DA. Utility of confirmatory factor analysis in test validation research. J. Consult. Clin. Psychol. 1987; 55: 584–594. 28. Hair J, Black B, Babin B, Anderson R, Tatham R. Multivariate Data Analysis, 6th edn. Pearson Prentice Hall, Upper Saddle River, 2006. 29. Kaji T, Mishima K, Kitamura S et al. Relationship between late-life depression and life stressors: Large-scale crosssectional study of a representative sample of the Japanese general population. Psychiatry Clin. Neurosci. 2010; 64: 426–434. 30. Richman WL, Kiesler S, Weisband S, Drasgow F. A metaanalytic study of social desirability distortion in computeradministered questionnaires, traditional questionnaires, and interviews. J. Appl. Psychol. 1999; 84: 754–775.
14.
15.
16.
17.
18.
19.
20.
21.
© 2014 The Authors Psychiatry and Clinical Neurosciences © 2014 Japanese Society of Psychiatry and Neurology
doi:10.1111/pcn.12151
Regular Article
Development of a Japanese version of the Reported and Intended Behaviour Scale: Reliability and validity Sosei Yamaguchi, PhD,1* Shinsuke Koike, MD, PhD,2 Kei-ichiro Watanabe, MD, PhD2 and Shuntaro Ando, MD, MScPH3,4 1
Department of Psychiatric Rehabilitation, Institute of Mental Health, National Center of Neurology and Psychiatry, 2Office for Mental Health Support, Division for Counseling and Support, 3Department of Neuropsychiatry, Graduate School of Medicine, The University of Tokyo, and 4Department of Psychiatry and Behavioral Sciences, Tokyo Metropolitan Institute of Medical Science, Tokyo, Japan
Aim: The Reported and Intended Behaviour Scale (RIBS) was developed in the UK to measure mental health-related behavior. The current study aimed to evaluate the applicability, and reliability of a Japanese version of the RIBS (RIBS-J) in a Japanese context, and further examine the construct validity of the RIBS-J. Methods: The sample included 224 undergraduate and postgraduate students at a Japanese university. Cronbach’s alpha was used to assess internal consistency. Pearson’s correlation coefficient was used to examine the divergent validity between the RIBS-J and the Mental Health Knowledge Schedule and the convergent validity between the second subscale of the RIBS-J and Japanese version of the Social Distance Scale. Confirmatory factor analysis assessed the goodness of model fit of the RIBS-J. We also examined test–retest reliability with another undergraduate sample (n = 29). Results: Most items exhibited no floor/ceiling effect. High internal consistency (α = 0.83) was reported.
HE STIGMA ATTACHED to mental health by the general public is a global concern, as it adversely influences all aspects of the lives of people with mental health problems, including income levels, housing, unemployment, the lower rate of marriage
T
*Correspondence: Sosei Yamaguchi, PhD, Department of Psychiatric Rehabilitation, Institute of Mental Health, National Center of Neurology and Psychiatry, 4-1-1 Ogawa-Higashi, Kodaira, Tokyo 187-8553, Japan. Email: [email protected] Received 21 October 2013; revised 14 December 2013; accepted 19 December 2013.
448
The second subscale of the RIBS-J, measuring intended behavior, correlated with the Mental Health Knowledge Schedule (r = 0.33, P < 0.001) and the Japanese version of the Social Distance Scale (r = −0.60, P < 0.001). In addition, confirmatory factor analysis found good model fit for the RIBS-J (χ2 = 41.001, d.f. = 19, P = 0.002, goodness-of-fit index = 0.956, adjusted goodness-of-fit index = 0.916, comparative fit index = 0.955, root mean square error of approximation = 0.072). Overall test–retest reliability (ρc) was 0.71.
Conclusion: The RIBS-J is an appropriate and psychometrically robust measure of behavior towards individuals with mental health problems in Japan. Further studies using a community sample could assess the generalizability of our findings. Key words: behavior, discrimination, psychometric properties, scale development, validity.
among this group, and smaller social networks.1 A conceptual framework of mental health-related stigma comprises the following three dimensions: ignorance (lack of knowledge), prejudice (negative attitudes and feelings), and discrimination (behavioral enactment of prejudice).2 Studies have consistently emphasized that the elimination of discrimination against the mentally ill is the most important aspect for combating mental health-related stigma.1,3 Nevertheless, recent systematic reviews of anti-stigma interventions have revealed that most studies seldom focused on others’ actual discriminatory behavior towards people with mental
© 2014 The Authors Psychiatry and Clinical Neurosciences © 2014 Japanese Society of Psychiatry and Neurology
Psychiatry and Clinical Neurosciences 2014; 68: 448–455
health problems as research outcomes.4–8 It is imperative to directly assess others’ discriminatory behavior (behavioral problems) in addition to knowledge and attitudes, as the improvements in proximal outcomes, such as people’s knowledge and attitudes, do not always guarantee their behavioral changes.9,10 Japanese studies in particular have rarely reported discrimination towards people with mental health problems, although Japanese adults and children both tend to hold more negative attitudes than individuals in other countries.11,12 This lack of research evidence on discrimination could be attributed to the absence of scales to accurately assess individuals’ behavior towards people with mental health problems. In fact, most of the scales that assess such stigma focus only on others’ knowledge of mental health issues and attitudes towards people with mental health problems.13 To the best of our knowledge, there is only one reliable scale, namely, the Reported and Intended Behaviour Scale (RIBS), which was designed to assess people’s past and present behaviors and behavioral intentions towards those with mental health problems, using two subscales comprising eight items in total.14 Overall, the RIBS was identified as having good feasibility (average time required for completion = 1 min) and good reliability as a result of the internal consistency of the second subscale (α = 0.85) and overall test–retest reliability (Lin’s concordance statistic: ρc = 0.75).14 Indeed, the RIBS has been utilized in several recent local and national studies on stigma, particularly in the UK.15–17 The RIBS has been mainly used in research conducted in the UK; therefore, little is known about its applicability in Japan or other countries. In addition, previous study has only examined one type of validity relating to the RIBS, namely, content validity.14 Therefore, this study aimed to develop a Japanese version of the RIBS (RBIS-J) and empirically test its applicability, reliability, (construct) validity, including convergent validity, divergent validity and goodness of model fit in a Japanese context.
METHODS Study 1 Participants Participants were undergraduate and graduate students of a university in Tokyo enrolled in several courses unrelated to mental health.
Validation of RIBS-J 449
Table 1 Participant characteristics n = 224 Sex School Major subjects
Mean age (SD)
Male Female Undergraduate Postgraduate Humanities† Science‡ Other 22.61 (2.47)
n
%
163 61 153 71 126 96 2
72.8 27.2 68.3 31.7 56.3 42.9 0.9
† Major subjects in the ‘Humanities’ include courses in art, linguistics, sociology, education, and psychology. ‡ Major subjects in ‘Science’ include courses in medicine, engineering, physics, mathematics, and chemistry.
The questionnaires were distributed to 235 students who attended a ‘stress management’ lecture. Participants were informed of the aim of the study, data collection procedures, and the implications of participation or non-participation in this study via the cover page of the questionnaire or verbal account from the second author (S.K.). Questionnaires were completed only by students who voluntarily consented to participate in the study. Informed consent was obtained from all participants (224 participants in total, response rate: 95.3%). Participants’ sociodemographic information is shown in Table 1. Approximately 70% of participants were male, with a mean age of 22.61 years (SD = 2.47). In addition, undergraduate students represented about 70% of the study sample. Around half of the participants were majoring in the humanities. Instrument The RIBS comprises two subscales containing eight items in total. The first subscale includes four statements relating to past or present contact with people with mental health problems. Each item receives a score of 1 for ‘yes’ answers and 0 for ‘no’ or ‘don’t know’ (score range of the subscale: 0–4). Higher scores indicate more past or present contact. The second subscale consists of four statements relating to participants’ future behavioral intentions when they come into contact with people with mental health problems. Responses to the items are rated as ‘agree strongly’, ‘agree slightly’, ‘neither agree nor
© 2014 The Authors Psychiatry and Clinical Neurosciences © 2014 Japanese Society of Psychiatry and Neurology
450 S. Yamaguchi et al.
Psychiatry and Clinical Neurosciences 2014; 68: 448–455
disagree’, ‘disagree slightly’, ‘disagree strongly’ and ‘don’t know’. For each item, a score of 5 was allocated for strong agreement, while a score of 1 was assigned to strong disagreement (score range of the subscale: 4–20). The response option ‘don’t know’, is coded as neutral (i.e., 3). Higher scores indicate more favorable behavioral intentions.
The SDSJ comprises five items measuring individuals’ desire for social distance from people with schizophrenia (e.g., ‘If a person with schizophrenia is a taxi driver, I would not want to take his/her taxi’); lower scores indicate a more positive attitude. This measure has shown good construct validity as shown by both exploratory and confirmatory factor analyses, good internal consistency (α = 0.82), and good test– retest reliability (r = 0.88) in Japanese settings.19 Confirmatory factor analysis was also performed to test the goodness of model fit of the RIBS-J in the Japanese context. All analyses were conducted with Stata version 12 and AMOS version 20 software.
Japanese translation We undertook the following three steps in order to translate the RIBS into Japanese. The original RIBS was translated into Japanese by the first author (S.Y.). Back-translation was conducted by a native English speaker from England, whose second language was Japanese. Finally, the back-translated version of the RIBS was confirmed and approved by the researchers who had originally developed the RIBS. Reliability/validity and statistical methods We examined several aspects relating to the reliability and validity of the RIBS-J. In terms of applicability, the floor and ceiling effects were assessed on the basis of the proportions of responses and means (SD) obtained for each item. T-tests and Pearson’s correlation coefficients were calculated to confirm whether scores on the RIBS-J differed according to sociodemographic characteristics. Similar to the UK study, Cronbach’s alpha (α) was calculated in order to determine the internal consistency of the second subscale of the RIBS-J, which measures intended behavior.14 Three types of construct validity were tested in this study. To test divergent and convergent validity, we calculated Pearson’s correlation coefficient (r) in order to determine the association between the RIBS-J and the Mental Health Knowledge Schedule (MAKS), and between the RIBS-J and the Japanese version of the Social Distance Scale (SDSJ). The MAKS comprises six items that assess knowledge of mental illness (e.g., ‘Medication can be an effective treatment for people with mental health problems’).18 Higher scores indicate greater knowledge of mental health problems. The MAKS was developed by the same research team that developed the RIBS. The content validity, internal consistency (α = 0.65), and overall test–retest reliability (ρc = 0.71) of the MAKS have been demonstrated from a previous study.18 The process of translation for the MAKS was identical to that of the RIBS-J.
Study 2 We recruited a second sample of 35 undergraduates from the same university by advertising on a university message board. Of these 35 participants, 29 agreed to complete the RIBS-J twice (at an interval of 1 month) to enable the evaluation of test–retest reliability. In this sample, 24 participants were male (82.8%) and the mean age was 19.62 (SD = 1.12). All the participants were undergraduate students. In accordance with Evans-Lacko et al., we used Lin’s concordance statistic to calculate test–retest reliability, not only for the first and second subscales of the RIBS-J, but also the entire RIBS-J scale.14 Lin’s concordance statistic (ρc) has often been employed in recent studies to examine test–retest reliability of measures assessing mental health-related stigma.14,15,20,21
Ethics approval This study was approved by the Research Ethics Committee at the University of Tokyo (approval number: 12-102).
RESULTS Study 1 Applicability (responses and means) Table 2 presents participants’ responses to the eight items of the RIBS-J, also indicating the distribution of responses according to all of the response options provided for each item. The non-response option constituted more than 75% of participants’ responses on the first subscale. Similarly, preference for ‘agree
© 2014 The Authors Psychiatry and Clinical Neurosciences © 2014 Japanese Society of Psychiatry and Neurology
Psychiatry and Clinical Neurosciences 2014; 68: 448–455
Validation of RIBS-J 451
Table 2 Distribution of participants’ responses Item no.
Statements in subscale 1: Reported behavior (n = 224)
RIBS-J 1
Are you currently living with, or have you ever lived with, someone with a mental health problem? RIBS-J 2 Are you currently working with, or have you ever worked with, someone with a mental health problem? RIBS-J 3 Do you currently have, or have you ever had, a neighbor with a mental health problem? RIBS-J 4 Do you currently have, or have you ever had, a close friend with a mental health problem? Total: Subscale 1†
Item no. RIBS-J 5
Statements in subscale 2: Intended behavior (n = 224)
In the future, I would be willing to live with someone with a mental health problem. RIBS-J 6 In the future, I would be willing to work with someone with a mental health problem. RIBS-J 7 In the future, I would be willing to live near someone with a mental health problem. RIBS-J 8 In the future, I would be willing to continue a relationship with a friend who has developed a mental health problem. Total: Subscale 2‡ Total: Subscales 1 and 2
n (%) Yes
Mean† (SD)
No
Don’t know
50 (22.3)
154 (68.8)
20 (8.94)
0.22 (0.42)
33 (14.7)
162 (72.3)
29 (12.9)
0.15 (0.36)
46 (20.5)
123 (54.9)
55 (24.6)
0.21 (0.40)
89 (39.7)
96 (43.3)
38 (17.0)
0.40 (0.49) 0.97 (1.07)
Agree strongly
Agree slightly
Neither agree nor disagree
Disagree slightly
Disagree strongly
Don’t know
Mean‡ (SD)
10 (4.5)
34 (15.2)
42 (18.8)
54 (24.1)
44 (19.6)
40 (17.9)
2.61 (1.10)
14 (6.3)
45 (20.1)
69 (30.8)
37 (16.5)
28 (12.5)
31 (13.8)
2.91 (1.06)
13 (5.8)
37 (16.5)
51 (22.8)
45 (20.1)
40 (17.9)
38 (17.0)
2.72 (1.11)
101 (45.1)
87 (38.8)
11 (4.9)
11 (4.9)
3 (1.3)
11 (4.9)
4.21 (0.91) 12.46 (3.42) 13.43 (3.83)
Calculation of mean scores: †‘Yes’ = 1, ‘No’ = 0, and ‘Don’t know’ = 0. ‡‘Agree strongly’ = 5, ‘Agree slightly’ = 4, ‘Neither agree nor disagree’ = 3 ‘Disagree slightly’ = 2, ‘Disagree strongly’ = 1, and ‘Don’t know’ = 3.
(disagree) strongly’ and ‘agree (disagree) slightly’ made up no more than 50% of participants’ total responses on three items of the second subscale. One item (RIBS-J 8: ‘In the future, I would be willing to continue a relationship with a friend who developed a mental health condition.’) had a slight ceiling effect (mean = 4.21, SD = 0.91). In addition, ‘don’t know’ as a response option made up no more than 30% for all of the items in both subscales. The mean scores on the first and second subscales of the RIBS-J were 0.97 (SD = 1.07) and 12.46 (SD = 3.42), respectively; the mean total score was 13.43 (SD = 3.83). The students’ age showed a slight, yet significant, correlation with the scores on the first subscale (r = 0.17, P = 0.011). However, no significant associations were found between scores on the RIBS-J (for both subscales) and other sociodemographic characteristics. Internal consistency and construct validity (convergent validity, divergent validity, and model fit) A Cronbach’s alpha (α) of 0.83 was obtained for the second subscale of the RIBS-J. The mean scores on the
MAKS and the SDSJ were 21.25 (SD = 2.93) and 6.90 (SD = 3.34), respectively. With regard to divergent validity, the score on the first subscale of the RIBS-J did not show a significant correlation with either the SDS-J score or the MAKS score. The MAKS score had a relatively weak correlation with the score on the second subscale of the RIBS-J (r = 0.33, P < 0.001). These indicated good divergent validity of the RIBS-J. Conversely, a moderate negative correlation was found between the score on the second subscale of the RIBS-J and the SDS-J score (r = −0.60, P < 0.001), indicating good convergent validity. The results of confirmatory factor analysis are presented in Table 3. The χ2-test for goodness-of-fit was significant (χ2 = 41.001, d.f. = 19, P = 0.002). However, the other three fitness indices were quite high (goodness-of-fit index [GFI] = 0.956, adjusted goodness-of-fit index [AGFI] = 0.916, comparative fit index [CFI] = 0.955), while one was less than 0.1 (RMSEA = 0.072). The standardized path coefficients for item 3 (RIBS-J 3: ‘Do you currently have, or have you ever had, a neighbor with a mental health problem?’) was relatively low (β = 0.36, P = 0.010). The standardized path coefficients of the remaining seven items were 0.44–0.87 (P < 0.001).
© 2014 The Authors Psychiatry and Clinical Neurosciences © 2014 Japanese Society of Psychiatry and Neurology
452 S. Yamaguchi et al.
Psychiatry and Clinical Neurosciences 2014; 68: 448–455
Table 3 Confirmatory factor analysis and model fit n = 224: χ2 = 41.001, d.f. = 19, (P = 0.002), GFI = 0.956, AGFI = 0.916, CFI = 0.955, RMSEA = 0.072 Correlation between subscale 1 (items 1–4) and subscale 2 (items 5–8) = 0.38, P < 0.001 Standardized path coefficients (stranded error) Subscale 1: Reported behavior
Item no.
Statements
RIBS-J 1
Are you currently living with, or have you ever lived with, someone with a mental health problem? Are you currently working with, or have you ever worked with, someone with a mental health problem? Do you currently have, or have you ever had, a neighbor with a mental health problem? Do you currently have, or have you ever had, a close friend with a mental health problem? In the future, I would be willing to live with someone with a mental health problem. In the future, I would be willing to work with someone with a mental health problem. In the future, I would be willing to live near someone with a mental health problem. In the future, I would be willing to continue a relationship with a friend who developed a mental health problem.
RIBS-J 2 RIBS-J 3 RIBS-J 4 RIBS-J 5 RIBS-J 6 RIBS-J 7 RIBS-J 8
Subscale 2: Intended behavior
0.44 (0.04)* 0.46 (0.04)* 0.36 (0.04)* 0.56 (0.05)* 0.81 (0.06)* 0.87 (0.06)* 0.85 (0.06)* 0.44 (0.06)*
*P < 0.01. AGFI, adjusted goodness-of-fit index; CFI, comparative fit index; GFI, goodness-of-fit index; RMSEA, root mean square error of approximation.
Study 2 Lin’s concordance statistic (ρc) for both the first and second subscales of the RIBS-J was 0.72 (SE = 0.09) and 0.70 (SE = 0.10), respectively. The entire scale was 0.71 (SE = 0.10), indicating the overall test– retest reliability of the RIBS-J.
DISCUSSION We tested the applicability and reliability of the RIBS-J in a Japanese setting, also evaluating several aspects relating to the validity of the RIBS-J. This study adds new evidence for the psychometric properties of the RIBS, particularly regarding convergent validity, divergent validity, and goodness of model fit.
Applicability in Japan Other than item 8 of the second subscale, the distribution of responses was not skewed for the other
items of the RIBS-J. A ceiling effect was observed for item 8 (mean = 4.21, SD = 0.91); however, EvansLacko et al. also observed a ceiling effect for this item (mean = 4.04, SD = 1.02) in a study conducted in the UK.14 In other words, the responses given by our sample may have been somewhat similar to those provided by the community sample used in the UK study. While the score on the RIBS-J showed no association with sex, major subjects, or academic grades (undergraduate/postgraduate), we found a weak correlation between age and scores on the first subscale of the RIBS-J, which addresses participants’ past and present experiences. Not surprisingly, older participants had more opportunities to have encountered various individuals, including people with mental health problems, in their lives, as opposed to younger participants. Therefore, we cannot assume that the weak correlation between age and scores on the first subscale of the RIBS-J completely nullifies the validity and applicability of the RIBS-J in Japan.
© 2014 The Authors Psychiatry and Clinical Neurosciences © 2014 Japanese Society of Psychiatry and Neurology
Psychiatry and Clinical Neurosciences 2014; 68: 448–455
Reliability The internal consistency of the second subscale of the RIBS-J was quite high, and satisfactory (α = 0.83) considering that, in general, the minimum threshold is 0.7.22 In addition, overall test–retest reliability for the RIBS-J (ρc = 0.71) was acceptable, when taking into account past studies aimed at developing scales measuring mental health-related stigma.14,18,20 In addition, it has been noted that the Cronbach’s alpha and Lin’s concordance statistic obtained in this study were similar to those reported for a study conducted in the UK (α = 0.85 and ρc = 0.75).14
Validity This study illustrates the originality of the RIBS-J in its assessment of behavior and behavioral intentions through the divergent validity test. Low correlation coefficients between a measure and other conceptually different measures are given as evidence of divergent validity.23 The lack of correlation between the first subscale of the RIBS-J and either the MAKS or the SDSJ means that the first subscale of the RIBS-J does not measure participants’ knowledge and attitudes, and thus indicates good divergent validity of the first subscale of the RIBS-J. The second subscale of the RIBS-J showed a weak correlation with the MAKS. In relation to this, past studies have also reported associations between participants’ knowledge of mental health issues and their behavioral intentions or social distance towards people with mental health problems.12,24 Therefore, it seems that the weak correlation, rather than strong or moderate correlation, observed between the second subscale of the RIBS-J and the MAKS further indicated the divergent validity of the RIBS-J. A significant correlation was found between the second subscale of the RIBS-J and the SDSJ. Strictly, the second subscale of the RIBS-J does not measure behavior, as such, but participants’ future intentions to be in close proximity to individuals with mental health problems in four given situations. The SDSJ measures participants’ attitudes or feelings, assuming participants’ desire for social distance from individuals with schizophrenia in five particular situations. In other words, the second subscale of the RIBS-J partially resembles a social distance scale. It is assumed that a relatively high correlation between the second subscale of the RIBS-J and the SDSJ ensures good convergent validity of the RIBS-J.
Validation of RIBS-J 453
On the other hand, the difference in the target conditions between RIBS-J (mental health problems) and SDSJ (schizophrenia) may affect the results. Generally, the general public tends to hold more stigmatizing attitudes towards people with schizophrenia than those with other psychiatric disorders, like depression, not only in Japan but also in Western countries.11,25,26 It is presumed that higher correlation coefficient would be found, if we employed other measures for people’s desire for social distance from people with mental health problems rather than schizophrenia. Overall, this study obtained good model fit for the RIBS-J. Although the χ2 for goodness of fit proved significant, when sample size is enlarged, the power to detect statistically significant differences also increases.27,28 In short, the fact that the χ2 yielded statistically significant results does not always indicate that the model is poor.28 Indeed, other indices of model fit for the RIBS-J in this study seemed to yield good model fit values; the minimum threshold of the GFI, AGFI, and CFI were more than 0.9, while the RMSEA was less than 0.1.28 The standardized path coefficients in seven of eight items were higher than 0.4 and all the standardized path coefficients were significant. The reason for the relatively low coefficient found for item 3 is unclear. However, it is assumed that this low coefficient may be due to the inherent differences in the constructs measured by item 3 and the other seven items. For instance, item 3 requires participants to provide answers about an unfamiliar person (a neighbor), whereas the other three items in the first subscale of the RIBS-J specifically relate to closer acquaintances, such as family members, friends, and colleagues. In Japanese culture, people often hesitate to mention distant acquaintances and their illnesses.11 Moreover, because communication with neighbors is relatively low in large cities, such as Tokyo, in Japan, people may not be well acquainted with their neighbors.29 As a result, the content of item 3 in relation to broader Japanese culture may have contributed to the relatively higher frequency (24.6%) of the ‘don’t know’ response compared to other items; in turn, this may have influenced the low coefficient that was obtained for this item.
Study limitations Some limitations of the study were identified. First, as opposed to a community sample, participants in this
© 2014 The Authors Psychiatry and Clinical Neurosciences © 2014 Japanese Society of Psychiatry and Neurology
454 S. Yamaguchi et al.
Psychiatry and Clinical Neurosciences 2014; 68: 448–455
study comprised university students who attended a lecture on ‘stress management’; thus, the study’s findings may not apply to the general public. Accordingly, the generalizability of the results could pose a challenge in the future. Second, we used the MAKS to test the divergent validity of the RIBS-J, because the same research team in the UK developed these two scales.14,18 However, the psychometrics of the MAKS in a Japanese setting is unclear. Therefore, we may not properly evaluate the divergent validity between RIBS-J and MAKS. The third limitation is the issue of social desirability. It cannot be denied that students may have reduced the extent of self-disclosure when completing the questionnaires due to social desirability distortion. To counter this, Evans-Lacko et al. conducted an online survey for their UK study to test the psychometric properties of the RIBS, as opposed to conducting interviews.14 This was because an online survey enhances participants’ perception of anonymity as compared to interviews, leading to a reduced tendency to give socially desirable responses.30 Although this study was not conducted online, its findings were similar to those obtained in the UK study, particularly with regard to the distribution of responses and reliability values. These similarities indicate that the RIBS-J may not be strongly affected by participants’ social desirability distortion. Despite this, we are not certain as to how social desirability influenced participants’ scores on the RIBS-J in this sample.
original Reported and Intended Behaviour Scale, for permitting us to develop a Japanese version of the scale. This study was supported by grants from the JSPS/MEXT (KAKENHI no. 25870143 to SK). All authors declare no conflicts of interest.
Conclusion We tested the psychometric properties of the RIBS-J, and found that most items did not have a floor or ceiling effect. Good internal consistency and reasonable test–retest reliability of the RIBS-J were obtained, similar to the UK study.14 Good construct validity, including convergent validity, divergent validity and good model fit, were also reported. Therefore, we consider that the RIBS-J is an appropriate and psychometrically robust scale for assessing behavior for mental health-related stigma in a Japanese university setting.
ACKNOWLEDGMENTS We are profoundly grateful to Professor Graham Thornicroft and Dr Sara Evans-Lacko of the Institute of Psychiatry, King’s College London, authors of the
REFERENCES 1. Thornicroft G. Shunned: Discrimination against People with Mental Illness. Oxford University Press, New York, 2006. 2. Thornicroft G, Rose D, Kassam A, Sartorius N. Stigma: Ignorance, prejudice or discrimination? Br. J. Psychiatry 2007; 190: 192–193. 3. Stuart H, Arboleda-Flórez J, Sartorius N. Paradigms Lost: Fighting Stigma and the Lessons Learned. Oxford University Press, New York, 2012. 4. Ando S, Clement S, Barley EA, Thornicroft G. The simulation of hallucinations to reduce the stigma of schizophrenia: A systematic review. Schizophr. Res. 2011; 133: 8–16. 5. Yamaguchi S, Wu S-I, Biswas M et al. Effects of short-term interventions to reduce mental health-related stigma in university or college students: A systematic review. J. Nerv. Ment. Dis. 2013; 201: 490–503. 6. Yamaguchi S, Mino Y, Uddin S. Strategies and future attempts to reduce stigmatization and increase awareness of mental health problems among young people: A narrative review of educational interventions. Psychiatry Clin. Neurosci. 2011; 65: 405–415. 7. Clement S, Lassman F, Barley E et al. Mass media interventions for reducing mental health-related stigma. Cochrane Database Syst. Rev. 2013. doi: 10.1002/14651858 .CD009453.pub2 8. Corrigan PW, Morris SB, Michaels PJ, Rafacz JD, Rusch N. Challenging the public stigma of mental illness: A metaanalysis of outcome studies. Psychiatr. Serv. 2012; 63: 963–973. 9. Booker A, Malcarne VL, Sadler GR. Evaluating outcomes of community-based cancer education interventions: A 10-year review of studies. J. Cancer Educ. 2013. doi: 10.1007/s13187-013-0578-6 10. Eisenberg D, Speer N, Hunt JB. Attitudes and beliefs about treatment among college students with untreated mental health problems. Psychiatr. Serv. 2012; 63: 711–713. 11. Ando S, Yamaguchi S, Aoki Y, Thornicroft G. Review of mental health-related stigma in Japan. Psychiatry Clin. Neurosci. 2013; 67: 471–482. 12. Yamaguchi S, Ling H, Kim K, Mino Y. Stigmatisation towards people with mental health problems in secondary school students: An international cross-sectional study between three cities in Japan, China and South-Korea. Int. J. Cult. Ment. Health 2013. doi: 10.1080/17542863.2013 .786108 13. Kassam A, Glozier N, Leese M, Henderson C, Thornicroft G. Development and responsiveness of a scale to measure
© 2014 The Authors Psychiatry and Clinical Neurosciences © 2014 Japanese Society of Psychiatry and Neurology
Psychiatry and Clinical Neurosciences 2014; 68: 448–455
Validation of RIBS-J 455
clinicians’ attitudes to people with mental illness (medical student version). Acta Psychiatr. Scand. 2010; 122: 153– 161. Evans-Lacko S, Rose D, Little K et al. Development and psychometric properties of the Reported and Intended Behaviour Scale (RIBS): A stigma-related behaviour measure. Epidemiol. Psychiatr. Sci. 2011; 20: 263–271. Evans-Lacko S, Henderson C, Thornicroft G. Public knowledge, attitudes and behaviour regarding people with mental illness in England 2009–2012. Br. J. Psychiatry Suppl. 2013; 202: s51–s57. Rusch N, Evans-Lacko SE, Henderson C, Flach C, Thornicroft G. Knowledge and attitudes as predictors of intentions to seek help for and disclose a mental illness. Psychiatr. Serv. 2011; 62: 675–678. Clement S, van Nieuwenhuizen A, Kassam A et al. Filmed v. live social contact interventions to reduce stigma: Randomised controlled trial. Br. J. Psychiatry 2012; 201: 57–64. Evans-Lacko S, Little K, Meltzer H et al. Development and psychometric properties of the mental health knowledge schedule. Can. J. Psychiatry 2010; 55: 157–165. Makita K. Development and reliability of the Japanese language version of Social Distance Scale (SDSJ). Jpn. Bull. Soc. Psychiat. 2006; 14: 231–241 (in Japanese). Clement S, Brohan E, Jeffery D, Henderson C, Hatch S, Thornicroft G. Development and psychometric properties the Barriers to Access to Care Evaluation scale (BACE) related to people with mental ill health. BMC Psychiatry 2012; 12: 36. Lin LIK. A concordance correlation coefficient to evaluate reproducibility. Biometrics 1989; 45: 255–268.
22. Santos JRA. Cronbach’s alpha: A tool for assessing the reliability of scales. J. Extension 1999; 37. [Cited 15 October 2013.] Available from URL: http://www.joe.org/ joe/1999april/tt3.php (last accessed 15 October 2013). 23. Prince M. Measurement in psychiatry. In: Prince M, Stewart R, Ford T, Hotopf M (eds). Practical Psychiatric Epidemiology. Oxford University Press, Oxford, 2003; 13–42. 24. Corrigan P, Markowitz FE, Watson A, Rowan D, Kubiak MA. An attribution model of public discrimination towards persons with mental illness. J. Health Soc. Behav. 2003; 44: 162–179. 25. Crisp AH, Gelder MG, Rix S, Meltzer HI, Rowlands OJ. Stigmatisation of people with mental illnesses. Br. J. Psychiatry 2000; 177: 4–7. 26. Angermeyer MC, Matschinger H. Public beliefs about schizophrenia and depression: Similarities and differences. Soc. Psychiatry Psychiatr. Epidemiol. 2003; 38: 526–534. 27. Cole DA. Utility of confirmatory factor analysis in test validation research. J. Consult. Clin. Psychol. 1987; 55: 584–594. 28. Hair J, Black B, Babin B, Anderson R, Tatham R. Multivariate Data Analysis, 6th edn. Pearson Prentice Hall, Upper Saddle River, 2006. 29. Kaji T, Mishima K, Kitamura S et al. Relationship between late-life depression and life stressors: Large-scale crosssectional study of a representative sample of the Japanese general population. Psychiatry Clin. Neurosci. 2010; 64: 426–434. 30. Richman WL, Kiesler S, Weisband S, Drasgow F. A metaanalytic study of social desirability distortion in computeradministered questionnaires, traditional questionnaires, and interviews. J. Appl. Psychol. 1999; 84: 754–775.
14.
15.
16.
17.
18.
19.
20.
21.
© 2014 The Authors Psychiatry and Clinical Neurosciences © 2014 Japanese Society of Psychiatry and Neurology
