AIDS Behav (2014) 18:1133–1141 DOI 10.1007/s10461-014-0698-y
ORIGINAL PAPER
The Validity of the Substance Abuse and Mental Illness Symptom Screener (SAMISS) in People Living with HIV/AIDS in Primary HIV Care in Cape Town, South Africa Erica Breuer • Kevin Stoloff • Landon Myer • Soraya Seedat • Dan J. Stein • John A. Joska
Published online: 23 January 2014 Ó Springer Science+Business Media New York 2014
Abstract Given the high prevalence of HIV in South Africa and co-morbid mental disorders in people living with HIV/AIDs (PLWHA) we sought to validate a brief screening tool in primary HIV care. Methods: 366 PLWHA were recruited prior to combination anti-retroviral treatment (CART) initiation from two primary health HIV clinics. A mental health nurse administered a sociodemographic questionnaire and the Mini Neuropsychiatric Interview (MINI) and a lay counsellor administered the Substance and Mental Illness Symptom Screener (SAMISS). Results: Using the MINI, 17 % of participants were identified with either depression, anxiety disorders or adjustment disorder and 18 % with substance or alcohol abuse/dependence. The sensitivity and specificity of the SAMISS was 94 % (95 % CI: 88–98 %) and 58 % (95 % CI: 52–65 %) respectively, with the alcohol component (sensitivity: 94 %; specificity: 85 %) performing better
E. Breuer (&) K. Stoloff J. A. Joska Alan J. Flisher Centre for Public Mental Health, Department of Psychiatry and Mental Health, University of Cape Town, Building B, 46 Sawkins Rd, Rondebosch, Cape Town 7700, South Africa e-mail: [email protected] L. Myer D. J. Stein Division of Epidemiology & Biostatistics, School of Public Health and Family Medicine, University of Cape Town, Cape Town, South Africa S. Seedat Department of Psychiatry, University of Stellenbosch, Stellenbosch, South Africa S. Seedat Medical Research Council of South Africa, Cape Town, South Africa
than the mental illness component of the SAMISS (sensitivity: 97 %; specificity: 60 %). The specificity of the tool improved when the cut-off for the mental illness component was increased. Conclusion: The SAMISS may provide a useful first tier screening tool for common mental disorders in primary care for PLWHA. Keywords HIV/AIDS Mental illness Screening sub-Saharan Africa
Introduction The prevalence of common mental disorders (CMD) in people living with HIV/AIDS (PLWHA) is high [1] [2]. In the United States, around half of PLWHA, screen positive for a mental illness [3]. Similar results are found in subSaharan Africa [2], where both HIV prevalence and incidence remains high. UNAIDS estimate that around 23.5 million PLWHA live in sub-Saharan Africa with 1.8 million people infected per year [4]. In South Africa, a nationally representative study of PLWHA attending public health facilities found that 44 % had a mental illness according to the WHO Composite International Diagnostic Interview [5]. This corresponds with other South African studies which report the prevalence of mental illness in PLWHA attending HIV clinics between 19 and 56 % [6–8]. Among the most prevalent CMD are depression, anxiety, substance abuse and posttraumatic stress disorder [1]. This increased prevalence compared with general population or health facility attendees is likely due to the combination of biological, psychological and social factors, such as the neurological effects of HIV on the brain [9] and HIV-associated stigma [10].
123
1134
Mental disorders in PLWHA lead to poorer HIV outcomes. A recent systematic review by Mayston and colleagues reported that PLWHA with mental disorders in low and middle income countries have poorer adherence to CART [11]. In addition, PLWHA with concomitant mental illness have delayed CART initiation [12], poorer immunological outcomes [13] and higher morbidity and mortality [14, 15]. However, antidepressant treatment has been found to improve adherence to antiretroviral therapy [16] and initiation [17]. Therefore the identification and treatment of mental illness in PLWHA may be an important way of increasing adherence, and by extension, healthoutcomes, particularly in settings with a high prevalence of HIV. Given these poorer health outcomes and the high estimated prevalence of HIV (10.6 %) [18], there is a need to identify patients with co-morbid mental illness in order to improve both mental health and HIV outcomes. However, given the large burden of care facing health professionals working in South Africa [19] and the paucity of mental health professionals in primary care settings [20], the addition of lengthy or complex screening protocols are not feasible. Good specificity of the screening tool is important as over-detection can lead to additional financial and human resource burden on the health system, which is particularly important to avoid in already resource constrained settings such as South Africa [21]. Therefore we sought to find a short reliable and valid screening tool with good sensitivity and specificity which could be used at the point of initiation of CART to screen for mental illness. As lay counsellors in South Africa currently have a key role in preparing patients for CART initiation as well as providing basic psychosocial counselling and monitoring adherence, they are ideally placed to administer a screening tool within a primary care setting. Various screening tools have been validated in PLWHA in sub-Saharan Africa [22]. These include the Centre for Epidemiological Studies Depression Scale, the Alcohol Use Disorders Identification Test [8], the Kessler Psychological Distress Scale-10 [23] the Hopkins Symptom Checklist-25 (HSCL-25) in antenatal women [24] and the Edinburgh Postnatal Depression Scale in postnatal women [25]. However, despite moderate to high sensitivity and specificity, these screen only either for alcohol and substance abuse or for depression/anxiety and would require the administration of multiple screening tools in order to provide coverage for the range of CMD which affect PLWHA. The Substance Use and Mental Illness Symptom Screener (SAMISS) was developed in the United States to screen for alcohol, substance use and common mental illness specifically in PLWHA. There have been several versions of this tool published: a 13 item tool [26] and a 16
123
AIDS Behav (2014) 18:1133–1141
item tool [27]. The 16 item tool contains items from the Alcohol Use Disorders Identification Test, the Two-Item Conjoint Screener and the Composite Diagnostic Interview Screening tool as well as some designed specifically for the SAMISS. This version was validated by Pence and colleagues against the Structured Clinical Interview for DSMIV at the University of North Carolina at Chapel Hill Hospital [27]. The sensitivity and the specificity of the alcohol and substance use component of the SAMISS was 86 and 75 % respectively and 95 and 49 % sensitivity and specificity for the mental illness component. In a previous study in the same setting, we compared mental health nurse and lay counsellor administration of the SAMISS. The reliability of lay counsellor administration was fair (j = 0.39, 95 % CI 0.29–0.49, P \ 0.01) with a higher proportion of PLWHA screening positive when lay counsellors administered the tool compared to nurses [28]. However, as the validity of the SAMISS had not been established, we could not determine whether counsellors were over detecting cases of mental illness. This study aimed to determine the validity of a lay counsellor administered SAMISS compared to the Mini International Neuropsychiatric Interview in a pre-HAART PLWHA in two primary care HIV clinics in South Africa.
Methods The study was a cross-sectional validity study performed at two primary health care clinics: Langa and Crossroads Community Health Clinics in Cape Town, South Africa. We included PLWHA who were CART naı¨ve and excluded those that had been previously assessed with the mental health screening tools at the clinic in order to minimise bias due to practice effects. A sample size of 360 subjects was estimated by calculating the width of confidence intervals for the sensitivity and specificity using values from Pence et al.’s [27] validation study. Potential participants were referred by nursing staff for inclusion. Following screening for suitability and written informed consent, a mental health nurse administered a socio-demographic questionnaire. Variables such as age, sex, level of education, employment status, marital status, current abode, access to clinic and home language were obtained. Participants were then randomized using a predefined list based on a random number table to either having the Substance and Mental Illness Symptom Screener (SAMISS) administered first followed by the Mini International Neuropsychiatric Interview (MINI), or vice versa. The order was randomised to reduce any bias introduced by administering one questionnaire before the other.
AIDS Behav (2014) 18:1133–1141
The MINI was administered by a mental health nurse trained in its administration. The training for counsellors in both clinics consisted of four sessions of one hour, with ongoing feedback and consultation with the study psychiatrist (KS) throughout the project. The MINI was administered in English, however, there were some cases where the questions were clarified in isiXhosa, the first language of the majority of participants. Initially the MINI Plus was used but this was changed due to programmatic reasons to the MINI Six with the addition of the adjustment disorder module from the MINI Plus. Only the depression, anxiety, adjustment disorder, bipolar disorder and substance and alcohol use components of the MINI were administered to align with the disorders screened for on the SAMISS. The SAMISS was administered in either isiXhosa or English by lay counsellors currently working at the clinic (n = 3 at Langa and n = 1 at Crossroads). They all received the same training by the study psychiatrist (KS) in four sessions of 2 hours each over 3 weeks. We used the same SAMISS version used in previous reliability study conducted at Langa which compared the inter-rater reliability between counsellors and mental health nurse [28]: the 16-Item SAMISS with questions that were identical to those used in Pence et al. (2005). However, due to the availability of different versions, the multiple choice answers to the alcohol and substance abuse questions were from the version used in Whetten et al. (2005). The SAMISS was translated into Xhosa using both forward and backward translation. We did not establish the inter-rater reliability between the four counsellors in this study. The lay counsellors were blinded to the results of the nurse administered interview and the nurse only viewed the results of the SAMISS once the MINI had been completed. Following the conclusion of the interview, participants who had screened positive for a current MINI defined mental disorder either received counselling by the study mental health nurse or were referred to the HIV clinician or a psychiatrist for further treatment. Data analysis was conducted using STATA version 12 [29]. Means and medians were calculated for continuous variables and compared using t-tests and Wilcoxon rank sum tests. Categorical variables were summarised using frequencies and compared using Chi squared statistics. The performance of the lay counsellor administered SAMISS was examined by calculating the sensitivity and specificity and their respective confidence intervals and conducting a non-parametric receiver operating characteristic curve analysis. As the SAMISS reports symptoms in the last 12 months and the MINI reports both current (past 2 weeks) and past depression, we compared the SAMISS to both past and current depression. Where data was missing we calculated the statistics using participants with complete data relevant to the test. The study was approved by
1135
the Human Research Ethics Committee of the Health Sciences Faculty of the University of Cape Town.
Results We recruited 366 participants into the study with complete data for both MINI and SAMISS on 356 participants. The majority of participants were isiXhosa speaking, female, had secondary education or higher and had been diagnosed with HIV a median of 1 year previously. Socio-demographic information is summarised in Table 1. The prevalence of MINI defined mental illness in the sample was 30 % (Table 2). The most common diagnoses were alcohol use disorders and past mood disorders. There was a significant difference in the prevalence of MINI defined disorder according to site. The prevalence was 13 and 23 % in Langa and Crossroads respectively for alcohol and substance use and 12 and 23 % for mental illness (p \ 0.001). Having a MINI-defined disorder was also associated with older age, male gender, lack of employment and living in one’s own/family house compared to a shack, backyard dwelling or not having a home (Table 1). The sensitivity of the SAMISS was 94 % (95 % CI: 88–98 %) and the specificity was 58 % (95 % CI: 52–65 %) with an area under the ROC curve of 0.76 (95 % CI: 0.75–0.80 %). The sensitivity and specificity were higher in the alcohol and substance abuse component of the SAMISS at 94 % (95 % CI: 85–98 %) and 85 % (95 % CI: 81–89 %) respectively (Table 3). This did not change significantly between genders, age groups, levels of education or employment (Table 4). It did change for the site of assessment with the specificity being much lower at Crossroads than Langa- 29 % (95 % CI: 21–38 %) compared to 81.4 % (95 % CI: 75–89 %). In order to determine whether we could increase the specificity of the mental illness component of the test, we conducted a ROC analysis (Figure 1; Table 5). When we raised the cut-off from 1 (yes to any of the mental illness questions) to 2 (yes to two or more of the mental illness questions) the specificity of the mental illness component increased from 60 % (95 % CI: 54–65 %) to 78 % (95 % CI: 73–83 %) with only a moderate decrease in sensitivity from 97 % (95 % CI: 88–100 %) to 85 % (95 % CI: 73–93 %). The percentage correctly classified increased from 66 % to 80 %. For the overall SAMISS the sensitivity with the new cut-off was 88 % (95 % CI: 81–94 %) and specificity 72 % (95 % CI: 66–77 %). The positive and negative predictive values increased to 58 % (95 % CI: 50–66 %) and 93 % (95 % CI: 89–96 %) respectively and the percentage correctly classified increased from 64 to 77 %.
123
1136 Table 1 Sociodemographic characteristics of sample according to MINI diagnosis
AIDS Behav (2014) 18:1133–1141
Variable
P-valuea
Mini negative (n = 256)
Mini positive (n = 110)
Total (n = 356)
32.1 (± 7.3)
34.0 (± 10.1)
32.7 (± 8.2)
0.05*
2 (IQR 0–3)
1 (IQR 1–4)
1 (IQR 0–3)
0.17**
Age (mean, SD) Years since diagnosis Gender \0.01
Female
187 (76.0 %)
67 (60.9 %)
254
Male
59 (24.0 %)
43 (39.1 %)
102
Langa
156 (63.4 %)
45 (40.9 %)
201
Crossroads
90 (36.6 %)
65 (59.1 %)
155
0.00
240 (97.5 %) 6 (2.5 %)
104 (94.5 %) 6 (5.5 %)
344 12
0.15
Primary or less
32 (13.0 %)
23 (20.9 %)
55
0.06
Secondary or tertiary
214 (87.0 %)
87 (79.1 %)
301
No
240 (98 %)
110 (100 %)
350
Yes
5 (2.0 %)
0 (0.0 %)
5
Site
Home language Xhosa Other Level of education
Disability Grant 0.33:
Employment status Unemployed
134 (54.7 %)
76 (69.1 %)
210
Fulltime employment
111 (45.3 %)
34 (30.9 %)
145
0.01 0.34:
Marital status Married
56 (22.7 %)
22 (20 %)
78
Divorced
4 (1.6 %)
3 (2.7 %)
7
Widowed
3 (1.2 %)
5 (4.5 %)
8
Single Common law marriage
177 (71.9 %) 6 (2.4 %)
78 (70.9 %) 2 (1.8 %)
255 8
Shack, backyard dwelling or none
144 (58.4 %)
52 (47.3 %)
196
Own/family house
102 (41.5 %)
58 (52.7 %)
160
Current abode
0.05
0.50:
Access to clinic Walk \30 min
145 (59.2 %)
61 (55.5 %)
206
Walk [30 min
57 (23.3 %)
24 (22.8 %)
81
Public transport
41 (16.7 %)
25 (18.6 %)
66
Own transport
2 (0.8 %)
0 (0.6 %)
2
Medical illness
83 (34.2 %)
41 (37.6 %)
124
VCT
80 (32.9 %)
47 (43.1 %)
127
Partner tested positive
11 (4.5 %)
2 (1.8 %)
13
Pregnancy
65 (26.7 %)
16 (14.7 %)
81
Other
4 (1.6 %)
3 (2.8 %)
7
Reason for HIV testing
a
Chi squared test
:
Fisher’s exact test where expected \ 5 * t test, ** Wilcoxon rank sum test
Discussion In this study we examined the diagnostic validity of the SAMISS as a substance abuse and mental illness screening tool in primary health care settings in Cape Town, South Africa. The tool demonstrates a high sensitivity and
123
0.06
moderate specificity against the MINI as the gold standard. In addition, the negative predictive value was high, while the positive predictive value was fair, suggesting that few cases would be missed during screening. The alcohol use component performed better than the mental illness component of the tool which improved when the cut-off was
AIDS Behav (2014) 18:1133–1141 Table 2 Percentage of sample with a MINI defined disorder by gender
1137
MINI defined disorder
Male (n = 105)
Female (n = 261)
Mini positive n = 356
Alcohol use
34 (32.4 %)
29 (11.1 %)
63 (17.2 %)
Substance use
1 (1.0 %)
0
1 (0.3 %)
Total substance and alcohol use
35 (33.3 %)
29 (11.1 %)
64 (17.5 %)
Current mood disorder
1 (1.0 %)
11 (4.2 %)
12 (3.3 %)
Past mood disorder
10 (9.5 %)
24 (9.2 %)
34 (9.3 %)
Anxiety disorder
2 (1.9 %)
8 (3.1 %)
10 (2.7 %)
Adjustment disorder
1 (1.0 %)
9 (3.5 %)
10 (2.8 %)
Bipolar disorder
0
0
0
Total Mental Illness
34 (13.0 %)
34 (13 %)
59 (16.6 %)
Total MINI positive
43 (42.2 %)
67 (26.4 %)
110 (30.1 %)
Table 3 Test characteristics of the SAMISS SAMISS module
Sensitivity (95 % CI)
Specificity (95 % CI)
Positive predictive value
Negative predictive value
Positive likelihood ratio
Negative likelihood ratio
AUROC
Alcohol and Substance Use *
93.8 % (84.8–98.3 %)
85.3 % (80.28–89.1 %)
57.7 % (47.6–67.3 %)
98.5 (96.1–99.6 %)
6.4 (4.8–8.5)
0.07 (0.03–0.19)
0.90 (0.90–0.93)
Mental illness**
96.6 % (88.1–99.6 %)
59.9 % (54.1–65.6 %)
32.4 % (25.5 %– 39.9 %)
98.9 % (96.0–99.9 %)
2.41 (2.08–2.8)
0.06 (0.01–0.23)
0.78 (0.75–0.82)
Combined
94.4 % (88.2–97.9 %)
58.3 % (51.8–64.5 %)
50.0 % (42.9–57.1 %)
95.9 % (91.3–98.5 %)
2.26 (1.94–2.64)
0.10 (0.04–0.21)
0.76 (0.75–0.80)
* Positive on MINI for substance dependence, alcohol abuse/dependence ** Positive on MINI current major depression, past major depression, bipolar disorder, adjustment disorder, PTSD, panic disorder, GAD
Positive on MINI for any mental illness or substance use disorder
altered. The prevalence of MINI defined mental illness was consistent with similar studies. The high sensitivity of the SAMISS suggests that the SAMISS can be used with confidence as screening tool for mental illness and alcohol use in pre-CART patients in primary health care. This is similar to the results reported by Pence et al. [27] in a study at the University of North Carolina at Chapel Hill Hospital who reported a sensitivity and specificity of 86 and 75 % for substance and alcohol use component and 95 and 49 % for the mental illness component respectively. The SAMISS appears to perform differently as a screener across its substance abuse and mental illness components. We noted a sensitivity and specificity of 93.8 and 85.3 % respectively for the alcohol and 96.6 and 59.9 % for the mental illness component. This difference was noted previously by our group when inter-rater reliability of the SAMISS between lay counsellor and mental health nurses was established, the reliability for the alcohol component was higher [1]. Similarly, Myer et al. [8] validated both the Center for Epidemiological Studies Depression Scale (CESD) and the Alcohol Use Disorders Identification Test (AUDIT) in a comparable setting and
found that the sensitivity and specificity of the AUDIT was higher than the CESD. This reflects the blurred boundaries of normal mood variation in CMD compared to the more objective assessment of numbers and types of drinks assessed in screening for alcohol use. Increasing the cut-off from 1 to 2 for the mental illness component of the SAMISS improved the sensitivity and specificity. Therefore, we suggest that the cut-off be increased to 2, particularly where second tier screening and referral is unavailable. Overall, we found high rates of MINI-defined substance abuse and mental disorder. These approximate other studies in South Africa and other countries in sub-Saharan Africa [8]. The most common disorders were alcohol use (17 %) and past and current mood disorder (9 and 3 % respectively). Alcohol use has been linked with HIV risk behaviour in southern Africa indicating that people who exhibit risky drinking may be at higher risk of becoming infected with HIV [30]. The prevalence of CMD was higher amongst women and alcohol use was higher among men. This is consistent with other studies in this context in both HIV positive and negative populations [31]. The low prevalence of MINI diagnosed adjustment disorder (1 %) which is similar to other studies [32] is interesting,
123
1138
AIDS Behav (2014) 18:1133–1141
Table 4 Test Characteristics of the SAMISS by Sociodemographic Variables Characteristic
Substance and alcohol Use
Mental illness
Prevalence 95 % confidence interval)
Sensitivity (95 % confidence interval)
Specificity (95 % confidence interval)
Prevalence (95 % confidence interval)
Sensitivity (95 % confidence interval)
Specificity (95 % confidence interval)
Male
33.0 % (24.0–43.2 %)
94.3 % (80.8–99.3 %)
74.3 % (62.4–84 %)
13.0 % (7.0–21.0 %)
92.3 % (64.0–99.8 %)
58 % (47.0–68.4 %)
Female
11.0 % (7.6–15.7 %)
93.1 % (77.2–99.2 %)
88.7 % (83.9–92.5 %)
18.0 % (13.0–23.4 %)
97.8 % (88.2–99.9 %)
60.8 % (53.7–67.5 %)
Langa
13.0 % (9.1–18.9 %)
85.7 % (67.3–96.0 %)
96.7 % (92.9–98.8 %)
12.0 % (7.6–17.2 %)
95.7 % (78.1–99.9 %)
81.4 % (74.8–86.9 %)
Crossroads
23.0 % (16.0–30.0 %)
100.0 % (90.0–100.0 %)
68.3 % (59.2–76.5 %)
23.0 % (16.0–30.2 %)
97.1 % (85.1–99.9 %)
28.6 % (20.7–37.6 %)
16–29
12.0 % (7.4–19.1 %)
88.2 % (63.6–98.5 %)
84.2 % (76.4–90.2 %)
16 % (83.9–100 %)
100 % (83.9–100 %)
55 % (45.2–64.4 %)
30–39
15.0 % (9.6–22.2 %)
100 % (83.9–100 %)
89 % (81.9–94 %)
16 % (10–23 %)
90.5 % (69.6–98.8 %)
66.4 % (56.9–75 %)
[40
34.0 % (22.0–47.4 %)
90 % (68.3–98.8 %)
79.5 % (63.5–90.7 %)
19 % (9.9–31.4 %)
100 % (71.5–100 %)
53.2 % (38.1–67.9 %)
Primary or less
27.0 % (16.0–41.0 %)
100 % (78.2–100 %)
80 % (64.4–90.9 %)
18 % (9.1–30.9 %)
100 % (69.2–100 %)
46.7 % (31.7–62.1 %)
Secondary or tertiary
16.0 % (12.0–20.4 %)
91.8 % (80.4–97.7 %)
86.2 % (81.3–90.1 %)
16 % (12.0–21.0 %)
95.8 % (85.7–99.5 %)
62.3 % (56.0–68.4 %)
Unemployed
21.5 % (15.0–26.7 %)
95.5 % (84.5–99.4 %)
82.8 % (76.3–88.2 %)
19 % (14.0–25.3 %)
95 % (83.1–99.4 %)
57.1 % (49.3–64.7 %)
Fulltime employment
13.0 % (8.3–19.8 %)
90 % (68.3–98.8 %)
88.5 % (81.7–93.4 %)
13 % (7.7–19.4 %)
100 % (81.5–100.0 %)
63.4 % (54.3–71.9 %)
Gender
Site
Age
Level of education
1.00
Employment status
Sensitivity (%)
Specificity (%)
Correctly classified (%)
LR?
LR-
(C0)
100.0
0.0
16.6
1.00
(C1)
96.6
59.9
66.0
2.41
0.06
84.5
79.5
80.3
4.11
0.20
0.25
0.50
Cut-point
(C2) (C3)
55.2
92.1
86.0
7.00
0.49
0.00
Sensitivity
0.75
Table 5 Sensitivity, specificity, positive and negative likelihood ratios (LR) for the mental illness component of the SAMISS
(C4) (C5)
36.2 20.7
95.6 97.3
85.7 84.6
8.13 7.55
0.67 0.82
0.00
0.25
0.50
0.75
1.00
1 - Specificity Area under ROC curve = 0.8788
Fig. 1 Receiver operating curve for the mental illness component of the SAMISS
123
(C6)
15.5
99.0
85.1
15.10
0.85
(C7)
8.6
99.7
84.6
25.17
0.92
(C8)
1.7
100.0
83.7
0.98
([8)
0.0
100.0
83.4
1.00
AIDS Behav (2014) 18:1133–1141
particularly as 28 % of participants screened positive for the adjustment disorder item on the SAMISS. We noted that the prevalence of MINI defined disorder differed between the two primary care clinics by two-fold. This could be an artefact or due to either clinic, patient or study characteristics. Although the referral mechanism was the same in both study sites, we did not ensure that all or a random sub-sample of pre-CART patients were referred by nurses into the study. Therefore referral mechanisms in the clinic may have influenced these findings. There were some differences in the patient profiles between Langa and Crossroads clinics. In post hoc bivariate analysis, patients at Cross-roads were significantly older, unemployed, living further from the clinic, were more likely to live in their own home and had a lower level of education than patients in Langa. Many of these variables were associated with an increased prevalence of MINI defined disorder in this study as well as the literature [33]. The use of the SAMISS is not without challenges, including case over-detection. The modest positive predictive value in this study indicated that cases were overdetected by the SAMISS with only a third of patients who screened positive on the SAMISS having a MINI defined diagnosis. This was particularly marked in the mental illness component of the tool. We postulate that, despite rigorous initial training of counsellors, drift in their technique might have occurred for several reasons. Firstly, symptoms of mental disorders such as depression do not always have direct isiXhosa equivalents which may result in counsellors refining or further clarifying symptoms after they have first read them. Secondly, counsellors are a unique group of healthcare providers who have limited training, often limited supervision, and no professional registration at present, who revert to basic and often didactic approaches [34]. There was some variation in the performance of the tool between Crossroads and Langa. The specificity of the tool was lower at Crossroads which may be due to the increased experience of the lay counsellors at Langa who participated in our previous study [28]. The differences in the performance of the tool in two quite similar primary HIV clinics suggest caution in generalising these data to other settings. It remains to be determined who the ideal administrators of the SAMISS would be. This would depend on the availability of mental health nurses or other cadres of mental health workers in primary care. Over-detection may have substantial implications following the introduction of routine screening tools in resource constrained settings by substantially increasing referrals for further assessment and treatment [21]. This could be mitigated by providing adequate training and supervision to counsellors [35] as well
1139
as increasing the cut-off of the mental illness component of the SAMISS to 2 or more, particularly when administered by lay counsellors. A limitation of this study was the use of two different versions of the MINI as the gold-standard. However, internal validity of the modules should be unaffected by the use of two versions as the individual items and skip patterns remain unchanged. We did find a trend towards increased sensitivity and decreased specificity of the SAMISS when we compared the MINI-6 to the MINI. Due to the difference in specificity between study sites it is difficult to determine the true effect of this, particularly as we did not establish the inter-reliability between counsellors. Another difficulty was the difference in the time periods between the SAMISS and the MINI. The SAMISS records symptoms in the last 12 months whereas the MINI records more recent symptoms. Therefore we combined past and current depression diagnoses on the MINI which may have inflated the reported sensitivity of the SAMISS. As the MINI only provides diagnosis at a single point in time, a longitudinal, expert, all data procedure for psychiatric diagnosis would have provided a better gold standard than either of the two versions of the MINI for comparison with the SAMISS [36].
Conclusion As the call for task-shifting increases to meet the demands of mental health services in low and middle income countries, the role of lay counsellors is likely to increase to encompass screening, identification, treatment and follow up for mental illness [35]. This study demonstrates that the SAMISS can be used by lay HIV counsellors to screen for alcohol, substance use and mental illness in pre-CART patients in busy primary care HIV clinics in South Africa. The validity of the SAMISS may vary according to the administration of the tool and the patient profiles. Using a higher cut-off and providing second-tier screening may be useful in mitigating the problem of over-detection. Acknowledgments We would like to thank Dr Karen Jennings and the Health Management Team of the City of Cape Town for their support for this study. In particular, we would like to thank Sr Yoliswa Mtingeni, Mrs Kareema Poggenpoel, the counsellors and staff at Langa and Crossroads CHC and the study participants for their contributions to this study. The work conducted in collecting data for this paper was funded by the US President’s Emergency Plan for AIDS Relief (PEPFAR), through USAID under the terms of Award no. 674-A-00-0800009-00 to the Anova Health Institute. EB is funded by a grant from UK aid from the UK Government. DS is funded by the Medical Research Council of South Africa. The opinions expressed in this article are not necessarily those of the funders.
123
1140
AIDS Behav (2014) 18:1133–1141
References 1. Breuer E, Myer L, Struthers H, Joska JA. HIV/AIDS and mental health research in sub-Saharan Africa: a systematic review. Afr J AIDS Res. 2011;10(2):101–22. 2. Nakimuli-Mpungu E, Bass JK, Alexandre P, Mills EJ, Musisi S, Ram M, et al. Depression, alcohol use and adherence to antiretroviral therapy in sub-Saharan Africa: a systematic review. AIDS Behav. 2012;16(8):2101–18 Epub 2011/11/26. 3. Bing EG, Burnam MA, Longshore D, Fleishman JA, Sherbourne CD, London AS, et al. Psychiatric disorders and drug use among human immunodeficiency virus-infected adults in the United States. Arch Gen Psychiatry. 2001;58:721–8. 4. UNAIDS. Regional Fact Sheet 2012:Sub-Saharan Africa. 2012. http://www.unaids.org/en/media/unaids/contentassets/documents/epi demiology/2012/gr2012/2012_FS_regional_ssa_en.pdf. Accessed 7th July 2013. 5. Freeman M, Nkomo N, Kafaar Z, Kelly K. Mental disorder in people living with HIV/AIDS in South Africa. S Afr J Psychol. 2008;38(3):489–500. 6. Thom R. Common mental disorders in people living with HIV/ AIDS. S Afr J HIV Med. 2009;35:8–13. 7. Olley BO, Seedat S, Nei DG, Stein DJ. Predictors of major depression in recently diagnosed patients with HIV/AIDS in South Africa. AIDS Patient Care STDS. 2004;18:481–7. 8. Myer L, Smit J, Roux LL, Parker S, Stein DJ, Seedat S. Common mental disorders among HIV-infected individuals in South Africa: prevalence, predictors, and validation of brief psychiatric rating scales. AIDS Patient Care STDS. 2008;22:147–58. 9. Dube´ B, Benton T, Cruess DG, Evans DL. Examen critique Neuropsychiatric manifestations of HIV infection and AIDS. J Psychiatry Neurosci. 2005;30:237–46. 10. Simbayi LC, Kalichman S, Strebel A, Cloete A, Henda N, Mqeketo A. Internalized stigma, discrimination, and depression among men and women living with HIV/AIDS in Cape Town. S Afr Soc Sci Med. 2007;64(9):1823–31 Epub 2007/03/06. 11. Mayston R, Kinyanda E, Chishinga N, Prince M, Patel V. Mental disorder and the outcome of HIV/AIDS in low-income and middleincome countries: a systematic review. AIDS. 2012;26(Suppl 2): S117–35 Epub 2013/01/18. 12. Fairfield KM, Libman H, Davis RB, Eisenberg DM. Delays in protease inhibitor use in clinical practice. J Gen Intern Med. 1999;14(7):395–401 Epub 1999/07/27. 13. Ironson G, Balbin E, Stuetzle R, Fletcher MA, O’Cleirigh C, Laurenceau JP, et al. Dispositional optimism and the mechanisms by which it predicts slower disease progression in HIV: proactive behavior, avoidant coping, and depression. Int J Behav Med. 2005;12(2):86–97 Epub 2005/05/20. 14. Ickovics JR, Hamburger ME, Vlahov D, Schoenbaum EE, Schuman P, Boland RJ, et al. Mortality, CD4 cell count decline, and depressive symptoms among HIV-seropositive women: longitudinal analysis from the HIV Epidemiology Research Study. JAMA, J Am Med Assoc. 2001;285:1466–74. 15. Antelman G, Kaaya S, Wei R, Mbwambo J, Msamanga GI, Fawzi WW, et al. Depressive symptoms increase risk of HIV disease progression and mortality among women in Tanzania. J Acquir Immune Defic Syndr. 2007;44(4):470–7. 16. Yun LW, Maravi M, Kobayashi JS, Barton PL, Davidson AJ. Antidepressant treatment improves adherence to antiretroviral therapy among depressed HIV-infected patients. J Acquir Immune Defic Syndr. 2005;38(4):432–8 Epub 2005/03/15. 17. Tegger MK, Crane HM, Tapia KA, Uldall KK, Holte SE, Kitahata MM. The effect of mental illness, substance use, and treatment for depression on the initiation of highly active
123
18.
19.
20.
21.
22.
23.
24.
25.
26.
27.
28.
29. 30.
31.
32.
33.
antiretroviral therapy among HIV-infected individuals. AIDS Patient Care STDS. 2008;22(3):233–43 Epub 2008/02/23. Shisana O, Rehle T, Simbayi LC, Zuma K, Jooste S, Pillay-vanWyk S, et al. South African national HIV prevalence, incidence, behaviour and communication survey, 2008 a turning tide among teenagers?. Cape Town: HSRC; 2008. Chopra M, Lawn JE, Sanders D, Barron P, Karim SSA, Bradshaw D, et al. Achieving the health Millennium Development Goals for South Africa: challenges and priorities. Lancet. 2009;374(9694): 1023–31. Lund C, Kleintjes S, Kakuma R, Flisher AJ. Public sector mental health systems in South Africa: inter-provincial comparisons and policy implications. Soc Psychiatry Psychiatr Epidemiol. 2010; 45:393–404. Kagee A, Tsai AC, Lund C, Tomlinson M. Screening for common mental disorders in low resource settings: reasons for caution and a way forward. Int Health. 2013;5(1):11–4. Akena D, Joska J, Obuku EA, Amos T, Musisi S, Stein DJ. Comparing the accuracy of brief versus long depression screening instruments which have been validated in low and middle income countries: a systematic review. BMC Psychiatry. 2012;12:187 Epub 2012/11/03. Spies G, Kader K, Kidd M, Smit J, Myer L, Stein DJ, et al. Validity of the K-10 in detecting DSM-IV-defined depression and anxiety disorders among HIV-infected individuals. AIDS care. 2009;21(9):1163–8 Epub 2009/12/22. Kaaya SF, Fawzi M, Mbwambo J, Lee B, Msamanga G, Fawzi W. Validity of the Hopkins Symptom Checklist-25 amongst HIVpositive pregnant women in Tanzania. Acta Psychiatr Scand. 2002;106(1):9–19. Chibanda D, Mangezi W, Tshimanga M, Woelk G, Rusakaniko S, Stranix-Chibanda L, et al. Postnatal depression by HIV status among women in Zimbabwe. J Women’s Health. 2010;19(11): 2071–7 Epub 2010/09/21. Whetten K, Reif S, Swartz M, Stevens R, Ostermann J, Hanisch L, et al. A brief mental health and substance abuse screener for persons with HIV. AIDS patient care STDS. 2005;19:89–99. Pence BW, Gaynes BN, Whetten K, Eron JJ, Ryder RW, Miller WC. Validation of a brief screening instrument for substance abuse and mental illness in HIV-positive patients. J Acquir Immune Defic Syndr. 1999;2005(40):434–44. Breuer E, Stoloff K, Myer L, Seedat S, Stein DJ, Joska J. Reliability of the lay adherence counsellor administered substance abuse and mental illness symptoms screener (SAMISS) and the International HIV Dementia Scale (IHDS) in a primary care HIV clinic in Cape Town. S Afr AIDS behav. 2012;16(6):1464–71 Epub 2011/10/18. Statcorp. STATA. 12th ed. College Station: Statcorp; 2011. Kalichman SC, Simbayi LC, Kaufman M, Cain D, Jooste S. Alcohol use and sexual risks for HIV/AIDS in sub-Saharan Africa: systematic review of empirical findings. Prev sci. 2007;8(2):141–51 Epub 2007/02/01. Simbayi LC, Kalichman SC, Jooste S, Mathirti V, Cain D, Cherry C. Alcohol use and sexual risks for HIV infection among men and women receiving sexually transmitted infection clinic services in Cape Town, South Africa. J Stud Alcohol. 2004;65(4):434–42. Olley BO, Gxamza F, Seedat S, Theron H, Stein DJ, Taljaard J, et al. Psychopathology and coping in recently diagnosed HIV/ AIDS patients—The role of gender. S Afr J Psychiatr. 2004; 10(1):21–4. Lund C, Breen A, Flisher AJ, Kakuma R, Corrigall J, Joska JA, et al. Poverty and common mental disorders in low and middle income countries: a systematic review. Soc Sci Med. 2010;71(3): 517–28 Epub 2010/07/14.
AIDS Behav (2014) 18:1133–1141 34. Dewing S, Mathews C, Schaay N, Cloete A, Louw J, Simbayi L. ‘‘It’s important to take your medication everyday okay?’’ An evaluation of counselling by lay counsellors for ARV adherence support in the Western Cape. S Afr AIDS Behav. 2013;17(1): 203–12 Epub 2012/05/23. 35. Kakuma R, Minas H, van Ginneken N, Dal Poz MR, Desiraju K, Morris JE. Human resources for mental health care: current
1141 situation and strategies for action. Lancet. 2011;378(9803): 1654–63 Epub 2011/10/20. 36. Kranzler HR, Tennen H, Babor TF, Kadden RM, Rounsaville BJ. Validity of the longitudinal, expert, all data procedure for psychiatric diagnosis in patients with psychoactive substance use disorders. Drug Alcohol Depend. 1997;45(1):93–104.
123
ORIGINAL PAPER
The Validity of the Substance Abuse and Mental Illness Symptom Screener (SAMISS) in People Living with HIV/AIDS in Primary HIV Care in Cape Town, South Africa Erica Breuer • Kevin Stoloff • Landon Myer • Soraya Seedat • Dan J. Stein • John A. Joska
Published online: 23 January 2014 Ó Springer Science+Business Media New York 2014
Abstract Given the high prevalence of HIV in South Africa and co-morbid mental disorders in people living with HIV/AIDs (PLWHA) we sought to validate a brief screening tool in primary HIV care. Methods: 366 PLWHA were recruited prior to combination anti-retroviral treatment (CART) initiation from two primary health HIV clinics. A mental health nurse administered a sociodemographic questionnaire and the Mini Neuropsychiatric Interview (MINI) and a lay counsellor administered the Substance and Mental Illness Symptom Screener (SAMISS). Results: Using the MINI, 17 % of participants were identified with either depression, anxiety disorders or adjustment disorder and 18 % with substance or alcohol abuse/dependence. The sensitivity and specificity of the SAMISS was 94 % (95 % CI: 88–98 %) and 58 % (95 % CI: 52–65 %) respectively, with the alcohol component (sensitivity: 94 %; specificity: 85 %) performing better
E. Breuer (&) K. Stoloff J. A. Joska Alan J. Flisher Centre for Public Mental Health, Department of Psychiatry and Mental Health, University of Cape Town, Building B, 46 Sawkins Rd, Rondebosch, Cape Town 7700, South Africa e-mail: [email protected] L. Myer D. J. Stein Division of Epidemiology & Biostatistics, School of Public Health and Family Medicine, University of Cape Town, Cape Town, South Africa S. Seedat Department of Psychiatry, University of Stellenbosch, Stellenbosch, South Africa S. Seedat Medical Research Council of South Africa, Cape Town, South Africa
than the mental illness component of the SAMISS (sensitivity: 97 %; specificity: 60 %). The specificity of the tool improved when the cut-off for the mental illness component was increased. Conclusion: The SAMISS may provide a useful first tier screening tool for common mental disorders in primary care for PLWHA. Keywords HIV/AIDS Mental illness Screening sub-Saharan Africa
Introduction The prevalence of common mental disorders (CMD) in people living with HIV/AIDS (PLWHA) is high [1] [2]. In the United States, around half of PLWHA, screen positive for a mental illness [3]. Similar results are found in subSaharan Africa [2], where both HIV prevalence and incidence remains high. UNAIDS estimate that around 23.5 million PLWHA live in sub-Saharan Africa with 1.8 million people infected per year [4]. In South Africa, a nationally representative study of PLWHA attending public health facilities found that 44 % had a mental illness according to the WHO Composite International Diagnostic Interview [5]. This corresponds with other South African studies which report the prevalence of mental illness in PLWHA attending HIV clinics between 19 and 56 % [6–8]. Among the most prevalent CMD are depression, anxiety, substance abuse and posttraumatic stress disorder [1]. This increased prevalence compared with general population or health facility attendees is likely due to the combination of biological, psychological and social factors, such as the neurological effects of HIV on the brain [9] and HIV-associated stigma [10].
123
1134
Mental disorders in PLWHA lead to poorer HIV outcomes. A recent systematic review by Mayston and colleagues reported that PLWHA with mental disorders in low and middle income countries have poorer adherence to CART [11]. In addition, PLWHA with concomitant mental illness have delayed CART initiation [12], poorer immunological outcomes [13] and higher morbidity and mortality [14, 15]. However, antidepressant treatment has been found to improve adherence to antiretroviral therapy [16] and initiation [17]. Therefore the identification and treatment of mental illness in PLWHA may be an important way of increasing adherence, and by extension, healthoutcomes, particularly in settings with a high prevalence of HIV. Given these poorer health outcomes and the high estimated prevalence of HIV (10.6 %) [18], there is a need to identify patients with co-morbid mental illness in order to improve both mental health and HIV outcomes. However, given the large burden of care facing health professionals working in South Africa [19] and the paucity of mental health professionals in primary care settings [20], the addition of lengthy or complex screening protocols are not feasible. Good specificity of the screening tool is important as over-detection can lead to additional financial and human resource burden on the health system, which is particularly important to avoid in already resource constrained settings such as South Africa [21]. Therefore we sought to find a short reliable and valid screening tool with good sensitivity and specificity which could be used at the point of initiation of CART to screen for mental illness. As lay counsellors in South Africa currently have a key role in preparing patients for CART initiation as well as providing basic psychosocial counselling and monitoring adherence, they are ideally placed to administer a screening tool within a primary care setting. Various screening tools have been validated in PLWHA in sub-Saharan Africa [22]. These include the Centre for Epidemiological Studies Depression Scale, the Alcohol Use Disorders Identification Test [8], the Kessler Psychological Distress Scale-10 [23] the Hopkins Symptom Checklist-25 (HSCL-25) in antenatal women [24] and the Edinburgh Postnatal Depression Scale in postnatal women [25]. However, despite moderate to high sensitivity and specificity, these screen only either for alcohol and substance abuse or for depression/anxiety and would require the administration of multiple screening tools in order to provide coverage for the range of CMD which affect PLWHA. The Substance Use and Mental Illness Symptom Screener (SAMISS) was developed in the United States to screen for alcohol, substance use and common mental illness specifically in PLWHA. There have been several versions of this tool published: a 13 item tool [26] and a 16
123
AIDS Behav (2014) 18:1133–1141
item tool [27]. The 16 item tool contains items from the Alcohol Use Disorders Identification Test, the Two-Item Conjoint Screener and the Composite Diagnostic Interview Screening tool as well as some designed specifically for the SAMISS. This version was validated by Pence and colleagues against the Structured Clinical Interview for DSMIV at the University of North Carolina at Chapel Hill Hospital [27]. The sensitivity and the specificity of the alcohol and substance use component of the SAMISS was 86 and 75 % respectively and 95 and 49 % sensitivity and specificity for the mental illness component. In a previous study in the same setting, we compared mental health nurse and lay counsellor administration of the SAMISS. The reliability of lay counsellor administration was fair (j = 0.39, 95 % CI 0.29–0.49, P \ 0.01) with a higher proportion of PLWHA screening positive when lay counsellors administered the tool compared to nurses [28]. However, as the validity of the SAMISS had not been established, we could not determine whether counsellors were over detecting cases of mental illness. This study aimed to determine the validity of a lay counsellor administered SAMISS compared to the Mini International Neuropsychiatric Interview in a pre-HAART PLWHA in two primary care HIV clinics in South Africa.
Methods The study was a cross-sectional validity study performed at two primary health care clinics: Langa and Crossroads Community Health Clinics in Cape Town, South Africa. We included PLWHA who were CART naı¨ve and excluded those that had been previously assessed with the mental health screening tools at the clinic in order to minimise bias due to practice effects. A sample size of 360 subjects was estimated by calculating the width of confidence intervals for the sensitivity and specificity using values from Pence et al.’s [27] validation study. Potential participants were referred by nursing staff for inclusion. Following screening for suitability and written informed consent, a mental health nurse administered a socio-demographic questionnaire. Variables such as age, sex, level of education, employment status, marital status, current abode, access to clinic and home language were obtained. Participants were then randomized using a predefined list based on a random number table to either having the Substance and Mental Illness Symptom Screener (SAMISS) administered first followed by the Mini International Neuropsychiatric Interview (MINI), or vice versa. The order was randomised to reduce any bias introduced by administering one questionnaire before the other.
AIDS Behav (2014) 18:1133–1141
The MINI was administered by a mental health nurse trained in its administration. The training for counsellors in both clinics consisted of four sessions of one hour, with ongoing feedback and consultation with the study psychiatrist (KS) throughout the project. The MINI was administered in English, however, there were some cases where the questions were clarified in isiXhosa, the first language of the majority of participants. Initially the MINI Plus was used but this was changed due to programmatic reasons to the MINI Six with the addition of the adjustment disorder module from the MINI Plus. Only the depression, anxiety, adjustment disorder, bipolar disorder and substance and alcohol use components of the MINI were administered to align with the disorders screened for on the SAMISS. The SAMISS was administered in either isiXhosa or English by lay counsellors currently working at the clinic (n = 3 at Langa and n = 1 at Crossroads). They all received the same training by the study psychiatrist (KS) in four sessions of 2 hours each over 3 weeks. We used the same SAMISS version used in previous reliability study conducted at Langa which compared the inter-rater reliability between counsellors and mental health nurse [28]: the 16-Item SAMISS with questions that were identical to those used in Pence et al. (2005). However, due to the availability of different versions, the multiple choice answers to the alcohol and substance abuse questions were from the version used in Whetten et al. (2005). The SAMISS was translated into Xhosa using both forward and backward translation. We did not establish the inter-rater reliability between the four counsellors in this study. The lay counsellors were blinded to the results of the nurse administered interview and the nurse only viewed the results of the SAMISS once the MINI had been completed. Following the conclusion of the interview, participants who had screened positive for a current MINI defined mental disorder either received counselling by the study mental health nurse or were referred to the HIV clinician or a psychiatrist for further treatment. Data analysis was conducted using STATA version 12 [29]. Means and medians were calculated for continuous variables and compared using t-tests and Wilcoxon rank sum tests. Categorical variables were summarised using frequencies and compared using Chi squared statistics. The performance of the lay counsellor administered SAMISS was examined by calculating the sensitivity and specificity and their respective confidence intervals and conducting a non-parametric receiver operating characteristic curve analysis. As the SAMISS reports symptoms in the last 12 months and the MINI reports both current (past 2 weeks) and past depression, we compared the SAMISS to both past and current depression. Where data was missing we calculated the statistics using participants with complete data relevant to the test. The study was approved by
1135
the Human Research Ethics Committee of the Health Sciences Faculty of the University of Cape Town.
Results We recruited 366 participants into the study with complete data for both MINI and SAMISS on 356 participants. The majority of participants were isiXhosa speaking, female, had secondary education or higher and had been diagnosed with HIV a median of 1 year previously. Socio-demographic information is summarised in Table 1. The prevalence of MINI defined mental illness in the sample was 30 % (Table 2). The most common diagnoses were alcohol use disorders and past mood disorders. There was a significant difference in the prevalence of MINI defined disorder according to site. The prevalence was 13 and 23 % in Langa and Crossroads respectively for alcohol and substance use and 12 and 23 % for mental illness (p \ 0.001). Having a MINI-defined disorder was also associated with older age, male gender, lack of employment and living in one’s own/family house compared to a shack, backyard dwelling or not having a home (Table 1). The sensitivity of the SAMISS was 94 % (95 % CI: 88–98 %) and the specificity was 58 % (95 % CI: 52–65 %) with an area under the ROC curve of 0.76 (95 % CI: 0.75–0.80 %). The sensitivity and specificity were higher in the alcohol and substance abuse component of the SAMISS at 94 % (95 % CI: 85–98 %) and 85 % (95 % CI: 81–89 %) respectively (Table 3). This did not change significantly between genders, age groups, levels of education or employment (Table 4). It did change for the site of assessment with the specificity being much lower at Crossroads than Langa- 29 % (95 % CI: 21–38 %) compared to 81.4 % (95 % CI: 75–89 %). In order to determine whether we could increase the specificity of the mental illness component of the test, we conducted a ROC analysis (Figure 1; Table 5). When we raised the cut-off from 1 (yes to any of the mental illness questions) to 2 (yes to two or more of the mental illness questions) the specificity of the mental illness component increased from 60 % (95 % CI: 54–65 %) to 78 % (95 % CI: 73–83 %) with only a moderate decrease in sensitivity from 97 % (95 % CI: 88–100 %) to 85 % (95 % CI: 73–93 %). The percentage correctly classified increased from 66 % to 80 %. For the overall SAMISS the sensitivity with the new cut-off was 88 % (95 % CI: 81–94 %) and specificity 72 % (95 % CI: 66–77 %). The positive and negative predictive values increased to 58 % (95 % CI: 50–66 %) and 93 % (95 % CI: 89–96 %) respectively and the percentage correctly classified increased from 64 to 77 %.
123
1136 Table 1 Sociodemographic characteristics of sample according to MINI diagnosis
AIDS Behav (2014) 18:1133–1141
Variable
P-valuea
Mini negative (n = 256)
Mini positive (n = 110)
Total (n = 356)
32.1 (± 7.3)
34.0 (± 10.1)
32.7 (± 8.2)
0.05*
2 (IQR 0–3)
1 (IQR 1–4)
1 (IQR 0–3)
0.17**
Age (mean, SD) Years since diagnosis Gender \0.01
Female
187 (76.0 %)
67 (60.9 %)
254
Male
59 (24.0 %)
43 (39.1 %)
102
Langa
156 (63.4 %)
45 (40.9 %)
201
Crossroads
90 (36.6 %)
65 (59.1 %)
155
0.00
240 (97.5 %) 6 (2.5 %)
104 (94.5 %) 6 (5.5 %)
344 12
0.15
Primary or less
32 (13.0 %)
23 (20.9 %)
55
0.06
Secondary or tertiary
214 (87.0 %)
87 (79.1 %)
301
No
240 (98 %)
110 (100 %)
350
Yes
5 (2.0 %)
0 (0.0 %)
5
Site
Home language Xhosa Other Level of education
Disability Grant 0.33:
Employment status Unemployed
134 (54.7 %)
76 (69.1 %)
210
Fulltime employment
111 (45.3 %)
34 (30.9 %)
145
0.01 0.34:
Marital status Married
56 (22.7 %)
22 (20 %)
78
Divorced
4 (1.6 %)
3 (2.7 %)
7
Widowed
3 (1.2 %)
5 (4.5 %)
8
Single Common law marriage
177 (71.9 %) 6 (2.4 %)
78 (70.9 %) 2 (1.8 %)
255 8
Shack, backyard dwelling or none
144 (58.4 %)
52 (47.3 %)
196
Own/family house
102 (41.5 %)
58 (52.7 %)
160
Current abode
0.05
0.50:
Access to clinic Walk \30 min
145 (59.2 %)
61 (55.5 %)
206
Walk [30 min
57 (23.3 %)
24 (22.8 %)
81
Public transport
41 (16.7 %)
25 (18.6 %)
66
Own transport
2 (0.8 %)
0 (0.6 %)
2
Medical illness
83 (34.2 %)
41 (37.6 %)
124
VCT
80 (32.9 %)
47 (43.1 %)
127
Partner tested positive
11 (4.5 %)
2 (1.8 %)
13
Pregnancy
65 (26.7 %)
16 (14.7 %)
81
Other
4 (1.6 %)
3 (2.8 %)
7
Reason for HIV testing
a
Chi squared test
:
Fisher’s exact test where expected \ 5 * t test, ** Wilcoxon rank sum test
Discussion In this study we examined the diagnostic validity of the SAMISS as a substance abuse and mental illness screening tool in primary health care settings in Cape Town, South Africa. The tool demonstrates a high sensitivity and
123
0.06
moderate specificity against the MINI as the gold standard. In addition, the negative predictive value was high, while the positive predictive value was fair, suggesting that few cases would be missed during screening. The alcohol use component performed better than the mental illness component of the tool which improved when the cut-off was
AIDS Behav (2014) 18:1133–1141 Table 2 Percentage of sample with a MINI defined disorder by gender
1137
MINI defined disorder
Male (n = 105)
Female (n = 261)
Mini positive n = 356
Alcohol use
34 (32.4 %)
29 (11.1 %)
63 (17.2 %)
Substance use
1 (1.0 %)
0
1 (0.3 %)
Total substance and alcohol use
35 (33.3 %)
29 (11.1 %)
64 (17.5 %)
Current mood disorder
1 (1.0 %)
11 (4.2 %)
12 (3.3 %)
Past mood disorder
10 (9.5 %)
24 (9.2 %)
34 (9.3 %)
Anxiety disorder
2 (1.9 %)
8 (3.1 %)
10 (2.7 %)
Adjustment disorder
1 (1.0 %)
9 (3.5 %)
10 (2.8 %)
Bipolar disorder
0
0
0
Total Mental Illness
34 (13.0 %)
34 (13 %)
59 (16.6 %)
Total MINI positive
43 (42.2 %)
67 (26.4 %)
110 (30.1 %)
Table 3 Test characteristics of the SAMISS SAMISS module
Sensitivity (95 % CI)
Specificity (95 % CI)
Positive predictive value
Negative predictive value
Positive likelihood ratio
Negative likelihood ratio
AUROC
Alcohol and Substance Use *
93.8 % (84.8–98.3 %)
85.3 % (80.28–89.1 %)
57.7 % (47.6–67.3 %)
98.5 (96.1–99.6 %)
6.4 (4.8–8.5)
0.07 (0.03–0.19)
0.90 (0.90–0.93)
Mental illness**
96.6 % (88.1–99.6 %)
59.9 % (54.1–65.6 %)
32.4 % (25.5 %– 39.9 %)
98.9 % (96.0–99.9 %)
2.41 (2.08–2.8)
0.06 (0.01–0.23)
0.78 (0.75–0.82)
Combined
94.4 % (88.2–97.9 %)
58.3 % (51.8–64.5 %)
50.0 % (42.9–57.1 %)
95.9 % (91.3–98.5 %)
2.26 (1.94–2.64)
0.10 (0.04–0.21)
0.76 (0.75–0.80)
* Positive on MINI for substance dependence, alcohol abuse/dependence ** Positive on MINI current major depression, past major depression, bipolar disorder, adjustment disorder, PTSD, panic disorder, GAD
Positive on MINI for any mental illness or substance use disorder
altered. The prevalence of MINI defined mental illness was consistent with similar studies. The high sensitivity of the SAMISS suggests that the SAMISS can be used with confidence as screening tool for mental illness and alcohol use in pre-CART patients in primary health care. This is similar to the results reported by Pence et al. [27] in a study at the University of North Carolina at Chapel Hill Hospital who reported a sensitivity and specificity of 86 and 75 % for substance and alcohol use component and 95 and 49 % for the mental illness component respectively. The SAMISS appears to perform differently as a screener across its substance abuse and mental illness components. We noted a sensitivity and specificity of 93.8 and 85.3 % respectively for the alcohol and 96.6 and 59.9 % for the mental illness component. This difference was noted previously by our group when inter-rater reliability of the SAMISS between lay counsellor and mental health nurses was established, the reliability for the alcohol component was higher [1]. Similarly, Myer et al. [8] validated both the Center for Epidemiological Studies Depression Scale (CESD) and the Alcohol Use Disorders Identification Test (AUDIT) in a comparable setting and
found that the sensitivity and specificity of the AUDIT was higher than the CESD. This reflects the blurred boundaries of normal mood variation in CMD compared to the more objective assessment of numbers and types of drinks assessed in screening for alcohol use. Increasing the cut-off from 1 to 2 for the mental illness component of the SAMISS improved the sensitivity and specificity. Therefore, we suggest that the cut-off be increased to 2, particularly where second tier screening and referral is unavailable. Overall, we found high rates of MINI-defined substance abuse and mental disorder. These approximate other studies in South Africa and other countries in sub-Saharan Africa [8]. The most common disorders were alcohol use (17 %) and past and current mood disorder (9 and 3 % respectively). Alcohol use has been linked with HIV risk behaviour in southern Africa indicating that people who exhibit risky drinking may be at higher risk of becoming infected with HIV [30]. The prevalence of CMD was higher amongst women and alcohol use was higher among men. This is consistent with other studies in this context in both HIV positive and negative populations [31]. The low prevalence of MINI diagnosed adjustment disorder (1 %) which is similar to other studies [32] is interesting,
123
1138
AIDS Behav (2014) 18:1133–1141
Table 4 Test Characteristics of the SAMISS by Sociodemographic Variables Characteristic
Substance and alcohol Use
Mental illness
Prevalence 95 % confidence interval)
Sensitivity (95 % confidence interval)
Specificity (95 % confidence interval)
Prevalence (95 % confidence interval)
Sensitivity (95 % confidence interval)
Specificity (95 % confidence interval)
Male
33.0 % (24.0–43.2 %)
94.3 % (80.8–99.3 %)
74.3 % (62.4–84 %)
13.0 % (7.0–21.0 %)
92.3 % (64.0–99.8 %)
58 % (47.0–68.4 %)
Female
11.0 % (7.6–15.7 %)
93.1 % (77.2–99.2 %)
88.7 % (83.9–92.5 %)
18.0 % (13.0–23.4 %)
97.8 % (88.2–99.9 %)
60.8 % (53.7–67.5 %)
Langa
13.0 % (9.1–18.9 %)
85.7 % (67.3–96.0 %)
96.7 % (92.9–98.8 %)
12.0 % (7.6–17.2 %)
95.7 % (78.1–99.9 %)
81.4 % (74.8–86.9 %)
Crossroads
23.0 % (16.0–30.0 %)
100.0 % (90.0–100.0 %)
68.3 % (59.2–76.5 %)
23.0 % (16.0–30.2 %)
97.1 % (85.1–99.9 %)
28.6 % (20.7–37.6 %)
16–29
12.0 % (7.4–19.1 %)
88.2 % (63.6–98.5 %)
84.2 % (76.4–90.2 %)
16 % (83.9–100 %)
100 % (83.9–100 %)
55 % (45.2–64.4 %)
30–39
15.0 % (9.6–22.2 %)
100 % (83.9–100 %)
89 % (81.9–94 %)
16 % (10–23 %)
90.5 % (69.6–98.8 %)
66.4 % (56.9–75 %)
[40
34.0 % (22.0–47.4 %)
90 % (68.3–98.8 %)
79.5 % (63.5–90.7 %)
19 % (9.9–31.4 %)
100 % (71.5–100 %)
53.2 % (38.1–67.9 %)
Primary or less
27.0 % (16.0–41.0 %)
100 % (78.2–100 %)
80 % (64.4–90.9 %)
18 % (9.1–30.9 %)
100 % (69.2–100 %)
46.7 % (31.7–62.1 %)
Secondary or tertiary
16.0 % (12.0–20.4 %)
91.8 % (80.4–97.7 %)
86.2 % (81.3–90.1 %)
16 % (12.0–21.0 %)
95.8 % (85.7–99.5 %)
62.3 % (56.0–68.4 %)
Unemployed
21.5 % (15.0–26.7 %)
95.5 % (84.5–99.4 %)
82.8 % (76.3–88.2 %)
19 % (14.0–25.3 %)
95 % (83.1–99.4 %)
57.1 % (49.3–64.7 %)
Fulltime employment
13.0 % (8.3–19.8 %)
90 % (68.3–98.8 %)
88.5 % (81.7–93.4 %)
13 % (7.7–19.4 %)
100 % (81.5–100.0 %)
63.4 % (54.3–71.9 %)
Gender
Site
Age
Level of education
1.00
Employment status
Sensitivity (%)
Specificity (%)
Correctly classified (%)
LR?
LR-
(C0)
100.0
0.0
16.6
1.00
(C1)
96.6
59.9
66.0
2.41
0.06
84.5
79.5
80.3
4.11
0.20
0.25
0.50
Cut-point
(C2) (C3)
55.2
92.1
86.0
7.00
0.49
0.00
Sensitivity
0.75
Table 5 Sensitivity, specificity, positive and negative likelihood ratios (LR) for the mental illness component of the SAMISS
(C4) (C5)
36.2 20.7
95.6 97.3
85.7 84.6
8.13 7.55
0.67 0.82
0.00
0.25
0.50
0.75
1.00
1 - Specificity Area under ROC curve = 0.8788
Fig. 1 Receiver operating curve for the mental illness component of the SAMISS
123
(C6)
15.5
99.0
85.1
15.10
0.85
(C7)
8.6
99.7
84.6
25.17
0.92
(C8)
1.7
100.0
83.7
0.98
([8)
0.0
100.0
83.4
1.00
AIDS Behav (2014) 18:1133–1141
particularly as 28 % of participants screened positive for the adjustment disorder item on the SAMISS. We noted that the prevalence of MINI defined disorder differed between the two primary care clinics by two-fold. This could be an artefact or due to either clinic, patient or study characteristics. Although the referral mechanism was the same in both study sites, we did not ensure that all or a random sub-sample of pre-CART patients were referred by nurses into the study. Therefore referral mechanisms in the clinic may have influenced these findings. There were some differences in the patient profiles between Langa and Crossroads clinics. In post hoc bivariate analysis, patients at Cross-roads were significantly older, unemployed, living further from the clinic, were more likely to live in their own home and had a lower level of education than patients in Langa. Many of these variables were associated with an increased prevalence of MINI defined disorder in this study as well as the literature [33]. The use of the SAMISS is not without challenges, including case over-detection. The modest positive predictive value in this study indicated that cases were overdetected by the SAMISS with only a third of patients who screened positive on the SAMISS having a MINI defined diagnosis. This was particularly marked in the mental illness component of the tool. We postulate that, despite rigorous initial training of counsellors, drift in their technique might have occurred for several reasons. Firstly, symptoms of mental disorders such as depression do not always have direct isiXhosa equivalents which may result in counsellors refining or further clarifying symptoms after they have first read them. Secondly, counsellors are a unique group of healthcare providers who have limited training, often limited supervision, and no professional registration at present, who revert to basic and often didactic approaches [34]. There was some variation in the performance of the tool between Crossroads and Langa. The specificity of the tool was lower at Crossroads which may be due to the increased experience of the lay counsellors at Langa who participated in our previous study [28]. The differences in the performance of the tool in two quite similar primary HIV clinics suggest caution in generalising these data to other settings. It remains to be determined who the ideal administrators of the SAMISS would be. This would depend on the availability of mental health nurses or other cadres of mental health workers in primary care. Over-detection may have substantial implications following the introduction of routine screening tools in resource constrained settings by substantially increasing referrals for further assessment and treatment [21]. This could be mitigated by providing adequate training and supervision to counsellors [35] as well
1139
as increasing the cut-off of the mental illness component of the SAMISS to 2 or more, particularly when administered by lay counsellors. A limitation of this study was the use of two different versions of the MINI as the gold-standard. However, internal validity of the modules should be unaffected by the use of two versions as the individual items and skip patterns remain unchanged. We did find a trend towards increased sensitivity and decreased specificity of the SAMISS when we compared the MINI-6 to the MINI. Due to the difference in specificity between study sites it is difficult to determine the true effect of this, particularly as we did not establish the inter-reliability between counsellors. Another difficulty was the difference in the time periods between the SAMISS and the MINI. The SAMISS records symptoms in the last 12 months whereas the MINI records more recent symptoms. Therefore we combined past and current depression diagnoses on the MINI which may have inflated the reported sensitivity of the SAMISS. As the MINI only provides diagnosis at a single point in time, a longitudinal, expert, all data procedure for psychiatric diagnosis would have provided a better gold standard than either of the two versions of the MINI for comparison with the SAMISS [36].
Conclusion As the call for task-shifting increases to meet the demands of mental health services in low and middle income countries, the role of lay counsellors is likely to increase to encompass screening, identification, treatment and follow up for mental illness [35]. This study demonstrates that the SAMISS can be used by lay HIV counsellors to screen for alcohol, substance use and mental illness in pre-CART patients in busy primary care HIV clinics in South Africa. The validity of the SAMISS may vary according to the administration of the tool and the patient profiles. Using a higher cut-off and providing second-tier screening may be useful in mitigating the problem of over-detection. Acknowledgments We would like to thank Dr Karen Jennings and the Health Management Team of the City of Cape Town for their support for this study. In particular, we would like to thank Sr Yoliswa Mtingeni, Mrs Kareema Poggenpoel, the counsellors and staff at Langa and Crossroads CHC and the study participants for their contributions to this study. The work conducted in collecting data for this paper was funded by the US President’s Emergency Plan for AIDS Relief (PEPFAR), through USAID under the terms of Award no. 674-A-00-0800009-00 to the Anova Health Institute. EB is funded by a grant from UK aid from the UK Government. DS is funded by the Medical Research Council of South Africa. The opinions expressed in this article are not necessarily those of the funders.
123
1140
AIDS Behav (2014) 18:1133–1141
References 1. Breuer E, Myer L, Struthers H, Joska JA. HIV/AIDS and mental health research in sub-Saharan Africa: a systematic review. Afr J AIDS Res. 2011;10(2):101–22. 2. Nakimuli-Mpungu E, Bass JK, Alexandre P, Mills EJ, Musisi S, Ram M, et al. Depression, alcohol use and adherence to antiretroviral therapy in sub-Saharan Africa: a systematic review. AIDS Behav. 2012;16(8):2101–18 Epub 2011/11/26. 3. Bing EG, Burnam MA, Longshore D, Fleishman JA, Sherbourne CD, London AS, et al. Psychiatric disorders and drug use among human immunodeficiency virus-infected adults in the United States. Arch Gen Psychiatry. 2001;58:721–8. 4. UNAIDS. Regional Fact Sheet 2012:Sub-Saharan Africa. 2012. http://www.unaids.org/en/media/unaids/contentassets/documents/epi demiology/2012/gr2012/2012_FS_regional_ssa_en.pdf. Accessed 7th July 2013. 5. Freeman M, Nkomo N, Kafaar Z, Kelly K. Mental disorder in people living with HIV/AIDS in South Africa. S Afr J Psychol. 2008;38(3):489–500. 6. Thom R. Common mental disorders in people living with HIV/ AIDS. S Afr J HIV Med. 2009;35:8–13. 7. Olley BO, Seedat S, Nei DG, Stein DJ. Predictors of major depression in recently diagnosed patients with HIV/AIDS in South Africa. AIDS Patient Care STDS. 2004;18:481–7. 8. Myer L, Smit J, Roux LL, Parker S, Stein DJ, Seedat S. Common mental disorders among HIV-infected individuals in South Africa: prevalence, predictors, and validation of brief psychiatric rating scales. AIDS Patient Care STDS. 2008;22:147–58. 9. Dube´ B, Benton T, Cruess DG, Evans DL. Examen critique Neuropsychiatric manifestations of HIV infection and AIDS. J Psychiatry Neurosci. 2005;30:237–46. 10. Simbayi LC, Kalichman S, Strebel A, Cloete A, Henda N, Mqeketo A. Internalized stigma, discrimination, and depression among men and women living with HIV/AIDS in Cape Town. S Afr Soc Sci Med. 2007;64(9):1823–31 Epub 2007/03/06. 11. Mayston R, Kinyanda E, Chishinga N, Prince M, Patel V. Mental disorder and the outcome of HIV/AIDS in low-income and middleincome countries: a systematic review. AIDS. 2012;26(Suppl 2): S117–35 Epub 2013/01/18. 12. Fairfield KM, Libman H, Davis RB, Eisenberg DM. Delays in protease inhibitor use in clinical practice. J Gen Intern Med. 1999;14(7):395–401 Epub 1999/07/27. 13. Ironson G, Balbin E, Stuetzle R, Fletcher MA, O’Cleirigh C, Laurenceau JP, et al. Dispositional optimism and the mechanisms by which it predicts slower disease progression in HIV: proactive behavior, avoidant coping, and depression. Int J Behav Med. 2005;12(2):86–97 Epub 2005/05/20. 14. Ickovics JR, Hamburger ME, Vlahov D, Schoenbaum EE, Schuman P, Boland RJ, et al. Mortality, CD4 cell count decline, and depressive symptoms among HIV-seropositive women: longitudinal analysis from the HIV Epidemiology Research Study. JAMA, J Am Med Assoc. 2001;285:1466–74. 15. Antelman G, Kaaya S, Wei R, Mbwambo J, Msamanga GI, Fawzi WW, et al. Depressive symptoms increase risk of HIV disease progression and mortality among women in Tanzania. J Acquir Immune Defic Syndr. 2007;44(4):470–7. 16. Yun LW, Maravi M, Kobayashi JS, Barton PL, Davidson AJ. Antidepressant treatment improves adherence to antiretroviral therapy among depressed HIV-infected patients. J Acquir Immune Defic Syndr. 2005;38(4):432–8 Epub 2005/03/15. 17. Tegger MK, Crane HM, Tapia KA, Uldall KK, Holte SE, Kitahata MM. The effect of mental illness, substance use, and treatment for depression on the initiation of highly active
123
18.
19.
20.
21.
22.
23.
24.
25.
26.
27.
28.
29. 30.
31.
32.
33.
antiretroviral therapy among HIV-infected individuals. AIDS Patient Care STDS. 2008;22(3):233–43 Epub 2008/02/23. Shisana O, Rehle T, Simbayi LC, Zuma K, Jooste S, Pillay-vanWyk S, et al. South African national HIV prevalence, incidence, behaviour and communication survey, 2008 a turning tide among teenagers?. Cape Town: HSRC; 2008. Chopra M, Lawn JE, Sanders D, Barron P, Karim SSA, Bradshaw D, et al. Achieving the health Millennium Development Goals for South Africa: challenges and priorities. Lancet. 2009;374(9694): 1023–31. Lund C, Kleintjes S, Kakuma R, Flisher AJ. Public sector mental health systems in South Africa: inter-provincial comparisons and policy implications. Soc Psychiatry Psychiatr Epidemiol. 2010; 45:393–404. Kagee A, Tsai AC, Lund C, Tomlinson M. Screening for common mental disorders in low resource settings: reasons for caution and a way forward. Int Health. 2013;5(1):11–4. Akena D, Joska J, Obuku EA, Amos T, Musisi S, Stein DJ. Comparing the accuracy of brief versus long depression screening instruments which have been validated in low and middle income countries: a systematic review. BMC Psychiatry. 2012;12:187 Epub 2012/11/03. Spies G, Kader K, Kidd M, Smit J, Myer L, Stein DJ, et al. Validity of the K-10 in detecting DSM-IV-defined depression and anxiety disorders among HIV-infected individuals. AIDS care. 2009;21(9):1163–8 Epub 2009/12/22. Kaaya SF, Fawzi M, Mbwambo J, Lee B, Msamanga G, Fawzi W. Validity of the Hopkins Symptom Checklist-25 amongst HIVpositive pregnant women in Tanzania. Acta Psychiatr Scand. 2002;106(1):9–19. Chibanda D, Mangezi W, Tshimanga M, Woelk G, Rusakaniko S, Stranix-Chibanda L, et al. Postnatal depression by HIV status among women in Zimbabwe. J Women’s Health. 2010;19(11): 2071–7 Epub 2010/09/21. Whetten K, Reif S, Swartz M, Stevens R, Ostermann J, Hanisch L, et al. A brief mental health and substance abuse screener for persons with HIV. AIDS patient care STDS. 2005;19:89–99. Pence BW, Gaynes BN, Whetten K, Eron JJ, Ryder RW, Miller WC. Validation of a brief screening instrument for substance abuse and mental illness in HIV-positive patients. J Acquir Immune Defic Syndr. 1999;2005(40):434–44. Breuer E, Stoloff K, Myer L, Seedat S, Stein DJ, Joska J. Reliability of the lay adherence counsellor administered substance abuse and mental illness symptoms screener (SAMISS) and the International HIV Dementia Scale (IHDS) in a primary care HIV clinic in Cape Town. S Afr AIDS behav. 2012;16(6):1464–71 Epub 2011/10/18. Statcorp. STATA. 12th ed. College Station: Statcorp; 2011. Kalichman SC, Simbayi LC, Kaufman M, Cain D, Jooste S. Alcohol use and sexual risks for HIV/AIDS in sub-Saharan Africa: systematic review of empirical findings. Prev sci. 2007;8(2):141–51 Epub 2007/02/01. Simbayi LC, Kalichman SC, Jooste S, Mathirti V, Cain D, Cherry C. Alcohol use and sexual risks for HIV infection among men and women receiving sexually transmitted infection clinic services in Cape Town, South Africa. J Stud Alcohol. 2004;65(4):434–42. Olley BO, Gxamza F, Seedat S, Theron H, Stein DJ, Taljaard J, et al. Psychopathology and coping in recently diagnosed HIV/ AIDS patients—The role of gender. S Afr J Psychiatr. 2004; 10(1):21–4. Lund C, Breen A, Flisher AJ, Kakuma R, Corrigall J, Joska JA, et al. Poverty and common mental disorders in low and middle income countries: a systematic review. Soc Sci Med. 2010;71(3): 517–28 Epub 2010/07/14.
AIDS Behav (2014) 18:1133–1141 34. Dewing S, Mathews C, Schaay N, Cloete A, Louw J, Simbayi L. ‘‘It’s important to take your medication everyday okay?’’ An evaluation of counselling by lay counsellors for ARV adherence support in the Western Cape. S Afr AIDS Behav. 2013;17(1): 203–12 Epub 2012/05/23. 35. Kakuma R, Minas H, van Ginneken N, Dal Poz MR, Desiraju K, Morris JE. Human resources for mental health care: current
1141 situation and strategies for action. Lancet. 2011;378(9803): 1654–63 Epub 2011/10/20. 36. Kranzler HR, Tennen H, Babor TF, Kadden RM, Rounsaville BJ. Validity of the longitudinal, expert, all data procedure for psychiatric diagnosis in patients with psychoactive substance use disorders. Drug Alcohol Depend. 1997;45(1):93–104.
123
