Validation of the World Health Organization Adult ADHD Self-Report Scale for adolescents KARIN SONNBY, KONSTANTINOS SKORDAS, SUSANNE OLOFSDOTTER, SOFIA VADLIN, KENT W. NILSSON, MIA RAMKLINT
Sonnby K, Skordas K, Olofsdotter S, Vadlin S, Nilsson KW, Ramklint M. Validation of the World Health Organization Adult ADHD Self-Report Scale for adolescents. Nord J Psychiatry 2015;69:216–223. Background: The World Health Organization Adult ADHD Self Report Scale (ASRS) is a widely used diagnostic tool for assessment of attention-deficit hyperactivity disorder (ADHD) symptoms in clinical psychiatry in Sweden. The ASRS consists of 18 questions, the first six of which can be used as a short screening version (ASRS-S). There is a version for adolescents— ASRS-Adolescent (ASRS-A)—and the corresponding screening version (ASRS-A-S), which has not been validated to date. Aim: The aim was to validate the ASRS-A and the ASRS-A-S for use in adolescent clinical populations. Methods: Adolescent psychiatric outpatients (n 134, mean age 15 years, 40% boys) reported on the ASRS-A, and were interviewed with the Kiddie Schedule of Affective Disorders and Schizophrenia (K-SADS), a semi-structured interview, together with a parent. Results: Internal consistency was 0.79 for the ASRS-A-S and 0.92 for the ASRS-A (Cronbach’s alpha). Internal consistency values were 0.79 and 0.87 for the inattention subscale, and 0.68 and 0.89 for the hyperactivity subscale, respectively. Concurrent validity values, measured with Spearman’s correlation coefficient, between the total K-SADS ADHD symptom severity score and the sum of ASRS-A-S and ASRS-A total scores were 0.51 and 0.60, respectively. Psychometric properties of the ASRS-A-S and the ASRS-A were: sensitivity 74% and 79%; negative predictive value 81% and 84%; specificity 59% and 60%; and positive predictive value 49% and 51%, respectively. Both versions showed better properties for girls than for boys. Conclusion: Both the ASRS-A-S and the ASRS-A showed promising psychometric properties for use in adolescent clinical populations. • Adolescent, Attention deficit disorder with hyperactivity, Validation study. Karin Sonnby, M.D., Ph.D., Centre for Clinical Research, Uppsala University, Västmanlands County Hospital, SE 721 89 Västerås, Sweden, E-mail: [email protected]; Accepted 18 September 2014.
A
ttention-deficit hyperactivity disorder (ADHD) is characterized by inattention and/or hyperactivity– impulsivity and has, in adolescence, a lifetime prevalence in the range 4–9% (1, 2). Symptoms of ADHD have a negative impact on school performance (3, 4), social relationships (5) and physical health (6), and increase suicidal risk (7). Early identification and treatment improves prognosis, including occupational outcome (8). ADHD is also characterized by high comorbidity rates (2, 9). In clinical work, patients with comorbid ADHD may be help-seeking because of a comorbid disorder, for example due to depression. Girls are less helpseeking for ADHD but might suffer from it as a comorbid disorder (10). Therefore, there is a need for
© 2014 Informa Healthcare
questionnaires that assess ADHD symptoms and are suitable for screening clinical populations to identify affected adolescents, especially girls. One such questionnaire is the World Health Organization Adult ADHD Self-Report Scale (ASRS) (11). The ASRS has been developed for adults and consists of two subscales targeting inattentiveness and hyperactivity–impulsivity, respectively. The first six questions of the ASRS can be used alone as a short screening scale and are collectively called the ASRS-Screening Test (ASRS-S). In the USA, the ASRS and the ASRS-S have been validated against an ADHD diagnosis in a stratified, population-based sample (n 154, gender rate not reported) (11). The ASRS-S has also been validated among drug-dependent
DOI: 10.3109/08039488.2014.968203
VALIDITY STUDY OF
patients (n 102, 83% male) (12). Both studies indicated that ASRS and ASRS-S are valid and reliable instruments for the screening of ADHD in adults (11, 12). The Chinese translation of the ASRS was investigated in two large, population-based young adult samples (n 1031, 100% males; n 3298, 38% males) (13). These studies found that the ASRS had good internal consistency among the subscales, and moderate to high correlations between the subscales and the Wender Utah Rating Scale (13). This supported the ASRS as a reliable and valid instrument for ADHD screening in adults (13). In two independent samples consisting of 232 (41% male) university students and 157 (68% male) drug-dependent outpatients, Hesse found that the ASRS-S measured the two constructs of inattention and hyperactivity independently, although the two domains were correlated with each other (14). The ASRS has also been studied in an adolescent sample of ADHD patients (n 88, 76% male) (15). Test–retest reliability and concurrent validity of the ASRS were examined in comparison with another clinician-administered ADHD rating scale. The results showed good test–retest reliability, high internal consistency and high concurrent validity (15). However, the study did not explore sensitivity, specificity, positive predictive value (PPV) or negative predictive value (NPV), as non-ADHD cases were not part of the study population (15). The Standards for Reporting of Diagnostic Accuracy (STARD) are international guidelines for investigators conducting diagnostic accuracy studies (16). No studies have used the Swedish translation of the ASRS in accordance with STARD (17), either in an adult or an adolescent population. Nevertheless, the Swedish translation of the ASRS is often used for the screening of ADHD in both adult and adolescent populations (18). In Sweden, versions of the ASRS, the ASRS-Adolescent (ASRS-A) and the ASRS-A-Screener (ASRS-A-S) are used among adolescents. However, there are no reports of the diagnostic accuracy of these versions. There is thus a need to examine the diagnostic accuracy of the Swedish version of the ASRS-A-S/ASRS-A.
Aims Aims of the study were to evaluate the psychometric properties and diagnostic accuracy of the Swedish translation of the ASRS-A and the shorter screening version (ASRS-A-S) in a group of adolescent psychiatric patients, including separate analyses for boys and girls.
Materials and methods Study design This study was a cross-sectional, diagnostic accuracy study. NORD J PSYCHIATRY·VOL 69 NO 3·2015
THE
ASRS
FOR ADOLESCENTS FOR CLINICAL USE
Participants Consecutively referred psychiatric outpatients aged 12–17 years old, from three Swedish cites (Enköping, Västerås and Sala) were invited to participate. Patients in need of acute psychiatric or somatic inpatient care at the time of the diagnostic interview, and those who required a Swedish interpreter, were excluded. The data collection took place from September 2011 to June 2013. The Västerås and Sala clinics are both part of the Child and Adolescent Psychiatric Clinic of the county of Västmanland. The Västerås clinic is situated in an area with 130,000 inhabitants, and the Sala clinic is situated in an area with 20,000 inhabitants. In Västerås and Sala, 433 adolescents were referred during the study period. Of those, 307 were invited and 157 (51%) agreed to participate, and of those 114 had full information for the analyses. The Enköping clinic is part of the Child and Adolescent Psychiatric Clinic of the county of Uppsala and is situated in an area with 40,000 inhabitants. One hundred and sixty-nine adolescents were referred to the Enköping clinic during the study period, of whom 55 were invited and 28 agreed to participate, and of those 20 had full information. In total from the three cites there were 134 patients with full information for the analyses. There were no significant differences between participants and internal drop-outs in sex or age. However, there was a difference between participants and external drop-outs related to sex, with a lower proportion of boys participating in the study (χ2 4.99, P 0.05). Participating boys were younger than non-participating boys (t 3.422, P 0.001), but there was no such difference in mean age between participating and non-participating girls (t 0.078, P 0.531).
Procedure Before the first appointment, the patients and their families were informed about the study. The first clinical assessment was a traditional non-structured psychiatric interview with the parent and adolescent together. At this appointment, the patient was invited to participate in the study. Only dyads of each one adolescent and a parent, who both agreed to participate, were included in the study. During the second visit, the parent and the adolescent were interviewed together using the K-SADS. In Enköping, the adolescents filled out the ASRS-A questionnaire immediately before completing the K-SADS interview. Responses on the ASRS-A were withheld from the K-SADS interviewer by placing the questionnaire inside an envelope. In Västerås and Sala, the ASRS-A was completed during the initial appointment. For 52 (39%) patients, the period between the initial appointment and the diagnostic interview was more than 7 days, which was considered the maximum time limit. In these cases, the ASRS-A questionnaire was completed again immediately after the K-SADS interview. The K-SADS
217
KARIN SONNBY ET AL
interviewers in Västerås and Sala were blind to the results of the ASRS-A.
Assessments THE ADULT ADHD SELF-REPORT SCALE ADOLESCENT VERSION (ASRS-A AND ASRS-A-S) The ASRS is a self-rating scale regarding ADHD in adults and is designed in accordance with DSM-IV criteria A for inattention and hyperactivity–impulsivity for ADHD. The questionnaire has 18 items and is composed of two subscales: inattention (IA) and hyperactivity–impulsivity (HD), with nine questions each. The response options are on a 0–4 Likert scale with the alternatives never, rarely, sometimes, often and very often. The total sum of the ASRS ranges from 0 to 72. The first six questions can be used alone as a short screening scale called the ASRS-S. The total sum of the ASRS-S ranges from 0 to 24. Kessler et al. (11) recommend a summation based on dichotomized cut-off points for all items, shown as shadowed areas on the questionnaire, and proposed a cutoff score of 9 for the ASRS (range 0–18), and 4 for the ASRS-S (range 0–6 points). In the original validation study where the dichotomized scoring method was used, the six-question ASRS-S screening test and the 18-question ASRS had sensitivity of 68.7% and 56.3%, and specificity of 99.5% and 98.3%, respectively. In a clinical, drug-dependent population, the ASRS-S had sensitivity of 66.7% and specificity of 81.2%, with a PPV of 41.8% and an NPV of 92.6% (12). The Swedish version of the ASRS was translated from English by Nyberg and back into English by Fernholm (18). Minor adaption of the wording to suit adolescents was made by one of the authors (KWN). KIDDIE-SCHEDULE OF AFFECTIVE DISORDERS AND SCHIZOPHRENIA (K-SADS) The K-SADS is a semi-structured diagnostic interview that examines current and previous psychopathology in children and adolescents between the ages of 6 and 18 years (19). The K-SADS consists of one screening part and eight supplements. If general symptoms are indicated in the screening interview, questions from the appropriate supplement are used to verify the diagnosis. The K-SADS is regarded as the gold standard for child psychiatric diagnoses (19–21). Conducting one K-SADS interview takes between 1.5 and 8 h. K-SADS ADHD SEVERITY SCORE The symptoms of ADHD in the K-SADS, according to DSM-IV criterion A, are rated on a scale ranging of 0–3. “Clinical significant symptoms” are coded as 3, “subclinical symptoms” as 2, “no symptoms” as 1 and “no information” as 0. A summation of the total ADHD symptom score in the K-SADS ranges between 0 and 57.
218
Training and reliability of interviewers All interviewers (n 6) were trained in the use of the K-SADS before they started the data collection. The interviewers consisted of a specialist in child and adolescent psychiatry, two residents in child and adolescent psychiatry, two clinical psychologists and a clinical social worker, all working in child psychiatric services. All interviewers completed their basic training in accordance with the Swedish standard for K-SADS training, with 4 days of theory and practice. The specialist in child and adolescent psychiatry (the first author) had received national training to train other interviewers. She planned the training for the others and was considered the standard when calculating inter-rater reliability. The training was based on viewing recorded interviews made by “master interviewers”. The ratings of these interviews were estimated individually and, then discussed among the group. Thereafter, the interviewers recorded themselves interviewing patients. The others assessed the recordings, and inter-rater reliability was calculated. Inter-rater reliability was determined before the study began, based on five consecutive interviews. It was calculated as both kappa and prevalenceand bias-adjusted kappa (PABAK) (22). First, inter-rater reliability was calculated for each interviewer for each diagnosis. The mean was then calculated for all diagnoses, and finally, the mean inter-rater reliability for the group was calculated. For the first five interviews before the study began, the mean inter-rater reliability for the group yielded a kappa of 0.84 (range 0.51–1.00) and a PABAK of 0.90 (range 0.77–1.00). For the diagnosis of ADHD, kappa was 0.83 (range 0.54–1.00) and PABAK was 0.84 (range 0.60–1.00). During the study, a joint assessment meeting took place each month to ensure continuous high levels of agreement. In total, 25 interviews were rated by the first author, and other interviewers each rated between five and 22 interviews. The mean ( standard deviation) number of interviews rated by the interviewers was 16 7.0. The overall inter-rater reliability, based on all ratings, yielded a kappa of 0.84 (range 0.73–1.00) and a PABAK of 0.91 (range 0.86–1.00). For the diagnosis of ADHD, kappa was 0.81 (range 0.70–1.00) and PABAK was 0.82 (range 0.70–1.00).
Statistical analysis Kappa statistics were used for agreement between interviewers (22, 23). Kappa is considered slight if kappa 0.2, fair if kappa is 0.21–0.40, moderate if kappa is 0.41– 0.60, substantial if kappa is 0.61–0.80 and almost perfect if kappa is 0.81–1.0. When the distribution of reported categories (the diagnoses) is unevenly distributed to a large extent, Cohen’s kappa is known to be less reliable (23, 24). Therefore, we also used the PABAK (23). Univariate analyses were performed to analyse differences between groups, by using t-tests for continuous NORD J PSYCHIATRY·VOL 69 NO 3·2015
VALIDITY STUDY OF
and ordinal data; however, they were validated by the non-parametric Mann–Whitney U test. Chi-square tests were used for dichotomous and categorical variables. The internal consistency of the ASRS-A-S and the ASRS-A was calculated with Cronbach’s alpha. Spearman’s rho was calculated for correlations between the total ASRS-A-S scores (0–24) and the total K-SADS ADHD symptom severity scores (0–57) as well as between the ASRS-A total scores (0–72) and the total K-SADS ADHD symptom severity scores (0–57). A dichotomous cut-off point was set for each item in accordance with Kessler et al.’s suggestions for ASRS-S and ASRS (11). The sums were used for receiver operating characteristic (ROC) analysis for both versions of the ASRS-A (25). The reference test, against which the ASRS-A-S and the ASRS-A were compared, was the K-SADS interview. All ADHD diagnoses, including ADHD (n 33), attention-deficit disorder (ADD; n 2), HD (n 2) and ADHD not otherwise specified (ADHD NOS; n 11), were pooled together as a reference diagnosis of ADHD (n 48). The original cut-offs were used for calculations of sensitivity, specificity, PPV and NPV as well as the odds ratio (OR) for an ADHD diagnosis. ROC analyses were used for area under the curve calculations (AUC) (25). Separate analyses were made for boys and girls. The study was approved by the Regional Ethics Committee of Uppsala University.
Results The mean age was 15 years among both boys and girls in the total sample (n 134). A little more than one
THE
ASRS
FOR ADOLESCENTS FOR CLINICAL USE
third of the sample had an ADHD diagnosis, with a larger proportion among boys than among girls. Comorbidity was present in a large majority of the patients, and one quarter had no comorbidity. The comorbidity rates were higher among girls than among boys, and this was more pronounced among individuals with ADHD. Among individuals with ADHD with four or more disorders, the proportion of girls was almost 10 times as large as among boys (Table 1). Not surprisingly, when comparing individuals with and without ADHD, the ADHD groups had higher K-SADS-ADHD severity score, as well as higher mean scores in the ASRS-A and ASRSA-S (Table 2).
Psychometric properties of the ASRS-A-S and the ASRS-A Internal consistency for the ASRS-A-S, measured by Cronbach’s alpha, was 0.79 (boys 0.81, girls 0.79). For the inattention subscale, internal consistency was 0.79 (boys 0.84, girls 0.76) and the hyperactivity subscale yielded a score of 0.68 (boys 0.71, girls 0.65). Internal consistency for the ASRS-A total scale, measured by Cronbach’s alpha, was 0.92 (boys 0.91, girls 0.92). For inattention, it was 0.87 (boys 0.88, girls 0.87) and for hyperactivity, it was 0.89 (boys 0.89, girls 0.89).
Concurrent validity of the ASRS-A-S and the ASRS-A Concurrent validity, measured by Spearman’s rho, for the correlation between the total K-SADS ADHD symptom severity score and the total sum of all ASRS-A-S ratings was 0.51 (boys 0.58, girls 0.47). Spearman’s rho for the
Table 1. Comparison between boys and girls among participating adolescent psychiatric patients (n134).
Mean age (s) No. of patients with ADHD, n (%) No. of disorders diagnosed, n (%) 0 1 2 3 4 5 6 8 Mean no. of disorders (s) Mean no. of diagnoses among patients with ADHD (s) No. of patients without ADHD with 4 disorders, n (%) No. of patients with ADHD with 4 disorders, n (%)
Boys, n 53 (39.6%)
Girls, n 81 (60.4%)
15.0 (1.5) 24 (45.3)
15.2 (1.5) 23 (28.4)
6 (11.3) 14 (26.4) 23 (43.4) 6 (11.3) 1 (1.9) 3 (5.7) 0 (0.0) 0 (0.0) 1.8 (1.2) 2.0 (0.9) 3 (10.3) 1 (4.2)
2 (2.5) 19 (23.5) 29 (35.8) 10 (12.3) 7 (8.6) 7 (8.6) 6 (7.4) 1 (1.2) 2.6 (1.7) 3.3 (2.0) 12 (20.7) 9 (39.1)
Statistics *t 0.650, P 0.517 χ2 4.012, P 0.045
**t 3.270, P 0.001 ***t 2.873, P 0.007
χ2 1.450, P 0.229 χ2 8.572, P 0.003
s, standard deviation; ADHD, attention-deficit hyperactivity disorder. The Mann–Whitney test showed significant differences in mean rank for boys and girls, *P 0.503, **P 0.008, ***P 0.023. NORD J PSYCHIATRY·VOL 69 NO 3·2015
219
KARIN SONNBY ET AL
Table 2. Comparison of reported attention-deficit hyperactivity disorder (ADHD) symptoms among adolescent psychiatric patients (n134) with (n47) and without (n87) ADHD according to an ADHD diagnosis with the Kiddie Schedule of Affective Disorders and Schizophrenia (K-SADS).
K-SADS ADHD symptom severity score (range 0–57), mean (s) ASRS-A mean number of assented dichotomized items (range 0–18), mean (s) ASRS-A mean total sum of all ratings (range 0–72 points), mean (s) ASRS-A-S mean number of assented dichotomized items (range 0–6), mean (s) ASRS-A-S mean total sum of all ratings (range 0–24 points), mean (s)
Diagnosis of ADHD
No diagnosis of ADHD
48 (7) 11 (4) 45 (13) 4.3 (1.4) 16 (4)
22 (18) 7 (4) 31 (12) 3.0 (1.6) 12 (5)
Statistics *t 11.684, P 0.001 *t 5.840, P 0.001 *t 6.167, P 0.001 *t 6.770, P 0.001 *t 4.725, P 0.001
s, standard deviation; ASRS-A, World Health Organization Adult ADHD Self Report Scale, version for adolescents; ASRS-A-S, the corresponding screening version of the ASRS-A. *The Mann–Whitney test showed significant differences in mean rank for patients with or without ADHD; P 0.001.
correlation between the total K-SADS ADHD symptom severity score and the total sum of all ASRS-A ratings was 0.60 (boys 0.64, girls 0.58).
Diagnostic accuracy of the ASRS-A-S For the total sample, AUC was 0.73, the 95% confidence interval (CI) 0.64–0.81, P 0.001), with a score of 0.68 for boys (95% CI 0.54–0.83, P 0.023) and a score of 0.76 for girls (95% CI 0.64–0.89, P 0.001) (Fig. 1). Sensitivity, specificity, PPV, NPV and the OR for an ADHD diagnosis are presented in Table 3.
Diagnostic accuracy of the ASRS-A For the total sample, AUC was 0.78 (95% CI 0.70–0.87, P 0.001), with a score of 0.77 for boys (95% CI 0.65– 0.90, P 0.001) and a score of 0.81 for girls (95% CI: 0.70–0.93, P 0.001) (Fig. 2). Sensitivity, specificity, PPV, NPV and the OR for an ADHD diagnosis are presented in Table 3.
In the total sample, 82 individuals filled out the ASRS-A questionnaire before the K-SADS interview and 52 filled out the questionnaire after the interview. There were no significant differences on mean score on the ASRS-A between those rated before (8.6 4.6) and those rated after the K-SADS (8.3 4.7; Student’s t-test, t 0.369; P 0.0713; P 0.737).
Discussion The Swedish translation of the ASRS adapted for adolescents, ASRS-A/ASRS-A-S, showed similar psychometric properties to the original English adult version, with high internal consistency and concurrent validity (11). However, the diagnostic accuracy differed with higher sensitivity and lower specificity. Another dissimilarity, compared with the original ASRS, was better psychometric properties for the total scale in comparison to the screening version. Both the ASRS-A-S and the ASRS-A had better psychometric properties for girls than for boys.
Fig. 1. Receiver operating characteristic (ROC) analysis for the screening version of the World Health Organization Adult ADHD Self Report Scale, version for adolescents (ASRS-A-S) with dichotomized answers (0–6 points) in a clinical adolescent sample (n 134; boys n 53).
220
NORD J PSYCHIATRY·VOL 69 NO 3·2015
VALIDITY STUDY OF
THE
ASRS
FOR ADOLESCENTS FOR CLINICAL USE
Table 3. Sensitivity, specificity, positive and negative predictive values and odds ratios for the ASRSA-S/ASRS-A compared with an attention-deficit hyperactivity disorder (ADHD) diagnosis in a clinical adolescent psychiatric sample (n134). ASRS-A-S (0–6 points, cut-off 4)
Sens.
Spec.
PPV
NPV
OR
95% CI
P
Total sample, n 134 Boys, n 53 Girls, n 81 ASRS-A-S rated before K-SADS, n 82 ASRS-A-S rated after K-SADS, n 52 ASRS-A (0–18 points, cut-off 9) Total sample, n 134 Boys, n 53 Girls, n 81 ASRS-A rated before K-SADS, n 82 ASRS-A rated after K-SADS, n 52
74% 71% 78% 75% 74%
59% 55% 60% 53% 69%
49% 57% 44% 40% 65%
81% 70% 88% 84% 77%
4.132 2.989 5.478 3.444 6.296
1.890–9.034 0.951–9.390 1.784–16.821 1.195–9.924 1.861–21.298
0.001 0.061 0.003 0.022 0.003
79% 75% 83% 79% 78%
60% 59% 60% 57% 66%
51% 60% 45% 43% 64%
84% 74% 90% 87% 79%
5.497 4.250 7.228 5.016 6.840
2.422–12.475 1.302–13.874 2.178–23.992 1.647–15.278 1.955–23.927
0.001 0.017 0.001 0.005 0.003
ASRS-A, World Health Organization Adult ADHD Self Report Scale, version for adolescents; ASRS-A-S) the corresponding screening version of the ASRS-A; PPV, positive predictive value; NPV, negative predictive value; OR, odds ratio; CI, confidence interval; K-SADS, Kiddie Schedule of Affective Disorders and Schizophrenia (K-SADS). Results are presented for boys and girls and for those who filled in the ASRS-A-S/ASRS-A before or after the K-SADS interview.
Moreover, whether or not the ratings of the ASRS-A and ASRS-A-S were performed before or after the K-SADSinterview, did not have any major impact on the diagnostic accuracy. Studies among adults, even when performed in different countries, have found consistent psychometric properties for the ASRS (13), which are similar to those found in this adolescent sample. However, diagnostic accuracy for the ASRS-A in this sample differed from that of previous studies. The only other study performed in the same age group (15) consisted exclusively of ADHD patients, with a preponderance of males, and adolescents perceived as unreliable reporters of their ADHD symptoms were excluded. Differences between participants, such as life stage (adolescents vs. adults)
and sex, may influence sensitivity and specificity because comorbidity patterns of individuals with ADHD might differ between groups. Comorbidity patterns may influence the extent of overlapping symptoms, as for instance attention problems are present in depression and anxiety disorders as well. The prevalence, which varies between population-based and clinical samples, influences PPV and NPV. Increasing prevalence increases PPV but decreases NPV. Therefore, the difference in diagnostic accuracy between this and previous studies is probably explained by differences in the studied populations. The results of the present study should be interpreted in the context of several limitations. The external dropout rate was high. Severely disordered patients might have dropped out more frequently, resulting in a less
Fig. 2. Receiver operating characteristic (ROC) analysis for the World Health Organization Adult ADHD Self Report Scale, version for adolescents (ASRS-A) with dichotomized answers (0–18 points) in a clinical adolescent sample (n 134; boys n 53). NORD J PSYCHIATRY·VOL 69 NO 3·2015
221
KARIN SONNBY ET AL
impaired study sample than the true patient population. However, it was not possible to investigate this. Moreover, some adolescents are not motivated help-seekers themselves but are brought to the clinic by their parents, which may result in less reliable ASRS-A self-reports. This might have occurred more frequently in boys, who more often chose not to participate. Symptoms of ADHD have been shown to be under-reported by adolescent boys. Therefore, parents are considered more reliable (26). By interviewing the adolescent and parent together, the reference standard was more likely to be valid. Furthermore, almost half of the participants completed the ASRS-A after the K-SADS interview, which may have sensitized them to their symptoms and influenced their answers. However, there were no differences in diagnostic accuracy or mean symptom level in the ASRS-A between those who answered the ASRS-A before the K-SADS interview and those who answered the ASRS-A afterwards. With regard to the different psychometric properties among boys and girls, caution must be paid to sample size; there were fewer boys than girls, which implies greater distribution variety among boys. Additionally, the comorbidity rates were considerably higher among girls than boys and were most pronounced among the patients with ADHD. Comorbidity may influence the ratings of the ASRS-A due to overlap with symptoms of the comorbid Moreover, boys with severe ADHD and high comorbidity rates may seek help more often before adolescence, compared with girls with severe ADHD and high comorbidity rates (10, 27). Therefore, help-seeking boys and girls may have different characteristics in adolescence. The internal consistency of the ASRS-A was high for both boys and girls, indicating it is reliable for both sexes. Finally, it is uncertain whether the dichotomized scoring method of Kessler et al is the optimal scoring method among adolescent psychiatric patients (11). A strength of the present study is that the ASRS-A-S/ ASRS-A was compared with a gold standard diagnosis of ADHD made by K-SADS interviewers with good inter-rater reliability before and during the data collection. Moreover, the time between the index test and the reference test was not more than 7 days. Another methodological strength is that the ADHD diagnosis from the K-SADS uses information from two informants—the adolescent and a parent. A further strength is that the study sample consisted of unselected adolescent outpatients with mental health problems, rather than patients in treatment for ADHD or patients from a specialist ADHD clinic. Additionally, the fact that the patients were included from three different sites, two small towns and one large city, is a strength that may further enhance the generalizability of the results. As a great deal of research on ADHD has been conducted in solely or predominately male samples, the present study contributes
222
more knowledge about the less often studied, yet equally impaired, females with ADHD. Finally, the present study fulfils the need for validation studies of the Swedish translation of the ASRS-A-S and ASRS-A.
Conclusion In a clinical population of adolescents, the ASRS-A-S and ASRS-A showed psychometric properties and diagnostic accuracy, which support clinical usefulness as well as further evaluation of the most optimal scoring method in this age group. Acknowledgements—Grants to KWN from the following research funds are acknowledged: the Swedish Brain Foundation, the Swedish Alcohol Monopoly Research Council (SRA), the Swedish Council for Working Life and Social Research (FAS), the Uppsala-Örebro Regional Research Council, the Fredrik and Ingrid Thurings Foundation, the County Council of Västmanland, the König-Söderströmska Foundation, the Swedish Psychiatric Foundation, and the Svenska Spel Research Foundation. The funding sources had no further role in study design, the collection, analysis and interpretation of data, the writing of the report, or the decision to submit the paper for publication. The authors Karin Sonnby, Konstantinos Skordas, Susanne Olofsdotter, Sofia Vadlin, Kent W. Nilsson and Mia Ramklint have no conflict of interests to disclose.
Declaration of interest: The authors report no conflicts of interest. The authors alone are responsible for the content and writing of the paper.
References 1. Merikangas KR, He JP, Burstein M, Swanson SA, Avenevoli S, Cui L, et al. Lifetime prevalence of mental disorders in U.S. adolescents: Results from the National Comorbidity Survey Replication—Adolescent Supplement (NCS-A). J Am Acad Child Adolesc Psychiatry 2010;49:980–9. 2. Smalley SL, McGough JJ, Moilanen IK, Loo SK, Taanila A, Ebeling H, et al. Prevalence and psychiatric comorbidity of attentiondeficit/hyperactivity disorder in an adolescent Finnish population. J Am Acad Child Adolesc Psychiatry 2007;46:1575–83. 3. Pham AV. Differentiating behavioral ratings of inattention, impulsivity, and hyperactivity in children: Effects on reading achievement. J Atten Disord 2013, Feb 11 [Epub ahead of print]. 4. Bauermeister JJ, Shrout PE, Chavez L, Rubio-Stipec M, Ramirez R, Padilla L, et al. ADHD and gender: Are risks and sequela of ADHD the same for boys and girls? J Child Psychol Psychiatry 2007;48:831–9. 5. Becker SP, Luebbe AM, Langberg JM. Co-occurring mental health problems and peer functioning among youth with attentiondeficit/hyperactivity disorder: A review and recommendations for future research. Clin Child Fam Psychol Rev 2012;15:279–302. 6. Pulgaron ER. Childhood obesity: A review of increased risk for physical and psychological comorbidities. Clin Ther 2013;35: A18–32. 7. Impey M, Heun R. Completed suicide, ideation and attempt in attention deficit hyperactivity disorder. Acta Psychiatr Scand 2012;125:93–102. 8. Halmoy A, Fasmer OB, Gillberg C, Haavik J. Occupational outcome in adult ADHD: Impact of symptom profile, comorbid psychiatric problems, and treatment: A cross-sectional study of 414 clinically diagnosed adult ADHD patients. J Atten Disord 2009;13:175–87. 9. Yoshimasu K, Barbaresi WJ, Colligan RC, Voigt RG, Killian JM, Weaver AL, et al. Childhood ADHD is strongly associated with NORD J PSYCHIATRY·VOL 69 NO 3·2015
VALIDITY STUDY OF
10.
11.
12.
13. 14. 15.
16. 17.
18.
19.
a broad range of psychiatric disorders during adolescence: A population-based birth cohort study. J Child Psychol Psychiatry 2012;53:1036–43. Merikangas KR, He JP, Burstein M, Swendsen J, Avenevoli S, Case B, et al. Service utilization for lifetime mental disorders in U.S. adolescents: Results of the National Comorbidity Survey— Adolescent Supplement (NCS-A). J Am Acad Child Adolesc Psychiatry 2011;50:32–45. Kessler RC, Adler L, Ames M, Demler O, Faraone S, Hiripi E, et al. The World Health Organization Adult ADHD Self-Report Scale (ASRS): A short screening scale for use in the general population. Psychol Med 2005;35:245–56. Dakwar E, Mahony A, Pavlicova M, Glass A, Brooks D, Mariani JJ, et al. The utility of attention-deficit/hyperactivity disorder screening instruments in individuals seeking treatment for substance use disorders. J Clin Psychiatry 2012;73:e1372–8. Yeh CB, Gau SS, Kessler RC, Wu YY. Psychometric properties of the Chinese version of the adult ADHD Self-report Scale. Int J Methods Psychiatr Res 2008;17:45–54. Hesse M. The ASRS-6 has two latent factors: Attention deficit and hyperactivity. J Atten Disord 2013;17:203–7. Adler LA, Shaw DM, Spencer TJ, Newcorn JH, Hammerness P, Sitt DJ, et al. Preliminary examination of the reliability and concurrent validity of the attention-deficit/hyperactivity disorder self-report scale v1.1 symptom checklist to rate symptoms of attention-deficit/ hyperactivity disorder in adolescents. J Child Adolesc Psychopharmacol 2012;22:238–44. Bossuyt PM, Reitsma JB, Bruns DE, Gatsonis CA, Glasziou PP, Irwig LM, et al. Towards complete and accurate reporting of studies of diagnostic accuracy: The STARD initiative. BMJ 2003 4;326:41–4. Swedish Council on Health Technology Assessment. ADHD— Diagnostics and treatment, the organization of health care and the participation of the patient. A systematic review of the literature 2013 91–85413–58–4. Rodriguez A, Ginsberg Y, Fernholm A, Nyberg L. [ADHD difficult to diagnose in adults. ASRS v1.1 Self-Report Scales valuable help—Now translated to Swedish]. Lakartidningen 2007; 2–8;104(18). Ambrosini PJ. Historical development and present status of the schedule for affective disorders and schizophrenia for school-age children (K-SADS). J Am Acad Child Adolesc Psychiatry 2000;39:49–58.
NORD J PSYCHIATRY·VOL 69 NO 3·2015
THE
ASRS
FOR ADOLESCENTS FOR CLINICAL USE
20. Fitzpatrick C, Abayomi NN, Kehoe A, Devlin N, Glackin S, Power L, et al. Do we miss depressive disorders and suicidal behaviours in clinical practice? Clin Child Psychol Psychiatry 2012;17:449–58. 21. Lauth B, Magnusson P, Ferrari P, Petursson H. An Icelandic version of the Kiddie-SADS-PL: Translation, cross-cultural adaptation and inter-rater reliability. Nord J Psychiatry 2008;62:379–85. 22. Fleiss JL, Cohen, J., Everitt, B.S. Large sample standard errors of kappa and weighted kappa. Psychological Bulletin 1969;72:323–7. 23. Byrt T, Bishop J, Carlin JB. Bias, prevalence and kappa. J Clin Epidemiol 1993;46:423–9. 24. Chen G, Faris P, Hemmelgarn B, Walker RL, Quan H. Measuring agreement of administrative data with chart data using prevalence unadjusted and adjusted kappa. BMC medical research methodology 2009;9:5. 25. Peacock. Oxford handbook of medical statistics. New York: Oxford University Press; 2011. 26. Biederman J, Ball SW, Mick E, Monuteaux MC, Kaiser R, Bristol E, et al. Informativeness of maternal reports on the diagnosis of ADHD: An analysis of mother and youth reports. J Atten Disord 2007;10:410–7. 27. The Swedish Social Insurance Agency, The childcare allowance— Developments to date and support situation tomorrow. Stockholm; 2012. Karin Sonnby, M.D., Ph.D., Centre for Clinical Research, Uppsala University, Västmanlands County Hospital, SE 721 89 Västerås, Sweden. Konstantinos Skordas, M.D., Department of Neuroscience, Psychiatry, Uppsala University, Akademiska sjukhuset, ing 10, SE 751 85 Uppsala, Sweden. Susanne Olofsdotter, Centre for Clinical Research, Uppsala University, Västmanlands County Hospital, SE 721 89 Västerås, Sweden. Sofia Vadlin, Centre for Clinical Research, Uppsala University, Västmanlands County Hospital, SE 721 89 Västerås, Sweden. Kent W. Nilsson, Ph.D., Centre for Clinical Research, Uppsala University, Västmanlands County Hospital, SE 721 89 Västerås, Sweden. Mia Ramklint, M.D., Ph.D., Department of Neuroscience, Psychiatry, Uppsala University, Akademiska sjukhuset, ing 10, SE 751 85 Uppsala, Sweden.
223
Copyright of Nordic Journal of Psychiatry is the property of Taylor & Francis Ltd and its content may not be copied or emailed to multiple sites or posted to a listserv without the copyright holder's express written permission. However, users may print, download, or email articles for individual use.
Sonnby K, Skordas K, Olofsdotter S, Vadlin S, Nilsson KW, Ramklint M. Validation of the World Health Organization Adult ADHD Self-Report Scale for adolescents. Nord J Psychiatry 2015;69:216–223. Background: The World Health Organization Adult ADHD Self Report Scale (ASRS) is a widely used diagnostic tool for assessment of attention-deficit hyperactivity disorder (ADHD) symptoms in clinical psychiatry in Sweden. The ASRS consists of 18 questions, the first six of which can be used as a short screening version (ASRS-S). There is a version for adolescents— ASRS-Adolescent (ASRS-A)—and the corresponding screening version (ASRS-A-S), which has not been validated to date. Aim: The aim was to validate the ASRS-A and the ASRS-A-S for use in adolescent clinical populations. Methods: Adolescent psychiatric outpatients (n 134, mean age 15 years, 40% boys) reported on the ASRS-A, and were interviewed with the Kiddie Schedule of Affective Disorders and Schizophrenia (K-SADS), a semi-structured interview, together with a parent. Results: Internal consistency was 0.79 for the ASRS-A-S and 0.92 for the ASRS-A (Cronbach’s alpha). Internal consistency values were 0.79 and 0.87 for the inattention subscale, and 0.68 and 0.89 for the hyperactivity subscale, respectively. Concurrent validity values, measured with Spearman’s correlation coefficient, between the total K-SADS ADHD symptom severity score and the sum of ASRS-A-S and ASRS-A total scores were 0.51 and 0.60, respectively. Psychometric properties of the ASRS-A-S and the ASRS-A were: sensitivity 74% and 79%; negative predictive value 81% and 84%; specificity 59% and 60%; and positive predictive value 49% and 51%, respectively. Both versions showed better properties for girls than for boys. Conclusion: Both the ASRS-A-S and the ASRS-A showed promising psychometric properties for use in adolescent clinical populations. • Adolescent, Attention deficit disorder with hyperactivity, Validation study. Karin Sonnby, M.D., Ph.D., Centre for Clinical Research, Uppsala University, Västmanlands County Hospital, SE 721 89 Västerås, Sweden, E-mail: [email protected]; Accepted 18 September 2014.
A
ttention-deficit hyperactivity disorder (ADHD) is characterized by inattention and/or hyperactivity– impulsivity and has, in adolescence, a lifetime prevalence in the range 4–9% (1, 2). Symptoms of ADHD have a negative impact on school performance (3, 4), social relationships (5) and physical health (6), and increase suicidal risk (7). Early identification and treatment improves prognosis, including occupational outcome (8). ADHD is also characterized by high comorbidity rates (2, 9). In clinical work, patients with comorbid ADHD may be help-seeking because of a comorbid disorder, for example due to depression. Girls are less helpseeking for ADHD but might suffer from it as a comorbid disorder (10). Therefore, there is a need for
© 2014 Informa Healthcare
questionnaires that assess ADHD symptoms and are suitable for screening clinical populations to identify affected adolescents, especially girls. One such questionnaire is the World Health Organization Adult ADHD Self-Report Scale (ASRS) (11). The ASRS has been developed for adults and consists of two subscales targeting inattentiveness and hyperactivity–impulsivity, respectively. The first six questions of the ASRS can be used alone as a short screening scale and are collectively called the ASRS-Screening Test (ASRS-S). In the USA, the ASRS and the ASRS-S have been validated against an ADHD diagnosis in a stratified, population-based sample (n 154, gender rate not reported) (11). The ASRS-S has also been validated among drug-dependent
DOI: 10.3109/08039488.2014.968203
VALIDITY STUDY OF
patients (n 102, 83% male) (12). Both studies indicated that ASRS and ASRS-S are valid and reliable instruments for the screening of ADHD in adults (11, 12). The Chinese translation of the ASRS was investigated in two large, population-based young adult samples (n 1031, 100% males; n 3298, 38% males) (13). These studies found that the ASRS had good internal consistency among the subscales, and moderate to high correlations between the subscales and the Wender Utah Rating Scale (13). This supported the ASRS as a reliable and valid instrument for ADHD screening in adults (13). In two independent samples consisting of 232 (41% male) university students and 157 (68% male) drug-dependent outpatients, Hesse found that the ASRS-S measured the two constructs of inattention and hyperactivity independently, although the two domains were correlated with each other (14). The ASRS has also been studied in an adolescent sample of ADHD patients (n 88, 76% male) (15). Test–retest reliability and concurrent validity of the ASRS were examined in comparison with another clinician-administered ADHD rating scale. The results showed good test–retest reliability, high internal consistency and high concurrent validity (15). However, the study did not explore sensitivity, specificity, positive predictive value (PPV) or negative predictive value (NPV), as non-ADHD cases were not part of the study population (15). The Standards for Reporting of Diagnostic Accuracy (STARD) are international guidelines for investigators conducting diagnostic accuracy studies (16). No studies have used the Swedish translation of the ASRS in accordance with STARD (17), either in an adult or an adolescent population. Nevertheless, the Swedish translation of the ASRS is often used for the screening of ADHD in both adult and adolescent populations (18). In Sweden, versions of the ASRS, the ASRS-Adolescent (ASRS-A) and the ASRS-A-Screener (ASRS-A-S) are used among adolescents. However, there are no reports of the diagnostic accuracy of these versions. There is thus a need to examine the diagnostic accuracy of the Swedish version of the ASRS-A-S/ASRS-A.
Aims Aims of the study were to evaluate the psychometric properties and diagnostic accuracy of the Swedish translation of the ASRS-A and the shorter screening version (ASRS-A-S) in a group of adolescent psychiatric patients, including separate analyses for boys and girls.
Materials and methods Study design This study was a cross-sectional, diagnostic accuracy study. NORD J PSYCHIATRY·VOL 69 NO 3·2015
THE
ASRS
FOR ADOLESCENTS FOR CLINICAL USE
Participants Consecutively referred psychiatric outpatients aged 12–17 years old, from three Swedish cites (Enköping, Västerås and Sala) were invited to participate. Patients in need of acute psychiatric or somatic inpatient care at the time of the diagnostic interview, and those who required a Swedish interpreter, were excluded. The data collection took place from September 2011 to June 2013. The Västerås and Sala clinics are both part of the Child and Adolescent Psychiatric Clinic of the county of Västmanland. The Västerås clinic is situated in an area with 130,000 inhabitants, and the Sala clinic is situated in an area with 20,000 inhabitants. In Västerås and Sala, 433 adolescents were referred during the study period. Of those, 307 were invited and 157 (51%) agreed to participate, and of those 114 had full information for the analyses. The Enköping clinic is part of the Child and Adolescent Psychiatric Clinic of the county of Uppsala and is situated in an area with 40,000 inhabitants. One hundred and sixty-nine adolescents were referred to the Enköping clinic during the study period, of whom 55 were invited and 28 agreed to participate, and of those 20 had full information. In total from the three cites there were 134 patients with full information for the analyses. There were no significant differences between participants and internal drop-outs in sex or age. However, there was a difference between participants and external drop-outs related to sex, with a lower proportion of boys participating in the study (χ2 4.99, P 0.05). Participating boys were younger than non-participating boys (t 3.422, P 0.001), but there was no such difference in mean age between participating and non-participating girls (t 0.078, P 0.531).
Procedure Before the first appointment, the patients and their families were informed about the study. The first clinical assessment was a traditional non-structured psychiatric interview with the parent and adolescent together. At this appointment, the patient was invited to participate in the study. Only dyads of each one adolescent and a parent, who both agreed to participate, were included in the study. During the second visit, the parent and the adolescent were interviewed together using the K-SADS. In Enköping, the adolescents filled out the ASRS-A questionnaire immediately before completing the K-SADS interview. Responses on the ASRS-A were withheld from the K-SADS interviewer by placing the questionnaire inside an envelope. In Västerås and Sala, the ASRS-A was completed during the initial appointment. For 52 (39%) patients, the period between the initial appointment and the diagnostic interview was more than 7 days, which was considered the maximum time limit. In these cases, the ASRS-A questionnaire was completed again immediately after the K-SADS interview. The K-SADS
217
KARIN SONNBY ET AL
interviewers in Västerås and Sala were blind to the results of the ASRS-A.
Assessments THE ADULT ADHD SELF-REPORT SCALE ADOLESCENT VERSION (ASRS-A AND ASRS-A-S) The ASRS is a self-rating scale regarding ADHD in adults and is designed in accordance with DSM-IV criteria A for inattention and hyperactivity–impulsivity for ADHD. The questionnaire has 18 items and is composed of two subscales: inattention (IA) and hyperactivity–impulsivity (HD), with nine questions each. The response options are on a 0–4 Likert scale with the alternatives never, rarely, sometimes, often and very often. The total sum of the ASRS ranges from 0 to 72. The first six questions can be used alone as a short screening scale called the ASRS-S. The total sum of the ASRS-S ranges from 0 to 24. Kessler et al. (11) recommend a summation based on dichotomized cut-off points for all items, shown as shadowed areas on the questionnaire, and proposed a cutoff score of 9 for the ASRS (range 0–18), and 4 for the ASRS-S (range 0–6 points). In the original validation study where the dichotomized scoring method was used, the six-question ASRS-S screening test and the 18-question ASRS had sensitivity of 68.7% and 56.3%, and specificity of 99.5% and 98.3%, respectively. In a clinical, drug-dependent population, the ASRS-S had sensitivity of 66.7% and specificity of 81.2%, with a PPV of 41.8% and an NPV of 92.6% (12). The Swedish version of the ASRS was translated from English by Nyberg and back into English by Fernholm (18). Minor adaption of the wording to suit adolescents was made by one of the authors (KWN). KIDDIE-SCHEDULE OF AFFECTIVE DISORDERS AND SCHIZOPHRENIA (K-SADS) The K-SADS is a semi-structured diagnostic interview that examines current and previous psychopathology in children and adolescents between the ages of 6 and 18 years (19). The K-SADS consists of one screening part and eight supplements. If general symptoms are indicated in the screening interview, questions from the appropriate supplement are used to verify the diagnosis. The K-SADS is regarded as the gold standard for child psychiatric diagnoses (19–21). Conducting one K-SADS interview takes between 1.5 and 8 h. K-SADS ADHD SEVERITY SCORE The symptoms of ADHD in the K-SADS, according to DSM-IV criterion A, are rated on a scale ranging of 0–3. “Clinical significant symptoms” are coded as 3, “subclinical symptoms” as 2, “no symptoms” as 1 and “no information” as 0. A summation of the total ADHD symptom score in the K-SADS ranges between 0 and 57.
218
Training and reliability of interviewers All interviewers (n 6) were trained in the use of the K-SADS before they started the data collection. The interviewers consisted of a specialist in child and adolescent psychiatry, two residents in child and adolescent psychiatry, two clinical psychologists and a clinical social worker, all working in child psychiatric services. All interviewers completed their basic training in accordance with the Swedish standard for K-SADS training, with 4 days of theory and practice. The specialist in child and adolescent psychiatry (the first author) had received national training to train other interviewers. She planned the training for the others and was considered the standard when calculating inter-rater reliability. The training was based on viewing recorded interviews made by “master interviewers”. The ratings of these interviews were estimated individually and, then discussed among the group. Thereafter, the interviewers recorded themselves interviewing patients. The others assessed the recordings, and inter-rater reliability was calculated. Inter-rater reliability was determined before the study began, based on five consecutive interviews. It was calculated as both kappa and prevalenceand bias-adjusted kappa (PABAK) (22). First, inter-rater reliability was calculated for each interviewer for each diagnosis. The mean was then calculated for all diagnoses, and finally, the mean inter-rater reliability for the group was calculated. For the first five interviews before the study began, the mean inter-rater reliability for the group yielded a kappa of 0.84 (range 0.51–1.00) and a PABAK of 0.90 (range 0.77–1.00). For the diagnosis of ADHD, kappa was 0.83 (range 0.54–1.00) and PABAK was 0.84 (range 0.60–1.00). During the study, a joint assessment meeting took place each month to ensure continuous high levels of agreement. In total, 25 interviews were rated by the first author, and other interviewers each rated between five and 22 interviews. The mean ( standard deviation) number of interviews rated by the interviewers was 16 7.0. The overall inter-rater reliability, based on all ratings, yielded a kappa of 0.84 (range 0.73–1.00) and a PABAK of 0.91 (range 0.86–1.00). For the diagnosis of ADHD, kappa was 0.81 (range 0.70–1.00) and PABAK was 0.82 (range 0.70–1.00).
Statistical analysis Kappa statistics were used for agreement between interviewers (22, 23). Kappa is considered slight if kappa 0.2, fair if kappa is 0.21–0.40, moderate if kappa is 0.41– 0.60, substantial if kappa is 0.61–0.80 and almost perfect if kappa is 0.81–1.0. When the distribution of reported categories (the diagnoses) is unevenly distributed to a large extent, Cohen’s kappa is known to be less reliable (23, 24). Therefore, we also used the PABAK (23). Univariate analyses were performed to analyse differences between groups, by using t-tests for continuous NORD J PSYCHIATRY·VOL 69 NO 3·2015
VALIDITY STUDY OF
and ordinal data; however, they were validated by the non-parametric Mann–Whitney U test. Chi-square tests were used for dichotomous and categorical variables. The internal consistency of the ASRS-A-S and the ASRS-A was calculated with Cronbach’s alpha. Spearman’s rho was calculated for correlations between the total ASRS-A-S scores (0–24) and the total K-SADS ADHD symptom severity scores (0–57) as well as between the ASRS-A total scores (0–72) and the total K-SADS ADHD symptom severity scores (0–57). A dichotomous cut-off point was set for each item in accordance with Kessler et al.’s suggestions for ASRS-S and ASRS (11). The sums were used for receiver operating characteristic (ROC) analysis for both versions of the ASRS-A (25). The reference test, against which the ASRS-A-S and the ASRS-A were compared, was the K-SADS interview. All ADHD diagnoses, including ADHD (n 33), attention-deficit disorder (ADD; n 2), HD (n 2) and ADHD not otherwise specified (ADHD NOS; n 11), were pooled together as a reference diagnosis of ADHD (n 48). The original cut-offs were used for calculations of sensitivity, specificity, PPV and NPV as well as the odds ratio (OR) for an ADHD diagnosis. ROC analyses were used for area under the curve calculations (AUC) (25). Separate analyses were made for boys and girls. The study was approved by the Regional Ethics Committee of Uppsala University.
Results The mean age was 15 years among both boys and girls in the total sample (n 134). A little more than one
THE
ASRS
FOR ADOLESCENTS FOR CLINICAL USE
third of the sample had an ADHD diagnosis, with a larger proportion among boys than among girls. Comorbidity was present in a large majority of the patients, and one quarter had no comorbidity. The comorbidity rates were higher among girls than among boys, and this was more pronounced among individuals with ADHD. Among individuals with ADHD with four or more disorders, the proportion of girls was almost 10 times as large as among boys (Table 1). Not surprisingly, when comparing individuals with and without ADHD, the ADHD groups had higher K-SADS-ADHD severity score, as well as higher mean scores in the ASRS-A and ASRSA-S (Table 2).
Psychometric properties of the ASRS-A-S and the ASRS-A Internal consistency for the ASRS-A-S, measured by Cronbach’s alpha, was 0.79 (boys 0.81, girls 0.79). For the inattention subscale, internal consistency was 0.79 (boys 0.84, girls 0.76) and the hyperactivity subscale yielded a score of 0.68 (boys 0.71, girls 0.65). Internal consistency for the ASRS-A total scale, measured by Cronbach’s alpha, was 0.92 (boys 0.91, girls 0.92). For inattention, it was 0.87 (boys 0.88, girls 0.87) and for hyperactivity, it was 0.89 (boys 0.89, girls 0.89).
Concurrent validity of the ASRS-A-S and the ASRS-A Concurrent validity, measured by Spearman’s rho, for the correlation between the total K-SADS ADHD symptom severity score and the total sum of all ASRS-A-S ratings was 0.51 (boys 0.58, girls 0.47). Spearman’s rho for the
Table 1. Comparison between boys and girls among participating adolescent psychiatric patients (n134).
Mean age (s) No. of patients with ADHD, n (%) No. of disorders diagnosed, n (%) 0 1 2 3 4 5 6 8 Mean no. of disorders (s) Mean no. of diagnoses among patients with ADHD (s) No. of patients without ADHD with 4 disorders, n (%) No. of patients with ADHD with 4 disorders, n (%)
Boys, n 53 (39.6%)
Girls, n 81 (60.4%)
15.0 (1.5) 24 (45.3)
15.2 (1.5) 23 (28.4)
6 (11.3) 14 (26.4) 23 (43.4) 6 (11.3) 1 (1.9) 3 (5.7) 0 (0.0) 0 (0.0) 1.8 (1.2) 2.0 (0.9) 3 (10.3) 1 (4.2)
2 (2.5) 19 (23.5) 29 (35.8) 10 (12.3) 7 (8.6) 7 (8.6) 6 (7.4) 1 (1.2) 2.6 (1.7) 3.3 (2.0) 12 (20.7) 9 (39.1)
Statistics *t 0.650, P 0.517 χ2 4.012, P 0.045
**t 3.270, P 0.001 ***t 2.873, P 0.007
χ2 1.450, P 0.229 χ2 8.572, P 0.003
s, standard deviation; ADHD, attention-deficit hyperactivity disorder. The Mann–Whitney test showed significant differences in mean rank for boys and girls, *P 0.503, **P 0.008, ***P 0.023. NORD J PSYCHIATRY·VOL 69 NO 3·2015
219
KARIN SONNBY ET AL
Table 2. Comparison of reported attention-deficit hyperactivity disorder (ADHD) symptoms among adolescent psychiatric patients (n134) with (n47) and without (n87) ADHD according to an ADHD diagnosis with the Kiddie Schedule of Affective Disorders and Schizophrenia (K-SADS).
K-SADS ADHD symptom severity score (range 0–57), mean (s) ASRS-A mean number of assented dichotomized items (range 0–18), mean (s) ASRS-A mean total sum of all ratings (range 0–72 points), mean (s) ASRS-A-S mean number of assented dichotomized items (range 0–6), mean (s) ASRS-A-S mean total sum of all ratings (range 0–24 points), mean (s)
Diagnosis of ADHD
No diagnosis of ADHD
48 (7) 11 (4) 45 (13) 4.3 (1.4) 16 (4)
22 (18) 7 (4) 31 (12) 3.0 (1.6) 12 (5)
Statistics *t 11.684, P 0.001 *t 5.840, P 0.001 *t 6.167, P 0.001 *t 6.770, P 0.001 *t 4.725, P 0.001
s, standard deviation; ASRS-A, World Health Organization Adult ADHD Self Report Scale, version for adolescents; ASRS-A-S, the corresponding screening version of the ASRS-A. *The Mann–Whitney test showed significant differences in mean rank for patients with or without ADHD; P 0.001.
correlation between the total K-SADS ADHD symptom severity score and the total sum of all ASRS-A ratings was 0.60 (boys 0.64, girls 0.58).
Diagnostic accuracy of the ASRS-A-S For the total sample, AUC was 0.73, the 95% confidence interval (CI) 0.64–0.81, P 0.001), with a score of 0.68 for boys (95% CI 0.54–0.83, P 0.023) and a score of 0.76 for girls (95% CI 0.64–0.89, P 0.001) (Fig. 1). Sensitivity, specificity, PPV, NPV and the OR for an ADHD diagnosis are presented in Table 3.
Diagnostic accuracy of the ASRS-A For the total sample, AUC was 0.78 (95% CI 0.70–0.87, P 0.001), with a score of 0.77 for boys (95% CI 0.65– 0.90, P 0.001) and a score of 0.81 for girls (95% CI: 0.70–0.93, P 0.001) (Fig. 2). Sensitivity, specificity, PPV, NPV and the OR for an ADHD diagnosis are presented in Table 3.
In the total sample, 82 individuals filled out the ASRS-A questionnaire before the K-SADS interview and 52 filled out the questionnaire after the interview. There were no significant differences on mean score on the ASRS-A between those rated before (8.6 4.6) and those rated after the K-SADS (8.3 4.7; Student’s t-test, t 0.369; P 0.0713; P 0.737).
Discussion The Swedish translation of the ASRS adapted for adolescents, ASRS-A/ASRS-A-S, showed similar psychometric properties to the original English adult version, with high internal consistency and concurrent validity (11). However, the diagnostic accuracy differed with higher sensitivity and lower specificity. Another dissimilarity, compared with the original ASRS, was better psychometric properties for the total scale in comparison to the screening version. Both the ASRS-A-S and the ASRS-A had better psychometric properties for girls than for boys.
Fig. 1. Receiver operating characteristic (ROC) analysis for the screening version of the World Health Organization Adult ADHD Self Report Scale, version for adolescents (ASRS-A-S) with dichotomized answers (0–6 points) in a clinical adolescent sample (n 134; boys n 53).
220
NORD J PSYCHIATRY·VOL 69 NO 3·2015
VALIDITY STUDY OF
THE
ASRS
FOR ADOLESCENTS FOR CLINICAL USE
Table 3. Sensitivity, specificity, positive and negative predictive values and odds ratios for the ASRSA-S/ASRS-A compared with an attention-deficit hyperactivity disorder (ADHD) diagnosis in a clinical adolescent psychiatric sample (n134). ASRS-A-S (0–6 points, cut-off 4)
Sens.
Spec.
PPV
NPV
OR
95% CI
P
Total sample, n 134 Boys, n 53 Girls, n 81 ASRS-A-S rated before K-SADS, n 82 ASRS-A-S rated after K-SADS, n 52 ASRS-A (0–18 points, cut-off 9) Total sample, n 134 Boys, n 53 Girls, n 81 ASRS-A rated before K-SADS, n 82 ASRS-A rated after K-SADS, n 52
74% 71% 78% 75% 74%
59% 55% 60% 53% 69%
49% 57% 44% 40% 65%
81% 70% 88% 84% 77%
4.132 2.989 5.478 3.444 6.296
1.890–9.034 0.951–9.390 1.784–16.821 1.195–9.924 1.861–21.298
0.001 0.061 0.003 0.022 0.003
79% 75% 83% 79% 78%
60% 59% 60% 57% 66%
51% 60% 45% 43% 64%
84% 74% 90% 87% 79%
5.497 4.250 7.228 5.016 6.840
2.422–12.475 1.302–13.874 2.178–23.992 1.647–15.278 1.955–23.927
0.001 0.017 0.001 0.005 0.003
ASRS-A, World Health Organization Adult ADHD Self Report Scale, version for adolescents; ASRS-A-S) the corresponding screening version of the ASRS-A; PPV, positive predictive value; NPV, negative predictive value; OR, odds ratio; CI, confidence interval; K-SADS, Kiddie Schedule of Affective Disorders and Schizophrenia (K-SADS). Results are presented for boys and girls and for those who filled in the ASRS-A-S/ASRS-A before or after the K-SADS interview.
Moreover, whether or not the ratings of the ASRS-A and ASRS-A-S were performed before or after the K-SADSinterview, did not have any major impact on the diagnostic accuracy. Studies among adults, even when performed in different countries, have found consistent psychometric properties for the ASRS (13), which are similar to those found in this adolescent sample. However, diagnostic accuracy for the ASRS-A in this sample differed from that of previous studies. The only other study performed in the same age group (15) consisted exclusively of ADHD patients, with a preponderance of males, and adolescents perceived as unreliable reporters of their ADHD symptoms were excluded. Differences between participants, such as life stage (adolescents vs. adults)
and sex, may influence sensitivity and specificity because comorbidity patterns of individuals with ADHD might differ between groups. Comorbidity patterns may influence the extent of overlapping symptoms, as for instance attention problems are present in depression and anxiety disorders as well. The prevalence, which varies between population-based and clinical samples, influences PPV and NPV. Increasing prevalence increases PPV but decreases NPV. Therefore, the difference in diagnostic accuracy between this and previous studies is probably explained by differences in the studied populations. The results of the present study should be interpreted in the context of several limitations. The external dropout rate was high. Severely disordered patients might have dropped out more frequently, resulting in a less
Fig. 2. Receiver operating characteristic (ROC) analysis for the World Health Organization Adult ADHD Self Report Scale, version for adolescents (ASRS-A) with dichotomized answers (0–18 points) in a clinical adolescent sample (n 134; boys n 53). NORD J PSYCHIATRY·VOL 69 NO 3·2015
221
KARIN SONNBY ET AL
impaired study sample than the true patient population. However, it was not possible to investigate this. Moreover, some adolescents are not motivated help-seekers themselves but are brought to the clinic by their parents, which may result in less reliable ASRS-A self-reports. This might have occurred more frequently in boys, who more often chose not to participate. Symptoms of ADHD have been shown to be under-reported by adolescent boys. Therefore, parents are considered more reliable (26). By interviewing the adolescent and parent together, the reference standard was more likely to be valid. Furthermore, almost half of the participants completed the ASRS-A after the K-SADS interview, which may have sensitized them to their symptoms and influenced their answers. However, there were no differences in diagnostic accuracy or mean symptom level in the ASRS-A between those who answered the ASRS-A before the K-SADS interview and those who answered the ASRS-A afterwards. With regard to the different psychometric properties among boys and girls, caution must be paid to sample size; there were fewer boys than girls, which implies greater distribution variety among boys. Additionally, the comorbidity rates were considerably higher among girls than boys and were most pronounced among the patients with ADHD. Comorbidity may influence the ratings of the ASRS-A due to overlap with symptoms of the comorbid Moreover, boys with severe ADHD and high comorbidity rates may seek help more often before adolescence, compared with girls with severe ADHD and high comorbidity rates (10, 27). Therefore, help-seeking boys and girls may have different characteristics in adolescence. The internal consistency of the ASRS-A was high for both boys and girls, indicating it is reliable for both sexes. Finally, it is uncertain whether the dichotomized scoring method of Kessler et al is the optimal scoring method among adolescent psychiatric patients (11). A strength of the present study is that the ASRS-A-S/ ASRS-A was compared with a gold standard diagnosis of ADHD made by K-SADS interviewers with good inter-rater reliability before and during the data collection. Moreover, the time between the index test and the reference test was not more than 7 days. Another methodological strength is that the ADHD diagnosis from the K-SADS uses information from two informants—the adolescent and a parent. A further strength is that the study sample consisted of unselected adolescent outpatients with mental health problems, rather than patients in treatment for ADHD or patients from a specialist ADHD clinic. Additionally, the fact that the patients were included from three different sites, two small towns and one large city, is a strength that may further enhance the generalizability of the results. As a great deal of research on ADHD has been conducted in solely or predominately male samples, the present study contributes
222
more knowledge about the less often studied, yet equally impaired, females with ADHD. Finally, the present study fulfils the need for validation studies of the Swedish translation of the ASRS-A-S and ASRS-A.
Conclusion In a clinical population of adolescents, the ASRS-A-S and ASRS-A showed psychometric properties and diagnostic accuracy, which support clinical usefulness as well as further evaluation of the most optimal scoring method in this age group. Acknowledgements—Grants to KWN from the following research funds are acknowledged: the Swedish Brain Foundation, the Swedish Alcohol Monopoly Research Council (SRA), the Swedish Council for Working Life and Social Research (FAS), the Uppsala-Örebro Regional Research Council, the Fredrik and Ingrid Thurings Foundation, the County Council of Västmanland, the König-Söderströmska Foundation, the Swedish Psychiatric Foundation, and the Svenska Spel Research Foundation. The funding sources had no further role in study design, the collection, analysis and interpretation of data, the writing of the report, or the decision to submit the paper for publication. The authors Karin Sonnby, Konstantinos Skordas, Susanne Olofsdotter, Sofia Vadlin, Kent W. Nilsson and Mia Ramklint have no conflict of interests to disclose.
Declaration of interest: The authors report no conflicts of interest. The authors alone are responsible for the content and writing of the paper.
References 1. Merikangas KR, He JP, Burstein M, Swanson SA, Avenevoli S, Cui L, et al. Lifetime prevalence of mental disorders in U.S. adolescents: Results from the National Comorbidity Survey Replication—Adolescent Supplement (NCS-A). J Am Acad Child Adolesc Psychiatry 2010;49:980–9. 2. Smalley SL, McGough JJ, Moilanen IK, Loo SK, Taanila A, Ebeling H, et al. Prevalence and psychiatric comorbidity of attentiondeficit/hyperactivity disorder in an adolescent Finnish population. J Am Acad Child Adolesc Psychiatry 2007;46:1575–83. 3. Pham AV. Differentiating behavioral ratings of inattention, impulsivity, and hyperactivity in children: Effects on reading achievement. J Atten Disord 2013, Feb 11 [Epub ahead of print]. 4. Bauermeister JJ, Shrout PE, Chavez L, Rubio-Stipec M, Ramirez R, Padilla L, et al. ADHD and gender: Are risks and sequela of ADHD the same for boys and girls? J Child Psychol Psychiatry 2007;48:831–9. 5. Becker SP, Luebbe AM, Langberg JM. Co-occurring mental health problems and peer functioning among youth with attentiondeficit/hyperactivity disorder: A review and recommendations for future research. Clin Child Fam Psychol Rev 2012;15:279–302. 6. Pulgaron ER. Childhood obesity: A review of increased risk for physical and psychological comorbidities. Clin Ther 2013;35: A18–32. 7. Impey M, Heun R. Completed suicide, ideation and attempt in attention deficit hyperactivity disorder. Acta Psychiatr Scand 2012;125:93–102. 8. Halmoy A, Fasmer OB, Gillberg C, Haavik J. Occupational outcome in adult ADHD: Impact of symptom profile, comorbid psychiatric problems, and treatment: A cross-sectional study of 414 clinically diagnosed adult ADHD patients. J Atten Disord 2009;13:175–87. 9. Yoshimasu K, Barbaresi WJ, Colligan RC, Voigt RG, Killian JM, Weaver AL, et al. Childhood ADHD is strongly associated with NORD J PSYCHIATRY·VOL 69 NO 3·2015
VALIDITY STUDY OF
10.
11.
12.
13. 14. 15.
16. 17.
18.
19.
a broad range of psychiatric disorders during adolescence: A population-based birth cohort study. J Child Psychol Psychiatry 2012;53:1036–43. Merikangas KR, He JP, Burstein M, Swendsen J, Avenevoli S, Case B, et al. Service utilization for lifetime mental disorders in U.S. adolescents: Results of the National Comorbidity Survey— Adolescent Supplement (NCS-A). J Am Acad Child Adolesc Psychiatry 2011;50:32–45. Kessler RC, Adler L, Ames M, Demler O, Faraone S, Hiripi E, et al. The World Health Organization Adult ADHD Self-Report Scale (ASRS): A short screening scale for use in the general population. Psychol Med 2005;35:245–56. Dakwar E, Mahony A, Pavlicova M, Glass A, Brooks D, Mariani JJ, et al. The utility of attention-deficit/hyperactivity disorder screening instruments in individuals seeking treatment for substance use disorders. J Clin Psychiatry 2012;73:e1372–8. Yeh CB, Gau SS, Kessler RC, Wu YY. Psychometric properties of the Chinese version of the adult ADHD Self-report Scale. Int J Methods Psychiatr Res 2008;17:45–54. Hesse M. The ASRS-6 has two latent factors: Attention deficit and hyperactivity. J Atten Disord 2013;17:203–7. Adler LA, Shaw DM, Spencer TJ, Newcorn JH, Hammerness P, Sitt DJ, et al. Preliminary examination of the reliability and concurrent validity of the attention-deficit/hyperactivity disorder self-report scale v1.1 symptom checklist to rate symptoms of attention-deficit/ hyperactivity disorder in adolescents. J Child Adolesc Psychopharmacol 2012;22:238–44. Bossuyt PM, Reitsma JB, Bruns DE, Gatsonis CA, Glasziou PP, Irwig LM, et al. Towards complete and accurate reporting of studies of diagnostic accuracy: The STARD initiative. BMJ 2003 4;326:41–4. Swedish Council on Health Technology Assessment. ADHD— Diagnostics and treatment, the organization of health care and the participation of the patient. A systematic review of the literature 2013 91–85413–58–4. Rodriguez A, Ginsberg Y, Fernholm A, Nyberg L. [ADHD difficult to diagnose in adults. ASRS v1.1 Self-Report Scales valuable help—Now translated to Swedish]. Lakartidningen 2007; 2–8;104(18). Ambrosini PJ. Historical development and present status of the schedule for affective disorders and schizophrenia for school-age children (K-SADS). J Am Acad Child Adolesc Psychiatry 2000;39:49–58.
NORD J PSYCHIATRY·VOL 69 NO 3·2015
THE
ASRS
FOR ADOLESCENTS FOR CLINICAL USE
20. Fitzpatrick C, Abayomi NN, Kehoe A, Devlin N, Glackin S, Power L, et al. Do we miss depressive disorders and suicidal behaviours in clinical practice? Clin Child Psychol Psychiatry 2012;17:449–58. 21. Lauth B, Magnusson P, Ferrari P, Petursson H. An Icelandic version of the Kiddie-SADS-PL: Translation, cross-cultural adaptation and inter-rater reliability. Nord J Psychiatry 2008;62:379–85. 22. Fleiss JL, Cohen, J., Everitt, B.S. Large sample standard errors of kappa and weighted kappa. Psychological Bulletin 1969;72:323–7. 23. Byrt T, Bishop J, Carlin JB. Bias, prevalence and kappa. J Clin Epidemiol 1993;46:423–9. 24. Chen G, Faris P, Hemmelgarn B, Walker RL, Quan H. Measuring agreement of administrative data with chart data using prevalence unadjusted and adjusted kappa. BMC medical research methodology 2009;9:5. 25. Peacock. Oxford handbook of medical statistics. New York: Oxford University Press; 2011. 26. Biederman J, Ball SW, Mick E, Monuteaux MC, Kaiser R, Bristol E, et al. Informativeness of maternal reports on the diagnosis of ADHD: An analysis of mother and youth reports. J Atten Disord 2007;10:410–7. 27. The Swedish Social Insurance Agency, The childcare allowance— Developments to date and support situation tomorrow. Stockholm; 2012. Karin Sonnby, M.D., Ph.D., Centre for Clinical Research, Uppsala University, Västmanlands County Hospital, SE 721 89 Västerås, Sweden. Konstantinos Skordas, M.D., Department of Neuroscience, Psychiatry, Uppsala University, Akademiska sjukhuset, ing 10, SE 751 85 Uppsala, Sweden. Susanne Olofsdotter, Centre for Clinical Research, Uppsala University, Västmanlands County Hospital, SE 721 89 Västerås, Sweden. Sofia Vadlin, Centre for Clinical Research, Uppsala University, Västmanlands County Hospital, SE 721 89 Västerås, Sweden. Kent W. Nilsson, Ph.D., Centre for Clinical Research, Uppsala University, Västmanlands County Hospital, SE 721 89 Västerås, Sweden. Mia Ramklint, M.D., Ph.D., Department of Neuroscience, Psychiatry, Uppsala University, Akademiska sjukhuset, ing 10, SE 751 85 Uppsala, Sweden.
223
Copyright of Nordic Journal of Psychiatry is the property of Taylor & Francis Ltd and its content may not be copied or emailed to multiple sites or posted to a listserv without the copyright holder's express written permission. However, users may print, download, or email articles for individual use.
