This article was downloaded by: [University of New Hampshire] On: 09 February 2015, At: 00:56 Publisher: Routledge Informa Ltd Registered in England and Wales Registered Number: 1072954 Registered office: Mortimer House, 37-41 Mortimer Street, London W1T 3JH, UK
Journal of Personality Assessment Publication details, including instructions for authors and subscription information: http://www.tandfonline.com/loi/hjpa20
External Validity of the New Devereux Adolescent Behavior Rating Scales Carolyn L. Williams , Yossef S. Ben-Porath , Craig Uchiyama , Nathan C. Weed & Robert P. Archer Published online: 22 Jun 2011.
To cite this article: Carolyn L. Williams , Yossef S. Ben-Porath , Craig Uchiyama , Nathan C. Weed & Robert P. Archer (1990) External Validity of the New Devereux Adolescent Behavior Rating Scales, Journal of Personality Assessment, 55:1-2, 73-85, DOI: 10.1080/00223891.1990.9674048 To link to this article: http://dx.doi.org/10.1080/00223891.1990.9674048
PLEASE SCROLL DOWN FOR ARTICLE Taylor & Francis makes every effort to ensure the accuracy of all the information (the “Content”) contained in the publications on our platform. However, Taylor & Francis, our agents, and our licensors make no representations or warranties whatsoever as to the accuracy, completeness, or suitability for any purpose of the Content. Any opinions and views expressed in this publication are the opinions and views of the authors, and are not the views of or endorsed by Taylor & Francis. The accuracy of the Content should not be relied upon and should be independently verified with primary sources of information. Taylor and Francis shall not be liable for any losses, actions, claims, proceedings, demands, costs, expenses, damages, and other liabilities whatsoever or howsoever caused arising directly or indirectly in connection with, in relation to or arising out of the use of the Content. This article may be used for research, teaching, and private study purposes. Any substantial or systematic reproduction, redistribution, reselling, loan, sub-licensing, systematic supply, or distribution in any form to anyone is expressly forbidden. Terms & Conditions of access and use can be found at http://www.tandfonline.com/page/terms-and-conditions
JOURNAL OF PERSONALITY ASSESSMENT, 1990, 55(1&2),73-85 Copyright o 1990, Lawrence Erlbaum Associates, Inc.
External Validity of the New Devereux Adolescent Behavior Rating Scales Carolyn L. Williams, Yossef S. Ben-Porath, Craig Uchiyama, and Nathan C. Weed Downloaded by [University of New Hampshire] at 00:56 09 February 2015
University of Minnesota
Robert P. Archer Eastern Virginia Medical School
The discriminant and concurrent validity of the five new scales for the Devereux Adolescent Behavior Rating Scale (DAB) was explored using a heterogeneous sample of psychiatric and substance abuse patients. Consistent with predictions, the substance abuse patients scored higher on the Acting Out Behaviors (AOB:) and Heterosexual Interests (HI) scales, and psychiatric patients scored higher on the Psychotic Behaviors scale. Gender differences also were found, including boys being rated higher on Acting Out Behaviors, and girls higher on Heterosexuall Interests. The new DAB scales demonstrated sufficient concurrent validity using aL thorough record review and a parent rating scale (the Child Behavior Checklist [CBCL]). The NeurotidDependent Behaviors scale (NDB) showed a consistent relationship with substance abuse and several other measures of more externalizing behaviors, in addition to the predicted relationshipswith anxious, tense, and dependent behaviors. The WithdrawdTimid Behaviors scale (WTB) proved to be a purer measure of internalizing behaviors in both sexes.
The DAB was developed during the 1960s by Spivack and colleagues (Spivajck, Haimes, & Spotts, 1967; Spivack & Spotts, 1967) to assess behavior problems occurring in adolescents in residential treatment settings and to provide relevant prognostic and treatment information. Unlike other behavior rating scales for adolescents that rely on parents' reports, for example, the Personality Inventory for Children (PIC; Wia, Lachar, Klinedinst, & Seat, 1977) or the Louisville Behavior Checklist (LBC; Miller & Roid, 1988),the DAB uses treatment staff to provide ratings and can be used for older (i.e., 17 to 18 year olds), as well as younger, adolescents. This can be an advantage in residential settings where parents may not be available to provide ratings and where older adolescents may
Downloaded by [University of New Hampshire] at 00:56 09 February 2015
74
WILLIAMS, BEN-PORATH, UCHIYAMA, WEED, ARCHER
be placed. Because of its potential usefulness in contemporary adolescent treatment settings, we began a series of studies on the DAB in conjunction with a larger project on the assessment of psychopathology in adolescents (Williams & Butcher, 1989a, 198913). Our first study of the DAB used a contemporary sample of adolescents hospitalized for substance abuse (Williams, Ben-Porath, & Weed, in press) and revealed low internal consistencies, as measured by Cronbach's (1951) coefficient alpha, for 9 of the 15 original DAB scales developed by Spivack and colleagues (Spivack et al., 1967; Spivack & Spotts, 1967). The low internal consistencies, plus the finding of high intercorrelations for many of the DAB original scales, led us to develop broadband scales measuring acting out behaviors, psychotic behaviors, attention-seekindexpansive behaviors, and inner turrnoil/frailty. The broadband scales were developed using a principal components analysis of the 15 original DAB scales. A second study with a more heterogeneous adolescent treatment sample (i.e., the 404 substance abuse subjects from Williams et al., in press, plus an additional 333 subjects in psychiatric treatment) found that only two of the four higher order scales (i.e., acting out behaviors and psychotic behdviors), previously developed in the substance abuse sample, replicated adequately in the heterogeneous sample (Ben-Porath, Williams, & Uchiyama, 1989). Consequently, an item-levelfactor analysis was used to develop five new scales for the DAB: AOB, PB, WTB, NDB, a d HI, Following the demonstration of cross-gender factorial comparability in separate analyses by gender, we used a combined-gender sample for our item-level factor analysis to derive the most stable and representative structure. Both principal-components and principal-factor analyses revealed identical structures (Ben-Porath et al., 1989). Our new scales were more internally consistent than the original factorially derived DAB scales (alphas of the new scales range from .80 to .95, M = .87; alphas ofthe old scales range from .44 to ,90, M = .75) and their item content was congruent with the narrow- and broadband dimensions of adolescent psychopathology identified in previous studies using other rating scales (Ben-Porath et al., 1989). Our study explores aspects of the new DAB scales' external validity in our large adolescent clinical sample. Our first look at the scales' validity examines discriminant validity by determining whether two groups of adolescents in treatment for different problems (i.e., substance abuse and ~ s ~ c h i a t rdisorders) ic score differently on particular DAB scales. We hypothesize that for both sexes the mean values of the DAB acting out scales (i.e., AOB and HI) are greater in the substance abuse sample and that the PB scale are higher in the ~ s ~ c h i a t r i c patients. Concurrent validity of the new DAB scales is examined in two ways, using a standardized Record Review form (Williams & Butcher, 1989a) completed by research assistants, and parents' ratings on the CBCL (Achenbach 6. Edelbrock, 1983). Items from the Record Review form are included in the
Downloaded by [University of New Hampshire] at 00:56 09 February 2015
VALIDITY OF NEW DAB SCALES
75
validity analyses on the basis of their hypothesized relation to our new DlAB scales. Our specific hypotheses are that the AOB scale is related to the following Record Review items: acting out/irresponsible behzvior, anger outbursts/ fighting, oppositional behavior, running away, drug use/abuse, impulsivity, history of assaultive behavior, and juvenile court involvement for various offenses; the PB scale is related to bizarre behavior, bizarre sensory experiences, and bizarre thought processes/delusions; the WTB scale is related to social withdrawal, low self-esteem, depression, suicide ideatiordgestures, and having few or no friends; the NDB scale is related to tension/nervousness and somatic complaints; and the HI scale is related to being sexually active and sexual acting out. We believe that these predictions hold for both boys and girls. Our hypotheses concerning the relations between DAB and CBCL scales are as follows: for boys, the CBCL scales measuring acting out (i.e., Immature, Delinquent, and Aggressive) are related to the DAB AtOB and HI scales; the CBCL Schizoid scale is related to the DAB PB scale; the CBCL Somatic and Obsessive/Compulsive scales are related to the DAB NBD scale; and the CBCL Uncommunicative scale is related to the DAB Q l l 3 scale. Similar predictiions are made for girls: CBCL Delinquent, Aggressive, and Cruel scales are related to the DAB AOB and HI scales; the CBCL Schizoid scale ir; related to the DAB PB scale; the CBCL Anxious/Obsessive and Somatic scales are related to DAB NDB scale; the CBCL DepressecUWithdrawn scale is related to DAB WTB scale. In making these predictions we are aware that not all the hypothesized CBCL-DAB relationships might be found. As mentioned earlier, the CBCL was developed and normed for younger adolescents (i.e., upper limit of 16 years) and our sample included 17- and 18-year-olds. And, as youths mature, solme forms of psychopathology (e.g., depressed/anxious cogr~itionsor feelings) may not be as easily observed by others. Nevertheless, we believe it is important to present information about the new DAB scales' relationship to a more established behavior rating scale measuring similar constructs.
METHOD Subjects The sample of 737 adolescents (420 boys, 317 girls) describled by Ben-Porath et al. (1989) served as subjects in this study. This sample was drawn from three types of adolescent treatment facilities: inpatient substance abuse units, inpatient psychiatric units, and day-treatment psychiatric facilities. The substance abuse sires consisted of two unlocked units at two separate ho'spitals in Minneapolis that were designed for long-term residential treatment. Three inpatient psychiatric units were included in the study, two located in one hospital in Minnr-lp-
Downloaded by [University of New Hampshire] at 00:56 09 February 2015
76
WILLIAMS, BEN-PORATH, UCHIYAMA, WEED, ARCHER
olis and one in Norfolk, VA. The Minneapolis sites consisted of a short-term crisis intervention unit, focused primarily on assessment and evaluation, and a longer term psychiatric treatment unit that primarily emphasized therapeutic intervention. The Norfolk inpatient psychiatric unit was designed for acute psychiatric care. The Zday treatment facilities, located in Minneapolis, functioned as school alternatives for emotionally and behaviorally disturbed adolescents. Given the number of different community-based substance abuse and psychiatic units sampled, results from this study would likely generalize to other adolescents in similar units. Criteria for inclusion in the study were that the individual was 13 to 18 years of age, had a treatment stay of sufficient length to allow an accurate evaluation of the patient by staff, was not tested at another data collection site, had parental consent to participate, and had their physicians' approval, Of the 1,283 adolescents admitted to the facilities during data collection, 959 were eligible for inclusion in the study, resulting in an eligibility rate of 75%. Data were collected on 737 of the eligible adolescents, producing a 77% response rate. Most nanrespondents were due to administrative errors (i.e., failing to record new admissions on the study roster, determining a DAB was missing too long past discharge for staff to make an accurate assessment). The total sample ranged in age from 13 to 18 years (M = 15,5, SD = 1.3). Subjects were primarily White (84%), with 7% Native American, 5% Black, 1% Hispanic, 1% Asian American, and 2% unknown. Table 1 presents a demographic breakdown by site. Chi-square analyses were used to determine if there were demographic differences in subjects at the different treatment sites. These results revealed that the substance abuse sample had more boys than the psychiatric sample, more older subjects than the inpatient psychiatric or the day treatment psychiatric samples, and fewer Blacks, Hispanics, and Asian Americans (these frequencies were collapsed into one category for the analysis) than the other sites. The day treatment sites had significantly fewer Whites and proportionately more younger subjects than the other units. The psychiatric inpatient units had fewer Native Americans. These demographic differences by site did not lead us to change our predictions about the discriminant validity of the DAB scales.
Procedure The DAB was completed by psychiatric nurses, master's-level psychologists, or chemical-dependency counselors, depending on the setting. Between 12 and 15 raters participated at each site. These raters were not responsible for determining whether the subject was to be admitted to a psychiatric or substance abuse unit. Raters were given specific instructions on how and when to complete the DAB (as specified in the manual), and research assistants coordinated, monitored, and helped resolve any difficulties encountered in data collection.
77
VALIDITY OF NEW DAB SCALES TABLE 1 Breakdown of Gender, Age, and Race by Setting Male
Gender
Substance abuse Psychiatric: Inpatient Day treatment Total
Downloaded by [University of New Hampshire] at 00:56 09 February 2015
Age Substance abuse Psychiatric: Inpatient Day treatment Total Race
Substance abuse Psychiatric: Inpatient Day treatment Total
Female
Total
-
267
137
404
133 20 420
149 31 317
282 51 737
13
14
15
16
17
18
Total
6
43
84
132
114
25
404
42 18 66
54 12 109
69 17 170
68 3 203
42 1 157
7 0 32
282 51 737
Asian
Black
Hispanic
Native American
White
Unknown
Total
3
14
6
35
346
0
404
4
21 6 41
5 0 11
7 11 53
237 34 617
8 0 8
282 51 737
0
7
-
Some of the psychiatric inpatients were hospitalized fewer than the 14 days suggested by the DAB manual. In such cases, ratings were obtained only when stafffelt confident of ~rovidingvalid information. This ~ r i m a r ioccurred l~ in the Minneapolis crisis intervention unit, and made up approximately 13% of the total sample. Record review data were completed on 96% of the subjects (n = 347 boys, n = 257 girls) from the Minneapolis treatment facilities by a team of nine research assistants who met weekly to insure consistency in their reviews. Although1 an individual DAB rater could write observations about a subject's behavior during his or her shift in the hospital record, usually in the nurse's daily section, an individual rater's impressions comprised only a small fraction of the patient's total hospital record. Thus, information from the record review was independent of the DAB. A standardized Record Review form was used and interrater reliability was assessed by having two raters complete independent Record Review forms on 117 subjects. Only Record Review items with sufficient variability in the sample (i.e., frequency of occurrence greater than 10% and less than 90%) and statistically significant kappa coefficients (Fleiss, 1971) indicating acceptable reliability were selected for study as possible DAB correlates. Exceptions to this rule were made for the bizarre behavior categories and history of few or no friends. These infrequent categories were considlered to represent such
Downloaded by [University of New Hampshire] at 00:56 09 February 2015
78
WILLIAMS, BEN-PORATH, UCHIYAMA, WEED, ARCHER
important potential descriptors that they were included despite their infrequency. As noted by Williams and Butcher (1989a)' however, although the Record Review variables demonstrated significant interrater reliability, the magnitude of some of the significant kappas was low (e.g., .25). This relatively low reliability, coupled with the infrequency of some of the Record Review variables, may contribute to lower correlation coefficients. Williams and Butcher (1989a) present further information about the Record Review. The CBCLs were administered to the adolescents' parents by treatment staff at admission after the parent signed an informed consent form. The CBCL was modified slightly with the author's permission (T. M. Achenbach, personal communication,January 15,1985)to make it more appropriate for older, as well as younger, adolescents. The computer scoring program provided by the author was used in this study and because norms were not available for subjects aged 17to 18-years-old,the norms for 12- to 16-year-olds were used in the analyses for all subjects, as suggested in the manual (Achenbach & Edelbrock, 1983). CBCLs were collected from 284 boys and 200 girls. They were not available for the day treatment subjects, as that staff was reluctant to request them from parents. Consistent with the earlier discussion, CBCLs could not be obtained from parents of 29% of the residential samples, underscoring the need for an assessment tool that does not require the cooperation of parents of adolescents in residential settings. Neither parents, staff, nor research assistants had knowledge of the subjects' DAB scores when completing the CBCL or Record Review.
RESULTS Three sets of analyses were conducted to examine the validity of the new DAB scales. The first set of analyses approached the question of validity by examining whether the new scales could successfully discriminate between two groups of adolescents in treatment for different problems-psychiatric patients and substance abusers. The next two involved computations of correlations between the new scales and the two external sources of data on our subjects-the structured Record Review and the CBCL described earlier.
Group Comparisons Table 2 presents the means and standard deviations for each of the five DAB scales by gender and setting (i.e., substance abuse and psychiatric) and the results of five seperate 2 x 2 analyses of variance (ANOVAs). We decided that a correction for multiple comparisons like the Bonferroni (Grove & Andreasen, 1982) would cause too many Type I1 errors in these analyses that were unwarranted, because we had made a priori hypotheses about the discriminant validity of the DAB scales in these settings. Instead, we used a p 5 .O1 to determine
Downloaded by [University of New Hampshire] at 00:56 09 February 2015
TABLE 2 Means, Standard Deviations, and ANOVA Results of Substance Abuse and Psychiatric Patients on the New DAB Scales Boys
Girls
Substance Abusea
Psychiatricb
Substance Abusec
DAB Scales
M
SD
M
SD
M
AOB 'WTB PB NDB HI
81.50 41.31 23.16 23.27 16.34
21.52 13.65 5.57 7.16 6.60
73.86 41.11 26.18 19.10 16.03
23.93 15.18 7.52 6.14 5.62
75.51 41.50 23.86 24.47 21.80
SD 19.65 13.87 6.72 6.21 7.44
~sychiatric~
M
SD
64.83 44.86 22.74 20.32 18.83
Journal of Personality Assessment Publication details, including instructions for authors and subscription information: http://www.tandfonline.com/loi/hjpa20
External Validity of the New Devereux Adolescent Behavior Rating Scales Carolyn L. Williams , Yossef S. Ben-Porath , Craig Uchiyama , Nathan C. Weed & Robert P. Archer Published online: 22 Jun 2011.
To cite this article: Carolyn L. Williams , Yossef S. Ben-Porath , Craig Uchiyama , Nathan C. Weed & Robert P. Archer (1990) External Validity of the New Devereux Adolescent Behavior Rating Scales, Journal of Personality Assessment, 55:1-2, 73-85, DOI: 10.1080/00223891.1990.9674048 To link to this article: http://dx.doi.org/10.1080/00223891.1990.9674048
PLEASE SCROLL DOWN FOR ARTICLE Taylor & Francis makes every effort to ensure the accuracy of all the information (the “Content”) contained in the publications on our platform. However, Taylor & Francis, our agents, and our licensors make no representations or warranties whatsoever as to the accuracy, completeness, or suitability for any purpose of the Content. Any opinions and views expressed in this publication are the opinions and views of the authors, and are not the views of or endorsed by Taylor & Francis. The accuracy of the Content should not be relied upon and should be independently verified with primary sources of information. Taylor and Francis shall not be liable for any losses, actions, claims, proceedings, demands, costs, expenses, damages, and other liabilities whatsoever or howsoever caused arising directly or indirectly in connection with, in relation to or arising out of the use of the Content. This article may be used for research, teaching, and private study purposes. Any substantial or systematic reproduction, redistribution, reselling, loan, sub-licensing, systematic supply, or distribution in any form to anyone is expressly forbidden. Terms & Conditions of access and use can be found at http://www.tandfonline.com/page/terms-and-conditions
JOURNAL OF PERSONALITY ASSESSMENT, 1990, 55(1&2),73-85 Copyright o 1990, Lawrence Erlbaum Associates, Inc.
External Validity of the New Devereux Adolescent Behavior Rating Scales Carolyn L. Williams, Yossef S. Ben-Porath, Craig Uchiyama, and Nathan C. Weed Downloaded by [University of New Hampshire] at 00:56 09 February 2015
University of Minnesota
Robert P. Archer Eastern Virginia Medical School
The discriminant and concurrent validity of the five new scales for the Devereux Adolescent Behavior Rating Scale (DAB) was explored using a heterogeneous sample of psychiatric and substance abuse patients. Consistent with predictions, the substance abuse patients scored higher on the Acting Out Behaviors (AOB:) and Heterosexual Interests (HI) scales, and psychiatric patients scored higher on the Psychotic Behaviors scale. Gender differences also were found, including boys being rated higher on Acting Out Behaviors, and girls higher on Heterosexuall Interests. The new DAB scales demonstrated sufficient concurrent validity using aL thorough record review and a parent rating scale (the Child Behavior Checklist [CBCL]). The NeurotidDependent Behaviors scale (NDB) showed a consistent relationship with substance abuse and several other measures of more externalizing behaviors, in addition to the predicted relationshipswith anxious, tense, and dependent behaviors. The WithdrawdTimid Behaviors scale (WTB) proved to be a purer measure of internalizing behaviors in both sexes.
The DAB was developed during the 1960s by Spivack and colleagues (Spivajck, Haimes, & Spotts, 1967; Spivack & Spotts, 1967) to assess behavior problems occurring in adolescents in residential treatment settings and to provide relevant prognostic and treatment information. Unlike other behavior rating scales for adolescents that rely on parents' reports, for example, the Personality Inventory for Children (PIC; Wia, Lachar, Klinedinst, & Seat, 1977) or the Louisville Behavior Checklist (LBC; Miller & Roid, 1988),the DAB uses treatment staff to provide ratings and can be used for older (i.e., 17 to 18 year olds), as well as younger, adolescents. This can be an advantage in residential settings where parents may not be available to provide ratings and where older adolescents may
Downloaded by [University of New Hampshire] at 00:56 09 February 2015
74
WILLIAMS, BEN-PORATH, UCHIYAMA, WEED, ARCHER
be placed. Because of its potential usefulness in contemporary adolescent treatment settings, we began a series of studies on the DAB in conjunction with a larger project on the assessment of psychopathology in adolescents (Williams & Butcher, 1989a, 198913). Our first study of the DAB used a contemporary sample of adolescents hospitalized for substance abuse (Williams, Ben-Porath, & Weed, in press) and revealed low internal consistencies, as measured by Cronbach's (1951) coefficient alpha, for 9 of the 15 original DAB scales developed by Spivack and colleagues (Spivack et al., 1967; Spivack & Spotts, 1967). The low internal consistencies, plus the finding of high intercorrelations for many of the DAB original scales, led us to develop broadband scales measuring acting out behaviors, psychotic behaviors, attention-seekindexpansive behaviors, and inner turrnoil/frailty. The broadband scales were developed using a principal components analysis of the 15 original DAB scales. A second study with a more heterogeneous adolescent treatment sample (i.e., the 404 substance abuse subjects from Williams et al., in press, plus an additional 333 subjects in psychiatric treatment) found that only two of the four higher order scales (i.e., acting out behaviors and psychotic behdviors), previously developed in the substance abuse sample, replicated adequately in the heterogeneous sample (Ben-Porath, Williams, & Uchiyama, 1989). Consequently, an item-levelfactor analysis was used to develop five new scales for the DAB: AOB, PB, WTB, NDB, a d HI, Following the demonstration of cross-gender factorial comparability in separate analyses by gender, we used a combined-gender sample for our item-level factor analysis to derive the most stable and representative structure. Both principal-components and principal-factor analyses revealed identical structures (Ben-Porath et al., 1989). Our new scales were more internally consistent than the original factorially derived DAB scales (alphas of the new scales range from .80 to .95, M = .87; alphas ofthe old scales range from .44 to ,90, M = .75) and their item content was congruent with the narrow- and broadband dimensions of adolescent psychopathology identified in previous studies using other rating scales (Ben-Porath et al., 1989). Our study explores aspects of the new DAB scales' external validity in our large adolescent clinical sample. Our first look at the scales' validity examines discriminant validity by determining whether two groups of adolescents in treatment for different problems (i.e., substance abuse and ~ s ~ c h i a t rdisorders) ic score differently on particular DAB scales. We hypothesize that for both sexes the mean values of the DAB acting out scales (i.e., AOB and HI) are greater in the substance abuse sample and that the PB scale are higher in the ~ s ~ c h i a t r i c patients. Concurrent validity of the new DAB scales is examined in two ways, using a standardized Record Review form (Williams & Butcher, 1989a) completed by research assistants, and parents' ratings on the CBCL (Achenbach 6. Edelbrock, 1983). Items from the Record Review form are included in the
Downloaded by [University of New Hampshire] at 00:56 09 February 2015
VALIDITY OF NEW DAB SCALES
75
validity analyses on the basis of their hypothesized relation to our new DlAB scales. Our specific hypotheses are that the AOB scale is related to the following Record Review items: acting out/irresponsible behzvior, anger outbursts/ fighting, oppositional behavior, running away, drug use/abuse, impulsivity, history of assaultive behavior, and juvenile court involvement for various offenses; the PB scale is related to bizarre behavior, bizarre sensory experiences, and bizarre thought processes/delusions; the WTB scale is related to social withdrawal, low self-esteem, depression, suicide ideatiordgestures, and having few or no friends; the NDB scale is related to tension/nervousness and somatic complaints; and the HI scale is related to being sexually active and sexual acting out. We believe that these predictions hold for both boys and girls. Our hypotheses concerning the relations between DAB and CBCL scales are as follows: for boys, the CBCL scales measuring acting out (i.e., Immature, Delinquent, and Aggressive) are related to the DAB AtOB and HI scales; the CBCL Schizoid scale is related to the DAB PB scale; the CBCL Somatic and Obsessive/Compulsive scales are related to the DAB NBD scale; and the CBCL Uncommunicative scale is related to the DAB Q l l 3 scale. Similar predictiions are made for girls: CBCL Delinquent, Aggressive, and Cruel scales are related to the DAB AOB and HI scales; the CBCL Schizoid scale ir; related to the DAB PB scale; the CBCL Anxious/Obsessive and Somatic scales are related to DAB NDB scale; the CBCL DepressecUWithdrawn scale is related to DAB WTB scale. In making these predictions we are aware that not all the hypothesized CBCL-DAB relationships might be found. As mentioned earlier, the CBCL was developed and normed for younger adolescents (i.e., upper limit of 16 years) and our sample included 17- and 18-year-olds. And, as youths mature, solme forms of psychopathology (e.g., depressed/anxious cogr~itionsor feelings) may not be as easily observed by others. Nevertheless, we believe it is important to present information about the new DAB scales' relationship to a more established behavior rating scale measuring similar constructs.
METHOD Subjects The sample of 737 adolescents (420 boys, 317 girls) describled by Ben-Porath et al. (1989) served as subjects in this study. This sample was drawn from three types of adolescent treatment facilities: inpatient substance abuse units, inpatient psychiatric units, and day-treatment psychiatric facilities. The substance abuse sires consisted of two unlocked units at two separate ho'spitals in Minneapolis that were designed for long-term residential treatment. Three inpatient psychiatric units were included in the study, two located in one hospital in Minnr-lp-
Downloaded by [University of New Hampshire] at 00:56 09 February 2015
76
WILLIAMS, BEN-PORATH, UCHIYAMA, WEED, ARCHER
olis and one in Norfolk, VA. The Minneapolis sites consisted of a short-term crisis intervention unit, focused primarily on assessment and evaluation, and a longer term psychiatric treatment unit that primarily emphasized therapeutic intervention. The Norfolk inpatient psychiatric unit was designed for acute psychiatric care. The Zday treatment facilities, located in Minneapolis, functioned as school alternatives for emotionally and behaviorally disturbed adolescents. Given the number of different community-based substance abuse and psychiatic units sampled, results from this study would likely generalize to other adolescents in similar units. Criteria for inclusion in the study were that the individual was 13 to 18 years of age, had a treatment stay of sufficient length to allow an accurate evaluation of the patient by staff, was not tested at another data collection site, had parental consent to participate, and had their physicians' approval, Of the 1,283 adolescents admitted to the facilities during data collection, 959 were eligible for inclusion in the study, resulting in an eligibility rate of 75%. Data were collected on 737 of the eligible adolescents, producing a 77% response rate. Most nanrespondents were due to administrative errors (i.e., failing to record new admissions on the study roster, determining a DAB was missing too long past discharge for staff to make an accurate assessment). The total sample ranged in age from 13 to 18 years (M = 15,5, SD = 1.3). Subjects were primarily White (84%), with 7% Native American, 5% Black, 1% Hispanic, 1% Asian American, and 2% unknown. Table 1 presents a demographic breakdown by site. Chi-square analyses were used to determine if there were demographic differences in subjects at the different treatment sites. These results revealed that the substance abuse sample had more boys than the psychiatric sample, more older subjects than the inpatient psychiatric or the day treatment psychiatric samples, and fewer Blacks, Hispanics, and Asian Americans (these frequencies were collapsed into one category for the analysis) than the other sites. The day treatment sites had significantly fewer Whites and proportionately more younger subjects than the other units. The psychiatric inpatient units had fewer Native Americans. These demographic differences by site did not lead us to change our predictions about the discriminant validity of the DAB scales.
Procedure The DAB was completed by psychiatric nurses, master's-level psychologists, or chemical-dependency counselors, depending on the setting. Between 12 and 15 raters participated at each site. These raters were not responsible for determining whether the subject was to be admitted to a psychiatric or substance abuse unit. Raters were given specific instructions on how and when to complete the DAB (as specified in the manual), and research assistants coordinated, monitored, and helped resolve any difficulties encountered in data collection.
77
VALIDITY OF NEW DAB SCALES TABLE 1 Breakdown of Gender, Age, and Race by Setting Male
Gender
Substance abuse Psychiatric: Inpatient Day treatment Total
Downloaded by [University of New Hampshire] at 00:56 09 February 2015
Age Substance abuse Psychiatric: Inpatient Day treatment Total Race
Substance abuse Psychiatric: Inpatient Day treatment Total
Female
Total
-
267
137
404
133 20 420
149 31 317
282 51 737
13
14
15
16
17
18
Total
6
43
84
132
114
25
404
42 18 66
54 12 109
69 17 170
68 3 203
42 1 157
7 0 32
282 51 737
Asian
Black
Hispanic
Native American
White
Unknown
Total
3
14
6
35
346
0
404
4
21 6 41
5 0 11
7 11 53
237 34 617
8 0 8
282 51 737
0
7
-
Some of the psychiatric inpatients were hospitalized fewer than the 14 days suggested by the DAB manual. In such cases, ratings were obtained only when stafffelt confident of ~rovidingvalid information. This ~ r i m a r ioccurred l~ in the Minneapolis crisis intervention unit, and made up approximately 13% of the total sample. Record review data were completed on 96% of the subjects (n = 347 boys, n = 257 girls) from the Minneapolis treatment facilities by a team of nine research assistants who met weekly to insure consistency in their reviews. Although1 an individual DAB rater could write observations about a subject's behavior during his or her shift in the hospital record, usually in the nurse's daily section, an individual rater's impressions comprised only a small fraction of the patient's total hospital record. Thus, information from the record review was independent of the DAB. A standardized Record Review form was used and interrater reliability was assessed by having two raters complete independent Record Review forms on 117 subjects. Only Record Review items with sufficient variability in the sample (i.e., frequency of occurrence greater than 10% and less than 90%) and statistically significant kappa coefficients (Fleiss, 1971) indicating acceptable reliability were selected for study as possible DAB correlates. Exceptions to this rule were made for the bizarre behavior categories and history of few or no friends. These infrequent categories were considlered to represent such
Downloaded by [University of New Hampshire] at 00:56 09 February 2015
78
WILLIAMS, BEN-PORATH, UCHIYAMA, WEED, ARCHER
important potential descriptors that they were included despite their infrequency. As noted by Williams and Butcher (1989a)' however, although the Record Review variables demonstrated significant interrater reliability, the magnitude of some of the significant kappas was low (e.g., .25). This relatively low reliability, coupled with the infrequency of some of the Record Review variables, may contribute to lower correlation coefficients. Williams and Butcher (1989a) present further information about the Record Review. The CBCLs were administered to the adolescents' parents by treatment staff at admission after the parent signed an informed consent form. The CBCL was modified slightly with the author's permission (T. M. Achenbach, personal communication,January 15,1985)to make it more appropriate for older, as well as younger, adolescents. The computer scoring program provided by the author was used in this study and because norms were not available for subjects aged 17to 18-years-old,the norms for 12- to 16-year-olds were used in the analyses for all subjects, as suggested in the manual (Achenbach & Edelbrock, 1983). CBCLs were collected from 284 boys and 200 girls. They were not available for the day treatment subjects, as that staff was reluctant to request them from parents. Consistent with the earlier discussion, CBCLs could not be obtained from parents of 29% of the residential samples, underscoring the need for an assessment tool that does not require the cooperation of parents of adolescents in residential settings. Neither parents, staff, nor research assistants had knowledge of the subjects' DAB scores when completing the CBCL or Record Review.
RESULTS Three sets of analyses were conducted to examine the validity of the new DAB scales. The first set of analyses approached the question of validity by examining whether the new scales could successfully discriminate between two groups of adolescents in treatment for different problems-psychiatric patients and substance abusers. The next two involved computations of correlations between the new scales and the two external sources of data on our subjects-the structured Record Review and the CBCL described earlier.
Group Comparisons Table 2 presents the means and standard deviations for each of the five DAB scales by gender and setting (i.e., substance abuse and psychiatric) and the results of five seperate 2 x 2 analyses of variance (ANOVAs). We decided that a correction for multiple comparisons like the Bonferroni (Grove & Andreasen, 1982) would cause too many Type I1 errors in these analyses that were unwarranted, because we had made a priori hypotheses about the discriminant validity of the DAB scales in these settings. Instead, we used a p 5 .O1 to determine
Downloaded by [University of New Hampshire] at 00:56 09 February 2015
TABLE 2 Means, Standard Deviations, and ANOVA Results of Substance Abuse and Psychiatric Patients on the New DAB Scales Boys
Girls
Substance Abusea
Psychiatricb
Substance Abusec
DAB Scales
M
SD
M
SD
M
AOB 'WTB PB NDB HI
81.50 41.31 23.16 23.27 16.34
21.52 13.65 5.57 7.16 6.60
73.86 41.11 26.18 19.10 16.03
23.93 15.18 7.52 6.14 5.62
75.51 41.50 23.86 24.47 21.80
SD 19.65 13.87 6.72 6.21 7.44
~sychiatric~
M
SD
64.83 44.86 22.74 20.32 18.83
