Psychiatry and Clinical Neurosciences 2014; 68: 759–767
doi:10.1111/pcn.12188
Regular Article
Reliability and validity of Brief Problem Monitor, an abbreviated form of the Child Behavior Checklist Brian J. Piper,
PhD,1,2*
Hilary M. Gray,
MS,2,4
Jacob Raber,
PhD2,3
and Melissa A. Birkett,
PhD5
1
Department of Basic Pharmaceutical Sciences, Husson University, Maine, 2Department of Behavioral Neuroscience, Departments of Neurology & Radiation Medicine, Oregon Health and Science University, 4Department of Counselor Education, Portland State University, Oregon, and 5Department of Psychology, Northern Arizona University, Arizona, USA 3
Aim: The parent form of the 113-item Child Behavior Checklist (CBCL) is widely utilized by child psychiatrists and psychologists. This report examines the reliability and validity of a recently developed abbreviated version of the CBCL, the Brief Problem Monitor (BPM). Methods: Caregivers (n = 567) completed the CBCL online and the 19 BPM items were examined separately. Results: Internal consistency of the BPM was high (Cronbach’s alpha = 0.91) and satisfactory for the Internalizing (0.78), Externalizing (0.86), and Attention (0.87) scales. High correlations between the CBCL and BPM were identified for the total score (r = 0.95) as well as the Internalizing (0.86), Externalizing (0.93), and Attention (0.97) scales. The BPM and scales were sensitive and identified significantly higher behavioral and emotional problems among children whose caregiver reported a psychiatric diagnosis of attention-deficit hyperactivity disorder, bipolar disorder, depression, anxiety, developmental
HE CHILD BEHAVIOR Checklist (CBCL) is a caregiver-completed questionnaire of child behavioral and emotional problems that is standardized, objective, and widely utilized by child psychiatrists, pediatricians, developmental psychologists, and other mental health professionals for clinical and research purposes.1 The CBCL has been revised since
T
*Correspondence: Brian J. Piper, PhD, Department of Basic Pharmaceutical Sciences, One College Circle, Husson University, Bangor, ME 04401, USA. Email: [email protected] Received 26 August 2013; revised 22 February 2014; accepted 11 April 2014.
disabilities, or autism spectrum disorders relative to a comparison group that had not been diagnosed with these disorders. BPM ratings also differed by the socioeconomic status and education of the caregiver. Mothers with higher annual incomes rated their children as having 38.8% fewer total problems (Cohen’s d = 0.62) as well as 42.8% lower Internalizing (d = 0.53), 44.1% less Externalizing (d = 0.62), and 30.9% decreased Attention (d = 0.39). A similar pattern was evident for maternal education (d = 0.30–0.65).
Conclusion: Overall, these findings provide strong psychometric support for the BPM, although the differences based on the characteristics of the parent indicate that additional information from other sources (e.g., teachers) should be obtained to complement parental reports. Key words: adolescents, anxiety, children, reliability, validity.
its original introduction by Thomas Achenbach, PhD, and colleagues and there are various forms of the instrument available depending on the information source (parent, teacher, and self-report), language of the respondent, and child age (preschool or school age). The CBCL/6–18 parent-report has high test– retest reliability, internal consistency, criterion validity and shows good agreement between maternal and paternal ratings.2–6 The influence of differences in the socioeconomic environment for CBCL ratings is a matter of some contention.3,7,8 Various investigators are in general agreement that children living in more economically disadvantaged households are rated as
© 2014 The Authors Psychiatry and Clinical Neurosciences © 2014 Japanese Society of Psychiatry and Neurology
759
760 B. J. Piper et al.
Psychiatry and Clinical Neurosciences 2014; 68: 759–767
having more problems but the magnitude of this effect has been reported as either modest9 or, more commonly, as sizable.10–12 One potential concern with the CBCL/6–18 is its length. There are 113 problem items that take approximately 10 min to complete and the optional competence items require another 5–10 min. The development of an abbreviated version of the CBCL offers several advantages. First, a shorter form allows clinicians to more easily determine the potential presence of behavioral and emotional problems. This could involve repeated measurement of a single child to assess the utility of a pharmacological or psychoeducational intervention. Second, this instrument could provide non-specialists with a screening tool to identify children and adolescents for whom follow up with a specialist is appropriate (e.g. for a neurodevelopmental disorder, schizophrenia, bipolar disorder). Third, a shortened version would be useful for investigators interested in learning about individual difference in emotion and behavioral functioning but who view the full-length form as prohibitively long within the context of multiple data collection demands (e.g. an epidemiological neurogenetics study). The Brief Problem Monitor (BPM) parent form was introduced in 2011 by Achenbach and coworkers and can be completed in only 2 min as it consists of only 19 items (Appendix A) from the CBCL that form scales for Internalizing, Externalizing, and Attention.13 The BPM is a modification and extension of the 12-item Brief Problem Checklist (BPC).14 The six items that form the Internalizing and Externalizing scales of the BPC were identified via application of item response theory and factor analysis to a large archival CBCL dataset. Three-month test–retest reliabilities of the BPC when the instrument was administered in an interview format were satisfactory (r = 0.73–0.82). An advantage of the BPM over the BPC is that the former contains additional items to assess attention. The prevalence of attention-deficit hyperactivity disorder (ADHD, 5%)15 is sufficiently high to warrant the addition of the six additional items to the BPC for the BPM (Table 2 of the report by Chorpita et al.14 contains a complete list of BPC items). The BPM13 also includes an additional Externalizing item on disobedience. Therefore, the objective of the present report was to build and expand upon the earlier work and examine several psychometric properties of the BPM, including internal consistency and validity, similarities with the CBCL, and also to determine if there are any indi-
vidual differences associated with demographic variables (e.g. education of the respondent).
METHODS Procedures Research Electronic Data Capture (REDCap), version 1.3.9, a web-based program for obtaining online databases, which has multiple safeguards for protecting personal information,16 administered the anonymous survey. Flyers were posted throughout the Oregon Health and Science University (OHSU) and Doernbecher Children’s hospitals, the Portland, Oregon metro area, and western Oregon and western Washington (e.g. coffee shops, laundromats, grocery stores). Links to the investigation were placed on the community and volunteer sections of Craigslist (craigslist.org) in the USA and Canada for 1 year. The Institutional Review Boards of OHSU and Northern Arizona University approved all procedures, which were in accordance with the Declaration of Helsinki.
Measures After completing an online consent to complete this investigation, caregivers began the survey, which took approximately 20 min to finish. Items on the first half (Appendix B) included maternal and child demographics (age, sex, ethnicity, education and household income). ‘Has your child ever been diagnosed with any of the following?’ was listed with 28 options, including common psychiatric and medical conditions (e.g. ADHD, autism spectrum disorder [ASD], attachment disorder, bipolar disorder, developmental disorder, diabetes, schizophrenia). Data about paternal characteristics were not obtained in this initial study due to a desire to minimize the survey completion time for each participant. Information from items about maternal drug use and child outcomes is presented elsewhere.17 The last half of the survey consisted of the CBCL. Caregivers were instructed to rate over the past half-year whether an item (e.g. inattentive) was not, somewhat, or very true about their child with scores of 0, 1, or 2 points.
Data analysis All analyses were completed in Systat, v 13.0 (Chicago, IL, USA). Exclusionary criteria included omitting items about the child age and sex; child age
© 2014 The Authors Psychiatry and Clinical Neurosciences © 2014 Japanese Society of Psychiatry and Neurology
Psychiatry and Clinical Neurosciences 2014; 68: 759–767
Brief Problem Monitor 761
(>18 or
doi:10.1111/pcn.12188
Regular Article
Reliability and validity of Brief Problem Monitor, an abbreviated form of the Child Behavior Checklist Brian J. Piper,
PhD,1,2*
Hilary M. Gray,
MS,2,4
Jacob Raber,
PhD2,3
and Melissa A. Birkett,
PhD5
1
Department of Basic Pharmaceutical Sciences, Husson University, Maine, 2Department of Behavioral Neuroscience, Departments of Neurology & Radiation Medicine, Oregon Health and Science University, 4Department of Counselor Education, Portland State University, Oregon, and 5Department of Psychology, Northern Arizona University, Arizona, USA 3
Aim: The parent form of the 113-item Child Behavior Checklist (CBCL) is widely utilized by child psychiatrists and psychologists. This report examines the reliability and validity of a recently developed abbreviated version of the CBCL, the Brief Problem Monitor (BPM). Methods: Caregivers (n = 567) completed the CBCL online and the 19 BPM items were examined separately. Results: Internal consistency of the BPM was high (Cronbach’s alpha = 0.91) and satisfactory for the Internalizing (0.78), Externalizing (0.86), and Attention (0.87) scales. High correlations between the CBCL and BPM were identified for the total score (r = 0.95) as well as the Internalizing (0.86), Externalizing (0.93), and Attention (0.97) scales. The BPM and scales were sensitive and identified significantly higher behavioral and emotional problems among children whose caregiver reported a psychiatric diagnosis of attention-deficit hyperactivity disorder, bipolar disorder, depression, anxiety, developmental
HE CHILD BEHAVIOR Checklist (CBCL) is a caregiver-completed questionnaire of child behavioral and emotional problems that is standardized, objective, and widely utilized by child psychiatrists, pediatricians, developmental psychologists, and other mental health professionals for clinical and research purposes.1 The CBCL has been revised since
T
*Correspondence: Brian J. Piper, PhD, Department of Basic Pharmaceutical Sciences, One College Circle, Husson University, Bangor, ME 04401, USA. Email: [email protected] Received 26 August 2013; revised 22 February 2014; accepted 11 April 2014.
disabilities, or autism spectrum disorders relative to a comparison group that had not been diagnosed with these disorders. BPM ratings also differed by the socioeconomic status and education of the caregiver. Mothers with higher annual incomes rated their children as having 38.8% fewer total problems (Cohen’s d = 0.62) as well as 42.8% lower Internalizing (d = 0.53), 44.1% less Externalizing (d = 0.62), and 30.9% decreased Attention (d = 0.39). A similar pattern was evident for maternal education (d = 0.30–0.65).
Conclusion: Overall, these findings provide strong psychometric support for the BPM, although the differences based on the characteristics of the parent indicate that additional information from other sources (e.g., teachers) should be obtained to complement parental reports. Key words: adolescents, anxiety, children, reliability, validity.
its original introduction by Thomas Achenbach, PhD, and colleagues and there are various forms of the instrument available depending on the information source (parent, teacher, and self-report), language of the respondent, and child age (preschool or school age). The CBCL/6–18 parent-report has high test– retest reliability, internal consistency, criterion validity and shows good agreement between maternal and paternal ratings.2–6 The influence of differences in the socioeconomic environment for CBCL ratings is a matter of some contention.3,7,8 Various investigators are in general agreement that children living in more economically disadvantaged households are rated as
© 2014 The Authors Psychiatry and Clinical Neurosciences © 2014 Japanese Society of Psychiatry and Neurology
759
760 B. J. Piper et al.
Psychiatry and Clinical Neurosciences 2014; 68: 759–767
having more problems but the magnitude of this effect has been reported as either modest9 or, more commonly, as sizable.10–12 One potential concern with the CBCL/6–18 is its length. There are 113 problem items that take approximately 10 min to complete and the optional competence items require another 5–10 min. The development of an abbreviated version of the CBCL offers several advantages. First, a shorter form allows clinicians to more easily determine the potential presence of behavioral and emotional problems. This could involve repeated measurement of a single child to assess the utility of a pharmacological or psychoeducational intervention. Second, this instrument could provide non-specialists with a screening tool to identify children and adolescents for whom follow up with a specialist is appropriate (e.g. for a neurodevelopmental disorder, schizophrenia, bipolar disorder). Third, a shortened version would be useful for investigators interested in learning about individual difference in emotion and behavioral functioning but who view the full-length form as prohibitively long within the context of multiple data collection demands (e.g. an epidemiological neurogenetics study). The Brief Problem Monitor (BPM) parent form was introduced in 2011 by Achenbach and coworkers and can be completed in only 2 min as it consists of only 19 items (Appendix A) from the CBCL that form scales for Internalizing, Externalizing, and Attention.13 The BPM is a modification and extension of the 12-item Brief Problem Checklist (BPC).14 The six items that form the Internalizing and Externalizing scales of the BPC were identified via application of item response theory and factor analysis to a large archival CBCL dataset. Three-month test–retest reliabilities of the BPC when the instrument was administered in an interview format were satisfactory (r = 0.73–0.82). An advantage of the BPM over the BPC is that the former contains additional items to assess attention. The prevalence of attention-deficit hyperactivity disorder (ADHD, 5%)15 is sufficiently high to warrant the addition of the six additional items to the BPC for the BPM (Table 2 of the report by Chorpita et al.14 contains a complete list of BPC items). The BPM13 also includes an additional Externalizing item on disobedience. Therefore, the objective of the present report was to build and expand upon the earlier work and examine several psychometric properties of the BPM, including internal consistency and validity, similarities with the CBCL, and also to determine if there are any indi-
vidual differences associated with demographic variables (e.g. education of the respondent).
METHODS Procedures Research Electronic Data Capture (REDCap), version 1.3.9, a web-based program for obtaining online databases, which has multiple safeguards for protecting personal information,16 administered the anonymous survey. Flyers were posted throughout the Oregon Health and Science University (OHSU) and Doernbecher Children’s hospitals, the Portland, Oregon metro area, and western Oregon and western Washington (e.g. coffee shops, laundromats, grocery stores). Links to the investigation were placed on the community and volunteer sections of Craigslist (craigslist.org) in the USA and Canada for 1 year. The Institutional Review Boards of OHSU and Northern Arizona University approved all procedures, which were in accordance with the Declaration of Helsinki.
Measures After completing an online consent to complete this investigation, caregivers began the survey, which took approximately 20 min to finish. Items on the first half (Appendix B) included maternal and child demographics (age, sex, ethnicity, education and household income). ‘Has your child ever been diagnosed with any of the following?’ was listed with 28 options, including common psychiatric and medical conditions (e.g. ADHD, autism spectrum disorder [ASD], attachment disorder, bipolar disorder, developmental disorder, diabetes, schizophrenia). Data about paternal characteristics were not obtained in this initial study due to a desire to minimize the survey completion time for each participant. Information from items about maternal drug use and child outcomes is presented elsewhere.17 The last half of the survey consisted of the CBCL. Caregivers were instructed to rate over the past half-year whether an item (e.g. inattentive) was not, somewhat, or very true about their child with scores of 0, 1, or 2 points.
Data analysis All analyses were completed in Systat, v 13.0 (Chicago, IL, USA). Exclusionary criteria included omitting items about the child age and sex; child age
© 2014 The Authors Psychiatry and Clinical Neurosciences © 2014 Japanese Society of Psychiatry and Neurology
Psychiatry and Clinical Neurosciences 2014; 68: 759–767
Brief Problem Monitor 761
(>18 or
