Families, Systems, & Health 2014, Vol. 32, No. 1, 89 –100
© 2014 American Psychological Association 1091-7527/14/$12.00 DOI: 10.1037/fsh0000014
This document is copyrighted by the American Psychological Association or one of its allied publishers. This article is intended solely for the personal use of the individual user and is not to be disseminated broadly.
Reliability and Normative Data for the Behavioral Health Measure (BHM) in Primary Care Behavioral Health Settings Craig J. Bryan, PsyD, ABPP
Tabatha Blount, PhD
National Center for Veterans Studies, Salt Lake City, Utah, and University of Utah
University of Texas Health Science Center at San Antonio
Kathryn A. Kanzler, PsyD, ABPP
Chad E. Morrow, PsyD, ABPP
Malcolm Grow Medical Clinic, Washington, DC
Hurlburt Field, Valparaiso, Florida
Kent A. Corso, PsyD
Meghan A. Corso, PsyD
NCR Behavioral Health LLC, Springfield, Virginia
U.S. Navy Bureau of Medicine and Surgery, Bethesda, Maryland
Bobbie Ray-Sannerud, PsyD National Center for Veterans Studies, Salt Lake City, Utah The Behavioral Health Measure (BHM) is a brief self-report measure of general psychological distress and functioning developed for the tracking of mental health outcomes in outpatient psychotherapy settings (Kopta & Lowry, 2002). Although the BHM is used in integrated primary care behavioral health clinics, the scale’s psychometric properties have not been evaluated in these settings. The current study investigated the BHM’s psychometric properties, including its factor structure and reliability, and presents normative data from 3 large integrated primary care clinics. Mean scores for each of the BHM’s 4 scales were significantly lower (i.e., more distress) for women than men, with scores being stable across the 3 primary care samples. Confirmatory factor analysis demonstrated adequate fit for the 3-factor and 1-factor models, with fit improving when 3 items were omitted. Internal consistency estimates for the BHM’s 4 scales ranged from adequate to very good (␣ range: .72⫺.93). The 4 scales were highly intercorrelated, suggesting they measure similar constructs. Results suggest a revised, 17-item version of the BHM has adequate structure and reliability estimates, and is appropriate for use in primary care settings. Keywords: behavioral health, primary care, outcome, factor analysis, psychometrics
The Behavioral Health Measure (BHM; CelestHealth Solutions, 2008; Kopta & Lowry, 2002) is a brief, 20-item self-report question-
naire that can be quickly administered (typical completion time is approximately 3 min) in a wide range of clinical settings to assess a pa-
This article was published Online First January 20, 2014. Craig J. Bryan, PsyD, ABPP, National Center for Veterans Studies, Salt Lake City, Utah, and University of Utah; Tabatha Blount, PhD, School of Medicine, University of Texas Health Science Center at San Antonio; Kathryn A. Kanzler, PsyD, ABPP, Malcolm Grow Medical Clinic, Washington, DC; Chad E. Morrow, PsyD, ABPP, Hurlburt Field, Valparaiso, Florida; Kent A. Corso, PsyD, NCR Behavioral Health LLC, Springfield, Virginia; Meghan A. Corso, PsyD, U.S. Navy Bureau of Medicine and Surgery, Bethesda, Maryland;
and Bobbie Ray-Sannerud, PsyD, National Center for Veterans Studies, Salt Lake City, Utah. The views expressed in this article are those of the authors and do not necessarily represent the official position or policy of the U.S. Government, Department of Defense, Department of the Air Force, or the Department of the Navy. Correspondence concerning this article should be addressed to Craig J. Bryan, PsyD, ABPP, 260 South Central Campus Drive, Room 205, Salt Lake City, UT 84105. E-mail: [email protected] 89
This document is copyrighted by the American Psychological Association or one of its allied publishers. This article is intended solely for the personal use of the individual user and is not to be disseminated broadly.
90
BRYAN ET AL.
tient’s mental health status in three proposed domains of mental health: Well-Being, which measures generalized distress, motivation, and energy; Symptoms, which measures psychological symptoms and problems common to many mental health conditions; and Life Functioning, which measures functional impairment across several domains of life including relationships and work/school (Kopta & Lowry, 2002). These three domains are based on the phase model of psychotherapy (Howard, Lueger, Maling, & Martinovich, 1993), which posits that clinical improvement in mental health treatment occurs sequentially, beginning with subjective improvement and increased hopefulness (i.e., enhanced well-being), followed by a reduction in psychological symptoms, and, finally, improvement in day-to-day functioning. The BHM’s 20 items also aggregate to a full-scale Global Mental Health score, which can be used as a global metric of overall mental health functioning. Used clinically, the BHM can be included in an initial assessment of patient functioning as a tool to aid treatment planning and can be administered repeatedly across the course of treatment to track clinical change over time. The BHM’s psychometric properties were previously reported by Kopta and Lowry (2002) in two clinical (i.e., outpatient psychotherapy and college counseling center) and two nonclinical (i.e., community adults and undergraduate students) samples. In this report (Kopta and Lowry, 2002), we found adequate internal consistency (i.e., coefficient alpha) for the four primary subscales of the BHM (␣ ⱖ .72); construct validity, based on moderate to large correlations (rs ⬎ .41) of each subscale with other measures of psychological distress (i.e., the Outcomes Questionnaire⫺45 [OQ-45] [Lambert et al., 2004]; BASIS-32 [Eisen, Dill, & Grob, 1994]; COMPASS [Howard, Brill, Lueger, & O’Mahoney, 1992]); and the ability of the four subscales to differentiate clinical from nonclinical groups to support its use as a clinical tool. The BHM was initially developed for use in traditional outpatient psychotherapy settings to provide a benchmark for considering a patient’s overall mental health status compared with various reference groups, and to track change (or lack thereof) in clinical status across psychotherapy sessions. Over the past few decades, however, primary care medical settings have become the predominant source of mental health care in the United States, with rapid
expansion of utilization by the general population that has far exceeded growth in traditional mental health settings, regardless of the severity of mental health impairment or distress (Wang et al., 2006). In response to this rapidly changing pattern of mental health treatment delivery, mental health providers with specialized training are increasingly being integrated into primary care clinics to collaborate with primary care providers (PCPs; e.g., physicians, nurse practitioners, physician assistants) to improve the detection, diagnosis, and treatment of behavioral and other psychosocial health issues. As mental health services have increasingly expanded into integrated primary care clinics, increased interest in monitoring clinical status for patients receiving integrated services has driven discussions about how best to measure clinical outcomes, especially in light of research findings that measuring and monitoring clinical status across treatment enhances mental health outcomes (Lambert, Harmon, Slade, Whipple, & Hawkins, 2005). Specific to this topic, a key point of discussion involves identifying ways to gather sufficient clinical information while working within a nontraditional behavioral health model. Commonly used measures in primary care include the Patient Health Questionnaire⫺9 (Kroenke, Spitzer, & Williams, 2001) and General Anxiety Disorder 7-item scale (Spitzer, Kroenke, Williams, & Löwe, 2006). But even though these instruments provide a brief assessment of specific clinical problems, they may be overly narrow for use in a medical setting marked by diverse behavioral and psychosocial presentations that affect day-to-day functioning (e.g., general distress, sleep disturbance, chronic pain). Consequently, more general measures of emotional state, health-related behaviors, and daily functioning, such as the OQ-45 (Lambert et al., 2004), may be more clinically useful. However, given the time constraints of primary care settings, abbreviated instruments that can be administered much more quickly than the OQ-45 may be preferred. Because it was designed to assess general distress and life functioning with an average administration time of less than 5 min, the 20item version of the BHM has been implemented by primary care behavioral health providers during the past few years, which has contributed to a number of studies focused on integrated services. For instance, the BHM has been used
This document is copyrighted by the American Psychological Association or one of its allied publishers. This article is intended solely for the personal use of the individual user and is not to be disseminated broadly.
BHM NORMS IN PRIMARY CARE
to model patient improvement in less than four 30-min primary care behavioral health appointments (Bryan, Morrow, & Appolonio, 2009; Bryan, Corso, Corso, Morrow, et al., 2012; Cigrang et al., 2011) and to demonstrate the sustained, long-term effects of brief, short-term primary care behavioral health interventions and treatment (Ray-Sannerud et al., 2012). The sensitivity of the BHM subscales to change, in response to relatively low-intensity behavioral interventions delivered in only a few sessions, has therefore been supported. The BHM has additionally been used to determine predictors of clinical improvement across primary care behavioral health appointments (Bryan, Corso, Corso, Kanzler, et al., 2012; Corso et al., 2012), and has been associated with increased likelihood of patient disclosure of suicidal ideation compared with routine care by PCPs (Bryan, Corso, Rudd, & Cordero, 2008), suggesting potential benefit as a suicide risk screener. Despite the expanded use of the BHM in primary care behavioral health clinics, to date there have been no published studies that evaluate the following aspects of the BHM’s psychometric properties: (a) its proposed threefactor structure, (b) normative data among primary care patients, and (c) possible gender differences. Given continued use of the BHM in primary care behavioral health clinics as well as in studies investigating mental health outcomes in primary care settings (e.g., Bryan, Morrow,
91
& Appolonio, 2009; Cigrang et al., 2011; Corso, Bryan, Morrow, Appolonio, & Dodendorf, 2009), further psychometric examination of its properties in primary care clinics is warranted, especially when considering that medical tests and psychological instruments do not necessarily perform similarly across settings and populations (Kraemer, 1992). The primary aims of the current study were, therefore, to examine the factor structure of the BHM, to evaluate the internal consistency of the measure’s various scales, and to provide normative data from several primary care clinics. Method Participants Descriptive variables for each of the three primary care samples are displayed in Table 1. Each sample was comprised of U.S. Department of Defense beneficiaries (i.e., active duty, military retirees, and family members of both) who were accessing behavioral health consultant (BHC) services within integrated family medicine clinics. As can be seen in Table 1, these three samples were very similar in terms of mean age and distribution of gender and race, although a large amount (56.8%) of race and ethnicity data were missing from the first sample and age data were completely unavailable for the third sample. In general, the samples
Table 1 Demographic Statistics Across Three Primary Care Samples Primary care samples Demographics Gender Male Female Unknown Race Caucasian African American Hispanic/Latino Asian/Pacific Islander Other Unknown Age M (SD) Range
1 (n ⫽ 338)
2 (n ⫽ 1,478)
3 (n ⫽ 1,256)
126 (37.3%) 212 (62.7%) —
519 (35.1%) 863 (58.4%) 96 (6.5%)
480 (38.2%) 738 (58.8%) 38 (3.0%)
120 (35.5%) 19 (5.6%) 1 (0.3%) 3 (0.9%) 1 (0.3%) 192 (56.8%)
793 (53.7%) 202 (13.7%) 213 (14.4%) 35 (2.4%) 48 (3.2%) 187 (12.7%)
550 (43.8%) 126 (10.0%) 197 (15.7%) 38 (3.0%) 29 (3.1%) 316 (25.2%) — — —
35.86 (13.14) 16–73
36.41 (12.23) 14–73
92
BRYAN ET AL.
were predominantly female and White, although adequate gender and racial distribution was observed. Mean age was approximately 35⫺36 years, with an overall range that covered the majority of the life span.
This document is copyrighted by the American Psychological Association or one of its allied publishers. This article is intended solely for the personal use of the individual user and is not to be disseminated broadly.
Procedures For all three primary care samples, participants included consecutive patients referred by their PCPs to BHC services integrated and collocated within two large family medicine clinics located at U.S. Air Force training hospitals in the Southwestern and Southeastern United States. Both clinics serve beneficiaries spanning across the entire life span, including active duty personnel, retired military personnel, and family members. The BHC model of integrated primary care is described in detail elsewhere (e.g., Robinson & Reiter, 2007), but, in general, involves the PCP referring patients to an embedded psychologist (i.e., the BHC) for evaluation and treatment recommendations on identification of psychosocial health issues during routine medical appointments. Patients were referred to the BHC for a wide range of problems in isolation (e.g., depression alone) or comorbid with other problems (e.g., depression, insomnia, pain). In each sample, patients completed the BHM (see next section) at each BHC appointment as a part of routine clinical care. At check-in, patients were given a paper-and-pencil version of the BHM to complete while awaiting the start of their appointment. The BHM was then reviewed by the BHC as a part of the appointment and was used as a tool to augment clinical decision-making. Either during or following the appointment, responses were entered by the BHC into a secured electronic database used to enhance clinical tracking across appointments. Data were extracted from the electronic database and deidentified prior to analysis. The Wilford Hall Medical Center Institutional Review Board reviewed and approved the study as exempt research; informed consent was, therefore, not obtained from patients. Behavioral Health Measureⴚ20. The Behavior Health Measure⫺20 (BHM-20; CelestHealth Solutions, 2008) is a 20-item self-report questionnaire that uses Likert rating scales to assess the three domains of mental health treatment, consistent with the phase model of psychotherapy. The 20 items aggregate to a Global Mental Health (GMH) scale, which is calculated by taking the mean score of all 20 items, to provide an index of
overall health. The BHM also has three subscales developed to correspond to the three phases of change in psychotherapy: first, subjective distress and overall well-being improves, followed by improvement in psychiatric symptoms (e.g., mood disturbance, anxiety), and, finally, improvement in day-to-day functioning (e.g., relationships, socialoccupational performance). The Well-Being (WB) subscale is comprised of three items that assess the intensity of subjective distress, general life satisfaction, and energy and motivation on a Likert scale ranging from 0 (not at all) to 4 (extremely), and corresponds to the first phase of psychotherapy change. The Symptoms (SYM) subscale is comprised of 13 items that assess the frequency of various symptoms of depression and anxiety, substance use, suicidal ideation, and aggressive or violent urges on a Likert scale ranging from 0 (almost always) to 4 (never), and corresponds to the second phase of psychotherapy change. The Life Functioning (LF) subscale is comprised of four items that assess the extent to which the respondent perceives he or she is performing in terms of work or school, intimate relationships, nonfamily social relationships, and overall life enjoyment (e.g., recreational activities) on a scale ranging from 0 (terribly) to 4 (very well). Scores for each subscale are calculated by taking the mean score for all subscale items. The BHM is keyed such that higher scores indicate better health for all four subscales. Kopta and Lowry (2002) have previously reported adequate to good internal consistency for the GMH (␣ ⬎ .89) and SYM (␣ ⬎ .85) subscales, and marginal to moderate internal consistency for the WB (␣ ⬎ .65) and LF (␣ ⬎ .72) subscales. Subscales have also been found to correlate highly with other measures of overall mental health functioning, generalized distress, and psychiatric symptoms. Results Confirmatory Factor Analysis The BHM-20’s initial design included three subscales (WB, SYM, and LF) corresponding to the phase model of psychotherapy, and a second-order global scale indicating overall mental health well-being and functioning. We used confirmatory factor analysis (CFA) with robust maximum likelihood estimation to determine if the BHM’s items conformed to this proposed three-factor model as well as the
This document is copyrighted by the American Psychological Association or one of its allied publishers. This article is intended solely for the personal use of the individual user and is not to be disseminated broadly.
BHM NORMS IN PRIMARY CARE
one-factor model. Because second-order models that are defined by two or three firstorder solutions tend to yield similar fit estimates as the related higher order models (Brown, 2006), we reasoned that formulation of a second-order solution would not be necessary. All analyses were performed with the Mplus, Version 6.12 program (Muthen & Muthen, 1998 –2011). Good fit for each model was assessed using the following criteria: comparative fit index (CFI) greater than .90, Tucker⫺Lewis index (TLI) greater than .90, root mean square error of approximation (RMSEA) less than .08, standardized root mean square residual (SRMR) less than .08, and low Akaike information criterion. As can be seen in Table 2, results of the CFA suggested the BHM-20’s proposed structure was marginally adequate, with the proposed threefactor structure providing a slightly better fit than the one-factor structure in all three samples. All of the standardized item-factor loadings were statistically significant (ps ⬍ .001) for both the threeand the one-factor solutions. Table 3 displays the item-factor loadings across all three samples. All factor loadings exceeded .350, except three items loading onto the SYM scale: Item 5 (“alcohol/ drug use interfering with your performance at
93
school or work”), Item 6 (“wanting to harm someone”), and Item 14 (“alcohol/drug use interfering with your relationships with family and/or friends”). We therefore decided to drop these three items from the SYM subscale and repeated the CFAs for both the three- and the one-factor models. Removing these three items improved overall model fit for both the three- and the one-factor models, with the three-factor model demonstrating a statistically significant better fit than the one-factor model (see Table 2). The revised threefactor scale demonstrated good fit based on the RMSEA (M ⫽ .079) and SRMR values (M ⫽ .053), and demonstrated marginal fit based on the CFI (M ⫽ .881) and TLI (M ⫽ .861) values. Item-factor loadings for the 17 retained items remained above .350. This revised, 17-item version of the BHM-20 will subsequently be referred to as the BHM-17-R, the revised SYM scale will be subsequently be referred to as SYM-17-R, and the revised GMH scale will be subsequently referred to as the GMH-17-R (the WB and LF subscales remain unchanged with the dropped items). The correlations of the original GMH scale with the revised GMH-17-R scale ranged from .841⫺.996, and the correlations of the original SYM subscale with the revised SYM-17-R subscale ranged from .846⫺.989.
Table 2 Fit Statistics for BHM-20 Confirmatory Factor Analyses Across Three Primary Care Samples PCC 1 (n ⫽ 338) Scales Full BHM (20 items) 2 CFI TLI RMSEA [90% CI] SRMR AIC Revised BHM (17 items) 2 CFI TLI RMSEA [90% CI] SRMR AIC
PCC 2 (n ⫽ 1,478)
PCC 3 (n ⫽ 1,256)
Three-factor
One-factor
Three-factor
One-factor
Three-factor
One-factor
675.824ⴱ .801 .773 .095 [.088, .102] .081 16,712.625
787.131ⴱ .758 .730 .104 [.096, .111] .086 16,849.364
1727.660ⴱ .835 .812 .080 [.077, .084] .067 71,858.620
1998.181ⴱ .807 .784 .086 [.083, .089] .071 72,243.476
1591.843ⴱ .831 .807 .083 [.079, .086] .067 64,562.169
1841.052ⴱ .801 .778 .088 [.085, .092] .070 64,913.520
381.002ⴱ .887 .868 .082 [.073, .091] .055 15,279.492
493.568ⴱ .840 .817 .097 [.088, .105] .062 15,408.932
1068.620ⴱ .869 .846 .075 [.070, .080] .054 81,164.190
1137.138ⴱ .860 .840 .076 [.072, .080] .054 81,243.555
1071.410ⴱ .888 .869 .081 [.077, .085] .050 58,378.826
1348.222ⴱ .856 .836 .091 [.086, .095] .054 58,823.103
Note. BHM ⫽ Behavioral Health Measure; PCC ⫽ primary care clinic; CFI ⫽ comparative fit index; TLI ⫽ Tucker⫺Lewis index; RMSEA ⫽ root mean square error of approximation; CI ⫽ confidence interval; SRMR ⫽ standardized root mean square residual; AIC ⫽ Aikaike’s information criterion. ⴱ p ⬍ .001.
94
BRYAN ET AL.
Table 3 Standardized Item Factor Loadings for the BHM-20’s Proposed Three-Factor (Well-Being, Symptoms, Life Functioning) and One-Factor (Global Mental Health) Models PCC 1 Scales WB
This document is copyrighted by the American Psychological Association or one of its allied publishers. This article is intended solely for the personal use of the individual user and is not to be disseminated broadly.
SYM
LF
GMH
Item No. 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20
PCC 2
PCC 3
Estimate
SE
Estimate
SE
Estimate
SE
.695 .760 .618 .610 .223 .326 .709 .645 .491 .448 .851 .828 .674 .261 .634 .540 .600 .744 .782 .849 .682 .722 .586 .590 .206 .306 .691 .658 .472 .429 .840 .797 .661 .241 .616 .517 .574 .670 .692 .749
.040 .033 .046 .040 .062 .061 .030 .035 .045 .048 .021 .024 .034 .065 .041 .048 .044 .034 .033 .020 .036 .031 .044 .041 .052 .056 .031 .034 .044 .046 .020 .026 .036 .048 .040 .046 .044 .036 .041 .031
.693 .736 .666 .574 .185 .260 .725 .659 .483 .380 .857 .806 .667 .201 .651 .548 .569 .688 .768 .863 .696 .727 .656 .555 .176 .254 .712 .662 .478 .363 .843 .789 .656 .198 .631 .533 .544 .637 .694 .786
.017 .015 .018 .021 .034 .029 .015 .018 .022 .026 .010 .012 .017 .030 .019 .023 .022 .017 .015 .010 .016 .014 .017 .021 .033 .028 .015 .017 .021 .026 .009 .012 .017 .029 .019 .022 .022 .018 .017 .012
.679 .745 .655 .615 .206 .303 .729 .648 .479 .449 .848 .811 .647 .238 .655 .562 .531 .699 .775 .848 .675 .728 .639 .595 .198 .293 .716 .653 .467 .434 .838 .793 .642 .234 .639 .547 .522 .637 .682 .757
.018 .016 .021 .021 .028 .027 .015 .018 .024 .023 .011 .013 .019 .029 .021 .023 .025 .019 .016 .012 .018 .015 .020 .021 .026 .026 .015 .018 .023 .022 .010 .013 .019 .027 .020 .023 .023 .019 .019 .015
Distressed Satisfied with life Energetic and motivated Fearful, scared Alcohol/drug use ⫺ school/work Harm someone Not liking yourself Concentrating Eating problem Thoughts of ending life Sad Hopeless Mood swings Alcohol/drug use ⫺ relationships Nervous Heart pounding Work/school Intimate relationships Nonfamily social relationships Life enjoyment Distressed Satisfied with life Energetic and motivated Fearful, scared Alcohol/drug use ⫺ school/work Harm someone Not liking yourself Concentrating Eating problem Thoughts of ending life Sad Hopeless Mood swings Alcohol/drug use ⫺ relationships Nervous Heart pounding Work/School Intimate relationships Nonfamily social relationships Life enjoyment
Note. BHM ⫽ Behavioral Health Measure; PCC ⫽ primary care clinic; WB ⫽ Well-Being subscale; SYM ⫽ Symptoms subscale; LF ⫽ Life Functioning subscale; GMH ⫽ Global Mental Health scale.
Means and Standard Deviations by Gender and Sample Means and standard deviations for all four BHM-20 and BHM-17-R scales were calculated and are displayed in Table 4. Means scores are reported for the original SYM and GMH scales as well as the revised SYM-17-R and GMH-
17-R scales to facilitate comparisons with previously published norms and to identify the effects of item removal on scale scores. Multivariate analysis of variance was used to compare mean scores. We first compared scores according to gender, and found that men scored significantly higher (i.e., “healthier”) than women on all BHM scales in all three primary
BHM NORMS IN PRIMARY CARE
95
Table 4 Mean Scores on BHM-20 and BHM-17-R Scales for Full Sample and by Gender Across Three Primary Care Samples, With Pooled Results Full sample
This document is copyrighted by the American Psychological Association or one of its allied publishers. This article is intended solely for the personal use of the individual user and is not to be disseminated broadly.
Scales GMH PCC 1 PCC 2 PCC 3 Pooled GMH-17-R PCC 1 PCC 2 PCC 3 Pooled WB PCC 1 PCC 2 PCC 3 Pooled SYM PCC 1 PCC 2 PCC 3 Pooled SYM-17-R PCC 1 PCC 2 PCC 3 Pooled LF PCC 1 PCC 2 PCC 3 Pooled
Male
Female
n
M
SD
n
M
SD
n
M
SD
pa
332 1357 1211 2902
2.73 2.70 2.72 2.71
.70 .69 .71 .70
124 512 480 1117
2.95 2.84 2.89 2.87
.62 .68 .68 .67
208 845 731 1785
2.60 2.62 2.61 2.61
.72 .69 .71 .70
⬍.001 ⬍.001 ⬍.001
© 2014 American Psychological Association 1091-7527/14/$12.00 DOI: 10.1037/fsh0000014
This document is copyrighted by the American Psychological Association or one of its allied publishers. This article is intended solely for the personal use of the individual user and is not to be disseminated broadly.
Reliability and Normative Data for the Behavioral Health Measure (BHM) in Primary Care Behavioral Health Settings Craig J. Bryan, PsyD, ABPP
Tabatha Blount, PhD
National Center for Veterans Studies, Salt Lake City, Utah, and University of Utah
University of Texas Health Science Center at San Antonio
Kathryn A. Kanzler, PsyD, ABPP
Chad E. Morrow, PsyD, ABPP
Malcolm Grow Medical Clinic, Washington, DC
Hurlburt Field, Valparaiso, Florida
Kent A. Corso, PsyD
Meghan A. Corso, PsyD
NCR Behavioral Health LLC, Springfield, Virginia
U.S. Navy Bureau of Medicine and Surgery, Bethesda, Maryland
Bobbie Ray-Sannerud, PsyD National Center for Veterans Studies, Salt Lake City, Utah The Behavioral Health Measure (BHM) is a brief self-report measure of general psychological distress and functioning developed for the tracking of mental health outcomes in outpatient psychotherapy settings (Kopta & Lowry, 2002). Although the BHM is used in integrated primary care behavioral health clinics, the scale’s psychometric properties have not been evaluated in these settings. The current study investigated the BHM’s psychometric properties, including its factor structure and reliability, and presents normative data from 3 large integrated primary care clinics. Mean scores for each of the BHM’s 4 scales were significantly lower (i.e., more distress) for women than men, with scores being stable across the 3 primary care samples. Confirmatory factor analysis demonstrated adequate fit for the 3-factor and 1-factor models, with fit improving when 3 items were omitted. Internal consistency estimates for the BHM’s 4 scales ranged from adequate to very good (␣ range: .72⫺.93). The 4 scales were highly intercorrelated, suggesting they measure similar constructs. Results suggest a revised, 17-item version of the BHM has adequate structure and reliability estimates, and is appropriate for use in primary care settings. Keywords: behavioral health, primary care, outcome, factor analysis, psychometrics
The Behavioral Health Measure (BHM; CelestHealth Solutions, 2008; Kopta & Lowry, 2002) is a brief, 20-item self-report question-
naire that can be quickly administered (typical completion time is approximately 3 min) in a wide range of clinical settings to assess a pa-
This article was published Online First January 20, 2014. Craig J. Bryan, PsyD, ABPP, National Center for Veterans Studies, Salt Lake City, Utah, and University of Utah; Tabatha Blount, PhD, School of Medicine, University of Texas Health Science Center at San Antonio; Kathryn A. Kanzler, PsyD, ABPP, Malcolm Grow Medical Clinic, Washington, DC; Chad E. Morrow, PsyD, ABPP, Hurlburt Field, Valparaiso, Florida; Kent A. Corso, PsyD, NCR Behavioral Health LLC, Springfield, Virginia; Meghan A. Corso, PsyD, U.S. Navy Bureau of Medicine and Surgery, Bethesda, Maryland;
and Bobbie Ray-Sannerud, PsyD, National Center for Veterans Studies, Salt Lake City, Utah. The views expressed in this article are those of the authors and do not necessarily represent the official position or policy of the U.S. Government, Department of Defense, Department of the Air Force, or the Department of the Navy. Correspondence concerning this article should be addressed to Craig J. Bryan, PsyD, ABPP, 260 South Central Campus Drive, Room 205, Salt Lake City, UT 84105. E-mail: [email protected] 89
This document is copyrighted by the American Psychological Association or one of its allied publishers. This article is intended solely for the personal use of the individual user and is not to be disseminated broadly.
90
BRYAN ET AL.
tient’s mental health status in three proposed domains of mental health: Well-Being, which measures generalized distress, motivation, and energy; Symptoms, which measures psychological symptoms and problems common to many mental health conditions; and Life Functioning, which measures functional impairment across several domains of life including relationships and work/school (Kopta & Lowry, 2002). These three domains are based on the phase model of psychotherapy (Howard, Lueger, Maling, & Martinovich, 1993), which posits that clinical improvement in mental health treatment occurs sequentially, beginning with subjective improvement and increased hopefulness (i.e., enhanced well-being), followed by a reduction in psychological symptoms, and, finally, improvement in day-to-day functioning. The BHM’s 20 items also aggregate to a full-scale Global Mental Health score, which can be used as a global metric of overall mental health functioning. Used clinically, the BHM can be included in an initial assessment of patient functioning as a tool to aid treatment planning and can be administered repeatedly across the course of treatment to track clinical change over time. The BHM’s psychometric properties were previously reported by Kopta and Lowry (2002) in two clinical (i.e., outpatient psychotherapy and college counseling center) and two nonclinical (i.e., community adults and undergraduate students) samples. In this report (Kopta and Lowry, 2002), we found adequate internal consistency (i.e., coefficient alpha) for the four primary subscales of the BHM (␣ ⱖ .72); construct validity, based on moderate to large correlations (rs ⬎ .41) of each subscale with other measures of psychological distress (i.e., the Outcomes Questionnaire⫺45 [OQ-45] [Lambert et al., 2004]; BASIS-32 [Eisen, Dill, & Grob, 1994]; COMPASS [Howard, Brill, Lueger, & O’Mahoney, 1992]); and the ability of the four subscales to differentiate clinical from nonclinical groups to support its use as a clinical tool. The BHM was initially developed for use in traditional outpatient psychotherapy settings to provide a benchmark for considering a patient’s overall mental health status compared with various reference groups, and to track change (or lack thereof) in clinical status across psychotherapy sessions. Over the past few decades, however, primary care medical settings have become the predominant source of mental health care in the United States, with rapid
expansion of utilization by the general population that has far exceeded growth in traditional mental health settings, regardless of the severity of mental health impairment or distress (Wang et al., 2006). In response to this rapidly changing pattern of mental health treatment delivery, mental health providers with specialized training are increasingly being integrated into primary care clinics to collaborate with primary care providers (PCPs; e.g., physicians, nurse practitioners, physician assistants) to improve the detection, diagnosis, and treatment of behavioral and other psychosocial health issues. As mental health services have increasingly expanded into integrated primary care clinics, increased interest in monitoring clinical status for patients receiving integrated services has driven discussions about how best to measure clinical outcomes, especially in light of research findings that measuring and monitoring clinical status across treatment enhances mental health outcomes (Lambert, Harmon, Slade, Whipple, & Hawkins, 2005). Specific to this topic, a key point of discussion involves identifying ways to gather sufficient clinical information while working within a nontraditional behavioral health model. Commonly used measures in primary care include the Patient Health Questionnaire⫺9 (Kroenke, Spitzer, & Williams, 2001) and General Anxiety Disorder 7-item scale (Spitzer, Kroenke, Williams, & Löwe, 2006). But even though these instruments provide a brief assessment of specific clinical problems, they may be overly narrow for use in a medical setting marked by diverse behavioral and psychosocial presentations that affect day-to-day functioning (e.g., general distress, sleep disturbance, chronic pain). Consequently, more general measures of emotional state, health-related behaviors, and daily functioning, such as the OQ-45 (Lambert et al., 2004), may be more clinically useful. However, given the time constraints of primary care settings, abbreviated instruments that can be administered much more quickly than the OQ-45 may be preferred. Because it was designed to assess general distress and life functioning with an average administration time of less than 5 min, the 20item version of the BHM has been implemented by primary care behavioral health providers during the past few years, which has contributed to a number of studies focused on integrated services. For instance, the BHM has been used
This document is copyrighted by the American Psychological Association or one of its allied publishers. This article is intended solely for the personal use of the individual user and is not to be disseminated broadly.
BHM NORMS IN PRIMARY CARE
to model patient improvement in less than four 30-min primary care behavioral health appointments (Bryan, Morrow, & Appolonio, 2009; Bryan, Corso, Corso, Morrow, et al., 2012; Cigrang et al., 2011) and to demonstrate the sustained, long-term effects of brief, short-term primary care behavioral health interventions and treatment (Ray-Sannerud et al., 2012). The sensitivity of the BHM subscales to change, in response to relatively low-intensity behavioral interventions delivered in only a few sessions, has therefore been supported. The BHM has additionally been used to determine predictors of clinical improvement across primary care behavioral health appointments (Bryan, Corso, Corso, Kanzler, et al., 2012; Corso et al., 2012), and has been associated with increased likelihood of patient disclosure of suicidal ideation compared with routine care by PCPs (Bryan, Corso, Rudd, & Cordero, 2008), suggesting potential benefit as a suicide risk screener. Despite the expanded use of the BHM in primary care behavioral health clinics, to date there have been no published studies that evaluate the following aspects of the BHM’s psychometric properties: (a) its proposed threefactor structure, (b) normative data among primary care patients, and (c) possible gender differences. Given continued use of the BHM in primary care behavioral health clinics as well as in studies investigating mental health outcomes in primary care settings (e.g., Bryan, Morrow,
91
& Appolonio, 2009; Cigrang et al., 2011; Corso, Bryan, Morrow, Appolonio, & Dodendorf, 2009), further psychometric examination of its properties in primary care clinics is warranted, especially when considering that medical tests and psychological instruments do not necessarily perform similarly across settings and populations (Kraemer, 1992). The primary aims of the current study were, therefore, to examine the factor structure of the BHM, to evaluate the internal consistency of the measure’s various scales, and to provide normative data from several primary care clinics. Method Participants Descriptive variables for each of the three primary care samples are displayed in Table 1. Each sample was comprised of U.S. Department of Defense beneficiaries (i.e., active duty, military retirees, and family members of both) who were accessing behavioral health consultant (BHC) services within integrated family medicine clinics. As can be seen in Table 1, these three samples were very similar in terms of mean age and distribution of gender and race, although a large amount (56.8%) of race and ethnicity data were missing from the first sample and age data were completely unavailable for the third sample. In general, the samples
Table 1 Demographic Statistics Across Three Primary Care Samples Primary care samples Demographics Gender Male Female Unknown Race Caucasian African American Hispanic/Latino Asian/Pacific Islander Other Unknown Age M (SD) Range
1 (n ⫽ 338)
2 (n ⫽ 1,478)
3 (n ⫽ 1,256)
126 (37.3%) 212 (62.7%) —
519 (35.1%) 863 (58.4%) 96 (6.5%)
480 (38.2%) 738 (58.8%) 38 (3.0%)
120 (35.5%) 19 (5.6%) 1 (0.3%) 3 (0.9%) 1 (0.3%) 192 (56.8%)
793 (53.7%) 202 (13.7%) 213 (14.4%) 35 (2.4%) 48 (3.2%) 187 (12.7%)
550 (43.8%) 126 (10.0%) 197 (15.7%) 38 (3.0%) 29 (3.1%) 316 (25.2%) — — —
35.86 (13.14) 16–73
36.41 (12.23) 14–73
92
BRYAN ET AL.
were predominantly female and White, although adequate gender and racial distribution was observed. Mean age was approximately 35⫺36 years, with an overall range that covered the majority of the life span.
This document is copyrighted by the American Psychological Association or one of its allied publishers. This article is intended solely for the personal use of the individual user and is not to be disseminated broadly.
Procedures For all three primary care samples, participants included consecutive patients referred by their PCPs to BHC services integrated and collocated within two large family medicine clinics located at U.S. Air Force training hospitals in the Southwestern and Southeastern United States. Both clinics serve beneficiaries spanning across the entire life span, including active duty personnel, retired military personnel, and family members. The BHC model of integrated primary care is described in detail elsewhere (e.g., Robinson & Reiter, 2007), but, in general, involves the PCP referring patients to an embedded psychologist (i.e., the BHC) for evaluation and treatment recommendations on identification of psychosocial health issues during routine medical appointments. Patients were referred to the BHC for a wide range of problems in isolation (e.g., depression alone) or comorbid with other problems (e.g., depression, insomnia, pain). In each sample, patients completed the BHM (see next section) at each BHC appointment as a part of routine clinical care. At check-in, patients were given a paper-and-pencil version of the BHM to complete while awaiting the start of their appointment. The BHM was then reviewed by the BHC as a part of the appointment and was used as a tool to augment clinical decision-making. Either during or following the appointment, responses were entered by the BHC into a secured electronic database used to enhance clinical tracking across appointments. Data were extracted from the electronic database and deidentified prior to analysis. The Wilford Hall Medical Center Institutional Review Board reviewed and approved the study as exempt research; informed consent was, therefore, not obtained from patients. Behavioral Health Measureⴚ20. The Behavior Health Measure⫺20 (BHM-20; CelestHealth Solutions, 2008) is a 20-item self-report questionnaire that uses Likert rating scales to assess the three domains of mental health treatment, consistent with the phase model of psychotherapy. The 20 items aggregate to a Global Mental Health (GMH) scale, which is calculated by taking the mean score of all 20 items, to provide an index of
overall health. The BHM also has three subscales developed to correspond to the three phases of change in psychotherapy: first, subjective distress and overall well-being improves, followed by improvement in psychiatric symptoms (e.g., mood disturbance, anxiety), and, finally, improvement in day-to-day functioning (e.g., relationships, socialoccupational performance). The Well-Being (WB) subscale is comprised of three items that assess the intensity of subjective distress, general life satisfaction, and energy and motivation on a Likert scale ranging from 0 (not at all) to 4 (extremely), and corresponds to the first phase of psychotherapy change. The Symptoms (SYM) subscale is comprised of 13 items that assess the frequency of various symptoms of depression and anxiety, substance use, suicidal ideation, and aggressive or violent urges on a Likert scale ranging from 0 (almost always) to 4 (never), and corresponds to the second phase of psychotherapy change. The Life Functioning (LF) subscale is comprised of four items that assess the extent to which the respondent perceives he or she is performing in terms of work or school, intimate relationships, nonfamily social relationships, and overall life enjoyment (e.g., recreational activities) on a scale ranging from 0 (terribly) to 4 (very well). Scores for each subscale are calculated by taking the mean score for all subscale items. The BHM is keyed such that higher scores indicate better health for all four subscales. Kopta and Lowry (2002) have previously reported adequate to good internal consistency for the GMH (␣ ⬎ .89) and SYM (␣ ⬎ .85) subscales, and marginal to moderate internal consistency for the WB (␣ ⬎ .65) and LF (␣ ⬎ .72) subscales. Subscales have also been found to correlate highly with other measures of overall mental health functioning, generalized distress, and psychiatric symptoms. Results Confirmatory Factor Analysis The BHM-20’s initial design included three subscales (WB, SYM, and LF) corresponding to the phase model of psychotherapy, and a second-order global scale indicating overall mental health well-being and functioning. We used confirmatory factor analysis (CFA) with robust maximum likelihood estimation to determine if the BHM’s items conformed to this proposed three-factor model as well as the
This document is copyrighted by the American Psychological Association or one of its allied publishers. This article is intended solely for the personal use of the individual user and is not to be disseminated broadly.
BHM NORMS IN PRIMARY CARE
one-factor model. Because second-order models that are defined by two or three firstorder solutions tend to yield similar fit estimates as the related higher order models (Brown, 2006), we reasoned that formulation of a second-order solution would not be necessary. All analyses were performed with the Mplus, Version 6.12 program (Muthen & Muthen, 1998 –2011). Good fit for each model was assessed using the following criteria: comparative fit index (CFI) greater than .90, Tucker⫺Lewis index (TLI) greater than .90, root mean square error of approximation (RMSEA) less than .08, standardized root mean square residual (SRMR) less than .08, and low Akaike information criterion. As can be seen in Table 2, results of the CFA suggested the BHM-20’s proposed structure was marginally adequate, with the proposed threefactor structure providing a slightly better fit than the one-factor structure in all three samples. All of the standardized item-factor loadings were statistically significant (ps ⬍ .001) for both the threeand the one-factor solutions. Table 3 displays the item-factor loadings across all three samples. All factor loadings exceeded .350, except three items loading onto the SYM scale: Item 5 (“alcohol/ drug use interfering with your performance at
93
school or work”), Item 6 (“wanting to harm someone”), and Item 14 (“alcohol/drug use interfering with your relationships with family and/or friends”). We therefore decided to drop these three items from the SYM subscale and repeated the CFAs for both the three- and the one-factor models. Removing these three items improved overall model fit for both the three- and the one-factor models, with the three-factor model demonstrating a statistically significant better fit than the one-factor model (see Table 2). The revised threefactor scale demonstrated good fit based on the RMSEA (M ⫽ .079) and SRMR values (M ⫽ .053), and demonstrated marginal fit based on the CFI (M ⫽ .881) and TLI (M ⫽ .861) values. Item-factor loadings for the 17 retained items remained above .350. This revised, 17-item version of the BHM-20 will subsequently be referred to as the BHM-17-R, the revised SYM scale will be subsequently be referred to as SYM-17-R, and the revised GMH scale will be subsequently referred to as the GMH-17-R (the WB and LF subscales remain unchanged with the dropped items). The correlations of the original GMH scale with the revised GMH-17-R scale ranged from .841⫺.996, and the correlations of the original SYM subscale with the revised SYM-17-R subscale ranged from .846⫺.989.
Table 2 Fit Statistics for BHM-20 Confirmatory Factor Analyses Across Three Primary Care Samples PCC 1 (n ⫽ 338) Scales Full BHM (20 items) 2 CFI TLI RMSEA [90% CI] SRMR AIC Revised BHM (17 items) 2 CFI TLI RMSEA [90% CI] SRMR AIC
PCC 2 (n ⫽ 1,478)
PCC 3 (n ⫽ 1,256)
Three-factor
One-factor
Three-factor
One-factor
Three-factor
One-factor
675.824ⴱ .801 .773 .095 [.088, .102] .081 16,712.625
787.131ⴱ .758 .730 .104 [.096, .111] .086 16,849.364
1727.660ⴱ .835 .812 .080 [.077, .084] .067 71,858.620
1998.181ⴱ .807 .784 .086 [.083, .089] .071 72,243.476
1591.843ⴱ .831 .807 .083 [.079, .086] .067 64,562.169
1841.052ⴱ .801 .778 .088 [.085, .092] .070 64,913.520
381.002ⴱ .887 .868 .082 [.073, .091] .055 15,279.492
493.568ⴱ .840 .817 .097 [.088, .105] .062 15,408.932
1068.620ⴱ .869 .846 .075 [.070, .080] .054 81,164.190
1137.138ⴱ .860 .840 .076 [.072, .080] .054 81,243.555
1071.410ⴱ .888 .869 .081 [.077, .085] .050 58,378.826
1348.222ⴱ .856 .836 .091 [.086, .095] .054 58,823.103
Note. BHM ⫽ Behavioral Health Measure; PCC ⫽ primary care clinic; CFI ⫽ comparative fit index; TLI ⫽ Tucker⫺Lewis index; RMSEA ⫽ root mean square error of approximation; CI ⫽ confidence interval; SRMR ⫽ standardized root mean square residual; AIC ⫽ Aikaike’s information criterion. ⴱ p ⬍ .001.
94
BRYAN ET AL.
Table 3 Standardized Item Factor Loadings for the BHM-20’s Proposed Three-Factor (Well-Being, Symptoms, Life Functioning) and One-Factor (Global Mental Health) Models PCC 1 Scales WB
This document is copyrighted by the American Psychological Association or one of its allied publishers. This article is intended solely for the personal use of the individual user and is not to be disseminated broadly.
SYM
LF
GMH
Item No. 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20
PCC 2
PCC 3
Estimate
SE
Estimate
SE
Estimate
SE
.695 .760 .618 .610 .223 .326 .709 .645 .491 .448 .851 .828 .674 .261 .634 .540 .600 .744 .782 .849 .682 .722 .586 .590 .206 .306 .691 .658 .472 .429 .840 .797 .661 .241 .616 .517 .574 .670 .692 .749
.040 .033 .046 .040 .062 .061 .030 .035 .045 .048 .021 .024 .034 .065 .041 .048 .044 .034 .033 .020 .036 .031 .044 .041 .052 .056 .031 .034 .044 .046 .020 .026 .036 .048 .040 .046 .044 .036 .041 .031
.693 .736 .666 .574 .185 .260 .725 .659 .483 .380 .857 .806 .667 .201 .651 .548 .569 .688 .768 .863 .696 .727 .656 .555 .176 .254 .712 .662 .478 .363 .843 .789 .656 .198 .631 .533 .544 .637 .694 .786
.017 .015 .018 .021 .034 .029 .015 .018 .022 .026 .010 .012 .017 .030 .019 .023 .022 .017 .015 .010 .016 .014 .017 .021 .033 .028 .015 .017 .021 .026 .009 .012 .017 .029 .019 .022 .022 .018 .017 .012
.679 .745 .655 .615 .206 .303 .729 .648 .479 .449 .848 .811 .647 .238 .655 .562 .531 .699 .775 .848 .675 .728 .639 .595 .198 .293 .716 .653 .467 .434 .838 .793 .642 .234 .639 .547 .522 .637 .682 .757
.018 .016 .021 .021 .028 .027 .015 .018 .024 .023 .011 .013 .019 .029 .021 .023 .025 .019 .016 .012 .018 .015 .020 .021 .026 .026 .015 .018 .023 .022 .010 .013 .019 .027 .020 .023 .023 .019 .019 .015
Distressed Satisfied with life Energetic and motivated Fearful, scared Alcohol/drug use ⫺ school/work Harm someone Not liking yourself Concentrating Eating problem Thoughts of ending life Sad Hopeless Mood swings Alcohol/drug use ⫺ relationships Nervous Heart pounding Work/school Intimate relationships Nonfamily social relationships Life enjoyment Distressed Satisfied with life Energetic and motivated Fearful, scared Alcohol/drug use ⫺ school/work Harm someone Not liking yourself Concentrating Eating problem Thoughts of ending life Sad Hopeless Mood swings Alcohol/drug use ⫺ relationships Nervous Heart pounding Work/School Intimate relationships Nonfamily social relationships Life enjoyment
Note. BHM ⫽ Behavioral Health Measure; PCC ⫽ primary care clinic; WB ⫽ Well-Being subscale; SYM ⫽ Symptoms subscale; LF ⫽ Life Functioning subscale; GMH ⫽ Global Mental Health scale.
Means and Standard Deviations by Gender and Sample Means and standard deviations for all four BHM-20 and BHM-17-R scales were calculated and are displayed in Table 4. Means scores are reported for the original SYM and GMH scales as well as the revised SYM-17-R and GMH-
17-R scales to facilitate comparisons with previously published norms and to identify the effects of item removal on scale scores. Multivariate analysis of variance was used to compare mean scores. We first compared scores according to gender, and found that men scored significantly higher (i.e., “healthier”) than women on all BHM scales in all three primary
BHM NORMS IN PRIMARY CARE
95
Table 4 Mean Scores on BHM-20 and BHM-17-R Scales for Full Sample and by Gender Across Three Primary Care Samples, With Pooled Results Full sample
This document is copyrighted by the American Psychological Association or one of its allied publishers. This article is intended solely for the personal use of the individual user and is not to be disseminated broadly.
Scales GMH PCC 1 PCC 2 PCC 3 Pooled GMH-17-R PCC 1 PCC 2 PCC 3 Pooled WB PCC 1 PCC 2 PCC 3 Pooled SYM PCC 1 PCC 2 PCC 3 Pooled SYM-17-R PCC 1 PCC 2 PCC 3 Pooled LF PCC 1 PCC 2 PCC 3 Pooled
Male
Female
n
M
SD
n
M
SD
n
M
SD
pa
332 1357 1211 2902
2.73 2.70 2.72 2.71
.70 .69 .71 .70
124 512 480 1117
2.95 2.84 2.89 2.87
.62 .68 .68 .67
208 845 731 1785
2.60 2.62 2.61 2.61
.72 .69 .71 .70
⬍.001 ⬍.001 ⬍.001
