562912
research-article2014
JPCXXX10.1177/2150131914562912Journal of Primary Care & Community HealthValleley et al
Pilot Studies
Behavioral Health Screening in Pediatric Primary Care: A Pilot Study
Journal of Primary Care & Community Health 2015, Vol. 6(3) 199–204 © The Author(s) 2014 Reprints and permissions: sagepub.com/journalsPermissions.nav DOI: 10.1177/2150131914562912 jpc.sagepub.com
Rachel J. Valleley1, Natalie Romer2, Sara Kupzyk1, Joseph H. Evans1, and Keith D. Allen1
Abstract This pilot study investigated the effect of the Pediatric Symptom Checklist (PSC) on identification of and physician response to behavioral health (BH) concerns. Researchers reviewed 1211 charts of youth aged 4 to 16 years. Records were compared during baseline and an intervention consisting of implementation of the PSC to determine the rate of BH identification and pediatrician response. Access to PSC data resulted in a trivial difference in BH concerns identified by physicians and did not affect physician responses. This case study demonstrates that simply implementing BH screening in primary care may not result in improved outcomes for these children. Keywords children, impact evaluation, pediatrics, primary care, program evaluation In primary care (PC), 15% to 25% of children present with a behavioral health (BH) concern.1,2 These problems are managed most effectively if detected and treated in childhood,3 and PC has been deemed a major point of access to brief and early intervention.4 Unfortunately, primary care physicians (PCPs) have struggled to fill this role5 and many children with BH concerns are not detected early or referred for services.6 One solution to early detection is to use BH screening. BH screening has been recommended as a cost-effective and efficient practice for early identification of BH concerns in PC.7,8 Previous research has focused on validating screening tools that identify patients at risk for psychopathology,9,10 including the Pediatric Symptom Checklist.11 Despite the support for screening instruments,12 it is not clear how BH screening affects actual physician behavior. Research to date on these issues has been mixed. One study found that there was a significant increase in rates of referrals when the PSC was implemented in health clinics serving a low-income, Hispanic community.13 However, other studies have found that few children identified with BH concerns are referred for services14,15 and that screening alone may have little impact on physician practices in terms of referral or initiation of treatment.16,17 Thus, there is still much to learn about how BH screening affects physician identification and referral of children with BH concerns. The purpose of this case study was to examine how the implementation of a BH screening instrument affected PCPs’ identification and response to BH concerns in a realworld setting.
Method Participants and Setting A privately owned pediatric PC clinic in a Midwestern city participated. Seven pediatricians (5 full time; 3 female), with an average of 11 years of experience (range 3-17 years) staffed this clinic. BH services were provided on-site 2 days per week by a psychologist. Researchers reviewed electronic charts for 1211 patient visits with ages ranging from 4 to 16 years attending visits. Table 1 summarizes characteristics of the visits for the 2 groups (screening and no screening).
Measure Independent Variable. Implementation of the Pediatric Symptom Checklist (PSC) was the independent variable. The PSC is a 1-page, 35-item screening questionnaire designed to capture a parent’s impression of their child’s psychosocial functioning in a PC setting.11,18 Each item is assigned a score of 2 = often; 1 = sometimes; and 0 = never. Cut scores are provided for different ages. Parents are also asked 2 additional questions: if their child has a BH concern 1
University of Nebraska Medical Center, Omaha, NE, USA University of South Florida, Tampa, FL, USA
2
Corresponding Author: Rachel J. Valleley, Munroe-Meyer Institute, 985450 University of Nebraska Medical Center, Omaha, NE 68144, USA. Email: [email protected]
Downloaded from jpc.sagepub.com at University of Sussex Library on July 26, 2015
200
Journal of Primary Care & Community Health 6(3)
Table 1. Descriptive Characteristics of the No Screening and Screening Groups (N = 1,211). No Screening
n
Patient gender Female 282 Male 385 Age, years 4-7 302 8-12 227 13-16 138 Visit with primary care physician Yes 541 No 126 Type of visit Sick 300 Well 367 Unknown Existing behavioral health concern Yes 54 No 613 Current psychotropic medication Yes 43 No 624 Unknown 0 Newly identified behavioral health concerna Total 58 Physician 1 (part time) 1 Physician 2 12 Physician 3 8 Physician 4 12 Physician 5 20 Physician 6 5 Physician 7 (part time) 0 Average 8.3 Action taken for new behavioral health concernb Total 22 Physician 1 (part time) 1 Physician 2 3 Physician 3 5 Physician 4 4 Physician 5 7 Physician 6 2 Physician 7 (part time) N/A Average 3.1
Screening %
n
%
42.3 57.7
262 282
48.2 51.8
45.3 34.0 20.7
199 215 130
36.6 39.5 23.9
81.1 18.9
488 56
89.7 10.3
45.0 55.0
127 415
23.3 76.3
8.1 91.9
53 491
9.7 90.3
6.4 93.6 0.0
33 511 2
6.1 93.9 0.4
8.7 2.7 7.2 6.4 12.5 14.5 5.4 0 7.0
73 8 13 6 13 24 7 2 10.4
13.4 13.1 9.5 11.8 13.3 20.5 10 20.0 14.0
37.3 100 18.8 55.6 50.0 35.0 40.0 N/A 50.0
22 1 3 3 3 7 3 2 3.14
29.3 12.5 21.4 50.0 21.4 29.2 42.9 100 39.6
a
Percentage calculated as number of newly identified behavioral health concerns divided by total number of patients seen. Percentage calculated as number of actions taken for newly identified behavioral health concerns divided by total number of patients newly identified.
b
that he/she needs help with and if they would like additional services. It has been found to be reliable18 and valid.18-20 Dependent Variables. The primary dependent variables were whether a BH concern was identified during the visit and whether the pediatrician addressed the BH concern. Newly
identified BH concerns and physician responses were ascertained from a variety sources in the visit note (eg, physician made note of a BH concern, physician added a new BH diagnosis, physician prescribed a psychotropic medication for first time, noted psychology referral, counseled on behavior).
Downloaded from jpc.sagepub.com at University of Sussex Library on July 26, 2015
201
Valleley et al
Procedure A quasi-experimental research design was employed, consisting of 2 groups: no screening (5 weeks) and screening using the PSC (6 weeks). Prior to PSC implementation, pediatricians were provided with a description of the PSC. During PSC implementation, researchers approached caregivers and provided them with the PSC to complete. The researcher scored the PSC and placed it at the pediatrician’s station. Pediatricians initialed PSC forms after review. In all, 82.9% of the PSC forms scanned into patient charts were initialed. The administration of the PSC was managed by the research team to ensure that the majority of patients were screened; however, decision making regarding whether a child was at-risk for a BH concern and management of the BH concern was left to the PCP. Charts were reviewed by the researchers. Coded data included whether the pediatrician identified a BH concern and type, pediatrician response to the BH concern, whether an existing BH concern had been identified and if services were being provided, whether the PSC was in the at-risk range, and whether the caregiver requested assistance or services on the PSC. A second rater coded data for 20.3% of the sample. Interrater agreement was calculated as the number of instances of agreement divided by agreements plus disagreements multiplied by 100%. Percentage agreement for the primary dependent variables was as follows: identified BH concern = 94.7%; pediatrician addressed BH concern = 91.5%.
Results Descriptive Analysis All providers detected new behavioral health concerns in their patient population. The percentages for full-time providers ranged from 5.4% to 14.5% during the no screening phase and from 9.5% to 20.5% in the screening phase. All pediatricians identified a higher percentage of patients with a behavioral health concern during screening. Physician response to the concerns varied across full time providers ranging from 18.8% to 55.6% during no screening and 21.4% to 50% during screening. In the no-screening group, 8.7% of visits included documentation of newly identified BH concern(s). In visits with newly identified BH concern(s), physicians responded to 37.3%. During the screening phase, new BH concern(s) were identified for 13.4% of the 544 patients, and of these, pediatricians responded to 29.3%. Based on caregiver ratings on the PSC, 5.0% of patients (n = 27) exceeded the cut score indicating risk of a BH problem. Of the 27 identified as at-risk, 11.1% had a BH concern identified, 44.4% had a previous BH concern
documented, and 48.1% had existing behavioral health services documented. When responding to the 2 questions at the end of the PSC, 30 caregivers, regardless if the PSC was at risk, indicated that their child had a behavioral problem for which he or she needed help. Notes showed that pediatricians addressed six of these concerns. Twenty caregivers indicated interest in services to address their child’s behavioral problem, and of these, pediatricians documented taking action 5 times. Figure 1 summarizes the types of newly identified BH. Most common problems identified were feeding, sleep, and attention deficit/hyperactivity disorder (ADHD) symptoms. The pediatricians were most likely to provide a behavioral health recommendation to address concerns (Figure 2).
Statistical Analysis Behavioral Health Concern Identified. Results of a 2-variable chi-square test showed a significant difference in the proportion of BH concerns identified between the screening and no screening groups: χ2(N = 1211, 1) = 6.93, P = .008. In the no-screening group, behavioral health concerns were identified in 8.7% (n = 58) of visits and in the screening group behavioral health concerns were identified in 13.4% (n = 73) of visits. When reviewing the output from the chi-square test, we also made the observation that these 2 percentages were similar, yet the chi-square test significant. Thus, given the large sample (n = 1211), we decided to also consider effect size, which was indeed small (r = 0.075 and when converted Cohen’s d = 0.15). This suggests that although the chi-square test was statistically significant, the difference was not meaningful. Pediatrician Response to Identified Behavioral Health Concern. A 2-variable chi-square test was also used to evaluate differences in the proportion of cases where BH concerns where identified and the pediatrician responded between the conditions. The difference in the proportion of patients with identified BH concerns where the pediatrician responded was not significantly different between groups: χ2(N = 131, 1) = 0.88, P = .348.
Discussion Access to BH screening data in this pediatric office was not sufficient in increasing the number of identified BH concerns by pediatricians; nor did implementation of the PSC affect pediatrician response to identified BH concerns. During all 1211 visits in both the screening and no screening phases, pediatricians documented 10.8% of patients having an identified BH concerns and 8.8% of the patients having had a previously identified BH
Downloaded from jpc.sagepub.com at University of Sussex Library on July 26, 2015
202
Journal of Primary Care & Community Health 6(3)
Figure 1. Proportion of different types of newly identified behavioral health concerns by group (no screening n = 73, screening n = 58).
concern. However, physicians responded to only 33% of identified BH concerns. Of the patients whose caregivers completed the PSC, pediatricians documented identified BH concern(s) for 13.4%. Interestingly, only 5.0% of the patients whose caregivers completed a PSC exceeded the cut score (ie, fewer BH concerns were identified using the PSC than practice as usual). In the majority of cases where a child was identified as at-risk or parents indicated that they were interested in seeking assistance, pediatricians did not take action. Pediatric settings have been identified as an optimal setting for behavioral screening, yet, our findings are consistent with research demonstrating that when BH concerns are identified in PC, problems are often undertreated and receive minimal follow-up.21,22 This study was conducted in a metro area clinic, with a population of primarily English speaking, and privately insured families. The clinic also has a colocated psychologist providing services. When considering this context, our findings are particularly discouraging given that preventative care adherence has been associated with
patients being better insured, English speaking, and nonminority.23,24
Limitations This study has some limitations that may have affected the results. First, implementation of the PSC by the researchers may have limited integration of the screening procedure into the day-to-day procedures. Second, there were no direct measures of procedural integrity. Pediatricians reviewed the majority of the completed PSCs; however, we do not know if pediatricians reviewed the PSC prior to, during, or following the appointment. Our review of records focused on the notes from the appointment during which the PSC was completed and any prior records. Pediatricians may have followed up on an elevated PSC score during a future encounter. Third, this was a sample of convenience and external validity is limited given that data were collected within 1 clinic. Finally, given that the data were collected from patients’ charts, it is possible that physicians discussed concerns with the patient, but failed to document information.
Downloaded from jpc.sagepub.com at University of Sussex Library on July 26, 2015
203
Valleley et al
Figure 2. Proportion of types of actions taken by the pediatricians to address the newly identified behavioral health concerns by phase (phase A no screening, phase B screening).
Future Directions
Declaration of Conflicting Interests
This study demonstrated that implementation of the PSC in this clinic did not improve the identification or response by pediatricians to identified BH concerns. This may be explained by some of the limitations, but it may also be an example of the gap between evidence-based practice and implementation.25,26 Research is needed to identify the essential, valued, feasible, and effective components of BH screening procedures within PC and evaluate outcomes over time. Our findings are aligned with the call for more information and training on the requisite systems for effective screening and follow-up within PC.27,28 The psychometric validity of BH screening measures has been well established in the research. If pediatricians are to commit to integrating screening into PC, however, more research is needed on implementation (eg, provider/ staff education and materials, pay-for performance initiatives targeted at screening, automated identification such as chart flagging, improvements in office procedures) that results in improved access to care for children and adolescents identified at-risk, and ultimately, improved outcomes over time.
The author(s) declared no potential conflicts of interest with respect to the research, authorship, and/or publication of this article.
Funding The author(s) disclosed receipt of the following financial support for the research, authorship, and/or publication of this article: This work was supported by grant numbers HRSA D40HP02597-08-00 and M01HP25184-01-00.
References 1. Cooper S, Valleley RJ, Polaha J, Begeny J, Evans JH. Running out of time: physician management of behavioral health concerns in rural pediatric primary care. Pediatrics. 2006;118:e132-e138. 2. Briggs-Gowan MJ, Horwitz SM, Schwab-Stone ME, Leventhal JM, Leaf PJ. Mental health in pediatric settings: distribution of disorders and factors related to service use. J Am Acad Child Adolesc Psychiatry. 2000;39:841-849. 3. Kim-Cohen J, Caspi A, Moffitt TE, Harrington H, Milne BJ, Poulton R. Prior juvenile diagnoses in adults with mental
Downloaded from jpc.sagepub.com at University of Sussex Library on July 26, 2015
204
Journal of Primary Care & Community Health 6(3)
disorder: developmental follow-back of a prospective-longitudinal cohort. Arch Gen Psychiatry. 2003;60:709-717. 4. Tolan PH, Dodge KA. Children’s mental health as a primary care and concern: a system for comprehensive support and service. Am Psychol. 2005;60:601-614. 5. Sheldrick RC, Merchant S, Perrin EC. Identification of developmental-behavioral problems in primary care: a systematic review. Pediatrics. 2011;128:356-363. 6. Cooper JL, Masi R, Vick J. Social and Emotional Development in Early Childhood: What Every Policymaker Should Know. New York, NY: National Center for Children in Poverty; 2009. 7. Hogan MF. New Freedom Commission report: The President’s New Freedom Commission: recommendations to transform mental health care in America. Psychiatr Serv. 2003;54:1467-1474. 8. Appendix S4: the case for routine mental health screening. Pediatrics. 2010;125(suppl 3):S133-S139. 9. Wolraich ML, Lambert W, Doffing MA, Bickman L, Simmons T, Worley K. Psychometric properties of the Vanderbilt ADHD Diagnostic Parent Rating Scale in a referred population. J Pediatr Psychol. 2003;28:559-568. 10. Johnson JG, Harris ES, Spitzer RL, Williams JB. The patient health questionnaire for adolescents: validation of an instrument for the assessment of mental disorders among adolescent primary care patients. J Adolesc Health. 2002;30:196-204. 11. Jellinek M, Evans N, Knight RB. Use of a behavior checklist on a pediatric inpatient unit. J Pediatr. 1979;94:156-158. 12. Carroll AE, Biondich P, Anand V, Dugan TM, Downs SM. A randomized controlled trial of screening for maternal depression with a clinical decision support system. J Am Med Inform Assoc. 2013;20:311-316. 13. Murphy JM, Ichinose C, Hicks RC, et al. Utility of the Pediatric Symptom Checklist as a psychosocial screen to meet the federal Early and Periodic Screening, Diagnosis, and Treatment (EPSDT) standards: a pilot study. J Pediatr. 1996;129:864-869. 14. Lavigne JV, Binns HJ, Christoffel KK, et al. Behavioral and emotional problems among preschool children in pediatric primary care: prevalence and pediatricians’ recognition. Pediatric Practice Research Group. Pediatrics. 1993;91:649-655. 15. Rushton J, Bruckman D, Kelleher K. Primary care referral of children with psychosocial problems. Arch Pediatr Adolesc Med. 2002;156:592-598. 16. Gilbody S, Sheldon T, House A. Screening and case-finding instruments for depression: a meta-analysis. CMAJ. 2008;178:997-1003. 17. Wissow LS, Brown J, Fothergill KE, et al. Universal mental health screening in pediatric primary care: a systematic review. J Am Acad Child Adolesc Psychiatry. 2013;52:1134. e23-1147.e23. 18. Jellinek MS, Murphy JM, Burns BJ. Brief psychoso cial screening in outpatient pediatric practice. J Pediatr. 1986;109:371-378. 19. Simonian SJ, Tarnowski KJ. Utility of the Pediatric Symptom Checklist for behavioral screening of disadvantaged children. Child Psychiatry Hum Dev. 2001;31:269-278. 20. Walker WO Jr, LaGrone RG, Atkinson AW. Psychosocial screening in pediatric practice: identifying high-risk children. J Dev Behav Pediatr. 1989;10:134-138.
21. Gardner W, Kelleher KJ, Pajer KA, Campo JV. Primary care clinicians’ use of standardized tools to assess child psychosocial problems. Ambul Pediatr. 2003;3:191-195. 22. Pidano AE, Kimmelblatt CA, Neace WP. Behavioral health in the pediatric primary care setting: needs, barriers, and implications for psychologists. Psychol Serv. 2011;8:151-165. 23. Chung PJ, Lee TC, Morrison JL, Schuster MA. Preventive care for children in the United States: quality and barriers. Annu Rev Public Health. 2006;27:491-515. 24. Stevens GD, Seid M, Mistry R, Halfon N. Disparities in primary care for vulnerable children: the influence of multiple risk factors. Health Serv Res. 2006;41:507-531. 25. Hacker K, Goldstein J, Link D, et al. Pediatric provider processes for behavioral health screening, decision making, and referral in sites with colocated mental health services. J Dev Behav Pediatr. 2013;34:680-687. 26. Proctor EK, Landsverk J, Aarons G, Chambers D, Glisson C, Mittman B. Implementation research in mental health services: an emerging science with conceptual, methodological, and training challenges. Adm Policy Ment Health. 2009;36:24-34. 27. Van Cleave J, Kuhlthau KA, Bloom S, et al. Interventions to improve screening and follow-up in primary care: a systematic review of the evidence. Acad Pediatr. 2012;12:269-282. 28. Weitzman CC, Leventhal JM. Screening for behavioral health problems in primary care. Curr Opin Pediatr. 2006;18:641-648.
Author Biographies Rachel J. Valleley, PhD, is an associate professor of Psychology and Pediatrics at the Munroe-Meyer Institute and the University of Nebraska Medical Center. Her main research interests involve integrating behavioral health services into pediatric practices. Natalie Romer, PhD, is an assistant professor in the Department of Child and Family Studies at the University of South Florida. Her main research interests include universal screening, strengthbased assessment, and implementation of multi-tiered behavior and mental health support systems. Sara Kupzyk, PhD, is an assistant professor of Psychology with the Munroe-Meyer Institute at the University of Nebraska Medical Center. She provides outpatient behavioral health services in pediatric primary care clinics. Joseph H. Evans is a professor and director of Psychology at the Munroe-Meyer Institute and at the University of Nebraska Medical Center. With training funds from the Health Resources and Services Administration, he has developed an “integrated behavioral health in pediatric primary care model” that is now being used in 32 rural and urban sites across Nebraska and that is being replicated in two additional states. Dr. Evans specializes in training psychology interns and post-doctoral fellows in methods to integrate behavioral services into patients’ primary care “medical homes.” Keith D. Allen, PhD, is a professor of Psychology and Pediatrics at the Munroe-Meyer Institute and the University of Nebraska Medical Center. His interests include pediatric pain management, health-related behavior in children, parent training, stress-related disorders, and management of distress during invasive medical and dental procedures.
Downloaded from jpc.sagepub.com at University of Sussex Library on July 26, 2015
research-article2014
JPCXXX10.1177/2150131914562912Journal of Primary Care & Community HealthValleley et al
Pilot Studies
Behavioral Health Screening in Pediatric Primary Care: A Pilot Study
Journal of Primary Care & Community Health 2015, Vol. 6(3) 199–204 © The Author(s) 2014 Reprints and permissions: sagepub.com/journalsPermissions.nav DOI: 10.1177/2150131914562912 jpc.sagepub.com
Rachel J. Valleley1, Natalie Romer2, Sara Kupzyk1, Joseph H. Evans1, and Keith D. Allen1
Abstract This pilot study investigated the effect of the Pediatric Symptom Checklist (PSC) on identification of and physician response to behavioral health (BH) concerns. Researchers reviewed 1211 charts of youth aged 4 to 16 years. Records were compared during baseline and an intervention consisting of implementation of the PSC to determine the rate of BH identification and pediatrician response. Access to PSC data resulted in a trivial difference in BH concerns identified by physicians and did not affect physician responses. This case study demonstrates that simply implementing BH screening in primary care may not result in improved outcomes for these children. Keywords children, impact evaluation, pediatrics, primary care, program evaluation In primary care (PC), 15% to 25% of children present with a behavioral health (BH) concern.1,2 These problems are managed most effectively if detected and treated in childhood,3 and PC has been deemed a major point of access to brief and early intervention.4 Unfortunately, primary care physicians (PCPs) have struggled to fill this role5 and many children with BH concerns are not detected early or referred for services.6 One solution to early detection is to use BH screening. BH screening has been recommended as a cost-effective and efficient practice for early identification of BH concerns in PC.7,8 Previous research has focused on validating screening tools that identify patients at risk for psychopathology,9,10 including the Pediatric Symptom Checklist.11 Despite the support for screening instruments,12 it is not clear how BH screening affects actual physician behavior. Research to date on these issues has been mixed. One study found that there was a significant increase in rates of referrals when the PSC was implemented in health clinics serving a low-income, Hispanic community.13 However, other studies have found that few children identified with BH concerns are referred for services14,15 and that screening alone may have little impact on physician practices in terms of referral or initiation of treatment.16,17 Thus, there is still much to learn about how BH screening affects physician identification and referral of children with BH concerns. The purpose of this case study was to examine how the implementation of a BH screening instrument affected PCPs’ identification and response to BH concerns in a realworld setting.
Method Participants and Setting A privately owned pediatric PC clinic in a Midwestern city participated. Seven pediatricians (5 full time; 3 female), with an average of 11 years of experience (range 3-17 years) staffed this clinic. BH services were provided on-site 2 days per week by a psychologist. Researchers reviewed electronic charts for 1211 patient visits with ages ranging from 4 to 16 years attending visits. Table 1 summarizes characteristics of the visits for the 2 groups (screening and no screening).
Measure Independent Variable. Implementation of the Pediatric Symptom Checklist (PSC) was the independent variable. The PSC is a 1-page, 35-item screening questionnaire designed to capture a parent’s impression of their child’s psychosocial functioning in a PC setting.11,18 Each item is assigned a score of 2 = often; 1 = sometimes; and 0 = never. Cut scores are provided for different ages. Parents are also asked 2 additional questions: if their child has a BH concern 1
University of Nebraska Medical Center, Omaha, NE, USA University of South Florida, Tampa, FL, USA
2
Corresponding Author: Rachel J. Valleley, Munroe-Meyer Institute, 985450 University of Nebraska Medical Center, Omaha, NE 68144, USA. Email: [email protected]
Downloaded from jpc.sagepub.com at University of Sussex Library on July 26, 2015
200
Journal of Primary Care & Community Health 6(3)
Table 1. Descriptive Characteristics of the No Screening and Screening Groups (N = 1,211). No Screening
n
Patient gender Female 282 Male 385 Age, years 4-7 302 8-12 227 13-16 138 Visit with primary care physician Yes 541 No 126 Type of visit Sick 300 Well 367 Unknown Existing behavioral health concern Yes 54 No 613 Current psychotropic medication Yes 43 No 624 Unknown 0 Newly identified behavioral health concerna Total 58 Physician 1 (part time) 1 Physician 2 12 Physician 3 8 Physician 4 12 Physician 5 20 Physician 6 5 Physician 7 (part time) 0 Average 8.3 Action taken for new behavioral health concernb Total 22 Physician 1 (part time) 1 Physician 2 3 Physician 3 5 Physician 4 4 Physician 5 7 Physician 6 2 Physician 7 (part time) N/A Average 3.1
Screening %
n
%
42.3 57.7
262 282
48.2 51.8
45.3 34.0 20.7
199 215 130
36.6 39.5 23.9
81.1 18.9
488 56
89.7 10.3
45.0 55.0
127 415
23.3 76.3
8.1 91.9
53 491
9.7 90.3
6.4 93.6 0.0
33 511 2
6.1 93.9 0.4
8.7 2.7 7.2 6.4 12.5 14.5 5.4 0 7.0
73 8 13 6 13 24 7 2 10.4
13.4 13.1 9.5 11.8 13.3 20.5 10 20.0 14.0
37.3 100 18.8 55.6 50.0 35.0 40.0 N/A 50.0
22 1 3 3 3 7 3 2 3.14
29.3 12.5 21.4 50.0 21.4 29.2 42.9 100 39.6
a
Percentage calculated as number of newly identified behavioral health concerns divided by total number of patients seen. Percentage calculated as number of actions taken for newly identified behavioral health concerns divided by total number of patients newly identified.
b
that he/she needs help with and if they would like additional services. It has been found to be reliable18 and valid.18-20 Dependent Variables. The primary dependent variables were whether a BH concern was identified during the visit and whether the pediatrician addressed the BH concern. Newly
identified BH concerns and physician responses were ascertained from a variety sources in the visit note (eg, physician made note of a BH concern, physician added a new BH diagnosis, physician prescribed a psychotropic medication for first time, noted psychology referral, counseled on behavior).
Downloaded from jpc.sagepub.com at University of Sussex Library on July 26, 2015
201
Valleley et al
Procedure A quasi-experimental research design was employed, consisting of 2 groups: no screening (5 weeks) and screening using the PSC (6 weeks). Prior to PSC implementation, pediatricians were provided with a description of the PSC. During PSC implementation, researchers approached caregivers and provided them with the PSC to complete. The researcher scored the PSC and placed it at the pediatrician’s station. Pediatricians initialed PSC forms after review. In all, 82.9% of the PSC forms scanned into patient charts were initialed. The administration of the PSC was managed by the research team to ensure that the majority of patients were screened; however, decision making regarding whether a child was at-risk for a BH concern and management of the BH concern was left to the PCP. Charts were reviewed by the researchers. Coded data included whether the pediatrician identified a BH concern and type, pediatrician response to the BH concern, whether an existing BH concern had been identified and if services were being provided, whether the PSC was in the at-risk range, and whether the caregiver requested assistance or services on the PSC. A second rater coded data for 20.3% of the sample. Interrater agreement was calculated as the number of instances of agreement divided by agreements plus disagreements multiplied by 100%. Percentage agreement for the primary dependent variables was as follows: identified BH concern = 94.7%; pediatrician addressed BH concern = 91.5%.
Results Descriptive Analysis All providers detected new behavioral health concerns in their patient population. The percentages for full-time providers ranged from 5.4% to 14.5% during the no screening phase and from 9.5% to 20.5% in the screening phase. All pediatricians identified a higher percentage of patients with a behavioral health concern during screening. Physician response to the concerns varied across full time providers ranging from 18.8% to 55.6% during no screening and 21.4% to 50% during screening. In the no-screening group, 8.7% of visits included documentation of newly identified BH concern(s). In visits with newly identified BH concern(s), physicians responded to 37.3%. During the screening phase, new BH concern(s) were identified for 13.4% of the 544 patients, and of these, pediatricians responded to 29.3%. Based on caregiver ratings on the PSC, 5.0% of patients (n = 27) exceeded the cut score indicating risk of a BH problem. Of the 27 identified as at-risk, 11.1% had a BH concern identified, 44.4% had a previous BH concern
documented, and 48.1% had existing behavioral health services documented. When responding to the 2 questions at the end of the PSC, 30 caregivers, regardless if the PSC was at risk, indicated that their child had a behavioral problem for which he or she needed help. Notes showed that pediatricians addressed six of these concerns. Twenty caregivers indicated interest in services to address their child’s behavioral problem, and of these, pediatricians documented taking action 5 times. Figure 1 summarizes the types of newly identified BH. Most common problems identified were feeding, sleep, and attention deficit/hyperactivity disorder (ADHD) symptoms. The pediatricians were most likely to provide a behavioral health recommendation to address concerns (Figure 2).
Statistical Analysis Behavioral Health Concern Identified. Results of a 2-variable chi-square test showed a significant difference in the proportion of BH concerns identified between the screening and no screening groups: χ2(N = 1211, 1) = 6.93, P = .008. In the no-screening group, behavioral health concerns were identified in 8.7% (n = 58) of visits and in the screening group behavioral health concerns were identified in 13.4% (n = 73) of visits. When reviewing the output from the chi-square test, we also made the observation that these 2 percentages were similar, yet the chi-square test significant. Thus, given the large sample (n = 1211), we decided to also consider effect size, which was indeed small (r = 0.075 and when converted Cohen’s d = 0.15). This suggests that although the chi-square test was statistically significant, the difference was not meaningful. Pediatrician Response to Identified Behavioral Health Concern. A 2-variable chi-square test was also used to evaluate differences in the proportion of cases where BH concerns where identified and the pediatrician responded between the conditions. The difference in the proportion of patients with identified BH concerns where the pediatrician responded was not significantly different between groups: χ2(N = 131, 1) = 0.88, P = .348.
Discussion Access to BH screening data in this pediatric office was not sufficient in increasing the number of identified BH concerns by pediatricians; nor did implementation of the PSC affect pediatrician response to identified BH concerns. During all 1211 visits in both the screening and no screening phases, pediatricians documented 10.8% of patients having an identified BH concerns and 8.8% of the patients having had a previously identified BH
Downloaded from jpc.sagepub.com at University of Sussex Library on July 26, 2015
202
Journal of Primary Care & Community Health 6(3)
Figure 1. Proportion of different types of newly identified behavioral health concerns by group (no screening n = 73, screening n = 58).
concern. However, physicians responded to only 33% of identified BH concerns. Of the patients whose caregivers completed the PSC, pediatricians documented identified BH concern(s) for 13.4%. Interestingly, only 5.0% of the patients whose caregivers completed a PSC exceeded the cut score (ie, fewer BH concerns were identified using the PSC than practice as usual). In the majority of cases where a child was identified as at-risk or parents indicated that they were interested in seeking assistance, pediatricians did not take action. Pediatric settings have been identified as an optimal setting for behavioral screening, yet, our findings are consistent with research demonstrating that when BH concerns are identified in PC, problems are often undertreated and receive minimal follow-up.21,22 This study was conducted in a metro area clinic, with a population of primarily English speaking, and privately insured families. The clinic also has a colocated psychologist providing services. When considering this context, our findings are particularly discouraging given that preventative care adherence has been associated with
patients being better insured, English speaking, and nonminority.23,24
Limitations This study has some limitations that may have affected the results. First, implementation of the PSC by the researchers may have limited integration of the screening procedure into the day-to-day procedures. Second, there were no direct measures of procedural integrity. Pediatricians reviewed the majority of the completed PSCs; however, we do not know if pediatricians reviewed the PSC prior to, during, or following the appointment. Our review of records focused on the notes from the appointment during which the PSC was completed and any prior records. Pediatricians may have followed up on an elevated PSC score during a future encounter. Third, this was a sample of convenience and external validity is limited given that data were collected within 1 clinic. Finally, given that the data were collected from patients’ charts, it is possible that physicians discussed concerns with the patient, but failed to document information.
Downloaded from jpc.sagepub.com at University of Sussex Library on July 26, 2015
203
Valleley et al
Figure 2. Proportion of types of actions taken by the pediatricians to address the newly identified behavioral health concerns by phase (phase A no screening, phase B screening).
Future Directions
Declaration of Conflicting Interests
This study demonstrated that implementation of the PSC in this clinic did not improve the identification or response by pediatricians to identified BH concerns. This may be explained by some of the limitations, but it may also be an example of the gap between evidence-based practice and implementation.25,26 Research is needed to identify the essential, valued, feasible, and effective components of BH screening procedures within PC and evaluate outcomes over time. Our findings are aligned with the call for more information and training on the requisite systems for effective screening and follow-up within PC.27,28 The psychometric validity of BH screening measures has been well established in the research. If pediatricians are to commit to integrating screening into PC, however, more research is needed on implementation (eg, provider/ staff education and materials, pay-for performance initiatives targeted at screening, automated identification such as chart flagging, improvements in office procedures) that results in improved access to care for children and adolescents identified at-risk, and ultimately, improved outcomes over time.
The author(s) declared no potential conflicts of interest with respect to the research, authorship, and/or publication of this article.
Funding The author(s) disclosed receipt of the following financial support for the research, authorship, and/or publication of this article: This work was supported by grant numbers HRSA D40HP02597-08-00 and M01HP25184-01-00.
References 1. Cooper S, Valleley RJ, Polaha J, Begeny J, Evans JH. Running out of time: physician management of behavioral health concerns in rural pediatric primary care. Pediatrics. 2006;118:e132-e138. 2. Briggs-Gowan MJ, Horwitz SM, Schwab-Stone ME, Leventhal JM, Leaf PJ. Mental health in pediatric settings: distribution of disorders and factors related to service use. J Am Acad Child Adolesc Psychiatry. 2000;39:841-849. 3. Kim-Cohen J, Caspi A, Moffitt TE, Harrington H, Milne BJ, Poulton R. Prior juvenile diagnoses in adults with mental
Downloaded from jpc.sagepub.com at University of Sussex Library on July 26, 2015
204
Journal of Primary Care & Community Health 6(3)
disorder: developmental follow-back of a prospective-longitudinal cohort. Arch Gen Psychiatry. 2003;60:709-717. 4. Tolan PH, Dodge KA. Children’s mental health as a primary care and concern: a system for comprehensive support and service. Am Psychol. 2005;60:601-614. 5. Sheldrick RC, Merchant S, Perrin EC. Identification of developmental-behavioral problems in primary care: a systematic review. Pediatrics. 2011;128:356-363. 6. Cooper JL, Masi R, Vick J. Social and Emotional Development in Early Childhood: What Every Policymaker Should Know. New York, NY: National Center for Children in Poverty; 2009. 7. Hogan MF. New Freedom Commission report: The President’s New Freedom Commission: recommendations to transform mental health care in America. Psychiatr Serv. 2003;54:1467-1474. 8. Appendix S4: the case for routine mental health screening. Pediatrics. 2010;125(suppl 3):S133-S139. 9. Wolraich ML, Lambert W, Doffing MA, Bickman L, Simmons T, Worley K. Psychometric properties of the Vanderbilt ADHD Diagnostic Parent Rating Scale in a referred population. J Pediatr Psychol. 2003;28:559-568. 10. Johnson JG, Harris ES, Spitzer RL, Williams JB. The patient health questionnaire for adolescents: validation of an instrument for the assessment of mental disorders among adolescent primary care patients. J Adolesc Health. 2002;30:196-204. 11. Jellinek M, Evans N, Knight RB. Use of a behavior checklist on a pediatric inpatient unit. J Pediatr. 1979;94:156-158. 12. Carroll AE, Biondich P, Anand V, Dugan TM, Downs SM. A randomized controlled trial of screening for maternal depression with a clinical decision support system. J Am Med Inform Assoc. 2013;20:311-316. 13. Murphy JM, Ichinose C, Hicks RC, et al. Utility of the Pediatric Symptom Checklist as a psychosocial screen to meet the federal Early and Periodic Screening, Diagnosis, and Treatment (EPSDT) standards: a pilot study. J Pediatr. 1996;129:864-869. 14. Lavigne JV, Binns HJ, Christoffel KK, et al. Behavioral and emotional problems among preschool children in pediatric primary care: prevalence and pediatricians’ recognition. Pediatric Practice Research Group. Pediatrics. 1993;91:649-655. 15. Rushton J, Bruckman D, Kelleher K. Primary care referral of children with psychosocial problems. Arch Pediatr Adolesc Med. 2002;156:592-598. 16. Gilbody S, Sheldon T, House A. Screening and case-finding instruments for depression: a meta-analysis. CMAJ. 2008;178:997-1003. 17. Wissow LS, Brown J, Fothergill KE, et al. Universal mental health screening in pediatric primary care: a systematic review. J Am Acad Child Adolesc Psychiatry. 2013;52:1134. e23-1147.e23. 18. Jellinek MS, Murphy JM, Burns BJ. Brief psychoso cial screening in outpatient pediatric practice. J Pediatr. 1986;109:371-378. 19. Simonian SJ, Tarnowski KJ. Utility of the Pediatric Symptom Checklist for behavioral screening of disadvantaged children. Child Psychiatry Hum Dev. 2001;31:269-278. 20. Walker WO Jr, LaGrone RG, Atkinson AW. Psychosocial screening in pediatric practice: identifying high-risk children. J Dev Behav Pediatr. 1989;10:134-138.
21. Gardner W, Kelleher KJ, Pajer KA, Campo JV. Primary care clinicians’ use of standardized tools to assess child psychosocial problems. Ambul Pediatr. 2003;3:191-195. 22. Pidano AE, Kimmelblatt CA, Neace WP. Behavioral health in the pediatric primary care setting: needs, barriers, and implications for psychologists. Psychol Serv. 2011;8:151-165. 23. Chung PJ, Lee TC, Morrison JL, Schuster MA. Preventive care for children in the United States: quality and barriers. Annu Rev Public Health. 2006;27:491-515. 24. Stevens GD, Seid M, Mistry R, Halfon N. Disparities in primary care for vulnerable children: the influence of multiple risk factors. Health Serv Res. 2006;41:507-531. 25. Hacker K, Goldstein J, Link D, et al. Pediatric provider processes for behavioral health screening, decision making, and referral in sites with colocated mental health services. J Dev Behav Pediatr. 2013;34:680-687. 26. Proctor EK, Landsverk J, Aarons G, Chambers D, Glisson C, Mittman B. Implementation research in mental health services: an emerging science with conceptual, methodological, and training challenges. Adm Policy Ment Health. 2009;36:24-34. 27. Van Cleave J, Kuhlthau KA, Bloom S, et al. Interventions to improve screening and follow-up in primary care: a systematic review of the evidence. Acad Pediatr. 2012;12:269-282. 28. Weitzman CC, Leventhal JM. Screening for behavioral health problems in primary care. Curr Opin Pediatr. 2006;18:641-648.
Author Biographies Rachel J. Valleley, PhD, is an associate professor of Psychology and Pediatrics at the Munroe-Meyer Institute and the University of Nebraska Medical Center. Her main research interests involve integrating behavioral health services into pediatric practices. Natalie Romer, PhD, is an assistant professor in the Department of Child and Family Studies at the University of South Florida. Her main research interests include universal screening, strengthbased assessment, and implementation of multi-tiered behavior and mental health support systems. Sara Kupzyk, PhD, is an assistant professor of Psychology with the Munroe-Meyer Institute at the University of Nebraska Medical Center. She provides outpatient behavioral health services in pediatric primary care clinics. Joseph H. Evans is a professor and director of Psychology at the Munroe-Meyer Institute and at the University of Nebraska Medical Center. With training funds from the Health Resources and Services Administration, he has developed an “integrated behavioral health in pediatric primary care model” that is now being used in 32 rural and urban sites across Nebraska and that is being replicated in two additional states. Dr. Evans specializes in training psychology interns and post-doctoral fellows in methods to integrate behavioral services into patients’ primary care “medical homes.” Keith D. Allen, PhD, is a professor of Psychology and Pediatrics at the Munroe-Meyer Institute and the University of Nebraska Medical Center. His interests include pediatric pain management, health-related behavior in children, parent training, stress-related disorders, and management of distress during invasive medical and dental procedures.
Downloaded from jpc.sagepub.com at University of Sussex Library on July 26, 2015
