Substance Abuse
ISSN: 0889-7077 (Print) 1547-0164 (Online) Journal homepage: http://www.tandfonline.com/loi/wsub20
Implementing Adolescent Screening, Brief Intervention, and Referral to Treatment (SBIRT) Education in a Pediatric Residency Curriculum Patricia Schram MD, Sion K. Harris PhD, Shari Van Hook PA-C, MPH, Sara Forman MD, Enrico Mezzacappa MD, Roman Pavlyuk BA & Sharon Levy MD, MPH To cite this article: Patricia Schram MD, Sion K. Harris PhD, Shari Van Hook PA-C, MPH, Sara Forman MD, Enrico Mezzacappa MD, Roman Pavlyuk BA & Sharon Levy MD, MPH (2015) Implementing Adolescent Screening, Brief Intervention, and Referral to Treatment (SBIRT) Education in a Pediatric Residency Curriculum, Substance Abuse, 36:3, 332-338, DOI: 10.1080/08897077.2014.936576 To link to this article: http://dx.doi.org/10.1080/08897077.2014.936576
Accepted author version posted online: 18 Jul 2014. Published online: 18 Jul 2014.
Submit your article to this journal
Article views: 78
View related articles
View Crossmark data
Citing articles: 3 View citing articles
Full Terms & Conditions of access and use can be found at http://www.tandfonline.com/action/journalInformation?journalCode=wsub20 Download by: [Washington University in St Louis]
Date: 05 November 2015, At: 22:46
SUBSTANCE ABUSE, 36: 332–338, 2015 Copyright Ó Taylor and Francis Group, LLC ISSN: 0889-7077 print / 1547-0164 online DOI: 10.1080/08897077.2014.936576
ORIGINAL RESEARCH
Downloaded by [Washington University in St Louis] at 22:46 05 November 2015
Implementing Adolescent Screening, Brief Intervention, and Referral to Treatment (SBIRT) Education in a Pediatric Residency Curriculum Patricia Schram, MD,1,2 Sion K. Harris, PhD,1,3 Shari Van Hook, PA-C, MPH,4 Sara Forman, MD,1,3 Enrico Mezzacappa, MD,5 Roman Pavlyuk, BA,1,2 and Sharon Levy, MD, MPH1,2 ABSTRACT. Background: Screening, brief intervention, and referral to treatment (SBIRT) is recommended as part of routine health care for adolescents as well as adults. In an effort to promote universal SBIRT, the Substance Abuse and Mental Health Services Administration awarded funding to residency programs to develop and implement SBIRT education and training. Our project focused on creating scientifically based, developmentally appropriate strategies and teaching materials for the adolescent age range. This paper describes curriculum development and implementation and presents evaluation data. Methods: Pediatric and child psychiatry residents were trained. The training consisted of 4 activities: (1) case-based teaching modules, (2) role-play of motivational interviewing and brief interventions, (3) mock interviews with trained adolescents, and (4) supervised “hands-on” screening and brief interventions. Main outcome measures included trainee satisfaction, and SBIRT knowledge, perceived self-efficacy, and self- and observer report of use of the SBIRT algorithm. Results: Among 150 total participants completing the SBIRT training modules, nearly all (92.3%) were satisfied/very satisfied with the training modules. Knowledge accuracy immediately post training was high, but declined significantly by the end of the first residency year, with little change across subsequent years of residency. Confidence ratings also declined over time. Use of the SBIRT algorithm during the Adolescent Medicine rotation was high according to trainee self- and faculty observer report. Conclusions: We found evidence of training satisfaction, increased confidence in talking to adolescents about substance use, and widespread use of recommended practices immediately following training. Use of a highly structured algorithm to guide practice, and simple, highly structured brief interventions was a successful training approach, as residents self-reported accurate use of the SBIRT algorithm immediately after training. Knowledge and self-confidence declined over time. It is possible that “booster” sessions and ongoing opportunities to review materials could help residents retain knowledge and skills.
Keywords: SBIRT training, adolescent substance abuse, residents, brief intervention 1 Department of Pediatrics, Harvard Medical School, Boston, Massachusetts, USA 2 Division of Developmental Medicine, Department of Medicine, Boston Children’s Hospital, Boston, Massachusetts, USA 3 Division of Adolescent/Young Adult Medicine, Department of Medicine, Boston Children’s Hospital, Boston, Massachusetts, USA 4 Inova Health System, Falls Church, Virginia, USA 5 Department of Psychiatry, Boston Children’s Hospital, Boston, Massachusetts, USA Correspondence should be addressed to Patricia Schram, MD, Center for Adolescent Substance Abuse Research, Boston Children’s Hospital, 300 Longwood Avenue, Fegan 10, Boston, MA 02115, USA. E-mail: [email protected]
INTRODUCTION The Substance Abuse and Mental Health Services Administration (SAMHSA)1 and the American Academy of Pediatrics2 recommend screening, brief intervention, and referral to treatment (SBIRT) for substance use as part of routine health care for all adolescents, although pediatricians self-reported rates of screening for substance use with validated tools are low.3,4 In an attempt to inculcate SBIRT into general medical practice, SAMHSA recently provided funding for 19 medical centers to develop, implement, and disseminate SBIRT
SCHRAM ET AL.
Downloaded by [Washington University in St Louis] at 22:46 05 November 2015
training materials for residents in a variety of medical disciplines.5 Medical students generally receive little instruction in the identification and treatment of substance use disorders.6 Most teaching materials that address substance use focus on medical management of the consequences of addiction, leaving medical school graduates entering pediatric residencies with very little experience in talking to adolescents about substance use, discriminating between low- and high-risk use, and making appropriate interventions. Our team focused our SBIRT training efforts on creating strategies and teaching materials specifically targeting adolescent patients. This paper describes the process of curriculum development and implementation and presents evaluation data from the project.
METHODS Setting and Participants This project was conducted in Pediatric Residency (41 residents per year) and Child and Adolescent Psychiatry Residency (5 residents per year) programs. Prior to the initiation of this project, pediatric residents were exposed to SBIRT via a 2-year, SAMHSA-funded initiative to screen adolescents presenting for emergency care. Trained paraprofessionals (“health promotion advocates”) screened adolescent patients and conducted brief interventions as appropriate. Residents were aware of the screen results and were invited to observe the health promotion advocates, although there was no formal resident training component.
Curriculum Development SBIRT training consisted of 4 activities: (1) case-based teaching modules, (2) role-play of motivational interviewing and brief interventions, (3) mock interviews with trained adolescents, and (4) supervised “hands-on” screening and brief interventions conducted in Adolescent Medicine and Outpatient Psychiatry programs. SAMHSA monitored all aspects of project implementation and adherence to goals and timeline, and approved additions to the specific aims.
Case-based modules We developed a series of teaching modules that were integrated into the formal residency curriculum and taught by faculty. We pilot-tested 3 modules (SBIRT Overview, Motivational Interviewing Techniques, and ADHD and Substance Use) and found that residents had a clear preference for case-based presentations. Therefore, cases served as a focal point for discussion whenever possible. Throughout the project, we developed additional modules in response to requests from residents, clinicians, and other staff members. These new modules were presented at general teaching sessions when requested. Each module had both “presentation” and “individual tutorial” (which had the main teaching point directly on each slide). After completion, each module was reviewed by a topic expert identified by SAMHSA; a SAMHSA staff member reviewed final versions to ensure revisions were consistent with reviewer comments. Overall, we
333
developed a total of 11 modules, 6 were presented at least annually in a live lecture format, and all were available online (Table 1).
Brief Intervention Role-play In a 90-minute session, residents role-played brief vignettes in which they took turns playing an adolescent in various stages of “readiness to change,”7 or a clinician. Residents swapped roles if they felt “stuck” or wanted to offer a suggestion. Faculty then used vignettes with challenging scenarios as a basis for group discussion and asked residents to reflect on previous experiences and how they may have handled them differently.
Mock interviews During their Adolescent Medicine rotation, pediatric residents conducted mock interviews with “adolescent educators,” i.e., adolescents who were trained to portray characters with complex social histories, including substance use. Residents were expected to use the SBIRT algorithm8,9 and then counsel appropriately based on what they had learned in the training modules. Residents received feedback from both a faculty member and the teenage “adolescent educator.”
Clinical algorithm To facilitate SBIRT practice, we developed a decision-based algorithm that we produced as a single-page reference card in order to reinforce learning and confidence.2 The algorithm prompted residents to screen with the CRAFFT,8,9 give positive feedback to adolescents that reported no past-year use, brief advice to those with “low risk” (i.e., CRAFFT D 0 or 1) or conduct a brief motivational intervention with those with “high risk “ (i.e., CRAFFT 2). For a detailed description of the clinical algorithm, see the American Academy of Pediatrics (AAP) Adolescent SBIRT policy statement.2
Curriculum Delivery Pediatrics residents First-year pediatrics residents received the SBIRT Overview module, Motivational Interviewing (MI) Techniques module, and TABLE 1 Modules Developed Presented live to pediatrics and psychiatry residents: SBIRT Overview Motivational Interviewing Techniques ADHD and Substance Use Drug Testing Pain Management Opioid Dependence* Presented at request of the chief residents: Inhalants Abuse Confidentiality Parent Guidance Smoking Cessation Neurobiology and the Impact of Alcohol and Marijuana on the Developing Brain Note. All modules available online. *Optional for pediatrics residents.
334
SUBSTANCE ABUSE
a mock interview in small groups (2–3 residents) during a required month-long block in Adolescent Medicine, and a module on attention-deficit/hyperactivity disorder (ADHD) and substance abuse during a required rotation in Developmental Medicine. Cases for this module highlighted screening, brief interventions, anticipatory guidance, and safe prescribing practices for youth with ADHD. Modules on pain management and drug testing were presented by a faculty member to residents from all 3 training years and repeated at least yearly at regularly scheduled noon time conferences. Two modules, “Inhalants Abuse” and “Neurobiology and the Impact of Alcohol and Marijuana on the Developing Brain,” were taught by faculty upon request by chief residents.
Downloaded by [Washington University in St Louis] at 22:46 05 November 2015
Child psychiatry residents Modules were modified slightly for child psychiatry residents to ensure salience. For example, the sample brief interventions presented to them focused more on motivational interviewing techniques that can be accommodated in therapy sessions as opposed to brief interventions designed to be delivered in a general medical session. Each core training module was delivered by a faculty member to the entire group of first-year child psychiatry trainees. An additional module on opioid dependence was delivered during the psychopharmacology lecture series.
Evaluation Measures Our evaluation assessed the following outcomes: trainee satisfaction, knowledge about SBIRT, perceived self-efficacy to conduct screening and a brief motivational intervention, and self- and observer report of use of the SBIRT algorithm. This project was not subject to IRB review as it was a training and quality improvement program.
Trainee satisfaction and feedback Immediately after each training module, participants completed SAMHSA’s Center for Substance Abuse Treatment (CSAT) Baseline Training Satisfaction Survey10 consisting of 24 items eliciting ratings of the quality and perceived utility of the training. The 18-item CSAT Training Follow-Up survey, which reassessed perceived utility and benefits of the training11 was administered 1 month later. Both baseline and follow-up surveys asked open-ended questions about what was most useful and how training could be improved.
Knowledge We administered brief questionnaires, consisting of multiplechoice or short-response items, after each training module to assess posttraining knowledge accuracy, and at the end of each residency year (each July of 2009–2012) to assess long-term knowledge retention. The year-end questionnaire items were varied from year to year to avoid recollection bias. All knowledge questionnaires were completed anonymously, so we are unable to link data across time for an individual. After the MI Techniques module, participants were asked 2 open-ended items to test their understanding and ability to apply MI principles in a hypothetical encounter with a substance-involved patient. The first item asked for an example of an open-ended question in keeping with MI principles that could be used to initiate a discussion with the
patient. The second asked for a response to patient resistance that demonstrated MI technique. Members of the study team collaboratively developed a standardized coding scheme. Two independent reviewers (a psychologist and a physician) coded responses to the first item as either open-ended or not, and responses to the second item as demonstrating any MI principle (i.e., “developing a discrepancy,” “expressing empathy,” “supporting self-efficacy,” and “rolling with resistance”) or not. Discrepant ratings were reviewed and resolved by a third rater using the same coding scheme.
Self-efficacy After the SBIRT Overview and in each end-of-year survey, we asked trainees to rate their level of confidence in their ability to conduct screening and brief intervention. We assessed trainees’ confidence in their ability to implement 2 specific MI skills, developing a discrepancy and reengaging a resistant teen, by having them rate their confidence level immediately before and after MI training, and in each end-of-year survey. All confidence rating items used a scale of 1–10, with 10 being highest.
SBIRT practice measures. To assess use of the SBIRT algorithm during patient visits, we developed a 1-page visit tracking form for completion by trainees immediately after seeing an adolescent primary care patient. Trainees recorded whether they completed each of the necessary SBIRT steps as determined by the patients’ substance use profile, as well as any reasons for not doing so. This self-report form was implemented during 3 of the 4 years of the project. There were some gaps in data collection in which some residents either failed to receive the tracking form to complete, or failed to complete the form. Administrative and clinical staff supported and reminded residents whenever possible, although competing demands at times supplanted completing this form. Therefore, the number of forms returned was lower than the number of patients seen by the residents. On average, residents returned forms for about half of the physicals that they performed. Each trainee was also observed by a preceptor during a patient visit, with preceptors completing an observation form on trainees’ use of the SBIRT algorithm during that visit. Similar to the trainee visit tracking forms, there were a number of months (about 6 total) in which faculty members failed to receive the evaluation form because of inconsistent administrative support. Faculty observers completed the SBIRT observation form whenever it was included in their evaluation packet, and we included all forms that we received.
Data Analysis We computed response frequency percentages for training quality and satisfaction ratings immediately post training and at 1-month follow-up. Two coders reviewed text comments regarding training highlights and suggested improvements to identify the most common themes. For knowledge, we computed the percentage of items answered correctly, and calculated postmodule and year-end score means and standard deviations. For both knowledge scores and confidence ratings, we used 1-way analysis of variance (ANOVA) to compare means by trainee year (years 1–3), including all trainees across all years of data collection. We also analyzed the trend in scores over time for the first cohort of trainees, for whom we
SCHRAM ET AL.
have 3 full years of follow-up. For confidence ratings related to MI skills, we were able to link pre- and posttraining data and used the paired t test to assess pre-post change. For trainee- and observerreported use of the SBIRT algorithm during patient visits, we computed the percentage of visits in which each step of the SBIRT algorithm was correctly applied. We used a P < .05 level to indicate statistical significance and used SPSS version 19.0 (IBM, Armonk, NY) for all analyses.
RESULTS
Downloaded by [Washington University in St Louis] at 22:46 05 November 2015
Training Satisfaction and Feedback Between July 2009 and May 2013, a total of 182 residents attended at least one training module (65.7% females, 81.5% white non-Hispanic) and completed a training satisfaction questionnaire. Nearly all participants (92.3%) were satisfied/very satisfied with the training modules. Table 2 shows that the vast majority felt the training was relevant and useful, enhanced their clinical skills, and they expected to use the information gained from the training. Not surprisingly, immediately post training, only half felt effective when working in this topic area. At 1month follow-up, however, nearly all reported having applied what they learned to their work. Feedback from participants indicated that the most useful aspects of the trainings included the easy-to-follow guides (utilizing mnemonics to aid recall) and practical tools for SBIRT (e.g., the algorithm and sample wording for use in brief counseling); the use of representative case stories; opportunities for practice through role-playing; and additional resources. The most common suggestions for improvement included provision of a manual, more opportunities for interaction and role-play, ongoing access to training materials beyond the initial training, and inclusion of examples showing difficult and challenging patient encounters.
Knowledge Among the 150 participants who completed the SBIRT Overview module across the 4-year evaluation period, knowledge accuracy immediately post training was high (Table 3). However, knowledge scores had declined significantly by the end of the first
335
residency year, with little change across subsequent years of residency. The observed trend over time was similar when comparing all first-year residents with all second- or third-year residents, or when examining the performance of the first cohort of trained residents (N D 35) over 3 subsequent years of follow-up. A total of 162 residents completed the end-of-year survey at the completion of their first year. Survey respondents that reported completing the SBIRT Overview module (n D 137) had significantly higher knowledge scores at the end of the first residency year than those that missed the training (n D 25) (mean percent accuracy score [95% confidence interval, CI]: 70.3 [67.1–73.4] vs. 60.0 [52.7–67.3]). However, for subsequent residency years, knowledge scores were not significantly different between training participants and nonparticipants (data not shown). Knowledge assessment immediately after the MI Techniques module indicated that nearly all trainees (95.6% of 114 participants) were able to formulate a MI-consistent open-ended question to begin a discussion about substance use with an adolescent. When this hypothetical patient showed resistance to the notion that there was a problem, the vast majority of trainees were able to formulate a response that demonstrated the MI principles of “rolling with resistance” (87.3%), “empathy” for the patient (78.2%), and helping to “develop a discrepancy” (74.5%) between the patient’s substance use and life values/goals.
Self-Efficacy Among those completing the SBIRT Overview module (N D 150), immediate-post-training confidence ratings were significantly higher for screening than for administering a brief intervention (Table 4), perhaps because the screening process is more structured and, therefore, easier to implement. Among MI Techniques module participants who completed both pre- and postmodule ratings (n D 109 out of 114), there was a significant pre- to posttest improvement in their confidence regarding MI skills (“develop a discrepancy” and “reengage a resistant teen”). However, as with knowledge scores, confidence ratings declined over time, likely due to lack of reinforcement and few occasions to practice these skills. Confidence ratings for MI skills fell to nearly pretest levels by the first end-of-year assessment, and continued to drop over subsequent years.
TABLE 2 Participant Evaluations of Training Immediately Post Training (N D 182) and at 1-Month Follow-Up (N D 131) Evaluation Ratings immediately post training I am currently effective when working in this topic area The material presented will be useful to me in dealing with substance abuse I expect to use the information gained from this training I expect this training to benefit my clients I would recommend this training to a colleague Follow-up ratings The material has been useful to me in dealing with substance abuse The training enhanced my skills in this area The training is relevant to my career This training has enabled me to serve my clients better Have you applied any of what you learned in the training to your work? (% Yes)
Strongly agree/Agree (%)
51.4 94.5 97.3 96.7 90.7 92.3 91.6 90.1 89.2 93.8
336
SUBSTANCE ABUSE TABLE 3 SBIRT Knowledge Scores Over Time Among Training Participantsa
Time point Post training End-of-year First-year trainees Second-year trainees Third-year trainees a
Downloaded by [Washington University in St Louis] at 22:46 05 November 2015
b
n
Knowledge score Mean percent correct (95% CI)
150
84.2 (82.0–86.3)
137 90 55
70.3 (67.1–73.4)b 66.1 (62.4–69.8)b 66.1 (61.2–71.2)b
SBIRT Overview module completers (N D 150). Significantly different from scores immediately post training, 2-tailed P < .05.
In their first end-of-year survey, residents who reported completing the MI Techniques module (n D 112) had a significantly higher average confidence rating for helping a teen “develop a discrepancy” than respondents who did not complete the module (n D 50) (6.3 [95% CI: 6.0–6.6] vs. 5.3 [95% CI: 4.9–5.8]). These 2 groups did not differ, however, in their confidence scores regarding “reengaging a resistant teen,” nor did they differ in confidence ratings for either MI skill on the end-of-year surveys after their second or third year (data not shown). Not surprisingly, confidence ratings for screening declined more slowly over time compared with intervention skills, with a significant decline not seen until the end of the second year. These findings highlight the importance of providing trainees with ongoing and easily accessible opportunities for learning and applying SBIRT skills.
Clinical Practice
(interquartile range [IQR]: 2–6). Among the 410 documented patients, 57.2% were female, and the mean age was 16.7 § 2.7 years (range: 11–24 years), similar to the clinic population as a whole. The residents reported consistent use of the SBIRT algorithm, completing an initial “quick-screen” for any past-12-month use on nearly all patients (96.1%), followed by the appropriate next steps in the SBIRT algorithm (e.g., giving praise and encouragement and asking the first question of the CRAFFT, CAR, to nonusing patients) at 92.5% of visits. In response to the CRAFFT screen result, residents conducted the appropriate next step (brief advice or brief assessment) at all recorded visits, and 71.1% of CRAFFT-positive patients had a follow-up arranged or were referred to treatment. The most common reasons cited for not arranging follow-up or referral to treatment were (1) the patient was already linked to treatment or working to reduce or quit use; (2) the patient declined further action; (3) the patient was unable to return due to distance or other barriers; and (4) the resident provided brief counseling during the visit.
Trainee self-report During the 3-year period in which residents completed visit tracking forms during their Adolescent Medicine rotation, 100 of 130 pediatric residents (76.9%) reported on a total of 410 patient visits. The median number of forms returned per resident was 3
Observer report During the period in which observation forms were completed on trainees, supervising faculty returned forms for 50 (64.1%) out of a total of 78. Similar to the findings from the trainee-completed
TABLE 4 Self-rated Confidencea in Screening and Brief Intervention Skills Over Time Among Training Participants Motivational Interviewing relatedb
Time point Pretraining Posttraining End-of-year survey First-year trainees Second-year trainees Third-year trainees a
n
Appropriately screen an adolescent for substance Conduct a brief intervention use Mean (95% CI) Mean (95% CI)
n
Reengage a teen who Help teen develop a has become resistant to discrepancy between behavior and medical recommendations stated goals Mean (95% CI) Mean (95% CI)
— 150
— 6.99 (6.78–7.21)
— 6.32 (6.06–6.58)
109b 109
6.14 (5.89-6.39) 7.46 (7.26–7.63)c
5.17 (4.88-5.46) 6.83 (6.59–7.06)c
137 90 55
6.57 (6.29–6.85) 6.08 (5.70–6.45)d,* 5.73 (5.28–6.17)d,*
6.05 (5.76–6.34) 5.70 (5.33–6.07) 5.62 (5.05–6.18)
112 82 53
6.27 (5.99–6.55) 5.65 (5.30–5.99)d,* 5.55 (5.07–6.02)d,*
5.22 (4.95–5.50) 4.56 (4.19–4.93)d,* 4.73 (4.27–5.20)
Rating scale 1–10 with 1 being lowest, 10 being highest. For motivational interviewing-related skills: both pre- and immediate-post-training ratings available on 109 trainees (out of 114 completing module); compared using paired t test. c Scores immediately post training higher than pretraining scores, 2-tailed P < .001. d Ratings at end of second/third year lower than at end of first year. *P < .05. b
SCHRAM ET AL.
forms, patient substance use screening was high among residents according to observer report, with 44 of 50 residents (88%) administering the “quick-screen” for substance use. Among the 44 patients screened, 37 (74.0%) had no past-12-month substance use, and 29 of the 37 (78.4%) appropriately received praise and encouragement and were asked the CAR question. Of the 13 patients identified with any substance use, 12 (96.7%) were administered the CRAFFT screen and 3 were found CRAFFTC. All except one patient screened with the CRAFFT received the appropriate follow-up advice or assessment and had follow-up or referral arranged.
Downloaded by [Washington University in St Louis] at 22:46 05 November 2015
DISCUSSION Although adding any new material to a residency curriculum can be challenging, adolescent SBIRT training was well received by residents and seemed to result in use of recommended practices immediately following the training. Use of a highly structured algorithm to guide practice and simple, highly structured brief interventions appear to be a successful training approach, as residents and preceptors consistently documented accurate use of the SBIRT algorithm immediately after the training. Our data indicated a decline in knowledge and self-confidence over time after training. Other programs have demonstrated that small-group sessions to educate residents about screening and brief intervention are effective.12,13 It is possible that these types of sessions administered as “booster” and ongoing opportunities to review materials could help residents retain knowledge and skills. Through individual tutorials, we offered open access and opportunity for resident learning despite restricted work hours. Future research could determine whether Web-based tutorials are as effective as in-person instruction and which methods are most effective for sustaining SBIRT practices. The extent to which residency graduates who received SBIRT training will continue to apply these skills in the practice setting is yet unknown. Ongoing SBIRT training for practicing physicians (through CME courses, certification requirements, etc.) may help disseminate and sustain these important practices within that population of practitioners. Our project had a number of limitations. We are describing outcomes of an educational curriculum that was not designed as a research project. Although most residents attended the in-person teaching sessions, some missed them because of other responsibilities or work hour limitations. We provided online modules to all residents but were not able to track who completed them, or in what depth. We do not know exactly how much exposure each resident got to materials—if residents who missed in-person sessions reviewed self-tutorials nor do we know if some residents reviewed on line materials after participating in a training session and in so doing received additional exposure. Since many training programs have similar infrastructure for resident teaching, we believe our results will be of interest. We did not assess SBIRT knowledge or practices before training, and it is possible that some of our learners may have had SBIRT experience before participating in the curriculum. However, the decline in knowledge scores from immediate post training to end-of-year follow-ups suggests that SBIRT knowledge needs reinforcement to be maintained over time. Although we had an observed measure of adherence to the SBIRT algorithm (the faculty observation forms), we did not assess brief intervention
337
quality or competence. Nonetheless, we believe our findings are likely generalizable to other similar settings and thus are informative. Our findings suggest that perhaps the greatest challenge is to help residents continue to improve their SBIRT skills over time. Shorter, more highly structured brief interventions might be easier for learners to use, and we have revised training materials to include a more highly structured version of the “brief motivational intervention.” Providing more opportunities for residents to practice SBIRT skills outside of the official curriculum training sites—such as inpatient units and subspecialty clinics—could help residents retain and refine skills. In this regard, there is reason to be hopeful. Adolescent SBIRT was officially recommended in American Academy of Pediatric policy guidelines published in 2011,2 and the National Institute on Alcoholism and Alcohol Abuse produced a youth alcohol screening and brief intervention guide for providers14 in order to increase SBIRT practice. The Substance Abuse and Mental Health Services Administration has invested substantial resources into SBIRT training and dissemination and the Joint Commission on Accreditation of Healthcare Organizations (JCAHO) has developed standardized performance measures for hospitals on tobacco and alcohol screening and cessation counseling.15 The National Institute on Drug Abuse (NIDA) also delineated a set of standard core data elements related to drug use for inclusion in all electronic health record systems and is funding a large SBIRT outreach and training initiative targeting health care providers.16,17 The ongoing investment in SBIRT research, policies, guidelines, recommendations, and trainings makes it likely that this practice will continue to spread and normalize and that future residents will be exposed to SBIRT as part of routine medical practice for every patient encounter.
ACKNOWLEDGMENTS The authors would like to acknowledge Drs. Christina Nordt, Susan Gray, and Miriam Schizer for their contributions as faculty teaching the SBIRT curriculum; and Rosemary Ziemnik and Joy Gabrielli for their help in manuscript preparation.
FUNDING This research was supported by SAMHSA grant 020267. SAMHSA monitored adherence to goals and timeline, approved additions to the specific aims, and funded a Technical Assistance grant with JBS International to conduct peer review of the modules. SAMHSA did not participate in data analysis or preparation of this publication.
AUTHOR CONTRIBUTIONS Patricia Schram was involved in the project design, development of the modules, and collection of the forms, and analysis and interpretation of some of the results. She wrote, edited, and revised the manuscript. Sharon Levy was the Principal Investigator (PI) of this project, and provided oversight for all aspects of development
Downloaded by [Washington University in St Louis] at 22:46 05 November 2015
338
SUBSTANCE ABUSE
and implementation. She was involved in development of teaching materials, training oversight, and data analysis and interpretation. She participated in writing, editing, and revising the manuscript. Patricia Schram and Sharon Levy are responsible for the integrity of the work as a whole, from inception to published article. Sion Kim Harris was involved in project design, development of measures, data analysis and interpretation, and drafting, editing, and final approval of the manuscript. Roman Pavlyuk was involved in the project implementation, collection and analysis of data, as well editing and revising the manuscript. Shari Van Hook was involved in the project design and management as well as review of results, and editing and revising the manuscript. Sara Forman was involved in design of some of the resident evaluation forms (resident observation and SBIRT forms) and helped develop the first overview module. She was also involved with teaching and implementation of the teaching protocol on the Adolescent Clinic, and in collection of data forms from residents. Dr. Forman also assisted in data interpretation and writing and revising the paper. Enrico Mezzacappa was involved in the conception, design, and adaptation of the modules for the child psychiatry residents, the collection of data (from the child psychiatry residents), and writing and revising of the manuscript. All authors agree to be accountable for all aspects of the work in ensuring that questions related to the accuracy or integrity of any part of the work are appropriately investigated and resolved.
REFERENCES [1] Substance Abuse and Mental Health Services Administration. Screening, Brief Intervention, and Referral to Treatment: What is SBIRT? Rockville, MD: Center for Substance Abuse Treatment; 2009. Available at: http://sbirt.samhsa.gov/index.htm. Accessed August 15, 2013. [2] Levy SJ, Kokotailo PK. Substance use screening, brief intervention, and referral to treatment for pediatricians. Pediatrics. 2011;128:e1330– e1340. Available at: http://www.ncbi.nlm.nih.gov/entrez/query.fcgi? cmd=Retrieve&db=PubMed&dopt=Citation&list_uids=22042818. Accessed August 15, 2013. [3] Ozer EM, Adams SH, Lustig JL, et al. Increasing the screening and counseling of adolescents for risky health behaviors: a primary care intervention. Pediatrics. 2005;115:960–968. [4] Millstein SG, Marcell A V. Screening and counseling for adolescent alcohol use among primary care physicians in the United States. Pediatrics. 2003;111:114–122.
[5] Gordon AJ. Alford DP. SBIRT curricular innovations: addressing a training gap. Subst Abuse. 2012;33:227–230. [6] O’Connor PG, Nyquist JG, McLellan AT. Integrating addiction medicine into graduate medical education in primary care: the time has come. Ann Intern Med. 2011;154:56–59. [7] Prochaska JO, DiClemente CC. The Transtheoretical Approach: Crossing Traditional Boundaries of Therapy. Homewood, IL: Dow/ Jones, Irwin; 1984. [8] Knight JR, Sherritt L, Shrier LA, Harris SK, Chang G. Validity of the CRAFFT substance abuse screening test among adolescent clinic patients. Arch Pediatr Adolesc Med. 2002;156:607–614. Available at: http://www.ncbi.nlm.nih.gov/entrez/query.fcgi?cmd=Retrie ve&db=PubMed&dopt = Citation&list_uids=12038895. Accessed August 15, 2013. [9] Dhalla S, Zumbo BD, Poole G, Poolem G. A review of the psychometric properties of the CRAFFT instrument. Curr Drug Abus Rev. 2011;4:57–64. [10] Substance Abuse and Mental Health Services Administration. Center for Substance Abuse Treatment Baseline Satisfaction Survey Training Form. Rockville, MD: Substance Abuse and Mental Health Services Administration; 2007. [11] Substance Abuse and Mental Health Services Administration. Center for Substance Abuse Treatment Training Follow-up Satisfaction Survey. Rockville, MD: Substance Abuse and Mental Health Services Administration; 2007. [12] Nigwekar SU, Morse DS. Educational half day: an innovative way to incorporate substance abuse curriculum into residency training. Subst Abus. 2006;27:1–3. [13] Ryan SA, Martel S, Pantalon M, et al. Screening, brief intervention, and referral to treatment (SBIRT) for alcohol and other drug use among adolescents: evaluation of a pediatric residency curriculum. Subst Abus. 2012;33:251–260. [14] National Institute on Alcohol Abuse and Alcoholism. Alcohol Screening and Brief Intervention for Youth: A Practitioner’s Guide. Bethesda, MD: National Institutes of Health; 2011. NIH Publication No. 11-7805. [15] The Joint Commission on Accreditation of Healthcare Organizations. Substance Use Core Measure Set. Available at: http://www. jointcommission.org/substance_use. Accessed August 15, 2013. [16] Ghitza UE, Gore-Langton RE, Lindblad R, Shide D, Subramaniam G, Tai B. Common data elements for substance use disorders in electronic health records: the NIDA Clinical Trials Network experience. Addiction. 2013;108:3–8. [17] National Institute on Drug Abuse. Request for Proposal (RFP) No. N01DA-14-1152. Outreach and Education to Health Care Providers on Substance Use. Available at: https://www.fbo.gov/ index?s=opportunity&mode=form&tab=core&id=f193868abdcba2 25871b8022eac46e16&_cview=0. Published 2014. Accessed August 15, 2013.
ISSN: 0889-7077 (Print) 1547-0164 (Online) Journal homepage: http://www.tandfonline.com/loi/wsub20
Implementing Adolescent Screening, Brief Intervention, and Referral to Treatment (SBIRT) Education in a Pediatric Residency Curriculum Patricia Schram MD, Sion K. Harris PhD, Shari Van Hook PA-C, MPH, Sara Forman MD, Enrico Mezzacappa MD, Roman Pavlyuk BA & Sharon Levy MD, MPH To cite this article: Patricia Schram MD, Sion K. Harris PhD, Shari Van Hook PA-C, MPH, Sara Forman MD, Enrico Mezzacappa MD, Roman Pavlyuk BA & Sharon Levy MD, MPH (2015) Implementing Adolescent Screening, Brief Intervention, and Referral to Treatment (SBIRT) Education in a Pediatric Residency Curriculum, Substance Abuse, 36:3, 332-338, DOI: 10.1080/08897077.2014.936576 To link to this article: http://dx.doi.org/10.1080/08897077.2014.936576
Accepted author version posted online: 18 Jul 2014. Published online: 18 Jul 2014.
Submit your article to this journal
Article views: 78
View related articles
View Crossmark data
Citing articles: 3 View citing articles
Full Terms & Conditions of access and use can be found at http://www.tandfonline.com/action/journalInformation?journalCode=wsub20 Download by: [Washington University in St Louis]
Date: 05 November 2015, At: 22:46
SUBSTANCE ABUSE, 36: 332–338, 2015 Copyright Ó Taylor and Francis Group, LLC ISSN: 0889-7077 print / 1547-0164 online DOI: 10.1080/08897077.2014.936576
ORIGINAL RESEARCH
Downloaded by [Washington University in St Louis] at 22:46 05 November 2015
Implementing Adolescent Screening, Brief Intervention, and Referral to Treatment (SBIRT) Education in a Pediatric Residency Curriculum Patricia Schram, MD,1,2 Sion K. Harris, PhD,1,3 Shari Van Hook, PA-C, MPH,4 Sara Forman, MD,1,3 Enrico Mezzacappa, MD,5 Roman Pavlyuk, BA,1,2 and Sharon Levy, MD, MPH1,2 ABSTRACT. Background: Screening, brief intervention, and referral to treatment (SBIRT) is recommended as part of routine health care for adolescents as well as adults. In an effort to promote universal SBIRT, the Substance Abuse and Mental Health Services Administration awarded funding to residency programs to develop and implement SBIRT education and training. Our project focused on creating scientifically based, developmentally appropriate strategies and teaching materials for the adolescent age range. This paper describes curriculum development and implementation and presents evaluation data. Methods: Pediatric and child psychiatry residents were trained. The training consisted of 4 activities: (1) case-based teaching modules, (2) role-play of motivational interviewing and brief interventions, (3) mock interviews with trained adolescents, and (4) supervised “hands-on” screening and brief interventions. Main outcome measures included trainee satisfaction, and SBIRT knowledge, perceived self-efficacy, and self- and observer report of use of the SBIRT algorithm. Results: Among 150 total participants completing the SBIRT training modules, nearly all (92.3%) were satisfied/very satisfied with the training modules. Knowledge accuracy immediately post training was high, but declined significantly by the end of the first residency year, with little change across subsequent years of residency. Confidence ratings also declined over time. Use of the SBIRT algorithm during the Adolescent Medicine rotation was high according to trainee self- and faculty observer report. Conclusions: We found evidence of training satisfaction, increased confidence in talking to adolescents about substance use, and widespread use of recommended practices immediately following training. Use of a highly structured algorithm to guide practice, and simple, highly structured brief interventions was a successful training approach, as residents self-reported accurate use of the SBIRT algorithm immediately after training. Knowledge and self-confidence declined over time. It is possible that “booster” sessions and ongoing opportunities to review materials could help residents retain knowledge and skills.
Keywords: SBIRT training, adolescent substance abuse, residents, brief intervention 1 Department of Pediatrics, Harvard Medical School, Boston, Massachusetts, USA 2 Division of Developmental Medicine, Department of Medicine, Boston Children’s Hospital, Boston, Massachusetts, USA 3 Division of Adolescent/Young Adult Medicine, Department of Medicine, Boston Children’s Hospital, Boston, Massachusetts, USA 4 Inova Health System, Falls Church, Virginia, USA 5 Department of Psychiatry, Boston Children’s Hospital, Boston, Massachusetts, USA Correspondence should be addressed to Patricia Schram, MD, Center for Adolescent Substance Abuse Research, Boston Children’s Hospital, 300 Longwood Avenue, Fegan 10, Boston, MA 02115, USA. E-mail: [email protected]
INTRODUCTION The Substance Abuse and Mental Health Services Administration (SAMHSA)1 and the American Academy of Pediatrics2 recommend screening, brief intervention, and referral to treatment (SBIRT) for substance use as part of routine health care for all adolescents, although pediatricians self-reported rates of screening for substance use with validated tools are low.3,4 In an attempt to inculcate SBIRT into general medical practice, SAMHSA recently provided funding for 19 medical centers to develop, implement, and disseminate SBIRT
SCHRAM ET AL.
Downloaded by [Washington University in St Louis] at 22:46 05 November 2015
training materials for residents in a variety of medical disciplines.5 Medical students generally receive little instruction in the identification and treatment of substance use disorders.6 Most teaching materials that address substance use focus on medical management of the consequences of addiction, leaving medical school graduates entering pediatric residencies with very little experience in talking to adolescents about substance use, discriminating between low- and high-risk use, and making appropriate interventions. Our team focused our SBIRT training efforts on creating strategies and teaching materials specifically targeting adolescent patients. This paper describes the process of curriculum development and implementation and presents evaluation data from the project.
METHODS Setting and Participants This project was conducted in Pediatric Residency (41 residents per year) and Child and Adolescent Psychiatry Residency (5 residents per year) programs. Prior to the initiation of this project, pediatric residents were exposed to SBIRT via a 2-year, SAMHSA-funded initiative to screen adolescents presenting for emergency care. Trained paraprofessionals (“health promotion advocates”) screened adolescent patients and conducted brief interventions as appropriate. Residents were aware of the screen results and were invited to observe the health promotion advocates, although there was no formal resident training component.
Curriculum Development SBIRT training consisted of 4 activities: (1) case-based teaching modules, (2) role-play of motivational interviewing and brief interventions, (3) mock interviews with trained adolescents, and (4) supervised “hands-on” screening and brief interventions conducted in Adolescent Medicine and Outpatient Psychiatry programs. SAMHSA monitored all aspects of project implementation and adherence to goals and timeline, and approved additions to the specific aims.
Case-based modules We developed a series of teaching modules that were integrated into the formal residency curriculum and taught by faculty. We pilot-tested 3 modules (SBIRT Overview, Motivational Interviewing Techniques, and ADHD and Substance Use) and found that residents had a clear preference for case-based presentations. Therefore, cases served as a focal point for discussion whenever possible. Throughout the project, we developed additional modules in response to requests from residents, clinicians, and other staff members. These new modules were presented at general teaching sessions when requested. Each module had both “presentation” and “individual tutorial” (which had the main teaching point directly on each slide). After completion, each module was reviewed by a topic expert identified by SAMHSA; a SAMHSA staff member reviewed final versions to ensure revisions were consistent with reviewer comments. Overall, we
333
developed a total of 11 modules, 6 were presented at least annually in a live lecture format, and all were available online (Table 1).
Brief Intervention Role-play In a 90-minute session, residents role-played brief vignettes in which they took turns playing an adolescent in various stages of “readiness to change,”7 or a clinician. Residents swapped roles if they felt “stuck” or wanted to offer a suggestion. Faculty then used vignettes with challenging scenarios as a basis for group discussion and asked residents to reflect on previous experiences and how they may have handled them differently.
Mock interviews During their Adolescent Medicine rotation, pediatric residents conducted mock interviews with “adolescent educators,” i.e., adolescents who were trained to portray characters with complex social histories, including substance use. Residents were expected to use the SBIRT algorithm8,9 and then counsel appropriately based on what they had learned in the training modules. Residents received feedback from both a faculty member and the teenage “adolescent educator.”
Clinical algorithm To facilitate SBIRT practice, we developed a decision-based algorithm that we produced as a single-page reference card in order to reinforce learning and confidence.2 The algorithm prompted residents to screen with the CRAFFT,8,9 give positive feedback to adolescents that reported no past-year use, brief advice to those with “low risk” (i.e., CRAFFT D 0 or 1) or conduct a brief motivational intervention with those with “high risk “ (i.e., CRAFFT 2). For a detailed description of the clinical algorithm, see the American Academy of Pediatrics (AAP) Adolescent SBIRT policy statement.2
Curriculum Delivery Pediatrics residents First-year pediatrics residents received the SBIRT Overview module, Motivational Interviewing (MI) Techniques module, and TABLE 1 Modules Developed Presented live to pediatrics and psychiatry residents: SBIRT Overview Motivational Interviewing Techniques ADHD and Substance Use Drug Testing Pain Management Opioid Dependence* Presented at request of the chief residents: Inhalants Abuse Confidentiality Parent Guidance Smoking Cessation Neurobiology and the Impact of Alcohol and Marijuana on the Developing Brain Note. All modules available online. *Optional for pediatrics residents.
334
SUBSTANCE ABUSE
a mock interview in small groups (2–3 residents) during a required month-long block in Adolescent Medicine, and a module on attention-deficit/hyperactivity disorder (ADHD) and substance abuse during a required rotation in Developmental Medicine. Cases for this module highlighted screening, brief interventions, anticipatory guidance, and safe prescribing practices for youth with ADHD. Modules on pain management and drug testing were presented by a faculty member to residents from all 3 training years and repeated at least yearly at regularly scheduled noon time conferences. Two modules, “Inhalants Abuse” and “Neurobiology and the Impact of Alcohol and Marijuana on the Developing Brain,” were taught by faculty upon request by chief residents.
Downloaded by [Washington University in St Louis] at 22:46 05 November 2015
Child psychiatry residents Modules were modified slightly for child psychiatry residents to ensure salience. For example, the sample brief interventions presented to them focused more on motivational interviewing techniques that can be accommodated in therapy sessions as opposed to brief interventions designed to be delivered in a general medical session. Each core training module was delivered by a faculty member to the entire group of first-year child psychiatry trainees. An additional module on opioid dependence was delivered during the psychopharmacology lecture series.
Evaluation Measures Our evaluation assessed the following outcomes: trainee satisfaction, knowledge about SBIRT, perceived self-efficacy to conduct screening and a brief motivational intervention, and self- and observer report of use of the SBIRT algorithm. This project was not subject to IRB review as it was a training and quality improvement program.
Trainee satisfaction and feedback Immediately after each training module, participants completed SAMHSA’s Center for Substance Abuse Treatment (CSAT) Baseline Training Satisfaction Survey10 consisting of 24 items eliciting ratings of the quality and perceived utility of the training. The 18-item CSAT Training Follow-Up survey, which reassessed perceived utility and benefits of the training11 was administered 1 month later. Both baseline and follow-up surveys asked open-ended questions about what was most useful and how training could be improved.
Knowledge We administered brief questionnaires, consisting of multiplechoice or short-response items, after each training module to assess posttraining knowledge accuracy, and at the end of each residency year (each July of 2009–2012) to assess long-term knowledge retention. The year-end questionnaire items were varied from year to year to avoid recollection bias. All knowledge questionnaires were completed anonymously, so we are unable to link data across time for an individual. After the MI Techniques module, participants were asked 2 open-ended items to test their understanding and ability to apply MI principles in a hypothetical encounter with a substance-involved patient. The first item asked for an example of an open-ended question in keeping with MI principles that could be used to initiate a discussion with the
patient. The second asked for a response to patient resistance that demonstrated MI technique. Members of the study team collaboratively developed a standardized coding scheme. Two independent reviewers (a psychologist and a physician) coded responses to the first item as either open-ended or not, and responses to the second item as demonstrating any MI principle (i.e., “developing a discrepancy,” “expressing empathy,” “supporting self-efficacy,” and “rolling with resistance”) or not. Discrepant ratings were reviewed and resolved by a third rater using the same coding scheme.
Self-efficacy After the SBIRT Overview and in each end-of-year survey, we asked trainees to rate their level of confidence in their ability to conduct screening and brief intervention. We assessed trainees’ confidence in their ability to implement 2 specific MI skills, developing a discrepancy and reengaging a resistant teen, by having them rate their confidence level immediately before and after MI training, and in each end-of-year survey. All confidence rating items used a scale of 1–10, with 10 being highest.
SBIRT practice measures. To assess use of the SBIRT algorithm during patient visits, we developed a 1-page visit tracking form for completion by trainees immediately after seeing an adolescent primary care patient. Trainees recorded whether they completed each of the necessary SBIRT steps as determined by the patients’ substance use profile, as well as any reasons for not doing so. This self-report form was implemented during 3 of the 4 years of the project. There were some gaps in data collection in which some residents either failed to receive the tracking form to complete, or failed to complete the form. Administrative and clinical staff supported and reminded residents whenever possible, although competing demands at times supplanted completing this form. Therefore, the number of forms returned was lower than the number of patients seen by the residents. On average, residents returned forms for about half of the physicals that they performed. Each trainee was also observed by a preceptor during a patient visit, with preceptors completing an observation form on trainees’ use of the SBIRT algorithm during that visit. Similar to the trainee visit tracking forms, there were a number of months (about 6 total) in which faculty members failed to receive the evaluation form because of inconsistent administrative support. Faculty observers completed the SBIRT observation form whenever it was included in their evaluation packet, and we included all forms that we received.
Data Analysis We computed response frequency percentages for training quality and satisfaction ratings immediately post training and at 1-month follow-up. Two coders reviewed text comments regarding training highlights and suggested improvements to identify the most common themes. For knowledge, we computed the percentage of items answered correctly, and calculated postmodule and year-end score means and standard deviations. For both knowledge scores and confidence ratings, we used 1-way analysis of variance (ANOVA) to compare means by trainee year (years 1–3), including all trainees across all years of data collection. We also analyzed the trend in scores over time for the first cohort of trainees, for whom we
SCHRAM ET AL.
have 3 full years of follow-up. For confidence ratings related to MI skills, we were able to link pre- and posttraining data and used the paired t test to assess pre-post change. For trainee- and observerreported use of the SBIRT algorithm during patient visits, we computed the percentage of visits in which each step of the SBIRT algorithm was correctly applied. We used a P < .05 level to indicate statistical significance and used SPSS version 19.0 (IBM, Armonk, NY) for all analyses.
RESULTS
Downloaded by [Washington University in St Louis] at 22:46 05 November 2015
Training Satisfaction and Feedback Between July 2009 and May 2013, a total of 182 residents attended at least one training module (65.7% females, 81.5% white non-Hispanic) and completed a training satisfaction questionnaire. Nearly all participants (92.3%) were satisfied/very satisfied with the training modules. Table 2 shows that the vast majority felt the training was relevant and useful, enhanced their clinical skills, and they expected to use the information gained from the training. Not surprisingly, immediately post training, only half felt effective when working in this topic area. At 1month follow-up, however, nearly all reported having applied what they learned to their work. Feedback from participants indicated that the most useful aspects of the trainings included the easy-to-follow guides (utilizing mnemonics to aid recall) and practical tools for SBIRT (e.g., the algorithm and sample wording for use in brief counseling); the use of representative case stories; opportunities for practice through role-playing; and additional resources. The most common suggestions for improvement included provision of a manual, more opportunities for interaction and role-play, ongoing access to training materials beyond the initial training, and inclusion of examples showing difficult and challenging patient encounters.
Knowledge Among the 150 participants who completed the SBIRT Overview module across the 4-year evaluation period, knowledge accuracy immediately post training was high (Table 3). However, knowledge scores had declined significantly by the end of the first
335
residency year, with little change across subsequent years of residency. The observed trend over time was similar when comparing all first-year residents with all second- or third-year residents, or when examining the performance of the first cohort of trained residents (N D 35) over 3 subsequent years of follow-up. A total of 162 residents completed the end-of-year survey at the completion of their first year. Survey respondents that reported completing the SBIRT Overview module (n D 137) had significantly higher knowledge scores at the end of the first residency year than those that missed the training (n D 25) (mean percent accuracy score [95% confidence interval, CI]: 70.3 [67.1–73.4] vs. 60.0 [52.7–67.3]). However, for subsequent residency years, knowledge scores were not significantly different between training participants and nonparticipants (data not shown). Knowledge assessment immediately after the MI Techniques module indicated that nearly all trainees (95.6% of 114 participants) were able to formulate a MI-consistent open-ended question to begin a discussion about substance use with an adolescent. When this hypothetical patient showed resistance to the notion that there was a problem, the vast majority of trainees were able to formulate a response that demonstrated the MI principles of “rolling with resistance” (87.3%), “empathy” for the patient (78.2%), and helping to “develop a discrepancy” (74.5%) between the patient’s substance use and life values/goals.
Self-Efficacy Among those completing the SBIRT Overview module (N D 150), immediate-post-training confidence ratings were significantly higher for screening than for administering a brief intervention (Table 4), perhaps because the screening process is more structured and, therefore, easier to implement. Among MI Techniques module participants who completed both pre- and postmodule ratings (n D 109 out of 114), there was a significant pre- to posttest improvement in their confidence regarding MI skills (“develop a discrepancy” and “reengage a resistant teen”). However, as with knowledge scores, confidence ratings declined over time, likely due to lack of reinforcement and few occasions to practice these skills. Confidence ratings for MI skills fell to nearly pretest levels by the first end-of-year assessment, and continued to drop over subsequent years.
TABLE 2 Participant Evaluations of Training Immediately Post Training (N D 182) and at 1-Month Follow-Up (N D 131) Evaluation Ratings immediately post training I am currently effective when working in this topic area The material presented will be useful to me in dealing with substance abuse I expect to use the information gained from this training I expect this training to benefit my clients I would recommend this training to a colleague Follow-up ratings The material has been useful to me in dealing with substance abuse The training enhanced my skills in this area The training is relevant to my career This training has enabled me to serve my clients better Have you applied any of what you learned in the training to your work? (% Yes)
Strongly agree/Agree (%)
51.4 94.5 97.3 96.7 90.7 92.3 91.6 90.1 89.2 93.8
336
SUBSTANCE ABUSE TABLE 3 SBIRT Knowledge Scores Over Time Among Training Participantsa
Time point Post training End-of-year First-year trainees Second-year trainees Third-year trainees a
Downloaded by [Washington University in St Louis] at 22:46 05 November 2015
b
n
Knowledge score Mean percent correct (95% CI)
150
84.2 (82.0–86.3)
137 90 55
70.3 (67.1–73.4)b 66.1 (62.4–69.8)b 66.1 (61.2–71.2)b
SBIRT Overview module completers (N D 150). Significantly different from scores immediately post training, 2-tailed P < .05.
In their first end-of-year survey, residents who reported completing the MI Techniques module (n D 112) had a significantly higher average confidence rating for helping a teen “develop a discrepancy” than respondents who did not complete the module (n D 50) (6.3 [95% CI: 6.0–6.6] vs. 5.3 [95% CI: 4.9–5.8]). These 2 groups did not differ, however, in their confidence scores regarding “reengaging a resistant teen,” nor did they differ in confidence ratings for either MI skill on the end-of-year surveys after their second or third year (data not shown). Not surprisingly, confidence ratings for screening declined more slowly over time compared with intervention skills, with a significant decline not seen until the end of the second year. These findings highlight the importance of providing trainees with ongoing and easily accessible opportunities for learning and applying SBIRT skills.
Clinical Practice
(interquartile range [IQR]: 2–6). Among the 410 documented patients, 57.2% were female, and the mean age was 16.7 § 2.7 years (range: 11–24 years), similar to the clinic population as a whole. The residents reported consistent use of the SBIRT algorithm, completing an initial “quick-screen” for any past-12-month use on nearly all patients (96.1%), followed by the appropriate next steps in the SBIRT algorithm (e.g., giving praise and encouragement and asking the first question of the CRAFFT, CAR, to nonusing patients) at 92.5% of visits. In response to the CRAFFT screen result, residents conducted the appropriate next step (brief advice or brief assessment) at all recorded visits, and 71.1% of CRAFFT-positive patients had a follow-up arranged or were referred to treatment. The most common reasons cited for not arranging follow-up or referral to treatment were (1) the patient was already linked to treatment or working to reduce or quit use; (2) the patient declined further action; (3) the patient was unable to return due to distance or other barriers; and (4) the resident provided brief counseling during the visit.
Trainee self-report During the 3-year period in which residents completed visit tracking forms during their Adolescent Medicine rotation, 100 of 130 pediatric residents (76.9%) reported on a total of 410 patient visits. The median number of forms returned per resident was 3
Observer report During the period in which observation forms were completed on trainees, supervising faculty returned forms for 50 (64.1%) out of a total of 78. Similar to the findings from the trainee-completed
TABLE 4 Self-rated Confidencea in Screening and Brief Intervention Skills Over Time Among Training Participants Motivational Interviewing relatedb
Time point Pretraining Posttraining End-of-year survey First-year trainees Second-year trainees Third-year trainees a
n
Appropriately screen an adolescent for substance Conduct a brief intervention use Mean (95% CI) Mean (95% CI)
n
Reengage a teen who Help teen develop a has become resistant to discrepancy between behavior and medical recommendations stated goals Mean (95% CI) Mean (95% CI)
— 150
— 6.99 (6.78–7.21)
— 6.32 (6.06–6.58)
109b 109
6.14 (5.89-6.39) 7.46 (7.26–7.63)c
5.17 (4.88-5.46) 6.83 (6.59–7.06)c
137 90 55
6.57 (6.29–6.85) 6.08 (5.70–6.45)d,* 5.73 (5.28–6.17)d,*
6.05 (5.76–6.34) 5.70 (5.33–6.07) 5.62 (5.05–6.18)
112 82 53
6.27 (5.99–6.55) 5.65 (5.30–5.99)d,* 5.55 (5.07–6.02)d,*
5.22 (4.95–5.50) 4.56 (4.19–4.93)d,* 4.73 (4.27–5.20)
Rating scale 1–10 with 1 being lowest, 10 being highest. For motivational interviewing-related skills: both pre- and immediate-post-training ratings available on 109 trainees (out of 114 completing module); compared using paired t test. c Scores immediately post training higher than pretraining scores, 2-tailed P < .001. d Ratings at end of second/third year lower than at end of first year. *P < .05. b
SCHRAM ET AL.
forms, patient substance use screening was high among residents according to observer report, with 44 of 50 residents (88%) administering the “quick-screen” for substance use. Among the 44 patients screened, 37 (74.0%) had no past-12-month substance use, and 29 of the 37 (78.4%) appropriately received praise and encouragement and were asked the CAR question. Of the 13 patients identified with any substance use, 12 (96.7%) were administered the CRAFFT screen and 3 were found CRAFFTC. All except one patient screened with the CRAFFT received the appropriate follow-up advice or assessment and had follow-up or referral arranged.
Downloaded by [Washington University in St Louis] at 22:46 05 November 2015
DISCUSSION Although adding any new material to a residency curriculum can be challenging, adolescent SBIRT training was well received by residents and seemed to result in use of recommended practices immediately following the training. Use of a highly structured algorithm to guide practice and simple, highly structured brief interventions appear to be a successful training approach, as residents and preceptors consistently documented accurate use of the SBIRT algorithm immediately after the training. Our data indicated a decline in knowledge and self-confidence over time after training. Other programs have demonstrated that small-group sessions to educate residents about screening and brief intervention are effective.12,13 It is possible that these types of sessions administered as “booster” and ongoing opportunities to review materials could help residents retain knowledge and skills. Through individual tutorials, we offered open access and opportunity for resident learning despite restricted work hours. Future research could determine whether Web-based tutorials are as effective as in-person instruction and which methods are most effective for sustaining SBIRT practices. The extent to which residency graduates who received SBIRT training will continue to apply these skills in the practice setting is yet unknown. Ongoing SBIRT training for practicing physicians (through CME courses, certification requirements, etc.) may help disseminate and sustain these important practices within that population of practitioners. Our project had a number of limitations. We are describing outcomes of an educational curriculum that was not designed as a research project. Although most residents attended the in-person teaching sessions, some missed them because of other responsibilities or work hour limitations. We provided online modules to all residents but were not able to track who completed them, or in what depth. We do not know exactly how much exposure each resident got to materials—if residents who missed in-person sessions reviewed self-tutorials nor do we know if some residents reviewed on line materials after participating in a training session and in so doing received additional exposure. Since many training programs have similar infrastructure for resident teaching, we believe our results will be of interest. We did not assess SBIRT knowledge or practices before training, and it is possible that some of our learners may have had SBIRT experience before participating in the curriculum. However, the decline in knowledge scores from immediate post training to end-of-year follow-ups suggests that SBIRT knowledge needs reinforcement to be maintained over time. Although we had an observed measure of adherence to the SBIRT algorithm (the faculty observation forms), we did not assess brief intervention
337
quality or competence. Nonetheless, we believe our findings are likely generalizable to other similar settings and thus are informative. Our findings suggest that perhaps the greatest challenge is to help residents continue to improve their SBIRT skills over time. Shorter, more highly structured brief interventions might be easier for learners to use, and we have revised training materials to include a more highly structured version of the “brief motivational intervention.” Providing more opportunities for residents to practice SBIRT skills outside of the official curriculum training sites—such as inpatient units and subspecialty clinics—could help residents retain and refine skills. In this regard, there is reason to be hopeful. Adolescent SBIRT was officially recommended in American Academy of Pediatric policy guidelines published in 2011,2 and the National Institute on Alcoholism and Alcohol Abuse produced a youth alcohol screening and brief intervention guide for providers14 in order to increase SBIRT practice. The Substance Abuse and Mental Health Services Administration has invested substantial resources into SBIRT training and dissemination and the Joint Commission on Accreditation of Healthcare Organizations (JCAHO) has developed standardized performance measures for hospitals on tobacco and alcohol screening and cessation counseling.15 The National Institute on Drug Abuse (NIDA) also delineated a set of standard core data elements related to drug use for inclusion in all electronic health record systems and is funding a large SBIRT outreach and training initiative targeting health care providers.16,17 The ongoing investment in SBIRT research, policies, guidelines, recommendations, and trainings makes it likely that this practice will continue to spread and normalize and that future residents will be exposed to SBIRT as part of routine medical practice for every patient encounter.
ACKNOWLEDGMENTS The authors would like to acknowledge Drs. Christina Nordt, Susan Gray, and Miriam Schizer for their contributions as faculty teaching the SBIRT curriculum; and Rosemary Ziemnik and Joy Gabrielli for their help in manuscript preparation.
FUNDING This research was supported by SAMHSA grant 020267. SAMHSA monitored adherence to goals and timeline, approved additions to the specific aims, and funded a Technical Assistance grant with JBS International to conduct peer review of the modules. SAMHSA did not participate in data analysis or preparation of this publication.
AUTHOR CONTRIBUTIONS Patricia Schram was involved in the project design, development of the modules, and collection of the forms, and analysis and interpretation of some of the results. She wrote, edited, and revised the manuscript. Sharon Levy was the Principal Investigator (PI) of this project, and provided oversight for all aspects of development
Downloaded by [Washington University in St Louis] at 22:46 05 November 2015
338
SUBSTANCE ABUSE
and implementation. She was involved in development of teaching materials, training oversight, and data analysis and interpretation. She participated in writing, editing, and revising the manuscript. Patricia Schram and Sharon Levy are responsible for the integrity of the work as a whole, from inception to published article. Sion Kim Harris was involved in project design, development of measures, data analysis and interpretation, and drafting, editing, and final approval of the manuscript. Roman Pavlyuk was involved in the project implementation, collection and analysis of data, as well editing and revising the manuscript. Shari Van Hook was involved in the project design and management as well as review of results, and editing and revising the manuscript. Sara Forman was involved in design of some of the resident evaluation forms (resident observation and SBIRT forms) and helped develop the first overview module. She was also involved with teaching and implementation of the teaching protocol on the Adolescent Clinic, and in collection of data forms from residents. Dr. Forman also assisted in data interpretation and writing and revising the paper. Enrico Mezzacappa was involved in the conception, design, and adaptation of the modules for the child psychiatry residents, the collection of data (from the child psychiatry residents), and writing and revising of the manuscript. All authors agree to be accountable for all aspects of the work in ensuring that questions related to the accuracy or integrity of any part of the work are appropriately investigated and resolved.
REFERENCES [1] Substance Abuse and Mental Health Services Administration. Screening, Brief Intervention, and Referral to Treatment: What is SBIRT? Rockville, MD: Center for Substance Abuse Treatment; 2009. Available at: http://sbirt.samhsa.gov/index.htm. Accessed August 15, 2013. [2] Levy SJ, Kokotailo PK. Substance use screening, brief intervention, and referral to treatment for pediatricians. Pediatrics. 2011;128:e1330– e1340. Available at: http://www.ncbi.nlm.nih.gov/entrez/query.fcgi? cmd=Retrieve&db=PubMed&dopt=Citation&list_uids=22042818. Accessed August 15, 2013. [3] Ozer EM, Adams SH, Lustig JL, et al. Increasing the screening and counseling of adolescents for risky health behaviors: a primary care intervention. Pediatrics. 2005;115:960–968. [4] Millstein SG, Marcell A V. Screening and counseling for adolescent alcohol use among primary care physicians in the United States. Pediatrics. 2003;111:114–122.
[5] Gordon AJ. Alford DP. SBIRT curricular innovations: addressing a training gap. Subst Abuse. 2012;33:227–230. [6] O’Connor PG, Nyquist JG, McLellan AT. Integrating addiction medicine into graduate medical education in primary care: the time has come. Ann Intern Med. 2011;154:56–59. [7] Prochaska JO, DiClemente CC. The Transtheoretical Approach: Crossing Traditional Boundaries of Therapy. Homewood, IL: Dow/ Jones, Irwin; 1984. [8] Knight JR, Sherritt L, Shrier LA, Harris SK, Chang G. Validity of the CRAFFT substance abuse screening test among adolescent clinic patients. Arch Pediatr Adolesc Med. 2002;156:607–614. Available at: http://www.ncbi.nlm.nih.gov/entrez/query.fcgi?cmd=Retrie ve&db=PubMed&dopt = Citation&list_uids=12038895. Accessed August 15, 2013. [9] Dhalla S, Zumbo BD, Poole G, Poolem G. A review of the psychometric properties of the CRAFFT instrument. Curr Drug Abus Rev. 2011;4:57–64. [10] Substance Abuse and Mental Health Services Administration. Center for Substance Abuse Treatment Baseline Satisfaction Survey Training Form. Rockville, MD: Substance Abuse and Mental Health Services Administration; 2007. [11] Substance Abuse and Mental Health Services Administration. Center for Substance Abuse Treatment Training Follow-up Satisfaction Survey. Rockville, MD: Substance Abuse and Mental Health Services Administration; 2007. [12] Nigwekar SU, Morse DS. Educational half day: an innovative way to incorporate substance abuse curriculum into residency training. Subst Abus. 2006;27:1–3. [13] Ryan SA, Martel S, Pantalon M, et al. Screening, brief intervention, and referral to treatment (SBIRT) for alcohol and other drug use among adolescents: evaluation of a pediatric residency curriculum. Subst Abus. 2012;33:251–260. [14] National Institute on Alcohol Abuse and Alcoholism. Alcohol Screening and Brief Intervention for Youth: A Practitioner’s Guide. Bethesda, MD: National Institutes of Health; 2011. NIH Publication No. 11-7805. [15] The Joint Commission on Accreditation of Healthcare Organizations. Substance Use Core Measure Set. Available at: http://www. jointcommission.org/substance_use. Accessed August 15, 2013. [16] Ghitza UE, Gore-Langton RE, Lindblad R, Shide D, Subramaniam G, Tai B. Common data elements for substance use disorders in electronic health records: the NIDA Clinical Trials Network experience. Addiction. 2013;108:3–8. [17] National Institute on Drug Abuse. Request for Proposal (RFP) No. N01DA-14-1152. Outreach and Education to Health Care Providers on Substance Use. Available at: https://www.fbo.gov/ index?s=opportunity&mode=form&tab=core&id=f193868abdcba2 25871b8022eac46e16&_cview=0. Published 2014. Accessed August 15, 2013.
