This article was downloaded by: [Aston University] On: 06 September 2014, At: 11:31 Publisher: Routledge Informa Ltd Registered in England and Wales Registered Number: 1072954 Registered office: Mortimer House, 37-41 Mortimer Street, London W1T 3JH, UK
Journal of Clinical Child & Adolescent Psychology Publication details, including instructions for authors and subscription information: http://www.tandfonline.com/loi/hcap20
MAP as a Model for Practice-Based Learning and Improvement in Child Psychiatry Training Sheryl H. Kataoka
a b
& Martha Bates Jura a
a
, Jennifer L. Podell , Bonnie T. Zima
a b
a
, Karin Best , Shawn Sidhu
a
a
UCLA Semel Institute of Neuroscience and Human Behavior
b
Center for Health Services and Society Published online: 18 Nov 2013.
To cite this article: Sheryl H. Kataoka , Jennifer L. Podell , Bonnie T. Zima , Karin Best , Shawn Sidhu & Martha Bates Jura (2014) MAP as a Model for Practice-Based Learning and Improvement in Child Psychiatry Training, Journal of Clinical Child & Adolescent Psychology, 43:2, 312-322, DOI: 10.1080/15374416.2013.848773 To link to this article: http://dx.doi.org/10.1080/15374416.2013.848773
PLEASE SCROLL DOWN FOR ARTICLE Taylor & Francis makes every effort to ensure the accuracy of all the information (the “Content”) contained in the publications on our platform. However, Taylor & Francis, our agents, and our licensors make no representations or warranties whatsoever as to the accuracy, completeness, or suitability for any purpose of the Content. Any opinions and views expressed in this publication are the opinions and views of the authors, and are not the views of or endorsed by Taylor & Francis. The accuracy of the Content should not be relied upon and should be independently verified with primary sources of information. Taylor and Francis shall not be liable for any losses, actions, claims, proceedings, demands, costs, expenses, damages, and other liabilities whatsoever or howsoever caused arising directly or indirectly in connection with, in relation to or arising out of the use of the Content. This article may be used for research, teaching, and private study purposes. Any substantial or systematic reproduction, redistribution, reselling, loan, sub-licensing, systematic supply, or distribution in any form to anyone is expressly forbidden. Terms & Conditions of access and use can be found at http:// www.tandfonline.com/page/terms-and-conditions
Journal of Clinical Child & Adolescent Psychology, 43(2), 312–322, 2014 Copyright # Taylor & Francis Group, LLC ISSN: 1537-4416 print=1537-4424 online DOI: 10.1080/15374416.2013.848773
MAP as a Model for Practice-Based Learning and Improvement in Child Psychiatry Training Sheryl H. Kataoka UCLA Semel Institute of Neuroscience and Human Behavior, Center for Health Services and Society
Downloaded by [Aston University] at 11:31 06 September 2014
Jennifer L. Podell UCLA Semel Institute of Neuroscience and Human Behavior
Bonnie T. Zima UCLA Semel Institute of Neuroscience and Human Behavior, Center for Health Services and Society
Karin Best, Shawn Sidhu, and Martha Bates Jura UCLA Semel Institute of Neuroscience and Human Behavior
Not only is there a growing literature demonstrating the positive outcomes that result from implementing evidence based treatments (EBTs) but also studies that suggest a lack of delivery of these EBTs in ‘‘usual care’’ practices. One way to address this deficit is to improve the quality of psychotherapy teaching for clinicians-in-training. The Accreditation Council for Graduate Medical Education (ACGME) requires all training programs to assess residents in a number of competencies including Practice-Based Learning and Improvements (PBLI). This article describes the piloting of Managing and Adapting Practice (MAP) for child psychiatry fellows, to teach them both EBT and PBLI skills. Eight child psychiatry trainees received 5 full days of MAP training and are delivering MAP in a year-long outpatient teaching clinic. In this setting, MAP is applied to the complex, multiply diagnosed psychiatric patients that present to this clinic. This article describes how MAP tools and resources assist in teaching trainees each of the eight required competency components of PBLI, including identifying deficits in expertise, setting learning goals, performing learning activities, conducting quality improvement methods in practice, incorporating formative feedback, using scientific studies to inform practice, using technology for learning, and participating in patient education. A case example illustrates the use of MAP in teaching PBLI. MAP provides a unique way to teach important quality improvement and practicebased learning skills to trainees while training them in important psychotherapy competence. We thank Dr. Bruce Chorpita for training and consulting with our clinic, all of the Child and Adolescent Psychiatry Fellows and Psychology Interns that participated in the piloting of MAP in the UCLA General Child Outpatient Clinic, and Dr. James T. McCracken for all of his support for the training program. Correspondence should be addressed to Sheryl H. Kataoka, Department of Psychiatry and Biobehavioral Sciences, UCLA Health Services Research Center, 10920 Wilshire Boulevard, Suite 300, Los Angeles, CA 90024-6505. E-mail: [email protected]
For more than four decades, health care quality improvement has been a focus in medicine (Brook, Kamberg, & McGlynn, 1996; Schuster, McGlynn, & Brook, 2005), and improving the quality of child mental health care has been a national priority area (U.S. Department of Health and Human Services, 2003; U.S. Public Health Service, 2000). In their seminal
PRACTICE-BASED LEARNING AND IMPROVEMENT
313
TABLE 1 ACGME PBLI Competency and Relationship With MAP System Components PBLI Component Identifying strengths and deficiencies in knowledge and expertise Identifying and performing appropriate learning activities Incorporating formative evaluative feedback into daily practice Systematically analyzing practice and implement changes to improve practice Appraising and using scientific evidence Using technology to optimize learning Participating in the education of patients and families
Map Component Therapist Portfolio Therapist Portfolio Dashboards The MAP Process Guide Focus-Interference-Framework Dashboard PWEBS database Use of MAP system (online tools, PWEBS clinical trials, Dashboard) MAP Practitioner Guides MAP Process Guides
Downloaded by [Aston University] at 11:31 06 September 2014
Note: ACGME ¼ Accreditation Council for Graduate Medical Education; PBLI ¼ Practice-Based Learning and Improvement; MAP ¼ Managing and Adapting Practice; PWEBS ¼ database of evidence-based child mental health treatments.
report, the Institute of Medicine recommended that effective care, defined as providing services based on scientific knowledge, was one of the six main aims to transform the U.S. health care system (Institute of Medicine, 2001). These national recommendations led to a growing body of literature supporting quality improvement efforts, including quality improvement in primary care settings for adult and adolescent depression (Asarnow et al., 2005; Wells et al., 2000). Nevertheless, despite increasing scientific evidence for mental health interventions, few clinicians in community-based mental health programs implement empirically supported psychotherapies (Weisz, 2000). Researchers have argued that there exist multiple barriers to improving the quality of mental health services commonly delivered, including clinicians’ lack of training in evidencebased treatments (Beidas & Kendell, 2010). In response to the growing emphasis on improving the quality of patient care in the medical field, the Accreditation Council for Graduate Medical Education (ACGME), a nonprofit organization with an established mission to improve the health of the public by insuring quality graduate medical education, has implemented competency-based requirements for all medical residency training programs (ACGME, 2009). The ACGME sets standards and requirements for training in various medical specialties, and compliance with the ACGME standards is required for program accreditation. In 1999, the ACGME introduced the Outcome Project, which requires that graduate medical education programs foster resident physicians’ development of competencies in six domains of clinical care and collect performance data that documents residents’ ability to care for patients and work effectively in healthcare delivery systems (Swing, 2007). These domains include patient care, medical knowledge, practice-based learning and improvement (PBLI), interpersonal and
communication skills, professionalism, and systemsbased practice. In 2009 the ACGME began basing training program accreditation on these six competency areas. In the area of PBLI, the ACGME (2009) requires that residents become proficient at quality improvement and lifelong learning by demonstrating ‘‘the ability to investigate and evaluate their care of patients, to appraise and assimilate scientific evidence, and to continuously improve patient care based on constant self-evaluation and lifelong learning.’’ There are eight goals of the PBLI competency: 1. Setting learning and improvement goals. 2. Identifying and performing appropriate learning activities. 3. Incorporating formative evaluation feedback into daily practice. 4. Systematically analyzing practice using quality improvement methods and implementing changes with the goal of practice improvement. 5. Locating, appraising, and assimilating evidence from scientific studies related to their patients’ health problems. 6. Using information technology to optimize learning. 7. Participating in the education of patients, families, students, residents, and other health professionals. 8. Identifying strengths, deficiencies, and limits in one’s knowledge and expertise. Psychiatry residency programs have implemented PBLI curricula in a number of ways. Many have focused on short-term quality improvement projects that primarily address hospital system changes and safety, whereas others have taught these requirements in the form of didactics and workshops (Reardon, Ogrinc, & Walaszek, 2011; Sockalingam,
Downloaded by [Aston University] at 11:31 06 September 2014
314
KATAOKA ET AL.
Stergiopoulous, Maggi, & Zaretsky, 2010), and yearround quality improvement didactics with ongoing projects (Arbuckle et al., 2013). Treatment guidelines have also been introduced to residents, with case vignettes used to reinforce learning (Falzer & Garman, 2012). Nevertheless, psychiatry residency training programs face the challenge of determining specific, observable, and measureable ways to evaluate meeting the ACGME required competencies, with few programs integrating PBLI into outpatient practice and the use of evidence-based psychotherapies, despite psychotherapy also being a requirement of psychiatric training (Swick, Hall, & Beresin, 2006). As the evidence for psychotherapy interventions continues to grow, an expanding array of skills need to be part of the armamentarium taught to cliniciansin-training. However, training programs have noted challenges to teaching evidence-based practices (EBPs), such as lack of qualified faculty, difficulty in teaching EBPs, and lack of trainee interest (Weissman et al., 2006). In addition, Weerasekera and colleagues (Weerasekera, Manring, & Lynn, 2010) found that evidence-based methods for assessing residents in psychotherapy competence need to be utilized by training programs. In response to general calls to continuously improve patient care and clinician training in mental health, our Child and Adolescent Psychiatry Training Program has begun piloting Managing and Adapting Practice (MAP) in a general outpatient child psychiatry teaching clinic (Chorpita & Daleiden, 2013). MAP is a system of tools designed to improve the quality, efficiency, and outcomes of children’s mental health services by giving clinicians easy access to the most current scientific information about psychotherapies for children and by providing user-friendly measurement tools and clinical protocols. Using an online database, the system provides detailed recommendations about discrete components of EBPs relevant to a specific child’s characteristics. These recommendations are organized into an individualized treatment plan, which is then flexibly implemented using structured, procedural guides in the context of a measurement feedback system, which allows the practitioner to adjust the plan over time (Chorpita & Daleiden, 2009). Our general outpatient clinic was well suited to serve as the pilot site given that MAP fit well with the clinic’s emphasis on short-term interventions, and its flexibility had the potential to address differences in provider education and the wide variation in patient diagnosis and acuity. In addition, MAP provides structure and resources for delivering child mental health services in a quality improvement framework that can assist trainees in learning each of the PBLI competencies and gaining skills in evidencebased psychotherapy components of care. This article
thus describes each PBLI component and the corresponding MAP tools that address this requirement (see Table 1), as well as a case example to illustrate ‘‘MAP in action.’’ Guided by our clinical observations and early lessons learned, recommendations for adaptation of MAP for use in a child psychiatry training clinic are also presented.
METHODS Clinic Setting The general outpatient child psychiatry training clinic provides diverse experiences for advanced trainees to the treatment and management of child populations (ages 3–17) with a wide variety of presenting problems and diagnostic complexities including anxiety, mood disorders, psychosis, attention deficit=hyperactivity disorder (ADHD) and other disruptive behaviors, medical comorbidity with psychiatric symptoms, and developmental disabilities. The level of clinical acuity also varies, from requests for first-time psychiatric assessments to follow-up care after recent psychiatric hospitalizations. As a result of our reorganization to include MAP, the clinic now consists of a didactic=supervision hour followed by four hours of patient care. The clinic is structured for 90-minutes new patient assessments, 30- to 60-minutes follow-up appointments, and 1 dedicated therapy hour for MAP therapy cases. Training All psychiatry and psychology faculty that supervise in the outpatient clinic participated in the 1-day intensive MAP Supervisor Training, which provided tools to conduct supervision within the MAP systems model framework. All trainees that participate in the outpatient clinic (eight 2nd-year child psychiatry fellows and two psychology interns) attended the 5-day MAP training. A MAP developer (Bruce Chorpita, Ph.D.) and one of the present authors, a MAP trainer, taught all the major components of MAP. Training consisted of learning 33 key psychotherapy skills (e.g., communication skills training, relaxation training, exposure for anxiety= trauma) and utilization of the components of MAP such as the Dashboard, the database of evidence-based child mental health treatments (PWEBS), the FocusInterference-Framework, and other implementation tools (described next). Moreover the training provided an introduction to the MAP evidence-based systems approach to clinical care and treatment planning (for more details on the MAP components, see Chorpita & Daleiden, 2009). For the purposes of this article, we focus on the training of the child psychiatry fellows
PRACTICE-BASED LEARNING AND IMPROVEMENT
given the emphasis of this article on the competency requirements of psychiatrists.
Downloaded by [Aston University] at 11:31 06 September 2014
Supervision Trainees are directly supervised during clinic, with supervision occurring in the context of both group and individual supervision. Supervising faculty includes board-certified child and adolescent psychologist and psychiatrists and two clinical psychologists with specialized child psychology training. Rounds consist of one hour each week at the beginning of clinic dedicated to didactics and group supervision. During group supervision, trainees present MAP cases with a focus on case conceptualization, use of psychotherapeutic skills, and treatment progress and review of videotaped segments of sessions. Trainees also receive live feedback on clinical skills, often with supervisors who observe the parent and child interviews behind a one-way mirror. PBLI Competencies and Corresponding MAP Tools
315
perform the component of therapy, with minimal errors and no longer having to think through each step. Finally, ‘‘habit’’ in performing the skill, recognizes that the trainee has the ability to incorporate the therapy component with ease into sessions and adapt those skills to the situation at hand. The Therapist Portfolio operationalizes areas of competence and areas for growth for each trainee, which can be used to identify the trainee’s strengths and deficiencies. The MAP Therapist Portfolio contains both a Direct Service Learning Record and a Direct Service Case Record. The Direct Service Learning Record is used to chart the clinician’s level of competence in various MAP skills including the evidence-based systems model, clinical problem solving, process essentials, MAP online tools, and specific therapeutic techniques (see Figure 1). Trainees use this record to track their individual professional development. The case record is used to track the clinician’s utilization of various MAP skills with specific clients. Trainees use this record to document the amount of time they have seen the client, the target problem area, how they are measuring treatment progress, and
The following describe how components of the MAP system address the eight goals of the PBLI competency for residency training. PBLI Goal 1. Setting learning and improvement goals and PBLI Goal 2. Identifying and performing appropriate learning activities. As part of the PBLI framework, trainees identify where deficits in training may exist and develop a learning plan to improve their level of expertise in that area. Part of the MAP system includes the utilization of an achievement based portfolio system (referred to as the Therapist Portfolio), which tracks and evaluates the knowledge and level of expertise of clinicians on various psychotherapeutic practices and processes of care. The design, structure, and process of completing the Therapist Portfolio readily allow trainees and supervisors to identify relative strengths and deficiencies in their psychotherapy training. These portfolios have six levels of skill attainment that the trainee self-rates from exposure to the technique from a training or lecture experience to becoming facile with using the technique with clients. Ratings begin with recognizing which skills the trainee has learned through ‘‘review,’’ which they have then ‘‘rehearsed’’ or practiced. When skills are then applied with clients, a trainee determines to what extent they have expertise in that skill, progressing from knowledge to habit. ‘‘Knowledge’’ refers to understanding when a skill is used and the ability to recall key aspects of the skill. ‘‘Production,’’ or being able to perform a practice, indicates that a trainee still needs to think through the steps during delivery of the skill. ‘‘Skill’’ refers to being able to
FIGURE 1 MAP Therapist Portfolio: Direct Service Learning Record. Note: #PracticeWise, LLC. Reprinted with permission. (Figure appears in color online.)
316
KATAOKA ET AL.
what specific practice elements were delivered in treatment. The Therapist Portfolio is a useful tool that helps trainees identify with which of the 33 psychotherapy skills they have experience and have achieved expertise, and in which skills they should gain greater training.
Downloaded by [Aston University] at 11:31 06 September 2014
PBLI Goal 3. Incorporating formative evaluative feedback into daily practice. One key component in learning how to improve practice involves incorporating formative evaluation into ongoing patient care in order to assess the process of care and identifying when changes in the treatment strategy may be needed. In developing skills in this area of the PBLI competency, trainees learn to use a MAP Dashboard as well as the Treatment Pathway guides. The Dashboard allows the clinician to visually track at each treatment session several metrics or measures each week that represent
key symptoms (e.g., amount of sleep per night, number of tantrums per week) or level of functioning (e.g., number of days missed from school per week, number of weekly play-dates). The clinician along with the child and parents determine together what these metrics of treatment progress should be so that outcomes are patient centered (see Figure 2). Given that in our psychiatry clinic most children are receiving both psychotherapy and medication treatments, we have also incorporated measurements of medication compliance (number of missed doses per week) and safety (level of side effects such as weight gain), in addition to symptom and functioning metrics. The MAP Dashboard allows the trainee to examine the patient’s progress over time as well as track which treatment components have been delivered and to decide whether quality improvement in the treatment process is
FIGURE 2 MAP Dashboard for Formative Evaluative Feedback. Note: # PracticeWise, LLC. Reprinted with permission. (Figure appears in color online.)
Downloaded by [Aston University] at 11:31 06 September 2014
PRACTICE-BASED LEARNING AND IMPROVEMENT
warranted. During group and direct individual supervision, trainees obtain feedback from their peers and supervisors as they present cases and review their Dashboards and learn how to problem-solve when treatment is not progressing. The trainees can also use the Dashboard to track where they are in the course of treatment and if they are following the treatment plan that had been formulated at the beginning of care. For example, based on a PWEBS search, a treatment pathway for anxiety usually includes psychoeducation, monitoring, exposure, and maintenance sessions. The trainee can use the Dashboard to track how the client is progressing in treatment and see which practices have been delivered. If the observed outcomes are not congruent with the expected outcomes, the trainee can modify and adapt treatment. For example, if the trainee has completed sessions on psychoeducation and monitoring, yet sees that the child’s anxiety ratings are not improving, the Dashboard would show that the expected component of exposure has not been completed. The data from the Dashboard provides feedback that (a) the child is not making progress as expected and (b) certain practice elements (e.g., exposure) have not been delivered. The Dashboard as well as the Treatment Pathway guides can provide informative feedback on both observed and expected values of treatment progression.
317
PBLI Goal 4. Systematically analyzing practice using quality improvement methods and implementing changes with the goal of practice improvement. As trainees learn skills in QI, MAP can serve as a good example of implementing improvements in everyday clinical practice. A classic quality improvement framework that has been used across medical specialties is the Plan-Do-Study-Act cycle (Cleghorn & Headrick, 1996). During the ‘‘Plan’’ phase, the trainee will identify a target for improvement. The MAP Process Guide (see Figure 3) is an overall ‘‘map’’ that helps the trainee problem solve when treatment is not progressing. For example, if the family is not attending sessions and not engaged in treatment, the MAP Process Guide suggests focusing on engagement strategies, which are outlined in the Process Guide. Next the ‘‘Do’’ phase includes piloting the proposed change in treatment. The MAP Focus-Interference-Framework (F-I-F) assists the trainee in outlining the focus of treatment and what factors may be interfering with progress, such as the ‘‘crisis of the day.’’ In the F-I-F, trainees delineate and operationalize three specific steps of treatment, referred to by the MAP model as connect, cultivate, and consolidate. The connect step includes assessment, engagement, education, and orienting the family to treatment. The cultivate step includes the development and rehearsal of new skills, and the consolidate step includes reviewing material,
FIGURE 3 MAP Process Guide for implementing quality improvement. Note: # PracticeWise, LLC. Reprinted with permission. (Figure appears in color online.)
318
KATAOKA ET AL.
Downloaded by [Aston University] at 11:31 06 September 2014
trying skills in new situations, building independence, and preparing for termination. In the ‘‘Study’’ phase, the metrics from the MAP Dashboard can be assessed to determine if the ‘‘Plan’’ is effectively addressing the focus of treatment. Finally, trainees can ‘‘Act’’ or continue along with the course of treatment if metrics are moving in a positive direction and the interference factors don’t become more significant than the areas of focus. PBLI Goal 5. Locating, appraising, and assimilating evidence from scientific studies related to their patients’ health problems. The MAP searchable database, known as PWEBS, currently summarizes 668 randomized clinical trials and 1,689 treatments for children’s mental health in the areas of childhood anxiety, depression, disruptive behavior, eating disorders, substance use, traumatic stress, ADHD, suicidality, mania, and autism (see Chorpita et al., 2011), which is continuously being updated as new studies are published. By having these studies rated by their level of evidence and coded for clinically relevant factors in treatment (e.g., setting of implementation, sociodemographics of the child), trainees have a quick and easily accessible way of appraising the current child mental health literature (see Figure 4). By using this database, trainees can review a bibliography and summaries of the original studies and view the most salient set of treatment components as determined by the strength of all current published studies. Thus trainees learn to directly apply the evidence from the scientific literature to their treatment decisions and practice. To record the trainee’s
progress in this area, the Therapist Portfolio, Direct Service Case Record, includes the level of Evidence-Based Service Knowledge Integration that the clinician implements during practice. PBLI Goal 6. Using information technology to optimize learning. Another aspect of PBLI that is highly relevant to changes in the health care system nationally is assuring that trainees are competent in using technology in clinical care and learning. Given that all of the MAP tools and PWEBS database are on the Practice Wise website, trainees have access to technology to optimize their learning. They can access assessment tools and scoring algorithms, templates for their Dashboards, and all of the Practice Guides and treatment protocols. In addition, the website provides slides and handouts from the original training for reference. In our group supervision, we demonstrate how to access various components of MAP, and trainees bring electronic copies of their Dashboards, FocusInterference-Frameworks and Clinical Event Structures to review. The Clinical Event Structure, a MAP tool to assist in session planning, provides a checklist of items for a clinician to do before and after session to increase the utility and effectiveness of the session (e.g., before the session confirm the appointment with the family, review treatment progress; after the session note practices used in session, record homework assigned). The Clinical Event Structure also has a section for the agenda items to be done during the session, including what will be done during the opening phase, working phase, and closing phase. Further, with the
FIGURE 4 Example of MAP Database, PWEBS, which summarizes the evidence from the scientific literature on psychotherapy interventions for children and adolescents. Note: # PracticeWise, LLC. Reprinted with permission. (Figure appears in color online.)
PRACTICE-BASED LEARNING AND IMPROVEMENT
Downloaded by [Aston University] at 11:31 06 September 2014
implementation of an electronic health record system at our university health care system under way, the option to create customized templates for individual clinics will provide the opportunity to integrate MAP tools into the electronic medical record. PBLI Goal 7. Participating in the education of patients, families, students, residents, and other health professionals. One critical component to PBLI is learning best practices in educating family members and other professionals about treatment strategies. MAP Practitioner Guides can be readily downloaded from the PracticeWise website and shared with families. These guides include brief summaries of common psychotherapy techniques such as exposure for anxiety, praise and reward for disruptive behaviors, activity planning and cognitive restructuring for depression, which facilitate psychoeducation with families. For example, handouts on each parent-training component can be shared with parents to reinforce practice in between sessions. Rating forms with thermometers are available that easily allow youth to rate their mood throughout the week. PBLI Goal 8. Identifying strengths, deficiencies, and limits in one’s knowledge and expertise. An important aspect of PBLI is self-reflection to identify areas of one’s own relative strengths and weaknesses and to track progression along a professional developmental pathway. Areas for practice improvement and additional supervision can then be recognized and addressed by both the supervisor and trainee, and a training plan for improvement can result. Within the MAP framework, supervisors and trainees can assess the trainee’s level of competence on each MAP case by utilizing the MAP Competence Model: Framework for Professional Development (http://www.practicewise.com). The MAP Competence Model Framework is a structured way to assess strengths and deficiencies in multiple domains and can be applied as it pertains to a case or caseload. In the future, our clinic will be utilizing this framework as a starting point for competency in the context of a specific MAP case, with each fellow and supervisor identifying from this list of characteristics areas of strength and areas for growth. Case Example To bring PBLI and MAP tools to life, we provide a case illustration of the use of MAP with a family seen by a child psychiatry trainee in our general outpatient clinic. This case, modified to protect the identity of the child and family, is typical of the complexity of cases seen in our teaching clinic. The identified patient is an
319
8-year-old girl, whom we will call ‘‘Julia,’’ who was previously diagnosed with Autism Spectrum Disorder, Intermittent Explosive Disorder, Tourette’s Syndrome, and Depressive Disorder Not Otherwise Specified. Julia presented to clinic after being seen in the emergency room for aggressive behavior toward her mother and suicidal ideation, with an attempt to hurt herself with a knife. Julia has received mental health services since the age of 4, and she has been on a number of psychotropic medications throughout her life with varying degrees of success. When she presented to clinic, Julia was taking daily doses of Guanfacine and Citalopram, and also Chlorpromazine as needed for exacerbations of aggressive behavior, and was experiencing significant disruptive behavior at home and aggressiveness toward her mother and father, depressed mood, impairment in social relatedness at school, and ongoing motor tics. Julia and her parents participated in a thorough intake evaluation over several sessions. As part of the evaluation, the child psychiatry fellow gathered collateral information from prior hospital records and school personnel. She also collected standardized symptom rating scales from parents and from Julia herself. Julia’s diagnostic profile and presenting problems were discussed and reviewed in rounds with supervisors and other trainees. Julia’s treatment initially focused on medication management for the first three sessions; however, these visits involved significant crisis management related to her oppositional and aggressive behavior. In addition, the parents had difficulty refraining from arguing with each other during sessions, which made treatment planning unproductive. This case was selected to begin treatment from the MAP systems model approach. Using PWEBS, the trainee searched the literature for EBPs relevant to this patient’s demographic characteristics and presenting problems. Although this patient had a number of clinical diagnoses, the trainee identified that Julia’s most impairing problem currently was disruptive and aggressive behavior. The PWEBS search helped the trainee to generate ideas for treatment and to learn from the research which practice elements were most effective for children with similar presenting problems (disruptive behavior). Next the trainee generated a F-I-F, where she specified the focus of treatment (disruptive behavior), delineated steps of treatment (connect, cultivate, and consolidate), and identified areas of potential interference including parental discord, motor tics, and depression. The trainee also outlined a Treatment Pathway and reviewed it with the family during the first MAP therapy session. The Treatment Pathway is a tool used to organize the treatment plan. This tool is a flow sheet similar to the F-I-F and served as a useful anchor for the family. Every time the family introduced a new ‘‘crisis of
Downloaded by [Aston University] at 11:31 06 September 2014
320
KATAOKA ET AL.
the week,’’ the trainee helped focus the session by referring back to the Treatment Pathway. The trainee also created a Dashboard with ideographic and objective measures specific to this family. The Dashboard for this patient included the number of weekly Chlorpromazine tablets needed, ratings from a behavior rating scale, hours of sleep, and number of weekly incidents of aggression. The focus of treatment was disruptive behavior and as such the treatment was targeted to improve parenting skills. Practice elements including praise, positive attending, rewards, time-out, and effective consequences were taught, modeled, and role-played with the parents. One Plan-Do-Study-Act cycle that the trainee implemented involved the problem of the parents arguing during sessions, which interfered significantly with further progress. The ‘‘Plan’’ was to split the session into two parts and meet with each parent separately but review the same content with each parent. In the ‘‘Study’’ of this improvement plan, parents went from not absorbing any skills to progressing each week and gaining competency in applying skills at home. With the success of this modification, the trainee continued the sessions in this manner. After approximately four sessions, the trainee was able to bring the parents together in the same room for part of the session. To date the family has made significant progress as tracked on the Dashboard. The child’s number of aggressive outbursts toward the mother has decreased from 30 per week to approximately five per week, the number of Chlorpromazine doses needed has declined from six per week to zero in the past 2 weeks, and the parents note significant improvement on behavior rating scales. By using formative evaluative feedback throughout this case, the trainee could readily assess each week the progress of the patient. When progress was not occurring, the Plan-Do-Study-Act cycle resulted in improvement of Dashboard measures. Through the use of the MAP systems model and specific MAP tools (PWEBS, Dashboard, Practitioner Guides), the trainee effectively structured sessions and implemented EBPs for this family. The use of MAP with this case has engendered personal and professional growth in the trainee, consistent with the ACGME area of PBLI. The Therapist Portfolio for this trainee shows progress in the areas of clinical problem solving, the use of MAP tools, and the application of various practice domains. The trainee has demonstrated a knowledge and production of the F-IF as well as knowledge and production in the use of PWEBS, the Dashboard, and the Practitioner Guides. This trainee has progressed from the didactic and rehearsal level to the knowledge and production level for practice elements including praise, monitoring, problem solving, time-out, communication skills, and
response cost. With regard to the overall MAP Professional Competence Model, the trainee has demonstrated increased self-awareness and insight into her own skills, an increase in openness to learning new material and different therapeutic techniques, improvement in goal setting, and progress in the utilization of systems-based practice.
DISCUSSION Overall, the MAP system provides useful tools and structure that address each of the core components of PBLI for child psychiatry training. MAP provides not only a foundation for teaching trainees how to practice evidence-based care but also a framework for reflective learning, an important component for self-assessment and professional development (Carraccio & Englander, 2004). As an adjunct to traditional quality improvement projects, MAP also provides a novel approach for teaching PBLI in the context of learning and practicing a wide variety of clinically indicated evidence-based psychotherapy techniques in the outpatient setting. Within this enhanced framework that aligns MAP with the ACGME core competencies, we further apply some of our early lessons learned to guide recommendations for future use of MAP to improve psychotherapy training and teach PBLI for child psychiatry fellows. To our knowledge, this is the first description of PBLI for child psychiatry fellows that teach this competency while learning a psychotherapy approach. In fact, MAP was adopted by the faculty of this clinic to improve the quality of psychotherapy taught in this clinic. Prior attempts to implement traditional evidencebased treatment manuals in this general outpatient clinic had been met with challenges. These challenges included difficulties in fitting complex and comorbid children and their families into any one treatment protocol and having supervisors well versed in multiple treatment protocols, and then teaching those protocols to fellows as cases presented to clinic. MAP had the potential to teach psychotherapy in a quality improvement paradigm and teach a broad base of psychotherapy skills through an integrated and easily navigated system of core skills. Given this early stage of delivering MAP in a teaching clinic, we have yet to conduct a formal evaluation. However we did gather informal program evaluation information from the trainees about (a) their frequency of use of the MAP tools (e.g., PWEBS, F-I-F, Clinical Event Structures, Practitioner Guides), (b) familiarity and comfort using the various Practitioner Guides, and (c) challenges in implementing MAP. Fellows used PWEBS and the Practitioner Guides most frequently of the MAP tools, and they reported feeling most
Downloaded by [Aston University] at 11:31 06 September 2014
PRACTICE-BASED LEARNING AND IMPROVEMENT
comfortable using the ‘‘Engagement with Caregiver’’ and ‘‘Engagement with Child’’ Practitioner Guides. Challenges reported by the fellows in implementing MAP during clinic included the number of cases that were covered during supervision time as well as the additional time spent on preparing Dashboards. Overall, these results as well as anecdotal evidence from the course of the academic year to date suggest that child psychiatry fellows demonstrate willingness to learn the therapeutic techniques in MAP, find the MAP tools useful, and appreciate the flexibility inherent in the MAP system. Areas for continued growth, with regard to trainee attitudes toward the implementation of MAP in our clinic are centered on the clinic format itself (e.g., need for additional supervision time, need for designated time to prepare for MAP sessions and complete tools such as Dashboards). Fellows also incorporated treatment protocols from our divisional faculty for select cases when specific Practitioner Guides were not available in the MAP library, yet still using other MAP tools such as the Dashboard. As highlighted by Borntrager, Chorpita, HigaMcMillan, and Weisz (2009), provider attitudes toward EBPs influence the dissemination of these practices in real-world settings. As part of a randomized clinical effectiveness trial, Borntrager et al. assessed the attitudes of therapists before and after training for a standard evidence-based treatment protocol and for a modular evidence-based treatment protocol. Results of this study suggested that compared with the standard condition, in the modular condition therapists’ attitudes were significantly more favorable toward EBPs. Therapists in the modular condition perceived more flexibility in their approach to treatment, and this is also something anecdotally we have seen in our clinic. It is important to note, however, that trainees in our clinic have varying degrees of experience with traditional evidence-based treatment protocols. Further research is needed to gather information regarding acceptability of standard compared to modular evidence-based protocols in training settings. In further recognizing that we are utilizing the MAP system beyond the boundaries of its evidence base, we would also like to propose areas for future research. Qualitative data from child psychiatry fellows and faculty is needed to further inform how MAP should be used with child psychiatry patients and their families as well as within the context of a teaching clinic within a large academic medical center. Following adaptations guided from this feedback, pilot testing the feasibility and acceptability of the revised implementation of MAP would be merited. Within this pilot, development of methods to measure educational outcomes such as retaining information about PBLI and how this might relate to changes in practice short- and
321
longer term would be important to assess. With these developmental stages achieved, future research examining the impact of MAP on improved quality of care mediated through improved clinician training may be proposed. In addition, given its flexible model, MAP has the potential to be adapted for child psychiatric practice, which often includes patients who require combined psychotherapy and psychopharmacological treatment. During the early piloting of MAP, we observed that not infrequently trainees include medication related metrics on their Dashboard, as in our case example. Measures of treatment compliance, amount of ‘‘as needed’’ medications used, and side effects can be included in weekly monitoring to inform medication management decisions. In addition, guidelines for common psychotropic medication treatment decisions are readily available (American Academy of Pediatrics, 2000; Greenhill, Pliszka, Dulcan, & the Work Group on Quality Issues, 2002), and a few national quality measures for child mental health include recommended frequencies for stimulant medication monitoring (Zima et al., 2013). Thus, adaptation of MAP for psychotropic medication management may be a natural extension of the framework to develop a potential tool for transparently integrating recommended care practices for psychotropic medication treatment with evidence-based psychotherapies. Yet in considering these prospective areas for new research, we also observed potential barriers that merit addressing in applying MAP to a child psychiatry teaching clinic. Similar to other evidence-based psychotherapies, time and resources are needed to train, implement, and sustain their use. Protected time for fellows and faculty to receive training and ongoing support in implementing MAP, as well as training and resource costs, need to be considered. Preparatory work is also often required outside of the clinic visit; therefore, time should be protected for child psychiatry fellows to prepare for patient visits and supervision. Engagement and continuous support to enhance the organizational climate to support successful implementation over time may be needed, given prior research showing that organizational climate can influence implementation of EBPs (Aarons & Sawitzky, 2006; Glisson & Hemmelgarn, 1998). Despite these potential barriers, MAP provides a practical way to teach PBLI competencies while delivering psychotherapy in the flexible way that is frequently needed in the child psychiatric outpatient setting. By teaching PBLI in the context of psychotherapy practice, during a yearlong outpatient training experience, lifelong learning and quality improvement will hopefully become embedded in the routine practice of graduates.
322
KATAOKA ET AL.
Downloaded by [Aston University] at 11:31 06 September 2014
REFERENCES Aarons, G. A., & Sawitzky, A. C. (2006). Organizational culture and climate and mental health provider attitudes toward evidence-based practice. Psychological Services, 3, 61. Accreditation Council for Graduate Medical Education. (2009). Practice-based Learning and Improvement: Common Program Requirement. Available from http://www.acgme.org American Academy of Pediatrics, Committee on Quality Improvement, Subcommittee on Attention-Deficit=Hyperactivity Disorder. (2000). Clinical practice guideline: Diagnosis and evaluation of the child with attention-deficit=hyperactivity disorder. Pediatrics, 105, 1158–1170. Arbuckle, M. R., Weinberg, M., Cabaniss, D. L., Kistler, S. C., Isaacs, A. J., Sederer, L. I., et al. (2013). Training psychiatry residents in quality improvement: An integrated, year-long curriculum. Academic Psychiatry, 37, 42–45. Asarnow, J. R., Jaycox, L. H., Duan, N., LaBorde, A. P., Rea, M. M., Murray, P., et al. (2005). Effectiveness of a quality improvement intervention for adolescent depression in primary care clinics: A randomized controlled trial. The Journal of the American Medical Association, 293, 311–319. Beidas, R. S., & Kendall, P. C. (2010). Training therapists in evidencebased practice: A critical review of studies from a systems-contextual perspective. Clinical Psychology: Science and Practice, 17, 1–30. Borntrager, C., Chorpita, B., Higa-McMillan, C., & Weisz, J. (2009). Provider attitudes toward evidence-based practices: Are the concerns with the evidence or with the manuals? Psychiatric Services, 60, 677–681. Brook, R. H., Kamberg, C. J., & McGlynn, E. A. (1996). Health system reform and quality. Journal of the American Medical Association, 276, 476–480. Carraccio, C., & Englander, R. (2004). Evaluating competence using a portfolio: A literature review and web-based application to the ACGME competencies. Teaching and Learning in Medicine, 16, 381–387. Chorpita, B. F., & Daleiden, E. L. (2009). Mapping evidence-based treatments for children and adolescents: Application of the distillation and matching model to 615 treatments from 322 randomized trials. Journal of Consulting and Clinical Psychology, 77, 566–579. Chorpita, B. F., & Daleiden, E. L. (2013). Structuring the collaboration of science and service in the pursuit of a shared vision. Journal of Child and Adolescent Clinical Psychology. Advance online publication. doi: 10.1080/15374416.2013.828297 Chorpita, B. F., Daleiden, E. L., Ebesutani, C., Young, J., Becker, K. D., Nakamura, B. J., et al. (2011). Evidence-based treatments for children and adolescents: An updated review of indicators of efficacy and effectiveness. Clinical Psychology: Science and Practice, 18, 153–171. Cleghorn, G. D., & Headrick, L. A. (1996). The PDSA cycle at the core of learning in health professions education. Joint Commission Journal of Quality Improvement, 22, 206–212. Falzer, P. R., & Garman, D. M. (2012). Evidence-based decisionmaking as a practice-based learning skill: A pilot study. Academic Psychiatry, 36, 104–109.
Glisson, C., & Hemmelgarn, A. (1998). The effects of organizational climate and interorganizational coordination on the quality and outcomes of children’s service systems. Child Abuse & Neglect, 22, 401–421. Greenhill, L. L., Pliszka, S., Dulcan, M. K., & Work Group on Quality Issues. (2002). Practice parameters for the use of stimulant medications in the treatment of children, adolescents, and adults. Journal of the American Academy of Child and Adolescent Psychiatry, 41(2 Suppl.), 26S–49S. Institute of Medicine, Committee on Quality of Health Care in America. (2001). Crossing the quality chasm: A new health system for the 21st century. Washington, DC: National Academy Press. Reardon, C. L., Ogrinc, G., & Walaszek, A. (2011). A didactic and experiential quality improvement curriculum for psychiatry residents. Journal of Graduate Medical Education, 562–565. Schuster, M. A., McGlynn, E. A., & Brook, R. H. (2005). How good is the quality of health care in the United States? Milbank Quarterly, 83, 843–895. Sockalingam, S., Stergiopoulous, V., Maggi, J., & Zaretsky, A. (2010). Quality education: A pilot quality improvement curriculum for psychiatry residents. Medical Teacher, 32, e221–e226. Swick, S., Hall, S., & Beresin, E. (2006). Assessing the ACGME competencies in psychiatry training programs. Academic Psychiatry, 30, 330–351. Swing, S. R. (2007). The ACGME outcome project: Retrospective and prospective. Medical Teacher, 29, 648–654. U.S. Department of Health, & Human Services. (2003). New Freedom Commission on Mental Health, achieving the promise: Transforming mental health care in America (Final report). Rockville, MD: Author. U.S. Public Health Service. (2000). Report of the Surgeon General’s Conference on Children’s Mental Health: A National Action Agenda. Washington, DC: Department of Health and Human Services. Weerasekera, P., Manring, J., & Lynn, D. (2010). Psychotherapy training for residents: Reconciling requirements with evidence-based, competency-focused practice. Academic Psychiatry, 34, 5–12. Weissman, M. M., Verdeli, H., Gameroff, M. J., Bledsoe, S. E., Betts, K., Mufson, L., et al. (2006). National survey of psychotherapy training in psychiatry, psychology, and social work. Archives of General Psychiatry, 63, 925–934. Weisz, J. R. (2000). Agenda for child and adolescent psychotherapy research: On the need to put science into practice. Archives of General Psychiatry, 57, 837. Wells, K. B., Sherbourne, C., Schoenbaum, M., Duan, N., Meredith, L., Unutzer, J. R., et al. (2000). Impact of disseminating quality improvement programs for depression in managed primary care. The Journal of the American Medical Association, 283, 212–220. Zima, B. T., Murphy, J. M., Scholle, S. H., Hoagwood, K. E., Sachdeva, R. C., Mangione-Smith, R., et al. (2013). National quality measures for child mental health care: Background, progress, and next steps. Pediatrics, 131, S38.
Journal of Clinical Child & Adolescent Psychology Publication details, including instructions for authors and subscription information: http://www.tandfonline.com/loi/hcap20
MAP as a Model for Practice-Based Learning and Improvement in Child Psychiatry Training Sheryl H. Kataoka
a b
& Martha Bates Jura a
a
, Jennifer L. Podell , Bonnie T. Zima
a b
a
, Karin Best , Shawn Sidhu
a
a
UCLA Semel Institute of Neuroscience and Human Behavior
b
Center for Health Services and Society Published online: 18 Nov 2013.
To cite this article: Sheryl H. Kataoka , Jennifer L. Podell , Bonnie T. Zima , Karin Best , Shawn Sidhu & Martha Bates Jura (2014) MAP as a Model for Practice-Based Learning and Improvement in Child Psychiatry Training, Journal of Clinical Child & Adolescent Psychology, 43:2, 312-322, DOI: 10.1080/15374416.2013.848773 To link to this article: http://dx.doi.org/10.1080/15374416.2013.848773
PLEASE SCROLL DOWN FOR ARTICLE Taylor & Francis makes every effort to ensure the accuracy of all the information (the “Content”) contained in the publications on our platform. However, Taylor & Francis, our agents, and our licensors make no representations or warranties whatsoever as to the accuracy, completeness, or suitability for any purpose of the Content. Any opinions and views expressed in this publication are the opinions and views of the authors, and are not the views of or endorsed by Taylor & Francis. The accuracy of the Content should not be relied upon and should be independently verified with primary sources of information. Taylor and Francis shall not be liable for any losses, actions, claims, proceedings, demands, costs, expenses, damages, and other liabilities whatsoever or howsoever caused arising directly or indirectly in connection with, in relation to or arising out of the use of the Content. This article may be used for research, teaching, and private study purposes. Any substantial or systematic reproduction, redistribution, reselling, loan, sub-licensing, systematic supply, or distribution in any form to anyone is expressly forbidden. Terms & Conditions of access and use can be found at http:// www.tandfonline.com/page/terms-and-conditions
Journal of Clinical Child & Adolescent Psychology, 43(2), 312–322, 2014 Copyright # Taylor & Francis Group, LLC ISSN: 1537-4416 print=1537-4424 online DOI: 10.1080/15374416.2013.848773
MAP as a Model for Practice-Based Learning and Improvement in Child Psychiatry Training Sheryl H. Kataoka UCLA Semel Institute of Neuroscience and Human Behavior, Center for Health Services and Society
Downloaded by [Aston University] at 11:31 06 September 2014
Jennifer L. Podell UCLA Semel Institute of Neuroscience and Human Behavior
Bonnie T. Zima UCLA Semel Institute of Neuroscience and Human Behavior, Center for Health Services and Society
Karin Best, Shawn Sidhu, and Martha Bates Jura UCLA Semel Institute of Neuroscience and Human Behavior
Not only is there a growing literature demonstrating the positive outcomes that result from implementing evidence based treatments (EBTs) but also studies that suggest a lack of delivery of these EBTs in ‘‘usual care’’ practices. One way to address this deficit is to improve the quality of psychotherapy teaching for clinicians-in-training. The Accreditation Council for Graduate Medical Education (ACGME) requires all training programs to assess residents in a number of competencies including Practice-Based Learning and Improvements (PBLI). This article describes the piloting of Managing and Adapting Practice (MAP) for child psychiatry fellows, to teach them both EBT and PBLI skills. Eight child psychiatry trainees received 5 full days of MAP training and are delivering MAP in a year-long outpatient teaching clinic. In this setting, MAP is applied to the complex, multiply diagnosed psychiatric patients that present to this clinic. This article describes how MAP tools and resources assist in teaching trainees each of the eight required competency components of PBLI, including identifying deficits in expertise, setting learning goals, performing learning activities, conducting quality improvement methods in practice, incorporating formative feedback, using scientific studies to inform practice, using technology for learning, and participating in patient education. A case example illustrates the use of MAP in teaching PBLI. MAP provides a unique way to teach important quality improvement and practicebased learning skills to trainees while training them in important psychotherapy competence. We thank Dr. Bruce Chorpita for training and consulting with our clinic, all of the Child and Adolescent Psychiatry Fellows and Psychology Interns that participated in the piloting of MAP in the UCLA General Child Outpatient Clinic, and Dr. James T. McCracken for all of his support for the training program. Correspondence should be addressed to Sheryl H. Kataoka, Department of Psychiatry and Biobehavioral Sciences, UCLA Health Services Research Center, 10920 Wilshire Boulevard, Suite 300, Los Angeles, CA 90024-6505. E-mail: [email protected]
For more than four decades, health care quality improvement has been a focus in medicine (Brook, Kamberg, & McGlynn, 1996; Schuster, McGlynn, & Brook, 2005), and improving the quality of child mental health care has been a national priority area (U.S. Department of Health and Human Services, 2003; U.S. Public Health Service, 2000). In their seminal
PRACTICE-BASED LEARNING AND IMPROVEMENT
313
TABLE 1 ACGME PBLI Competency and Relationship With MAP System Components PBLI Component Identifying strengths and deficiencies in knowledge and expertise Identifying and performing appropriate learning activities Incorporating formative evaluative feedback into daily practice Systematically analyzing practice and implement changes to improve practice Appraising and using scientific evidence Using technology to optimize learning Participating in the education of patients and families
Map Component Therapist Portfolio Therapist Portfolio Dashboards The MAP Process Guide Focus-Interference-Framework Dashboard PWEBS database Use of MAP system (online tools, PWEBS clinical trials, Dashboard) MAP Practitioner Guides MAP Process Guides
Downloaded by [Aston University] at 11:31 06 September 2014
Note: ACGME ¼ Accreditation Council for Graduate Medical Education; PBLI ¼ Practice-Based Learning and Improvement; MAP ¼ Managing and Adapting Practice; PWEBS ¼ database of evidence-based child mental health treatments.
report, the Institute of Medicine recommended that effective care, defined as providing services based on scientific knowledge, was one of the six main aims to transform the U.S. health care system (Institute of Medicine, 2001). These national recommendations led to a growing body of literature supporting quality improvement efforts, including quality improvement in primary care settings for adult and adolescent depression (Asarnow et al., 2005; Wells et al., 2000). Nevertheless, despite increasing scientific evidence for mental health interventions, few clinicians in community-based mental health programs implement empirically supported psychotherapies (Weisz, 2000). Researchers have argued that there exist multiple barriers to improving the quality of mental health services commonly delivered, including clinicians’ lack of training in evidencebased treatments (Beidas & Kendell, 2010). In response to the growing emphasis on improving the quality of patient care in the medical field, the Accreditation Council for Graduate Medical Education (ACGME), a nonprofit organization with an established mission to improve the health of the public by insuring quality graduate medical education, has implemented competency-based requirements for all medical residency training programs (ACGME, 2009). The ACGME sets standards and requirements for training in various medical specialties, and compliance with the ACGME standards is required for program accreditation. In 1999, the ACGME introduced the Outcome Project, which requires that graduate medical education programs foster resident physicians’ development of competencies in six domains of clinical care and collect performance data that documents residents’ ability to care for patients and work effectively in healthcare delivery systems (Swing, 2007). These domains include patient care, medical knowledge, practice-based learning and improvement (PBLI), interpersonal and
communication skills, professionalism, and systemsbased practice. In 2009 the ACGME began basing training program accreditation on these six competency areas. In the area of PBLI, the ACGME (2009) requires that residents become proficient at quality improvement and lifelong learning by demonstrating ‘‘the ability to investigate and evaluate their care of patients, to appraise and assimilate scientific evidence, and to continuously improve patient care based on constant self-evaluation and lifelong learning.’’ There are eight goals of the PBLI competency: 1. Setting learning and improvement goals. 2. Identifying and performing appropriate learning activities. 3. Incorporating formative evaluation feedback into daily practice. 4. Systematically analyzing practice using quality improvement methods and implementing changes with the goal of practice improvement. 5. Locating, appraising, and assimilating evidence from scientific studies related to their patients’ health problems. 6. Using information technology to optimize learning. 7. Participating in the education of patients, families, students, residents, and other health professionals. 8. Identifying strengths, deficiencies, and limits in one’s knowledge and expertise. Psychiatry residency programs have implemented PBLI curricula in a number of ways. Many have focused on short-term quality improvement projects that primarily address hospital system changes and safety, whereas others have taught these requirements in the form of didactics and workshops (Reardon, Ogrinc, & Walaszek, 2011; Sockalingam,
Downloaded by [Aston University] at 11:31 06 September 2014
314
KATAOKA ET AL.
Stergiopoulous, Maggi, & Zaretsky, 2010), and yearround quality improvement didactics with ongoing projects (Arbuckle et al., 2013). Treatment guidelines have also been introduced to residents, with case vignettes used to reinforce learning (Falzer & Garman, 2012). Nevertheless, psychiatry residency training programs face the challenge of determining specific, observable, and measureable ways to evaluate meeting the ACGME required competencies, with few programs integrating PBLI into outpatient practice and the use of evidence-based psychotherapies, despite psychotherapy also being a requirement of psychiatric training (Swick, Hall, & Beresin, 2006). As the evidence for psychotherapy interventions continues to grow, an expanding array of skills need to be part of the armamentarium taught to cliniciansin-training. However, training programs have noted challenges to teaching evidence-based practices (EBPs), such as lack of qualified faculty, difficulty in teaching EBPs, and lack of trainee interest (Weissman et al., 2006). In addition, Weerasekera and colleagues (Weerasekera, Manring, & Lynn, 2010) found that evidence-based methods for assessing residents in psychotherapy competence need to be utilized by training programs. In response to general calls to continuously improve patient care and clinician training in mental health, our Child and Adolescent Psychiatry Training Program has begun piloting Managing and Adapting Practice (MAP) in a general outpatient child psychiatry teaching clinic (Chorpita & Daleiden, 2013). MAP is a system of tools designed to improve the quality, efficiency, and outcomes of children’s mental health services by giving clinicians easy access to the most current scientific information about psychotherapies for children and by providing user-friendly measurement tools and clinical protocols. Using an online database, the system provides detailed recommendations about discrete components of EBPs relevant to a specific child’s characteristics. These recommendations are organized into an individualized treatment plan, which is then flexibly implemented using structured, procedural guides in the context of a measurement feedback system, which allows the practitioner to adjust the plan over time (Chorpita & Daleiden, 2009). Our general outpatient clinic was well suited to serve as the pilot site given that MAP fit well with the clinic’s emphasis on short-term interventions, and its flexibility had the potential to address differences in provider education and the wide variation in patient diagnosis and acuity. In addition, MAP provides structure and resources for delivering child mental health services in a quality improvement framework that can assist trainees in learning each of the PBLI competencies and gaining skills in evidencebased psychotherapy components of care. This article
thus describes each PBLI component and the corresponding MAP tools that address this requirement (see Table 1), as well as a case example to illustrate ‘‘MAP in action.’’ Guided by our clinical observations and early lessons learned, recommendations for adaptation of MAP for use in a child psychiatry training clinic are also presented.
METHODS Clinic Setting The general outpatient child psychiatry training clinic provides diverse experiences for advanced trainees to the treatment and management of child populations (ages 3–17) with a wide variety of presenting problems and diagnostic complexities including anxiety, mood disorders, psychosis, attention deficit=hyperactivity disorder (ADHD) and other disruptive behaviors, medical comorbidity with psychiatric symptoms, and developmental disabilities. The level of clinical acuity also varies, from requests for first-time psychiatric assessments to follow-up care after recent psychiatric hospitalizations. As a result of our reorganization to include MAP, the clinic now consists of a didactic=supervision hour followed by four hours of patient care. The clinic is structured for 90-minutes new patient assessments, 30- to 60-minutes follow-up appointments, and 1 dedicated therapy hour for MAP therapy cases. Training All psychiatry and psychology faculty that supervise in the outpatient clinic participated in the 1-day intensive MAP Supervisor Training, which provided tools to conduct supervision within the MAP systems model framework. All trainees that participate in the outpatient clinic (eight 2nd-year child psychiatry fellows and two psychology interns) attended the 5-day MAP training. A MAP developer (Bruce Chorpita, Ph.D.) and one of the present authors, a MAP trainer, taught all the major components of MAP. Training consisted of learning 33 key psychotherapy skills (e.g., communication skills training, relaxation training, exposure for anxiety= trauma) and utilization of the components of MAP such as the Dashboard, the database of evidence-based child mental health treatments (PWEBS), the FocusInterference-Framework, and other implementation tools (described next). Moreover the training provided an introduction to the MAP evidence-based systems approach to clinical care and treatment planning (for more details on the MAP components, see Chorpita & Daleiden, 2009). For the purposes of this article, we focus on the training of the child psychiatry fellows
PRACTICE-BASED LEARNING AND IMPROVEMENT
given the emphasis of this article on the competency requirements of psychiatrists.
Downloaded by [Aston University] at 11:31 06 September 2014
Supervision Trainees are directly supervised during clinic, with supervision occurring in the context of both group and individual supervision. Supervising faculty includes board-certified child and adolescent psychologist and psychiatrists and two clinical psychologists with specialized child psychology training. Rounds consist of one hour each week at the beginning of clinic dedicated to didactics and group supervision. During group supervision, trainees present MAP cases with a focus on case conceptualization, use of psychotherapeutic skills, and treatment progress and review of videotaped segments of sessions. Trainees also receive live feedback on clinical skills, often with supervisors who observe the parent and child interviews behind a one-way mirror. PBLI Competencies and Corresponding MAP Tools
315
perform the component of therapy, with minimal errors and no longer having to think through each step. Finally, ‘‘habit’’ in performing the skill, recognizes that the trainee has the ability to incorporate the therapy component with ease into sessions and adapt those skills to the situation at hand. The Therapist Portfolio operationalizes areas of competence and areas for growth for each trainee, which can be used to identify the trainee’s strengths and deficiencies. The MAP Therapist Portfolio contains both a Direct Service Learning Record and a Direct Service Case Record. The Direct Service Learning Record is used to chart the clinician’s level of competence in various MAP skills including the evidence-based systems model, clinical problem solving, process essentials, MAP online tools, and specific therapeutic techniques (see Figure 1). Trainees use this record to track their individual professional development. The case record is used to track the clinician’s utilization of various MAP skills with specific clients. Trainees use this record to document the amount of time they have seen the client, the target problem area, how they are measuring treatment progress, and
The following describe how components of the MAP system address the eight goals of the PBLI competency for residency training. PBLI Goal 1. Setting learning and improvement goals and PBLI Goal 2. Identifying and performing appropriate learning activities. As part of the PBLI framework, trainees identify where deficits in training may exist and develop a learning plan to improve their level of expertise in that area. Part of the MAP system includes the utilization of an achievement based portfolio system (referred to as the Therapist Portfolio), which tracks and evaluates the knowledge and level of expertise of clinicians on various psychotherapeutic practices and processes of care. The design, structure, and process of completing the Therapist Portfolio readily allow trainees and supervisors to identify relative strengths and deficiencies in their psychotherapy training. These portfolios have six levels of skill attainment that the trainee self-rates from exposure to the technique from a training or lecture experience to becoming facile with using the technique with clients. Ratings begin with recognizing which skills the trainee has learned through ‘‘review,’’ which they have then ‘‘rehearsed’’ or practiced. When skills are then applied with clients, a trainee determines to what extent they have expertise in that skill, progressing from knowledge to habit. ‘‘Knowledge’’ refers to understanding when a skill is used and the ability to recall key aspects of the skill. ‘‘Production,’’ or being able to perform a practice, indicates that a trainee still needs to think through the steps during delivery of the skill. ‘‘Skill’’ refers to being able to
FIGURE 1 MAP Therapist Portfolio: Direct Service Learning Record. Note: #PracticeWise, LLC. Reprinted with permission. (Figure appears in color online.)
316
KATAOKA ET AL.
what specific practice elements were delivered in treatment. The Therapist Portfolio is a useful tool that helps trainees identify with which of the 33 psychotherapy skills they have experience and have achieved expertise, and in which skills they should gain greater training.
Downloaded by [Aston University] at 11:31 06 September 2014
PBLI Goal 3. Incorporating formative evaluative feedback into daily practice. One key component in learning how to improve practice involves incorporating formative evaluation into ongoing patient care in order to assess the process of care and identifying when changes in the treatment strategy may be needed. In developing skills in this area of the PBLI competency, trainees learn to use a MAP Dashboard as well as the Treatment Pathway guides. The Dashboard allows the clinician to visually track at each treatment session several metrics or measures each week that represent
key symptoms (e.g., amount of sleep per night, number of tantrums per week) or level of functioning (e.g., number of days missed from school per week, number of weekly play-dates). The clinician along with the child and parents determine together what these metrics of treatment progress should be so that outcomes are patient centered (see Figure 2). Given that in our psychiatry clinic most children are receiving both psychotherapy and medication treatments, we have also incorporated measurements of medication compliance (number of missed doses per week) and safety (level of side effects such as weight gain), in addition to symptom and functioning metrics. The MAP Dashboard allows the trainee to examine the patient’s progress over time as well as track which treatment components have been delivered and to decide whether quality improvement in the treatment process is
FIGURE 2 MAP Dashboard for Formative Evaluative Feedback. Note: # PracticeWise, LLC. Reprinted with permission. (Figure appears in color online.)
Downloaded by [Aston University] at 11:31 06 September 2014
PRACTICE-BASED LEARNING AND IMPROVEMENT
warranted. During group and direct individual supervision, trainees obtain feedback from their peers and supervisors as they present cases and review their Dashboards and learn how to problem-solve when treatment is not progressing. The trainees can also use the Dashboard to track where they are in the course of treatment and if they are following the treatment plan that had been formulated at the beginning of care. For example, based on a PWEBS search, a treatment pathway for anxiety usually includes psychoeducation, monitoring, exposure, and maintenance sessions. The trainee can use the Dashboard to track how the client is progressing in treatment and see which practices have been delivered. If the observed outcomes are not congruent with the expected outcomes, the trainee can modify and adapt treatment. For example, if the trainee has completed sessions on psychoeducation and monitoring, yet sees that the child’s anxiety ratings are not improving, the Dashboard would show that the expected component of exposure has not been completed. The data from the Dashboard provides feedback that (a) the child is not making progress as expected and (b) certain practice elements (e.g., exposure) have not been delivered. The Dashboard as well as the Treatment Pathway guides can provide informative feedback on both observed and expected values of treatment progression.
317
PBLI Goal 4. Systematically analyzing practice using quality improvement methods and implementing changes with the goal of practice improvement. As trainees learn skills in QI, MAP can serve as a good example of implementing improvements in everyday clinical practice. A classic quality improvement framework that has been used across medical specialties is the Plan-Do-Study-Act cycle (Cleghorn & Headrick, 1996). During the ‘‘Plan’’ phase, the trainee will identify a target for improvement. The MAP Process Guide (see Figure 3) is an overall ‘‘map’’ that helps the trainee problem solve when treatment is not progressing. For example, if the family is not attending sessions and not engaged in treatment, the MAP Process Guide suggests focusing on engagement strategies, which are outlined in the Process Guide. Next the ‘‘Do’’ phase includes piloting the proposed change in treatment. The MAP Focus-Interference-Framework (F-I-F) assists the trainee in outlining the focus of treatment and what factors may be interfering with progress, such as the ‘‘crisis of the day.’’ In the F-I-F, trainees delineate and operationalize three specific steps of treatment, referred to by the MAP model as connect, cultivate, and consolidate. The connect step includes assessment, engagement, education, and orienting the family to treatment. The cultivate step includes the development and rehearsal of new skills, and the consolidate step includes reviewing material,
FIGURE 3 MAP Process Guide for implementing quality improvement. Note: # PracticeWise, LLC. Reprinted with permission. (Figure appears in color online.)
318
KATAOKA ET AL.
Downloaded by [Aston University] at 11:31 06 September 2014
trying skills in new situations, building independence, and preparing for termination. In the ‘‘Study’’ phase, the metrics from the MAP Dashboard can be assessed to determine if the ‘‘Plan’’ is effectively addressing the focus of treatment. Finally, trainees can ‘‘Act’’ or continue along with the course of treatment if metrics are moving in a positive direction and the interference factors don’t become more significant than the areas of focus. PBLI Goal 5. Locating, appraising, and assimilating evidence from scientific studies related to their patients’ health problems. The MAP searchable database, known as PWEBS, currently summarizes 668 randomized clinical trials and 1,689 treatments for children’s mental health in the areas of childhood anxiety, depression, disruptive behavior, eating disorders, substance use, traumatic stress, ADHD, suicidality, mania, and autism (see Chorpita et al., 2011), which is continuously being updated as new studies are published. By having these studies rated by their level of evidence and coded for clinically relevant factors in treatment (e.g., setting of implementation, sociodemographics of the child), trainees have a quick and easily accessible way of appraising the current child mental health literature (see Figure 4). By using this database, trainees can review a bibliography and summaries of the original studies and view the most salient set of treatment components as determined by the strength of all current published studies. Thus trainees learn to directly apply the evidence from the scientific literature to their treatment decisions and practice. To record the trainee’s
progress in this area, the Therapist Portfolio, Direct Service Case Record, includes the level of Evidence-Based Service Knowledge Integration that the clinician implements during practice. PBLI Goal 6. Using information technology to optimize learning. Another aspect of PBLI that is highly relevant to changes in the health care system nationally is assuring that trainees are competent in using technology in clinical care and learning. Given that all of the MAP tools and PWEBS database are on the Practice Wise website, trainees have access to technology to optimize their learning. They can access assessment tools and scoring algorithms, templates for their Dashboards, and all of the Practice Guides and treatment protocols. In addition, the website provides slides and handouts from the original training for reference. In our group supervision, we demonstrate how to access various components of MAP, and trainees bring electronic copies of their Dashboards, FocusInterference-Frameworks and Clinical Event Structures to review. The Clinical Event Structure, a MAP tool to assist in session planning, provides a checklist of items for a clinician to do before and after session to increase the utility and effectiveness of the session (e.g., before the session confirm the appointment with the family, review treatment progress; after the session note practices used in session, record homework assigned). The Clinical Event Structure also has a section for the agenda items to be done during the session, including what will be done during the opening phase, working phase, and closing phase. Further, with the
FIGURE 4 Example of MAP Database, PWEBS, which summarizes the evidence from the scientific literature on psychotherapy interventions for children and adolescents. Note: # PracticeWise, LLC. Reprinted with permission. (Figure appears in color online.)
PRACTICE-BASED LEARNING AND IMPROVEMENT
Downloaded by [Aston University] at 11:31 06 September 2014
implementation of an electronic health record system at our university health care system under way, the option to create customized templates for individual clinics will provide the opportunity to integrate MAP tools into the electronic medical record. PBLI Goal 7. Participating in the education of patients, families, students, residents, and other health professionals. One critical component to PBLI is learning best practices in educating family members and other professionals about treatment strategies. MAP Practitioner Guides can be readily downloaded from the PracticeWise website and shared with families. These guides include brief summaries of common psychotherapy techniques such as exposure for anxiety, praise and reward for disruptive behaviors, activity planning and cognitive restructuring for depression, which facilitate psychoeducation with families. For example, handouts on each parent-training component can be shared with parents to reinforce practice in between sessions. Rating forms with thermometers are available that easily allow youth to rate their mood throughout the week. PBLI Goal 8. Identifying strengths, deficiencies, and limits in one’s knowledge and expertise. An important aspect of PBLI is self-reflection to identify areas of one’s own relative strengths and weaknesses and to track progression along a professional developmental pathway. Areas for practice improvement and additional supervision can then be recognized and addressed by both the supervisor and trainee, and a training plan for improvement can result. Within the MAP framework, supervisors and trainees can assess the trainee’s level of competence on each MAP case by utilizing the MAP Competence Model: Framework for Professional Development (http://www.practicewise.com). The MAP Competence Model Framework is a structured way to assess strengths and deficiencies in multiple domains and can be applied as it pertains to a case or caseload. In the future, our clinic will be utilizing this framework as a starting point for competency in the context of a specific MAP case, with each fellow and supervisor identifying from this list of characteristics areas of strength and areas for growth. Case Example To bring PBLI and MAP tools to life, we provide a case illustration of the use of MAP with a family seen by a child psychiatry trainee in our general outpatient clinic. This case, modified to protect the identity of the child and family, is typical of the complexity of cases seen in our teaching clinic. The identified patient is an
319
8-year-old girl, whom we will call ‘‘Julia,’’ who was previously diagnosed with Autism Spectrum Disorder, Intermittent Explosive Disorder, Tourette’s Syndrome, and Depressive Disorder Not Otherwise Specified. Julia presented to clinic after being seen in the emergency room for aggressive behavior toward her mother and suicidal ideation, with an attempt to hurt herself with a knife. Julia has received mental health services since the age of 4, and she has been on a number of psychotropic medications throughout her life with varying degrees of success. When she presented to clinic, Julia was taking daily doses of Guanfacine and Citalopram, and also Chlorpromazine as needed for exacerbations of aggressive behavior, and was experiencing significant disruptive behavior at home and aggressiveness toward her mother and father, depressed mood, impairment in social relatedness at school, and ongoing motor tics. Julia and her parents participated in a thorough intake evaluation over several sessions. As part of the evaluation, the child psychiatry fellow gathered collateral information from prior hospital records and school personnel. She also collected standardized symptom rating scales from parents and from Julia herself. Julia’s diagnostic profile and presenting problems were discussed and reviewed in rounds with supervisors and other trainees. Julia’s treatment initially focused on medication management for the first three sessions; however, these visits involved significant crisis management related to her oppositional and aggressive behavior. In addition, the parents had difficulty refraining from arguing with each other during sessions, which made treatment planning unproductive. This case was selected to begin treatment from the MAP systems model approach. Using PWEBS, the trainee searched the literature for EBPs relevant to this patient’s demographic characteristics and presenting problems. Although this patient had a number of clinical diagnoses, the trainee identified that Julia’s most impairing problem currently was disruptive and aggressive behavior. The PWEBS search helped the trainee to generate ideas for treatment and to learn from the research which practice elements were most effective for children with similar presenting problems (disruptive behavior). Next the trainee generated a F-I-F, where she specified the focus of treatment (disruptive behavior), delineated steps of treatment (connect, cultivate, and consolidate), and identified areas of potential interference including parental discord, motor tics, and depression. The trainee also outlined a Treatment Pathway and reviewed it with the family during the first MAP therapy session. The Treatment Pathway is a tool used to organize the treatment plan. This tool is a flow sheet similar to the F-I-F and served as a useful anchor for the family. Every time the family introduced a new ‘‘crisis of
Downloaded by [Aston University] at 11:31 06 September 2014
320
KATAOKA ET AL.
the week,’’ the trainee helped focus the session by referring back to the Treatment Pathway. The trainee also created a Dashboard with ideographic and objective measures specific to this family. The Dashboard for this patient included the number of weekly Chlorpromazine tablets needed, ratings from a behavior rating scale, hours of sleep, and number of weekly incidents of aggression. The focus of treatment was disruptive behavior and as such the treatment was targeted to improve parenting skills. Practice elements including praise, positive attending, rewards, time-out, and effective consequences were taught, modeled, and role-played with the parents. One Plan-Do-Study-Act cycle that the trainee implemented involved the problem of the parents arguing during sessions, which interfered significantly with further progress. The ‘‘Plan’’ was to split the session into two parts and meet with each parent separately but review the same content with each parent. In the ‘‘Study’’ of this improvement plan, parents went from not absorbing any skills to progressing each week and gaining competency in applying skills at home. With the success of this modification, the trainee continued the sessions in this manner. After approximately four sessions, the trainee was able to bring the parents together in the same room for part of the session. To date the family has made significant progress as tracked on the Dashboard. The child’s number of aggressive outbursts toward the mother has decreased from 30 per week to approximately five per week, the number of Chlorpromazine doses needed has declined from six per week to zero in the past 2 weeks, and the parents note significant improvement on behavior rating scales. By using formative evaluative feedback throughout this case, the trainee could readily assess each week the progress of the patient. When progress was not occurring, the Plan-Do-Study-Act cycle resulted in improvement of Dashboard measures. Through the use of the MAP systems model and specific MAP tools (PWEBS, Dashboard, Practitioner Guides), the trainee effectively structured sessions and implemented EBPs for this family. The use of MAP with this case has engendered personal and professional growth in the trainee, consistent with the ACGME area of PBLI. The Therapist Portfolio for this trainee shows progress in the areas of clinical problem solving, the use of MAP tools, and the application of various practice domains. The trainee has demonstrated a knowledge and production of the F-IF as well as knowledge and production in the use of PWEBS, the Dashboard, and the Practitioner Guides. This trainee has progressed from the didactic and rehearsal level to the knowledge and production level for practice elements including praise, monitoring, problem solving, time-out, communication skills, and
response cost. With regard to the overall MAP Professional Competence Model, the trainee has demonstrated increased self-awareness and insight into her own skills, an increase in openness to learning new material and different therapeutic techniques, improvement in goal setting, and progress in the utilization of systems-based practice.
DISCUSSION Overall, the MAP system provides useful tools and structure that address each of the core components of PBLI for child psychiatry training. MAP provides not only a foundation for teaching trainees how to practice evidence-based care but also a framework for reflective learning, an important component for self-assessment and professional development (Carraccio & Englander, 2004). As an adjunct to traditional quality improvement projects, MAP also provides a novel approach for teaching PBLI in the context of learning and practicing a wide variety of clinically indicated evidence-based psychotherapy techniques in the outpatient setting. Within this enhanced framework that aligns MAP with the ACGME core competencies, we further apply some of our early lessons learned to guide recommendations for future use of MAP to improve psychotherapy training and teach PBLI for child psychiatry fellows. To our knowledge, this is the first description of PBLI for child psychiatry fellows that teach this competency while learning a psychotherapy approach. In fact, MAP was adopted by the faculty of this clinic to improve the quality of psychotherapy taught in this clinic. Prior attempts to implement traditional evidencebased treatment manuals in this general outpatient clinic had been met with challenges. These challenges included difficulties in fitting complex and comorbid children and their families into any one treatment protocol and having supervisors well versed in multiple treatment protocols, and then teaching those protocols to fellows as cases presented to clinic. MAP had the potential to teach psychotherapy in a quality improvement paradigm and teach a broad base of psychotherapy skills through an integrated and easily navigated system of core skills. Given this early stage of delivering MAP in a teaching clinic, we have yet to conduct a formal evaluation. However we did gather informal program evaluation information from the trainees about (a) their frequency of use of the MAP tools (e.g., PWEBS, F-I-F, Clinical Event Structures, Practitioner Guides), (b) familiarity and comfort using the various Practitioner Guides, and (c) challenges in implementing MAP. Fellows used PWEBS and the Practitioner Guides most frequently of the MAP tools, and they reported feeling most
Downloaded by [Aston University] at 11:31 06 September 2014
PRACTICE-BASED LEARNING AND IMPROVEMENT
comfortable using the ‘‘Engagement with Caregiver’’ and ‘‘Engagement with Child’’ Practitioner Guides. Challenges reported by the fellows in implementing MAP during clinic included the number of cases that were covered during supervision time as well as the additional time spent on preparing Dashboards. Overall, these results as well as anecdotal evidence from the course of the academic year to date suggest that child psychiatry fellows demonstrate willingness to learn the therapeutic techniques in MAP, find the MAP tools useful, and appreciate the flexibility inherent in the MAP system. Areas for continued growth, with regard to trainee attitudes toward the implementation of MAP in our clinic are centered on the clinic format itself (e.g., need for additional supervision time, need for designated time to prepare for MAP sessions and complete tools such as Dashboards). Fellows also incorporated treatment protocols from our divisional faculty for select cases when specific Practitioner Guides were not available in the MAP library, yet still using other MAP tools such as the Dashboard. As highlighted by Borntrager, Chorpita, HigaMcMillan, and Weisz (2009), provider attitudes toward EBPs influence the dissemination of these practices in real-world settings. As part of a randomized clinical effectiveness trial, Borntrager et al. assessed the attitudes of therapists before and after training for a standard evidence-based treatment protocol and for a modular evidence-based treatment protocol. Results of this study suggested that compared with the standard condition, in the modular condition therapists’ attitudes were significantly more favorable toward EBPs. Therapists in the modular condition perceived more flexibility in their approach to treatment, and this is also something anecdotally we have seen in our clinic. It is important to note, however, that trainees in our clinic have varying degrees of experience with traditional evidence-based treatment protocols. Further research is needed to gather information regarding acceptability of standard compared to modular evidence-based protocols in training settings. In further recognizing that we are utilizing the MAP system beyond the boundaries of its evidence base, we would also like to propose areas for future research. Qualitative data from child psychiatry fellows and faculty is needed to further inform how MAP should be used with child psychiatry patients and their families as well as within the context of a teaching clinic within a large academic medical center. Following adaptations guided from this feedback, pilot testing the feasibility and acceptability of the revised implementation of MAP would be merited. Within this pilot, development of methods to measure educational outcomes such as retaining information about PBLI and how this might relate to changes in practice short- and
321
longer term would be important to assess. With these developmental stages achieved, future research examining the impact of MAP on improved quality of care mediated through improved clinician training may be proposed. In addition, given its flexible model, MAP has the potential to be adapted for child psychiatric practice, which often includes patients who require combined psychotherapy and psychopharmacological treatment. During the early piloting of MAP, we observed that not infrequently trainees include medication related metrics on their Dashboard, as in our case example. Measures of treatment compliance, amount of ‘‘as needed’’ medications used, and side effects can be included in weekly monitoring to inform medication management decisions. In addition, guidelines for common psychotropic medication treatment decisions are readily available (American Academy of Pediatrics, 2000; Greenhill, Pliszka, Dulcan, & the Work Group on Quality Issues, 2002), and a few national quality measures for child mental health include recommended frequencies for stimulant medication monitoring (Zima et al., 2013). Thus, adaptation of MAP for psychotropic medication management may be a natural extension of the framework to develop a potential tool for transparently integrating recommended care practices for psychotropic medication treatment with evidence-based psychotherapies. Yet in considering these prospective areas for new research, we also observed potential barriers that merit addressing in applying MAP to a child psychiatry teaching clinic. Similar to other evidence-based psychotherapies, time and resources are needed to train, implement, and sustain their use. Protected time for fellows and faculty to receive training and ongoing support in implementing MAP, as well as training and resource costs, need to be considered. Preparatory work is also often required outside of the clinic visit; therefore, time should be protected for child psychiatry fellows to prepare for patient visits and supervision. Engagement and continuous support to enhance the organizational climate to support successful implementation over time may be needed, given prior research showing that organizational climate can influence implementation of EBPs (Aarons & Sawitzky, 2006; Glisson & Hemmelgarn, 1998). Despite these potential barriers, MAP provides a practical way to teach PBLI competencies while delivering psychotherapy in the flexible way that is frequently needed in the child psychiatric outpatient setting. By teaching PBLI in the context of psychotherapy practice, during a yearlong outpatient training experience, lifelong learning and quality improvement will hopefully become embedded in the routine practice of graduates.
322
KATAOKA ET AL.
Downloaded by [Aston University] at 11:31 06 September 2014
REFERENCES Aarons, G. A., & Sawitzky, A. C. (2006). Organizational culture and climate and mental health provider attitudes toward evidence-based practice. Psychological Services, 3, 61. Accreditation Council for Graduate Medical Education. (2009). Practice-based Learning and Improvement: Common Program Requirement. Available from http://www.acgme.org American Academy of Pediatrics, Committee on Quality Improvement, Subcommittee on Attention-Deficit=Hyperactivity Disorder. (2000). Clinical practice guideline: Diagnosis and evaluation of the child with attention-deficit=hyperactivity disorder. Pediatrics, 105, 1158–1170. Arbuckle, M. R., Weinberg, M., Cabaniss, D. L., Kistler, S. C., Isaacs, A. J., Sederer, L. I., et al. (2013). Training psychiatry residents in quality improvement: An integrated, year-long curriculum. Academic Psychiatry, 37, 42–45. Asarnow, J. R., Jaycox, L. H., Duan, N., LaBorde, A. P., Rea, M. M., Murray, P., et al. (2005). Effectiveness of a quality improvement intervention for adolescent depression in primary care clinics: A randomized controlled trial. The Journal of the American Medical Association, 293, 311–319. Beidas, R. S., & Kendall, P. C. (2010). Training therapists in evidencebased practice: A critical review of studies from a systems-contextual perspective. Clinical Psychology: Science and Practice, 17, 1–30. Borntrager, C., Chorpita, B., Higa-McMillan, C., & Weisz, J. (2009). Provider attitudes toward evidence-based practices: Are the concerns with the evidence or with the manuals? Psychiatric Services, 60, 677–681. Brook, R. H., Kamberg, C. J., & McGlynn, E. A. (1996). Health system reform and quality. Journal of the American Medical Association, 276, 476–480. Carraccio, C., & Englander, R. (2004). Evaluating competence using a portfolio: A literature review and web-based application to the ACGME competencies. Teaching and Learning in Medicine, 16, 381–387. Chorpita, B. F., & Daleiden, E. L. (2009). Mapping evidence-based treatments for children and adolescents: Application of the distillation and matching model to 615 treatments from 322 randomized trials. Journal of Consulting and Clinical Psychology, 77, 566–579. Chorpita, B. F., & Daleiden, E. L. (2013). Structuring the collaboration of science and service in the pursuit of a shared vision. Journal of Child and Adolescent Clinical Psychology. Advance online publication. doi: 10.1080/15374416.2013.828297 Chorpita, B. F., Daleiden, E. L., Ebesutani, C., Young, J., Becker, K. D., Nakamura, B. J., et al. (2011). Evidence-based treatments for children and adolescents: An updated review of indicators of efficacy and effectiveness. Clinical Psychology: Science and Practice, 18, 153–171. Cleghorn, G. D., & Headrick, L. A. (1996). The PDSA cycle at the core of learning in health professions education. Joint Commission Journal of Quality Improvement, 22, 206–212. Falzer, P. R., & Garman, D. M. (2012). Evidence-based decisionmaking as a practice-based learning skill: A pilot study. Academic Psychiatry, 36, 104–109.
Glisson, C., & Hemmelgarn, A. (1998). The effects of organizational climate and interorganizational coordination on the quality and outcomes of children’s service systems. Child Abuse & Neglect, 22, 401–421. Greenhill, L. L., Pliszka, S., Dulcan, M. K., & Work Group on Quality Issues. (2002). Practice parameters for the use of stimulant medications in the treatment of children, adolescents, and adults. Journal of the American Academy of Child and Adolescent Psychiatry, 41(2 Suppl.), 26S–49S. Institute of Medicine, Committee on Quality of Health Care in America. (2001). Crossing the quality chasm: A new health system for the 21st century. Washington, DC: National Academy Press. Reardon, C. L., Ogrinc, G., & Walaszek, A. (2011). A didactic and experiential quality improvement curriculum for psychiatry residents. Journal of Graduate Medical Education, 562–565. Schuster, M. A., McGlynn, E. A., & Brook, R. H. (2005). How good is the quality of health care in the United States? Milbank Quarterly, 83, 843–895. Sockalingam, S., Stergiopoulous, V., Maggi, J., & Zaretsky, A. (2010). Quality education: A pilot quality improvement curriculum for psychiatry residents. Medical Teacher, 32, e221–e226. Swick, S., Hall, S., & Beresin, E. (2006). Assessing the ACGME competencies in psychiatry training programs. Academic Psychiatry, 30, 330–351. Swing, S. R. (2007). The ACGME outcome project: Retrospective and prospective. Medical Teacher, 29, 648–654. U.S. Department of Health, & Human Services. (2003). New Freedom Commission on Mental Health, achieving the promise: Transforming mental health care in America (Final report). Rockville, MD: Author. U.S. Public Health Service. (2000). Report of the Surgeon General’s Conference on Children’s Mental Health: A National Action Agenda. Washington, DC: Department of Health and Human Services. Weerasekera, P., Manring, J., & Lynn, D. (2010). Psychotherapy training for residents: Reconciling requirements with evidence-based, competency-focused practice. Academic Psychiatry, 34, 5–12. Weissman, M. M., Verdeli, H., Gameroff, M. J., Bledsoe, S. E., Betts, K., Mufson, L., et al. (2006). National survey of psychotherapy training in psychiatry, psychology, and social work. Archives of General Psychiatry, 63, 925–934. Weisz, J. R. (2000). Agenda for child and adolescent psychotherapy research: On the need to put science into practice. Archives of General Psychiatry, 57, 837. Wells, K. B., Sherbourne, C., Schoenbaum, M., Duan, N., Meredith, L., Unutzer, J. R., et al. (2000). Impact of disseminating quality improvement programs for depression in managed primary care. The Journal of the American Medical Association, 283, 212–220. Zima, B. T., Murphy, J. M., Scholle, S. H., Hoagwood, K. E., Sachdeva, R. C., Mangione-Smith, R., et al. (2013). National quality measures for child mental health care: Background, progress, and next steps. Pediatrics, 131, S38.
