584929
research-article2015
CPJXXX10.1177/0009922815584929Clinical PediatricsZenlea et al
Original Article
Enhancing Patient Safety in Pediatric Primary Care: Implementing a Patient Safety Curriculum
Clinical Pediatrics 1–8 © The Author(s) 2015 Reprints and permissions: sagepub.com/journalsPermissions.nav DOI: 10.1177/0009922815584929 cpj.sagepub.com
Ian S. Zenlea, MD, MPH1,2, Elene Scheff, PT1,3, Barbara Szeidler, RN, BS, LNC, CPHQ4, Anjala Tess, MD2,5, Jean Santangelo, RN, BSN1, Luke Sato, MD2,4, Kathy J. Jenkins, MD, MPH2,3, and Glenn Focht, MD1
Abstract Objective. We developed and implemented a patient safety (PS) curriculum targeted at clinicians and nonclinical office practice staff within a large primary care pediatric network. Methods. Curricular content was informed by medical literature, local PS experts, and malpractice claims data. Sessions were centered on illustrative closed malpractice cases or informed by identified safety events. Participants provided subjective responses to the postsession evaluations. Results. Invited participants from 12 practices included both clinical and nonclinical practice staff (up to 24 attendees per session). Participants reported that they were confident in their knowledge and skills. Several participants engaged in improvement projects that included active surveillance of high-risk patients, improvements in referral and test result management processes, and the distribution of patient educational materials. Conclusions. We successfully developed and implemented a multifaceted PS curriculum for pediatric providers. Participants enjoyed the sessions and several engaged in new PS projects as a result of the program. Keywords patient safety, quality improvement, continuing education, pediatrics, primary care
Introduction Formal training in quality improvement (QI) and patient safety (PS) has become an integral component of medical education in the United States and internationally.1 In the United States, postgraduate trainees are expected to achieve proficiency in advocating for optimal patient care delivery systems, identifying medical errors, and working in interprofessional teams to enhance PS and care quality.2 More recently, the need to formally train practicing physicians has been recognized.3 In the United States, several states now require completion of risk management and PS continuing medical education (CME) hours for physician license renewal.4 As of June 2013, 14 of the American Board of Medical Specialties formally adopted QI activities as part of the Maintenance of Certification (MOC) Part 4–Practice Performance Assessments required for continued medical licensure.3 In response to a trend of increasing malpractice claims originating from the ambulatory setting, Boston Children’s Hospital (BCH), in 2012, was awarded funding by the Risk Management Foundation of the Harvard
Medical Institutions, Inc (CRICO), the Harvard captive malpractice insurer, to collaborate with the Pediatric Physicians’ Organization at Children’s (PPOC) to establish an ambulatory PS program. This PS program would be led by the PPOC, aligned with the existing inpatient BCH PS program, staffed by an ambulatory PS manager, and include the infrastructure to support a robust safety event reporting system and targeted performance improvement efforts. Recognizing the need to formally 1
Pediatric Physicians’ Organization at Children’s, Brookline, MA, USA 2 Harvard Medical School, Boston, MA, USA 3 Boston Children’s Hospital, Boston, MA, USA 4 Risk Management Foundation of the Harvard Medical Institutions, Inc, Cambridge, MA, USA 5 Beth Israel Deaconess Medical Center, Boston, MA, USA Corresponding Author: Ian S. Zenlea, Department of Paediatrics, Credit Valley Hospital, 2200 Eglinton Avenue West, Mississauga, Ontario, L5M 2N1, Canada. Email: [email protected]
Downloaded from cpj.sagepub.com at CALIFORNIA STATE UNIV FRESNO on July 26, 2015
2
Clinical Pediatrics
train PPOC clinicians and support staff in PS science, as an initial phase of our work, we developed an interprofessional PS curriculum that was implemented in 12 primary care pediatric practices. Although there are published reports describing QI interventions with continuing medical education and process improvement components targeted at practicing pediatric clinicians,5-10 there are limited reports specific to PS. In this article, we present the novel design and implementation of a multifaceted PS curriculum targeted at practicing pediatric clinicians and office staff, the teaching methods used, participant learning outcomes, and the next steps in our continued work toward mitigating risk in pediatric primary care.
Methods Educational Context The PPOC is an independent practice association of 77 privately owned primary care pediatric practices, composed of nearly 280 physicians, affiliated with BCH. The PPOC provides primary care to an estimated 400 000 patients in eastern Massachusetts. The PPOC has a centralized and integrated quality infrastructure comprised of QI consultants and information technology specialists who are trained in clinical microsystems and change management. They are deployed regionally to provide onsite practice support with quality measurement, electronic health record support and implementation, and process improvement efforts. Twelve pediatric practices (including BCH Primary Care at Longwood and BCH Adolescent Medicine) participated in the initial 2-year phase of the ambulatory patient safety educational program from May 2012 to May 2014. Participants were volunteers interested in learning more about PS or by invitation, members of practices in which a safety event had recently occurred. The PS curriculum was delivered over a 15-month period from late January 2013 to early May 2014. All members of the practice staff, both clinical and nonclinical, were invited to participate. Continuing medical education requirements for physician license renewal in Massachusetts include obtaining 10 Risk Management Study credits over the 2-year renewal cycle aimed at instructing physicians in medical malpractice prevention such as risk identification and mitigation, patient safety, and loss prevention.4 Additionally, all licensed nurses in Massachusetts are required to obtain 15 contact hours of continuing education (CE) for license renewal.11 Therefore, to further engage physician and nurse participants and facilitate fulfillment of licensure requirements, approval was obtained for physician Risk Management Study CME
credits through CRICO and nursing CE units through BCH.
Patient Safety Curriculum The blueprint for the curricular content and objectives was informed by publications characterizing ambulatory patient safety risk12-14 and data provided by CRICO regarding the nature of malpractice claims in the pediatric ambulatory setting (personal communication, August 2012) (Table 1). Content was also informed by focused onsite practice assessments, specific needs assessment surveys, and by prior experience of the authors (GF, KJJ, LS, BS) and local content experts (NR). The focus was on the following identified top contributing factors: communication failures (with the patient/family/caregiver and between providers) and clinical judgment failures (failure or delay in obtaining a diagnostic test, consult, or referral). The PS curriculum emphasized 2 key principles in PS science: (1) the importance of utilizing a systemsbased approach to care11,12 and (2) recognizing the impact of systems on effective closed-loop communication between patients/families/caregivers and members of the clinical care team across the care continuum. Targeted goals of the PS curriculum were to (1) increase the participant’s knowledge of PS science, the value of safety event and near miss identification and review, and the relationship between PS and care quality; (2) improve the participant’s knowledge of clinical microsystems tools to drive rapid cycle improvements in processes related to closed-loop communication, patient handoffs, results and referral management, and differentiating between complaints/grievances and patient safety risks; (3) impart participants with skills to identify a safety event or risk to PS in order to activate a root cause analysis; and (4) promote a culture of safety through shared storytelling and trust in reporting. As the primary focus of the educational sessions was to increase knowledge and change attitudes, completion of PS projects was encouraged, but not mandatory. For those practices that chose to undertake projects, the PPOC QI consultants and course instructors (GF, ISZ, ES, BS) provided support with project implementation. Our PS curriculum leveraged adult learning principles and combined a series of 4 in-person, interactive didactic sessions with a networking dinner (each session was 2.5 hours total in length; Table 1). Session content was centered on illustrative closed malpractice cases provided by CRICO and informed by reported safety events, or potential risks to PS identified within the PPOC practices (Table 1). Collaborative discussions around safety stories were used to teach PS concepts and raise awareness of PS science and initiatives. To encourage a safety
Downloaded from cpj.sagepub.com at CALIFORNIA STATE UNIV FRESNO on July 26, 2015
3
Zenlea et al Table 1. Blueprint for Patient Safety Curriculum. Content Area Session 1: Core Principles and Science of Patient Safety
Session 2: Referral Management
Session 3: Test result management
Session 4: Managing Complaints and Grievances
Topic
Teaching Methods
•• Financial impact and epidemiology of malpractice claims in the pediatric ambulatory setting •• Systems-based thinking •• Swiss cheese model of medical error15 •• Utilizing evidence-based algorithms and clinical decision aides for accurate diagnosing •• Closed loop communication strategies and effective handoffs •• Epidemiology of malpractice claims in the pediatric ambulatory setting specific to referral management •• Elements of highly reliable referral management processes •• Referral classification and prioritization •• Understanding the current process using clinical microsystems16 •• Application of 5 Ps clinical microsystems approach16 and Plan, Do, Study, Act cycles17 to design an ideal referral management process •• Patient and family engagement in the process •• Epidemiology of malpractice claims in the pediatric ambulatory setting specific to test result management •• Elements of highly reliable test result management processes •• Understanding the current process using clinical microsystems16 •• Applying of 5 Ps clinical microsystems approach16 and Plan, Do, Study, Act cycles17 to design an ideal referral management process •• Patient and family engagement in the process •• Change management and creating a culture of safety •• Definitions of complaints, grievances, near misses, and safety events •• Policies and processes for complaint management •• Identifying a safety event to initiate a root cause analysis •• Using clinical microsystems approach16 to drive improvement from complaints •• Peer support if named in malpractice case
•• Interactive didactic •• Closed malpractice case presentations of missed appendicitis •• Case discussion •• Reflection on current practice and identification of safety priorities
culture during the didactic sessions, practices discussed the challenges and successes of undertaken process improvement efforts and shared safety stories. To promote health systems–based thinking and collaboration across the care continuum, specialists from BCH were invited to participate in and present on collaborative efforts to improve closed-loop communication with primary care providers.
Program Evaluation Standardized needs assessment and postsession surveys were created using the curricular blueprint and session
•• Interactive didactic •• Closed malpractice case presentation of delayed specialty referral for evaluation and management of a facial mass •• Case discussion to identify systemsbased issues contributing to adverse outcomes •• Reflection on current practice
•• Interactive didactic •• Closed malpractice case presentation of a missed newborn screen result and absence of recommended follow-up •• Case discussion to identify systemsbased issues contributing to adverse outcome •• Role-modeling through practice presentations of quality improvement efforts •• Reflection on current practice •• Interactive didactic •• Case presentation of safety event initially captured as a caregiver complaint •• Group exercise incorporating process mapping, brainstorming, and root cause analysis •• Reflection on current practice •• Shared experience by physician involved in malpractice trial
objects to assess level 1 (reaction) and level 2 (learning) learner outcomes according to Kirkpatrick's four levels of learning evaluation18 . Survey items were rated on 4or 5-point Likert-type scales with additional open-ended questions to allow for comments from respondents. The following 3 open-ended questions were asked in each survey to further capture detailed level 1 (reaction) outcomes: “What changes to your clinical practice would you make?,” “What barriers do you anticipate encountering as you make changes in your practice?,” and “What were the two most valuable ‘take-aways’ from this meeting?” Participant suggestions for future topics and needs for further discussion or clarification of core
Downloaded from cpj.sagepub.com at CALIFORNIA STATE UNIV FRESNO on July 26, 2015
4
Clinical Pediatrics
concepts were also elicited. Level 3 (behavior) outcomes18 were assessed during the didactic sessions through presentations of improvement work by participants, onsite visits by the 2 of the authors (ISZ and ES), and via a follow-up email communication with the participants on completion of the educational series.
Analyses We summarized the program evaluation data descriptively, using percentages for categorical data. Two of the authors (ISZ and ES) and a PPOC intern independently reviewed participant responses to open-ended questions and marked and categorized key words and phrases to generate initial themes. Initial themes were then reviewed and final key themes were generated by group consensus. The study was deemed quality improvement and therefore exempt from review by the BCH Institutional Review Board.
Results Six of the 12 (50%) practices were represented at each session. Two of the PPOC practices attended only the first session prior to disenrolling. In addition to BCH and PPOC administrative, safety, and quality staff, 11 to 19 physicians, 1 to 5 nurses and advanced practice nurses, and 2 to 4 practice staff (2 office practice managers, 2 medical home care coordinators), attended each session. Because of increased interest within the participating practices, other practice members also attended some sessions resulting in a variation of participant numbers. Of those enrolled and with planned attendance, session participation was 95% (n = 19/20) for session 1; 73% (n = 11/15) for session 2; 82% (n = 18/22) for session 3; and 90% (n = 18/20) for session 4. Additionally, there were last minute, drop-in participants (between 2 and 4 at each session) from already enrolled practices, which increased attendance to 24 at sessions 1 and 3. A total of 59 CME credits and 19 CEs were granted. Overall, participants rated the sessions highly. The proportions of participants who rated the sessions as “Excellent” were greatest for sessions 1 and 2 (81% and 91%, respectively) and lowest for sessions 3 and 4 (60% and 67%, respectively). Participants reported improvements in knowledge and skills (level 2 outcomes) (Table 2). Themes and selected comments from the 3 open-ended questions reflecting level 1 outcomes (reaction) are presented in Table 3. Participants reported that anticipated changes to clinical practice would center on documentation and communication practices, improved understanding of current practice patterns, in-office safety culture, active surveillance and follow-up of test results,
and accessibility of information for sharing across the care continuum. The cited barriers to changing current practice included culture shift, general resistance to change, absence of shared responsibility and accountability, lack of information sharing, inconsistent communication, and limited resources. Participants understood the importance of effective communication and the need for highly reliable processes for surveillance and tracking of referrals, tests, and imaging results. Optional improvement projects (behavior; level 3 outcomes) undertaken by the participants addressed a variety of topics, including active surveillance and follow-up of newborn screen results, referral and test result management process redesign, formation of a patient safety council to actively review best practices, active tracking of patients referred for emergency care, and distribution of patient education tools such as “Ask Me 3” from the National Patient Safety Foundation.19 Selected improvement projects were shared during the didactic sessions with open discussion of successes and challenges with implementation and measurement. Practices were working across the care continuum and engaging with the emergency department at BCH to improve closed loop communication throughout the referral and discharge processes.
Discussion We developed and implemented a PS curriculum for pediatric primary care clinicians and office staff. Overall, the curriculum was highly rated, suggesting that it was an acceptable, well-received, and relevant modality for delivering PS content. We engaged the learners by relating the course material to participants’ experiences through actual malpractice cases, facilitating self-reflection on current practice, and using needs assessment surveys to tailor the curriculum. Nonclinical staff (medical home care coordinators and practice managers) participated in the sessions and appeared eager to learn new concepts and implementation tools. Providing networking opportunities helped create a sense of community and encouraged the sharing of safety stories, which is critical to creating a culture of safety. We were highly encouraged by the results of this initial curriculum, particularly with respect to participant reactionstoward the curriculum, self-reported changes in knowledge and skills, and the variety of voluntary improvement projects undertaken.
Lessons Learned Although this first iteration of the curriculum was well received, we identified opportunities for improvement. Based on participant feedback regarding difficulties
Downloaded from cpj.sagepub.com at CALIFORNIA STATE UNIV FRESNO on July 26, 2015
5
Zenlea et al Table 2. Participants’ Self-Assessment of Knowledge and Skills (Completed After Each Session).
Participants Who Responded ‘‘Definitely Agree” or “Agree”a; n (%) Session 1b I can identify systems-based issues contributing to malpractice claims in the outpatient setting I can identify major allegations driving outpatient malpractice claims specific to pediatrics I am able to list strategies my practice and colleagues can implement to improve patient safety in the outpatient setting for patients with abdominal pain Session 2c I can identify strategies to improve patient safety as related to referral management I understand the implications of error prone processes in my practice I can identify barriers to best practice Session 3d I am able to use the Swiss Cheese Model to articulate and explain potential breaches of care I am able to demonstrate at least one best practice for closing the loop of communication regarding a referral process I am able to demonstrate at least one best practice for closing the loop of communication regarding test results Session 4e I am better informed on how to drive improvement based on information gathered from complaint management data I am better able to describe what constitutes complaints, grievances, near misses, and safety events I am clear about which complaint and events should be shared with a patient safety manager
24 (100) 22 (91.7) 21 (87.5) 17 (100) 17 (100) 17 (100) 18 (100) 18 (100) 17 (94.4) 15 (93.8) 14 (87.5) 14 (87.5)
a
Respondents rated each of these items using a 4-point Likert-type scale (do not agree, somewhat agree, agree, definitely agree). N = 24 responses. c N = 18 responses. d N = 17 responses. e N = 16 responses. b
with implementing improvement projects, in future versions of the curriculum, we would incorporate change management and leadership methods to advance participants’ skills to address general staff resistance to change and perceived inequality in roles, responsibility, and accountability. Additionally, we better appreciate the need for ongoing project support at regular intervals and plan to offer intervening webinars to provide the requisite expertise for mentored implementation. With this infrastructure in place, we could take a staged approach to project implementation using Plan, Do, Study, Actcycles.17 Such a platform would also allow us to better capture results at the highest levels of Kirkpatrick’s learning evaluation model (behaviors and clinical and patient outcomes).18 In accordance with adult learning principles, we would incorporate more opportunities for self-reflection20 through specific survey items and role modeling by inviting presentations by past participants. We also noted a decline in session attendance, particularly for sessions 2 (July 2013) and 3 (November
2013), which was likely due to participant scheduling constraints (eg, vacation travel, close to holidays, family obligations etc). The 2 PPOC practices that discontinued were composed of solo practitioners whose ongoing practice obligations and commitments disallowed consistent involvement. To address logistical factors, we would adjust the timing of the sessions to avoid potential conflicts (eg, hold sessions in the morning rather than the evening and avoid holidays or medical conference dates). Additionally, we recognize an opportunity to further incentivize physician participation by offering American Board of Pediatrics (ABP) MOC Part 4 credit21 for improvement projects. Furthermore, sessions 3 and 4 were less highly rated than the preceding sessions. Participant feedback for these sessions centered on the lack of detail and unclear relevance of the case presentations. Although useful for illustrating broad PS themes, closed malpractice cases typically lack certain levels of clinical data, and several participants commented in the feedback questionnaires that more clinical
Downloaded from cpj.sagepub.com at CALIFORNIA STATE UNIV FRESNO on July 26, 2015
6
Clinical Pediatrics
Table 3. Themes From Postsession Evaluations. Question What changes to your clinical practice would you make?
Theme Documentation Communication
Information accessibility and sharing Active surveillance, tracking, and follow-up Evaluation of current process
Culture What barriers do you anticipate encountering as you make changes in your practice?
General resistance Lack of shared responsibility and accountability Culture
Lack of information sharing Lack of resources Inconsistent communication What were the two most valuable “take-aways” from this meeting?”
Communication Systems and processes for surveillance, tracking, or follow-up Reconciliation
detail would have increased the effectiveness of the presentations. To address the lack of detail, as the PPOC PS program expands and a robust, peer protected eventreporting system is established, we would use near miss and safety event reports for case presentations.
Limitations Our study has several limitations. This was the first iteration of a curriculum with data obtained from only a small number of attendees. Ongoing program evaluation
Example “We are going to take a close look at the documentation and delivery of patient instructions” “In cases where there is diagnostic uncertainty there needs to be a tight communication plan . . . Continue to reinforce to our staff that follow up calls to parents and patients are important and their work may be at some time critical” “I believe we need to have the information available for patients and parents to access more readily, perhaps revising our website information” “Use known Pediatric specialists, and ensure patient goes to see them, by having staff call for F/U and to encourage patient to keep [appointment], if specialist note not received in reasonable period of time” “We are planning to process map our current referrals process to identify the Swiss cheese moments . . . and would very much like the opportunity to employ best practices as shown in the process maps of high performing practices last night” “Continue promoting no shame/ blame atmosphere in our office, for the sake of patient care and safety” “Staff reluctance to add to their workflow” “Accountability of everyone in the communications loop is essential . . .” “Is difficult to change what has not been a culture of effective communication in the past . . . Long-standing and ingrained ‘head in the sand ethos that if we don’t acknowledge it, it must not exist’” “Labs and consultants not connected to [electronic health record] need a separate system and as such have greater potential for error in referral management” “Inadequate support staff to call patients who miss appointments” “Communication from specialists/ER/patients is not consistent” “Importance of communicating concerns to [emergency room] personnel when referring our patients” “Lack of standardized process for tracking newborn screening as well as other lab results” “Closing the loop on patient labs and referrals can improve outcomes”
by participants from the next phase of our curriculum is necessary to confirm our findings. We developed the postsession evaluations de novo and did not have the opportunity for validation. Therefore, participants may not have interpreted the questions as we intended and differently from each other. Additionally, we relied on self-assessed changes in knowledge and did not formally evaluate individual participant performance. Although we verified the initial undertaking of QI projects (level 3 outcomes; behavior) through onsite visits and invited presentations at the educational sessions, we
Downloaded from cpj.sagepub.com at CALIFORNIA STATE UNIV FRESNO on July 26, 2015
7
Zenlea et al have not assessed project sustainability other than by emailed follow-up. Given that project completion was voluntary, we did not require practices to report patientlevel or clinical data. Therefore, we were unable to capture level 4 outcomes (results) for the projects.
of the Harvard Medical Institutions, Inc. (CRICO). Dr. Tess is the Course Director for the Quality and Safety Educators Academy at the Society for Hospital Medicine and the Academic Alliance for Internal Medicine. The other authors have no financial disclosures or conflicts of interest.
Funding
Conclusions We successfully implemented a multifaceted, interprofessional PS curriculum in the pediatric primary care setting. Grounding the teachings in clinical microsystems imparted participants with the knowledge to recognize a safety event, near miss, or potential risk to PS and begin to analyze and enact change. Although curriculum development and session planning was time intensive, we now have a packaged curriculum that can serve as a shared resource for participants and for further dissemination. This is particularly appealing in the context of the collaboration between the PPOC and BCH to establish a comprehensive ambulatory PS program. Although this was a single institutional experience, we believe that our work can inform other ambulatory risk management programs and patient safety educational interventions targeted at practicing clinicians and office practice staff. Acknowledgments The authors wish to thank the Boston Children’s Hospital Program for Patient Safety and Quality and the Pediatric Physician’s Organization at Children’s for supporting this work. The authors wish to thank Elizabeth Lambert, MEd, for her administrative support, Nicole Pelletier, MPH for assistance with compiling and analyzing the survey data, and Nina Rauscher, MS, RN, CPHQ, for her support and thoughtful review of our manuscript. We thank Jane Gagne for coordinating the Risk Management CME credits and Gregory Durkin, MEd, RN-BC, for coordinating the CE units. We thank Graham T. McMahon, MBBCh, MD, MMSc, for his thoughtful comments and review of the manuscript.
Author Contributions ISZ, ES, BS, AT, JS, LS, KJJ, and GF contributed to the project conception or design; ISZ, ES, BS, LS, and GF contributed to data acquisition, analysis, or intrepretation; ISZ drafted the manuscript; ES, BS, AT, JS, LS, KJJ, and GF critically revised the manuscript; GF gave final approval of the manuscript; ISZ and GF agree to be accountable for all aspects of the work ensuring integrity and accuracy.
Declaration of Conflicting Interests The author(s) declared the following potential conflicts of interest with respect to the research, authorship, and/or publication of this article: Dr. Focht is a Safety & Operations Consultant for Ingenious Med and an active member of the Ambulatory Care Risk Group at Risk Management Foundation
The author(s) disclosed receipt of the following financial support for the research, authorship, and/or publication of this article: This project was supported by Risk Management Foundation of the Harvard Medical Institutions, Inc. (CRICO) through the Ambulatory Risk Management Grant to Boston Children’s Hospital and the Harvard Medical School Fellowship in Patient Safety and Quality.
References 1. Davis NL, Davis DA, Johnson NM, et al. Aligning academic continuing medical education with quality improvement: a model for the 21st century. Acad Med. 2013;88:1437-1441. 2. Accreditation Council for General Medical Education. Next Accreditation System (NAS). 2013.http://www.acgme.org/ acgmeweb/tabid/435/ProgramandInstitutionalAccreditation/ NextAccreditationSystem.aspx. Accessed November 27, 2013. 3. American Board of Medical Specialties. MOC Competencies and Criteria. 2014. http://www.abms.org/ Maintenance_of_Certification/MOC_competencies.aspx. Accessed May 21, 2014. 4. Massachusetts Medical Society. Continuing Medical Education Requirements for Physician License Renewal in Massachusetts. 2012. http://www.massmed.org/ Continuing-Education-and-Events/Continuing-MedicalEducation-Requirements-for-Physician-LicenseRenewal-in-Massachusetts/. Accessed May 20, 2014. 5. Boonyasai RT, Windish DM, Chakraborti C, Feldman LS, Rubin HR, Bass EB. Effectiveness of teaching quality improvement to clinicians: a systematic review. JAMA. 2007;298:1023-1037. 6. Margolis PA, Lannon CM, Stuart JM, Fried BJ, KeyesElstein L, Moore DE Jr. Practice based education to improve delivery systems for prevention in primary care: randomised trial. BMJ. 2004;328:388. 7. Rosenthal MS, Lannon CM, Stuart JM, Brown L, Miller WC, Margolis PA. A randomized trial of practice-based education to improve delivery systems for anticipatory guidance. Arch Pediatr Adolesc Med. 2005;159:456-463. 8. Homer CJ, Forbes P, Horvitz L, Peterson LE, Wypij D, Heinrich P. Impact of a quality improvement program on care and outcomes for children with asthma. Arch Pediatr Adolesc Med. 2005;159:464-469. 9. Vernacchio L, Francis ME, Epstein DM, et al. Effectiveness of an asthma quality improvement program designed for maintenance of certification. Pediatrics. 2014;134:e242-e248. 10. Vernacchio L, Trudell EK, Hresko MT, Karlin LI, Risko W. A quality improvement program to reduce
Downloaded from cpj.sagepub.com at CALIFORNIA STATE UNIV FRESNO on July 26, 2015
8
Clinical Pediatrics
unnecessary referrals for adolescent scoliosis. Pediatrics. 2013;131:e912-e920. 11. Mass.gov Health and Human Services. Continuing Education Information. 2014. http://www.mass.gov/ eohhs/gov/departments/dph/programs/hcq/dhpl/nursing/ faq/continuing-education-.html. Accessed May 20, 2014. 12. Neuspiel DR, Stubbs EH. Patient safety in ambulatory care. Pediatr Clin North Am. 2012;59:1341-1354. 13. Miller MR, Pronovost PJ, Burstin HR. Pediatric patient safety in the ambulatory setting. Ambul Pediatr. 2004;4:47-54. 14. Gandhi TK, Lee TH. Patient safety beyond the hospital. N Engl J Med. 2010;363:1001-1003. 15. Reason J. Human error: models and management. BMJ. 2000;320:768-770. 16. Quality by Design: A Clinical Microsystems Approach. 1st ed. San Francisco, CA: Jossey-Bass; 2007.
17. Langley GJ, Ronald M, Nolan KM, Nolan TW, Norman CL, Provost LP. The Improvement Guide: A Practical Approach to Enhancing Organizational Performance. San Francisco, CA: Jossey-Bass; 2009. 18. Kirkpatrick DL, Kirkpatrick JD. Evaluating Training Programs: The Four Levels. 3rd ed. San Francisco, CA: Berrett-Koehler; 2006. 19. National Patient Safety Foundation. Ask Me 3. 2014. http:// www.npsf.org/for-healthcare-professionals/programs/ ask-me-3/. Accessed June 16, 2014. 20. Wittich CM, Reed DA, Ting HH, et al. Measuring reflection on participation in quality improvement activities for maintenance of certification. Acad Med. 2014;89: 1392-1397. 21. American Board of Pediatrics. Performance in Practice (Part 4). 2015. https://www.abp.org/content/performancepractice-part-4. Accessed April 28, 2015.
Downloaded from cpj.sagepub.com at CALIFORNIA STATE UNIV FRESNO on July 26, 2015
research-article2015
CPJXXX10.1177/0009922815584929Clinical PediatricsZenlea et al
Original Article
Enhancing Patient Safety in Pediatric Primary Care: Implementing a Patient Safety Curriculum
Clinical Pediatrics 1–8 © The Author(s) 2015 Reprints and permissions: sagepub.com/journalsPermissions.nav DOI: 10.1177/0009922815584929 cpj.sagepub.com
Ian S. Zenlea, MD, MPH1,2, Elene Scheff, PT1,3, Barbara Szeidler, RN, BS, LNC, CPHQ4, Anjala Tess, MD2,5, Jean Santangelo, RN, BSN1, Luke Sato, MD2,4, Kathy J. Jenkins, MD, MPH2,3, and Glenn Focht, MD1
Abstract Objective. We developed and implemented a patient safety (PS) curriculum targeted at clinicians and nonclinical office practice staff within a large primary care pediatric network. Methods. Curricular content was informed by medical literature, local PS experts, and malpractice claims data. Sessions were centered on illustrative closed malpractice cases or informed by identified safety events. Participants provided subjective responses to the postsession evaluations. Results. Invited participants from 12 practices included both clinical and nonclinical practice staff (up to 24 attendees per session). Participants reported that they were confident in their knowledge and skills. Several participants engaged in improvement projects that included active surveillance of high-risk patients, improvements in referral and test result management processes, and the distribution of patient educational materials. Conclusions. We successfully developed and implemented a multifaceted PS curriculum for pediatric providers. Participants enjoyed the sessions and several engaged in new PS projects as a result of the program. Keywords patient safety, quality improvement, continuing education, pediatrics, primary care
Introduction Formal training in quality improvement (QI) and patient safety (PS) has become an integral component of medical education in the United States and internationally.1 In the United States, postgraduate trainees are expected to achieve proficiency in advocating for optimal patient care delivery systems, identifying medical errors, and working in interprofessional teams to enhance PS and care quality.2 More recently, the need to formally train practicing physicians has been recognized.3 In the United States, several states now require completion of risk management and PS continuing medical education (CME) hours for physician license renewal.4 As of June 2013, 14 of the American Board of Medical Specialties formally adopted QI activities as part of the Maintenance of Certification (MOC) Part 4–Practice Performance Assessments required for continued medical licensure.3 In response to a trend of increasing malpractice claims originating from the ambulatory setting, Boston Children’s Hospital (BCH), in 2012, was awarded funding by the Risk Management Foundation of the Harvard
Medical Institutions, Inc (CRICO), the Harvard captive malpractice insurer, to collaborate with the Pediatric Physicians’ Organization at Children’s (PPOC) to establish an ambulatory PS program. This PS program would be led by the PPOC, aligned with the existing inpatient BCH PS program, staffed by an ambulatory PS manager, and include the infrastructure to support a robust safety event reporting system and targeted performance improvement efforts. Recognizing the need to formally 1
Pediatric Physicians’ Organization at Children’s, Brookline, MA, USA 2 Harvard Medical School, Boston, MA, USA 3 Boston Children’s Hospital, Boston, MA, USA 4 Risk Management Foundation of the Harvard Medical Institutions, Inc, Cambridge, MA, USA 5 Beth Israel Deaconess Medical Center, Boston, MA, USA Corresponding Author: Ian S. Zenlea, Department of Paediatrics, Credit Valley Hospital, 2200 Eglinton Avenue West, Mississauga, Ontario, L5M 2N1, Canada. Email: [email protected]
Downloaded from cpj.sagepub.com at CALIFORNIA STATE UNIV FRESNO on July 26, 2015
2
Clinical Pediatrics
train PPOC clinicians and support staff in PS science, as an initial phase of our work, we developed an interprofessional PS curriculum that was implemented in 12 primary care pediatric practices. Although there are published reports describing QI interventions with continuing medical education and process improvement components targeted at practicing pediatric clinicians,5-10 there are limited reports specific to PS. In this article, we present the novel design and implementation of a multifaceted PS curriculum targeted at practicing pediatric clinicians and office staff, the teaching methods used, participant learning outcomes, and the next steps in our continued work toward mitigating risk in pediatric primary care.
Methods Educational Context The PPOC is an independent practice association of 77 privately owned primary care pediatric practices, composed of nearly 280 physicians, affiliated with BCH. The PPOC provides primary care to an estimated 400 000 patients in eastern Massachusetts. The PPOC has a centralized and integrated quality infrastructure comprised of QI consultants and information technology specialists who are trained in clinical microsystems and change management. They are deployed regionally to provide onsite practice support with quality measurement, electronic health record support and implementation, and process improvement efforts. Twelve pediatric practices (including BCH Primary Care at Longwood and BCH Adolescent Medicine) participated in the initial 2-year phase of the ambulatory patient safety educational program from May 2012 to May 2014. Participants were volunteers interested in learning more about PS or by invitation, members of practices in which a safety event had recently occurred. The PS curriculum was delivered over a 15-month period from late January 2013 to early May 2014. All members of the practice staff, both clinical and nonclinical, were invited to participate. Continuing medical education requirements for physician license renewal in Massachusetts include obtaining 10 Risk Management Study credits over the 2-year renewal cycle aimed at instructing physicians in medical malpractice prevention such as risk identification and mitigation, patient safety, and loss prevention.4 Additionally, all licensed nurses in Massachusetts are required to obtain 15 contact hours of continuing education (CE) for license renewal.11 Therefore, to further engage physician and nurse participants and facilitate fulfillment of licensure requirements, approval was obtained for physician Risk Management Study CME
credits through CRICO and nursing CE units through BCH.
Patient Safety Curriculum The blueprint for the curricular content and objectives was informed by publications characterizing ambulatory patient safety risk12-14 and data provided by CRICO regarding the nature of malpractice claims in the pediatric ambulatory setting (personal communication, August 2012) (Table 1). Content was also informed by focused onsite practice assessments, specific needs assessment surveys, and by prior experience of the authors (GF, KJJ, LS, BS) and local content experts (NR). The focus was on the following identified top contributing factors: communication failures (with the patient/family/caregiver and between providers) and clinical judgment failures (failure or delay in obtaining a diagnostic test, consult, or referral). The PS curriculum emphasized 2 key principles in PS science: (1) the importance of utilizing a systemsbased approach to care11,12 and (2) recognizing the impact of systems on effective closed-loop communication between patients/families/caregivers and members of the clinical care team across the care continuum. Targeted goals of the PS curriculum were to (1) increase the participant’s knowledge of PS science, the value of safety event and near miss identification and review, and the relationship between PS and care quality; (2) improve the participant’s knowledge of clinical microsystems tools to drive rapid cycle improvements in processes related to closed-loop communication, patient handoffs, results and referral management, and differentiating between complaints/grievances and patient safety risks; (3) impart participants with skills to identify a safety event or risk to PS in order to activate a root cause analysis; and (4) promote a culture of safety through shared storytelling and trust in reporting. As the primary focus of the educational sessions was to increase knowledge and change attitudes, completion of PS projects was encouraged, but not mandatory. For those practices that chose to undertake projects, the PPOC QI consultants and course instructors (GF, ISZ, ES, BS) provided support with project implementation. Our PS curriculum leveraged adult learning principles and combined a series of 4 in-person, interactive didactic sessions with a networking dinner (each session was 2.5 hours total in length; Table 1). Session content was centered on illustrative closed malpractice cases provided by CRICO and informed by reported safety events, or potential risks to PS identified within the PPOC practices (Table 1). Collaborative discussions around safety stories were used to teach PS concepts and raise awareness of PS science and initiatives. To encourage a safety
Downloaded from cpj.sagepub.com at CALIFORNIA STATE UNIV FRESNO on July 26, 2015
3
Zenlea et al Table 1. Blueprint for Patient Safety Curriculum. Content Area Session 1: Core Principles and Science of Patient Safety
Session 2: Referral Management
Session 3: Test result management
Session 4: Managing Complaints and Grievances
Topic
Teaching Methods
•• Financial impact and epidemiology of malpractice claims in the pediatric ambulatory setting •• Systems-based thinking •• Swiss cheese model of medical error15 •• Utilizing evidence-based algorithms and clinical decision aides for accurate diagnosing •• Closed loop communication strategies and effective handoffs •• Epidemiology of malpractice claims in the pediatric ambulatory setting specific to referral management •• Elements of highly reliable referral management processes •• Referral classification and prioritization •• Understanding the current process using clinical microsystems16 •• Application of 5 Ps clinical microsystems approach16 and Plan, Do, Study, Act cycles17 to design an ideal referral management process •• Patient and family engagement in the process •• Epidemiology of malpractice claims in the pediatric ambulatory setting specific to test result management •• Elements of highly reliable test result management processes •• Understanding the current process using clinical microsystems16 •• Applying of 5 Ps clinical microsystems approach16 and Plan, Do, Study, Act cycles17 to design an ideal referral management process •• Patient and family engagement in the process •• Change management and creating a culture of safety •• Definitions of complaints, grievances, near misses, and safety events •• Policies and processes for complaint management •• Identifying a safety event to initiate a root cause analysis •• Using clinical microsystems approach16 to drive improvement from complaints •• Peer support if named in malpractice case
•• Interactive didactic •• Closed malpractice case presentations of missed appendicitis •• Case discussion •• Reflection on current practice and identification of safety priorities
culture during the didactic sessions, practices discussed the challenges and successes of undertaken process improvement efforts and shared safety stories. To promote health systems–based thinking and collaboration across the care continuum, specialists from BCH were invited to participate in and present on collaborative efforts to improve closed-loop communication with primary care providers.
Program Evaluation Standardized needs assessment and postsession surveys were created using the curricular blueprint and session
•• Interactive didactic •• Closed malpractice case presentation of delayed specialty referral for evaluation and management of a facial mass •• Case discussion to identify systemsbased issues contributing to adverse outcomes •• Reflection on current practice
•• Interactive didactic •• Closed malpractice case presentation of a missed newborn screen result and absence of recommended follow-up •• Case discussion to identify systemsbased issues contributing to adverse outcome •• Role-modeling through practice presentations of quality improvement efforts •• Reflection on current practice •• Interactive didactic •• Case presentation of safety event initially captured as a caregiver complaint •• Group exercise incorporating process mapping, brainstorming, and root cause analysis •• Reflection on current practice •• Shared experience by physician involved in malpractice trial
objects to assess level 1 (reaction) and level 2 (learning) learner outcomes according to Kirkpatrick's four levels of learning evaluation18 . Survey items were rated on 4or 5-point Likert-type scales with additional open-ended questions to allow for comments from respondents. The following 3 open-ended questions were asked in each survey to further capture detailed level 1 (reaction) outcomes: “What changes to your clinical practice would you make?,” “What barriers do you anticipate encountering as you make changes in your practice?,” and “What were the two most valuable ‘take-aways’ from this meeting?” Participant suggestions for future topics and needs for further discussion or clarification of core
Downloaded from cpj.sagepub.com at CALIFORNIA STATE UNIV FRESNO on July 26, 2015
4
Clinical Pediatrics
concepts were also elicited. Level 3 (behavior) outcomes18 were assessed during the didactic sessions through presentations of improvement work by participants, onsite visits by the 2 of the authors (ISZ and ES), and via a follow-up email communication with the participants on completion of the educational series.
Analyses We summarized the program evaluation data descriptively, using percentages for categorical data. Two of the authors (ISZ and ES) and a PPOC intern independently reviewed participant responses to open-ended questions and marked and categorized key words and phrases to generate initial themes. Initial themes were then reviewed and final key themes were generated by group consensus. The study was deemed quality improvement and therefore exempt from review by the BCH Institutional Review Board.
Results Six of the 12 (50%) practices were represented at each session. Two of the PPOC practices attended only the first session prior to disenrolling. In addition to BCH and PPOC administrative, safety, and quality staff, 11 to 19 physicians, 1 to 5 nurses and advanced practice nurses, and 2 to 4 practice staff (2 office practice managers, 2 medical home care coordinators), attended each session. Because of increased interest within the participating practices, other practice members also attended some sessions resulting in a variation of participant numbers. Of those enrolled and with planned attendance, session participation was 95% (n = 19/20) for session 1; 73% (n = 11/15) for session 2; 82% (n = 18/22) for session 3; and 90% (n = 18/20) for session 4. Additionally, there were last minute, drop-in participants (between 2 and 4 at each session) from already enrolled practices, which increased attendance to 24 at sessions 1 and 3. A total of 59 CME credits and 19 CEs were granted. Overall, participants rated the sessions highly. The proportions of participants who rated the sessions as “Excellent” were greatest for sessions 1 and 2 (81% and 91%, respectively) and lowest for sessions 3 and 4 (60% and 67%, respectively). Participants reported improvements in knowledge and skills (level 2 outcomes) (Table 2). Themes and selected comments from the 3 open-ended questions reflecting level 1 outcomes (reaction) are presented in Table 3. Participants reported that anticipated changes to clinical practice would center on documentation and communication practices, improved understanding of current practice patterns, in-office safety culture, active surveillance and follow-up of test results,
and accessibility of information for sharing across the care continuum. The cited barriers to changing current practice included culture shift, general resistance to change, absence of shared responsibility and accountability, lack of information sharing, inconsistent communication, and limited resources. Participants understood the importance of effective communication and the need for highly reliable processes for surveillance and tracking of referrals, tests, and imaging results. Optional improvement projects (behavior; level 3 outcomes) undertaken by the participants addressed a variety of topics, including active surveillance and follow-up of newborn screen results, referral and test result management process redesign, formation of a patient safety council to actively review best practices, active tracking of patients referred for emergency care, and distribution of patient education tools such as “Ask Me 3” from the National Patient Safety Foundation.19 Selected improvement projects were shared during the didactic sessions with open discussion of successes and challenges with implementation and measurement. Practices were working across the care continuum and engaging with the emergency department at BCH to improve closed loop communication throughout the referral and discharge processes.
Discussion We developed and implemented a PS curriculum for pediatric primary care clinicians and office staff. Overall, the curriculum was highly rated, suggesting that it was an acceptable, well-received, and relevant modality for delivering PS content. We engaged the learners by relating the course material to participants’ experiences through actual malpractice cases, facilitating self-reflection on current practice, and using needs assessment surveys to tailor the curriculum. Nonclinical staff (medical home care coordinators and practice managers) participated in the sessions and appeared eager to learn new concepts and implementation tools. Providing networking opportunities helped create a sense of community and encouraged the sharing of safety stories, which is critical to creating a culture of safety. We were highly encouraged by the results of this initial curriculum, particularly with respect to participant reactionstoward the curriculum, self-reported changes in knowledge and skills, and the variety of voluntary improvement projects undertaken.
Lessons Learned Although this first iteration of the curriculum was well received, we identified opportunities for improvement. Based on participant feedback regarding difficulties
Downloaded from cpj.sagepub.com at CALIFORNIA STATE UNIV FRESNO on July 26, 2015
5
Zenlea et al Table 2. Participants’ Self-Assessment of Knowledge and Skills (Completed After Each Session).
Participants Who Responded ‘‘Definitely Agree” or “Agree”a; n (%) Session 1b I can identify systems-based issues contributing to malpractice claims in the outpatient setting I can identify major allegations driving outpatient malpractice claims specific to pediatrics I am able to list strategies my practice and colleagues can implement to improve patient safety in the outpatient setting for patients with abdominal pain Session 2c I can identify strategies to improve patient safety as related to referral management I understand the implications of error prone processes in my practice I can identify barriers to best practice Session 3d I am able to use the Swiss Cheese Model to articulate and explain potential breaches of care I am able to demonstrate at least one best practice for closing the loop of communication regarding a referral process I am able to demonstrate at least one best practice for closing the loop of communication regarding test results Session 4e I am better informed on how to drive improvement based on information gathered from complaint management data I am better able to describe what constitutes complaints, grievances, near misses, and safety events I am clear about which complaint and events should be shared with a patient safety manager
24 (100) 22 (91.7) 21 (87.5) 17 (100) 17 (100) 17 (100) 18 (100) 18 (100) 17 (94.4) 15 (93.8) 14 (87.5) 14 (87.5)
a
Respondents rated each of these items using a 4-point Likert-type scale (do not agree, somewhat agree, agree, definitely agree). N = 24 responses. c N = 18 responses. d N = 17 responses. e N = 16 responses. b
with implementing improvement projects, in future versions of the curriculum, we would incorporate change management and leadership methods to advance participants’ skills to address general staff resistance to change and perceived inequality in roles, responsibility, and accountability. Additionally, we better appreciate the need for ongoing project support at regular intervals and plan to offer intervening webinars to provide the requisite expertise for mentored implementation. With this infrastructure in place, we could take a staged approach to project implementation using Plan, Do, Study, Actcycles.17 Such a platform would also allow us to better capture results at the highest levels of Kirkpatrick’s learning evaluation model (behaviors and clinical and patient outcomes).18 In accordance with adult learning principles, we would incorporate more opportunities for self-reflection20 through specific survey items and role modeling by inviting presentations by past participants. We also noted a decline in session attendance, particularly for sessions 2 (July 2013) and 3 (November
2013), which was likely due to participant scheduling constraints (eg, vacation travel, close to holidays, family obligations etc). The 2 PPOC practices that discontinued were composed of solo practitioners whose ongoing practice obligations and commitments disallowed consistent involvement. To address logistical factors, we would adjust the timing of the sessions to avoid potential conflicts (eg, hold sessions in the morning rather than the evening and avoid holidays or medical conference dates). Additionally, we recognize an opportunity to further incentivize physician participation by offering American Board of Pediatrics (ABP) MOC Part 4 credit21 for improvement projects. Furthermore, sessions 3 and 4 were less highly rated than the preceding sessions. Participant feedback for these sessions centered on the lack of detail and unclear relevance of the case presentations. Although useful for illustrating broad PS themes, closed malpractice cases typically lack certain levels of clinical data, and several participants commented in the feedback questionnaires that more clinical
Downloaded from cpj.sagepub.com at CALIFORNIA STATE UNIV FRESNO on July 26, 2015
6
Clinical Pediatrics
Table 3. Themes From Postsession Evaluations. Question What changes to your clinical practice would you make?
Theme Documentation Communication
Information accessibility and sharing Active surveillance, tracking, and follow-up Evaluation of current process
Culture What barriers do you anticipate encountering as you make changes in your practice?
General resistance Lack of shared responsibility and accountability Culture
Lack of information sharing Lack of resources Inconsistent communication What were the two most valuable “take-aways” from this meeting?”
Communication Systems and processes for surveillance, tracking, or follow-up Reconciliation
detail would have increased the effectiveness of the presentations. To address the lack of detail, as the PPOC PS program expands and a robust, peer protected eventreporting system is established, we would use near miss and safety event reports for case presentations.
Limitations Our study has several limitations. This was the first iteration of a curriculum with data obtained from only a small number of attendees. Ongoing program evaluation
Example “We are going to take a close look at the documentation and delivery of patient instructions” “In cases where there is diagnostic uncertainty there needs to be a tight communication plan . . . Continue to reinforce to our staff that follow up calls to parents and patients are important and their work may be at some time critical” “I believe we need to have the information available for patients and parents to access more readily, perhaps revising our website information” “Use known Pediatric specialists, and ensure patient goes to see them, by having staff call for F/U and to encourage patient to keep [appointment], if specialist note not received in reasonable period of time” “We are planning to process map our current referrals process to identify the Swiss cheese moments . . . and would very much like the opportunity to employ best practices as shown in the process maps of high performing practices last night” “Continue promoting no shame/ blame atmosphere in our office, for the sake of patient care and safety” “Staff reluctance to add to their workflow” “Accountability of everyone in the communications loop is essential . . .” “Is difficult to change what has not been a culture of effective communication in the past . . . Long-standing and ingrained ‘head in the sand ethos that if we don’t acknowledge it, it must not exist’” “Labs and consultants not connected to [electronic health record] need a separate system and as such have greater potential for error in referral management” “Inadequate support staff to call patients who miss appointments” “Communication from specialists/ER/patients is not consistent” “Importance of communicating concerns to [emergency room] personnel when referring our patients” “Lack of standardized process for tracking newborn screening as well as other lab results” “Closing the loop on patient labs and referrals can improve outcomes”
by participants from the next phase of our curriculum is necessary to confirm our findings. We developed the postsession evaluations de novo and did not have the opportunity for validation. Therefore, participants may not have interpreted the questions as we intended and differently from each other. Additionally, we relied on self-assessed changes in knowledge and did not formally evaluate individual participant performance. Although we verified the initial undertaking of QI projects (level 3 outcomes; behavior) through onsite visits and invited presentations at the educational sessions, we
Downloaded from cpj.sagepub.com at CALIFORNIA STATE UNIV FRESNO on July 26, 2015
7
Zenlea et al have not assessed project sustainability other than by emailed follow-up. Given that project completion was voluntary, we did not require practices to report patientlevel or clinical data. Therefore, we were unable to capture level 4 outcomes (results) for the projects.
of the Harvard Medical Institutions, Inc. (CRICO). Dr. Tess is the Course Director for the Quality and Safety Educators Academy at the Society for Hospital Medicine and the Academic Alliance for Internal Medicine. The other authors have no financial disclosures or conflicts of interest.
Funding
Conclusions We successfully implemented a multifaceted, interprofessional PS curriculum in the pediatric primary care setting. Grounding the teachings in clinical microsystems imparted participants with the knowledge to recognize a safety event, near miss, or potential risk to PS and begin to analyze and enact change. Although curriculum development and session planning was time intensive, we now have a packaged curriculum that can serve as a shared resource for participants and for further dissemination. This is particularly appealing in the context of the collaboration between the PPOC and BCH to establish a comprehensive ambulatory PS program. Although this was a single institutional experience, we believe that our work can inform other ambulatory risk management programs and patient safety educational interventions targeted at practicing clinicians and office practice staff. Acknowledgments The authors wish to thank the Boston Children’s Hospital Program for Patient Safety and Quality and the Pediatric Physician’s Organization at Children’s for supporting this work. The authors wish to thank Elizabeth Lambert, MEd, for her administrative support, Nicole Pelletier, MPH for assistance with compiling and analyzing the survey data, and Nina Rauscher, MS, RN, CPHQ, for her support and thoughtful review of our manuscript. We thank Jane Gagne for coordinating the Risk Management CME credits and Gregory Durkin, MEd, RN-BC, for coordinating the CE units. We thank Graham T. McMahon, MBBCh, MD, MMSc, for his thoughtful comments and review of the manuscript.
Author Contributions ISZ, ES, BS, AT, JS, LS, KJJ, and GF contributed to the project conception or design; ISZ, ES, BS, LS, and GF contributed to data acquisition, analysis, or intrepretation; ISZ drafted the manuscript; ES, BS, AT, JS, LS, KJJ, and GF critically revised the manuscript; GF gave final approval of the manuscript; ISZ and GF agree to be accountable for all aspects of the work ensuring integrity and accuracy.
Declaration of Conflicting Interests The author(s) declared the following potential conflicts of interest with respect to the research, authorship, and/or publication of this article: Dr. Focht is a Safety & Operations Consultant for Ingenious Med and an active member of the Ambulatory Care Risk Group at Risk Management Foundation
The author(s) disclosed receipt of the following financial support for the research, authorship, and/or publication of this article: This project was supported by Risk Management Foundation of the Harvard Medical Institutions, Inc. (CRICO) through the Ambulatory Risk Management Grant to Boston Children’s Hospital and the Harvard Medical School Fellowship in Patient Safety and Quality.
References 1. Davis NL, Davis DA, Johnson NM, et al. Aligning academic continuing medical education with quality improvement: a model for the 21st century. Acad Med. 2013;88:1437-1441. 2. Accreditation Council for General Medical Education. Next Accreditation System (NAS). 2013.http://www.acgme.org/ acgmeweb/tabid/435/ProgramandInstitutionalAccreditation/ NextAccreditationSystem.aspx. Accessed November 27, 2013. 3. American Board of Medical Specialties. MOC Competencies and Criteria. 2014. http://www.abms.org/ Maintenance_of_Certification/MOC_competencies.aspx. Accessed May 21, 2014. 4. Massachusetts Medical Society. Continuing Medical Education Requirements for Physician License Renewal in Massachusetts. 2012. http://www.massmed.org/ Continuing-Education-and-Events/Continuing-MedicalEducation-Requirements-for-Physician-LicenseRenewal-in-Massachusetts/. Accessed May 20, 2014. 5. Boonyasai RT, Windish DM, Chakraborti C, Feldman LS, Rubin HR, Bass EB. Effectiveness of teaching quality improvement to clinicians: a systematic review. JAMA. 2007;298:1023-1037. 6. Margolis PA, Lannon CM, Stuart JM, Fried BJ, KeyesElstein L, Moore DE Jr. Practice based education to improve delivery systems for prevention in primary care: randomised trial. BMJ. 2004;328:388. 7. Rosenthal MS, Lannon CM, Stuart JM, Brown L, Miller WC, Margolis PA. A randomized trial of practice-based education to improve delivery systems for anticipatory guidance. Arch Pediatr Adolesc Med. 2005;159:456-463. 8. Homer CJ, Forbes P, Horvitz L, Peterson LE, Wypij D, Heinrich P. Impact of a quality improvement program on care and outcomes for children with asthma. Arch Pediatr Adolesc Med. 2005;159:464-469. 9. Vernacchio L, Francis ME, Epstein DM, et al. Effectiveness of an asthma quality improvement program designed for maintenance of certification. Pediatrics. 2014;134:e242-e248. 10. Vernacchio L, Trudell EK, Hresko MT, Karlin LI, Risko W. A quality improvement program to reduce
Downloaded from cpj.sagepub.com at CALIFORNIA STATE UNIV FRESNO on July 26, 2015
8
Clinical Pediatrics
unnecessary referrals for adolescent scoliosis. Pediatrics. 2013;131:e912-e920. 11. Mass.gov Health and Human Services. Continuing Education Information. 2014. http://www.mass.gov/ eohhs/gov/departments/dph/programs/hcq/dhpl/nursing/ faq/continuing-education-.html. Accessed May 20, 2014. 12. Neuspiel DR, Stubbs EH. Patient safety in ambulatory care. Pediatr Clin North Am. 2012;59:1341-1354. 13. Miller MR, Pronovost PJ, Burstin HR. Pediatric patient safety in the ambulatory setting. Ambul Pediatr. 2004;4:47-54. 14. Gandhi TK, Lee TH. Patient safety beyond the hospital. N Engl J Med. 2010;363:1001-1003. 15. Reason J. Human error: models and management. BMJ. 2000;320:768-770. 16. Quality by Design: A Clinical Microsystems Approach. 1st ed. San Francisco, CA: Jossey-Bass; 2007.
17. Langley GJ, Ronald M, Nolan KM, Nolan TW, Norman CL, Provost LP. The Improvement Guide: A Practical Approach to Enhancing Organizational Performance. San Francisco, CA: Jossey-Bass; 2009. 18. Kirkpatrick DL, Kirkpatrick JD. Evaluating Training Programs: The Four Levels. 3rd ed. San Francisco, CA: Berrett-Koehler; 2006. 19. National Patient Safety Foundation. Ask Me 3. 2014. http:// www.npsf.org/for-healthcare-professionals/programs/ ask-me-3/. Accessed June 16, 2014. 20. Wittich CM, Reed DA, Ting HH, et al. Measuring reflection on participation in quality improvement activities for maintenance of certification. Acad Med. 2014;89: 1392-1397. 21. American Board of Pediatrics. Performance in Practice (Part 4). 2015. https://www.abp.org/content/performancepractice-part-4. Accessed April 28, 2015.
Downloaded from cpj.sagepub.com at CALIFORNIA STATE UNIV FRESNO on July 26, 2015
![[Patient safety in primary care: PREFASEG project].](/assets/img/hold.png)
