Vol. 0 No. 0 Month 2015
1
Using Lean to Advance Quality Improvement Research Christopher Craig Blackmore, Barbara L. Williams, Joan M. Ching, Lynne A. Chafetz, Gary S. Kaplan
Introduction Awareness of the need for quality improvement in healthcare has risen dramatically since the Institute of Medicine report of 2001 called urgently for redesign of healthcare to address overwhelming quality and safety deficiencies (Hurtado et al., 2001). Unfortunately, research on the effectiveness of quality improvement has not always kept pace with improvement innovation (Auerbach et al., 2007). The insufficiency of quality improvement research could potentially slowdown progress in improvement, both by enabling diffusion of ineffective interventions and slowing down rapid diffusion of interventions that are effective. Furthermore, despite commonality between clinical and quality improvement research, the complex systems on which quality improvement interventions act require additional research methods, derived from the social sciences and engineering (Berwick, 2008). However, research methodology is not generally a component of quality improvement training, and there are only a limited number of individuals with both quality improvement and research methodological expertise. Hence, most quality improvement interventions are performed without the possibility for research on their effectiveness. Furthermore, although academic medical centers commonly provide education and infrastructure supporting clinical and basic science research, quality improvement research remains relatively underdeveloped. At our institution, the Lean methods serve as the foundation for all management and quality improvement activities. We successfully adapted the Toyota Production System for healthcare, providing the framework not simply for quality improvement but as an overall management strategy
Introduction: Quality improvement research skills are not commonplace among quality improvement practitioners, and research on the effectiveness of quality improvement has not always kept pace with improvement innovation. However, the Lean tools applied to quality improvement should be equally relevant to the advancement of quality improvement research. Methods: We applied the Lean methods to develop a simplified quality improvement publication pathway enabling a small research methodology group to increase quality improvement research throughout the institution. The key innovations of the pathway are horizontal integration of the quality improvement research methods group across the institution, implementation of a Lean quality improvement research pathway, and application of a just-in-time quality improvement research toolkit. Results: This work provides a road map and tools for the acceleration of quality improvement research. At our institution, the Lean quality improvement research approach was associated with statistically significant increases in the number (annual mean increase from 3.0 to 8.5, p = .03) and breadth of published quality improvement research articles, and in the number of quality improvement research projects currently in process. Discussion: Application of Lean methods to the quality improvement research process can aid in increasing publication of quality improvement articles from across the institution.
(Kaplan et al., 2014, Kenney, 2011). However, despite recognition for its Lean quality improvement efforts, our institution had published little quality improvement research. Accordingly, in 2012, we founded the Center for Health Services Research (CHSR), making an institutional commitment to provide methodological support and mentorship for an expansion of quality improvement research. We hypothesized that the Lean model we deployed through the CHSR would result in increased number and breadth
Keywords quality improvement Lean research
Journal for Healthcare Quality Vol. 0, No. 0, pp. 1–8 © 2015 National Association for Healthcare Quality
2
Journal for Healthcare Quality
of quality improvement publications (QuIPs), extending beyond traditional academic departments, to quality improvement teams from throughout the health system.
Methods This project consisted of development and assessment of the effectiveness of the CHSR.
Developing the CHSR Intervention The original framework for the CHSR came out of a Kaizen Event (Lean 2 day quality improvement event) in 2010, charged with increasing the output of academic QuIPs. Part of that event included a root cause analysis exercise using Ishikawa charts. The main barriers to publishing quality improvement research we identified related to the lack of research training and experience, and time among the quality improvement practitioners. We identified specific steps in the process that represented challenges to quality improvement specialists and developed tools to address those barriers. The CHSR was founded in 2012, based on the needs identified in the 2010 Kaizen Event, providing the staff necessary to support the QuIP process. With only a small team and a mandate to support the entire institution, we used Lean quality improvement methods to attempt to make the research process as efficient as possible and enable participation by the broad range of quality improvement specialists. First, the CHSR was horizontally integrated throughout the entire institution rather than in vertical silos, restructuring the research enterprise around a central methodological core and partnering with the quality improvement teams who can focus on improvement work without having to learn a new skill set in research methodology. Second, we developed and implemented a standardized Lean QuIP pathway for researchers to proceed from quality improvement idea to publication. Third, we have developed a quality improvement research toolkit supporting researchers at key points along the QuIP
pathway and making performance of research more accessible to quality improvement practitioners.
Center for Health Services Research Traditional medical research models concentrate research programs among academic experts, with success in clinical and basic science research. However, such models limit the scope of research to the content areas or “silos” of those experts. Because successful quality improvement efforts are by nature spread throughout all aspects of healthcare delivery, we centralize the quality improvement research infrastructure in a methodological core, the CHSR (Fig. 1). The CHSR design enables QI research from throughout the institution, including physician departments, nursing, hospital operations and support services, executive leadership, and the Lean methods and education unit without requiring the large number of Lean quality improvement specialists to also train in research. In effect, the research methodology support from the CHSR is available when needed, using the Lean concept of “just-in-time” (Ohno, 1988). This approach aligns the skills and tasks of both researchers and improvement specialists. The role of the CHSR is not limited to statistical support or research design as in many traditional clinical research centers. Instead, the CHSR can aid any aspects of quality improvement research, including research design, data collection, data analysis, and article preparation and editing. Quality improvement teams led by experienced clinical researchers may require only minimal support to perform QI research, whereas teams with limited research experience may require extensive input from the CHSR to achieve the goal of publication. The CHSR is supported by the institution as central to the Virginia Mason vision to transform healthcare. Dedicated resources for the CHSR included 0.5 full time equivalent (FTE) allocation for the physician/researcher who directs the CHSR, 1.0 FTE PhD-level research scientist, and 0.5 FTE for a data analyst/computer programmer. No specific
Vol. 0 No. 0 Month 2015
Figure 1. Organization of the Center for Health Services Research (CHSR) within the institution and range of research projects.
incentives were provided to quality improvement specialists to publish articles.
Standardized Pathway Quality improvement research overall is a relatively consistent process of investigating the effect of interventions, regardless of the specific topic. Accordingly, the Lean principles we are applying to improve quality and safety throughout the medical center are equally relevant to the
quality improvement research enterprise. Under the Lean process, we first mapped out each progressive step in the QuIP process. From this, we identified specific points in the QuIP process where there is waste, meaning delay, unnecessary rework, errors, or outright failure of the project to come to completion. The Lean process then focuses on identifying causes and corresponding solutions to the waste. From this, we developed a final QuIP Lean pathway (Fig. 2), a model for the efficient
3
4
Journal for Healthcare Quality
Figure 2. Standardized quality improvement publication (QuIP) pathway.
performance of quality improvement research. The QuIP Lean pathway identifies where quality improvement practitioners are likely to need research assistance, leading us to develop the tools detailed below to address those barriers. Standardization of the QI research process through the QuIP also allows us to track success through intermediate metrics at different phases of the process.
Tools The CHSR uses a Lean toolkit of practical and simple aids to facilitate the QuIP process. Among the CHSR tools is the QuIP Worksheet, which was developed in conjunction with the institutional review board (IRB), and serves as a standard template for application for exemption from IRB review (see Supplemental Digital Content 1, http://links.lww.com/JHQ/A3). Although quality improvement research generally does not require IRB approval, discerning which research does requires IRB oversight can be challenging. Accordingly, our IRB
considers every QI project with potential for publication, although most projects are determined to not require formal review. In addition, the QuIP Worksheet is submitted to our medical librarians who support quality improvement project owners by performing searches of the relevant academic literature. Other QuIP tools we have established include an article writing template (see Supplemental Digital Content 2, http://links.lww.com/JHQ/A4). A major challenge for beginning researchers is to translate their understanding of a project into the written structure of a research report. This template is the initial important step, specifically defining the content for each paragraph in the first draft. The SQUIRE guidelines (SQUIRE, 2008) are then used to expand the early draft into a full research article. We have also developed the I-PISO tool adapted from evidence-based medicine to help quality improvement practitioners frame an improvement project as research and to define which quality improvement projects
Vol. 0 No. 0 Month 2015
are worth pursuing as research endeavors (see Supplemental Digital Content 3, http://links.lww.com/JHQ/A5). A consistent challenge we have identified in QI research is the need for reliable effectiveness data. Specifically, investigators would often struggle to identify what to measure for quality improvement and how to put effectiveness measurement into the larger context of healthcare. Accordingly, we have adapted a model from outside of medicine called the Chinook effectiveness monitoring framework (Crawford and Rumsey, 2011), a simple approach to identifying what to measure to understand the effectiveness of an intervention. This Chinook framework identifies three levels of monitoring (Fig. 3). The highest level, the problem of interest,
focuses on the current state and any changes in the main patient relevant outcome (i.e., mortality, quality of life, length of stay). Since individual quality improvement events may have very small or delayed impacts on such high-level outcomes, second-level intermediate or process outcomes such as physiological markers (e.g., HbA1C, blood pressure), utilization metrics (e.g., colon cancer screening rates, appropriate imaging), and service markers (e.g., same day access, dropped calls) are required to track the effectiveness of each individual improvement event. Finally, because success requires not only solutions but also implementation, the third level of monitoring is implementation. As improvements are designed by quality improvement teams, implementation is
Figure 3. Chinook model for effectiveness monitoring.
5
6
Journal for Healthcare Quality
measured to both confirm that changes in outcome are temporally related to the intervention and to identify areas for further work. All levels of metrics under the Chinook model are then measured using statistical process control charts or other methods. By invoking this framework early in the improvement planning process, we improve data reliability for both the QI projects themselves and for subsequent publication.
Methods of Evaluation To understand the effectiveness of the CHSR model and toolkit, we monitor the program using the same approach we would for any other quality improvement intervention. Under the Chinook model, the main (level 1) outcome of interest is the number of articles published. This we determined yearly, with a single investigator assessing the number of institutional QI publications, through review of titles (and abstract and papers where necessary) of all institutional publications in PubMed. Additional searches were made in PubMed for articles by known institutional QI authors. Results were analyzed from 2006 to 2013 before and after initiation of the CHSR in 2012, using the t test in STATA 12 (STATA Corp, College Station, TX). We also qualitatively monitor the breadth of publications, assessing the source of QI publications, and whether they are from
Figure 4. Change in number of quality improvement publications and number of works-inprogress from 2006 to 2013.
physician departments, nursing, hospital operations, executive leadership, or the Lean methods and education unit. Although our overall goal is publication, the number of articles published is not a sensitive metric to immediate changes that we make to improve our program. Publication of articles is subject to long delays and variability in the time from submission to publication. Hence, as an intermediate metric (Chinook model level 2), we track number of QuIP Worksheets submitted to the IRB. This is not a perfect surrogate for publications, as not all will be published, but does serve as a more sensitive intermediate metric. Finally, implementation (Chinook level 3) is also tracked as the number of projects using the various QuIP tools.
Results With initiation of the CHSR in 2012, there was substantial increase in the number of QI research publications, from an average of 3.0 per year at baseline to 8.5 after founding of the CHSR (p = .03). Since the origination of the QuIP process in 2010, there has been a significant increase in QuIP Worksheet submissions every year, with now over 30 active projects (Fig. 4). The CHSR mission is to support quality improvement research throughout the institution, and the diversity of publications reflects the success in reaching the broad base of institutional QI initiatives (Fig. 1). Successful publications range from traditional physician departments including anesthesia (Porter 2014), medicine (Blackmore et al., 2013b), radiology (Hui et al., 2014), and nursing (Ching et al., 2013, Dewing et al., 2013), to traditionally nonacademic teams including hospital operations and support services (Blackmore et al., 2013a, Farrokhi et al., 2013), executive leadership (Kaplan et al., 2014, Calderon et al., 2014), and Lean methods and education (Russ et al., 2013, Ching et al., 2014).
Discussion Lean quality improvement methods can also be applied to improve quality improvement
Vol. 0 No. 0 Month 2015
research. Our increasing number and variety of QuIPs speaks to the success of this model. Through a small methodological core group and a relatively simple Lean toolkit, we enable successful publication of a broad range of QI projects originating throughout the institution. Key lessons include (1) horizontal integration across the entire institution, including formal collaborations with executive leadership, the clinical data systems group, and QI teams; (2) standardization of the QI research process by applying the tools of Lean production; and (3) implementation of a toolkit to enable investigators to overcome barriers in producing quality improvement research publications. The results we report are from a single institution, and we cannot claim to have perfected the approach or the tools. Furthermore, our simplistic analysis of the effectiveness of the program is meant as an illustration, not as a comprehensive program evaluation. Customer satisfaction data in particular would be valuable to help understand and improve the program. However, our program does highlight the potential value of viewing the research enterprise as another process to which quality improvement methods can be applied, an approach that we expect would have value throughout healthcare. In addition, other ways in which the CHSR may contribute to the institution are not captured in the analysis. Through working with the CHSR, staff members from throughout the institution have gained greater skill in determining the effectiveness of quality improvement interventions. Also, the quantitative methods used by the CHSR have been incorporated into institutional Lean training. We note that our objective in this report was not to compare different models for increasing research productivity. An alternate approach would have been for us to fund a research institute typical for many academic institutions. This may also have led to success in increasing research productivity. However, institutes are typically vertical in orientation with a specific research focus and concentrated content
knowledge in that arena. Our aim was not simply to establish a research program and produce publications in one targeted area (like iatrogenic infections or falls). Rather, our goal was to support staff and enable research on the broad range of quality improvement work being performed around the institution. Hence, success is not simply the number of articles but also the breadth of reach from the CHSR in publishing work from all areas. Although there have been broad calls for more research on quality improvement (Berwick, 2008, Wachter 2010, Davidoff et al., 2008) and rigorous debate on the methodology for quality improvement research (Auerbach et al., 2007, Shojania et al., 2002, Berwick, 2008), there is little other published literature on improving the efficiency of quality improvement research. Holzmueller and Pronovost (2013) developed a standard template for article writing, although its effectiveness has not been demonstrated. Through our Lean approach, we attempt to address inefficiencies in both the organization of quality research infrastructure and the process of performing quality improvement research projects. Advancing quality improvement requires rigorous research on the effectiveness of proposed interventions. In this work, we demonstrate how the tools of quality improvement themselves can be applied to the quality improvement research enterprise, enabling efficient and successful QI research from traditional academic departments, but also nontraditional areas including hospital operations and support services, and executive leadership. We believe that this approach can help drive the evidence base necessary to allow quality improvement to reach its potential for addressing the cost and quality challenges facing healthcare.
References Auerbach, AD, Landefeld, CS, & Shojania, KG. The tension between needing to improve care and knowing how to do it. N Engl J Med 2007;357:608–613.
7
8
Journal for Healthcare Quality
Berwick, DM. The science of improvement. JAMA 2008;299:1182–1184. Blackmore, CC, Bishop, R, Luker, S, & Williams, BL. Applying lean methods to improve quality and safety in surgical sterile instrument processing. Jt Comm J Qual Patient Safe 2013a;39:99–105. Blackmore, CC, Edwards, JW, & Searles, C, et al. Nurse practitioner-staffed clinic at Virginia Mason improves care and lowers costs for women with benign breast conditions. Health Aff (millwood) 2013b;32:20–26. Calderon, AS, Blackmore, CC, & Williams, BL, et al. Transforming ward rounds through rounding-in-flow. J Grad Med Educ 2014;4: 750–755. Ching, JM, Long, C, Williams, BL, & Blackmore, CC. Using lean to improve medication administration safety. Jt Comm J Qual Patient Safe 2013;39:199–204. Ching, JM, Williams, BL, Idemoto, LM, & Blackmore, CC. Using lean automation with a human touch to improve medication safety: a step closer to the perfect dose. Jt Comm J Qual Patient Safe 2014;40:341–350. Crawford, BA, & Rumsey, SM. Guidance for Monitoring Recovery of Pacific Northwest Salmon and Steelhead Listed Under the Endangered Species Act. Washington, DC: National Marine Fisheries Service. Davidoff, F, Batalden, P, & Stevens, D, et al. Publication guidelines for quality improvement in health care: evolution of the SQUIRE project. Qual Saf Health Care 2008;17:i3–i9. Dewing, K, Belza, B, Zierler, B, & LaCroix, AZ. Health maintenance model improves BMD testing. Nurse Pract 2013;38:37–41. Farrokhi, F, Gunther, M, Williams, BL, & Blackmore, CC. Application of lean methodology for improved quality and efficiency in operating room instrument availability. J Healthc Qual Sep 24: 1–10 (epub ahead of print). Holzmueller, CG, & Pronovost, P. Organising a manuscript reporting quality improvement or patient safety research. BMJ Qual Saf 2013;22:777–785. Hui, JS, Kramer, DJ, & Blackmore, CC, et al. A quality improvement initiative to reduce unnecessary follow-up for imaging for adnexal lesions. J Am Coll Radiol 2014;11:373–377. Hurtado, MP, Swift, EK, & Corrigan, JM. Crossing the Quality Chasm: A New Health Care System for the 21st Century. Washington, DC: National Academy Press; 2001. Kaplan, GS, Patterson, SH, Ching, JM, & Blackmore, CC. Why Lean doesn’t work for everyone. BMJ Qual Saf 2014;23:970–973.
Kenney, C. Transforming Health Care: Virginia Mason Medical Center’s Pursuit of the Perfect Patient Experience New York, NY: Productivity Press—Taylor & Francis Group; 2011. Porter, AJ, Narimasu, JY, Mulroy, MF, & Koehler, RP. Sustainable, effective implementation of a surgical preprocedural checklist: an “attestation” format for all operating team members. Jt Comm J Qual Patient Saf 2014;40:3–9. Russ, LR, Phillips, J, & Brzozowicz, K, et al. Experience-based design for integrating the patient care experience into healthcare improvement: identifying a set of reliable emotion words. Healthcare 2013;1:91–99. Shojania, KG, Duncan, BW, McDonald, KM, & Wachter, RM. Safe but not sound: patient safety meets evidence-based medicine. JAMA 2002;288:508–513. SQUIRE: Standards for Quality Improvement Reporting Excellence. 2008. Available at: http:// squire-statement.org/assets/pdfs/SQUIRE_ guidelines_table.pdf. Accessed May 15, 2014. Wachter, RM. Patient safety at ten: unmistakable progress, troubling gaps. Health Aff (Millwood) 2010;29:165–173.
Authors’ Biographies Christopher Craig Blackmore, MD, MPH, is the Director of the Center for Health Services Research at Virginia Mason, Seattle, WA. Barbara L. Williams, PhD, is Research Scientist at the Center for Health Services Research at Virginia Mason. Joan M. Ching, RN, MN, is Nursing Director for Quality and Safety. Lynne A. Chafetz, JD, is Senior Vice President and General Counsel, Virginia Mason. Gary S. Kaplan, MD, is President and Chief Executive Officer, Virginia Mason. For more information on this article, contact C. Craig Blackmore at [email protected]. Supplemental digital content is available for this article. Direct URL citations appear in the printed text and in the HTML and PDF versions of the article at www.jhqonline.com. The authors declare no conflict of interest.
1
Using Lean to Advance Quality Improvement Research Christopher Craig Blackmore, Barbara L. Williams, Joan M. Ching, Lynne A. Chafetz, Gary S. Kaplan
Introduction Awareness of the need for quality improvement in healthcare has risen dramatically since the Institute of Medicine report of 2001 called urgently for redesign of healthcare to address overwhelming quality and safety deficiencies (Hurtado et al., 2001). Unfortunately, research on the effectiveness of quality improvement has not always kept pace with improvement innovation (Auerbach et al., 2007). The insufficiency of quality improvement research could potentially slowdown progress in improvement, both by enabling diffusion of ineffective interventions and slowing down rapid diffusion of interventions that are effective. Furthermore, despite commonality between clinical and quality improvement research, the complex systems on which quality improvement interventions act require additional research methods, derived from the social sciences and engineering (Berwick, 2008). However, research methodology is not generally a component of quality improvement training, and there are only a limited number of individuals with both quality improvement and research methodological expertise. Hence, most quality improvement interventions are performed without the possibility for research on their effectiveness. Furthermore, although academic medical centers commonly provide education and infrastructure supporting clinical and basic science research, quality improvement research remains relatively underdeveloped. At our institution, the Lean methods serve as the foundation for all management and quality improvement activities. We successfully adapted the Toyota Production System for healthcare, providing the framework not simply for quality improvement but as an overall management strategy
Introduction: Quality improvement research skills are not commonplace among quality improvement practitioners, and research on the effectiveness of quality improvement has not always kept pace with improvement innovation. However, the Lean tools applied to quality improvement should be equally relevant to the advancement of quality improvement research. Methods: We applied the Lean methods to develop a simplified quality improvement publication pathway enabling a small research methodology group to increase quality improvement research throughout the institution. The key innovations of the pathway are horizontal integration of the quality improvement research methods group across the institution, implementation of a Lean quality improvement research pathway, and application of a just-in-time quality improvement research toolkit. Results: This work provides a road map and tools for the acceleration of quality improvement research. At our institution, the Lean quality improvement research approach was associated with statistically significant increases in the number (annual mean increase from 3.0 to 8.5, p = .03) and breadth of published quality improvement research articles, and in the number of quality improvement research projects currently in process. Discussion: Application of Lean methods to the quality improvement research process can aid in increasing publication of quality improvement articles from across the institution.
(Kaplan et al., 2014, Kenney, 2011). However, despite recognition for its Lean quality improvement efforts, our institution had published little quality improvement research. Accordingly, in 2012, we founded the Center for Health Services Research (CHSR), making an institutional commitment to provide methodological support and mentorship for an expansion of quality improvement research. We hypothesized that the Lean model we deployed through the CHSR would result in increased number and breadth
Keywords quality improvement Lean research
Journal for Healthcare Quality Vol. 0, No. 0, pp. 1–8 © 2015 National Association for Healthcare Quality
2
Journal for Healthcare Quality
of quality improvement publications (QuIPs), extending beyond traditional academic departments, to quality improvement teams from throughout the health system.
Methods This project consisted of development and assessment of the effectiveness of the CHSR.
Developing the CHSR Intervention The original framework for the CHSR came out of a Kaizen Event (Lean 2 day quality improvement event) in 2010, charged with increasing the output of academic QuIPs. Part of that event included a root cause analysis exercise using Ishikawa charts. The main barriers to publishing quality improvement research we identified related to the lack of research training and experience, and time among the quality improvement practitioners. We identified specific steps in the process that represented challenges to quality improvement specialists and developed tools to address those barriers. The CHSR was founded in 2012, based on the needs identified in the 2010 Kaizen Event, providing the staff necessary to support the QuIP process. With only a small team and a mandate to support the entire institution, we used Lean quality improvement methods to attempt to make the research process as efficient as possible and enable participation by the broad range of quality improvement specialists. First, the CHSR was horizontally integrated throughout the entire institution rather than in vertical silos, restructuring the research enterprise around a central methodological core and partnering with the quality improvement teams who can focus on improvement work without having to learn a new skill set in research methodology. Second, we developed and implemented a standardized Lean QuIP pathway for researchers to proceed from quality improvement idea to publication. Third, we have developed a quality improvement research toolkit supporting researchers at key points along the QuIP
pathway and making performance of research more accessible to quality improvement practitioners.
Center for Health Services Research Traditional medical research models concentrate research programs among academic experts, with success in clinical and basic science research. However, such models limit the scope of research to the content areas or “silos” of those experts. Because successful quality improvement efforts are by nature spread throughout all aspects of healthcare delivery, we centralize the quality improvement research infrastructure in a methodological core, the CHSR (Fig. 1). The CHSR design enables QI research from throughout the institution, including physician departments, nursing, hospital operations and support services, executive leadership, and the Lean methods and education unit without requiring the large number of Lean quality improvement specialists to also train in research. In effect, the research methodology support from the CHSR is available when needed, using the Lean concept of “just-in-time” (Ohno, 1988). This approach aligns the skills and tasks of both researchers and improvement specialists. The role of the CHSR is not limited to statistical support or research design as in many traditional clinical research centers. Instead, the CHSR can aid any aspects of quality improvement research, including research design, data collection, data analysis, and article preparation and editing. Quality improvement teams led by experienced clinical researchers may require only minimal support to perform QI research, whereas teams with limited research experience may require extensive input from the CHSR to achieve the goal of publication. The CHSR is supported by the institution as central to the Virginia Mason vision to transform healthcare. Dedicated resources for the CHSR included 0.5 full time equivalent (FTE) allocation for the physician/researcher who directs the CHSR, 1.0 FTE PhD-level research scientist, and 0.5 FTE for a data analyst/computer programmer. No specific
Vol. 0 No. 0 Month 2015
Figure 1. Organization of the Center for Health Services Research (CHSR) within the institution and range of research projects.
incentives were provided to quality improvement specialists to publish articles.
Standardized Pathway Quality improvement research overall is a relatively consistent process of investigating the effect of interventions, regardless of the specific topic. Accordingly, the Lean principles we are applying to improve quality and safety throughout the medical center are equally relevant to the
quality improvement research enterprise. Under the Lean process, we first mapped out each progressive step in the QuIP process. From this, we identified specific points in the QuIP process where there is waste, meaning delay, unnecessary rework, errors, or outright failure of the project to come to completion. The Lean process then focuses on identifying causes and corresponding solutions to the waste. From this, we developed a final QuIP Lean pathway (Fig. 2), a model for the efficient
3
4
Journal for Healthcare Quality
Figure 2. Standardized quality improvement publication (QuIP) pathway.
performance of quality improvement research. The QuIP Lean pathway identifies where quality improvement practitioners are likely to need research assistance, leading us to develop the tools detailed below to address those barriers. Standardization of the QI research process through the QuIP also allows us to track success through intermediate metrics at different phases of the process.
Tools The CHSR uses a Lean toolkit of practical and simple aids to facilitate the QuIP process. Among the CHSR tools is the QuIP Worksheet, which was developed in conjunction with the institutional review board (IRB), and serves as a standard template for application for exemption from IRB review (see Supplemental Digital Content 1, http://links.lww.com/JHQ/A3). Although quality improvement research generally does not require IRB approval, discerning which research does requires IRB oversight can be challenging. Accordingly, our IRB
considers every QI project with potential for publication, although most projects are determined to not require formal review. In addition, the QuIP Worksheet is submitted to our medical librarians who support quality improvement project owners by performing searches of the relevant academic literature. Other QuIP tools we have established include an article writing template (see Supplemental Digital Content 2, http://links.lww.com/JHQ/A4). A major challenge for beginning researchers is to translate their understanding of a project into the written structure of a research report. This template is the initial important step, specifically defining the content for each paragraph in the first draft. The SQUIRE guidelines (SQUIRE, 2008) are then used to expand the early draft into a full research article. We have also developed the I-PISO tool adapted from evidence-based medicine to help quality improvement practitioners frame an improvement project as research and to define which quality improvement projects
Vol. 0 No. 0 Month 2015
are worth pursuing as research endeavors (see Supplemental Digital Content 3, http://links.lww.com/JHQ/A5). A consistent challenge we have identified in QI research is the need for reliable effectiveness data. Specifically, investigators would often struggle to identify what to measure for quality improvement and how to put effectiveness measurement into the larger context of healthcare. Accordingly, we have adapted a model from outside of medicine called the Chinook effectiveness monitoring framework (Crawford and Rumsey, 2011), a simple approach to identifying what to measure to understand the effectiveness of an intervention. This Chinook framework identifies three levels of monitoring (Fig. 3). The highest level, the problem of interest,
focuses on the current state and any changes in the main patient relevant outcome (i.e., mortality, quality of life, length of stay). Since individual quality improvement events may have very small or delayed impacts on such high-level outcomes, second-level intermediate or process outcomes such as physiological markers (e.g., HbA1C, blood pressure), utilization metrics (e.g., colon cancer screening rates, appropriate imaging), and service markers (e.g., same day access, dropped calls) are required to track the effectiveness of each individual improvement event. Finally, because success requires not only solutions but also implementation, the third level of monitoring is implementation. As improvements are designed by quality improvement teams, implementation is
Figure 3. Chinook model for effectiveness monitoring.
5
6
Journal for Healthcare Quality
measured to both confirm that changes in outcome are temporally related to the intervention and to identify areas for further work. All levels of metrics under the Chinook model are then measured using statistical process control charts or other methods. By invoking this framework early in the improvement planning process, we improve data reliability for both the QI projects themselves and for subsequent publication.
Methods of Evaluation To understand the effectiveness of the CHSR model and toolkit, we monitor the program using the same approach we would for any other quality improvement intervention. Under the Chinook model, the main (level 1) outcome of interest is the number of articles published. This we determined yearly, with a single investigator assessing the number of institutional QI publications, through review of titles (and abstract and papers where necessary) of all institutional publications in PubMed. Additional searches were made in PubMed for articles by known institutional QI authors. Results were analyzed from 2006 to 2013 before and after initiation of the CHSR in 2012, using the t test in STATA 12 (STATA Corp, College Station, TX). We also qualitatively monitor the breadth of publications, assessing the source of QI publications, and whether they are from
Figure 4. Change in number of quality improvement publications and number of works-inprogress from 2006 to 2013.
physician departments, nursing, hospital operations, executive leadership, or the Lean methods and education unit. Although our overall goal is publication, the number of articles published is not a sensitive metric to immediate changes that we make to improve our program. Publication of articles is subject to long delays and variability in the time from submission to publication. Hence, as an intermediate metric (Chinook model level 2), we track number of QuIP Worksheets submitted to the IRB. This is not a perfect surrogate for publications, as not all will be published, but does serve as a more sensitive intermediate metric. Finally, implementation (Chinook level 3) is also tracked as the number of projects using the various QuIP tools.
Results With initiation of the CHSR in 2012, there was substantial increase in the number of QI research publications, from an average of 3.0 per year at baseline to 8.5 after founding of the CHSR (p = .03). Since the origination of the QuIP process in 2010, there has been a significant increase in QuIP Worksheet submissions every year, with now over 30 active projects (Fig. 4). The CHSR mission is to support quality improvement research throughout the institution, and the diversity of publications reflects the success in reaching the broad base of institutional QI initiatives (Fig. 1). Successful publications range from traditional physician departments including anesthesia (Porter 2014), medicine (Blackmore et al., 2013b), radiology (Hui et al., 2014), and nursing (Ching et al., 2013, Dewing et al., 2013), to traditionally nonacademic teams including hospital operations and support services (Blackmore et al., 2013a, Farrokhi et al., 2013), executive leadership (Kaplan et al., 2014, Calderon et al., 2014), and Lean methods and education (Russ et al., 2013, Ching et al., 2014).
Discussion Lean quality improvement methods can also be applied to improve quality improvement
Vol. 0 No. 0 Month 2015
research. Our increasing number and variety of QuIPs speaks to the success of this model. Through a small methodological core group and a relatively simple Lean toolkit, we enable successful publication of a broad range of QI projects originating throughout the institution. Key lessons include (1) horizontal integration across the entire institution, including formal collaborations with executive leadership, the clinical data systems group, and QI teams; (2) standardization of the QI research process by applying the tools of Lean production; and (3) implementation of a toolkit to enable investigators to overcome barriers in producing quality improvement research publications. The results we report are from a single institution, and we cannot claim to have perfected the approach or the tools. Furthermore, our simplistic analysis of the effectiveness of the program is meant as an illustration, not as a comprehensive program evaluation. Customer satisfaction data in particular would be valuable to help understand and improve the program. However, our program does highlight the potential value of viewing the research enterprise as another process to which quality improvement methods can be applied, an approach that we expect would have value throughout healthcare. In addition, other ways in which the CHSR may contribute to the institution are not captured in the analysis. Through working with the CHSR, staff members from throughout the institution have gained greater skill in determining the effectiveness of quality improvement interventions. Also, the quantitative methods used by the CHSR have been incorporated into institutional Lean training. We note that our objective in this report was not to compare different models for increasing research productivity. An alternate approach would have been for us to fund a research institute typical for many academic institutions. This may also have led to success in increasing research productivity. However, institutes are typically vertical in orientation with a specific research focus and concentrated content
knowledge in that arena. Our aim was not simply to establish a research program and produce publications in one targeted area (like iatrogenic infections or falls). Rather, our goal was to support staff and enable research on the broad range of quality improvement work being performed around the institution. Hence, success is not simply the number of articles but also the breadth of reach from the CHSR in publishing work from all areas. Although there have been broad calls for more research on quality improvement (Berwick, 2008, Wachter 2010, Davidoff et al., 2008) and rigorous debate on the methodology for quality improvement research (Auerbach et al., 2007, Shojania et al., 2002, Berwick, 2008), there is little other published literature on improving the efficiency of quality improvement research. Holzmueller and Pronovost (2013) developed a standard template for article writing, although its effectiveness has not been demonstrated. Through our Lean approach, we attempt to address inefficiencies in both the organization of quality research infrastructure and the process of performing quality improvement research projects. Advancing quality improvement requires rigorous research on the effectiveness of proposed interventions. In this work, we demonstrate how the tools of quality improvement themselves can be applied to the quality improvement research enterprise, enabling efficient and successful QI research from traditional academic departments, but also nontraditional areas including hospital operations and support services, and executive leadership. We believe that this approach can help drive the evidence base necessary to allow quality improvement to reach its potential for addressing the cost and quality challenges facing healthcare.
References Auerbach, AD, Landefeld, CS, & Shojania, KG. The tension between needing to improve care and knowing how to do it. N Engl J Med 2007;357:608–613.
7
8
Journal for Healthcare Quality
Berwick, DM. The science of improvement. JAMA 2008;299:1182–1184. Blackmore, CC, Bishop, R, Luker, S, & Williams, BL. Applying lean methods to improve quality and safety in surgical sterile instrument processing. Jt Comm J Qual Patient Safe 2013a;39:99–105. Blackmore, CC, Edwards, JW, & Searles, C, et al. Nurse practitioner-staffed clinic at Virginia Mason improves care and lowers costs for women with benign breast conditions. Health Aff (millwood) 2013b;32:20–26. Calderon, AS, Blackmore, CC, & Williams, BL, et al. Transforming ward rounds through rounding-in-flow. J Grad Med Educ 2014;4: 750–755. Ching, JM, Long, C, Williams, BL, & Blackmore, CC. Using lean to improve medication administration safety. Jt Comm J Qual Patient Safe 2013;39:199–204. Ching, JM, Williams, BL, Idemoto, LM, & Blackmore, CC. Using lean automation with a human touch to improve medication safety: a step closer to the perfect dose. Jt Comm J Qual Patient Safe 2014;40:341–350. Crawford, BA, & Rumsey, SM. Guidance for Monitoring Recovery of Pacific Northwest Salmon and Steelhead Listed Under the Endangered Species Act. Washington, DC: National Marine Fisheries Service. Davidoff, F, Batalden, P, & Stevens, D, et al. Publication guidelines for quality improvement in health care: evolution of the SQUIRE project. Qual Saf Health Care 2008;17:i3–i9. Dewing, K, Belza, B, Zierler, B, & LaCroix, AZ. Health maintenance model improves BMD testing. Nurse Pract 2013;38:37–41. Farrokhi, F, Gunther, M, Williams, BL, & Blackmore, CC. Application of lean methodology for improved quality and efficiency in operating room instrument availability. J Healthc Qual Sep 24: 1–10 (epub ahead of print). Holzmueller, CG, & Pronovost, P. Organising a manuscript reporting quality improvement or patient safety research. BMJ Qual Saf 2013;22:777–785. Hui, JS, Kramer, DJ, & Blackmore, CC, et al. A quality improvement initiative to reduce unnecessary follow-up for imaging for adnexal lesions. J Am Coll Radiol 2014;11:373–377. Hurtado, MP, Swift, EK, & Corrigan, JM. Crossing the Quality Chasm: A New Health Care System for the 21st Century. Washington, DC: National Academy Press; 2001. Kaplan, GS, Patterson, SH, Ching, JM, & Blackmore, CC. Why Lean doesn’t work for everyone. BMJ Qual Saf 2014;23:970–973.
Kenney, C. Transforming Health Care: Virginia Mason Medical Center’s Pursuit of the Perfect Patient Experience New York, NY: Productivity Press—Taylor & Francis Group; 2011. Porter, AJ, Narimasu, JY, Mulroy, MF, & Koehler, RP. Sustainable, effective implementation of a surgical preprocedural checklist: an “attestation” format for all operating team members. Jt Comm J Qual Patient Saf 2014;40:3–9. Russ, LR, Phillips, J, & Brzozowicz, K, et al. Experience-based design for integrating the patient care experience into healthcare improvement: identifying a set of reliable emotion words. Healthcare 2013;1:91–99. Shojania, KG, Duncan, BW, McDonald, KM, & Wachter, RM. Safe but not sound: patient safety meets evidence-based medicine. JAMA 2002;288:508–513. SQUIRE: Standards for Quality Improvement Reporting Excellence. 2008. Available at: http:// squire-statement.org/assets/pdfs/SQUIRE_ guidelines_table.pdf. Accessed May 15, 2014. Wachter, RM. Patient safety at ten: unmistakable progress, troubling gaps. Health Aff (Millwood) 2010;29:165–173.
Authors’ Biographies Christopher Craig Blackmore, MD, MPH, is the Director of the Center for Health Services Research at Virginia Mason, Seattle, WA. Barbara L. Williams, PhD, is Research Scientist at the Center for Health Services Research at Virginia Mason. Joan M. Ching, RN, MN, is Nursing Director for Quality and Safety. Lynne A. Chafetz, JD, is Senior Vice President and General Counsel, Virginia Mason. Gary S. Kaplan, MD, is President and Chief Executive Officer, Virginia Mason. For more information on this article, contact C. Craig Blackmore at [email protected]. Supplemental digital content is available for this article. Direct URL citations appear in the printed text and in the HTML and PDF versions of the article at www.jhqonline.com. The authors declare no conflict of interest.
