Beginner’s Guide to Practice Quality Improvement Using the Model for Improvement Cindy S. Lee, MDa, David B. Larson, MD, MBAb
Radiologists in the United States are required to complete the Practice Quality Improvement (PQI) program as part of their Maintenance of Certification by the ABR. The Institute for Healthcare Improvement’s (IHI) Model for Improvement (MFI) offers an alternative to the 3-phase approach currently advocated by the ABR. The MFI implicitly assumes that many interventions will need to be tested and refined for any meaningful project, and provides a project management approach that enables rapid assessment and improvement of performance. By collecting data continuously, rather than simply before and after interventions, more interventions can be tested simultaneously and projects can progress more rapidly. In this article, we describe the ABR’s 3-phase approach, and introduce the MFI and how it can be employed to affect positive changes. Using a radiology case study, we demonstrate how one can utilize the MFI to enable rapid quality improvement. Key Words: Model for improvement, maintenance of certification, practice quality improvement J Am Coll Radiol 2014;11:1131-1136. Copyright © 2014 American College of Radiology
INTRODUCTION
In 2007, the ABR established the Practice Quality Improvement (PQI) program as part of its Maintenance of Certification (MOC) program, with the goal to “improve the quality of health care through diplomateinitiated learning and quality improvement” [1]. A satisfactory PQI project must be relevant to one’s practice, be achievable in one’s clinical setting, produce results suited for repeat measurements during the MOC cycle, and be reasonably expected to result in quality improvement [2]. In this article, we describe the quality improvement (QI) methodology published by the ABR, introduce the Model for Improvement (MFI)—another popular model used in health care—and briefly describe how an individual or group can use that model to get started to effect positive change within an organization. QI METHODOLOGIES
Multiple project improvement methodologies have been developed over the past several decades. Among the a
Department of Radiology Biomedical Imaging, University of California, San Francisco, San Franciso, California.
b
Department of Radiology, Stanford University School of Medicine, Stanford, CA 94305-51-5. Corresponding author and reprints: Cindy S. Lee, MD, Department of Radiology and Biomedical Imaging, University of California, San Francisco, 505 Parnassus Avenue, Room L374, San Francisco, CA 94143; e-mail: cindy. [email protected]. ª 2014 American College of Radiology 1546-1440/14/$36.00 http://dx.doi.org/10.1016/j.jacr.2014.08.033
most popular programs are Six Sigma DMAIC (define, measure, analyze, improve, control) and Lean (define value, identify value streams, establish “pull” and “flow,” and pursue perfection) [3]. Perhaps the most popular methodology currently used in health care is the MFI, promoted by the Institute for Healthcare Improvement (IHI) [4]. All of the models reflect a common theme of goal definition, data acquisition and analysis, implementation of process change, and review of results. Each of the models uses some version of the Plan-DoStudy-Act (PDSA) cycle, which was initially devised by a founder of modern QI methodology, W. Edwards Deming. Its original intent was to teach factory workers how to conduct QI in a scientific manner [5]. The PDSA is generally introduced as a restatement of the scientific method. Framed in this manner, the scientific method is to form a hypothesis (plan), test the hypothesis (do), analyze the data (study), and draw actionable conclusions (act). The PDSA cycle has been widely adopted in manufacturing, although it has been modified according to local needs. For example, Toyota Motor Company modified the Plan-Do-Study-Act model to be the Plan-Do-Check-Act model, although these are often used interchangeably [6]. The methodology currently prescribed by the ABR for execution of the project is a version of the PDSA cycle. As it is described, the methodology has at least 3 phases: the baseline PDSA cycle, implementation of an improvement 1131
1132 Journal of the American College of Radiology/Vol. 11 No. 12PA December 2014
plan, and the posteimprovement plan PDSA cycle. This model has been incorporated into the MOC program, which requires radiologists to complete at least one improvement project every 3 years [7]. A useful alternative to the ABR approach is the MFI by the IHI, which is an organization committed to redesigning health care into a system without errors, waste, delay, or unsustainable costs, since the late 1980s [3,8]. Although the 2 models have similarities, they differ in important ways. 3-PHASE MODEL
The ABR currently frames the PQI requirement in terms of the PDSA methodology, as follows [7]. Phase 1: Baseline PDSA Cycle (Cycle #1)
Plan. An area of your practice that is judged to be in need of improvement is identified, and a measure is devised to assess the degree of need. A plan is developed to implement the measure and obtain the required data. Finally, a target or goal for the measure to be reached is set. Do. Your plan is set in motion, and data are collected. Study. You determine how well your measure compared to the desired goal. Root causes for lack of goal achievement are explored. Act. You consider what can be done to address the root causes and develop an improvement plan to implement in the next PDSA cycle. Phase 2: Implement Your Improvement Plan Phase 3: Post-improvement Plan Cycle (Cycle #2)
After improvement plan implementation, another PDSA cycle is begun to assess the degree of any gain achieved. The cycle can be used continuously until the goal is reached or employed intermittently to document the stability of any gain achieved. IHI MODEL FOR IMPROVEMENT
Promoted by W. Edwards Deming [3], the MFI frames a project with 3 questions (Fig. 1): (1) What are we trying to accomplish?; (2) How will we know a change is an improvement?; and (3) What changes can we make that will result in improvement? Proposed process changes are then iteratively tested, refined, and gradually implemented. Each test is conducted to gain additional knowledge about the process or about the potential effectiveness of proposed changes, based on the PDSA methodology. Essentially, each test is a focused scientific experiment, whereby the test is planned and conducted, the results are analyzed, and actions are taken based on the results that may include adopting the change, abandoning the change, or modifying the change and retesting it. COMPARING AND CONTRASTING THE MODELS
Both models promote an iterative approach to improvement, which consists of selecting an area of
Fig 1. Model for Improvement. The first 3 steps are used in planning the project. Subsequently, the Plan-Do-Study-Act cycle is used in an iterative fashion to test and refine changes.
improvement, developing performance measure(s), establishing a target, examining the perceived root causes of underperformance, identifying changes, and implementing and remeasuring those changes. However, the 2 models differ in their treatment of the PDSA cycle, which has important implications for how the project is conducted. In the 3-phase model, the first PDSA is largely a project management framework. By contrast, in the MFI, the term PDSA means “hypotheses testing” and is embedded within a larger project management framework. For example, in the 3-phase model, the “plan” of the first PDSA cycle includes these steps: identifying the need for improvement, developing a measure, and establishing the target performance. In the MFI, these 3 tasks are addressed by the first question: “What are we trying to accomplish?” In the 3-phase model, the “do” and “study” portions of the first PDSA cycle include collecting baseline measurements, analyzing the data to determine the current status relative to the target, and exploring root causes for performance deficits.
Lee, Larson/Model for Improvement 1133
In the MFI, these same tasks are addressed by the second question: “How will we know if a change is an improvement?” In the 3-phase model, the “act” of the first PDSA cycle includes possible changes to address the root causes and make improvement plan(s). In the MFI, these actions are addressed by the third question: “What changes can we make that will result in improvement?” However, in the MFI, the assumption is made that the changes, as they are originally conceived, will not be immediately successful and will require multiple revisions before they can be implemented. In contrast, in the ABR’s model, changes are immediately implemented after they are planned. In the 3-phase approach, after the changes are implemented, a second PDSA is conducted to plan and obtain the measurement, analyze the data, and assess whether the target is reached or another cycle is needed. In contrast, the MFI divides the project into 2 parts: (1) planning and management, and (2) testing and refinement of interventions. The project management portion includes activities encompassed in the 3 initial questions. Once the measures and targets are established, performance is continuously measured throughout the life of the project, rather than with a simple before-versus-after comparison. After investigation of the root causes of underperformance, multiple potential improvements are developed, with the recognition that many will fail and the successful few will likely require significant refinement before they are widely implemented. The 3-phase model calls for implementation of the planned changes immediately after developing an improvement plan. The MFI, on the other hand, dictates that potential improvements be tested and refined on as small a scale as possible before being implemented. Process changes are expanded gradually once they have been proven successful on a smaller scale. The final determination of whether changes are effective is whether they result in improved performance. Hence, it is critical to continuously monitor performance throughout the project’s life, rather than to simply obtain 2 measurements. Perhaps the primary advantage of the MFI is that it enables rapid iteration. With the 3-phase approach, changes are implemented and tested one at a time, with periods of measurement in between. In contrast, with the MFI, since measures are obtained continuously, the impact on outcome can be detected almost immediately. Because only 2 episodes of data collection are required, the 3-phase model implicitly assumes that implemented changes are likely to succeed on the first try, although additional cycles can be conducted if needed. By contrast, the MFI inherently assumes that most changes initially fail, or at least require significant refinement. The MFI applies the scientific method to study and test the proposed changes as quickly and with as little impact as possible before implementing those changes. This model assumes that one cannot fully
anticipate all possible nuances during implementation and therefore it is best to run small tests to understand the impact of these changes. Furthermore, because tests are small and outcome data are monitored continuously, multiple tests can be run either in parallel or in rapid succession, enabling rapid improvement and avoiding having to wait for data from implementation of one large change at a time. PQI CASE STUDY USING THE MFI
The following hypothetical case study illustrates how the IHI’s MFI might be used in the radiology setting. Project Goal
Reducing the “miss” rate of retained surgical foreign objects on operating room (OR) radiographs. Background
At a large community hospital, 2 cases of retained surgically placed objects have been reported in the past 3 months. In one case, the object was missed on the initial OR radiograph but identified on a later exam. In another case, the object was identified on the radiograph, but by the time the call was made to the OR, the patient was already out of surgery and in the recovery unit. The hospital safety office is launching an investigation; the CEO has made it clear to the radiologist-inchief that the group’s contract will be in jeopardy if the situation is not immediately remedied and wants to see evidence that the process has been changed to prevent recurrence. Typically, an intraoperative radiograph ordered to exclude foreign body is performed stat. When the study is ordered, the OR staff calls the technologist work area to summon the technologist. After acquiring the image, the technologist processes the computed radiography (CR) plate and walks to the reading room to review the case with the attending radiologist, who then calls the OR. During off hours, when no attending radiologist is in the hospital, the radiology resident is responsible for interpreting the study and calling the OR. The investigation revealed that, in the first case, the radiologist made the finding of a clamp but had assumed that it was external to the patient (Fig. 2A). In the second case, the surgery was performed overnight, and the radiology resident was busy performing an ultrasound when the OR call came. After waiting 15 minutes, the surgeon completed the case. The needle count had been incorrect, but the surgeon had reviewed the image himself and did not see the needle which, in retrospect, mimicked a surgical clip on the image (Fig. 2B). Project Planning
The chief radiologist met with his practice manager, chief technologist, and 3 senior radiologists to discuss this problem. One of the radiologists, the chief of the radiography section, was assigned to lead the improvement project.
1134 Journal of the American College of Radiology/Vol. 11 No. 12PA December 2014
Fig 2. (A) Postoperative radiograph of the abdomen and pelvis, demonstrating an overlying surgical clamp that was assumed to be outside of the patient in the hypothetical case study; (B) Postoperative radiograph demonstrating a needle overlying the left flank, which was initially assumed to be a surgical clip.
What are we trying to accomplish? Decrease the rate of missed retained foreign objects on OR radiographs from approximately 1 in 100 to
Radiologists in the United States are required to complete the Practice Quality Improvement (PQI) program as part of their Maintenance of Certification by the ABR. The Institute for Healthcare Improvement’s (IHI) Model for Improvement (MFI) offers an alternative to the 3-phase approach currently advocated by the ABR. The MFI implicitly assumes that many interventions will need to be tested and refined for any meaningful project, and provides a project management approach that enables rapid assessment and improvement of performance. By collecting data continuously, rather than simply before and after interventions, more interventions can be tested simultaneously and projects can progress more rapidly. In this article, we describe the ABR’s 3-phase approach, and introduce the MFI and how it can be employed to affect positive changes. Using a radiology case study, we demonstrate how one can utilize the MFI to enable rapid quality improvement. Key Words: Model for improvement, maintenance of certification, practice quality improvement J Am Coll Radiol 2014;11:1131-1136. Copyright © 2014 American College of Radiology
INTRODUCTION
In 2007, the ABR established the Practice Quality Improvement (PQI) program as part of its Maintenance of Certification (MOC) program, with the goal to “improve the quality of health care through diplomateinitiated learning and quality improvement” [1]. A satisfactory PQI project must be relevant to one’s practice, be achievable in one’s clinical setting, produce results suited for repeat measurements during the MOC cycle, and be reasonably expected to result in quality improvement [2]. In this article, we describe the quality improvement (QI) methodology published by the ABR, introduce the Model for Improvement (MFI)—another popular model used in health care—and briefly describe how an individual or group can use that model to get started to effect positive change within an organization. QI METHODOLOGIES
Multiple project improvement methodologies have been developed over the past several decades. Among the a
Department of Radiology Biomedical Imaging, University of California, San Francisco, San Franciso, California.
b
Department of Radiology, Stanford University School of Medicine, Stanford, CA 94305-51-5. Corresponding author and reprints: Cindy S. Lee, MD, Department of Radiology and Biomedical Imaging, University of California, San Francisco, 505 Parnassus Avenue, Room L374, San Francisco, CA 94143; e-mail: cindy. [email protected]. ª 2014 American College of Radiology 1546-1440/14/$36.00 http://dx.doi.org/10.1016/j.jacr.2014.08.033
most popular programs are Six Sigma DMAIC (define, measure, analyze, improve, control) and Lean (define value, identify value streams, establish “pull” and “flow,” and pursue perfection) [3]. Perhaps the most popular methodology currently used in health care is the MFI, promoted by the Institute for Healthcare Improvement (IHI) [4]. All of the models reflect a common theme of goal definition, data acquisition and analysis, implementation of process change, and review of results. Each of the models uses some version of the Plan-DoStudy-Act (PDSA) cycle, which was initially devised by a founder of modern QI methodology, W. Edwards Deming. Its original intent was to teach factory workers how to conduct QI in a scientific manner [5]. The PDSA is generally introduced as a restatement of the scientific method. Framed in this manner, the scientific method is to form a hypothesis (plan), test the hypothesis (do), analyze the data (study), and draw actionable conclusions (act). The PDSA cycle has been widely adopted in manufacturing, although it has been modified according to local needs. For example, Toyota Motor Company modified the Plan-Do-Study-Act model to be the Plan-Do-Check-Act model, although these are often used interchangeably [6]. The methodology currently prescribed by the ABR for execution of the project is a version of the PDSA cycle. As it is described, the methodology has at least 3 phases: the baseline PDSA cycle, implementation of an improvement 1131
1132 Journal of the American College of Radiology/Vol. 11 No. 12PA December 2014
plan, and the posteimprovement plan PDSA cycle. This model has been incorporated into the MOC program, which requires radiologists to complete at least one improvement project every 3 years [7]. A useful alternative to the ABR approach is the MFI by the IHI, which is an organization committed to redesigning health care into a system without errors, waste, delay, or unsustainable costs, since the late 1980s [3,8]. Although the 2 models have similarities, they differ in important ways. 3-PHASE MODEL
The ABR currently frames the PQI requirement in terms of the PDSA methodology, as follows [7]. Phase 1: Baseline PDSA Cycle (Cycle #1)
Plan. An area of your practice that is judged to be in need of improvement is identified, and a measure is devised to assess the degree of need. A plan is developed to implement the measure and obtain the required data. Finally, a target or goal for the measure to be reached is set. Do. Your plan is set in motion, and data are collected. Study. You determine how well your measure compared to the desired goal. Root causes for lack of goal achievement are explored. Act. You consider what can be done to address the root causes and develop an improvement plan to implement in the next PDSA cycle. Phase 2: Implement Your Improvement Plan Phase 3: Post-improvement Plan Cycle (Cycle #2)
After improvement plan implementation, another PDSA cycle is begun to assess the degree of any gain achieved. The cycle can be used continuously until the goal is reached or employed intermittently to document the stability of any gain achieved. IHI MODEL FOR IMPROVEMENT
Promoted by W. Edwards Deming [3], the MFI frames a project with 3 questions (Fig. 1): (1) What are we trying to accomplish?; (2) How will we know a change is an improvement?; and (3) What changes can we make that will result in improvement? Proposed process changes are then iteratively tested, refined, and gradually implemented. Each test is conducted to gain additional knowledge about the process or about the potential effectiveness of proposed changes, based on the PDSA methodology. Essentially, each test is a focused scientific experiment, whereby the test is planned and conducted, the results are analyzed, and actions are taken based on the results that may include adopting the change, abandoning the change, or modifying the change and retesting it. COMPARING AND CONTRASTING THE MODELS
Both models promote an iterative approach to improvement, which consists of selecting an area of
Fig 1. Model for Improvement. The first 3 steps are used in planning the project. Subsequently, the Plan-Do-Study-Act cycle is used in an iterative fashion to test and refine changes.
improvement, developing performance measure(s), establishing a target, examining the perceived root causes of underperformance, identifying changes, and implementing and remeasuring those changes. However, the 2 models differ in their treatment of the PDSA cycle, which has important implications for how the project is conducted. In the 3-phase model, the first PDSA is largely a project management framework. By contrast, in the MFI, the term PDSA means “hypotheses testing” and is embedded within a larger project management framework. For example, in the 3-phase model, the “plan” of the first PDSA cycle includes these steps: identifying the need for improvement, developing a measure, and establishing the target performance. In the MFI, these 3 tasks are addressed by the first question: “What are we trying to accomplish?” In the 3-phase model, the “do” and “study” portions of the first PDSA cycle include collecting baseline measurements, analyzing the data to determine the current status relative to the target, and exploring root causes for performance deficits.
Lee, Larson/Model for Improvement 1133
In the MFI, these same tasks are addressed by the second question: “How will we know if a change is an improvement?” In the 3-phase model, the “act” of the first PDSA cycle includes possible changes to address the root causes and make improvement plan(s). In the MFI, these actions are addressed by the third question: “What changes can we make that will result in improvement?” However, in the MFI, the assumption is made that the changes, as they are originally conceived, will not be immediately successful and will require multiple revisions before they can be implemented. In contrast, in the ABR’s model, changes are immediately implemented after they are planned. In the 3-phase approach, after the changes are implemented, a second PDSA is conducted to plan and obtain the measurement, analyze the data, and assess whether the target is reached or another cycle is needed. In contrast, the MFI divides the project into 2 parts: (1) planning and management, and (2) testing and refinement of interventions. The project management portion includes activities encompassed in the 3 initial questions. Once the measures and targets are established, performance is continuously measured throughout the life of the project, rather than with a simple before-versus-after comparison. After investigation of the root causes of underperformance, multiple potential improvements are developed, with the recognition that many will fail and the successful few will likely require significant refinement before they are widely implemented. The 3-phase model calls for implementation of the planned changes immediately after developing an improvement plan. The MFI, on the other hand, dictates that potential improvements be tested and refined on as small a scale as possible before being implemented. Process changes are expanded gradually once they have been proven successful on a smaller scale. The final determination of whether changes are effective is whether they result in improved performance. Hence, it is critical to continuously monitor performance throughout the project’s life, rather than to simply obtain 2 measurements. Perhaps the primary advantage of the MFI is that it enables rapid iteration. With the 3-phase approach, changes are implemented and tested one at a time, with periods of measurement in between. In contrast, with the MFI, since measures are obtained continuously, the impact on outcome can be detected almost immediately. Because only 2 episodes of data collection are required, the 3-phase model implicitly assumes that implemented changes are likely to succeed on the first try, although additional cycles can be conducted if needed. By contrast, the MFI inherently assumes that most changes initially fail, or at least require significant refinement. The MFI applies the scientific method to study and test the proposed changes as quickly and with as little impact as possible before implementing those changes. This model assumes that one cannot fully
anticipate all possible nuances during implementation and therefore it is best to run small tests to understand the impact of these changes. Furthermore, because tests are small and outcome data are monitored continuously, multiple tests can be run either in parallel or in rapid succession, enabling rapid improvement and avoiding having to wait for data from implementation of one large change at a time. PQI CASE STUDY USING THE MFI
The following hypothetical case study illustrates how the IHI’s MFI might be used in the radiology setting. Project Goal
Reducing the “miss” rate of retained surgical foreign objects on operating room (OR) radiographs. Background
At a large community hospital, 2 cases of retained surgically placed objects have been reported in the past 3 months. In one case, the object was missed on the initial OR radiograph but identified on a later exam. In another case, the object was identified on the radiograph, but by the time the call was made to the OR, the patient was already out of surgery and in the recovery unit. The hospital safety office is launching an investigation; the CEO has made it clear to the radiologist-inchief that the group’s contract will be in jeopardy if the situation is not immediately remedied and wants to see evidence that the process has been changed to prevent recurrence. Typically, an intraoperative radiograph ordered to exclude foreign body is performed stat. When the study is ordered, the OR staff calls the technologist work area to summon the technologist. After acquiring the image, the technologist processes the computed radiography (CR) plate and walks to the reading room to review the case with the attending radiologist, who then calls the OR. During off hours, when no attending radiologist is in the hospital, the radiology resident is responsible for interpreting the study and calling the OR. The investigation revealed that, in the first case, the radiologist made the finding of a clamp but had assumed that it was external to the patient (Fig. 2A). In the second case, the surgery was performed overnight, and the radiology resident was busy performing an ultrasound when the OR call came. After waiting 15 minutes, the surgeon completed the case. The needle count had been incorrect, but the surgeon had reviewed the image himself and did not see the needle which, in retrospect, mimicked a surgical clip on the image (Fig. 2B). Project Planning
The chief radiologist met with his practice manager, chief technologist, and 3 senior radiologists to discuss this problem. One of the radiologists, the chief of the radiography section, was assigned to lead the improvement project.
1134 Journal of the American College of Radiology/Vol. 11 No. 12PA December 2014
Fig 2. (A) Postoperative radiograph of the abdomen and pelvis, demonstrating an overlying surgical clamp that was assumed to be outside of the patient in the hypothetical case study; (B) Postoperative radiograph demonstrating a needle overlying the left flank, which was initially assumed to be a surgical clip.
What are we trying to accomplish? Decrease the rate of missed retained foreign objects on OR radiographs from approximately 1 in 100 to
