J Med Syst (2014) 38:144 DOI 10.1007/s10916-014-0144-8
PATIENT FACING SYSTEMS
Operating Room Metrics Score Card—Creating a Prototype for Individualized Feedback Rodney A. Gabriel & Robert Gimlich & Jesse M. Ehrenfeld & Richard D. Urman
Received: 17 August 2014 / Accepted: 6 October 2014 / Published online: 15 October 2014 # Springer Science+Business Media New York 2014
Abstract The balance between reducing costs and inefficiencies with that of patient safety is a challenging problem faced in the operating room suite. An ongoing challenge is the creation of effective strategies that reduce these inefficiencies and provide real-time personalized metrics and electronic feedback to anesthesia practitioners. We created a sample report card structure, utilizing existing informatics systems. This system allows to gather and analyze operating room metrics for each anesthesia provider and offer personalized feedback. To accomplish this task, we identified key metrics that represented time and quality parameters. We collected these data for individual anesthesiologists and compared performance to the overall group average. Data were presented as an electronic score card and made available to individual clinicians on a real-time basis in an effort to provide effective feedback. These metrics included number of cancelled cases, average turnover time, average time to operating room ready and patient in room, number of delayed first case starts, average induction time, average extubation time, average time to recovery room arrival to discharge, performance feedback from other providers, compliance to various protocols, and total anesthetic costs. The concept we propose can easily be generalized to a variety of operating room settings, types of facilities and OR health care professionals. Such a scorecard This work attributed to: Dept. of Anesthesiology, Brigham and Women’s Hospital, Boston, MA This article is part of the Topical Collection on Patient Facing Systems. R. A. Gabriel : R. Gimlich : R. D. Urman (*) Department of Anesthesiology, Perioperative and Pain Medicine, Brigham and Women’s Hospital, Harvard Medical School, 75 Francis Street, Boston, MA 02115, USA e-mail: [email protected] J. M. Ehrenfeld Department of Anesthesiology, Vanderbilt University School of Medicine, Nashville, TN 37232, USA
can be created using content that is important for operating room efficiency, research, and practice improvement for anesthesia providers. Keywords Operating room . Management . Quality . Performance improvement . Benchmarking . Outcomes . Report card
Introduction The operating room suite is often the largest contributor to a hospital’s financial success. However, it is also one of the most costly units within this environment (1). Operating room (OR) efficiency is thus a high priority for hospitals especially during a time in which the delivery of health care is becoming increasingly more challenging (2, 3). Central to the dynamics of OR functionality is patient safety, OR utilization efficiency, and financial costs of labor, medication management, and equipment usage. The balance between reducing costs and inefficiencies with that of patient safety is a challenging problem faced by the anesthesiologists and other perioperative providers in the OR setting. A number of studies have reported on factors and processes that contribute to OR inefficiencies (4–7). Several benchmarks, including anesthesia-controlled time (ACT) and turnover time (TOT) are used to gauge the efficiency of the ORs (8). Creating highly efficient ORs is challenging given the variability of patient problems, operating types, and unexpected events, especially in a complex, tertiary care academic environment. Additional barriers include infrastructure, human resource management, scheduling variation, process flow, technology issues, and information systems. The first step in improving OR efficiency is identifying appropriate metrics that highlight the key players that define an efficient OR. Some of these metrics include TOT, first case start time,
144, Page 2 of 6
surgical and anesthesia delays, OR utilization, and ACTs, among others (3, 8). Defining valuable, reliable, and useful quality indicators that measure OR efficiency is, however, only the first step in the improvement process. The key question is how one can use these metrics and execute actionable changes by individuals. The terminology “feedback” is defined as the act of providing knowledge of the results of the behavior or performance to an individual or group. Clinical performance feedback is a vital component of the ongoing development and education of health care providers and systems (9). Many studies have evaluated the feedback strategies applied in the clinical setting; however, the outcomes of these systems were mixed (10). Studies have shown that characteristics of effective feedback include: trust in data quality, timeliness, and confidentiality combined with educational implementation, verbal/graphic feedback delivery, and when it is presented close to the time of decision-making (11–15). Mugford et al. describe passive feedback as provided information without an underlying mechanism for requiring subsequent action, whereas active feedback occurs when clinician interest is stimulated and engaged concurrently (16). To be effective, feedback must incorporate some action or response from the receiver to close the identified gap. Benn and colleagues reviewed the literature in providing effective feedback in the clinical setting and synthesized from a diverse body of literature on how information from quality indicators can be fed back and used effectively to enhance care (10). Promoting OR efficiency is a challenge that when addressed, may lead to both economic and healthcare rewards. Although it is logical to incorporate practice-related feedback into the daily practice anesthesia providers, it is not done consistently at many institutions. Implementing a more immediate, formal feedback strategy that focuses on goal-oriented behavior into the OR setting is ideal (17). The OR consists of a multidisciplinary team of health care professionals, including surgeons, anesthesiologists, nurses, technicians, and pharmacists. There is a need to develop a reporting system that utilizes existing informatics systems to gather and analyze metrics pertaining to OR efficiency. The content of such a report would be important not only to anesthesiologists, but other stakeholders such as hospital administrators, nursing staff, surgeons and pharmacists. In fact, this concept can easily be generalized to other OR health care professionals. Previously, we published a study describing the creation of an anesthesia report card covering all aspects of anesthesiology practice including fulfilling compliance and credentialing requirements, academic achievements, clinical performance, and contributions to education (18). However, with the scorecard focused on OR performance, users will be able to identify in real-time how their individual OR costs and efficiency metrics compare to those of their colleagues. We hypothesize that such a scorecard can be created using content
J Med Syst (2014) 38:144
that is important for OR efficiency and as a way to improve clinical practice.
Methods We propose the creation of a tool that accurately measures individual metrics pertaining to OR efficiency and provides feedback in a timely manner with an effort to improve the quality of clinical care and decrease costs at a large academic institution. This system can aggregate the data of an entire group so that individuals can compare their performance to that of their peers. The type of information obtained from databases was picked based on whether it would help the facility keep track of the individual anesthesiologist’s impact on OR costs and efficiency. Input on what constitutes appropriate metrics for this tool was provided by anesthesiologists and other members of the OR leadership management. The selected metrics were chosen based on demonstrated factors contributing to OR costs and quality of care (4–7). As this is an ongoing dynamic project, current metrics could be changed or removed, and new metrics can be added. The scorecard contains metrics that assess OR efficiency, and these include metrics that fall within two parameters: time and quality. To obtain the appropriate metrics, we used hospital quality data to obtain ACTs and TOTs for individual providers as well as grouped statistics, which was grouped based on either all surgical cases or just cases in a specific surgical subdivisions. We used the ACT definition as established by the American Association of Clinical Directors (19). For each individual utilizing the scorecard, ACTs were calculated for each unique surgical case on record, utilizing a subset of timestamps that are routinely documented by the OR staff at our institution. These timestamps include: “Room Ready”, “Patient Into OR”, “Induction Complete”, “Surgery End”, and “Ready to Transfer”. From these timestamps, we were able to collect individual and grouped data for “average time to room ready and patient in room”, “average time to patient in room to induction complete”, and “average time to surgery end to extubation”. Other metrics were collected using hospital quality data, including number of cancelled cases, average TOT, average case time, case hours, amount of overtime, and average time to postanesthesia care unit (PACU) arrival and discharge. To measure quality of care, an important dimension of OR management, we collected provider data from performance evaluations and compliance to various protocols such as anesthesia record signing, time outs/checklist protocols, surgical antibiotics, and medication reconciliation. Total anesthetic costs per individual were also obtained through pharmacy databases. Each of these metrics were calculated for an individual, total group, and for each anesthesia subspecialty.
J Med Syst (2014) 38:144
Page 3 of 6, 144
Results
Discussion
Individual and group metrics were provided via a report card. This report card prototype consisted of metrics describing individual contribution to OR efficiency and costs. Institutional experiences and literature related to OR efficiency helped identify several factors that play an important role in defining the efficiency of an OR (4–6, 20–22). Figure 1 displays the format of the report card. Based on our prototype, the anesthesiologist should able to log onto this report card system at any time to review his/her metric scores for the quarter and how they compared to their peers (Fig. 2a). This can be further broken down based on surgical service (ie orthopedics, neurosurgery, cardiac) as different OR subspecialty areas can have different limitations and some anesthesiologists spent more time in one location versus other locations. Because frequent and updated feedback is likely more effective, the report card can be accessed at any time and allow the clinician to review current results over the past few months as well as compare to previous time periods and to the cumulative data from their colleagues. These real-time reports were additionally available to supervisors permitting them to monitor and intervene where inefficient OR practices are detected. By clicking on a specific metric, the user was provided with a graphical representation of their performance presented as a bell curve using data from the overall group or a surgical subspecialty (Fig. 2b).
Our prototype can lead to further development of the tools with the potential to gather accurate data on individual and group OR efficiency metrics. The tools can be designed to provide timely feedback that would hopefully lead to better quality and cost-effective anesthetic practices. Time and quality are two essential parameters to measure when assessing efficiency in the OR. The time parameters we chose to provide the best overall snapshot of efficiency include: “average time to room ready and patient in room”, “average time to patient in room to induction complete”, “average time to surgery end to extubation”, number of cancelled cases, average TOT, average case time, case hours, amount of overtime, and average time to PACU arrival and discharge. In addition to time saving, OR efficiency should also be based on quality, which can be defined as a measure of adherence to quality and compliance protocols. To measure quality, our scorecard presents information such as evaluations by other professional colleagues, as well as compliance in anesthesia record signing, time outs/ checklist use, and medication reconciliation. With our scorecard, OR managers can review efficiency on a quarterly and annual basis for each anesthesia provider. This data, in addition to improving OR efficiency, can provide a platform for research and improvement in clinical care and quality. However, in order for this system to improve OR efficiency, providers must regularly access and respond to the
Fig. 1 Operating Room Metrics Scorecard. This displays the general structure of the report card. The first column lists all the metrics measured for each individual and for the group. Data is presented as quarterly and annual averages, and is colorcoordinated
144, Page 4 of 6
J Med Syst (2014) 38:144
Fig. 2 Sample data from an anesthesiology provider. a. This displays real data obtained for an individual provider and how their quarterly and annual metric averages compare to the group based on cases performed in the Urology suite. Data that is increased compared to the group average
are colored in dark blue and dark orange for quarterly and annual averages, respectively. b. This is a sample graphical representation of provider data. The red-highlighted data point in A is previewed as a bell curve in B
information provided to them. Feedback is an invaluable aspect of the ongoing development of practitioners in health care. It is central to developing competence, professionalism, and confidence at all stages of a physician’s medical career. Despite the value inherent in effective feedback, there are integral challenges in implementing novel mechanisms of feedback into an existing system. The “buy-in” includes the view and expectation that feedback occurs on a routine basis, and those participating are held accountable (17). Feedback may often warrant negative connotations, resulting from providing constructive feedback and consequently damaging relationships with others (23). Furthermore, barriers to implementing a structured feedback system include cultural and financial infrastructures. These new systems need to be embraced and encouraged by senior members of a department. In additional, creating such as a system would require a financial investment into the IT infrastructure and continued technological support and development. There are major premises of successfully integrating feedback and establishing a culture of physician improvement (17). The purpose of effective feedback should focus on quality improvement of a health care group and thus should not be an isolated initiative. It should be viewed as part of a physician’s lifelong learning process. Feedback is best when it detects problems early on, providing information in real-time,
in an effort to curtail future problems. Finally, feedback data instruments should cater to the needs of the overall practicing group and not just the individual. We designed a report card that can provide anesthesiologists with an overview of their specific ACT averages, turnover times, anesthetic costs, and number of cancelled cases. These values are subsequently compared to group data and provide a graphical representation of how they compare statistically to their peers. Numerous characteristics that previously published literature and institutional experiences have shown to affect OR efficiency were included in this report card. The real-time data of this report card is available at anytime, is constantly updated, and thus provides feedback in a timely manner. The concept serves as a template to provide timely and effective feedback to improve OR efficiency at both the group and individual level. For example, if individual anesthesiologists can better visualize their costeffectiveness compared to others, this may encourage them to tailor their work to more efficient practices, provided they maintain patient safety and quality of care. The proposed feedback tool has its limitations, however. Some anesthesiologists, especially those trained in a particular subspecialty, may work the majority of their time in specific OR subspecialty areas, such as cardiac, thoracic, or orthopedic surgery. These subspecialty areas may have specific
J Med Syst (2014) 38:144
limitations in the measured OR efficiency data and thus are more or less prone to delays exemplified by various metrics (20). An example of a subspecialty-specific limitation is that anesthesiologists in the otorhinolaryngology area may on average have longer time periods between OR start to induction complete due the exposure to patients with more difficult airways. Another example is that anesthesiologists perform preoperative peripheral nerve blocks more often in the orthopedics area, which can understandably lead to increased TOT. Furthermore, some areas will require anesthetics that are consistently more costly than others. To address this limitation, our feedback system allows clinicians to not only have access to comparisons to the overall group, but also metrics based on subspecialty-specific cases in order to provide a more accurate peer to peer comparison. Occasionally, some anesthesiologists will feel that their caseloads consist of more complex cases and medically complicated patients, or patients who are more likely to have postoperative complications such as post-operative nausea and vomiting (PONV). The complexity of the case would require additional medications or procedures, such arterial or central line placement, which would add to total costs and time delays. The patient requiring anesthetic adjustments because of higher risk for PONV could mean more frequent utilization of total intravenous anesthetics and anti-emetic usage. Therefore, it is important to point out that the metrics reported should be an average over a specified time period in order to have a more accurate peer-to-peer comparison. Various health care groups have different methods of data allocation, collection and they utilize electronic medical record systems. This poses a challenge to creating a report that is universally available across the multitude of the utilized platforms. This highlights the IT challenges inherent in a tool of this type. In any case, it is still possible to integrate this concept into practices that have efficient means of collecting these metrics. Effective feedback should be an integral part of clinical practice. While the OR is often the largest contributor to a hospital’s financial success, it is also the most costly unit in this environment. OR efficiency is a high priority, and methods to improve the process are warranted. Providing real-time data to anesthesiologists regarding their contribution to OR costs is an effective method of monitoring inefficiencies and detecting patterns early in an effort to improve overall group OR metrics.
Page 5 of 6, 144
2.
3.
4.
5.
6.
7.
8.
9.
10.
11.
12.
13.
14.
15.
16.
17.
References 18. 1. Cima, R. R., Brown, M. J., Hebl, J. R., Moore, R., Rogers, J. C., Kollengode, A., et al., Use of lean and six sigma methodology to improve operating room efficiency in a high-volume tertiary-care
academic medical center. J Am Coll Surg 213(1):83–92, 2011. discussion 3–4. Epub 2011/03/23. Krupka, D. C., and Sandberg, W. S., Operating room design and its impact on operating room economics. Curr Opin Anaesthesiol 19(2): 185–91, 2006. Epub 2006/03/23. Kodali, B. S., Kim, D., Bleday, R., Flanagan, H., and Urman, R. D., Successful strategies for the reduction of operating room turnover times in a tertiary care academic medical center. J Surg Res 187(2): 403–11, 2014. Epub 2014/01/01. Friedman, D. M., Sokal, S. M., Chang, Y., and Berger, D. L., Increasing operating room efficiency through parallel processing. Ann Surg 243(1):10–4, 2006. Epub 2005/12/24. Harders, M., Malangoni, M. A., Weight, S., and Sidhu, T., Improving operating room efficiency through process redesign. Surgery 140(4): 509–14, 2006. discussion 14–6. Epub 2006/10/03. Stahl, J. E., Sandberg, W. S., Daily, B., Wiklund, R., Egan, M. T., Goldman, J. M., et al., Reorganizing patient care and workflow in the operating room: a cost-effectiveness study. Surgery 139(6):717–28, 2006. Epub 2006/06/20. Sokal, S. M., Craft, D. L., Chang, Y., Sandberg, W. S., and Berger, D. L., Maximizing operating room and recovery room capacity in an era of constrained resources. Arch Surg 141(4):389–93, 2006. discussion 93–5. Epub 2006/04/19. Kodali, B. S., Kim, K. D., Flanagan, H., Ehrenfeld, J. M., and Urman, R. D., Variability of subspecialty-specific anesthesia-controlled times at two academic institutions. J Med Syst 38(2):11, 2014. Epub 2014/01/29. Willig, J. H., Krawitz, M., Panjamapirom, A., Ray, M. N., Nevin, C. R., English, T. M., et al., Closing the feedback loop: an interactive voice response system to provide follow-up and feedback in primary care settings. Journal of medical systems 37(2):9905, 2013. Epub 2013/01/24. Benn, J., Arnold, G., Wei, I., Riley, C., and Aleva, F., Using quality indicators in anaesthesia: feeding back data to improve care. Br J Anaesth 109(1):80–91, 2012. Epub 2012/06/05. van der Veer, S. N., de Keizer, N. F., Ravelli, A. C., Tenkink, S., and Jager, K. J., Improving quality of care. A systematic review on how medical registries provide information feedback to health care providers. Int J Med Inform. 79(5):305–23, 2010. Epub 2010/03/02. de Vos, M., Graafmans, W., Kooistra, M., Meijboom, B., Van Der Voort, P., and Westert, G., Using quality indicators to improve hospital care: a review of the literature. Int J Qual Health Care 21(2): 119–29, 2009. Epub 2009/01/22. Hysong, S. J., Meta-analysis: audit and feedback features impact effectiveness on care quality. Med Care 47(3):356–63, 2009. Epub 2009/02/06. Jamtvedt, G., Young, J. M., Kristoffersen, D. T., O'Brien, M. A., and Oxman, A. D., Does telling people what they have been doing change what they do? A systematic review of the effects of audit and feedback. Qual Saf Health Care 15(6):433–6, 2006. Epub 2006/12/ 05. Veloski, J., Boex, J. R., Grasberger, M. J., Evans, A., and Wolfson, D. B., Systematic review of the literature on assessment, feedback and physicians’ clinical performance: BEME Guide No. 7. Med Teach 28(2):117–28, 2006. Mugford, M., Banfield, P., and O’Hanlon, M., Effects of feedback of information on clinical practice: a review. BMJ 303(6799):398–402, 1991. Epub 1991/08/17. Kaye, A. D., Okanlawon, O. J., and Urman, R. D., Clinical performance feedback and quality improvement opportunities for perioperative physicians. Adv Med Educ Pract. 5:115–23, 2014. Epub 2014/05/17. Peccora, C. D., Gimlich, R., Cornell, R. P., Vacanti, C. A., Ehrenfeld, J. M., and Urman, R. D., Anesthesia report card - a customizable tool for performance improvement. Journal of medical systems 38(9):105, 2014. Epub 2014/07/21.
144, Page 6 of 6 19. Glossary of times used for scheduling and monitoring of diagnostic and therapeutic procedures. AORN Journal. 1997;66(4):601–6. Epub 1997/10/24. 20. Kodali, B. S., Kim, K. D., Flanagan, H., Ehrenfeld, J. M., and Urman, R. D., Variability of subspecialty-specific anesthesia-controlled times at two academic institutions. Journal of Medical Systems 38(2):11, 2014. Epub 2014/01/29. 21. Friedman, D. M., Sokal, S. M., Chang, Y., and Berger, D. L., Increasing operating room efficiency through parallel processing. Annals of Surgery 243(1):10–4, 2006. Epub 2005/12/24.
J Med Syst (2014) 38:144 22. Sokal, S. M., Craft, D. L., Chang, Y., Sandberg, W. S., and Berger, D. L., Maximizing operating room and recovery room capacity in an era of constrained resources. Archives of Surgery 141(4):389–93, 2006. discussion 93–5. Epub 2006/04/19. 23. Heidegger, T., Husemann, Y., Nuebling, M., Morf, D., Sieber, T., Huth, A., et al., Patient satisfaction with anaesthesia care: development of a psychometric questionnaire and benchmarking among six hospitals in Switzerland and Austria. British Journal of Anaesthesia 89(6):863–72, 2002. Epub 2002/11/28.
PATIENT FACING SYSTEMS
Operating Room Metrics Score Card—Creating a Prototype for Individualized Feedback Rodney A. Gabriel & Robert Gimlich & Jesse M. Ehrenfeld & Richard D. Urman
Received: 17 August 2014 / Accepted: 6 October 2014 / Published online: 15 October 2014 # Springer Science+Business Media New York 2014
Abstract The balance between reducing costs and inefficiencies with that of patient safety is a challenging problem faced in the operating room suite. An ongoing challenge is the creation of effective strategies that reduce these inefficiencies and provide real-time personalized metrics and electronic feedback to anesthesia practitioners. We created a sample report card structure, utilizing existing informatics systems. This system allows to gather and analyze operating room metrics for each anesthesia provider and offer personalized feedback. To accomplish this task, we identified key metrics that represented time and quality parameters. We collected these data for individual anesthesiologists and compared performance to the overall group average. Data were presented as an electronic score card and made available to individual clinicians on a real-time basis in an effort to provide effective feedback. These metrics included number of cancelled cases, average turnover time, average time to operating room ready and patient in room, number of delayed first case starts, average induction time, average extubation time, average time to recovery room arrival to discharge, performance feedback from other providers, compliance to various protocols, and total anesthetic costs. The concept we propose can easily be generalized to a variety of operating room settings, types of facilities and OR health care professionals. Such a scorecard This work attributed to: Dept. of Anesthesiology, Brigham and Women’s Hospital, Boston, MA This article is part of the Topical Collection on Patient Facing Systems. R. A. Gabriel : R. Gimlich : R. D. Urman (*) Department of Anesthesiology, Perioperative and Pain Medicine, Brigham and Women’s Hospital, Harvard Medical School, 75 Francis Street, Boston, MA 02115, USA e-mail: [email protected] J. M. Ehrenfeld Department of Anesthesiology, Vanderbilt University School of Medicine, Nashville, TN 37232, USA
can be created using content that is important for operating room efficiency, research, and practice improvement for anesthesia providers. Keywords Operating room . Management . Quality . Performance improvement . Benchmarking . Outcomes . Report card
Introduction The operating room suite is often the largest contributor to a hospital’s financial success. However, it is also one of the most costly units within this environment (1). Operating room (OR) efficiency is thus a high priority for hospitals especially during a time in which the delivery of health care is becoming increasingly more challenging (2, 3). Central to the dynamics of OR functionality is patient safety, OR utilization efficiency, and financial costs of labor, medication management, and equipment usage. The balance between reducing costs and inefficiencies with that of patient safety is a challenging problem faced by the anesthesiologists and other perioperative providers in the OR setting. A number of studies have reported on factors and processes that contribute to OR inefficiencies (4–7). Several benchmarks, including anesthesia-controlled time (ACT) and turnover time (TOT) are used to gauge the efficiency of the ORs (8). Creating highly efficient ORs is challenging given the variability of patient problems, operating types, and unexpected events, especially in a complex, tertiary care academic environment. Additional barriers include infrastructure, human resource management, scheduling variation, process flow, technology issues, and information systems. The first step in improving OR efficiency is identifying appropriate metrics that highlight the key players that define an efficient OR. Some of these metrics include TOT, first case start time,
144, Page 2 of 6
surgical and anesthesia delays, OR utilization, and ACTs, among others (3, 8). Defining valuable, reliable, and useful quality indicators that measure OR efficiency is, however, only the first step in the improvement process. The key question is how one can use these metrics and execute actionable changes by individuals. The terminology “feedback” is defined as the act of providing knowledge of the results of the behavior or performance to an individual or group. Clinical performance feedback is a vital component of the ongoing development and education of health care providers and systems (9). Many studies have evaluated the feedback strategies applied in the clinical setting; however, the outcomes of these systems were mixed (10). Studies have shown that characteristics of effective feedback include: trust in data quality, timeliness, and confidentiality combined with educational implementation, verbal/graphic feedback delivery, and when it is presented close to the time of decision-making (11–15). Mugford et al. describe passive feedback as provided information without an underlying mechanism for requiring subsequent action, whereas active feedback occurs when clinician interest is stimulated and engaged concurrently (16). To be effective, feedback must incorporate some action or response from the receiver to close the identified gap. Benn and colleagues reviewed the literature in providing effective feedback in the clinical setting and synthesized from a diverse body of literature on how information from quality indicators can be fed back and used effectively to enhance care (10). Promoting OR efficiency is a challenge that when addressed, may lead to both economic and healthcare rewards. Although it is logical to incorporate practice-related feedback into the daily practice anesthesia providers, it is not done consistently at many institutions. Implementing a more immediate, formal feedback strategy that focuses on goal-oriented behavior into the OR setting is ideal (17). The OR consists of a multidisciplinary team of health care professionals, including surgeons, anesthesiologists, nurses, technicians, and pharmacists. There is a need to develop a reporting system that utilizes existing informatics systems to gather and analyze metrics pertaining to OR efficiency. The content of such a report would be important not only to anesthesiologists, but other stakeholders such as hospital administrators, nursing staff, surgeons and pharmacists. In fact, this concept can easily be generalized to other OR health care professionals. Previously, we published a study describing the creation of an anesthesia report card covering all aspects of anesthesiology practice including fulfilling compliance and credentialing requirements, academic achievements, clinical performance, and contributions to education (18). However, with the scorecard focused on OR performance, users will be able to identify in real-time how their individual OR costs and efficiency metrics compare to those of their colleagues. We hypothesize that such a scorecard can be created using content
J Med Syst (2014) 38:144
that is important for OR efficiency and as a way to improve clinical practice.
Methods We propose the creation of a tool that accurately measures individual metrics pertaining to OR efficiency and provides feedback in a timely manner with an effort to improve the quality of clinical care and decrease costs at a large academic institution. This system can aggregate the data of an entire group so that individuals can compare their performance to that of their peers. The type of information obtained from databases was picked based on whether it would help the facility keep track of the individual anesthesiologist’s impact on OR costs and efficiency. Input on what constitutes appropriate metrics for this tool was provided by anesthesiologists and other members of the OR leadership management. The selected metrics were chosen based on demonstrated factors contributing to OR costs and quality of care (4–7). As this is an ongoing dynamic project, current metrics could be changed or removed, and new metrics can be added. The scorecard contains metrics that assess OR efficiency, and these include metrics that fall within two parameters: time and quality. To obtain the appropriate metrics, we used hospital quality data to obtain ACTs and TOTs for individual providers as well as grouped statistics, which was grouped based on either all surgical cases or just cases in a specific surgical subdivisions. We used the ACT definition as established by the American Association of Clinical Directors (19). For each individual utilizing the scorecard, ACTs were calculated for each unique surgical case on record, utilizing a subset of timestamps that are routinely documented by the OR staff at our institution. These timestamps include: “Room Ready”, “Patient Into OR”, “Induction Complete”, “Surgery End”, and “Ready to Transfer”. From these timestamps, we were able to collect individual and grouped data for “average time to room ready and patient in room”, “average time to patient in room to induction complete”, and “average time to surgery end to extubation”. Other metrics were collected using hospital quality data, including number of cancelled cases, average TOT, average case time, case hours, amount of overtime, and average time to postanesthesia care unit (PACU) arrival and discharge. To measure quality of care, an important dimension of OR management, we collected provider data from performance evaluations and compliance to various protocols such as anesthesia record signing, time outs/checklist protocols, surgical antibiotics, and medication reconciliation. Total anesthetic costs per individual were also obtained through pharmacy databases. Each of these metrics were calculated for an individual, total group, and for each anesthesia subspecialty.
J Med Syst (2014) 38:144
Page 3 of 6, 144
Results
Discussion
Individual and group metrics were provided via a report card. This report card prototype consisted of metrics describing individual contribution to OR efficiency and costs. Institutional experiences and literature related to OR efficiency helped identify several factors that play an important role in defining the efficiency of an OR (4–6, 20–22). Figure 1 displays the format of the report card. Based on our prototype, the anesthesiologist should able to log onto this report card system at any time to review his/her metric scores for the quarter and how they compared to their peers (Fig. 2a). This can be further broken down based on surgical service (ie orthopedics, neurosurgery, cardiac) as different OR subspecialty areas can have different limitations and some anesthesiologists spent more time in one location versus other locations. Because frequent and updated feedback is likely more effective, the report card can be accessed at any time and allow the clinician to review current results over the past few months as well as compare to previous time periods and to the cumulative data from their colleagues. These real-time reports were additionally available to supervisors permitting them to monitor and intervene where inefficient OR practices are detected. By clicking on a specific metric, the user was provided with a graphical representation of their performance presented as a bell curve using data from the overall group or a surgical subspecialty (Fig. 2b).
Our prototype can lead to further development of the tools with the potential to gather accurate data on individual and group OR efficiency metrics. The tools can be designed to provide timely feedback that would hopefully lead to better quality and cost-effective anesthetic practices. Time and quality are two essential parameters to measure when assessing efficiency in the OR. The time parameters we chose to provide the best overall snapshot of efficiency include: “average time to room ready and patient in room”, “average time to patient in room to induction complete”, “average time to surgery end to extubation”, number of cancelled cases, average TOT, average case time, case hours, amount of overtime, and average time to PACU arrival and discharge. In addition to time saving, OR efficiency should also be based on quality, which can be defined as a measure of adherence to quality and compliance protocols. To measure quality, our scorecard presents information such as evaluations by other professional colleagues, as well as compliance in anesthesia record signing, time outs/ checklist use, and medication reconciliation. With our scorecard, OR managers can review efficiency on a quarterly and annual basis for each anesthesia provider. This data, in addition to improving OR efficiency, can provide a platform for research and improvement in clinical care and quality. However, in order for this system to improve OR efficiency, providers must regularly access and respond to the
Fig. 1 Operating Room Metrics Scorecard. This displays the general structure of the report card. The first column lists all the metrics measured for each individual and for the group. Data is presented as quarterly and annual averages, and is colorcoordinated
144, Page 4 of 6
J Med Syst (2014) 38:144
Fig. 2 Sample data from an anesthesiology provider. a. This displays real data obtained for an individual provider and how their quarterly and annual metric averages compare to the group based on cases performed in the Urology suite. Data that is increased compared to the group average
are colored in dark blue and dark orange for quarterly and annual averages, respectively. b. This is a sample graphical representation of provider data. The red-highlighted data point in A is previewed as a bell curve in B
information provided to them. Feedback is an invaluable aspect of the ongoing development of practitioners in health care. It is central to developing competence, professionalism, and confidence at all stages of a physician’s medical career. Despite the value inherent in effective feedback, there are integral challenges in implementing novel mechanisms of feedback into an existing system. The “buy-in” includes the view and expectation that feedback occurs on a routine basis, and those participating are held accountable (17). Feedback may often warrant negative connotations, resulting from providing constructive feedback and consequently damaging relationships with others (23). Furthermore, barriers to implementing a structured feedback system include cultural and financial infrastructures. These new systems need to be embraced and encouraged by senior members of a department. In additional, creating such as a system would require a financial investment into the IT infrastructure and continued technological support and development. There are major premises of successfully integrating feedback and establishing a culture of physician improvement (17). The purpose of effective feedback should focus on quality improvement of a health care group and thus should not be an isolated initiative. It should be viewed as part of a physician’s lifelong learning process. Feedback is best when it detects problems early on, providing information in real-time,
in an effort to curtail future problems. Finally, feedback data instruments should cater to the needs of the overall practicing group and not just the individual. We designed a report card that can provide anesthesiologists with an overview of their specific ACT averages, turnover times, anesthetic costs, and number of cancelled cases. These values are subsequently compared to group data and provide a graphical representation of how they compare statistically to their peers. Numerous characteristics that previously published literature and institutional experiences have shown to affect OR efficiency were included in this report card. The real-time data of this report card is available at anytime, is constantly updated, and thus provides feedback in a timely manner. The concept serves as a template to provide timely and effective feedback to improve OR efficiency at both the group and individual level. For example, if individual anesthesiologists can better visualize their costeffectiveness compared to others, this may encourage them to tailor their work to more efficient practices, provided they maintain patient safety and quality of care. The proposed feedback tool has its limitations, however. Some anesthesiologists, especially those trained in a particular subspecialty, may work the majority of their time in specific OR subspecialty areas, such as cardiac, thoracic, or orthopedic surgery. These subspecialty areas may have specific
J Med Syst (2014) 38:144
limitations in the measured OR efficiency data and thus are more or less prone to delays exemplified by various metrics (20). An example of a subspecialty-specific limitation is that anesthesiologists in the otorhinolaryngology area may on average have longer time periods between OR start to induction complete due the exposure to patients with more difficult airways. Another example is that anesthesiologists perform preoperative peripheral nerve blocks more often in the orthopedics area, which can understandably lead to increased TOT. Furthermore, some areas will require anesthetics that are consistently more costly than others. To address this limitation, our feedback system allows clinicians to not only have access to comparisons to the overall group, but also metrics based on subspecialty-specific cases in order to provide a more accurate peer to peer comparison. Occasionally, some anesthesiologists will feel that their caseloads consist of more complex cases and medically complicated patients, or patients who are more likely to have postoperative complications such as post-operative nausea and vomiting (PONV). The complexity of the case would require additional medications or procedures, such arterial or central line placement, which would add to total costs and time delays. The patient requiring anesthetic adjustments because of higher risk for PONV could mean more frequent utilization of total intravenous anesthetics and anti-emetic usage. Therefore, it is important to point out that the metrics reported should be an average over a specified time period in order to have a more accurate peer-to-peer comparison. Various health care groups have different methods of data allocation, collection and they utilize electronic medical record systems. This poses a challenge to creating a report that is universally available across the multitude of the utilized platforms. This highlights the IT challenges inherent in a tool of this type. In any case, it is still possible to integrate this concept into practices that have efficient means of collecting these metrics. Effective feedback should be an integral part of clinical practice. While the OR is often the largest contributor to a hospital’s financial success, it is also the most costly unit in this environment. OR efficiency is a high priority, and methods to improve the process are warranted. Providing real-time data to anesthesiologists regarding their contribution to OR costs is an effective method of monitoring inefficiencies and detecting patterns early in an effort to improve overall group OR metrics.
Page 5 of 6, 144
2.
3.
4.
5.
6.
7.
8.
9.
10.
11.
12.
13.
14.
15.
16.
17.
References 18. 1. Cima, R. R., Brown, M. J., Hebl, J. R., Moore, R., Rogers, J. C., Kollengode, A., et al., Use of lean and six sigma methodology to improve operating room efficiency in a high-volume tertiary-care
academic medical center. J Am Coll Surg 213(1):83–92, 2011. discussion 3–4. Epub 2011/03/23. Krupka, D. C., and Sandberg, W. S., Operating room design and its impact on operating room economics. Curr Opin Anaesthesiol 19(2): 185–91, 2006. Epub 2006/03/23. Kodali, B. S., Kim, D., Bleday, R., Flanagan, H., and Urman, R. D., Successful strategies for the reduction of operating room turnover times in a tertiary care academic medical center. J Surg Res 187(2): 403–11, 2014. Epub 2014/01/01. Friedman, D. M., Sokal, S. M., Chang, Y., and Berger, D. L., Increasing operating room efficiency through parallel processing. Ann Surg 243(1):10–4, 2006. Epub 2005/12/24. Harders, M., Malangoni, M. A., Weight, S., and Sidhu, T., Improving operating room efficiency through process redesign. Surgery 140(4): 509–14, 2006. discussion 14–6. Epub 2006/10/03. Stahl, J. E., Sandberg, W. S., Daily, B., Wiklund, R., Egan, M. T., Goldman, J. M., et al., Reorganizing patient care and workflow in the operating room: a cost-effectiveness study. Surgery 139(6):717–28, 2006. Epub 2006/06/20. Sokal, S. M., Craft, D. L., Chang, Y., Sandberg, W. S., and Berger, D. L., Maximizing operating room and recovery room capacity in an era of constrained resources. Arch Surg 141(4):389–93, 2006. discussion 93–5. Epub 2006/04/19. Kodali, B. S., Kim, K. D., Flanagan, H., Ehrenfeld, J. M., and Urman, R. D., Variability of subspecialty-specific anesthesia-controlled times at two academic institutions. J Med Syst 38(2):11, 2014. Epub 2014/01/29. Willig, J. H., Krawitz, M., Panjamapirom, A., Ray, M. N., Nevin, C. R., English, T. M., et al., Closing the feedback loop: an interactive voice response system to provide follow-up and feedback in primary care settings. Journal of medical systems 37(2):9905, 2013. Epub 2013/01/24. Benn, J., Arnold, G., Wei, I., Riley, C., and Aleva, F., Using quality indicators in anaesthesia: feeding back data to improve care. Br J Anaesth 109(1):80–91, 2012. Epub 2012/06/05. van der Veer, S. N., de Keizer, N. F., Ravelli, A. C., Tenkink, S., and Jager, K. J., Improving quality of care. A systematic review on how medical registries provide information feedback to health care providers. Int J Med Inform. 79(5):305–23, 2010. Epub 2010/03/02. de Vos, M., Graafmans, W., Kooistra, M., Meijboom, B., Van Der Voort, P., and Westert, G., Using quality indicators to improve hospital care: a review of the literature. Int J Qual Health Care 21(2): 119–29, 2009. Epub 2009/01/22. Hysong, S. J., Meta-analysis: audit and feedback features impact effectiveness on care quality. Med Care 47(3):356–63, 2009. Epub 2009/02/06. Jamtvedt, G., Young, J. M., Kristoffersen, D. T., O'Brien, M. A., and Oxman, A. D., Does telling people what they have been doing change what they do? A systematic review of the effects of audit and feedback. Qual Saf Health Care 15(6):433–6, 2006. Epub 2006/12/ 05. Veloski, J., Boex, J. R., Grasberger, M. J., Evans, A., and Wolfson, D. B., Systematic review of the literature on assessment, feedback and physicians’ clinical performance: BEME Guide No. 7. Med Teach 28(2):117–28, 2006. Mugford, M., Banfield, P., and O’Hanlon, M., Effects of feedback of information on clinical practice: a review. BMJ 303(6799):398–402, 1991. Epub 1991/08/17. Kaye, A. D., Okanlawon, O. J., and Urman, R. D., Clinical performance feedback and quality improvement opportunities for perioperative physicians. Adv Med Educ Pract. 5:115–23, 2014. Epub 2014/05/17. Peccora, C. D., Gimlich, R., Cornell, R. P., Vacanti, C. A., Ehrenfeld, J. M., and Urman, R. D., Anesthesia report card - a customizable tool for performance improvement. Journal of medical systems 38(9):105, 2014. Epub 2014/07/21.
144, Page 6 of 6 19. Glossary of times used for scheduling and monitoring of diagnostic and therapeutic procedures. AORN Journal. 1997;66(4):601–6. Epub 1997/10/24. 20. Kodali, B. S., Kim, K. D., Flanagan, H., Ehrenfeld, J. M., and Urman, R. D., Variability of subspecialty-specific anesthesia-controlled times at two academic institutions. Journal of Medical Systems 38(2):11, 2014. Epub 2014/01/29. 21. Friedman, D. M., Sokal, S. M., Chang, Y., and Berger, D. L., Increasing operating room efficiency through parallel processing. Annals of Surgery 243(1):10–4, 2006. Epub 2005/12/24.
J Med Syst (2014) 38:144 22. Sokal, S. M., Craft, D. L., Chang, Y., Sandberg, W. S., and Berger, D. L., Maximizing operating room and recovery room capacity in an era of constrained resources. Archives of Surgery 141(4):389–93, 2006. discussion 93–5. Epub 2006/04/19. 23. Heidegger, T., Husemann, Y., Nuebling, M., Morf, D., Sieber, T., Huth, A., et al., Patient satisfaction with anaesthesia care: development of a psychometric questionnaire and benchmarking among six hospitals in Switzerland and Austria. British Journal of Anaesthesia 89(6):863–72, 2002. Epub 2002/11/28.
