The American Journal of Surgery (2015) -, -–-

Editorial Opinion

Why the distribution of medical errors matters KEYWORDS: Medical errors; Gaussian; Power rule

Abstract During the last decade, interventions to reduce the number of medical errors have been largely ineffective. Although it is widely assumed that medical errors follow a Gaussian distribution, they may actually follow a Power Rule distribution. This article presents the evidence in favor of a Power Rule distribution for medical errors and then examines the consequences of such a distribution for medical errors. As the distribution of medical errors has real-world implications, further research is needed to determine whether medical errors follow a Gaussian or Power Rule distribution. Ó 2015 Elsevier Inc. All rights reserved.

Data from both sides of the Atlantic suggest that despite numerous interventions to improve the quality of health care during the last decade, there is little evidence of positive reform.1 For example, in the United States, a Rand Corporation study failed to demonstrate that Pay-for-Performance initiatives reduced medical error occurrences2; and in the United Kingdom, the large-scale Safer Patient Initiative ‘‘had no discernible additional effects on patient safety; care [improved] to the same extent in both the treatment and comparison hospitals.’’3 Worse, in some healthcare arenas, evidence actually exists which suggests that quality of care is deteriorating despite the fact that the healthcare providers have received extensive quality-of-care training.4 Although many explanations for our failure to reduce medical errors may exist, this article examines the possibility that our model for medical error reduction may be wrong. Part II reviews 2 models for medical distributions. Theoretically, medical errors should follow a Paretian or Power Rule distribution. On the other hand, virtually all interventions to reduce medical error assume that these errors follow a Gaussian or Bell-shaped distribution. Part III examines the behavioral economic arguments as to why we have systematically failed to consider that medical errors may not follow a Gaussian distribution. Part IV

The authors declare no conflicts of interest. * Corresponding author. Tel.: 11-913-526-5526; fax: 11-913-9620840. E-mail address: [email protected] Manuscript received September 2, 2014; revised manuscript September 30, 2014 0002-9610/$ - see front matter Ó 2015 Elsevier Inc. All rights reserved. http://dx.doi.org/10.1016/j.amjsurg.2014.10.032

addresses how behavioral economic techniques may be used to provide a nudge to get us to check our assumptions on how medical errors are distributed.

On the Nature of Medical Errors In its seminal paper on medical errors, the Institute of Medicine (IOM) stated that the delivery of health care is both complicated and complex.5 Healthcare delivery is complicated because of its multiple independent agents and clinical pathways. Healthcare delivery is complex because ‘‘one component of the system can interact with multiple other components’’ and these interactions are ‘‘time dependent.’’ According to the IOM, healthcare delivery can be ‘‘characterized by [its] specialization and interdependency’’ such that its ‘‘multiple feedback loops’’ are information sensitive. Next, the IOM defined a medical error as ‘‘a failure in the process of delivering care in a complex delivery system.’’ Given that healthcare delivery is both complicated and complex, a key question concerns how medical errors are distributed. Many medical error reduction studies have simply and explicitly assumed that medical errors follow a Gaussian distribution (or a Poisson distribution if the study’s authors consider a ‘‘ medical error’’ to be a discrete variable).6 Even more administrative intervention studies concerning medical error reduction implicitly assumed that medical error occurrences follow a Gaussian distribution.7 Given that at the time of the IOM’s report, quality improvement in the industrial and business services sectors was predicated on the reduction of variation, or more

2

The American Journal of Surgery, Vol -, No -, - 2015

specifically standard deviation (s or sigma),8 assuming medical errors were distributed along a bell-shaped curve was not unreasonable. (Herein, the term ‘‘sigma reduction’’ refers to any error reduction strategy or technique that assumes that medical errors follow a bell-shaped curve distribution and then attempts to reduce the incidence of medical errors by narrowing the standard deviation.)9 Still, times are changing. Taleb, in his 2013 book ‘‘Antifragile,’’ observed that errors in information-sensitive complicated and complex systems are distributed according to a Power Law distribution. A priori, Taleb’s observation implies that if healthcare delivery occurs as the IOM has described it, than medical errors should be distributed according to a Power Law. Similarly, O’Boyle demonstrated that across a wide range of industries and job types, ‘‘individual performance is not normally distributeddinstead, it follows a Paretian [Power Law] distribution.’’10 O’Boyle added that ‘‘[w]hen performance data do not conform to the normal distribution, then [researchers often conclude] that the error ‘must’ lie within the sample not the population.’’ That is, when researchers obtain unanticipated results, they often assume something is wrong with their data collection rather than check their assumptions. The works by Taleb and O’Boyle should encourage healthcare providers to re-examine their assumptions on how medical errors are distributed. Re-examining our assumptions about medical error distribution is limited by a paucity of public information concerning the occurrence of medical errors at the individual, hospital, state, and national levels.11 Therefore, to work around this knowledge limitation, we need to examine the distinguishing features of Power Law and Gaussian distributions, as well as employ the use of medical error proxies (eg, physician discipline and medical malpractice claims) for medical errors. Three factors distinguish Paretian and Gaussian distributions. Events that follow Gaussian distributions occur independently of each other, and this independence allows for both the reliable calculation of mean and statistical analysis based on the variance. The Gaussian distribution’s bell-shaped curve means that events that occur more than 3s beyond the mean are so rare that it is reasonable to assume that these events do not occur. Finally, a Gaussian distribution is scale dependent, that is, the shape of the distribution curve depends on the section of the curve under review. In contrast, because the interdependent events of a Power Law distribution map to a parabolic curve, the concept of a mean and variance is undermined.12 Moreover, the ‘‘fat tails’’ of a Power Law distribution means that occurrences that are more than 3s beyond the mean can never be assumed to be zero.13 Finally, both Mendelbrot and O’Boyle have observed that Power Law distributions demonstrate ‘‘scale invariance,’’ that is, the curve of a Power Law distribution looks the same when it is magnified ten, a hundred, or a thousand times. When combined with medical error proxy data, whether an occurrence curve is scalable or not can be used as a tool

to assess the distribution of medical errors. Consider first how medical error proxies are distributed at the provider level. Bismark found that the number of complaints per physician that a medical board received closely follows an 80-20 Paretian distribution.14 In an observational study of more than 9,000 medical malpractice claims, Rolph demonstrated that the number of claims filed against physicians closely adhered to an 80-20 Paretian distribution15da finding that has been observed by others16 including the National Practitioner Data Bank.17 Like physicians, hospitals are leery of reporting serious medical errors.18 Not surprisingly, finding hard medical error data for hospitals is not easy. Therefore, we need to turn to proxy data again. Consider hospital quality data from the Leapfrog Group, which is an employer-based coalition that advocates for improved hospital care by promoting transparency.19 Leapfrog ranks hospitals from ‘‘A’’ to ‘‘F’’ depending on the weighted measurements of 28 publically available quality metrics that reflect medical errors.20 If we consider Leapfrog’s data for 276 California hospitals, we find that 123 hospitals were considered A or relatively error free quality, 63 hospitals were ‘‘B’’ quality, 66 hospitals were considered ‘‘C’’ quality, 18 hospitals were consider ‘‘D’’ quality, 0 hospitals were consider ‘‘E’’ quality, and 6 hospitals were considered ‘‘F’’ or error prone quality.21 Although these data do not produce a perfect parabolic distribution curve that is consistent with a Power Law distribution, the data definitely do not conform to a bell-shaped distribution curve. At the state level, Leapfrog’s data also suggest that medical errors are distributed according to a Power Law. Fig. 1 demonstrates the percentage of hospitals in 50 reporting states that received an A rating.22 The above data strongly suggest that for physicians, hospitals, and states the distribution of medical errors is

Figure 1 Distribution of medical errors for hospitals in 50 reporting states. The x-axis displays the percentage of hospital in each state that received an A rating from Leapfrog. The data are displayed from the state with the highest score (Maine) to the lowest (New Mexico and Maryland). See text for details. Note: Only 48 bars appear because New Mexico and Maryland reported no hospitals receiving an A rating.

T.R. McLean

Distribution of medical errors

parabolic and shape invariant. These findings are not definitive, but it appears that the distribution of medical errors follows a Power Law distribution. Our preference to assume that medical errors are distributed according to a Gaussian curve therefore appears to reflect convenience. Yet, one must realize that assuming phenomena follow a Gaussian distribution when, in fact, the phenomena follow a Paretian distribution can have disastrous consequences.23

Medical Error Reduction Strategy If further research does demonstrate that medical errors follow a Paretian distribution, it could explain why the medical error interventions of the last decade have failed to reduce patient suffering. In short, we may have been applying the wrong model to medical error interventions. Therefore, besides mathematical convenience, it is import to consider other factors that keep us using Gaussian-based error reduction interventions. Behavioral economics can assist us with this analysis. In particular, behavioral economics teaches that ‘‘group think’’ tends to blind researchers to flaws in their assumptions,24 leading researchers ‘‘to overestimate how much we understand about the world and to underestimate the role of chance in events.’’ Researchers’ blindness is compounded by the ‘‘law of least effort,’’ which asserts ‘‘that if there are several ways of achieving the same goal, people will eventually gravitate to the least demanding course of action.’’ This law of least efforts has 2 corollaries. First, because overriding an intuitively plausible answer or conclusion requires hard work, many researchers do not take the time to think the problem through. The second corollary to this law is that when we must choose between alternatives, ‘‘recent events and the current context have the most weight in determining an interpretation’’ because ‘‘the consistency of the information that matters in a good story, not its completeness.’’ These principles also tend to explain how medical error reduction research has become corrupted. For example, contrary ‘‘to the rule of philosophers of science, who advise testing hypotheses by trying to refute them people . seek data that are likely to be compatible with the beliefs they currently hold.’’ Similarly, according to Kahneman, consensus papers and editorials can be misleading if the author(s) share a bias, the aggregation of judgments will not reduce it. . [Accordingly to] derive the most useful information from multiple sources of evidence you should always try to make these sources independent of each other. . The standard practice of open discussion gives too much weight to the opinions of those who speak early and assertively.

Applying these principles to medical error research goes a long way toward explaining our reluctance to jettison Gaussian-based error reduction techniques. Gaussian-based error reduction strategy (ie, sigma reduction strategy) was

3 first developed in the industrial section, and then widely adopted by the business services sector.25 Accordingly, in the wake of the IOM’s 1999 publishing of ‘‘To Err is Human,’’ the conventional group think among healthcare reformers was to apply sigma reduction strategy to the healthcare field, even though healthcare delivery is fundamentally different from the manufacturing and service industries because healthcare providers cannot specify the initial conditions of their patients. That is, the success of sigma reduction techniques in other fields allowed healthcare reformers ‘‘to overestimate’’ their understanding of nature of medical errors. Against this backdrop, the ‘‘law of least effort’’ creates substantial inertia to jettison sigma reduction techniques from the healthcare sector. In the healthcare sector, we hold sigma reduction techniques dear not only because their mathematics is easier than Paretian analysis, but also we are bombarded with information concerning the success of such techniques in other fields. In contemplating how to solve the complex problem of medical error reduction, we use an ‘‘availability heuristic’’ that directs us to look at what is currently available and find our answer there. Indeed, if one were to argue that error reduction in the healthcare sector is somehow different from industry and the business services sector, it would destroy ‘‘the consistency’’ of general applicability of the sigma reduction narrative. In short, when we are confronted with our inability to reduce medical errors by sigma reduction strategies, we have systematically failed to check our assumptions about medical errors. Worse, we continue to ‘‘double down’’ on this model in the healthcare sector as we continue to tweak the application of this model. That is, rather than check our fundamental assumptions about medical errors, we seek information not to refute the Gaussian distribution model, but rather information to confirm this model. How doubling-down on the sigma reduction model for medical error reduction leads only to further failure is well illustrated by interventions to reduce wrong site surgery (WSS). WSS occurs when a surgeon operates on the wrong anatomic location. A number of administrative interventions to reduce WSS have been introduced (eg, the use of checklists and standardized preoperative procedures).26 Characteristic of these interventions is the fact that they are designed to reduce variance and increase provider conformity. Accordingly, WSS interventions are aimed at reducing the number of these events by implicitly assuming a Gaussian distribution. This assumption ignores the fact that the interpersonal interactions between operating room personnel may create short- and long-term dependencies that would create a Paretian distribution. Therefore, it should come as no surprise that a decade’s long employment of sigma reduction techniques to WSS avoidance has failed to reduce the incidence of this medical error.27 Worse, the number of administrative procedures to combat WSS has only increased the complexity of performing surgery.

4

The American Journal of Surgery, Vol -, No -, - 2015

What Is to Be Done

handling medical error data will need to change. We may find that outcomes are improved if we learn to tolerate the more frequent ‘‘minimal impact’’ medical errors to avoid the less common ‘‘major impact’’ medical errors.33 For example, the outcomes for laparoscopic cholecystectomies may improve if we focus our attention on the number of procedures a surgeon preforms ‘‘without’’ a bile duct injury rather than worrying about how many near misses the surgeon experienced during the identification of the bile ducts. Alternatively, if we want to reduce the incidence of Power Law distributed medical errors, then it may be necessary to remove some of the complexities from our healthcare system.34 For example, we may want to rethink the Joint Commission’s Universal Protocol before surgery. This procedure adds complexity to the start of every surgical case as everyone in the operating room is required to stop to review the planned operation. Yet at the time of the surgical procedure, if the surgical site was mismarked before surgery, then because everyone depends on the affirmations of one or 2 individuals concerning the appropriateness of the surgical consent and the marking of the patient, application of the Universal Protocol in the operating room cannot always avoid a sentinel event. Therefore, we would have better outcomes if the Universal Protocol was simplified and we increased the penalties for negligently obtaining surgical consent and marking patients. Finally, the most important implication for a Power Rule distribution of medical errors is the recognition that such errors arise from the interdependence of several healthcare providers and not the surgeon alone. It is no one’s interest to blame surgeons for events out of their control.

Behavioral economics also provides some insight on what should be done. Thaler and Sunstein28 have observed that a ‘‘nudge’’ can be useful in solving behavioral economics problems. A nudge, as used by these investigators, refers to ‘‘any aspect of the choice architecture that alters people’s behavior in a predictable way without forbidding any options or significantly changing their economic incentives.’’ There are 6 potential nudge options open to choice architects in reducing medical errors. These options can be remembered by the mnemonic ‘‘NUDGES,’’ which stands for iNcentives, Understanding mapping, Defaults, Give feedback, Expect errors, Structure complex choices. Of these nudge options, 4 will not be discussed further here. Common to understanding mapping, setting defaults, anticipation of human error and structuring choices concern system modification is that they are all system specific. Absent a specific system to discuss, there is little to discuss. However, when designing future medical error reduction research, these nudges should be kept in mind. Incentives are a more interesting story. Regardless of whether medical errors follow a Gaussian or Paretian distribution, the publication of provider-specific medical error rates creates reputational incentives for providers.29 Providers who want to stay in the market must avoid being tagged as low-quality provider. Consequently, healthcare providers avoid high-risk patients thereby lowering healthcare cost by lowering the volume of services provided.30 To the degree that sigma reduction techniques fail to reduce medical errors, the increasing numbers of medical errors will only increase the reputational incentives on healthcare providers. This is a good thing if we are interested in controlling healthcare costs. Conversely, it seems unlikely that we can create incentives sufficient to get patient safety advocates to re-examine their assumptions on the distribution of medical errors. The reason is simple: Just as attorneys and accountants favor a complex tax code to protect their livelihood,31 the livelihood of healthcare experts in sigma reduction technology depends on the continuation of, and expansion of, sigma reduction techniques in medical error analysis. This brings us to the last nudge option: feedback. When feedback is ‘‘expressed confidently,’’ it can undermine group think-based conclusions. Feedback holds the potential for stimulating a ‘‘bandwagon effect’’ that can overcome the inertia of the status quo. The purpose of this article is to provide some feedback to patient safety advocates and specifically request that they review their assumptions concerning the distribution of medical errors. To do this, patient safety advocates will need to take a ‘‘big data’’ approach to analyzing large numbers of medical error occurrences.32 If subsequent research does suggest that medical errors are distributed according to a Power Law, our method of

Conclusions Increasingly, commentators are questioning whether patient safety initiatives (especially the ‘‘shot-gun’’ approach that applies a multistep process to achieve a single goal) cause the intended outcome or whether the outcome is because of Hawthorn Effect.35 Other commentators have pointed out that the positive outcomes of patient safety initiatives are merely manifestations of the statistical phenomena of ‘‘regression to the mean,’’ and as such are not evidence that the safety interventions actually caused the outcome ascribed to them.36 Herein, we have advanced another reason why patient safety initiatives may fail: the assumption that medical error occurrences are distributed along a bell-shaped curve may be wrong. Accordingly, we have attempted to provide thought-provoking feedback because it seems that the distribution of medical errors matters. Thomas R. McLean, M.D., J.D.* Third Millennium Consultants, LLC, 4970 Park, Shawnee, KS 66216, USA

T.R. McLean

Distribution of medical errors

References 1. Landrigan CP, Parry GJ, Bones CB, et al. Temporal trends in the trends of patient harm resulting from medical care. N Engl J Med 2010;363: 2124–34. 2. Mullen KJ, Frank RG, Rosenthal MB. Can you get what you pay for? Pay-for-performance and the quality of healthcare providers. Rand J Econ 2010;41:64–91. 3. Pronovost PJ, Berenholtz SM, Morlock LL. Is quality of care improving in the UK? BJM 2011;342:341–2. 4. Landro L. Step Rise in Complications of Childbirth Spurs Action. Wall Street Journal, Dec 10, 2012 at D1. 5. Institute of Medicine. To Err Is Human: Building a Better Healthcare System. Washington, DC: National Academy Press; 2000. 6. Anderson JG. The need for organizational change in patient safety initiatives. Int J Med Inform 2006;75:809–17. 7. Harrington HJ, Trusko B. A Bigger Problem than Terrorismd Healthcare Errors: Using Total Six Sigma. Harrington Institute; 2004 at 29. 8. Jaap van Ede CJ. Reducing Variation Improves Quality. BusinessImprove Eu; 2012. Available at: http://www.business-improvement. eu/six_sigma/six_sigma_eng.php; 2012. Accessed September 30, 2014. 9. Wiesenfelder H. Understanding Variation: Common Cause & Special Cause. Bright Hub PM; 2013. Available at: http://www. brighthubpm.com/six-sigma/27921-understanding-variation-commoncause-and-special-cause/; 2013. Accessed September 25, 2014. 10. O’Boyle E, Aguinis H. The best and the rest: revisiting the norm of normality of individual performance. Personnel Psychol 2012;65:79–119. 11. Levinson DR. Adverse Events in Hospitals: National Incidence among Medicare Beneficiaries. Department of Health and Human Services; 2010. Available at: https://oig.hhs.gov/oei/reports/oei-06-09-00090. pdf; 2010. Accessed September 30, 2014. 12. Ball P. Don’t panic, it might never happen. Nature. Available at: http:// www.nature.com/news/2005/050214/full/news050214-20.html; 2005. Accessed September 30, 2014. 13. Mendelbrot B, Hudson RL. The Misbehavior of Markets: A Fractal View of Financial Turbulence. New York, NY: Basic Books; 2006. 14. Bismark MM, Spittal MJ, Studdert DM. Prevalence and characteristics of complaint-prone doctors in private practice in Victoria. Med J Aust 2011;195:25–8. 15. Rolph JE, Adams JL, McGuigan KA. Identifying malpractice-prone physicians. J Empirical Legal Stud 2007;4:125–53. 16. Thomas M. Bad Apples Hurt Doctors’ Call for Malpractice Cap. Orlando Sentinel; April 6, 2003 (internet edition). 17. American Society of Anesthesiologists. Syllabus on Ethics Available at: http://education.asahq.org/sites/education.asahq.org/files/users/ 1392/2012-ethics-syllabus.pdf#page511; 2012. Accessed September 30, 2014.

5 18. Bernard B, Kohler J. Hospitals Are Leery of Reporting Serious Errors. St. Louis Dispatch; 2011 (internet edition). Available at: http://m.stltoday.com/news/local/metro/hospitalsleery-of-reporting-serious-errors/article_9403fd2a-b2f4-5aed-9f081d92dc9165b7.html?mobile_touch5true. 19. The Leapfrog Group. Home Page Available at: http://leapfroggroup. org/; 2013. Accessed September 30, 2014. 20. The Leapfrog Group. Hospital Safety Score Measurements in Detail Available at: http://www.hospitalsafetyscore.org/about-the-score/measures-indetail; 2013. Accessed September 30, 2014. 21. The Leapfrog Group. Hospital Safety Score Hospitals Near You Available at: http://www.hospitalsafetyscore.org/search-result?zip_code5 &hospital5&city5&state_prov5CA&agree5agree; 2013. Accessed September 30, 2014. 22. The Leapfrog Group. Hospital Safety Score State Ratings Available at: http://www.hospitalsafetyscore.org/state-rankings; 2013. Accessed September 30, 2014. 23. Taleb NN. Black Swan: The Impact of the Highly Improbable. New York, NY: Random House; 2007. 24. Kahneman Daniel. Thinking Fast and Slow. New York, NY: Farrar, Straus, and Giroux; 2012. 25. McLean TR. Implication of Patient Safety Research & Risk Managed Care. 2002; Southern Illinois University Law Review. 26: 227–281. 26. O’Reilly KB: Wrong-site Surgeries Risk Reduced during Pilot Project. AMA News July 18, 2011; Available at: http://www.ama-assn.org/ amednews/2011/07/18/prsb0718.htm. Accessed September 30, 2014. 27. Stahel PF, Sabel AL, Victoroff MS, et al. Wrong-site and wrongpatient procedures in the universal protocol era. Arch Surg 2010; 145:978–84. 28. Thaler RH, Sunstein CR. Nudge. New Haven, CT: Yale University Press; 2008. 29. McLean TR. Medical rationing: the implicit results of leadership by example. J Health Law 2003;26:325–63. 30. McLean TR. In New York State, do more percutaneous coronary interventions mean fewer or more complex referrals to cardiac surgeons? Am Heart Hosp J 2008;6:30–6. 31. Johnston DC. Fine Print. New York, NY: Portfolio; 2012. 32. Mayer-Scho¨nberger V, Cukier K. A Revolution that Will Transform How We Live, Work, and Think By. New York, NY: Portfolio; 2013. 33. Silver N. The Signal and the Noise Why So Many Predictions Fail but Some Don’t. New York, NY: The Penguin Press; 2012. 34. Rickards J. Currency wars: the making of the next global crisis. New York, NY: Portfolio Trade; 2012. 35. Clancy CM, Berwick DM. The science of safety improvement: learning while doing. Ann Int Med 2011;154:699–700. 36. Ted Countermanche. The 80-20 Rule and Why It’s Misleading in Health Care. Phytel Ap 2. Available at: http://blog.phytel.com/2011/04/the-8020-rule-and-why-its-misleading-in-health-care; 2011. Accessed December 31, 2011.

Why the distribution of medical errors matters.

During the last decade, interventions to reduce the number of medical errors have been largely ineffective. Although it is widely assumed that medical...
295KB Sizes 1 Downloads 13 Views