IDEAS AND OPINIONS
Annals of Internal Medicine
The Lake Wobegon Effect: Why Most Patients Are at Below-Average Risk Andrew J. Vickers, DPhil, and David M. Kent, MD, CM, MSc
n Garrison Keillor's fictional Midwestern town of Lake Wobegon featured in A Prairie Home Companion, “all the women are strong, all the men are good-looking, and all the children are above average.” It might also seem deliberately paradoxical to say that “most patients are at below average risk” or that “most patients within a trial experience less than the average level of benefit from a drug.” But these statements are true more often than not (1). To start with a simple example, the lifetime risk that an American will develop lung cancer is about 7.5%. But because smoking strongly increases the risk for lung cancer, the average risk of 7.5% can be stratified into smokers, who have a risk of 15% to 20%, and nonsmokers, who have a risk of around 1%. Because 60% to 70% of the population has never smoked, most Americans are substantially below the average risk for lung cancer. This is naturally a simplification because the probability of lung cancer is not the same for all smokers. Statisticians have created prediction models presenting lung cancer risk in terms of age, sex, pack-years of smoking, and current smoking status (2). The math underlying these prediction models is typically based on logarithms, and as such, the risk distribution will almost always be skewed to the right when the overall outcome rate is less than 50%. The Figure shows the distribution of risk from a typical prediction model. In this hypothetical example, 10% of patients develop disease, so the mean risk is 10%. The median risk is closer to 7%, and about two thirds of patients have a risk less than the mean. The implications of this “Lake Wobegon effect” are far from trivial. Take a typical randomized trial taught in an introductory evidence-based medicine class, with cardiovascular event rates of 10% in the control group compared with 5% in the drug group. This is a 5% absolute risk reduction with a number needed to treat of 20. Let us assume that most clinicians think that the burdens, costs, adverse effects, and risks of the drug are such that it would, in fact, be worth treating 20 patients, although no more than 20, to prevent 1 cardiovascular event. As such, the trial is deemed a success and the drug is widely prescribed. But let us further assume that a prediction model is available that can be used to determine the risk of an individual patient on the basis of risk factors, such as blood pressure and cholesterol level, with similar properties to the model shown in the Figure. If the relative risk reduction is roughly constant across different levels of absolute risk (50%), and if the absolute risk for harms from drug therapies does not vary importantly between patients, then This article was published online first at www.annals.org on 14 April 2015. 866 © 2015 American College of Physicians
Downloaded From: https://annals.org/ by a Columbia University User on 08/07/2017
only about one third of patients benefit sufficiently from the drug to outweigh its costs, burdens, and harms. In other words, although the treatment is worthwhile for patients on average, it is not worthwhile for the average patient. We have examined this effect empirically in patients with ST-segment elevation myocardial infarction. Using a previously developed risk model, we found that patients in the highest risk quartile have about a 16-fold risk for death compared with patients in the lowest risk quartile. In contrast, the typical patient has a risk that is only about half of the average. When we reanalyzed the GUSTO (Global Utilization of Streptokinase and tPA for Occluded coronary arteries) trial (3) using this risk model, we found that the more potent and expensive thrombolytic therapy, tissue plasminogen activator, was indeed effective on average. But for most patients, the degree of benefit probably did not warrant the extra risks and costs compared with the less effective but safer and less expensive alternative, streptokinase. In a similar study, we found that primary angioplasty saves more lives on average than thrombolytic therapy, but not in patients with typical risk; up to 75% of patients with ST-segment elevation derive no mortality benefit from angioplasty (1, 4, 5). Because such risk-stratified analyses are rare, the emphasis placed on the average summary results can lead to lowvalue care and overtreatment in many persons, particularly for treatments that carry substantial risks or costs. Figure. The distribution of risk in a typical prediction model.
Population Density
I
0
5
10
25
50
Risk, %
The mean risk is 10%. The median risk is 7%. The dark shaded area constitutes the 50% of patients with risk less than the median; the light shaded area represents the patients with risk greater than the mean (about one third). Approximately two thirds of patients have risk less than the mean.
IDEAS AND OPINIONS
The Lake Wobegon Effect
Perhaps the ground zero of overtreatment in contemporary medicine is prostate cancer. Population data show that, after the introduction of screening using the prostate-specific antigen (PSA) test, mortality decreased somewhat but incidence and treatment increased dramatically. We believe that typical approaches to prostate cancer screening, which assume that all men are at average risk, are a major cause of overdiagnosis. In fact, risk can be very strongly separated depending on PSA levels. Men in the top quartile of PSA levels at age 60 years (≥2 ng/mL) have a 20times greater risk for prostate cancer death than do those with lower PSA levels, and 90% of deaths by age 85 years occur in this group (6)—a clear example of the Lake Wobegon effect. We recently showed that screening only men at high risk rather than screening all men drastically reduced screening harms, in terms of overdiagnosis, but retained 100% of the screening benefits, in terms of mortality reductions, because screening did not reduce prostate cancer deaths in the group with low PSA levels (7). Of course, the distribution of risk is a modeldependent property. Although risk models are available for many clinically important outcomes, their implementation in research and clinical domains has been limited and barriers to wider use are not insubstantial. The relationship between harms and benefits might also be complex because it is likely that in some cases the risk for harms is somewhat correlated with the degree of benefit. Nevertheless, it is clear that the Lake Wobegon effect—the apparently paradoxical finding that most patients are at below-average risk and expect to have less-than-average benefit from treatment—is a common and underappreciated phenomenon in medicine with important clinical implications. In many cases, too many patients are screened, diagnosed, and treated. A better understanding of this effect, and a better use of risk prediction in both research and clinical practice, will be essential to ensure that we focus our attention on patients who stand most to gain. From Memorial Sloan Kettering Cancer Center, New York, New York, and Tufts Medical Center, Boston, Massachusetts. Grant Support: In part by funds from David H. Koch provided
through the Prostate Cancer Foundation, the Sidney Kimmel Center for Prostate and Urologic Cancers, and the National Cancer Institute (P50-CA92629 [Specialized Programs of Re-
www.annals.org
Downloaded From: https://annals.org/ by a Columbia University User on 08/07/2017
search Excellence grant] and P30-CA008748 [to Memorial Sloan Kettering Cancer Center]). Additional support was provided by the National Institutes of Health (grant U01 NS086294), the Patient-Centered Outcomes Research Institute (grant 1IP2PI000722), and a Clinical & Translational Science Awards U-award (UL1 TR001064). Disclosures: Disclosures can be viewed at www.acponline
.org/authors/icmje/ConflictOfInterestForms.do?msNum=M14 -2767. Requests for Single Reprints: Andrew J. Vickers, DPhil, Memo-
rial Sloan Kettering Cancer Center, 307 East 63rd Street, 2nd Floor, New York, NY 10065; e-mail, [email protected]. Current author addresses and author contributions are available at www.annals.org. Ann Intern Med. 2015;162:866-867. doi:10.7326/M14-2767
References 1. Kent DM, Hayward RA. Limitations of applying summary results of clinical trials to individual patients: the need for risk stratification. JAMA. 2007;298:1209-12. [PMID: 17848656] 2. Bach PB, Gould MK. When the average applies to no one: personalized decision making about potential benefits of lung cancer screening. Ann Intern Med. 2012;157:571-3. [PMID: 22893040] doi: 10.7326/0003-4819-157-8-201210160-00524 3. Kent DM, Hayward RA, Griffith JL, Vijan S, Beshansky JR, Califf RM, et al. An independently derived and validated predictive model for selecting patients with myocardial infarction who are likely to benefit from tissue plasminogen activator compared with streptokinase. Am J Med. 2002;113:104-11. [PMID: 12133748] 4. Kent DM, Schmid CH, Lau J, Selker HP. Is primary angioplasty for some as good as primary angioplasty for all? J Gen Intern Med. 2002;17:887-94. [PMID: 12472924] 5. Thune JJ, Hoefsten DE, Lindholm MG, Mortensen LS, Andersen HR, Nielsen TT, et al; Danish Multicenter Randomized Study on Fibrinolytic Therapy Versus Acute Coronary Angioplasty in Acute Myocardial Infarction (DANAMI)-2 Investigators. Simple risk stratification at admission to identify patients with reduced mortality from primary angioplasty. Circulation. 2005;112:2017-21. [PMID: 16186438] 6. Vickers AJ, Cronin AM, Bjo¨rk T, Manjer J, Nilsson PM, Dahlin A, et al. Prostate specific antigen concentration at age 60 and death or metastasis from prostate cancer: case-control study. BMJ. 2010;341: c4521. [PMID: 20843935] doi:10.1136/bmj.c4521 7. Carlsson S, Assel M, Sjoberg D, Ulmert D, Hugosson J, Lilja H, et al. Influence of blood prostate specific antigen levels at age 60 on benefits and harms of prostate cancer screening: population based cohort study. BMJ. 2014;348:g2296. [PMID: 24682399] doi:10.1136 /bmj.g2296
Annals of Internal Medicine • Vol. 162 No. 12 • 16 June 2015 867
Annals of Internal Medicine Current Author Addresses: Dr. Vickers: Department of Epidemiology and Biostatistics, Memorial Sloan Kettering Cancer Center, 485 Lexington Avenue, 2nd Floor, New York, NY 10017. Dr. Kent: Institute for Clinical Research and Health Policy Studies, Tufts Medical Center, 800 Washington Street, Boston, MA 02111.
www.annals.org
Downloaded From: https://annals.org/ by a Columbia University User on 08/07/2017
Annals of Internal Medicine • Vol. 162 No. 12 • 16 June 2015
Annals of Internal Medicine
The Lake Wobegon Effect: Why Most Patients Are at Below-Average Risk Andrew J. Vickers, DPhil, and David M. Kent, MD, CM, MSc
n Garrison Keillor's fictional Midwestern town of Lake Wobegon featured in A Prairie Home Companion, “all the women are strong, all the men are good-looking, and all the children are above average.” It might also seem deliberately paradoxical to say that “most patients are at below average risk” or that “most patients within a trial experience less than the average level of benefit from a drug.” But these statements are true more often than not (1). To start with a simple example, the lifetime risk that an American will develop lung cancer is about 7.5%. But because smoking strongly increases the risk for lung cancer, the average risk of 7.5% can be stratified into smokers, who have a risk of 15% to 20%, and nonsmokers, who have a risk of around 1%. Because 60% to 70% of the population has never smoked, most Americans are substantially below the average risk for lung cancer. This is naturally a simplification because the probability of lung cancer is not the same for all smokers. Statisticians have created prediction models presenting lung cancer risk in terms of age, sex, pack-years of smoking, and current smoking status (2). The math underlying these prediction models is typically based on logarithms, and as such, the risk distribution will almost always be skewed to the right when the overall outcome rate is less than 50%. The Figure shows the distribution of risk from a typical prediction model. In this hypothetical example, 10% of patients develop disease, so the mean risk is 10%. The median risk is closer to 7%, and about two thirds of patients have a risk less than the mean. The implications of this “Lake Wobegon effect” are far from trivial. Take a typical randomized trial taught in an introductory evidence-based medicine class, with cardiovascular event rates of 10% in the control group compared with 5% in the drug group. This is a 5% absolute risk reduction with a number needed to treat of 20. Let us assume that most clinicians think that the burdens, costs, adverse effects, and risks of the drug are such that it would, in fact, be worth treating 20 patients, although no more than 20, to prevent 1 cardiovascular event. As such, the trial is deemed a success and the drug is widely prescribed. But let us further assume that a prediction model is available that can be used to determine the risk of an individual patient on the basis of risk factors, such as blood pressure and cholesterol level, with similar properties to the model shown in the Figure. If the relative risk reduction is roughly constant across different levels of absolute risk (50%), and if the absolute risk for harms from drug therapies does not vary importantly between patients, then This article was published online first at www.annals.org on 14 April 2015. 866 © 2015 American College of Physicians
Downloaded From: https://annals.org/ by a Columbia University User on 08/07/2017
only about one third of patients benefit sufficiently from the drug to outweigh its costs, burdens, and harms. In other words, although the treatment is worthwhile for patients on average, it is not worthwhile for the average patient. We have examined this effect empirically in patients with ST-segment elevation myocardial infarction. Using a previously developed risk model, we found that patients in the highest risk quartile have about a 16-fold risk for death compared with patients in the lowest risk quartile. In contrast, the typical patient has a risk that is only about half of the average. When we reanalyzed the GUSTO (Global Utilization of Streptokinase and tPA for Occluded coronary arteries) trial (3) using this risk model, we found that the more potent and expensive thrombolytic therapy, tissue plasminogen activator, was indeed effective on average. But for most patients, the degree of benefit probably did not warrant the extra risks and costs compared with the less effective but safer and less expensive alternative, streptokinase. In a similar study, we found that primary angioplasty saves more lives on average than thrombolytic therapy, but not in patients with typical risk; up to 75% of patients with ST-segment elevation derive no mortality benefit from angioplasty (1, 4, 5). Because such risk-stratified analyses are rare, the emphasis placed on the average summary results can lead to lowvalue care and overtreatment in many persons, particularly for treatments that carry substantial risks or costs. Figure. The distribution of risk in a typical prediction model.
Population Density
I
0
5
10
25
50
Risk, %
The mean risk is 10%. The median risk is 7%. The dark shaded area constitutes the 50% of patients with risk less than the median; the light shaded area represents the patients with risk greater than the mean (about one third). Approximately two thirds of patients have risk less than the mean.
IDEAS AND OPINIONS
The Lake Wobegon Effect
Perhaps the ground zero of overtreatment in contemporary medicine is prostate cancer. Population data show that, after the introduction of screening using the prostate-specific antigen (PSA) test, mortality decreased somewhat but incidence and treatment increased dramatically. We believe that typical approaches to prostate cancer screening, which assume that all men are at average risk, are a major cause of overdiagnosis. In fact, risk can be very strongly separated depending on PSA levels. Men in the top quartile of PSA levels at age 60 years (≥2 ng/mL) have a 20times greater risk for prostate cancer death than do those with lower PSA levels, and 90% of deaths by age 85 years occur in this group (6)—a clear example of the Lake Wobegon effect. We recently showed that screening only men at high risk rather than screening all men drastically reduced screening harms, in terms of overdiagnosis, but retained 100% of the screening benefits, in terms of mortality reductions, because screening did not reduce prostate cancer deaths in the group with low PSA levels (7). Of course, the distribution of risk is a modeldependent property. Although risk models are available for many clinically important outcomes, their implementation in research and clinical domains has been limited and barriers to wider use are not insubstantial. The relationship between harms and benefits might also be complex because it is likely that in some cases the risk for harms is somewhat correlated with the degree of benefit. Nevertheless, it is clear that the Lake Wobegon effect—the apparently paradoxical finding that most patients are at below-average risk and expect to have less-than-average benefit from treatment—is a common and underappreciated phenomenon in medicine with important clinical implications. In many cases, too many patients are screened, diagnosed, and treated. A better understanding of this effect, and a better use of risk prediction in both research and clinical practice, will be essential to ensure that we focus our attention on patients who stand most to gain. From Memorial Sloan Kettering Cancer Center, New York, New York, and Tufts Medical Center, Boston, Massachusetts. Grant Support: In part by funds from David H. Koch provided
through the Prostate Cancer Foundation, the Sidney Kimmel Center for Prostate and Urologic Cancers, and the National Cancer Institute (P50-CA92629 [Specialized Programs of Re-
www.annals.org
Downloaded From: https://annals.org/ by a Columbia University User on 08/07/2017
search Excellence grant] and P30-CA008748 [to Memorial Sloan Kettering Cancer Center]). Additional support was provided by the National Institutes of Health (grant U01 NS086294), the Patient-Centered Outcomes Research Institute (grant 1IP2PI000722), and a Clinical & Translational Science Awards U-award (UL1 TR001064). Disclosures: Disclosures can be viewed at www.acponline
.org/authors/icmje/ConflictOfInterestForms.do?msNum=M14 -2767. Requests for Single Reprints: Andrew J. Vickers, DPhil, Memo-
rial Sloan Kettering Cancer Center, 307 East 63rd Street, 2nd Floor, New York, NY 10065; e-mail, [email protected]. Current author addresses and author contributions are available at www.annals.org. Ann Intern Med. 2015;162:866-867. doi:10.7326/M14-2767
References 1. Kent DM, Hayward RA. Limitations of applying summary results of clinical trials to individual patients: the need for risk stratification. JAMA. 2007;298:1209-12. [PMID: 17848656] 2. Bach PB, Gould MK. When the average applies to no one: personalized decision making about potential benefits of lung cancer screening. Ann Intern Med. 2012;157:571-3. [PMID: 22893040] doi: 10.7326/0003-4819-157-8-201210160-00524 3. Kent DM, Hayward RA, Griffith JL, Vijan S, Beshansky JR, Califf RM, et al. An independently derived and validated predictive model for selecting patients with myocardial infarction who are likely to benefit from tissue plasminogen activator compared with streptokinase. Am J Med. 2002;113:104-11. [PMID: 12133748] 4. Kent DM, Schmid CH, Lau J, Selker HP. Is primary angioplasty for some as good as primary angioplasty for all? J Gen Intern Med. 2002;17:887-94. [PMID: 12472924] 5. Thune JJ, Hoefsten DE, Lindholm MG, Mortensen LS, Andersen HR, Nielsen TT, et al; Danish Multicenter Randomized Study on Fibrinolytic Therapy Versus Acute Coronary Angioplasty in Acute Myocardial Infarction (DANAMI)-2 Investigators. Simple risk stratification at admission to identify patients with reduced mortality from primary angioplasty. Circulation. 2005;112:2017-21. [PMID: 16186438] 6. Vickers AJ, Cronin AM, Bjo¨rk T, Manjer J, Nilsson PM, Dahlin A, et al. Prostate specific antigen concentration at age 60 and death or metastasis from prostate cancer: case-control study. BMJ. 2010;341: c4521. [PMID: 20843935] doi:10.1136/bmj.c4521 7. Carlsson S, Assel M, Sjoberg D, Ulmert D, Hugosson J, Lilja H, et al. Influence of blood prostate specific antigen levels at age 60 on benefits and harms of prostate cancer screening: population based cohort study. BMJ. 2014;348:g2296. [PMID: 24682399] doi:10.1136 /bmj.g2296
Annals of Internal Medicine • Vol. 162 No. 12 • 16 June 2015 867
Annals of Internal Medicine Current Author Addresses: Dr. Vickers: Department of Epidemiology and Biostatistics, Memorial Sloan Kettering Cancer Center, 485 Lexington Avenue, 2nd Floor, New York, NY 10017. Dr. Kent: Institute for Clinical Research and Health Policy Studies, Tufts Medical Center, 800 Washington Street, Boston, MA 02111.
www.annals.org
Downloaded From: https://annals.org/ by a Columbia University User on 08/07/2017
Annals of Internal Medicine • Vol. 162 No. 12 • 16 June 2015
