Research Original Investigation
Cardiovascular Risk Calculator Controversy
Invited Commentary
Prevention Guidelines Bad Process, Bad Outcome Steven E. Nissen, MD
Thomas Jefferson, the third American President and author of the Declaration of Independence, famously opined, “Governments derive their just powers from the consent of the governed.” The principle underlying this simple statement is an essential component of any collection of rules that govern how individuals conRelated article page 1964 form to behavioral standards. Clinical practice guidelines should explicitly or implicitly follow the same principles. Guidelines are effective only when they involve the participation and consent of the stakeholders whose behavior they intend to govern. The 2011 Institute of Medicine (IOM) report1 in 2011 (Clinical Practice Guidelines We Can Trust) emphasizes transparency and public involvement as integral to the development process. In this context, the recently issued guidelines for cholesterol reduction2 represent an important failure of guideline governance and oversight process. Rather than forging a consensus on cholesterol management, the guidelines have further polarized the debate on appropriate use of statin medications. How did the process go awry? Many of the current concerns stem from a process lacking openness and transparency, as recommended by the IOM. The Prevention Guidelines were originally commissioned in 2008 by the National Institutes of Health (NIH), the supervising authority for all previous lipid guidelines. Writing efforts lasted more than 5 years from initiation to completion. During the development process, several committee members resigned, although the exact reasons for their withdrawal have never been revealed. An adequate explanation for the long period of development also has not been provided. Although 4 separate guidelines were issued, despite years in development, the guidelines are narrow, not comprehensive. The main drug therapy recommendations address only 3 “critical questions,” primarily describing which patients should receive statins. After commissioning the guidelines, in June 2013, the NIH withdrew as the supervising authority. 3 As a consequence of the NIH decision, the guideline committee was abandoned as a stateless entity, a group of writers with no oversight or commissioning organization. Rushing to fill the void, the American College of Cardiology (ACC) and American Heart Association (AHA), assumed responsibility for oversight, but instead of reconstituting the committee and recommissioning the document (in the usual thoughtful and deliberate fashion), they apparently just accepted the result “as is.” Worse yet, the AHA pressed to release the guidelines in advance of the AHA Scientific Sessions to take advantage of the publicity provided by such an important document. These decisions, the abandonment by the NIH, the unquestioning acceptance by the ACC and AHA, and the rush by 1972
AHA to release the document had unfortunate and preventable untoward effects. The “governed” (practicing physicians) were left in a bewildered state, eroding the confidence required for successful implementation. One of the most serious problems of the guidelines is carefully analyzed in the article by Cook and Ridker4 in this issue of JAMA Internal Medicine. The guidelines included a newly developed and heretofore unpublished cardiovascular risk calculator, designed to identify patients most likely to benefit from statins. In the current manuscript, Cook and Ridker4 apply this new risk calculator to a population with known outcomes and show quite convincingly that the risk calculator supplied is inaccurate. Using the Women’s Health Study cohort, the observed risk is substantially lower than the risk predicted by the guidelines. The extent of miscalibration is substantial, overestimating risk by at least 50%. This is not a trivial problem. While statins are valuable drugs, particularly in secondary prevention, they do have downsides, and prudence requires not administering drugs to patients who will likely not benefit. The implications of the overestimation of risk are profound. A 50% overestimation by the guideline risk equations would likely add millions of Americans to the roles of patients for whom statins are recommended. The poor performance of the risk calculator is selfevident to any clinician who attempts to make a calculation for sample patients. A 65-year-old white woman with a total cholesterol level of 175 mg/dL, an high-density lipoprotein cholesterol (HDL-C) level of 54 mg/dL, and a low-density lipoprotein cholesterol (LDL-C) level of 96 mg/dL, a blood pressure (BP) of 134 mm Hg (while taking lisinopril, 5 g/d) reaches a “predicted risk” of 7.7%, above the 7.5% level at which treatment with a statin is recommended. Similarly, a 47-year-old African American man with total cholesterol level of 170 mg/dL, an HDL-C level of 45 mg/dL, an LDL-C level of 95 mg/dL, and a BP of 129 mm Hg (while taking hydrochlorothiazide, 25 mg/d) exceeds the 7.5% threshold for lifelong treatment. Most practitioners would not treat such patients. (To convert total, HDL-C, and LDL-C cholesterol to millimoles per liter, multiply by 0.0259.) In their public response to this criticism, the guideline writers and professional societies organized a scrambled defense based mostly on hubris rather than thoughtful analysis. Rather than investigating the concerns, several authors5 stated that the guidelines were never intended to be taken at face value. Rather, they were a “starting point” for a conversation with the patient about treatment. As suggested by Montori et al,6 it is critically important that any decision to begin statins be discussed with patients, consistent with shared decisionmaking principles. Such advice is sensible, but issuing recommendations using a flawed risk calculator undermines the
JAMA Internal Medicine December 2014 Volume 174, Number 12
Copyright 2014 American Medical Association. All rights reserved.
Downloaded From: http://archinte.jamanetwork.com/ by a University of Arizona Health Sciences Library User on 05/25/2015
jamainternalmedicine.com
Cardiovascular Risk Calculator Controversy
Original Investigation Research
credibility of the process and the confidence of the medical community. Furthermore, the origin of the 7.5% threshold for risk is inadequately explained, leaving clinicians with serious doubts about the quality of the scientific process leading to this conclusion. How could the current controversy have been avoided? Instead of using an unpublished risk calculator, a peerreviewed publication of the equations and risk calculator methodology would have allowed other physician-scientists to test the calculator against known cohorts and determine whether ARTICLE INFORMATION Author Affiliation: Department of Cardiovascular Medicine, The Cleveland Clinic Foundation, Cleveland, Ohio. Corresponding Author: Steven E. Nissen, MD, Department of Cardiovascular Medicine, The Cleveland Clinic Foundation, 9500 Euclid Ave, Cleveland, OH 44195 ([email protected]). Published Online: October 6, 2014. doi:10.1001/jamainternmed.2014.3278. Conflict of Interest Disclosures: Dr Nissen reports that the Cleveland Clinic Center for Clinical Research receives funding to perform clinical trials from AstraZeneca, Amgen, Eli Lilly, Pfizer, The Medicines Company, Novartis, Takeda, Orexigen, Vivus, and Eli Lilly. Dr Nissen is involved in these clinical trials but receives no personal remuneration for his participation. Dr Nissen consults for many pharmaceutical companies but requires them to
jamainternalmedicine.com
the results were reliable. Furthermore, the process for guideline development should be more open and transparent and include a public comment period. How do we move forward in the public interest? The ACC and AHA should promptly revise the guidelines to address the criticisms offered by independent authorities. Any new guidelines should be posted for public comment to allow affected stakeholders to participate in the vetting of the recommendations. Transparency and openness are essential to good governance and good medicine.
donate all honoraria or consulting fees directly to charity so that he receives neither income nor a tax deduction. REFERENCES 1. Clinical practice guidelines we can trust. http: //www.iom.edu/Reports/2011/Clinical-Practice -Guidelines-We-Can-Trust.aspx. Accessed August 28, 2014. 2. Stone NJ, Robinson JG, Lichtenstein AH, et al; American College of Cardiology/American Heart Association Task Force on Practice Guidelines. 2013 ACC/AHA guideline on the treatment of blood cholesterol to reduce atherosclerotic cardiovascular risk in adults: a report of the American College of Cardiology/American Heart Association Task Force on Practice Guidelines. J Am Coll Cardiol. 2014;63(25, pt B):2889-2934.
guidelines: an announcement from the National Heart, Lung, and Blood Institute. J Am Coll Cardiol. 2013;62(15):1396-1398. 4. Cook NR, Ridker PM. Further insight into the cardiovascular risk calculator: the roles of statins, revascularizations, and underascertainment in the Women’s Health Study [published online October 6, 2014]. JAMA Intern Med. doi:10.1001 /jamainternmed.2014.5336. 5. New heart disease guidelines called into question. http://www.nydailynews.com/life-style /health/new-heart-disease-guidelines-called -question-article-1.1521031. Accessed August 28, 2014. 6. Montori VM, Brito JP, Murad MH. The optimal practice of evidence-based medicine: incorporating patient preferences in practice guidelines. JAMA. 2013;310(23):2503-2504.
3. Gibbons GH, Shurin SB, Mensah GA, Lauer MS. Refocusing the agenda on cardiovascular
JAMA Internal Medicine December 2014 Volume 174, Number 12
Copyright 2014 American Medical Association. All rights reserved.
Downloaded From: http://archinte.jamanetwork.com/ by a University of Arizona Health Sciences Library User on 05/25/2015
1973
Cardiovascular Risk Calculator Controversy
Invited Commentary
Prevention Guidelines Bad Process, Bad Outcome Steven E. Nissen, MD
Thomas Jefferson, the third American President and author of the Declaration of Independence, famously opined, “Governments derive their just powers from the consent of the governed.” The principle underlying this simple statement is an essential component of any collection of rules that govern how individuals conRelated article page 1964 form to behavioral standards. Clinical practice guidelines should explicitly or implicitly follow the same principles. Guidelines are effective only when they involve the participation and consent of the stakeholders whose behavior they intend to govern. The 2011 Institute of Medicine (IOM) report1 in 2011 (Clinical Practice Guidelines We Can Trust) emphasizes transparency and public involvement as integral to the development process. In this context, the recently issued guidelines for cholesterol reduction2 represent an important failure of guideline governance and oversight process. Rather than forging a consensus on cholesterol management, the guidelines have further polarized the debate on appropriate use of statin medications. How did the process go awry? Many of the current concerns stem from a process lacking openness and transparency, as recommended by the IOM. The Prevention Guidelines were originally commissioned in 2008 by the National Institutes of Health (NIH), the supervising authority for all previous lipid guidelines. Writing efforts lasted more than 5 years from initiation to completion. During the development process, several committee members resigned, although the exact reasons for their withdrawal have never been revealed. An adequate explanation for the long period of development also has not been provided. Although 4 separate guidelines were issued, despite years in development, the guidelines are narrow, not comprehensive. The main drug therapy recommendations address only 3 “critical questions,” primarily describing which patients should receive statins. After commissioning the guidelines, in June 2013, the NIH withdrew as the supervising authority. 3 As a consequence of the NIH decision, the guideline committee was abandoned as a stateless entity, a group of writers with no oversight or commissioning organization. Rushing to fill the void, the American College of Cardiology (ACC) and American Heart Association (AHA), assumed responsibility for oversight, but instead of reconstituting the committee and recommissioning the document (in the usual thoughtful and deliberate fashion), they apparently just accepted the result “as is.” Worse yet, the AHA pressed to release the guidelines in advance of the AHA Scientific Sessions to take advantage of the publicity provided by such an important document. These decisions, the abandonment by the NIH, the unquestioning acceptance by the ACC and AHA, and the rush by 1972
AHA to release the document had unfortunate and preventable untoward effects. The “governed” (practicing physicians) were left in a bewildered state, eroding the confidence required for successful implementation. One of the most serious problems of the guidelines is carefully analyzed in the article by Cook and Ridker4 in this issue of JAMA Internal Medicine. The guidelines included a newly developed and heretofore unpublished cardiovascular risk calculator, designed to identify patients most likely to benefit from statins. In the current manuscript, Cook and Ridker4 apply this new risk calculator to a population with known outcomes and show quite convincingly that the risk calculator supplied is inaccurate. Using the Women’s Health Study cohort, the observed risk is substantially lower than the risk predicted by the guidelines. The extent of miscalibration is substantial, overestimating risk by at least 50%. This is not a trivial problem. While statins are valuable drugs, particularly in secondary prevention, they do have downsides, and prudence requires not administering drugs to patients who will likely not benefit. The implications of the overestimation of risk are profound. A 50% overestimation by the guideline risk equations would likely add millions of Americans to the roles of patients for whom statins are recommended. The poor performance of the risk calculator is selfevident to any clinician who attempts to make a calculation for sample patients. A 65-year-old white woman with a total cholesterol level of 175 mg/dL, an high-density lipoprotein cholesterol (HDL-C) level of 54 mg/dL, and a low-density lipoprotein cholesterol (LDL-C) level of 96 mg/dL, a blood pressure (BP) of 134 mm Hg (while taking lisinopril, 5 g/d) reaches a “predicted risk” of 7.7%, above the 7.5% level at which treatment with a statin is recommended. Similarly, a 47-year-old African American man with total cholesterol level of 170 mg/dL, an HDL-C level of 45 mg/dL, an LDL-C level of 95 mg/dL, and a BP of 129 mm Hg (while taking hydrochlorothiazide, 25 mg/d) exceeds the 7.5% threshold for lifelong treatment. Most practitioners would not treat such patients. (To convert total, HDL-C, and LDL-C cholesterol to millimoles per liter, multiply by 0.0259.) In their public response to this criticism, the guideline writers and professional societies organized a scrambled defense based mostly on hubris rather than thoughtful analysis. Rather than investigating the concerns, several authors5 stated that the guidelines were never intended to be taken at face value. Rather, they were a “starting point” for a conversation with the patient about treatment. As suggested by Montori et al,6 it is critically important that any decision to begin statins be discussed with patients, consistent with shared decisionmaking principles. Such advice is sensible, but issuing recommendations using a flawed risk calculator undermines the
JAMA Internal Medicine December 2014 Volume 174, Number 12
Copyright 2014 American Medical Association. All rights reserved.
Downloaded From: http://archinte.jamanetwork.com/ by a University of Arizona Health Sciences Library User on 05/25/2015
jamainternalmedicine.com
Cardiovascular Risk Calculator Controversy
Original Investigation Research
credibility of the process and the confidence of the medical community. Furthermore, the origin of the 7.5% threshold for risk is inadequately explained, leaving clinicians with serious doubts about the quality of the scientific process leading to this conclusion. How could the current controversy have been avoided? Instead of using an unpublished risk calculator, a peerreviewed publication of the equations and risk calculator methodology would have allowed other physician-scientists to test the calculator against known cohorts and determine whether ARTICLE INFORMATION Author Affiliation: Department of Cardiovascular Medicine, The Cleveland Clinic Foundation, Cleveland, Ohio. Corresponding Author: Steven E. Nissen, MD, Department of Cardiovascular Medicine, The Cleveland Clinic Foundation, 9500 Euclid Ave, Cleveland, OH 44195 ([email protected]). Published Online: October 6, 2014. doi:10.1001/jamainternmed.2014.3278. Conflict of Interest Disclosures: Dr Nissen reports that the Cleveland Clinic Center for Clinical Research receives funding to perform clinical trials from AstraZeneca, Amgen, Eli Lilly, Pfizer, The Medicines Company, Novartis, Takeda, Orexigen, Vivus, and Eli Lilly. Dr Nissen is involved in these clinical trials but receives no personal remuneration for his participation. Dr Nissen consults for many pharmaceutical companies but requires them to
jamainternalmedicine.com
the results were reliable. Furthermore, the process for guideline development should be more open and transparent and include a public comment period. How do we move forward in the public interest? The ACC and AHA should promptly revise the guidelines to address the criticisms offered by independent authorities. Any new guidelines should be posted for public comment to allow affected stakeholders to participate in the vetting of the recommendations. Transparency and openness are essential to good governance and good medicine.
donate all honoraria or consulting fees directly to charity so that he receives neither income nor a tax deduction. REFERENCES 1. Clinical practice guidelines we can trust. http: //www.iom.edu/Reports/2011/Clinical-Practice -Guidelines-We-Can-Trust.aspx. Accessed August 28, 2014. 2. Stone NJ, Robinson JG, Lichtenstein AH, et al; American College of Cardiology/American Heart Association Task Force on Practice Guidelines. 2013 ACC/AHA guideline on the treatment of blood cholesterol to reduce atherosclerotic cardiovascular risk in adults: a report of the American College of Cardiology/American Heart Association Task Force on Practice Guidelines. J Am Coll Cardiol. 2014;63(25, pt B):2889-2934.
guidelines: an announcement from the National Heart, Lung, and Blood Institute. J Am Coll Cardiol. 2013;62(15):1396-1398. 4. Cook NR, Ridker PM. Further insight into the cardiovascular risk calculator: the roles of statins, revascularizations, and underascertainment in the Women’s Health Study [published online October 6, 2014]. JAMA Intern Med. doi:10.1001 /jamainternmed.2014.5336. 5. New heart disease guidelines called into question. http://www.nydailynews.com/life-style /health/new-heart-disease-guidelines-called -question-article-1.1521031. Accessed August 28, 2014. 6. Montori VM, Brito JP, Murad MH. The optimal practice of evidence-based medicine: incorporating patient preferences in practice guidelines. JAMA. 2013;310(23):2503-2504.
3. Gibbons GH, Shurin SB, Mensah GA, Lauer MS. Refocusing the agenda on cardiovascular
JAMA Internal Medicine December 2014 Volume 174, Number 12
Copyright 2014 American Medical Association. All rights reserved.
Downloaded From: http://archinte.jamanetwork.com/ by a University of Arizona Health Sciences Library User on 05/25/2015
1973
