Journal of Surgical Oncology 2014;110:500–508

Risk Prediction Tools in Surgical Oncology CHRISTINE V. KINNIER, MD,1,2 ELLIOT A. ASARE, MD,3,4 SANJAY MOHANTY, MD,3,5 JENNIFER L. PARUCH, MD,3,6 RAVI RAJARAM, MD,3,7,8 AND KARL Y. BILIMORIA, MD, MS1,9* 1

Department of Surgery, Surgical Outcomes and Quality Improvement Center, Feinberg School of Medicine, Northwestern University, Chicago, Illinois 2 Department of Surgery, Massachusetts General Hospital, Boston, Massachusetts 3 Division of Research and Optimal Patient Care, American College of Surgeons, Chicago, Illinois 4 Department of Surgery, Medical College of Wisconsin, Milwaukee, Wisconsin 5 Department of Surgery, Henry Ford Hospital, Detroit, Michigan 6 Department of Surgery, Pritzker School of Medicine, University of Chicago, Chicago, Illinois 7 Center for Healthcare Studies, Feinberg School of Medicine, Northwestern University, Chicago, Illinois 8 Department of Surgery, Northwestern Memorial Hospital, Chicago, Illinois 9 Northwestern Institute for Comparative Effectiveness Research (NICER) in Oncology, Robert H. Lurie Comprehensive Cancer Center, Chicago, Illinois

Healthcare has increasingly focused on patient engagement and shared decision‐making. Decision aids can promote engagement and shared decision making by providing patients and their providers with care options and outcomes. This article discusses decision aids for surgical oncology patients. Topics include: short‐term risk prediction following surgery, long‐term risk prediction of survival and recurrence, the combination of short‐ and long‐ term risk prediction to help guide treatment choice, and decision aid usability, transparency, and accessibility.

J. Surg. Oncol. 2014;110:500–508. ß 2014 Wiley Periodicals, Inc.

KEY WORDS: decision support techniques; patient‐centered care; nomograms; health literacy

INTRODUCTION AND OVERVIEW Patient‐centeredness in healthcare has received increasing attention in recent years [1–8]. In a report by the Institute of Medicine (IOM), patient engagement and shared decision making were outlined as central tenets to a high‐quality cancer care delivery system [9,10]. Additionally, the establishment of funding agencies such as the Patient Centered Outcomes Research Institute (PCORI) by the Patient Protection and Affordable Care Act underscores the increased emphasis placed on patients as their own decision‐makers [11–13]. Incorporating the use of decision aids may be one potential way to help facilitate a patient‐ centered healthcare system.

Decision Aids Decision aids are visual or audio tools that help patients make treatment decisions by explicitly naming the decision question, providing information about options and outcomes, clarifying personal values, and openly involving the patient in the decision making process [14]. Decision aids may take many forms, from educational videos and pamphlets to interactive decision boards. A Cochrane Review found that decision aids improve patient awareness regarding treatment options, decrease decisional conflict, and encourage patients to take an active role in their care [15]. Consequently, cancer patients, often faced with multiple, emotionally‐fraught treatment options, may benefit significantly from decision aids. For example, breast cancer patients are more likely to choose breast conservation therapy when treatment information is presented using an interactive decision board that displays diagrams and other visually engaging materials. These patients are also more likely to express satisfaction with their decision when compared to controls who receive consultation without the decision aid [16]. Patients with advanced cancer also benefit from decision aids. Patients who receive a booklet with an audio‐ or video‐guide during consultation are more aware of their prognosis, have

ß 2014 Wiley Periodicals, Inc.

decreased anxiety, and have increased clarity regarding end‐of‐life goals after using the decision aid [17–21]. Clearly decision aids can influence a patient’s decision and their overall experience with the healthcare system. Moreover, decision aids are not restricted to decisions with medical uncertainty. Even when medical evidence clearly favors one treatment over another, decision aids may improve patient understanding and foster patient‐centered care.

Risk Prediction Tools as Decision Aids Risk prediction tools, such as risk calculators, are one type of decision aid proposed for use in a patient‐centered model of care [22–24]. While many decision aids provide general information about a disease, prognosis, or treatment, risk prediction tools use patient‐ or tumor‐ specific information to estimate a patient’s disease or treatment outcomes. In cancer care, outcomes can take many forms; risk prediction tools may estimate the benefits of lung cancer screening or the 10‐year recurrence rate following colon cancer resection [25]. Many

Christine V. Kinnier, Elliot A. Asare, Sanjay Mohanty, Jennifer L. Paruch, and Ravi Rajaram contributed equally to this manuscript. Work performed at the Surgical Outcomes and Quality Improvement Center, Department of Surgery, Feinberg School of Medicine, Northwestern University, Chicago, IL. Conflict of interest: None. *Correspondence to: Karl Y. Bilimoria, MD, MS, Department of Surgery, Surgical Outcomes and Quality Improvement Center, Northwestern University, 676 N. St. Clair St, Suite 6‐650, Chicago, IL 60611. Fax: þ1‐312‐695‐1462. E‐mail: [email protected] Received 3 April 2014; Accepted 5 June 2014 DOI 10.1002/jso.23714 Published online 29 June 2014 in Wiley Online Library (wileyonlinelibrary.com).

Prediction Tools in Surgical Oncology of these tools are web‐based, and users can directly enter patient‐ or tumor‐specific information before calculating and reporting outcome estimates. Early risk prediction tools were typically directed towards clinicians with the expectation that clinicians would interpret the results for their patients. With recent attention on patient‐centeredness in healthcare, however, some risk prediction tools are now using plain‐ language in order to further encourage patient engagement. In this review, we will discuss the use of risk prediction tools during the care of cancer patients. First, we will review peri‐operative risk calculators that provide short‐term outcomes in cancer surgery. Next, we will highlight the use of risk calculators to estimate long‐term survival in cancer patients. We will then review how risk calculators might help patients and clinicians choose between different treatment options. Finally, we will discuss transparency, literacy, and numeracy in the development and implementation of risk prediction tools.

SHORT‐TERM, SURGERY‐SPECIFIC RISK CALCULATORS

501

There is also an option for surgeons to adjust the prediction, using their clinical judgment, if there are risk factors not captured by the calculator. The tool is designed to be used by surgeons and patients together, and is formatted to provide patient‐friendly reports that can then be printed or emailed.

Use of Surgical Risk Calculators for Cancer Patients Although important for all surgical patients, some aspects of care and decision‐making in cancer surgery lend themselves particularly well to the use of a surgical risk calculator. Most surgeries performed for cancer are elective procedures planned during multiple pre‐operative visits, allowing ample opportunity for use of surgical risk calculators. Information can then be used to tailor peri‐operative care or to focus patient education on strategies for preventing specific complications. For example, if a patient undergoing a gastrectomy is identified as being at increased risk for respiratory complications, they may be more vigilant about smoking cessation, preoperative conditioning regimens, ambulation, or use of incentive spirometers.

Need for Surgery‐Specific Risk Calculators Understanding the risks and benefits associated with treatment is critical for patient engagement, patient‐physician communication, and shared decision‐making [26]. In many cases, the risks of surgery provided to patients are based on either individual surgeon experience or data from single institution studies or clinical trials. None of these strategies incorporate patient comorbidities, which are important drivers of postoperative complications [27]. Cancer patients are older and may be more likely to have multiple health problems, and cancer surgeries are often high‐risk procedures (e.g., Whipple, esophagectomy) with significant complication rates. Therefore honest discussions based on high quality data are particularly important when treating cancer patients. As a result, surgical organizations have begun leveraging multi‐institutional, clinical databases to develop surgical risk calculators that provide empiric, patient‐ specific risk predictions [28,29]. Beginning in 2014, the Centers for Medicare and Medicaid Services (CMS) will begin tying use of these calculators to physician payment through the Physician Quality Reporting System (PQRS) [30]. There are several surgical risk calculators currently available that are specialty or procedure‐specific [29,31,32]. Since the American College of Surgeons National Surgical Quality Improvement Program (ACS NSQIP) surgical risk calculator is the only one currently available based on clinical data that can be used for most procedures, this section will specifically address the development and use of this tool.

Limitations and Future Directions Although surgical risk calculators are helpful for supporting shared‐ decision making and patient engagement in surgical oncology, current versions have some limitations that could be addressed to strengthen their use. First, ACS NSQIP is currently testing the surgical risk calculator with patients and surgeons to improve the design of the website to optimize it for patients and surgeons. In the future, users may also be able to perform side‐by‐side comparisons of patient‐specific risks for multiple procedures. Second, many currently available risk calculators do not include procedure‐specific outcomes (e.g., recurrent laryngeal nerve injury, anastomotic leak). Third, future surgical risk calculators should incorporate patient‐centered outcomes such as pain and return to normal function. Patient‐centered outcomes are a priority area of cancer research [9,35], and could be easily incorporated into the same framework as an additional outcome. Finally, surgical risk calculators should be combined with other tools, including long‐term risk prediction tools, to provide patients with personalized estimates for the risks and benefits of all treatment options (or phases for multi‐modal therapy).

LONG‐TERM RISK PREDICTION TOOLS IN ONCOLOGY

ACS NSQIP Surgical Risk Calculator

Need for Long‐Term Risk Calculators

ACS NSQIP is an ideal source of data for the development of a surgical risk calculator. Described in detail elsewhere [33,34], ACS NSQIP collects high‐quality data for 30‐day post‐operative complications from >500 hospitals ranging from academic tertiary care centers to community hospitals. It includes cases from nearly all specialties, with the exception of cardiac and transplant surgery. Data are collected by trained and audited Surgical Clinical Reviewers (SCRs) using standardized definitions. Over 200 variables are collected for each case, including comorbidities, operative variables, and complications. Outcomes are abstracted from 30‐ days following surgery, regardless of whether complications occur at the same hospital, a different hospital, or at home. The most recent version of the ACS NSQIP surgical risk calculator was released in 2013, and can be used for >2,500 different procedures [28]. The tool was developed using over 1.4 million cases from 2009 to 2012, and is publicly available at www. riskcalculator.facs.org. On the website, surgeons can enter 21 easily identified preoperative variables along with the CPT code. The ACS NSQIP surgical risk calculator then provides patient‐specific, 30‐day predicted risks for 11 different postoperative complications (Fig. 1).

Short‐term risk prediction is particularly germane to patient discussions regarding treatment complications, but most cancer patients are especially interested in long‐term risks such as survival, recurrence risk, or risk of metastasis. Clinicians who predict survival using clinical assessment, intuition, or experience are often overly optimistic [36–38]. In contrast, tools that integrate multiple patient, tumor and treatment factors more accurately predict survival [39], and accurate prediction of long‐term risks for cancer patients facilitates clinical decision‐making and guides treatment recommendations. When poor prognosis is recognized early, patients often decide to pursue comfort care rather than aggressive therapy in order to avoid undue morbidity and costs [40], and patients and their families are able to institute timely end of life planning [41].

Journal of Surgical Oncology

Development of Long‐Term Risk Prediction Tools Data for the development of prediction tools can come from large national databases like the Surveillance, Epidemiology and End Results (SEER) database or from prospective or retrospective single institution

502

Kinnier et al.

Fig. 1. American college of surgeons surgical risk calculator. Journal of Surgical Oncology

Prediction Tools in Surgical Oncology or multi‐institution databases. The common end points for available long‐term prediction tools include overall survival, disease specific survival, conditional survival, and recurrence free survival [42]. Variables used for prediction can be broadly classified into patient factors, tumor factors, and treatment factors. The choice of variables included in the model may be dictated by availability, clinical validity, previous published work, or statistical significance [43–47]. Once developed, the performance of the prediction tool is assessed by discrimination and calibration. Discrimination is the ability of a model to distinguish between patients with different outcomes while calibration is a comparison of the predicted probability of an event to the observed probability [40,48].

Selected Long‐Term Risk Prediction Tools A range of prediction tools are available for the use of the oncologist and the cancer patient. Rabin et al. published a detailed review of currently available prediction tools with web applications [42]. A few of the prediction tools that have been published in peer‐reviewed journals and are readily accessible to the public will be highlighted here.

Colon Cancer Survival Calculators The Memorial Sloan Kettering Cancer Center provides an online tool for predicting 5‐year overall survival and 5–10‐year disease free survival in patients with colorectal cancer who undergo curative resection (available at http://nomograms.mskcc.org/Colorectal) [45,49]. A cohort in the SEER database from 1994 to 2005 was used in model development and validation. The following variables are utilized in prediction: American Joint Committee on Cancer (AJCC) tumor (T) category, AJCC node (N) category, number of positive lymph nodes, total number of lymph nodes examined, age at time of surgery, gender, and histologic grade. Another readily available online prediction tool is the MD Anderson Cancer Center conditional survival prediction tool for patients with colon cancer [44,50]. Unlike overall survival, which is calculated at the time of diagnosis, this tool aims to predict survival for patients who have already survived up to a certain time point. It is more suitable for predicting survival in patients at follow‐up. Patients with colon cancer in the SEER database from 1988 to 2000 were used to develop and validate this model. The variables required for prediction are age, sex, grade, AJCC TNM stage, and race. The oncologist may want to choose one prediction tool over another depending on the particular patient scenario and the benefits offered by each calculator. For example, the Memorial Sloan Kettering colon cancer calculator may be useful when discussing prognosis with a newly diagnosed cancer patient while the MD Anderson cancer calculator may be appropriate when predicting a patient’s survival during follow‐up. Both calculators were developed using large data from a reliable national cancer database, so results may be generalizable.

Breast Cancer PREDICT is a tool developed and validated in the UK for predicting overall and disease specific survival for women with breast cancer (available at http://www.predict.nhs.uk/predict.shtml) [46,49]. The sources of data for model development and validation are the Eastern Cancer Registration and Information Center (ECRIC) and West Midlands Cancer Intelligence Unit (WMCIU), respectively. The variables used for prediction are age at diagnosis, mode of tumor detection, tumor size, grade, number of positive lymph nodes, ER status, HER2 status, Ki67 status, and chemotherapy regimen. The PREDICT tool notably includes molecular markers to enhance prediction accuracy and therefore suggests directions for future risk calculator improvement. Journal of Surgical Oncology

503

Unique Challenges in Long‐Term Risk Prediction A tool’s generalizability and accuracy significantly affect the utility of a long‐term risk prediction tool. Generalizability is frequently limited by underlying data limitations. For example, large retrospective databases often have missing variables. These are often excluded and may introduce bias and decrease the generalizability of the prediction tool. In addition, there are no standard guidelines on the minimum sample size needed to develop a prediction tool; hence the reported sample sizes have ranged from a few hundred to hundreds of thousands. Prediction tools developed with smaller sample sizes may lack generalizability. In contrast, tool accuracy may be limited by changing prognostic factors and care standards. For example, older long‐term risk prediction tools may not readily incorporate new prognostic factors due to database limitations. Consequently, a tool that was accurate at the time of publication may become less accurate over time. Similarly, changes in AJCC TNM staging may cause up‐ or down‐staging of patients. As a result, the cohort used in model development may have different characteristics than modern patients using the tool for risk prediction thereby leading to tool inaccuracy. Even when a tool is intrinsically accurate, comparing the accuracy of prediction tools from different institutions may still prove difficult. Different long‐term risk prediction tools often predict similar endpoints using different predictor variables. This is likely due to the different variables available in independent databases. Nevertheless, this lack of harmonization makes comparison of tool accuracy impractical. Finally, long‐term risk prediction tool utility is limited by the types of available outcomes. With the current emphasis on patient‐centered outcomes, prediction tools that capture this important component of healthcare will be more relevant, yet quality of life measures have not been incorporated into many existing prediction tools [42]. Most retrospective databases are not designed to collect information on quality of life parameters. Modifications to existing databases will have to be made to enable abstraction of quality of life measures. Despite some of the challenges with long‐term prediction outlined above, their incorporation into clinical decision‐making will facilitate the current push to tailor care for each patient based on pertinent individual variables as opposed to a one‐size‐fits‐all approach. Improvements in data collection, multi‐institutional collaboration with harmonization of data variables, and advances in computational analysis will help mitigate some of the challenges involved with long‐term risk prediction for cancer patients.

USING RISK CALCULATORS TO GUIDE TREATMENT CHOICE Short‐term risk prediction tools help patients make surgical decisions, and long‐term risk prediction tools help patients confront survival and make long‐term plans, but treatment choice is influenced by both short‐ and long‐term outcomes. Patients would benefit from a risk calculator that merges short‐ and long‐term prediction tools described so far into a single tool. In this section, we will address risk comparison calculators that could be developed to help patients and their providers choose among multiple treatment options shortly after diagnosis.

Comparing Surgery and Medical Therapy Choosing appropriate oncologic treatment for older patients with multiple comorbidities is often challenging due to their risk for complications from medical therapy, surgery, or both. Further complicating the picture, patients also differ in their values and risk aversion; a patient who highly values quality of life may choose a different treatment than one who prioritizes recurrence‐free survival [51]. A patient‐specific risk comparison calculator could provide invaluable

504

Kinnier et al.

information to such patients. For example, adjuvant chemotherapy provides additional survival benefit to many cancer patients but may also carry significant morbidity risks compared to surgical therapy alone. The web‐based tool Adjuvant! Online (available at www.adjuvantonline. com) already compares the 10‐year patient‐ and tumor‐specific mortality for patients who undergo surgery and those who undergo surgery with adjuvant chemotherapy [29,30,52]. Patients could make more informed decisions if chemotherapy adverse effect rates and surgical risks were also included in the calculator. Taking calculators a step further, patients with stage II or III squamous cell esophageal cancer may be treated with either chemoradiation or surgery [31]. A risk comparison calculator could provide patients with patient‐ and tumor‐specific rates of chemoradiation adverse effects, postoperative complications, recurrence, and survival. If these risks were presented side‐by‐side for chemoradiation and surgery, patients would be able to select the therapy most suited to their personal risks and values. In contrast, younger patients may need patient‐specific risk calculators that help them balance the immediate and long‐term risks and benefits of different treatments. For example, some ulcerative colitis patients require aggressive immunosuppressants or biologics to achieve adequate control [53,54]. Colectomy offers immediate symptom relief while minimizing risks of colon cancer and opportunistic infections [55]. However, colectomy also exposes patients to peri‐operative morbidity and mortality and long‐term risks of dehydration and incontinence. Like the calculator for esophageal cancer treatment described above, a risk comparison calculator could provide patients with patient‐ and treatment‐specific rates of opportunistic infections, ulcerative colitis recurrence, colon cancer, peri‐operative morbidity and mortality, dehydration, and even patient‐reported outcomes such as incontinence and quality of life. Such a calculator would be extremely helpful for ulcerative colitis patients struggling with disease control.

Comparing Different Surgeries Some diseases are amenable to two different procedures, each associated with its own risks and benefits. As with the examples given above, procedure‐specific, postoperative risks may diverge widely for patients with different comorbidities and tumor characteristics. Patient‐ and procedure‐specific risk comparison calculators would help elucidate these risks and benefits and allow patients to select treatments aligned with their values. The Breast Reconstruction Risk Assessment (BRA) Score (www.brascore.org) is a recently published example of this type of risk calculator (Fig. 2) [56,57]. The BRA score is a web‐based tool that estimates medical complication rates, flap loss risk, explantation risk, and 30‐ day reoperation risk for four breast reconstruction techniques. Patient‐ and procedure‐specific risks are estimated using the ACS NSQIP database and the Tracking Operations and Outcomes for Plastic Surgeons database. Due to underlying data constraints, the BRA Score is currently limited to 30‐day outcomes. Nevertheless, the calculator provides clear, patient‐ and procedure‐specific risks for providers and patients choosing among post‐mastectomy reconstruction options. Similar calculators could be developed for breast cancer patients choosing between mastectomy and lumpectomy with radiation or rectal cancer patients choosing between a very low anterior resection and abdominoperineal resection. In both circumstances, an informative risk calculator would also include long‐term postoperative outcomes such as lymphedema or incontinence and tumor recurrence rates.

Limitations As many of these examples have highlighted, patient‐specific risk comparison calculators could be powerful decision tools if they estimated Journal of Surgical Oncology

procedure‐specific, short‐term outcomes, and long‐term recurrence and mortality estimates. Developing these calculators, however, requires large national databases that include relevant predictor variables. Unfortunately, predictor variables significantly limit calculator development in three ways. First, many cancer‐specific databases collect 5‐year mortality and cancer recurrence [58], but recurrence abstraction is sometimes unreliable [59]. Risk calculator utility for cancer patients will be severely limited if tumor‐ and procedure‐specific recurrence rates are unavailable. Second, ACS NSQIP currently collects serious morbidity and mortality (e.g., renal failure and ventilator dependence) and complications common to all procedures (e.g., surgical site infection and urinary tract infection) [60]. The complications that most affect a patient’s quality of life, however, are often procedure‐specific, patient‐reported, and persist over time. For example, patients undergoing a very low anterior resection will want to know the likelihood of post‐operative incontinence while patients undergoing esophagectomy will want to know their anastomotic stricture risk. Both patient populations will want to know how previous patients rate their quality of life. The availability of these procedure‐ specific and patient‐reported data elements will critically affect the utility of a patient‐specific risk comparison calculator. Finally, medical and surgical databases are usually independent. If providers want to honestly compare medical and radiation outcomes with surgery, links between databases must be developed. Databases like SEER are a solid foundation, but reliable chemotherapy and radiation data elements are necessary to adequately estimate patient‐ and tumor‐specific treatment risks. While significant database hurdles still exist, the opportunities for patient‐specific risk comparison calculators are nearly limitless. If physicians build on the database and calculator infrastructure already in place, both patients and providers will benefit immensely from more accurate risk estimation.

ADDITIONAL CONSIDERATIONS BEFORE RISK CALCULATOR RELEASE Ideally, tools that provide individualized risk estimation communicate accurate, reliable, and objective data in a transparent fashion. In the specific case of cancer, the preoperative patient and tumor factors that must be accounted for when determining prognosis are numerous. The statistical challenges to this are considerable, but they represent only one part of creating such risk communication tools. Two other essential components when designing high quality prediction tools are their transparency and accessibility.

Transparency Transparency is essential to any risk prediction tool as it allows critical evaluation by users. A cross‐sectional study of available web‐ based prognostic tools demonstrated that many tools fail to provide adequate information on data sources, the underlying risk prediction model, and the reason certain risk factors were included in the prediction model [61]. In numerous prognostic tools for a variety of different cancers, the study noted a wide and disconcerting amount of variability at the level of tool function and format. First, the authors uncovered variability in how patient risk factors and outcomes were formatted between calculators. In the case of breast cancer, family history input ranged from a simple yes/no field to a free text space where the user could enter a relative’s age at the time of her breast cancer diagnosis. Second, tool output was rarely uniform across tools. Some tools provided numeric information such as percentages while others provided comparative estimates (e.g., “below average risk” or “very much above average risk”). Finally, tools frequently failed to include established risk factors in the risk prediction model. For example, tools for breast cancer were often missing risk factors identified by the Gail model, a widely‐known, validated model for individualized prediction

Prediction Tools in Surgical Oncology

505

Fig. 2. Breast reconstruction risk assessment (BRA) score web prediction tool.

of breast cancer risk. Most cancers do not have published risk assessment models such as the Gail model, and in these cases, information about model development is even more important. Without information on underlying prediction algorithms to allow assessment by users, doubt should be cast on the generated predictions. Even when output format was comparable, the authors found that actual risk predictions with hypothetical low‐ and high‐risk patients often exhibited poor correlation with an average correlation coefficient of 0.42 (range: 0.74 to 1.00). This indicates that at least some of the available tools provide inaccurate estimates [61]. A more recent systematic review used more stringent search and inclusion criteria and focused on interactive cancer prognostic tools. Journal of Surgical Oncology

Many of these tools, in contrast to the first review, did provide information on the data source and the underlying risk prediction model. Many tools specifically noted whether the data source was national or center‐specific. However, similar to the first study, the format for risk factor input, the output format, and the exact risk factors used varied considerably across tools. The study also noted that only one of the tools had information regarding its use in the real world. Importantly, these calculators were overwhelmingly directed at clinicians. Only four of twenty‐two tools mentioned patients as “potential users” but still recommended close consultation with the treating doctor when interpreting output [42]. The changes in prediction tools described by these two publications indicate that transparency is beginning to improve, but risk factor output

506

Kinnier et al.

and format variability persist. In addition, if a risk prediction tool’s purpose is to support decision‐making or facilitate shared decision making, the output format should also consider health literacy and numeracy issues. Neither were addressed by the above publications.

Accessibility: Health Literacy, and Numeracy Health literacy refers to an individual’s “capacity to obtain, process, and understand basic health information and services needed to make appropriate health decisions”[62]. Limited health literacy is often associated with less education, minority race, and increasing age as well as worse health outcomes and lower rates of healthcare use [63–65]. Patients with lower health literacy also have more difficulty using decision tools if health literacy has not been specifically addressed during tool development [66]. In contrast, health numeracy describes an individual’s “ability to use his or her judgment about whether to use math in a situation, what math to use, how to use it, and what degree of accuracy is appropriate” [67]. This includes interpreting tables, graphs, and percentage changes as well as problem solving. While health numeracy was historically considered a subcategory of health literacy, experts have recently noted that even health literate individuals with advanced education may have difficulty with health numeracy [67]. Remarkably, a recent literacy survey demonstrated that participants were weakest in quantitative tasks and mathematical assessment [67,68,69]. As patient risk calculators evolve, it is essential that researchers and interface designers consider an audience’s health literacy and numeracy during tool development, especially if the tool will be web‐based or publicly accessible. A calculator that is intended for physician use during preoperative consultation can use technical language and numbers without special consideration of health literacy and numeracy. In contrast, a calculator available on a popular news website must consider the widely variable levels of health literacy and numeracy more carefully, and word choice and presentation of health and risk information should be modified appropriately. While there is no clear consensus on the best way to design decision aids with health literacy and numeracy in mind, risk prediction tools are more useful for the general population when developers aim to maximize patient communication. This can be achieved by including action thresholds and plain language translations of numeric data during tool development. For example, if a physician intends for a patient to take action after viewing abnormal laboratory results, then presentation of raw, tabulated laboratory values may be ineffective. Instead, a tool could automatically instruct patients to call their physician when a patient’s results fall outside a selected range. Similarly, a cancer risk prediction tool that displays a simple risk percentage is unlikely to be useful for a patient with limited health numeracy. Instead, the tool could present the percentage along with plain language translations such as “above average risk” or “higher risk.” While some granularity is lost, communication with patients with limited health literacy and numeracy is improved. Organizations such as the Agency for Healthcare and Research Quality (AHRQ) have compiled a list of additional features that may be helpful when designing tools for patients with limited health literacy and numeracy. These features include: presenting essential health information, such as morbidity rate, by itself rather than among other information, though it is important to orient this rate with respect to an average or standard; presenting essential health information first; presenting quality information so that higher numbers indicate better quality; using the same denominator to present baseline risk and treatment benefit; adding icons or icon arrays like stars to accompany numerical information; and adding video to verbal narratives (Fig. 3) [66]. Tool developers should also aim to highlight key results and ensure that assistance is available and easily accessed directly from the tool. Journal of Surgical Oncology

Fig. 3.

Key considerations for the development of risk prediction tools.

Future Directions Risk calculators should play an important role in shared decision‐ making and patient‐centeredness in the future, but significant gaps in knowledge remain. First, more real‐world studies of these tools are necessary to better understand how and when to use risk calculators during shared decision‐making. Second, many physicians and patients choose not to use risk prediction tools or are unaware they exist. Further research is needed to understand the barriers to both physician and patient use. Finally, risk prediction tools must be accessible to patients with varying levels of health literacy and numeracy. Researchers and website designers must continue to refine existing and new prediction tools so that patients with low health literacy and numeracy are able to comprehend the presented outcomes. Despite these limitations, transparency and accessibility has already improved beyond early prediction tools. With continued focus, prediction tools have the potential to open new avenues for shared decision‐making and patient‐ centered care.

SUMMARY There are good short‐ and long‐term interactive prediction tools already available in a variety of formats. As we have discussed, prediction tools favorably affect communication, patient satisfaction, and risk perception, but these tools would still benefit from improvement. Further research is needed to understand how prediction tools work in real‐world settings and how current design limitations can be overcome. Standards for prediction tool development and transparency will need to be established, and developers will need to be mindful of health literacy and numeracy. Ultimately, risk prediction tools should also incorporate all known evidence‐based factors while easily permitting modifications as new prognostic information becomes available. An example of this ideal is illustrated by a patient with breast cancer. In the future, a well‐developed risk estimation tool might allow the patient to enter their risk factors, demographic characteristics, and various tumor and genetic characteristics at an appointment with a specialist. The calculator would consider possible treatment algorithms and then produce the risks and benefits of multiple surgical options as well as information about adjuvant chemoradiation, reconstructive options, and patient‐centered outcomes such as cosmetic result. Ideally, the patient would then enter the consultation with their physician with a

Prediction Tools in Surgical Oncology much clearer set of expectations than is currently possible now. While this scenario might sound implausible, many of the pieces for such a prediction tool are already in place. With continued collaboration, prediction tools have the ability to significantly change the way we counsel and communicate with our patients.

REFERENCES 1. Zuger A: Talking to patients in the 21st century. JAMA 2013;309:2384–2385. 2. White A, Danis M: Enhancing patient‐centered communication and collaboration by using the electronic health record in the examination room. JAMA 2013;309:2327–2328. 3. Weiner SJ, Schwartz A, Sharma G, et al.: Patient‐centered decision making and health care outcomes: An observational study. Ann Intern Med 2013;158:573–579. 4. Snyder CF, Jensen RE, Segal JB, et al.: Patient‐reported outcomes (PROs): Putting the patient perspective in patient‐centered outcomes research. Med Care 2013;51:S73–S79. 5. Lipkin M: Shared decision making. JAMA Intern Med 2013;173:1204–1205. 6. Gillick MR: The critical role of caregivers in achieving patient‐ centered care. JAMA 2013;310:575–576. 7. Carson‐Stevens A, Jones A, Hansen AS, et al.: “What can I do to improve your care today?”—One question closer to patient‐ centered care. Am J Med Qual 2013;28:174. 8. Bardes CL: Defining “patient‐centered medicine”. N Engl J Med 2012;366:782–783. 9. Delivering high‐quality cancer care: Charting a new course for a system in crisis. Washington, DC; 2013. 10. Ganz PA, Levit LA: Charting a new course for the delivery of high‐ quality cancer care. J Clin Oncol 2013;31:4485–4487. 11. Patient‐Centered Outcomes Research Institute. In “Patient‐Centered Outcomes Research Institute.” 2014. 12. Washington AE, Lipstein SH: The patient‐centered outcomes research institute—Promoting better information, decisions, and health. N Engl J Med 2011;365:e31. 13. Selby JV, Lipstein SH: PCORI at 3 years—Progress, lessons, and plans. N Engl J Med 2014;370:592–595. 14. Patient decision aids. In Ottawa Hospital Research Institute. 15. Stacey D, Legare F, Col NF, et al.: Decision aids for people facing health treatment or screening decisions. Cochrane Database Syst Rev 2014;1:CD001431. 16. Whelan T, Levine M, Willan A, et al.: Effect of a decision aid on knowledge and treatment decision making for breast cancer surgery: A randomized trial. JAMA 2004;292:435–441. 17. Fiset V, O’Connor AM, Evans W, et al.: Development and evaluation of a decision aid for patients with stage IV non‐small cell lung cancer. Health Expect 2000;3:125–136. 18. Leighl NB, Shepherd FA, Zawisza D, et al.: Enhancing treatment decision‐making: Pilot study of a treatment decision aid in stage IV non‐small cell lung cancer. Br J Cancer 2008;98:1769–1773. 19. Leighl NB, Shepherd HL, Butow PN, et al.: Supporting treatment decision making in advanced cancer: A randomized trial of a decision aid for patients with advanced colorectal cancer considering chemotherapy. J Clin Oncol 2011;29:2077–2084. 20. Volandes AE, Levin TT, Slovin S, et al.: Augmenting advance care planning in poor prognosis cancer with a video decision aid: a preintervention‐postintervention study. Cancer 2012;118:4331– 4338. 21. Yun YH, Lee MK, Park S, et al.: Use of a decision aid to help caregivers discuss terminal disease status with a family member with cancer: A randomized controlled trial. J Clin Oncol 2011;29: 4811–4819. 22. Barry MJ, Edgman‐Levitan S: Shared decision making—Pinnacle of patient‐centered care. N Engl J Med 2012;366:780–781. 23. Ozanne EM, Howe R, Omer Z, et al.: Development of a personalized decision aid for breast cancer risk reduction and management. BMC Med Inform Decis Mak 2014;14:4. 24. Paruch JL, Ko CY, Bilimoria KY: An opportunity to improve informed consent and shared decision making: the role of the ACS Journal of Surgical Oncology

25. 26. 27. 28.

29. 30. 31.

32. 33.

34.

35.

36. 37. 38. 39. 40. 41. 42.

43. 44. 45. 46. 47.

507

NSQIP Surgical Risk Calculator in oncology. Ann Surg Oncol 2014;21:5–7. Memorial Sloan Kettering Cancer Center (MSKCC). MSKCC Prediction Tools. Date accessed February 25, 2014. In http://www. mskcc.org/cancer‐care/prediction‐tools. Institute of Medicine. Delivering High‐Quality Cancer Care: Charting a New Course for a System in Crisis. Washington, DC: The National Academies Press, 2013. Ko CY, Chang JT, Chaudhry S, et al.: Are high‐volume surgeons and hospitals the most important predictors of in‐hospital outcome for colon cancer resection? Surgery 2002;132:268–273. Bilimoria KY, Liu Y, Paruch JL, et al.: Development and evaluation of the universal ACS NSQIP surgical risk calculator: A decision aid and informed consent tool for patients and surgeons. J Am Coll Surg 2013;217:833–842,e831–e833. Olivotto IA, Bajdik CD, Ravdin PM, et al.: Population‐based validation of the prognostic model ADJUVANT! For early breast cancer. J Clin Oncol 2005;23:2716–2725. Adjuvant! Inc. Adjuvant! Online. Date accessed February 26, 2014. In http://www.adjuvantonline.com/index.jsp. National Comprehensive Cancer Network (NCCN). NCCN Clinical Practice Guidelines in Oncology. Esophageal and Esophagogastric Junction Cancers. (Version 1.2014). In http://www.nccn.org/ professionals/physician_gls/pdf/esophageal.pdf. Parikh P, Shiloach M, Cohen ME, et al.: Pancreatectomy risk calculator: An ACS‐NSQIP resource. HPB 2010;12:488– 497. Hall BL, Hamilton BH, Richards K, et al.: Does surgical quality improve in the American College of Surgeons National Surgical Quality Improvement Program: An evaluation of all participating hospitals. Ann Surg 2009;250:363–376. Cohen ME, Ko CY, Bilimoria KY, et al.: Optimizing ACS NSQIP modeling for evaluation of surgical quality and risk: patient risk adjustment, procedure mix adjustment, shrinkage adjustment, and surgical focus. J Am Coll Surg 2013;217:336–346e331. Basch E, Abernethy AP, Mullins CD, et al.: Recommendations for incorporating patient‐reported outcomes into clinical comparative effectiveness research in adult oncology. J Clin Oncol 2012;30: 4249–4255. Parkes CM: Accuracy of predictions of survival in later stages of cancer. Br Med J 1972;2:29–31. Glare P, Virik K, Jones M, et al.: A systematic review of physicians’ survival predictions in terminally ill cancer patients. BMJ (Clin Res Ed) 2003,327:195–198. Vigano A, Dorgan M, Buckingham J, et al.: Survival prediction in terminal cancer patients: A systematic review of the medical literature. Palliat Med 2000;14:363–374. Bruera E, Hui D: Practical model for prognostication in advanced cancer patients: Is less more? J Clin Oncol 2008;26:5843– 5844. Mackillop WJ, Quirt CF: Measuring the accuracy of prognostic judgments in oncology. J Clin Epidemiol 1997;50:21–29. Hilden J, Habbema DF: Prognosis in medicine: An analysis of its meaning and roles. Theor Med 1987;8:349–365. Rabin BA, Gaglio B, Sanders T, et al.: Predicting cancer prognosis using interactive online tools: A systematic review and implications for cancer care providers. Cancer Epidemiol Biomarkers Prev 2013; 22:1645–1656. Chow E, Abdolell M, Panzarella T, et al.: Predictive model for survival in patients with advanced cancer. J Clin Oncol 2008;26:5863–5869. Chang GJ, Hu CY, Eng C, et al.: Practical application of a calculator for conditional survival in colon cancer. J Clin Oncol 2009;27:5938–5943. Weiser MR, Gonen M, Chou JF, et al.: Predicting survival after curative colectomy for cancer: Individualizing colon cancer staging. J Clin Oncol 2011;29:4796–4802. Wishart GC, Azzato EM, Greenberg DC, et al.: PREDICT: A new UK prognostic model that predicts survival following surgery for invasive breast cancer. Breast Cancer Res 2010;12:R1. Altman DG, Royston P: What do we mean by validating a prognostic model? Stat Med 2000;19:453–473.

508

Kinnier et al.

48. Mallett S, Royston P, Dutton S, et al.: Reporting methods in studies developing prognostic models in cancer: A review. BMC Med 2010;8:20. 49. Memorial Sloan Kettering Cancer Center (MSKCC). MSKCC Prediction Tools. Colorectal Cancer Nomogram. Date accessed March 7, 2014. In http://nomograms.mskcc.org/Colorectal. 50. MD Anderson Cancer Center. Clinical Calculators. Date accessed December 31, 2013. In http://www.mdanderson.org/education‐and‐ research/resources‐for‐professionals/clinical‐tools‐and‐resources/ clinical‐calculators/index.html. 51. Bruera E, Sweeney C, Calder K, et al.: Patient preferences versus physician perceptions of treatment decisions in cancer care. J Clin Oncol 2001;19:2883–2885. 52. Ravdin PM, Siminoff LA, Davis GJ, et al.: Computer program to assist in making decisions about adjuvant therapy for women with early breast cancer. J Clin Oncol 2001;19:980–991. 53. Faubion WA Jr, Loftus EV Jr, Harmsen WS, et al.: The natural history of corticosteroid therapy for inflammatory bowel disease: A population‐based study. Gastroenterology 2001;121:255–260. 54. Rutgeerts P, Sandborn WJ, Feagan BG, et al.: Infliximab for induction and maintenance therapy for ulcerative colitis. N Engl J Med 2005;353:2462–2476. 55. Jess T, Riis L, Vind I, et al.: Changes in clinical characteristics, course, and prognosis of inflammatory bowel disease during the last 5 decades: A population‐based study from Copenhagen, Denmark. Inflamm Bowel Dis 2007;13:481–489. 56. Khavanin N, Kim JYS, Davila AA, et al.: The Bra score: Creating a General Risk Calculator for Breast Reconstruction Outcomes. In “Plastic Surgery Research Council 59th Annual Meeting.” New York City, New York, 2014 abstract. 57. Breast Reconstruction Risk Assessment (BRA) Score. In Medicine NFSo (ed), 2014. 58. American College of Surgeons National Surgical Quality Improvement Program (ACS NSQIP). User Guide for the 2012 ACS NSQIP

Journal of Surgical Oncology

59. 60. 61. 62.

63. 64. 65. 66. 67. 68. 69.

Participant Use Data File. October 2013. In http://site.acsnsqip.org/wp‐ content/uploads/2013/10/ACSNSQIP.PUF_.UserGuide.2012.pdf. In H, Bilimoria KY, Stewart AK, et al.: Cancer recurrence: An important but missing variable in national cancer registries. Ann Surg Oncol 2014;21:1520–1529. User Guide for the 2012 ACS NSQIP Participant Use Data File. In “American College of Surgeons National Surgical Quality Improvement Program.” 2012. Levy AG, Sonnad SS, Kurichi JE, et al.: Making sense of cancer risk calculators on the web. J Gen Intern Med 2008;23:229–235. Ratzan S, Parker R: Introduction. In: Selden C, Zorn M, Ratzan S, Parker R, editors. National Library of Medicine current bibliographies in medicine: Health literacy. Bethesda, MD: National Institues of Health; 2000. Paasche‐Orlow MK, Parker RM, Gazmararian JA, et al.: The prevalence of limited health literacy. J Gen Intern Med 2005;20:175–184. Berkman ND, Sheridan SL, Donahue KE, et al.: Low health literacy and health outcomes: An updated systematic review. Ann Intern Med 2011;155:97–107. IOM (Institute of Medicine). Innovations in health literacy research: Workshop summary. Washington, DC: The National Academies Press; 2011. McCaffery K, Holmes‐Rovner M, Smith S, et al.: Addressing health literacy in patient decision aids. BMC Med Inform Decis Mak 2013;13:1–14. IOM (Insitute of Medicine). Health literacy and numeracy: Workshop summary. Washington, DC: The National Academies Press; 2014. NCES NCfES. Highlights from the 2003 international adult literacy and lifeskills survey (ALL). Washington, DC: U.S. Department of Education; 2005. Service GT: 2012 annual statistical report on the GED test. In: GED Testing Service, 2013.