Gynecologic Oncology 136 (2015) 30–36
Contents lists available at ScienceDirect
Gynecologic Oncology journal homepage: www.elsevier.com/locate/ygyno
Nomogram for predicting incomplete cytoreduction in advanced ovarian cancer patients☆ Seung-Hyuk Shim a, Sun Joo Lee a, Seon-Ok Kim b, Soo-Nyung Kim a, Dae-Yeon Kim c,⁎, Jong Jin Lee d, Jong-Hyeok Kim c, Yong-Man Kim c, Young-Tak Kim c, Joo-Hyun Nam c a
Department of Obstetrics and Gynecology, Konkuk University School of Medicine, Seoul, Republic of Korea Department of Clinical Epidemiology and Biostatistics, University of Ulsan College of Medicine, Asan Medical Center, Seoul, Republic of Korea Department of Obstetrics and Gynecology, University of Ulsan College of Medicine, Asan Medical Center, Seoul, Republic of Korea d Department of Nuclear Medicine, University of Ulsan College of Medicine, Asan Medical Center, Seoul, Republic of Korea b c
H I G H L I G H T S • 343 consecutive advanced-ovarian cancer patients undergoing PET/CT before primary surgery were analyzed. • Using surgical aggressiveness and PET/CT features, a nomogram for predicting incomplete cytoreduction was developed. • Nomogram performance was good across individual surgeons of heterogeneous surgical aggressiveness.
a r t i c l e
i n f o
Article history: Received 17 June 2014 Accepted 2 November 2014 Available online 9 November 2014 Keywords: Ovarian cancer Nomogram Residual cancer Positron emission tomography and computed tomography (PET/CT) Surgical specialty
a b s t r a c t Objective. Accurately predicting cytoreducibility in advanced-ovarian cancer is needed to establish preoperative plans, consider neoadjuvant chemotherapy, and improve clinical trial protocols. We aimed to develop a positronemission tomography/computed tomography-based nomogram for predicting incomplete cytoreduction in advanced-ovarian cancer patients. Methods. Between 2006 and 2012, 343 consecutive advanced-ovarian cancer patients underwent positronemission tomography/computed tomography before primary cytoreduction: 240 and 103 patients were assigned to the model development or validation cohort, respectively. After reviewing the detailed surgical documentation, incomplete cytoreduction was defined as a remaining gross residual tumor. We evaluated each individual surgeon's surgical aggressiveness index (number of high-complex surgeries/total number of surgeries). Possible predictors, including surgical aggressiveness index and positron-emission tomography/ computed tomography features, were analyzed using logistic regression modeling. A nomogram based on this model was developed and externally validated. Results. Complete cytoreduction was achieved in 120 patients (35%). Surgical aggressiveness index and five positron-emission tomography/computed tomography features were independent predictors of incomplete cytoreduction. Our nomogram predicted incomplete cytoreduction by incorporating these variables and demonstrated good predictive accuracy (concordance index = 0.881; 95% CI = 0.838–0.923). The predictive accuracy of our validation cohort was also good (concordance index = 0.881; 95% CI = 0.790–0.932) and the predicted probability was close to the actual observed outcome. Our model demonstrated good performance across surgeons with varying degrees of surgical aggressiveness. Conclusion. We have developed and validated a nomogram for predicting incomplete cytoreduction in advanced-ovarian cancer patients which may help stratify patients for clinical trials, establish meticulous preoperative plans, and determine if neoadjuvant chemotherapy is warranted. © 2014 Elsevier Inc. All rights reserved.
Introduction ☆ The results of this manuscript were presented in part at the SGO 44th Annual Meeting on Women's Cancer, which was held from March 9–12, 2013 at the Los Angeles Convention Center, Los Angeles, CA. ⁎ Corresponding author at: Department of Obstetrics and Gynecology, University of Ulsan College of Medicine, Asan Medical Center, 388-1, Pungnap-dong, Songpa-gu, Seoul 138-736, Republic of Korea. Fax: +82 2 476 7331. E-mail address: [email protected] (D.-Y. Kim).
http://dx.doi.org/10.1016/j.ygyno.2014.11.004 0090-8258/© 2014 Elsevier Inc. All rights reserved.
Epithelial ovarian cancer comprises 25% of malignancies in the female genital tract and is the most common gynecological cause of death in developed countries [1]. It is the second most common gynecological malignancy in Korea where an estimated 1300 new cases develop annually [2]. Primary cytoreductive surgery followed by taxane/
S.-H. Shim et al. / Gynecologic Oncology 136 (2015) 30–36
platinum-based chemotherapy is the first-line therapy for ovarian cancer. Maximal cytoreduction is one of the most important prognostic factors for treating advanced-ovarian cancer; patients who demonstrate complete cytoreduction (i.e. no macroscopic residual tumor left) demonstrate the best prognosis after adjuvant chemotherapy [3,4]. Accurately predicting incomplete cytoreduction is needed in advanced-ovarian cancer cases for several reasons [5]. First, if a patient is at high-risk of incomplete cytoreduction, alternative treatments such as neoadjuvant chemotherapy could be an option. For such cases, cytoreductive surgery following neoadjuvant chemotherapy seems to reduce operative morbidity and increase rates of complete cytoreduction [4,6]. Second, it may help to predict the surgical requirements to achieve complete cytoreduction. If greater surgical skill and multi-disciplinary resources are required to achieve complete cytoreduction, surgeons can consider whether to refer a patient to colleagues with more experience and resources. Third, further clinical trials are needed to confirm the efficacy of neoadjuvant chemotherapy in patients at high-risk of incomplete cytoreduction. In such trials, adequate stratification based on properly assessed surgical respectability will be necessary. As a triage tool, modeling can be useful for patient stratification in such trials. Biochemical markers and diagnostic imaging features are predictors of cytoreducibility [7–10]. Although models that incorporate these predictors have been developed [8,9,11], they cannot be independently validated [11] because of differences in individual surgical policies or skills [12]. To address the limitations of previous models, we assessed surgical aggressiveness of individual surgeons and developed a predictive nomogram for incomplete cytoreduction in advanced-ovarian cancer patients after primary surgery. Moreover, we used positronemission tomography/computed tomography to identify predictors of cytoreducibility [13,14]. This hybrid system more accurately detects extrapelvic metastasis in comparison with conventional imaging [15–20]. Methods Patients Patients included in this analysis were treated at our institution from 2006 to 2013 with a suspicious ovarian cancer on the basis of physical examination, ultrasound, computed tomography, presence of ascites or increased cancer antigen-125 (CA-125) level [21,22]. Initial preoperative positron-emission tomography/computed tomography has been recommended to all suspicious ovarian cancer patients and was performed on all patients except those who refused this procedure. All eligibility criteria for inclusion in our present study had to be met: age N18 and b80 years; pathologically confirmed ovarian cancer; positron-emission tomography/computed tomography performed 4 weeks prior to surgery; primary staging and subsequent cytoreductive surgery; and postoperative diagnosis of stages III–IV cancer according to the International Federation of Gynecology and Obstetrics (FIGO). We excluded patients who did not receive primary treatment at our institution, patients who received neoadjuvant chemotherapy, and patients with a history of other malignancies. Neoadjuvant chemotherapy was administered at the physician's discretion, especially for patients who were unable to receive surgical procedures due to poor physical condition or extraabdominal disease. Indication for neoadjuvant chemotherapy in our institution is described in the Supplemental Methods. In our institutional database of 444 advanced-ovarian cancer patients, 343 patients were eligible for this study. Before analysis, patients were allocated at a 7:3 ratio to either the model development (n = 240; January 2006 to June 2011) or validation cohort (n = 103; July 2011 to August 2013) (Supplemental Fig. S1). Clinicopathological data were collected from the medical records. Surgical staging was determined according to FIGO guidelines. If
31
indicated, patients received ≥ 6 cycles of taxane/platinum-based systemic chemotherapy after surgery. Surgical exploration of the abdominal cavity was performed systematically as described previously [23]. Visual estimation of tumor spread was based on the consensus of two operators. Multiple biopsies were obtained to confirm the results of macroscopic evaluation. After surgery, details regarding the presence, location, number, and size of the residual tumor were also recorded and illustrated on a surgical documentation form. This study was approved by our institutional review board (S2013-1521-0001). PET/CT scanning procedure Patients were instructed to avoid strenuous exercise for 24 h before positron-emission tomography/computed tomography in order to minimize radiotracer uptake into the muscles. They were also instructed to fast ≥6 h prior to the injection of 18F-fluorodeoxyglucose, which was produced in our center's radiopharmacy using standard synthetic techniques. Furosemide (40-mg tablet) and duspatalin (135-mg tablet) were orally administered just before venous blood glucose measurement. Venous blood glucose levels were maintained b 140 mg/dl. All patients were injected with 0.2 mCi/kg 18F-fluorodeoxyglucose and allowed to rest in a sitting or supine position for approximately 60 min prior to scanning. The patients were then positioned in the scanner with their arms above their heads. Positron-emission tomography/ computed tomography scans from the base of the skull to the midthigh were performed using Discovery STE (GE Healthcare, Waukesha, WI), Biograph Truepoint 16 (Siemens/CTI, Knoxville, TN), or Biograph Truepoint 40 (Siemens/CTI) scanners. The scanners obtained combination multislice computed tomography and positron-emission tomography tomographs. The computed tomography data were used for attenuation correction. A total of five to six bed positions for 2–3 min per position were acquired for emission scanning (3 min/bed with the Discovery STE and Biograph Truepoint 16; 2 min/bed with the Biograph Truepoint 40). All scans were reconstructed using an ordered-subsets expectation maximization algorithm (20 subsets and two iterations for the Discovery STE; 16 subsets and two iterations for the Biograph Truepoint 16; 21 subsets and three iterations for the Biograph Truepoint 40). Calibration of each scanner against dose calibrators and well counters was routinely performed. The measured standardized uptake value of the phantom was within the acceptable range of 90–110%. The mean standardized uptake value of the liver was also calculated by drawing a three-dimensional region of interest with a 3-cm diameter within the normal inferior right lobe. Further details with regard to the criteria used to interpret positron-emission tomography scans are described in the Supplemental Methods. Statistical analysis The following variables were assessed to identify predictors of incomplete cytoreduction: age, parity, menopausal status, American Society of Anesthesiology physical status, preoperative serum CA-125, serum albumin, and platelet count. We evaluated the extent of each surgery using surgical complexity score [24]. Based on the number and complexity of the surgical procedures, patients were assigned to three groups: low, intermediate, or high (Supplemental Table 1). The surgical aggressiveness index of the individual surgeons was calculated using the following formula: Surgical aggressiveness index of surgeon A = (number of high-surgical complexity score surgeries of surgeon A/total number of surgeries of surgeon A) × 100. The index was calculated using all primary cytoreductive surgeries for advanced ovarian cancer patients (excluding cervical or corpus cancer or early ovarian cancer) during the study period (2006–2013). Complete cytoreduction was defined as ‘no gross residual tumor’ [25]. Age, parity, preoperative serum CA-125, preoperative serum albumin, preoperative platelet count, surgical aggressiveness index, and
32
S.-H. Shim et al. / Gynecologic Oncology 136 (2015) 30–36
tumoral uptake ratio were considered continuous variables. Menopausal status and 11 positron-emission tomography/computed tomography features (diaphragm disease, large bowel mesentery implant, pleural effusion, large-volume ascites, peritoneal carcinomatosis, small bowel mesentery implant, rectal implant, pelvic lymph nodes, retroperitoneal lymph nodes, inguinal lymph nodes, lymph nodes outside the abdomen) were considered dichotomous variables. Nomograms were constructed as previously described [26,27]. To develop a robust and well-calibrated nomogram predicting the risk of incomplete cytoreduction, a logistic regression model was built using a development cohort of 240 patients and validated with a cohort of 103 patients. Categorical variables were grouped before modeling. Bivariate relationships between risk factors and incomplete cytoreduction were assessed using the model development cohort. Predictive values were determined using univariate analysis (P b 0.2) and tested using bootstrap resampling, in which 1000 repetitions were included in the logistic regression model with backward elimination. Criterion for inclusion in the final logistic model was 50% relative selection frequency. To assess model fit, the concordance index was used to measure discrimination by calculating the area under the receiver operating characteristics curve. The Hosmer–Lemeshow test was used to assess calibration. The model was applied to the validation cohort for external validation. Using the same methods, discrimination and model calibration were tested. Positive likelihood ratio was calculated using the following formula: Positive likelihood ratio = sensitivity/(1 − specificity). All analyses were performed using SPSS (version 19.0; SPSS, Chicago, IL) and R version 3.0.0 (http://cran.r-project.org/mirrors.html). In this study, P b 0.05 was considered significant.
Results Patient characteristics The characteristics of the model development and validation cohorts are summarized in Table 1. High-surgical complexity score surgery was performed on 27.9% (67 of 240) and 31.1% (32 of 103) of patients in the model development and validation cohorts, respectively. The complete cytoreduction rates of the model development and validation cohorts were 33.8% (81 of 240) and 37.9% (39 of 103 patients), respectively. The sensitivity, specificity, and positive and negative predictive values were calculated for positron-emission tomography/computed tomography using operative and histopathological findings as the reference standard. The results of the analyses made on a per-patient basis in different intra-abdominal areas are shown in Supplemental Table 2. Predicting incomplete cytoreduction in the model development and validation cohorts Table 2 shows the results of the logistic regression analyses used to identify predictors of incomplete cytoreduction. After bootstrap resampling, the final model indicated surgical aggressiveness index and five positron-emission tomography/computed tomography features (diaphragm disease, ascites, peritoneal carcinomatosis, small bowel mesentery implant, tumoral uptake ratio) as statistically significant predictors. A nomogram was constructed based on this logistic regression model (Fig. 1). The point value assigned to each factor was proportional to the odds ratio derived from its own beta coefficients determined by
Table 1 Characteristics of the model development and validation cohorts. Characteristics Age, years BMI, kg/m2 Parity, n Menopause, n (%) ASA score, n (%)
FIGO stage, n (%)
Histology, n (%)
Grade, n (%)
Surgical complexity score group
Preoperative CA-125, U/ml Preoperative albumin, g/dl Preoperative platelet count, 103/mm3 SUVmax in the primary tumor Tumoral uptake ratioa Residual tumor
Median (range) Median (range) Median (range) No Yes 1 2 3 IIIA IIIB IIIC IV Serous Mucinous Endometrioid Clear cell Transitional cell Carcinosarcoma Others 1 2 3 unknown ≤3 (low) 4–7 (intermediate) ≥8 (high) Median (range) Median (range) Median (range) Median (range) Median (range) No gross residual 0–1 cm N1 cm
Model development cohort (n = 240)
Validation cohort (n = 103)
P
55 (27–80) 23.7 (15.3–36.4) 2 (0–8) 85 (35.4) 155 (64.6) 36 (15.0) 194 (80.8) 10 (4.1) 6 (2.5) 10 (4.2) 173 (72.1) 51 (21.3) 185 (77.1) 5 (2.1) 9 (3.8) 19 (7.9) 6 (2.5) 10 (4.2) 6 (2.5) 5 (2.1) 19 (7.9) 185 (77.1) 31 (12.9) 26 (10.8) 147 (61.3) 67 (27.9) 645 (4–123000) 3.6 (1.9–4.6) 318 (111–648) 9.6 (1.6–45.3) 0.72 (0–2.06) 81 (33.8) 94 (39.2) 65 (27.1)
54 (20–76) 23.3 (15.3–34.3) 2 (0–7) 37 (35.9) 66 (64.1) 14 (13.6) 85 (82.5) 4 (3.9) 2 (1.9) 4 (3.9) 67 (65.0) 30 (29.1) 89 (86.4) 0 4 (3.9) 3 (2.9) 4 (3.9) 1 (1.0) 2 (1.9) 4 (3.9) 11 (10.7) 78 (75.7) 10 (9.7) 5 (4.9) 66 (64.1) 32 (31.1) 879 (13–17145) 3.6 (2.1–4.7) 330 (139–736) 10.1 (2.6–26.3) 0.69 (0–2.07) 39 (37.9) 34 (33.0) 30 (29.1)
0.572 0.569 0.131 0.511 0.855
0.473
0.197
0.147
0.203
0.198 0.692 0.963 0.587 0.461 0.270
Abbreviations: BMI, body mass index; ASA, American Society of Anesthesiology score; FIGO, International Federation of Gynecology and Obstetrics; CA-125, cancer antigen 125; SUVmax, maximum standard uptake value. a Tumoral uptake ratio = (highest SUVmax of the lesions in the upper abdominal region)/(highest SUVmax of the lesions in the lower abdominal region).
S.-H. Shim et al. / Gynecologic Oncology 136 (2015) 30–36
33
Table 2 Univariate and multivariate logistic regression models for predicting incomplete cytoreduction. Variables
n
Univariate analysis Odds ratio (95% CI)
Age, yearsa BMI, kg/m2a Parity, na Menopause ASA
Diaphragm disease Large bowel mesentery implant Pleural effusion Ascites Peritoneal carcinomatosis Small bowel mesentery implant Rectal implants Pelvic lymph nodes N1 cm Retroperitoneal lymph nodes N1 cm Inguinal lymph nodes N1 cm Lymph nodes outside abdomen N1 cm SUVmax in the primary tumora Tumoral uptake ratioa,b Log of preoperative CA-125a Preoperative albumin, g/dla Preoperative platelet count, 103/mm3a Surgical aggressiveness index, %a
No (62) Yes (96) 1 (36) 2 (194) 3 (10) No (144) Yes (96) No (152) Yes (88) No (209) Yes (31) No (101) Yes (139) No (35) Yes (205) No (155) Yes (85) No (202) Yes (38) No (156) Yes (84) No (125) Yes (115) No (234) Yes (6) No (154) Yes (86)
1.028 (1.003–1.052) 0.947 (0.858–1.044) 1.254 (1.032–1.524) 1 1.415 (0.813–2.460) 1 1.636 (0.619–4.327) 0.136 (0.014–1.351) 1 7.310 (3.601–14.837) 1 3.735 (1.966–7.094) 1 1.285 (0.562–2.935) 1 4.337 (2.477–7.736) 1 6.652 (3.004–14.731) 1 4.329 (2.214–8.463) 1 1.302 (0.610–2.779) 1 1.570 (0.880–2.801) 1 1.953 (1.129–3.378) 1 2.597 (0.298–22.611) 1 2.886 (1.553–5.365) 1.015 (0.964–1.068) 11.961 (5.157–27.745) 1.349 (1.120–1.624) 0.263 (0.148–0.467) 1.003 (1.000–1.006) 0.947 (0.922–0.974)
Multivariate analysis P
Odds ratio (95% CI)
P
0.026 0.275 0.023 0.219 0.321
b0.001
1 3.654 (1.604–8.324)
0.002
b0.001 0.552 b0.001 b0.001 b0.001
1 3.027 (1.446–6.338) 1 3.731 (1.268–10.982) 1 2.978 (1.276–6.951)
0.012
8.231 (3.036–22.311)
b0.001
0.926 (0.893–0.960)
b0.001
0.003 0.017
0.496 0.127 0.017 0.387 0.001 0.580 b0.001 0.002 b0.001 0.052 b0.001
Abbreviations: BMI, body mass index; ASA, American Society of Anesthesiology score; CA-125, cancer antigen 125; SUVmax, maximum standard uptake value; CI, confidence interval. a Continuous variable. b Tumoral uptake ratio = (highest SUVmax of the lesions in the upper abdominal region)/(highest SUVmax of the lesions in the lower abdominal region).
Fig. 1. Nomogram that can predict incomplete cytoreduction in patients with advanced epithelial ovarian cancer. This nomogram incorporates six variables. For each level of each prognostic variable, points were allocated according to the scale shown here. The total score was determined by adding individual parameter points and used to calculate the predicted probability of incomplete cytoreduction. A total score of 128 was assigned a value of 0.8 and used to define the groups at high-risk of incomplete cytoreduction.
34
S.-H. Shim et al. / Gynecologic Oncology 136 (2015) 30–36
Fig. 2. Performance of the nomogram for predicting incomplete cytoreduction in patients with advanced epithelial ovarian cancer. (A) After 1000 repetitions, the bootstrap-corrected concordance index of the model development cohort was 0.881 (95% CI = 0.838–0.923). In the validation cohort, the bootstrap-corrected concordance index was 0.861 (95% CI = 0.790–0.932). (B) Calibration plots of the nomogram for the model development cohort. (C) Calibration plots of the nomogram for the validation cohort. The dashed line represents the performance of an ideal nomogram, in which 100% of incomplete cytoreductions match the actuarial incomplete cytoreduction rate. The solid line represents the performance of the actual nomogram. The filled dots were obtained from a subcohort in the database, and the vertical bars indicate 95% confidence interval. When plotting the incomplete cytoreduction probabilities predicted by the current nomogram against the actuarial incomplete cytoreduction rate, the calibration curve was close to the dashed line.
the regression analysis. Internal validation was performed using bootstrapping resampling. After 1000 repetitions, the bootstrapcorrected concordance index of the model was 0.881 (95% CI = 0.838–0.923). In the validation cohort, the discrimination accuracy of the model was 0.861 (95% CI = 0.790–0.932) (Fig. 2A). Fig. 2B–C shows the calibration plots of the nomogram for the model development and validation cohorts, respectively. The Hosmer–Lemeshow test yielded a P value of 0.274 for the model development cohort, indicating that the nomogram was well-fitted. For the validation cohort, the nomogram also fit the data (P = 0.214; Hosmer–Lemeshow test). We also performed same analysis for suboptimal cytoreduction (maximum disease greater than 1 cm). The surgical aggressiveness index, lymph nodes outside the abdomen, and tumoral uptake ratio were revealed as statistically significant predictors for suboptimal cytoreduction (Supplemental Table 3). A nomogram was constructed based on these predictors (Supplemental Fig. S2). The concordance indices of the model were 0.793 (95% CI = 0.727–0.859) and 0.818 (95% CI = 0.737–0.899) for the model development and validation cohorts, respectively (Supplemental Fig. S3). Identification of patients at high risk of incomplete cytoreduction We defined the high-risk group as having a predicted probability of incomplete cytoreduction of greater than 80%. To keep a balance between positive likelihood ratio and false negativity, we set the cutoff value of 80% predicted probability of incomplete cytoreduction to identify the high-risk group. The nomogram thereby classified 115 of 240 patients (48%) in the model development cohort as high-risk. In that group, the predicted probability of incomplete cytoreduction was 92.7%, and the actual incomplete cytoreduction rate was 92.2%
(106 out of 115). In the validation cohort, 48 of 103 patients (47%) were classified as high-risk. In that group, the predicted probability of incomplete cytoreduction was 93.1%, and the actual incomplete cytoreduction rate was 87.5% (42 out of 48). After combining the model development and validation cohorts, the positive likelihood ratio for the high-risk group was 5.3 (95% CI = 3.3–8.6).
Model performance across individual surgeons We tested the performance of our nomogram by applying it to subsets of individual surgeons with varying degrees of surgical aggressiveness. Supplemental Table 5 summarizes the types of procedures performed. The surgical aggressiveness of the attending surgeons was heterogeneous: surgeon C performed high-surgical complexity score surgery in N40% of cases, while surgeon F performed high-surgical complexity score surgery in b5% of cases. The complete cytoreduction rate was also heterogeneous (range = 50.6–9.1%) (Table 3). For surgeon C, the concordance index of the model was 0.809 (95% CI = 0.713–0.905). For surgeon F, who performed less extensive procedures, our model demonstrated a concordance index of 0.950 (95% CI = 0.852–1.000). The model also demonstrated good performance for other surgeons with various surgical aggressiveness (Table 3).
Discussion In the present study, we developed a non-invasive and user-friendly nomogram for predicting incomplete cytoreduction in advancedovarian cancer patients using surgical aggressiveness index and positron-emission tomography/computed tomography. The nomogram
Table 3 Surgical aggressiveness indices and model discrimination for each attending surgeon. Surgeon
A B C D E F
Total surgeries (n)
58 92 77 48 46 22
Incomplete cytoreduction, n (%)
44 (76) 61 (66) 38 (49) 32 (67) 28 (61) 20 (91)
Abbreviations: AUC, area under the curve; CI, confidence interval.
Surgical complexity score
Surgical aggressiveness index
High, n (%)
Intermediate, n (%)
Low, n (%)
15 (26) 29 (32) 32 (42) 5 (10) 17 (37) 1 (5)
36 (62) 62 (67) 45 (58) 34 (71) 27 (59) 9 (41)
7 (12) 1 (1) 0 9 (19) 2 (4) 12 (54)
Model discrimination AUC (95% CI)
25.86 31.52 41.56 10.42 36.96 4.55
0.883 (0.799–0.968) 0.859 (0.775–0.943) 0.809 (0.713–0.905) 0.973 (0.937-1.000) 0.834 (0.705–0.964) 0.950 (0.852–1.000)
S.-H. Shim et al. / Gynecologic Oncology 136 (2015) 30–36
accurately identified the high-risk group of incomplete cytoreduction and was validated using external data. Compared with previous models, a major strength of our nomogram is that individual surgeon factor was considered. Surgeons with high surgical capability and a multidisciplinary team can achieve a high rate of complete cytoreduction [5]. To estimate the surgical complexity levels at an individual institution, Jung et al. designed the surgical skill index [28]. However, surgical aggressiveness will differ even within the same institution. Thus, we modified this parameter and calculated the surgical aggressiveness index for each attending surgeon and our nomogram demonstrated similarly high performances for different surgical efficiency. To develop a reliable model, all clinical predictors should be tested for inclusion. Although many previous studies have included only a small number of patients, our present study cohorts was sufficiently large to test candidate predictors. Potential predictors of cytoreducibility might include clinical characteristics, and the most important may be tumor location and size [5]. Commonly suggested predictors include diaphragmatic disease [8,10,11], large-volume ascites [8,9], mesenteric disease [8,10,11], and diffuse peritoneal implants [8,9,13]. We tested such clinically relevant predictors to construct our model. According to National Comprehensive Cancer Network and American College of Radiology Appropriateness Criteria, positronemission tomography/computed tomography is useful for initial staging [29,30]. For the pretreatment staging and evaluation of distant ovarian carcinoma metastases, the best performances have been reported using positron-emission tomography/computed tomography [15,17, 18,20,29,30]. Thus, this procedure could identify patients who are ineligible for complete cytoreduction [19]. We here used previously reported positron-emission tomography/computed tomography features as predictors for cytoreducibility [14]. The concordance index of our nomogram was 0.881, which indicates high predictive power [5]. The model was also tested using computed tomography instead of positron-emission tomography/computed tomography. The concordance index of the computed tomography-based model was 0.861 (95% CI = 0.814–0.908) (Supplemental Table 4). However, the Hosmer–Lemeshow test indicated poor fit in the validation cohort (P = 0.033). Thus, we chose the multivariate model based on positron-emission tomography/computed tomography. The intent of this study is not to recommend routine neoadjuvant chemotherapy. Although a recent European Organization for Research and Treatment of Cancer (EORTC)/National Cancer Institute of Canada (NCIC) trial reported that neoadjuvant chemotherapy strategy is not inferior to primary debulking surgery in advanced-ovarian cancer patients [31], some authors still suggest that neoadjuvant chemotherapy requires further investigation; a major criticism is that the survival outcomes of the primary cytoreductive surgery arm were alarmingly low (30 months) [4]. In another study that used identical inclusion criteria as the EORTC–NCIC trial, the optimal cytoreduction rate was 71% and the median overall survival period was 50 months [32]. In addition, the EORTC-NCIC trial did not elucidate the advantage of neoadjuvant chemotherapy in terms of adverse effects, or quality of life [5]. Accordingly, further trials are needed to confirm the efficacy of neoadjuvant chemotherapy in patients at high-risk of incomplete cytoreduction. Our current study demonstrated a positive likelihood ratio value of 5.3 for high-risk patients, suggesting that the nomogram is useful for identifying such patients [33]. If greater surgical skill and multi-disciplinary resources are required to achieve complete cytoreduction, surgeons can consider patient referrals. Our nomogram may be useful in such decision making. We do not think, however, that all surgeries require a more experienced or aggressive surgeon in attendance to allow complete cytoreduction until enough evidence is accumulated. From our current data, there was a difference in morbidity and resource consumption based on the surgical complexity score. In addition, surgeons with a higher surgical aggressiveness index can also have higher complication rates and cost.
35
Considering that cytoreduction comes at a cost, aggressive surgery should be tailored with respect to the general condition and extent of disease in the patient [6]. Our present study has several limitations. First, positron-emission tomography/computed tomography was not performed on all cases. However, this procedure has been recommended for all suspected cases of ovarian cancer and was performed on consecutive patients. Thus, most of our advanced ovarian cancer patients (343/444; 77%) underwent preoperative PET/CT (Supplemental Fig. S1). Second, the estimated surgical aggressiveness index still requires validation. Moreover, inclusion of the index in the nomogram may limit the generalizability beyond that of the surgeons tested. The model excluding this index is outlined in Supplemental Table 6 and Supplemental Fig. S4. However, although exclusion of the index may enhance the generalizability of the model, it may also limit the accuracy of the model among surgeons with different surgical aggressiveness. Third, although we validated the model using external data, it must still be validated by other medical institutions to assess its general applicability. In this regard, we created a web-based version of the nomogram (http:// glimmer.rstudio.com/jeongyoonlee/nomogram/). This web-based tool may encourage other institutions to evaluate whether our nomogram could be effectively used in their clinics. We are also conducting a multicenter-prospective study to validate our nomogram. Despite these limitations, our nomogram represents the first attempt to incorporate both positron-emission tomography/computed tomography and surgeon factor to predict incomplete cytoreduction in advancedovarian cancer patients. Eventually, the nomogram may produce better outcomes and reduced morbidity in advanced-ovarian cancer patients. Accordingly, our new model should be further tested using multi-institutional, prospective, validation studies. Supplementary data to this article can be found online at http://dx. doi.org/10.1016/j.ygyno.2014.11.004. Conflict of interest statement The authors have no conflicts of interest or financial ties to disclose.
Acknowledgments We thank Jeong-Yoon Lee, PhD who made a computerized version of the nomogram in line with the web-site. References [1] Jemal A, Siegel R, Xu J, Ward E. Cancer statistics, 2010. CA Cancer J Clin 2010;60: 277–300. [2] Lim MC, Moon EK, Shin A, Jung KW, Won YJ, Seo SS, et al. Incidence of cervical, endometrial, and ovarian cancer in Korea, 1999–2010. J Gynecol Oncol 2013;24: 298–302. [3] Bristow RE, Tomacruz RS, Armstrong DK, Trimble EL, Montz FJ. Survival effect of maximal cytoreductive surgery for advanced ovarian carcinoma during the platinum era: a meta-analysis. J Clin Oncol 2002;20:1248–59. [4] Chi DS, Bristow RE, Armstrong DK, Karlan BY. Is the easier way ever the better way? J Clin Oncol 2011;29:4073–5. [5] Kang S, Park SY. To predict or not to predict? The dilemma of predicting the risk of suboptimal cytoreduction in ovarian cancer. Ann Oncol 2011;22(Suppl. 8):viii23–8. [6] Vergote I, Trope CG, Amant F, Ehlen T, Reed NS, Casado A. Neoadjuvant chemotherapy is the better treatment option in some patients with stage IIIc to IV ovarian cancer. J Clin Oncol 2011;29:4076–8. [7] Parker D, Bradley C, Bogle SM, Lay J, Masood M, Hancock AK, et al. Serum albumin and CA125 are powerful predictors of survival in epithelial ovarian cancer. Br J Obstet Gynaecol 1994;101:888–93. [8] Bristow RE, Duska LR, Lambrou NC, Fishman EK, O'Neill MJ, Trimble EL, et al. A model for predicting surgical outcome in patients with advanced ovarian carcinoma using computed tomography. Cancer 2000;89:1532–40. [9] Dowdy SC, Mullany SA, Brandt KR, Huppert BJ, Cliby WA. The utility of computed tomography scans in predicting suboptimal cytoreductive surgery in women with advanced ovarian carcinoma. Cancer 2004;101:346–52. [10] Ferrandina G, Sallustio G, Fagotti A, Vizzielli G, Paglia A, Cucci E, et al. Role of CT scan-based and clinical evaluation in the preoperative prediction of optimal cytoreduction in advanced ovarian cancer: a prospective trial. Br J Cancer 2009; 101:1066–73.
36
S.-H. Shim et al. / Gynecologic Oncology 136 (2015) 30–36
[11] Axtell AE, Lee MH, Bristow RE, Dowdy SC, Cliby WA, Raman S, et al. Multi-institutional reciprocal validation study of computed tomography predictors of suboptimal primary cytoreduction in patients with advanced ovarian cancer. J Clin Oncol 2007;25:384–9. [12] Bristow RE. Predicting surgical outcome for advanced ovarian cancer, surgical standards of care, and the concept of kaizen. Gynecol Oncol 2009;112:1–3. [13] Risum S, Hogdall C, Loft A, Berthelsen AK, Hogdall E, Nedergaard L, et al. Prediction of suboptimal primary cytoreduction in primary ovarian cancer with combined positron emission tomography/computed tomography–a prospective study. Gynecol Oncol 2008;108:265–70. [14] Shim SH, Kim DY, Seo MJ, Lee SW, Park JY, Lee JJ, et al. Preoperative fluorine 18 fluorodeoxyglucose tumoral uptake ratio between upper and lower abdomen in primary advanced-stage ovarian cancer. Int J Gynecol Cancer 2013;23:1383–92. [15] Kitajima K, Murakami K, Yamasaki E, Kaji Y, Fukasawa I, Inaba N, et al. Diagnostic accuracy of integrated FDG-PET/contrast-enhanced CT in staging ovarian cancer: comparison with enhanced CT. Eur J Nucl Med Mol Imaging 2008;35:1912–20. [16] Fruscio R, Sina F, Dolci C, Signorelli M, Crivellaro C, Dell'Anna T, et al. Preoperative 18 F-FDG PET/CT in the management of advanced epithelial ovarian cancer. Gynecol Oncol 2013;131:689–93. [17] Nam EJ, Yun MJ, Oh YT, Kim JW, Kim JH, Kim S, et al. Diagnosis and staging of primary ovarian cancer: correlation between PET/CT, Doppler US, and CT or MRI. Gynecol Oncol 2010;116:389–94. [18] Yuan Y, Gu ZX, Tao XF, Liu SY. Computer tomography, magnetic resonance imaging, and positron emission tomography or positron emission tomography/computer tomography for detection of metastatic lymph nodes in patients with ovarian cancer: a meta-analysis. Eur J Radiol 2012;81:1002–6. [19] Prakash P, Cronin CG, Blake MA. Role of PET/CT in ovarian cancer. AJR Am J Roentgenol 2010;194:W464–70. [20] Castellucci P, Perrone AM, Picchio M, Ghi T, Farsad M, Nanni C, et al. Diagnostic accuracy of 18 F-FDG PET/CT in characterizing ovarian lesions and staging ovarian cancer: correlation with transvaginal ultrasonography, computed tomography, and histology. Nucl Med Commun 2007;28:589–95. [21] Timmerman D, Ameye L, Fischerova D, Epstein E, Melis GB, Guerriero S, et al. Simple ultrasound rules to distinguish between benign and malignant adnexal masses before surgery: prospective validation by IOTA group. BMJ 2010;341:c6839. [22] Iyer VR, Lee SI. MRI, CT, and PET/CT for ovarian cancer detection and adnexal lesion characterization. AJR Am J Roentgenol 2010;194:311–21.
[23] Park JY, Kim DY, Suh DS, Kim JH, Kim YM, Kim YT, et al. Comparison of laparoscopy and laparotomy in surgical staging of early-stage ovarian and fallopian tubal cancer. Ann Surg Oncol 2008;15:2012–9. [24] Aletti GD, Santillan A, Eisenhauer EL, Hu J, Aletti G, Podratz KC, et al. A new frontier for quality of care in gynecologic oncology surgery: multi-institutional assessment of short-term outcomes for ovarian cancer using a risk-adjusted model. Gynecol Oncol 2007;107:99–106. [25] Chi DS, Eisenhauer EL, Lang J, Huh J, Haddad L, Abu-Rustum NR, et al. What is the optimal goal of primary cytoreductive surgery for bulky stage IIIC epithelial ovarian carcinoma (EOC)? Gynecol Oncol 2006;103:559–64. [26] Iasonos A, Schrag D, Raj GV, Panageas KS. How to build and interpret a nomogram for cancer prognosis. J Clin Oncol 2008;26:1364–70. [27] Kim DY, Shim SH, Kim SO, Lee SW, Park JY, Suh DS, et al. Preoperative nomogram for the identification of lymph node metastasis in early cervical cancer. Br J Cancer 2014;110:34–41. [28] Jung DC, Kang S, Kim SC, Kim JW, Nam JH, Ryu SY, et al. Use of complex surgical procedures, patterns of tumor spread, and CA-125 predicts a risk of incomplete cytoreduction: a Korean Gynecologic Oncology Group study (KGOG-3022). Gynecol Oncol 2013;131:336–40. [29] Mitchell DG, Javitt MC, Glanc P, Bennett GL, Brown DL, Dubinsky T, et al. ACR appropriateness criteria staging and follow-up of ovarian cancer. J Am Coll Radiol 2013;10: 822–7. [30] Morgan Jr RJ, Alvarez RD, Armstrong DK, Burger RA, Chen LM, Copeland L, et al. Ovarian cancer, version 2.2013. J Natl Compr Cancer Netw 2013;11:1199–209. [31] Vergote I, Trope CG, Amant F, Kristensen GB, Ehlen T, Johnson N, et al. Neoadjuvant chemotherapy or primary surgery in stage IIIC or IV ovarian cancer. N Engl J Med 2010;363:943–53. [32] Chi DS, Musa F, Dao F, Zivanovic O, Sonoda Y, Leitao MM, et al. An analysis of patients with bulky advanced stage ovarian, tubal, and peritoneal carcinoma treated with primary debulking surgery (PDS) during an identical time period as the randomized EORTC-NCIC trial of PDS vs neoadjuvant chemotherapy (NACT). Gynecol Oncol 2012;124:10–4. [33] Jaeschke R, Guyatt GH, Sackett DL. Users' guides to the medical literature. III. How to use an article about a diagnostic test. B. What are the results and will they help me in caring for my patients? The Evidence-Based Medicine Working Group. JAMA 1994; 271:703–7.
Contents lists available at ScienceDirect
Gynecologic Oncology journal homepage: www.elsevier.com/locate/ygyno
Nomogram for predicting incomplete cytoreduction in advanced ovarian cancer patients☆ Seung-Hyuk Shim a, Sun Joo Lee a, Seon-Ok Kim b, Soo-Nyung Kim a, Dae-Yeon Kim c,⁎, Jong Jin Lee d, Jong-Hyeok Kim c, Yong-Man Kim c, Young-Tak Kim c, Joo-Hyun Nam c a
Department of Obstetrics and Gynecology, Konkuk University School of Medicine, Seoul, Republic of Korea Department of Clinical Epidemiology and Biostatistics, University of Ulsan College of Medicine, Asan Medical Center, Seoul, Republic of Korea Department of Obstetrics and Gynecology, University of Ulsan College of Medicine, Asan Medical Center, Seoul, Republic of Korea d Department of Nuclear Medicine, University of Ulsan College of Medicine, Asan Medical Center, Seoul, Republic of Korea b c
H I G H L I G H T S • 343 consecutive advanced-ovarian cancer patients undergoing PET/CT before primary surgery were analyzed. • Using surgical aggressiveness and PET/CT features, a nomogram for predicting incomplete cytoreduction was developed. • Nomogram performance was good across individual surgeons of heterogeneous surgical aggressiveness.
a r t i c l e
i n f o
Article history: Received 17 June 2014 Accepted 2 November 2014 Available online 9 November 2014 Keywords: Ovarian cancer Nomogram Residual cancer Positron emission tomography and computed tomography (PET/CT) Surgical specialty
a b s t r a c t Objective. Accurately predicting cytoreducibility in advanced-ovarian cancer is needed to establish preoperative plans, consider neoadjuvant chemotherapy, and improve clinical trial protocols. We aimed to develop a positronemission tomography/computed tomography-based nomogram for predicting incomplete cytoreduction in advanced-ovarian cancer patients. Methods. Between 2006 and 2012, 343 consecutive advanced-ovarian cancer patients underwent positronemission tomography/computed tomography before primary cytoreduction: 240 and 103 patients were assigned to the model development or validation cohort, respectively. After reviewing the detailed surgical documentation, incomplete cytoreduction was defined as a remaining gross residual tumor. We evaluated each individual surgeon's surgical aggressiveness index (number of high-complex surgeries/total number of surgeries). Possible predictors, including surgical aggressiveness index and positron-emission tomography/ computed tomography features, were analyzed using logistic regression modeling. A nomogram based on this model was developed and externally validated. Results. Complete cytoreduction was achieved in 120 patients (35%). Surgical aggressiveness index and five positron-emission tomography/computed tomography features were independent predictors of incomplete cytoreduction. Our nomogram predicted incomplete cytoreduction by incorporating these variables and demonstrated good predictive accuracy (concordance index = 0.881; 95% CI = 0.838–0.923). The predictive accuracy of our validation cohort was also good (concordance index = 0.881; 95% CI = 0.790–0.932) and the predicted probability was close to the actual observed outcome. Our model demonstrated good performance across surgeons with varying degrees of surgical aggressiveness. Conclusion. We have developed and validated a nomogram for predicting incomplete cytoreduction in advanced-ovarian cancer patients which may help stratify patients for clinical trials, establish meticulous preoperative plans, and determine if neoadjuvant chemotherapy is warranted. © 2014 Elsevier Inc. All rights reserved.
Introduction ☆ The results of this manuscript were presented in part at the SGO 44th Annual Meeting on Women's Cancer, which was held from March 9–12, 2013 at the Los Angeles Convention Center, Los Angeles, CA. ⁎ Corresponding author at: Department of Obstetrics and Gynecology, University of Ulsan College of Medicine, Asan Medical Center, 388-1, Pungnap-dong, Songpa-gu, Seoul 138-736, Republic of Korea. Fax: +82 2 476 7331. E-mail address: [email protected] (D.-Y. Kim).
http://dx.doi.org/10.1016/j.ygyno.2014.11.004 0090-8258/© 2014 Elsevier Inc. All rights reserved.
Epithelial ovarian cancer comprises 25% of malignancies in the female genital tract and is the most common gynecological cause of death in developed countries [1]. It is the second most common gynecological malignancy in Korea where an estimated 1300 new cases develop annually [2]. Primary cytoreductive surgery followed by taxane/
S.-H. Shim et al. / Gynecologic Oncology 136 (2015) 30–36
platinum-based chemotherapy is the first-line therapy for ovarian cancer. Maximal cytoreduction is one of the most important prognostic factors for treating advanced-ovarian cancer; patients who demonstrate complete cytoreduction (i.e. no macroscopic residual tumor left) demonstrate the best prognosis after adjuvant chemotherapy [3,4]. Accurately predicting incomplete cytoreduction is needed in advanced-ovarian cancer cases for several reasons [5]. First, if a patient is at high-risk of incomplete cytoreduction, alternative treatments such as neoadjuvant chemotherapy could be an option. For such cases, cytoreductive surgery following neoadjuvant chemotherapy seems to reduce operative morbidity and increase rates of complete cytoreduction [4,6]. Second, it may help to predict the surgical requirements to achieve complete cytoreduction. If greater surgical skill and multi-disciplinary resources are required to achieve complete cytoreduction, surgeons can consider whether to refer a patient to colleagues with more experience and resources. Third, further clinical trials are needed to confirm the efficacy of neoadjuvant chemotherapy in patients at high-risk of incomplete cytoreduction. In such trials, adequate stratification based on properly assessed surgical respectability will be necessary. As a triage tool, modeling can be useful for patient stratification in such trials. Biochemical markers and diagnostic imaging features are predictors of cytoreducibility [7–10]. Although models that incorporate these predictors have been developed [8,9,11], they cannot be independently validated [11] because of differences in individual surgical policies or skills [12]. To address the limitations of previous models, we assessed surgical aggressiveness of individual surgeons and developed a predictive nomogram for incomplete cytoreduction in advanced-ovarian cancer patients after primary surgery. Moreover, we used positronemission tomography/computed tomography to identify predictors of cytoreducibility [13,14]. This hybrid system more accurately detects extrapelvic metastasis in comparison with conventional imaging [15–20]. Methods Patients Patients included in this analysis were treated at our institution from 2006 to 2013 with a suspicious ovarian cancer on the basis of physical examination, ultrasound, computed tomography, presence of ascites or increased cancer antigen-125 (CA-125) level [21,22]. Initial preoperative positron-emission tomography/computed tomography has been recommended to all suspicious ovarian cancer patients and was performed on all patients except those who refused this procedure. All eligibility criteria for inclusion in our present study had to be met: age N18 and b80 years; pathologically confirmed ovarian cancer; positron-emission tomography/computed tomography performed 4 weeks prior to surgery; primary staging and subsequent cytoreductive surgery; and postoperative diagnosis of stages III–IV cancer according to the International Federation of Gynecology and Obstetrics (FIGO). We excluded patients who did not receive primary treatment at our institution, patients who received neoadjuvant chemotherapy, and patients with a history of other malignancies. Neoadjuvant chemotherapy was administered at the physician's discretion, especially for patients who were unable to receive surgical procedures due to poor physical condition or extraabdominal disease. Indication for neoadjuvant chemotherapy in our institution is described in the Supplemental Methods. In our institutional database of 444 advanced-ovarian cancer patients, 343 patients were eligible for this study. Before analysis, patients were allocated at a 7:3 ratio to either the model development (n = 240; January 2006 to June 2011) or validation cohort (n = 103; July 2011 to August 2013) (Supplemental Fig. S1). Clinicopathological data were collected from the medical records. Surgical staging was determined according to FIGO guidelines. If
31
indicated, patients received ≥ 6 cycles of taxane/platinum-based systemic chemotherapy after surgery. Surgical exploration of the abdominal cavity was performed systematically as described previously [23]. Visual estimation of tumor spread was based on the consensus of two operators. Multiple biopsies were obtained to confirm the results of macroscopic evaluation. After surgery, details regarding the presence, location, number, and size of the residual tumor were also recorded and illustrated on a surgical documentation form. This study was approved by our institutional review board (S2013-1521-0001). PET/CT scanning procedure Patients were instructed to avoid strenuous exercise for 24 h before positron-emission tomography/computed tomography in order to minimize radiotracer uptake into the muscles. They were also instructed to fast ≥6 h prior to the injection of 18F-fluorodeoxyglucose, which was produced in our center's radiopharmacy using standard synthetic techniques. Furosemide (40-mg tablet) and duspatalin (135-mg tablet) were orally administered just before venous blood glucose measurement. Venous blood glucose levels were maintained b 140 mg/dl. All patients were injected with 0.2 mCi/kg 18F-fluorodeoxyglucose and allowed to rest in a sitting or supine position for approximately 60 min prior to scanning. The patients were then positioned in the scanner with their arms above their heads. Positron-emission tomography/ computed tomography scans from the base of the skull to the midthigh were performed using Discovery STE (GE Healthcare, Waukesha, WI), Biograph Truepoint 16 (Siemens/CTI, Knoxville, TN), or Biograph Truepoint 40 (Siemens/CTI) scanners. The scanners obtained combination multislice computed tomography and positron-emission tomography tomographs. The computed tomography data were used for attenuation correction. A total of five to six bed positions for 2–3 min per position were acquired for emission scanning (3 min/bed with the Discovery STE and Biograph Truepoint 16; 2 min/bed with the Biograph Truepoint 40). All scans were reconstructed using an ordered-subsets expectation maximization algorithm (20 subsets and two iterations for the Discovery STE; 16 subsets and two iterations for the Biograph Truepoint 16; 21 subsets and three iterations for the Biograph Truepoint 40). Calibration of each scanner against dose calibrators and well counters was routinely performed. The measured standardized uptake value of the phantom was within the acceptable range of 90–110%. The mean standardized uptake value of the liver was also calculated by drawing a three-dimensional region of interest with a 3-cm diameter within the normal inferior right lobe. Further details with regard to the criteria used to interpret positron-emission tomography scans are described in the Supplemental Methods. Statistical analysis The following variables were assessed to identify predictors of incomplete cytoreduction: age, parity, menopausal status, American Society of Anesthesiology physical status, preoperative serum CA-125, serum albumin, and platelet count. We evaluated the extent of each surgery using surgical complexity score [24]. Based on the number and complexity of the surgical procedures, patients were assigned to three groups: low, intermediate, or high (Supplemental Table 1). The surgical aggressiveness index of the individual surgeons was calculated using the following formula: Surgical aggressiveness index of surgeon A = (number of high-surgical complexity score surgeries of surgeon A/total number of surgeries of surgeon A) × 100. The index was calculated using all primary cytoreductive surgeries for advanced ovarian cancer patients (excluding cervical or corpus cancer or early ovarian cancer) during the study period (2006–2013). Complete cytoreduction was defined as ‘no gross residual tumor’ [25]. Age, parity, preoperative serum CA-125, preoperative serum albumin, preoperative platelet count, surgical aggressiveness index, and
32
S.-H. Shim et al. / Gynecologic Oncology 136 (2015) 30–36
tumoral uptake ratio were considered continuous variables. Menopausal status and 11 positron-emission tomography/computed tomography features (diaphragm disease, large bowel mesentery implant, pleural effusion, large-volume ascites, peritoneal carcinomatosis, small bowel mesentery implant, rectal implant, pelvic lymph nodes, retroperitoneal lymph nodes, inguinal lymph nodes, lymph nodes outside the abdomen) were considered dichotomous variables. Nomograms were constructed as previously described [26,27]. To develop a robust and well-calibrated nomogram predicting the risk of incomplete cytoreduction, a logistic regression model was built using a development cohort of 240 patients and validated with a cohort of 103 patients. Categorical variables were grouped before modeling. Bivariate relationships between risk factors and incomplete cytoreduction were assessed using the model development cohort. Predictive values were determined using univariate analysis (P b 0.2) and tested using bootstrap resampling, in which 1000 repetitions were included in the logistic regression model with backward elimination. Criterion for inclusion in the final logistic model was 50% relative selection frequency. To assess model fit, the concordance index was used to measure discrimination by calculating the area under the receiver operating characteristics curve. The Hosmer–Lemeshow test was used to assess calibration. The model was applied to the validation cohort for external validation. Using the same methods, discrimination and model calibration were tested. Positive likelihood ratio was calculated using the following formula: Positive likelihood ratio = sensitivity/(1 − specificity). All analyses were performed using SPSS (version 19.0; SPSS, Chicago, IL) and R version 3.0.0 (http://cran.r-project.org/mirrors.html). In this study, P b 0.05 was considered significant.
Results Patient characteristics The characteristics of the model development and validation cohorts are summarized in Table 1. High-surgical complexity score surgery was performed on 27.9% (67 of 240) and 31.1% (32 of 103) of patients in the model development and validation cohorts, respectively. The complete cytoreduction rates of the model development and validation cohorts were 33.8% (81 of 240) and 37.9% (39 of 103 patients), respectively. The sensitivity, specificity, and positive and negative predictive values were calculated for positron-emission tomography/computed tomography using operative and histopathological findings as the reference standard. The results of the analyses made on a per-patient basis in different intra-abdominal areas are shown in Supplemental Table 2. Predicting incomplete cytoreduction in the model development and validation cohorts Table 2 shows the results of the logistic regression analyses used to identify predictors of incomplete cytoreduction. After bootstrap resampling, the final model indicated surgical aggressiveness index and five positron-emission tomography/computed tomography features (diaphragm disease, ascites, peritoneal carcinomatosis, small bowel mesentery implant, tumoral uptake ratio) as statistically significant predictors. A nomogram was constructed based on this logistic regression model (Fig. 1). The point value assigned to each factor was proportional to the odds ratio derived from its own beta coefficients determined by
Table 1 Characteristics of the model development and validation cohorts. Characteristics Age, years BMI, kg/m2 Parity, n Menopause, n (%) ASA score, n (%)
FIGO stage, n (%)
Histology, n (%)
Grade, n (%)
Surgical complexity score group
Preoperative CA-125, U/ml Preoperative albumin, g/dl Preoperative platelet count, 103/mm3 SUVmax in the primary tumor Tumoral uptake ratioa Residual tumor
Median (range) Median (range) Median (range) No Yes 1 2 3 IIIA IIIB IIIC IV Serous Mucinous Endometrioid Clear cell Transitional cell Carcinosarcoma Others 1 2 3 unknown ≤3 (low) 4–7 (intermediate) ≥8 (high) Median (range) Median (range) Median (range) Median (range) Median (range) No gross residual 0–1 cm N1 cm
Model development cohort (n = 240)
Validation cohort (n = 103)
P
55 (27–80) 23.7 (15.3–36.4) 2 (0–8) 85 (35.4) 155 (64.6) 36 (15.0) 194 (80.8) 10 (4.1) 6 (2.5) 10 (4.2) 173 (72.1) 51 (21.3) 185 (77.1) 5 (2.1) 9 (3.8) 19 (7.9) 6 (2.5) 10 (4.2) 6 (2.5) 5 (2.1) 19 (7.9) 185 (77.1) 31 (12.9) 26 (10.8) 147 (61.3) 67 (27.9) 645 (4–123000) 3.6 (1.9–4.6) 318 (111–648) 9.6 (1.6–45.3) 0.72 (0–2.06) 81 (33.8) 94 (39.2) 65 (27.1)
54 (20–76) 23.3 (15.3–34.3) 2 (0–7) 37 (35.9) 66 (64.1) 14 (13.6) 85 (82.5) 4 (3.9) 2 (1.9) 4 (3.9) 67 (65.0) 30 (29.1) 89 (86.4) 0 4 (3.9) 3 (2.9) 4 (3.9) 1 (1.0) 2 (1.9) 4 (3.9) 11 (10.7) 78 (75.7) 10 (9.7) 5 (4.9) 66 (64.1) 32 (31.1) 879 (13–17145) 3.6 (2.1–4.7) 330 (139–736) 10.1 (2.6–26.3) 0.69 (0–2.07) 39 (37.9) 34 (33.0) 30 (29.1)
0.572 0.569 0.131 0.511 0.855
0.473
0.197
0.147
0.203
0.198 0.692 0.963 0.587 0.461 0.270
Abbreviations: BMI, body mass index; ASA, American Society of Anesthesiology score; FIGO, International Federation of Gynecology and Obstetrics; CA-125, cancer antigen 125; SUVmax, maximum standard uptake value. a Tumoral uptake ratio = (highest SUVmax of the lesions in the upper abdominal region)/(highest SUVmax of the lesions in the lower abdominal region).
S.-H. Shim et al. / Gynecologic Oncology 136 (2015) 30–36
33
Table 2 Univariate and multivariate logistic regression models for predicting incomplete cytoreduction. Variables
n
Univariate analysis Odds ratio (95% CI)
Age, yearsa BMI, kg/m2a Parity, na Menopause ASA
Diaphragm disease Large bowel mesentery implant Pleural effusion Ascites Peritoneal carcinomatosis Small bowel mesentery implant Rectal implants Pelvic lymph nodes N1 cm Retroperitoneal lymph nodes N1 cm Inguinal lymph nodes N1 cm Lymph nodes outside abdomen N1 cm SUVmax in the primary tumora Tumoral uptake ratioa,b Log of preoperative CA-125a Preoperative albumin, g/dla Preoperative platelet count, 103/mm3a Surgical aggressiveness index, %a
No (62) Yes (96) 1 (36) 2 (194) 3 (10) No (144) Yes (96) No (152) Yes (88) No (209) Yes (31) No (101) Yes (139) No (35) Yes (205) No (155) Yes (85) No (202) Yes (38) No (156) Yes (84) No (125) Yes (115) No (234) Yes (6) No (154) Yes (86)
1.028 (1.003–1.052) 0.947 (0.858–1.044) 1.254 (1.032–1.524) 1 1.415 (0.813–2.460) 1 1.636 (0.619–4.327) 0.136 (0.014–1.351) 1 7.310 (3.601–14.837) 1 3.735 (1.966–7.094) 1 1.285 (0.562–2.935) 1 4.337 (2.477–7.736) 1 6.652 (3.004–14.731) 1 4.329 (2.214–8.463) 1 1.302 (0.610–2.779) 1 1.570 (0.880–2.801) 1 1.953 (1.129–3.378) 1 2.597 (0.298–22.611) 1 2.886 (1.553–5.365) 1.015 (0.964–1.068) 11.961 (5.157–27.745) 1.349 (1.120–1.624) 0.263 (0.148–0.467) 1.003 (1.000–1.006) 0.947 (0.922–0.974)
Multivariate analysis P
Odds ratio (95% CI)
P
0.026 0.275 0.023 0.219 0.321
b0.001
1 3.654 (1.604–8.324)
0.002
b0.001 0.552 b0.001 b0.001 b0.001
1 3.027 (1.446–6.338) 1 3.731 (1.268–10.982) 1 2.978 (1.276–6.951)
0.012
8.231 (3.036–22.311)
b0.001
0.926 (0.893–0.960)
b0.001
0.003 0.017
0.496 0.127 0.017 0.387 0.001 0.580 b0.001 0.002 b0.001 0.052 b0.001
Abbreviations: BMI, body mass index; ASA, American Society of Anesthesiology score; CA-125, cancer antigen 125; SUVmax, maximum standard uptake value; CI, confidence interval. a Continuous variable. b Tumoral uptake ratio = (highest SUVmax of the lesions in the upper abdominal region)/(highest SUVmax of the lesions in the lower abdominal region).
Fig. 1. Nomogram that can predict incomplete cytoreduction in patients with advanced epithelial ovarian cancer. This nomogram incorporates six variables. For each level of each prognostic variable, points were allocated according to the scale shown here. The total score was determined by adding individual parameter points and used to calculate the predicted probability of incomplete cytoreduction. A total score of 128 was assigned a value of 0.8 and used to define the groups at high-risk of incomplete cytoreduction.
34
S.-H. Shim et al. / Gynecologic Oncology 136 (2015) 30–36
Fig. 2. Performance of the nomogram for predicting incomplete cytoreduction in patients with advanced epithelial ovarian cancer. (A) After 1000 repetitions, the bootstrap-corrected concordance index of the model development cohort was 0.881 (95% CI = 0.838–0.923). In the validation cohort, the bootstrap-corrected concordance index was 0.861 (95% CI = 0.790–0.932). (B) Calibration plots of the nomogram for the model development cohort. (C) Calibration plots of the nomogram for the validation cohort. The dashed line represents the performance of an ideal nomogram, in which 100% of incomplete cytoreductions match the actuarial incomplete cytoreduction rate. The solid line represents the performance of the actual nomogram. The filled dots were obtained from a subcohort in the database, and the vertical bars indicate 95% confidence interval. When plotting the incomplete cytoreduction probabilities predicted by the current nomogram against the actuarial incomplete cytoreduction rate, the calibration curve was close to the dashed line.
the regression analysis. Internal validation was performed using bootstrapping resampling. After 1000 repetitions, the bootstrapcorrected concordance index of the model was 0.881 (95% CI = 0.838–0.923). In the validation cohort, the discrimination accuracy of the model was 0.861 (95% CI = 0.790–0.932) (Fig. 2A). Fig. 2B–C shows the calibration plots of the nomogram for the model development and validation cohorts, respectively. The Hosmer–Lemeshow test yielded a P value of 0.274 for the model development cohort, indicating that the nomogram was well-fitted. For the validation cohort, the nomogram also fit the data (P = 0.214; Hosmer–Lemeshow test). We also performed same analysis for suboptimal cytoreduction (maximum disease greater than 1 cm). The surgical aggressiveness index, lymph nodes outside the abdomen, and tumoral uptake ratio were revealed as statistically significant predictors for suboptimal cytoreduction (Supplemental Table 3). A nomogram was constructed based on these predictors (Supplemental Fig. S2). The concordance indices of the model were 0.793 (95% CI = 0.727–0.859) and 0.818 (95% CI = 0.737–0.899) for the model development and validation cohorts, respectively (Supplemental Fig. S3). Identification of patients at high risk of incomplete cytoreduction We defined the high-risk group as having a predicted probability of incomplete cytoreduction of greater than 80%. To keep a balance between positive likelihood ratio and false negativity, we set the cutoff value of 80% predicted probability of incomplete cytoreduction to identify the high-risk group. The nomogram thereby classified 115 of 240 patients (48%) in the model development cohort as high-risk. In that group, the predicted probability of incomplete cytoreduction was 92.7%, and the actual incomplete cytoreduction rate was 92.2%
(106 out of 115). In the validation cohort, 48 of 103 patients (47%) were classified as high-risk. In that group, the predicted probability of incomplete cytoreduction was 93.1%, and the actual incomplete cytoreduction rate was 87.5% (42 out of 48). After combining the model development and validation cohorts, the positive likelihood ratio for the high-risk group was 5.3 (95% CI = 3.3–8.6).
Model performance across individual surgeons We tested the performance of our nomogram by applying it to subsets of individual surgeons with varying degrees of surgical aggressiveness. Supplemental Table 5 summarizes the types of procedures performed. The surgical aggressiveness of the attending surgeons was heterogeneous: surgeon C performed high-surgical complexity score surgery in N40% of cases, while surgeon F performed high-surgical complexity score surgery in b5% of cases. The complete cytoreduction rate was also heterogeneous (range = 50.6–9.1%) (Table 3). For surgeon C, the concordance index of the model was 0.809 (95% CI = 0.713–0.905). For surgeon F, who performed less extensive procedures, our model demonstrated a concordance index of 0.950 (95% CI = 0.852–1.000). The model also demonstrated good performance for other surgeons with various surgical aggressiveness (Table 3).
Discussion In the present study, we developed a non-invasive and user-friendly nomogram for predicting incomplete cytoreduction in advancedovarian cancer patients using surgical aggressiveness index and positron-emission tomography/computed tomography. The nomogram
Table 3 Surgical aggressiveness indices and model discrimination for each attending surgeon. Surgeon
A B C D E F
Total surgeries (n)
58 92 77 48 46 22
Incomplete cytoreduction, n (%)
44 (76) 61 (66) 38 (49) 32 (67) 28 (61) 20 (91)
Abbreviations: AUC, area under the curve; CI, confidence interval.
Surgical complexity score
Surgical aggressiveness index
High, n (%)
Intermediate, n (%)
Low, n (%)
15 (26) 29 (32) 32 (42) 5 (10) 17 (37) 1 (5)
36 (62) 62 (67) 45 (58) 34 (71) 27 (59) 9 (41)
7 (12) 1 (1) 0 9 (19) 2 (4) 12 (54)
Model discrimination AUC (95% CI)
25.86 31.52 41.56 10.42 36.96 4.55
0.883 (0.799–0.968) 0.859 (0.775–0.943) 0.809 (0.713–0.905) 0.973 (0.937-1.000) 0.834 (0.705–0.964) 0.950 (0.852–1.000)
S.-H. Shim et al. / Gynecologic Oncology 136 (2015) 30–36
accurately identified the high-risk group of incomplete cytoreduction and was validated using external data. Compared with previous models, a major strength of our nomogram is that individual surgeon factor was considered. Surgeons with high surgical capability and a multidisciplinary team can achieve a high rate of complete cytoreduction [5]. To estimate the surgical complexity levels at an individual institution, Jung et al. designed the surgical skill index [28]. However, surgical aggressiveness will differ even within the same institution. Thus, we modified this parameter and calculated the surgical aggressiveness index for each attending surgeon and our nomogram demonstrated similarly high performances for different surgical efficiency. To develop a reliable model, all clinical predictors should be tested for inclusion. Although many previous studies have included only a small number of patients, our present study cohorts was sufficiently large to test candidate predictors. Potential predictors of cytoreducibility might include clinical characteristics, and the most important may be tumor location and size [5]. Commonly suggested predictors include diaphragmatic disease [8,10,11], large-volume ascites [8,9], mesenteric disease [8,10,11], and diffuse peritoneal implants [8,9,13]. We tested such clinically relevant predictors to construct our model. According to National Comprehensive Cancer Network and American College of Radiology Appropriateness Criteria, positronemission tomography/computed tomography is useful for initial staging [29,30]. For the pretreatment staging and evaluation of distant ovarian carcinoma metastases, the best performances have been reported using positron-emission tomography/computed tomography [15,17, 18,20,29,30]. Thus, this procedure could identify patients who are ineligible for complete cytoreduction [19]. We here used previously reported positron-emission tomography/computed tomography features as predictors for cytoreducibility [14]. The concordance index of our nomogram was 0.881, which indicates high predictive power [5]. The model was also tested using computed tomography instead of positron-emission tomography/computed tomography. The concordance index of the computed tomography-based model was 0.861 (95% CI = 0.814–0.908) (Supplemental Table 4). However, the Hosmer–Lemeshow test indicated poor fit in the validation cohort (P = 0.033). Thus, we chose the multivariate model based on positron-emission tomography/computed tomography. The intent of this study is not to recommend routine neoadjuvant chemotherapy. Although a recent European Organization for Research and Treatment of Cancer (EORTC)/National Cancer Institute of Canada (NCIC) trial reported that neoadjuvant chemotherapy strategy is not inferior to primary debulking surgery in advanced-ovarian cancer patients [31], some authors still suggest that neoadjuvant chemotherapy requires further investigation; a major criticism is that the survival outcomes of the primary cytoreductive surgery arm were alarmingly low (30 months) [4]. In another study that used identical inclusion criteria as the EORTC–NCIC trial, the optimal cytoreduction rate was 71% and the median overall survival period was 50 months [32]. In addition, the EORTC-NCIC trial did not elucidate the advantage of neoadjuvant chemotherapy in terms of adverse effects, or quality of life [5]. Accordingly, further trials are needed to confirm the efficacy of neoadjuvant chemotherapy in patients at high-risk of incomplete cytoreduction. Our current study demonstrated a positive likelihood ratio value of 5.3 for high-risk patients, suggesting that the nomogram is useful for identifying such patients [33]. If greater surgical skill and multi-disciplinary resources are required to achieve complete cytoreduction, surgeons can consider patient referrals. Our nomogram may be useful in such decision making. We do not think, however, that all surgeries require a more experienced or aggressive surgeon in attendance to allow complete cytoreduction until enough evidence is accumulated. From our current data, there was a difference in morbidity and resource consumption based on the surgical complexity score. In addition, surgeons with a higher surgical aggressiveness index can also have higher complication rates and cost.
35
Considering that cytoreduction comes at a cost, aggressive surgery should be tailored with respect to the general condition and extent of disease in the patient [6]. Our present study has several limitations. First, positron-emission tomography/computed tomography was not performed on all cases. However, this procedure has been recommended for all suspected cases of ovarian cancer and was performed on consecutive patients. Thus, most of our advanced ovarian cancer patients (343/444; 77%) underwent preoperative PET/CT (Supplemental Fig. S1). Second, the estimated surgical aggressiveness index still requires validation. Moreover, inclusion of the index in the nomogram may limit the generalizability beyond that of the surgeons tested. The model excluding this index is outlined in Supplemental Table 6 and Supplemental Fig. S4. However, although exclusion of the index may enhance the generalizability of the model, it may also limit the accuracy of the model among surgeons with different surgical aggressiveness. Third, although we validated the model using external data, it must still be validated by other medical institutions to assess its general applicability. In this regard, we created a web-based version of the nomogram (http:// glimmer.rstudio.com/jeongyoonlee/nomogram/). This web-based tool may encourage other institutions to evaluate whether our nomogram could be effectively used in their clinics. We are also conducting a multicenter-prospective study to validate our nomogram. Despite these limitations, our nomogram represents the first attempt to incorporate both positron-emission tomography/computed tomography and surgeon factor to predict incomplete cytoreduction in advancedovarian cancer patients. Eventually, the nomogram may produce better outcomes and reduced morbidity in advanced-ovarian cancer patients. Accordingly, our new model should be further tested using multi-institutional, prospective, validation studies. Supplementary data to this article can be found online at http://dx. doi.org/10.1016/j.ygyno.2014.11.004. Conflict of interest statement The authors have no conflicts of interest or financial ties to disclose.
Acknowledgments We thank Jeong-Yoon Lee, PhD who made a computerized version of the nomogram in line with the web-site. References [1] Jemal A, Siegel R, Xu J, Ward E. Cancer statistics, 2010. CA Cancer J Clin 2010;60: 277–300. [2] Lim MC, Moon EK, Shin A, Jung KW, Won YJ, Seo SS, et al. Incidence of cervical, endometrial, and ovarian cancer in Korea, 1999–2010. J Gynecol Oncol 2013;24: 298–302. [3] Bristow RE, Tomacruz RS, Armstrong DK, Trimble EL, Montz FJ. Survival effect of maximal cytoreductive surgery for advanced ovarian carcinoma during the platinum era: a meta-analysis. J Clin Oncol 2002;20:1248–59. [4] Chi DS, Bristow RE, Armstrong DK, Karlan BY. Is the easier way ever the better way? J Clin Oncol 2011;29:4073–5. [5] Kang S, Park SY. To predict or not to predict? The dilemma of predicting the risk of suboptimal cytoreduction in ovarian cancer. Ann Oncol 2011;22(Suppl. 8):viii23–8. [6] Vergote I, Trope CG, Amant F, Ehlen T, Reed NS, Casado A. Neoadjuvant chemotherapy is the better treatment option in some patients with stage IIIc to IV ovarian cancer. J Clin Oncol 2011;29:4076–8. [7] Parker D, Bradley C, Bogle SM, Lay J, Masood M, Hancock AK, et al. Serum albumin and CA125 are powerful predictors of survival in epithelial ovarian cancer. Br J Obstet Gynaecol 1994;101:888–93. [8] Bristow RE, Duska LR, Lambrou NC, Fishman EK, O'Neill MJ, Trimble EL, et al. A model for predicting surgical outcome in patients with advanced ovarian carcinoma using computed tomography. Cancer 2000;89:1532–40. [9] Dowdy SC, Mullany SA, Brandt KR, Huppert BJ, Cliby WA. The utility of computed tomography scans in predicting suboptimal cytoreductive surgery in women with advanced ovarian carcinoma. Cancer 2004;101:346–52. [10] Ferrandina G, Sallustio G, Fagotti A, Vizzielli G, Paglia A, Cucci E, et al. Role of CT scan-based and clinical evaluation in the preoperative prediction of optimal cytoreduction in advanced ovarian cancer: a prospective trial. Br J Cancer 2009; 101:1066–73.
36
S.-H. Shim et al. / Gynecologic Oncology 136 (2015) 30–36
[11] Axtell AE, Lee MH, Bristow RE, Dowdy SC, Cliby WA, Raman S, et al. Multi-institutional reciprocal validation study of computed tomography predictors of suboptimal primary cytoreduction in patients with advanced ovarian cancer. J Clin Oncol 2007;25:384–9. [12] Bristow RE. Predicting surgical outcome for advanced ovarian cancer, surgical standards of care, and the concept of kaizen. Gynecol Oncol 2009;112:1–3. [13] Risum S, Hogdall C, Loft A, Berthelsen AK, Hogdall E, Nedergaard L, et al. Prediction of suboptimal primary cytoreduction in primary ovarian cancer with combined positron emission tomography/computed tomography–a prospective study. Gynecol Oncol 2008;108:265–70. [14] Shim SH, Kim DY, Seo MJ, Lee SW, Park JY, Lee JJ, et al. Preoperative fluorine 18 fluorodeoxyglucose tumoral uptake ratio between upper and lower abdomen in primary advanced-stage ovarian cancer. Int J Gynecol Cancer 2013;23:1383–92. [15] Kitajima K, Murakami K, Yamasaki E, Kaji Y, Fukasawa I, Inaba N, et al. Diagnostic accuracy of integrated FDG-PET/contrast-enhanced CT in staging ovarian cancer: comparison with enhanced CT. Eur J Nucl Med Mol Imaging 2008;35:1912–20. [16] Fruscio R, Sina F, Dolci C, Signorelli M, Crivellaro C, Dell'Anna T, et al. Preoperative 18 F-FDG PET/CT in the management of advanced epithelial ovarian cancer. Gynecol Oncol 2013;131:689–93. [17] Nam EJ, Yun MJ, Oh YT, Kim JW, Kim JH, Kim S, et al. Diagnosis and staging of primary ovarian cancer: correlation between PET/CT, Doppler US, and CT or MRI. Gynecol Oncol 2010;116:389–94. [18] Yuan Y, Gu ZX, Tao XF, Liu SY. Computer tomography, magnetic resonance imaging, and positron emission tomography or positron emission tomography/computer tomography for detection of metastatic lymph nodes in patients with ovarian cancer: a meta-analysis. Eur J Radiol 2012;81:1002–6. [19] Prakash P, Cronin CG, Blake MA. Role of PET/CT in ovarian cancer. AJR Am J Roentgenol 2010;194:W464–70. [20] Castellucci P, Perrone AM, Picchio M, Ghi T, Farsad M, Nanni C, et al. Diagnostic accuracy of 18 F-FDG PET/CT in characterizing ovarian lesions and staging ovarian cancer: correlation with transvaginal ultrasonography, computed tomography, and histology. Nucl Med Commun 2007;28:589–95. [21] Timmerman D, Ameye L, Fischerova D, Epstein E, Melis GB, Guerriero S, et al. Simple ultrasound rules to distinguish between benign and malignant adnexal masses before surgery: prospective validation by IOTA group. BMJ 2010;341:c6839. [22] Iyer VR, Lee SI. MRI, CT, and PET/CT for ovarian cancer detection and adnexal lesion characterization. AJR Am J Roentgenol 2010;194:311–21.
[23] Park JY, Kim DY, Suh DS, Kim JH, Kim YM, Kim YT, et al. Comparison of laparoscopy and laparotomy in surgical staging of early-stage ovarian and fallopian tubal cancer. Ann Surg Oncol 2008;15:2012–9. [24] Aletti GD, Santillan A, Eisenhauer EL, Hu J, Aletti G, Podratz KC, et al. A new frontier for quality of care in gynecologic oncology surgery: multi-institutional assessment of short-term outcomes for ovarian cancer using a risk-adjusted model. Gynecol Oncol 2007;107:99–106. [25] Chi DS, Eisenhauer EL, Lang J, Huh J, Haddad L, Abu-Rustum NR, et al. What is the optimal goal of primary cytoreductive surgery for bulky stage IIIC epithelial ovarian carcinoma (EOC)? Gynecol Oncol 2006;103:559–64. [26] Iasonos A, Schrag D, Raj GV, Panageas KS. How to build and interpret a nomogram for cancer prognosis. J Clin Oncol 2008;26:1364–70. [27] Kim DY, Shim SH, Kim SO, Lee SW, Park JY, Suh DS, et al. Preoperative nomogram for the identification of lymph node metastasis in early cervical cancer. Br J Cancer 2014;110:34–41. [28] Jung DC, Kang S, Kim SC, Kim JW, Nam JH, Ryu SY, et al. Use of complex surgical procedures, patterns of tumor spread, and CA-125 predicts a risk of incomplete cytoreduction: a Korean Gynecologic Oncology Group study (KGOG-3022). Gynecol Oncol 2013;131:336–40. [29] Mitchell DG, Javitt MC, Glanc P, Bennett GL, Brown DL, Dubinsky T, et al. ACR appropriateness criteria staging and follow-up of ovarian cancer. J Am Coll Radiol 2013;10: 822–7. [30] Morgan Jr RJ, Alvarez RD, Armstrong DK, Burger RA, Chen LM, Copeland L, et al. Ovarian cancer, version 2.2013. J Natl Compr Cancer Netw 2013;11:1199–209. [31] Vergote I, Trope CG, Amant F, Kristensen GB, Ehlen T, Johnson N, et al. Neoadjuvant chemotherapy or primary surgery in stage IIIC or IV ovarian cancer. N Engl J Med 2010;363:943–53. [32] Chi DS, Musa F, Dao F, Zivanovic O, Sonoda Y, Leitao MM, et al. An analysis of patients with bulky advanced stage ovarian, tubal, and peritoneal carcinoma treated with primary debulking surgery (PDS) during an identical time period as the randomized EORTC-NCIC trial of PDS vs neoadjuvant chemotherapy (NACT). Gynecol Oncol 2012;124:10–4. [33] Jaeschke R, Guyatt GH, Sackett DL. Users' guides to the medical literature. III. How to use an article about a diagnostic test. B. What are the results and will they help me in caring for my patients? The Evidence-Based Medicine Working Group. JAMA 1994; 271:703–7.
