Original Paper
Urologia
Received: February 14, 2014 Accepted after revision: June 16, 2014 Published online: August 27, 2014
Urol Int 2015;94:262–269 DOI: 10.1159/000365292
Internationalis
The Clinical Use of Statistical Permutation Test Methodology: A Tool for Identifying Predictive Variables of Outcome M. Racioppi a L. Salmaso b C. Brombin b R. Arboretti b, p D. D'Agostino a R. Colombo c, d V. Serretta c, e M. Brausi c, f G. Casetta c, g P. Gontero c, h R. Hurle c, i R. Tenaglia c, j V. Altieri c, k R. Bartoletti c, l M. Maffezzini c, m S. Siracusano c, n G. Morgia c, o P.F. Bassi a, c
a
Department of Urology, Catholic University of the Sacred Heart, Rome, b Department of Management and Engineering, University of Padua, Padua, c G.I.S.Ca.V. Group, Rome, d Department of Urology and Urological Research Institute, University Vita-Salute, Milan, e Department of Urology, Paolo Giaccone General Hospital, Palermo, f Department of Urology, B. Ramazzini Hospital, Carpi-Modena, g Department of Urology 1, S. Giovanni Battista Hospital, and h Department of Urology, University of Turin, Turin, i Urology Unit, Humanitas-Gavazzeni Hospital, Bergamo, j Department of Urology, School of Urology, University of Chieti, Chieti, k Institute of Urology, University of Naples, Naples, l Department of Urology, University of Florence, Florence, m Department of Specialized Surgery, Urology, E.O. Ospedali Galliera, Genova, n Department of Urology, University of Trieste, Trieste, o Urological Department, University of Catania, Catania, and p Department of Land Use and Territorial Systems, University of Padova, Padova, Italy
Key Words Bladder cancer · Prognosis · Permutation test
Abstract Objectives: To identify the predictive variables affecting the outcome after radical surgery for bladder cancer by a newer statistical methodology, i.e. nonparametric combination (NPC). Methods: A multicenter study enrolled 1,312 patients who had undergone radical cystectomy for bladder cancer in 11 Italian oncological centers from January 1982 to December 2002. A statistical analysis of their medical history and diagnostic, pathological and postoperative variables was performed using a NPC test. The patients were included in a comprehensive database with medical history and clinical and pathological data. Five-year survival was used as the dependent variable, and p values were corrected for multiplicity using a closed testing
© 2014 S. Karger AG, Basel 0042–1138/14/0943–0262$39.50/0 E-Mail [email protected] www.karger.com/uin
procedure. The newer nonparametric approach was used to evaluate the prognostic importance of the variables. All of the analyses were performed using routines developed in MATLAB© and the significance level was set at α = 0.05. Results: A significant prognostic predictive value (p < 0.01) for tumor clinical staging, hydronephrosis, tumor pathological staging, grading, presence of concomitant carcinoma in situ, regional lymph node involvement, corpora cavernosa invasion, microvascular invasion, lymphatic invasion and prostatic stroma involvement was found. Conclusions: The NPC test could handle any type of variable (categorical and quantitative) and take into account the multivariate relation among variables. This newer methodology offers a significant contribution in biomedical studies with several endpoints and is recommended in presence of non-normal data and missing values, as well as solving high-dimensional data and problems relating to small sample sizes. © 2014 S. Karger AG, Basel
Dr. Daniele D’Agostino Department of Urology, Policlinico ‘A. Gemelli’ Catholic University of the Sacred Heart Largo Francesco Vito 1, IT–00168 Rome (Italy) E-Mail danieledagostino @ yahoo.it
Downloaded by: UCSF Library & CKM 169.230.243.252 - 4/16/2015 7:34:36 PM
Transitional cell carcinoma of the bladder is the second most common malignancy of the genitourinary tract [1]. Bladder cancer (BC) exhibits a wide spectrum of tumors with different clinical behaviors. Consequently, the treatment options differ according to the risk of disease recurrence, progression to more advanced stages, and death from the disease. Bladder tumors can be categorized into two forms: non-muscle-invasive and muscle-invasive cancer. Radical cystectomy (RC) with bilateral pelvic lymphadenectomy and urinary diversion is the gold standard treatment for muscle-invasive BC or refractory high-grade nonmuscle-invasive cancer. The overall 5-year survival rate after RC is nearly 50% [2]. Unfortunately, there is controversy surrounding the data about the independent prognostic variables of outcome of muscle-invasive or refractory non-muscle-invasive BC [3–6]. Optimal statistical analysis must identify the predictive variables of outcome, but unfortunately current models such as logistic regression (LoR) are limited because of their inadequate capacity to handle the variability and complexity of data [7]. Recently, newer statistical techniques such as nomograms and artificial intelligence (AI) have been proposed for BC, but with controversial results. This paper introduces nonparametric combination (NPC) of dependent permutation test methodology [8] to better identify prognostic factors in patients with a diagnosis of BC. With the NPC approach, we can carry out a more flexible analysis both in terms of specification of multivariate hypotheses and of the nature of the variables involved in the analysis. This methodology is able to handle any type of variable (categorical and quantitative) and to consider the multivariate relation among variables. The aim of the study was to test the capacity of the NPC statistical methodology to best identify the predictive variables affecting the outcome after radical surgery for BC.
Patients and Methods Patient Characteristics We performed a retrospective study on 1,312 patients, 1,149 (87.6%) male and 163 (12.4%) female, included in a comprehensive database with medical history and clinical and pathological data, who underwent RC, bilateral iliac and obturator node dissection, and urinary diversion for urothelial BC from January 1982 to December 2002 in 11 Italian urological centers of excellence in BC management. BC was diagnosed or confirmed by transurethral resection. Physical examination, chest X-ray, computerized tomography and/or magnetic resonance imaging were used for clinical staging. The tumors were staged according to the 2002 TNM classification and graded according to the 1997 WHO system [9, 10].
Statistical Permutation Test Methodology
Table 1. Variables evaluated and relative coding
Variable
Coding
Medical history Previous non-muscle-invasive TCC Focality Tumor stage Grading Concomitant CIS
1 2 3 4 5
Preoperative status Focality Tumor stage Concomitant CIS Grading Regional lymph nodes involvement Hydronephrosis
6 7 8 9 10 11
Postoperative status Tumor stage Concomitant CIS Grading Regional lymph node involvement Metastasis Concomitant squamous, adenocarcinoma or other differentiation Trigone invasion Corpora cavernosa invasion Urethral involvement Microvascular invasion Lymphatic invasion Prostatic stroma involvement Concomitant adenocarcinoma of the prostate Upper urinary tract TCC
12 13 14 15 16 17 18 19 20 21 22 23 24 25
TCC = Transitional cell carcinoma.
The indications for RC included: muscle-invasive BC, non-muscle-invasive BC refractory to intravesical therapy and multifocal stage T1G3 tumor associated with diffuse carcinoma in situ (CIS). No patient had clinically distant metastases at the time of operation, and the patients were periodically followed at the referral institution. Patients lost to follow-up or who died of causes other than BC were excluded. The total sample size (1,003 individuals) was divided into two groups: the first group (n = 469) included patients dead from disease and alive with relapse of disease, and the second group (n = 534) consisted of disease-free patients. The dependent variable was overall 5-year survival after RC. We took into consideration the many variables involved in the history of a patient’s BC and transformed them into categorical (ordinal) variables, and thus had 25 endpoints (table 1). NPC Methodology The NPC of dependent permutation tests is a conditional testing procedure, introduced in 2010 [11], which is appropriate and sometimes unavoidable when (1) sample sizes are smaller than the number of response variables, (2) in multivariate problems, some
Urol Int 2015;94:262–269 DOI: 10.1159/000365292
263
Downloaded by: UCSF Library & CKM 169.230.243.252 - 4/16/2015 7:34:36 PM
Introduction
Table 2. Clinical characteristics of the 1,003 evaluable patients
Variable
Result
Medical history Previous non-muscle-invasive TCC Focality Tumor stage Grading Concomitant CIS Preoperative status Focality Tumor stage Concomitant CIS Grading Regional lymph node involvement Hydronephrosis Postoperative status Tumor Concomitant CIS Grading Regional lymph nodes Metastases Concomitant squamous, adenocarcinoma or other differentiation Trigone invasion Corpora cavernosa invasion Urethral involvement Microvascular invasion Lymphatic invasion Prostatic stroma invasion Concomitant adenocarcinoma of prostate Upper urinary tract TCC Neoadjuvant chemotherapy Adjuvant chemotherapy
yes 402 single 295 Tx T0 56 5 Gx 31 yes 74
no 560 multiple 160 Ta T1 T2 T2a T2b T3 49 189 72 3 13 7 G1 18 G2 121 no 455
unk. 41 unk. 548 T3a T3b T4 T4a T4b unk. 0 3 2 2 2 600 G3 232 unk. 601 unk. 474
yes 629 Tx T0 23 9 yes 95 Gx 56 Nx 105 no 751
no 219 Ta T1 T2 T2a T2b T3 19 145 458 11 17 85 no 842 G1 6 G2 134 N0 862 N1 7 monolateral 176
unk. 155 T3b T4 T4a T4b unk. 47 4 18 4 92 unk. 66 G3 725 unk. 82 N2 3 unk. 26 bilateral 40 unk. 36
Tx T0 13 80 yes 189 Gx 32 Nx 118 M0 934 TCC 760
Ta T1 T2 T2a T2b T3 T3a T3b T4 T4a T4b unk. 16 134 138 18 22 30 138 207 11 109 11 76 no 738 unk. 76 G1 10 G2 119 G3 701 unk. 141 N0 710 N1 111 N2 49 unk. 15 M1 10 unk. 59 mixed 87 squamous ca. 24 adenocarcinoma 9 other 32 unk. 91
no 482 no 340 no 807 no 632 no 593 no 646 no 711 no 874 no 878 no 830
T3a 71
yes 411 yes 481 yes 67 yes 179 yes 249 yes 129 yes 149 yes 40 yes 25 yes 156
unk. 110 unk. 182 unk. 129 unk. 192 unk. 161 unk. 228 unk. 143 unk. 89 unk. 100 unk. 17
variables are categorical and others quantitative, (3) data sets contain missing values that cannot be ignored, and (4) data sets are obtained by selection-bias procedures. NPC tests are relatively efficient and much less demanding in terms of underlying assumptions than parametric competitors and standard distribution-free methods based on ranks, which are generally not conditional on sufficient statistics and almost never present better unconditional power behavior [11]. One major feature of the NPC of dependent tests is the fact that the researcher is not explicitly required to specify the dependence structure of response variables/endpoints. This aspect is of great importance especially for non-normal or binary variables/endpoints in which dependence relations are generally too difficult to define, and even when well defined are hard to cope with. In this sense, NPC methodology can provide an efficient and robust tool for use in the
264
Urol Int 2015;94:262–269 DOI: 10.1159/000365292
analysis of both experimental and observational studies. Due to the presence of categorical data and missing values, and given the fact that the distributional model is not well-specified (e.g. it is difficult to assume that observations are normally distributed), the complex data set at hand may be properly analyzed using NPC methodology. Once we obtained two independent samples (on the basis of the overall survival), we expanded the dataset of categorical variables in dummy variables and then performed a one-way multivariate ANOVA for comparing the means of the variables in the two samples (i.e. t test). Moreover, we adjusted p values for multiplicity using a closed testing procedure, controlling the FWE rate. All of the analyses were performed using routines developed in MATLAB© and the significance level was set at α = 0.05. Moreover, a multivariate analysis of the same variables was carried out using standard LoR to check for any possible differences.
Racioppi et al.
Downloaded by: UCSF Library & CKM 169.230.243.252 - 4/16/2015 7:34:36 PM
unk. = Unknown; TCC = transitional cell carcinoma.
Table 3. Results obtained using the NPC methodology
Variable
No.
p
p-FWE
p
p-fwe
Medical history Previous non-muscle-invasive TCC Focality Tumor stage Grading Concomitant CIS
1 2 3 4 5
n.s. n.s. n.s. n.s. n.s.
n.s. n.s. n.s. n.s. n.s.
n.s.
n.s.
Preoperative status Focality Tumor stage Concomitant CIS Grading Regional lymph node involvement Hydronephrosis
6 7 8 9 10 11
n.s. 0.0010 n.s. n.s. 0.0009 0.0002
n.s. 0.0141 n.s. n.s. 0.0126 0.0029
0.0010
0.0027
12 13 14 15 17
0.0003 0.0034 0.0003 0.0002 n.s.
0.0042 0.0391 0.0042 0.0029 n.s.
0.0004
0.0036
18 19 20 21 22 23 24 25
n.s. 0.0005 n.s. 0.0001 0.0001 0.0014 n.s. n.s.
n.s. 0.0069 n.s. 0.0011 0.0011 0.0174 n.s. n.s. 0.0011
0.0005
0.0016
n.s.
n.s.
Postoperative status Tumor stage Concomitant CIS Grading Regional lymph node involvement Concomitant squamous, adenocarcinoma or other differentiation Trigone infiltration Corpora cavernosa invasion Urethral involvement Microvascular invasion Lymphatic invasion Prostatic stroma involvement Concomitant adenocarcinoma of the prostate Upper urinary tract TCC
Results
The mean age of the patients was 64.3 years (range: 30–91). Mean and median follow-up were 60.1 and 39 months (SD ±58.3, range: 1–276), respectively. Table 2 shows the clinical characteristics of the included patients. With reference to clinical analysis, we did not find any differences between the two groups, dead and alive patients, in the medical history findings; however, in the preoperative status we found differences (p < 0.01) in terms of tumor clinical staging (variable No. 7), clinical regional lymph nodes (variable No. 10) and hydronephrosis (variable No. 11). Concerning the third group, postoperative status variables, we found significant difStatistical Permutation Test Methodology
ferences in terms of tumor pathological staging (variable No. 12), presence of concomitant CIS (variable No. 13), grading (variable No. 14), pathological regional lymph node involvement (variable No. 15), metastases (variable No. 16), corpora cavernosa invasion (variable No. 19), microscopic vascular invasion (variable No. 21), lymphatic invasion (variable No. 22) and prostatic stroma involvement (variable No. 23). All of these variables maintained their significance after adjustment for multiplicity: a closed testing procedure controlling the FWE (i.e. the probability of at least one type I error in the m tests) was applied (table 3). All of the analyses were performed using routines developed in MATLAB©, and the significance level was set at α = 0.05, B = 10,000 permutaUrol Int 2015;94:262–269 DOI: 10.1159/000365292
265
Downloaded by: UCSF Library & CKM 169.230.243.252 - 4/16/2015 7:34:36 PM
We show both raw and adjusted p values. Tippett combining function was used. 10,000 permutations were carried out. Combination of variables belonging to the same domain is also shown along with global p values. TCC = Transitional cell carcinoma.
Table 4. Results obtained using the LoR model
Variable
No.
Estimate
SE
z
p
OR
Medical history Previous non-muscle-invasive TCC Focality Tumor stage Grading Concomitant CIS
1 2 3 4 5
0.6319 0.8961 –0.1101 –0.1760 –0.5259
0.9883 0.4162 0.1658 0.2256 0.6056
0.6390 2.1530 –0.6640 –0.7800 –0.8680
0.5226 0.0313 0.5067 0.4353 0.3852
1.8812 2.4501 0.8958 0.8386 0.5911
Preoperative status Focality Tumor stage Concomitant CIS Grading Regional lymph node involvement Hydronephrosis
6 7 8 9 10 11
–0.1057 0.0685 0.1592 –0.1160 0.0969 0.4089
0.3543 0.0481 0.4814 0.2107 0.6530 0.1974
–0.2980 1.4250 0.3310 –0.5510 0.1480 2.0720
0.7654 0.1541 0.7410 0.5819 0.8820 0.0383
0.8997 1.0709 1.1725 0.8905 1.1018 1.5051
12 13 14 15 17
0.1569 0.2783 0.1575 0.4965 0.1074
0.0433 0.2745 0.2366 0.2217 0.1283
3.6210 1.0140 0.6660 2.2400 0.8370
0.0003 0.3107 0.5055 0.0251 0.4025
1.1698 1.3208 1.1706 1.6430 1.1134
18 19 20 21 22 23 24
–0.0375 –0.1973 –0.6318 0.2728 0.2296 0.4198 0.0568
0.2236 0.2709 0.4788 0.3759 0.2321 0.3361 0.2535
–0.1680 –0.7280 –1.3190 0.7260 0.9890 1.2490 0.2240
0.8668 0.4664 0.1870 0.4681 0.3226 0.2116 0.8226
0.9632 0.8210 0.5316 1.3136 1.2581 1.5217 1.0585
25
0.1296
0.5126
0.2530
0.8003
1.1384
Postoperative status Tumor stage Concomitant CIS Grading Regional lymph node involvement Concomitant squamous, adenocarcinoma or other differentiation Trigone infiltration Corpora cavernosa invasion Urethral involvement Microvascular invasion Lymphatic invasion Prostatic stroma involvement Concomitant adenocarcinoma of the prostate Upper urinary tract TCC
tions (Monte Carlo sampling) was used for estimating the permutation distribution. Results obtained using the LoR model are summarized in table 4. A statistical difference was observed in focality (variable No. 2), hydronephrosis (variable No. 11), tumor pathological staging (variable No. 12) and pathological regional lymph node involvement (variable No. 15).
Discussion
The newer nonparametric approach is a statistical methodology which we applied (to the best of our knowledge) for the first time in oncology to a large population of patients treated for BC. NPC highlighted the prognostic importance of factors not otherwise usually considered in BC. 266
Urol Int 2015;94:262–269 DOI: 10.1159/000365292
The appropriate and tailored assignment of risk is a pivotal principle in the management of any cancer. Many efforts have been addressed to develop statistical techniques that are able to identify prognostic factors of outcome after RC for BC. The TNM staging system originated as a response to the need for an accurate and consistent universal cancer outcome prediction system [12]. Universally, pathological stage together with grade are the most reliable predictors of bladder tumor behavior, and TNM prediction is accurate in 70% of tumors in the best series [13]. Furthermore, since the TNM staging system was introduced in the 1950s, new prognostic factors have been identified [14]. For example, the presence of CIS and the state of the bladder 3 months after surgery are important prognostic factors in non-muscle-invasive carcinoma [13]; lymphovascular invasion represents a highly significant and independent preRacioppi et al.
Downloaded by: UCSF Library & CKM 169.230.243.252 - 4/16/2015 7:34:36 PM
TCC = Transitional cell carcinoma. Bold values denote statistical significance.
The possible weakness is represented by the normality assumption which might often not be valid in clinical investigations based on observational studies with no randomization. Both methods of AI predicted relapse with an accuracy ranging from 88 to 95%, which is superior to standard statistical methods [13]. ANNs are complex mathematical models designed to replicate human decision-making by emulating processing pathways used by the human brain [7]. Unlike LoR models, ANNs can accommodate large numbers of prognostic variables and can analyze their varying degrees of impact on the outcome [18]. Bassi et al. [16] showed that ANNs have a prognostic performance comparable with that of LoR. In particular they demonstrated that ANNs could outperform LoR when applied to small or moderate datasets. Qureshi et al. [18] reported that ANNs were significantly more accurate than clinicians in predicting stage progression in a retrospective series of patients with T1G3 BC (overall accuracy: 82 vs. 40%). Catto et al. [13] confirmed that ANNs provide better accuracy (90 vs. 77%) than traditional statistical methods for predicting relapse in patients with non-muscle- and muscle-invasive BC, based on clinicopathological data. Comparing the TNM staging system’s predictive accuracy with that of ANNs, Burke et al. [14] showed that ANNs are significantly more accurate than the TNM staging system. Moreover, new prognostic factors can be added to ANNs to increase their prognostic accuracy further. On the other hand, the network is hidden within a functional ‘black box’: it is difficult to gain insight into the solution used to resolve the clinical data, making subsequent analysis (to ensure clinical sense prevails) and interrogation of new variables almost impossible. So, statisticians are reluctant to believe in the validity of ANNs [19]. NFM has a similar or superior predictive accuracy to ANNs, but without the ‘black box’, enabling validation from clinical knowledge and the manipulation of input variables to allow exploratory predictions [13]. Comparing both methods, Catto et al. [13] showed that NFM appeared significantly better than ANNs at predicting the timing of relapse, and significantly more accurate than LoR and linear regression. However, there seems to be no agreement on the specification of the prognostic factors determining the outcome after RC for BC. Alternative statistical procedures are represented by permutation methods. These tests are distribution-free and allow for efficient solutions even when the number of cases is less than the number of variables. We propose a nonparametric permutation approach in order to better and more easily identify the
Statistical Permutation Test Methodology
Urol Int 2015;94:262–269 DOI: 10.1159/000365292
267
Downloaded by: UCSF Library & CKM 169.230.243.252 - 4/16/2015 7:34:36 PM
dictor of cancer-specific survival in lymph node negative cystectomy patients after adjusting for pathological stage [15, 16]. Thus, clinicians are routinely forced to use their judgment to incorporate factors not included in the staging system. Traditionally, statistical techniques such as Cox’s proportional hazards and LoR were employed and, until the advent of AI, LoR was considered the best method for predicting tumor behavior [13]. When LoR is used, i.e. Cox multivariate regression analysis, Bassi et al. [16] found that in a retrospective study of patients who underwent RC for BC, only pathological stage and nodal involvement were predictive of survival. Solsona et al. [17], however, showed that together with the latter, prostatic stroma status is also an independent variable. In a similar study, Frazier et al. [15] found that pT stage, nodal involvement, positive surgical margins, the patient’s age at surgery and loss of histologic differentiation were predictive of poor cancer-specific survival in a multivariate analysis. However, the predictive accuracy of this method is poor, in the range of 69–77% [13, 14]. The traditional statistical methods in evaluating prognostic factors are limited because of their need for linear relationships between variables, large datasets and their poor performance when large amounts of ‘noise’ (inherent variation) contaminate a dataset [13]. In particular, in situations where newer explored markers interact in a complex fashion, the processing rules or mathematical equations are often very difficult to determine [18]. Moreover, they may be applicable to a population, but are not predictive for an individual [13]. In the last decades more advanced clinical studies have been focused on AI to accurately predict cancer behavior [13]. By using AI methods, such as artificial neural networks (ANNs) and neurofuzzy modeling (NFM), complex nonlinear relationships between dependent and independent variables in a population of which distribution may not be normal can be identified [13]. ANNs are computational models inspired by animals’ central nervous systems (in particular the brain) that are capable of machine learning and pattern recognition. They are usually presented as systems of interconnected ‘neurons’ that can compute values from input by feeding information through the network. NFM refers to combinations of ANNs and fuzzy logic. Neurofuzzy hybridization results in a hybrid intelligent system that synergizes these two techniques by combining the human-like reasoning style of fuzzy systems with the learning and connectionist structure of neural networks. The main strength of neurofuzzy systems is that they are universal approximators with the ability to solicit interpretable ‘IF-THEN’ rules.
268
Urol Int 2015;94:262–269 DOI: 10.1159/000365292
when considering the future of a patient who is to undergo radical surgery for BC. This method is surely transferable to all other human malignancies, as it allows the evaluation of new treatment strategies and may be useful in the design of clinical trials. We recommend the use of this method whenever the normality assumption is hard to justify, in the presence of missing values and when the number of variables is higher than the number of patients, this last aspect being very frequent in biomedical studies. Our study has some limitations. First of all, the study population came from 11 different centers and obviously the clinical evaluations and operations were performed by different physicians. However, the large number of patients limits the importance of this heterogeneity. Second, the study is retrospective. On the other hand, these aspects, which one could consider negative, can assume a positive value because the aim of the NPC methodology is to be rigorous but flexible to go beyond the problems linked to incomplete datasets, which is different from the traditional statistical methods.
Conclusions
NPC of dependent permutation test methodology is a newer statistical approach which can offer a significant contribution to successful research in biomedical studies with several endpoints. The advantages of this approach are its flexibility in handling any type of variable while at the same time taking dependencies among those variables into account without the need for modeling them [20]. In our study, NPC methodology identified new predictive variables affecting the outcome of patients who underwent radical surgery for BC. Along with lymph node involvement, pathological stage and grade at diagnosis, which are the most reliable predictors of bladder tumor behavior, we also found other potential prognostic factors. We wish to highlight that NPC methodologies can provide a more accurate prediction of clinical outcome.
References
1 Jemal A, Siegel R, Ward E, et al: Cancer statistics, 2008. CA Cancer J Clin 2008;58:71–96. 2 Herr HW, Faulkner JR, Grossman HB, et al: Surgical factors influence bladder cancer outcomes: a cooperative group report. J Clin Oncol 2004;22:2781–2789. 3 Bassi P, Ferrante GD, Piazza N, et al: Prognostic factors of outcome after radical cystectomy for bladder cancer: a retrospective study of a homogeneous patients cohort. J Urol 1999; 161:1494–1497.
Racioppi et al.
Downloaded by: UCSF Library & CKM 169.230.243.252 - 4/16/2015 7:34:36 PM
prognostic factors allowing to discriminate the outcome of patients with a BC diagnosis. Using this methodology, we are able to obtain more information that is qualitatively and quantitatively different from that provided by standard statistical methods. First of all, if we perform a standard LoR in our population, we have to invoke the principle of deletion or imputation in order to deal with missing data. However, following the previous nonparametric solution, we do not need to delete/impute. Moreover, NPC methodology does not have the drawback of using black boxes. By using NPC methodology and adjusting the p value, we have found 12 prognostic factors for survival. The test confirmed significant prognostic factors: tumor pathological staging (variable No. 12), grading (variable No. 14), pathological regional lymph nodes involvement (variable No. 16) and prostatic stroma involvement (variable No. 23). In this large series, however, by using NPC, a strong negative significance also arose for tumor clinical staging (variable No. 7), clinical regional lymph node involvement (variable No. 10), hydronephrosis (variable No. 11), presence of concomitant CIS (variable No. 13), corpora cavernosa invasion (variable No. 19), microscopic vascular invasion (variable No. 21) and lymphatic invasion (variable No. 22), all of which are factors not otherwise fully taken into consideration. Hence, we could assert that NPC methodology is an instrument that provides an accurate prediction of clinical outcome, capable of better evaluating datasets and giving significance to adjunctive prognostic factors, i.e. the outcome evaluation improves as the number of prognostic factors increases. Through this application to a real case study, we have shown how NPC methodology can offer a significant contribution to successful research in biomedical studies with several endpoints. Advantages of this approach are related to its flexibility in both handling any type of variables (categorical and quantitative with or without missing values) and simultaneously of taking into account the multivariate relation among those variables. Moreover, the underlying dependence relation structure is implicitly captured in a nonparametric way by the combination procedure. Hence, the researcher is not explicitly required to specify the dependence structure of response variables. This feature is of great importance especially for non-normal or categorical variables, in which dependence relations are too difficult to define and, if defined, are hard to cope with. In this study we have demonstrated that additional factors to those already known should be taken in account
Statistical Permutation Test Methodology
10 Epstein JI, Amin MB, Reuter VR, et al: The World Health Organization/International Society of Urological Pathology consensus classification of urothelial (transitional cell) neoplasms of the urinary bladder. Bladder Consensus Conference Committee. Am J Surg Pathol 1998;22:1435–1448. 11 Pesarin F, Salmaso L: Permutation Tests for Complex Data. Hoboken, John Wiley & Sons, 2010. 12 Beahrs OH: How can general surgery survive? Am Surg 1992;58:17–21. 13 Catto JW, Linkens DA, Abbod MF, et al: Artificial intelligence in predicting bladder cancer outcome: a comparison of neuro-fuzzy modeling and artificial neural networks. Clin Cancer Res 2003;9:4172–4177. 14 Burke HB, Goodman PH, Rosen DB, et al: Artificial neural networks improve the accuracy of cancer survival prediction. Cancer 1997;79: 857–862. 15 Frazier HA, Robertson JE, Dodge RK, Paulson DF: The value of pathologic factors in predicting cancer-specific survival among patients treated with radical cystectomy for transitional cell carcinoma of the bladder and prostate. Cancer 1993;71:3993–4001.
Urol Int 2015;94:262–269 DOI: 10.1159/000365292
16 Bassi P, Sacco E, De Marco V, et al: Prognostic accuracy of an artificial neural network in patients undergoing radical cystectomy for bladder cancer: a comparison with logistic regression analysis. BJU Int 2007; 99: 1007– 1012. 17 Solsona E, Iborra I, Dumont R, et al: Risk groups in patients with bladder cancer treated with radical cystectomy: statistical and clinical model improving homogeneity. J Urol 2005;174:1226–1230. 18 Qureshi KN, Naguib RN, Hamdy FC, et al: Neural network analysis of clinicopathological and molecular markers in bladder cancer. J Urol 2000;163:630–633. 19 Schwarzer G, Vach W, Schumacher M: On the misuses of artificial neural networks for prognostic and diagnostic classification in oncology. Stat Med 2000;19:541–561. 20 Pesarin F: Extending permutation conditional inference to unconditional one. Stat Methods Appt 2002;11:161–173.
269
Downloaded by: UCSF Library & CKM 169.230.243.252 - 4/16/2015 7:34:36 PM
4 Dalbagni G, Genega E, Hashibe M, et al: Cystectomy for bladder cancer: a contemporary series. J Urol 2001;165:1111–1116. 5 Madersbacher S, Hochreiter W, Burkhard F, et al: Radical cystectomy for bladder cancer today – a homogeneous series without neoadjuvant therapy. J Clin Oncol 2003;21:690–696. 6 Monzó Gardiner JI, Herranz Amo F, Díez Cordero JM, et al: Factores pronósticos en la supervivencia de los pacientes con carcinoma transicional de vejiga tratados con cistectomía radical. Actas Urol Esp 2009;33:249–257. 7 Shabsigh A, Bochner BH: Use of nomograms as predictive tools in bladder cancer. World J Urol 2006;24:489–498. 8 Finos L, Salmaso L: Weighted methods controlling the multiplicity when the number of variables is much higher than the number of observations. J Nonparametr Stat 2006; 18: 245–261. 9 Sobin LH, Greene FL: TNM classification: clarification of number of regional lymph nodes for pNo. Cancer 2001;92:452.
Urologia
Received: February 14, 2014 Accepted after revision: June 16, 2014 Published online: August 27, 2014
Urol Int 2015;94:262–269 DOI: 10.1159/000365292
Internationalis
The Clinical Use of Statistical Permutation Test Methodology: A Tool for Identifying Predictive Variables of Outcome M. Racioppi a L. Salmaso b C. Brombin b R. Arboretti b, p D. D'Agostino a R. Colombo c, d V. Serretta c, e M. Brausi c, f G. Casetta c, g P. Gontero c, h R. Hurle c, i R. Tenaglia c, j V. Altieri c, k R. Bartoletti c, l M. Maffezzini c, m S. Siracusano c, n G. Morgia c, o P.F. Bassi a, c
a
Department of Urology, Catholic University of the Sacred Heart, Rome, b Department of Management and Engineering, University of Padua, Padua, c G.I.S.Ca.V. Group, Rome, d Department of Urology and Urological Research Institute, University Vita-Salute, Milan, e Department of Urology, Paolo Giaccone General Hospital, Palermo, f Department of Urology, B. Ramazzini Hospital, Carpi-Modena, g Department of Urology 1, S. Giovanni Battista Hospital, and h Department of Urology, University of Turin, Turin, i Urology Unit, Humanitas-Gavazzeni Hospital, Bergamo, j Department of Urology, School of Urology, University of Chieti, Chieti, k Institute of Urology, University of Naples, Naples, l Department of Urology, University of Florence, Florence, m Department of Specialized Surgery, Urology, E.O. Ospedali Galliera, Genova, n Department of Urology, University of Trieste, Trieste, o Urological Department, University of Catania, Catania, and p Department of Land Use and Territorial Systems, University of Padova, Padova, Italy
Key Words Bladder cancer · Prognosis · Permutation test
Abstract Objectives: To identify the predictive variables affecting the outcome after radical surgery for bladder cancer by a newer statistical methodology, i.e. nonparametric combination (NPC). Methods: A multicenter study enrolled 1,312 patients who had undergone radical cystectomy for bladder cancer in 11 Italian oncological centers from January 1982 to December 2002. A statistical analysis of their medical history and diagnostic, pathological and postoperative variables was performed using a NPC test. The patients were included in a comprehensive database with medical history and clinical and pathological data. Five-year survival was used as the dependent variable, and p values were corrected for multiplicity using a closed testing
© 2014 S. Karger AG, Basel 0042–1138/14/0943–0262$39.50/0 E-Mail [email protected] www.karger.com/uin
procedure. The newer nonparametric approach was used to evaluate the prognostic importance of the variables. All of the analyses were performed using routines developed in MATLAB© and the significance level was set at α = 0.05. Results: A significant prognostic predictive value (p < 0.01) for tumor clinical staging, hydronephrosis, tumor pathological staging, grading, presence of concomitant carcinoma in situ, regional lymph node involvement, corpora cavernosa invasion, microvascular invasion, lymphatic invasion and prostatic stroma involvement was found. Conclusions: The NPC test could handle any type of variable (categorical and quantitative) and take into account the multivariate relation among variables. This newer methodology offers a significant contribution in biomedical studies with several endpoints and is recommended in presence of non-normal data and missing values, as well as solving high-dimensional data and problems relating to small sample sizes. © 2014 S. Karger AG, Basel
Dr. Daniele D’Agostino Department of Urology, Policlinico ‘A. Gemelli’ Catholic University of the Sacred Heart Largo Francesco Vito 1, IT–00168 Rome (Italy) E-Mail danieledagostino @ yahoo.it
Downloaded by: UCSF Library & CKM 169.230.243.252 - 4/16/2015 7:34:36 PM
Transitional cell carcinoma of the bladder is the second most common malignancy of the genitourinary tract [1]. Bladder cancer (BC) exhibits a wide spectrum of tumors with different clinical behaviors. Consequently, the treatment options differ according to the risk of disease recurrence, progression to more advanced stages, and death from the disease. Bladder tumors can be categorized into two forms: non-muscle-invasive and muscle-invasive cancer. Radical cystectomy (RC) with bilateral pelvic lymphadenectomy and urinary diversion is the gold standard treatment for muscle-invasive BC or refractory high-grade nonmuscle-invasive cancer. The overall 5-year survival rate after RC is nearly 50% [2]. Unfortunately, there is controversy surrounding the data about the independent prognostic variables of outcome of muscle-invasive or refractory non-muscle-invasive BC [3–6]. Optimal statistical analysis must identify the predictive variables of outcome, but unfortunately current models such as logistic regression (LoR) are limited because of their inadequate capacity to handle the variability and complexity of data [7]. Recently, newer statistical techniques such as nomograms and artificial intelligence (AI) have been proposed for BC, but with controversial results. This paper introduces nonparametric combination (NPC) of dependent permutation test methodology [8] to better identify prognostic factors in patients with a diagnosis of BC. With the NPC approach, we can carry out a more flexible analysis both in terms of specification of multivariate hypotheses and of the nature of the variables involved in the analysis. This methodology is able to handle any type of variable (categorical and quantitative) and to consider the multivariate relation among variables. The aim of the study was to test the capacity of the NPC statistical methodology to best identify the predictive variables affecting the outcome after radical surgery for BC.
Patients and Methods Patient Characteristics We performed a retrospective study on 1,312 patients, 1,149 (87.6%) male and 163 (12.4%) female, included in a comprehensive database with medical history and clinical and pathological data, who underwent RC, bilateral iliac and obturator node dissection, and urinary diversion for urothelial BC from January 1982 to December 2002 in 11 Italian urological centers of excellence in BC management. BC was diagnosed or confirmed by transurethral resection. Physical examination, chest X-ray, computerized tomography and/or magnetic resonance imaging were used for clinical staging. The tumors were staged according to the 2002 TNM classification and graded according to the 1997 WHO system [9, 10].
Statistical Permutation Test Methodology
Table 1. Variables evaluated and relative coding
Variable
Coding
Medical history Previous non-muscle-invasive TCC Focality Tumor stage Grading Concomitant CIS
1 2 3 4 5
Preoperative status Focality Tumor stage Concomitant CIS Grading Regional lymph nodes involvement Hydronephrosis
6 7 8 9 10 11
Postoperative status Tumor stage Concomitant CIS Grading Regional lymph node involvement Metastasis Concomitant squamous, adenocarcinoma or other differentiation Trigone invasion Corpora cavernosa invasion Urethral involvement Microvascular invasion Lymphatic invasion Prostatic stroma involvement Concomitant adenocarcinoma of the prostate Upper urinary tract TCC
12 13 14 15 16 17 18 19 20 21 22 23 24 25
TCC = Transitional cell carcinoma.
The indications for RC included: muscle-invasive BC, non-muscle-invasive BC refractory to intravesical therapy and multifocal stage T1G3 tumor associated with diffuse carcinoma in situ (CIS). No patient had clinically distant metastases at the time of operation, and the patients were periodically followed at the referral institution. Patients lost to follow-up or who died of causes other than BC were excluded. The total sample size (1,003 individuals) was divided into two groups: the first group (n = 469) included patients dead from disease and alive with relapse of disease, and the second group (n = 534) consisted of disease-free patients. The dependent variable was overall 5-year survival after RC. We took into consideration the many variables involved in the history of a patient’s BC and transformed them into categorical (ordinal) variables, and thus had 25 endpoints (table 1). NPC Methodology The NPC of dependent permutation tests is a conditional testing procedure, introduced in 2010 [11], which is appropriate and sometimes unavoidable when (1) sample sizes are smaller than the number of response variables, (2) in multivariate problems, some
Urol Int 2015;94:262–269 DOI: 10.1159/000365292
263
Downloaded by: UCSF Library & CKM 169.230.243.252 - 4/16/2015 7:34:36 PM
Introduction
Table 2. Clinical characteristics of the 1,003 evaluable patients
Variable
Result
Medical history Previous non-muscle-invasive TCC Focality Tumor stage Grading Concomitant CIS Preoperative status Focality Tumor stage Concomitant CIS Grading Regional lymph node involvement Hydronephrosis Postoperative status Tumor Concomitant CIS Grading Regional lymph nodes Metastases Concomitant squamous, adenocarcinoma or other differentiation Trigone invasion Corpora cavernosa invasion Urethral involvement Microvascular invasion Lymphatic invasion Prostatic stroma invasion Concomitant adenocarcinoma of prostate Upper urinary tract TCC Neoadjuvant chemotherapy Adjuvant chemotherapy
yes 402 single 295 Tx T0 56 5 Gx 31 yes 74
no 560 multiple 160 Ta T1 T2 T2a T2b T3 49 189 72 3 13 7 G1 18 G2 121 no 455
unk. 41 unk. 548 T3a T3b T4 T4a T4b unk. 0 3 2 2 2 600 G3 232 unk. 601 unk. 474
yes 629 Tx T0 23 9 yes 95 Gx 56 Nx 105 no 751
no 219 Ta T1 T2 T2a T2b T3 19 145 458 11 17 85 no 842 G1 6 G2 134 N0 862 N1 7 monolateral 176
unk. 155 T3b T4 T4a T4b unk. 47 4 18 4 92 unk. 66 G3 725 unk. 82 N2 3 unk. 26 bilateral 40 unk. 36
Tx T0 13 80 yes 189 Gx 32 Nx 118 M0 934 TCC 760
Ta T1 T2 T2a T2b T3 T3a T3b T4 T4a T4b unk. 16 134 138 18 22 30 138 207 11 109 11 76 no 738 unk. 76 G1 10 G2 119 G3 701 unk. 141 N0 710 N1 111 N2 49 unk. 15 M1 10 unk. 59 mixed 87 squamous ca. 24 adenocarcinoma 9 other 32 unk. 91
no 482 no 340 no 807 no 632 no 593 no 646 no 711 no 874 no 878 no 830
T3a 71
yes 411 yes 481 yes 67 yes 179 yes 249 yes 129 yes 149 yes 40 yes 25 yes 156
unk. 110 unk. 182 unk. 129 unk. 192 unk. 161 unk. 228 unk. 143 unk. 89 unk. 100 unk. 17
variables are categorical and others quantitative, (3) data sets contain missing values that cannot be ignored, and (4) data sets are obtained by selection-bias procedures. NPC tests are relatively efficient and much less demanding in terms of underlying assumptions than parametric competitors and standard distribution-free methods based on ranks, which are generally not conditional on sufficient statistics and almost never present better unconditional power behavior [11]. One major feature of the NPC of dependent tests is the fact that the researcher is not explicitly required to specify the dependence structure of response variables/endpoints. This aspect is of great importance especially for non-normal or binary variables/endpoints in which dependence relations are generally too difficult to define, and even when well defined are hard to cope with. In this sense, NPC methodology can provide an efficient and robust tool for use in the
264
Urol Int 2015;94:262–269 DOI: 10.1159/000365292
analysis of both experimental and observational studies. Due to the presence of categorical data and missing values, and given the fact that the distributional model is not well-specified (e.g. it is difficult to assume that observations are normally distributed), the complex data set at hand may be properly analyzed using NPC methodology. Once we obtained two independent samples (on the basis of the overall survival), we expanded the dataset of categorical variables in dummy variables and then performed a one-way multivariate ANOVA for comparing the means of the variables in the two samples (i.e. t test). Moreover, we adjusted p values for multiplicity using a closed testing procedure, controlling the FWE rate. All of the analyses were performed using routines developed in MATLAB© and the significance level was set at α = 0.05. Moreover, a multivariate analysis of the same variables was carried out using standard LoR to check for any possible differences.
Racioppi et al.
Downloaded by: UCSF Library & CKM 169.230.243.252 - 4/16/2015 7:34:36 PM
unk. = Unknown; TCC = transitional cell carcinoma.
Table 3. Results obtained using the NPC methodology
Variable
No.
p
p-FWE
p
p-fwe
Medical history Previous non-muscle-invasive TCC Focality Tumor stage Grading Concomitant CIS
1 2 3 4 5
n.s. n.s. n.s. n.s. n.s.
n.s. n.s. n.s. n.s. n.s.
n.s.
n.s.
Preoperative status Focality Tumor stage Concomitant CIS Grading Regional lymph node involvement Hydronephrosis
6 7 8 9 10 11
n.s. 0.0010 n.s. n.s. 0.0009 0.0002
n.s. 0.0141 n.s. n.s. 0.0126 0.0029
0.0010
0.0027
12 13 14 15 17
0.0003 0.0034 0.0003 0.0002 n.s.
0.0042 0.0391 0.0042 0.0029 n.s.
0.0004
0.0036
18 19 20 21 22 23 24 25
n.s. 0.0005 n.s. 0.0001 0.0001 0.0014 n.s. n.s.
n.s. 0.0069 n.s. 0.0011 0.0011 0.0174 n.s. n.s. 0.0011
0.0005
0.0016
n.s.
n.s.
Postoperative status Tumor stage Concomitant CIS Grading Regional lymph node involvement Concomitant squamous, adenocarcinoma or other differentiation Trigone infiltration Corpora cavernosa invasion Urethral involvement Microvascular invasion Lymphatic invasion Prostatic stroma involvement Concomitant adenocarcinoma of the prostate Upper urinary tract TCC
Results
The mean age of the patients was 64.3 years (range: 30–91). Mean and median follow-up were 60.1 and 39 months (SD ±58.3, range: 1–276), respectively. Table 2 shows the clinical characteristics of the included patients. With reference to clinical analysis, we did not find any differences between the two groups, dead and alive patients, in the medical history findings; however, in the preoperative status we found differences (p < 0.01) in terms of tumor clinical staging (variable No. 7), clinical regional lymph nodes (variable No. 10) and hydronephrosis (variable No. 11). Concerning the third group, postoperative status variables, we found significant difStatistical Permutation Test Methodology
ferences in terms of tumor pathological staging (variable No. 12), presence of concomitant CIS (variable No. 13), grading (variable No. 14), pathological regional lymph node involvement (variable No. 15), metastases (variable No. 16), corpora cavernosa invasion (variable No. 19), microscopic vascular invasion (variable No. 21), lymphatic invasion (variable No. 22) and prostatic stroma involvement (variable No. 23). All of these variables maintained their significance after adjustment for multiplicity: a closed testing procedure controlling the FWE (i.e. the probability of at least one type I error in the m tests) was applied (table 3). All of the analyses were performed using routines developed in MATLAB©, and the significance level was set at α = 0.05, B = 10,000 permutaUrol Int 2015;94:262–269 DOI: 10.1159/000365292
265
Downloaded by: UCSF Library & CKM 169.230.243.252 - 4/16/2015 7:34:36 PM
We show both raw and adjusted p values. Tippett combining function was used. 10,000 permutations were carried out. Combination of variables belonging to the same domain is also shown along with global p values. TCC = Transitional cell carcinoma.
Table 4. Results obtained using the LoR model
Variable
No.
Estimate
SE
z
p
OR
Medical history Previous non-muscle-invasive TCC Focality Tumor stage Grading Concomitant CIS
1 2 3 4 5
0.6319 0.8961 –0.1101 –0.1760 –0.5259
0.9883 0.4162 0.1658 0.2256 0.6056
0.6390 2.1530 –0.6640 –0.7800 –0.8680
0.5226 0.0313 0.5067 0.4353 0.3852
1.8812 2.4501 0.8958 0.8386 0.5911
Preoperative status Focality Tumor stage Concomitant CIS Grading Regional lymph node involvement Hydronephrosis
6 7 8 9 10 11
–0.1057 0.0685 0.1592 –0.1160 0.0969 0.4089
0.3543 0.0481 0.4814 0.2107 0.6530 0.1974
–0.2980 1.4250 0.3310 –0.5510 0.1480 2.0720
0.7654 0.1541 0.7410 0.5819 0.8820 0.0383
0.8997 1.0709 1.1725 0.8905 1.1018 1.5051
12 13 14 15 17
0.1569 0.2783 0.1575 0.4965 0.1074
0.0433 0.2745 0.2366 0.2217 0.1283
3.6210 1.0140 0.6660 2.2400 0.8370
0.0003 0.3107 0.5055 0.0251 0.4025
1.1698 1.3208 1.1706 1.6430 1.1134
18 19 20 21 22 23 24
–0.0375 –0.1973 –0.6318 0.2728 0.2296 0.4198 0.0568
0.2236 0.2709 0.4788 0.3759 0.2321 0.3361 0.2535
–0.1680 –0.7280 –1.3190 0.7260 0.9890 1.2490 0.2240
0.8668 0.4664 0.1870 0.4681 0.3226 0.2116 0.8226
0.9632 0.8210 0.5316 1.3136 1.2581 1.5217 1.0585
25
0.1296
0.5126
0.2530
0.8003
1.1384
Postoperative status Tumor stage Concomitant CIS Grading Regional lymph node involvement Concomitant squamous, adenocarcinoma or other differentiation Trigone infiltration Corpora cavernosa invasion Urethral involvement Microvascular invasion Lymphatic invasion Prostatic stroma involvement Concomitant adenocarcinoma of the prostate Upper urinary tract TCC
tions (Monte Carlo sampling) was used for estimating the permutation distribution. Results obtained using the LoR model are summarized in table 4. A statistical difference was observed in focality (variable No. 2), hydronephrosis (variable No. 11), tumor pathological staging (variable No. 12) and pathological regional lymph node involvement (variable No. 15).
Discussion
The newer nonparametric approach is a statistical methodology which we applied (to the best of our knowledge) for the first time in oncology to a large population of patients treated for BC. NPC highlighted the prognostic importance of factors not otherwise usually considered in BC. 266
Urol Int 2015;94:262–269 DOI: 10.1159/000365292
The appropriate and tailored assignment of risk is a pivotal principle in the management of any cancer. Many efforts have been addressed to develop statistical techniques that are able to identify prognostic factors of outcome after RC for BC. The TNM staging system originated as a response to the need for an accurate and consistent universal cancer outcome prediction system [12]. Universally, pathological stage together with grade are the most reliable predictors of bladder tumor behavior, and TNM prediction is accurate in 70% of tumors in the best series [13]. Furthermore, since the TNM staging system was introduced in the 1950s, new prognostic factors have been identified [14]. For example, the presence of CIS and the state of the bladder 3 months after surgery are important prognostic factors in non-muscle-invasive carcinoma [13]; lymphovascular invasion represents a highly significant and independent preRacioppi et al.
Downloaded by: UCSF Library & CKM 169.230.243.252 - 4/16/2015 7:34:36 PM
TCC = Transitional cell carcinoma. Bold values denote statistical significance.
The possible weakness is represented by the normality assumption which might often not be valid in clinical investigations based on observational studies with no randomization. Both methods of AI predicted relapse with an accuracy ranging from 88 to 95%, which is superior to standard statistical methods [13]. ANNs are complex mathematical models designed to replicate human decision-making by emulating processing pathways used by the human brain [7]. Unlike LoR models, ANNs can accommodate large numbers of prognostic variables and can analyze their varying degrees of impact on the outcome [18]. Bassi et al. [16] showed that ANNs have a prognostic performance comparable with that of LoR. In particular they demonstrated that ANNs could outperform LoR when applied to small or moderate datasets. Qureshi et al. [18] reported that ANNs were significantly more accurate than clinicians in predicting stage progression in a retrospective series of patients with T1G3 BC (overall accuracy: 82 vs. 40%). Catto et al. [13] confirmed that ANNs provide better accuracy (90 vs. 77%) than traditional statistical methods for predicting relapse in patients with non-muscle- and muscle-invasive BC, based on clinicopathological data. Comparing the TNM staging system’s predictive accuracy with that of ANNs, Burke et al. [14] showed that ANNs are significantly more accurate than the TNM staging system. Moreover, new prognostic factors can be added to ANNs to increase their prognostic accuracy further. On the other hand, the network is hidden within a functional ‘black box’: it is difficult to gain insight into the solution used to resolve the clinical data, making subsequent analysis (to ensure clinical sense prevails) and interrogation of new variables almost impossible. So, statisticians are reluctant to believe in the validity of ANNs [19]. NFM has a similar or superior predictive accuracy to ANNs, but without the ‘black box’, enabling validation from clinical knowledge and the manipulation of input variables to allow exploratory predictions [13]. Comparing both methods, Catto et al. [13] showed that NFM appeared significantly better than ANNs at predicting the timing of relapse, and significantly more accurate than LoR and linear regression. However, there seems to be no agreement on the specification of the prognostic factors determining the outcome after RC for BC. Alternative statistical procedures are represented by permutation methods. These tests are distribution-free and allow for efficient solutions even when the number of cases is less than the number of variables. We propose a nonparametric permutation approach in order to better and more easily identify the
Statistical Permutation Test Methodology
Urol Int 2015;94:262–269 DOI: 10.1159/000365292
267
Downloaded by: UCSF Library & CKM 169.230.243.252 - 4/16/2015 7:34:36 PM
dictor of cancer-specific survival in lymph node negative cystectomy patients after adjusting for pathological stage [15, 16]. Thus, clinicians are routinely forced to use their judgment to incorporate factors not included in the staging system. Traditionally, statistical techniques such as Cox’s proportional hazards and LoR were employed and, until the advent of AI, LoR was considered the best method for predicting tumor behavior [13]. When LoR is used, i.e. Cox multivariate regression analysis, Bassi et al. [16] found that in a retrospective study of patients who underwent RC for BC, only pathological stage and nodal involvement were predictive of survival. Solsona et al. [17], however, showed that together with the latter, prostatic stroma status is also an independent variable. In a similar study, Frazier et al. [15] found that pT stage, nodal involvement, positive surgical margins, the patient’s age at surgery and loss of histologic differentiation were predictive of poor cancer-specific survival in a multivariate analysis. However, the predictive accuracy of this method is poor, in the range of 69–77% [13, 14]. The traditional statistical methods in evaluating prognostic factors are limited because of their need for linear relationships between variables, large datasets and their poor performance when large amounts of ‘noise’ (inherent variation) contaminate a dataset [13]. In particular, in situations where newer explored markers interact in a complex fashion, the processing rules or mathematical equations are often very difficult to determine [18]. Moreover, they may be applicable to a population, but are not predictive for an individual [13]. In the last decades more advanced clinical studies have been focused on AI to accurately predict cancer behavior [13]. By using AI methods, such as artificial neural networks (ANNs) and neurofuzzy modeling (NFM), complex nonlinear relationships between dependent and independent variables in a population of which distribution may not be normal can be identified [13]. ANNs are computational models inspired by animals’ central nervous systems (in particular the brain) that are capable of machine learning and pattern recognition. They are usually presented as systems of interconnected ‘neurons’ that can compute values from input by feeding information through the network. NFM refers to combinations of ANNs and fuzzy logic. Neurofuzzy hybridization results in a hybrid intelligent system that synergizes these two techniques by combining the human-like reasoning style of fuzzy systems with the learning and connectionist structure of neural networks. The main strength of neurofuzzy systems is that they are universal approximators with the ability to solicit interpretable ‘IF-THEN’ rules.
268
Urol Int 2015;94:262–269 DOI: 10.1159/000365292
when considering the future of a patient who is to undergo radical surgery for BC. This method is surely transferable to all other human malignancies, as it allows the evaluation of new treatment strategies and may be useful in the design of clinical trials. We recommend the use of this method whenever the normality assumption is hard to justify, in the presence of missing values and when the number of variables is higher than the number of patients, this last aspect being very frequent in biomedical studies. Our study has some limitations. First of all, the study population came from 11 different centers and obviously the clinical evaluations and operations were performed by different physicians. However, the large number of patients limits the importance of this heterogeneity. Second, the study is retrospective. On the other hand, these aspects, which one could consider negative, can assume a positive value because the aim of the NPC methodology is to be rigorous but flexible to go beyond the problems linked to incomplete datasets, which is different from the traditional statistical methods.
Conclusions
NPC of dependent permutation test methodology is a newer statistical approach which can offer a significant contribution to successful research in biomedical studies with several endpoints. The advantages of this approach are its flexibility in handling any type of variable while at the same time taking dependencies among those variables into account without the need for modeling them [20]. In our study, NPC methodology identified new predictive variables affecting the outcome of patients who underwent radical surgery for BC. Along with lymph node involvement, pathological stage and grade at diagnosis, which are the most reliable predictors of bladder tumor behavior, we also found other potential prognostic factors. We wish to highlight that NPC methodologies can provide a more accurate prediction of clinical outcome.
References
1 Jemal A, Siegel R, Ward E, et al: Cancer statistics, 2008. CA Cancer J Clin 2008;58:71–96. 2 Herr HW, Faulkner JR, Grossman HB, et al: Surgical factors influence bladder cancer outcomes: a cooperative group report. J Clin Oncol 2004;22:2781–2789. 3 Bassi P, Ferrante GD, Piazza N, et al: Prognostic factors of outcome after radical cystectomy for bladder cancer: a retrospective study of a homogeneous patients cohort. J Urol 1999; 161:1494–1497.
Racioppi et al.
Downloaded by: UCSF Library & CKM 169.230.243.252 - 4/16/2015 7:34:36 PM
prognostic factors allowing to discriminate the outcome of patients with a BC diagnosis. Using this methodology, we are able to obtain more information that is qualitatively and quantitatively different from that provided by standard statistical methods. First of all, if we perform a standard LoR in our population, we have to invoke the principle of deletion or imputation in order to deal with missing data. However, following the previous nonparametric solution, we do not need to delete/impute. Moreover, NPC methodology does not have the drawback of using black boxes. By using NPC methodology and adjusting the p value, we have found 12 prognostic factors for survival. The test confirmed significant prognostic factors: tumor pathological staging (variable No. 12), grading (variable No. 14), pathological regional lymph nodes involvement (variable No. 16) and prostatic stroma involvement (variable No. 23). In this large series, however, by using NPC, a strong negative significance also arose for tumor clinical staging (variable No. 7), clinical regional lymph node involvement (variable No. 10), hydronephrosis (variable No. 11), presence of concomitant CIS (variable No. 13), corpora cavernosa invasion (variable No. 19), microscopic vascular invasion (variable No. 21) and lymphatic invasion (variable No. 22), all of which are factors not otherwise fully taken into consideration. Hence, we could assert that NPC methodology is an instrument that provides an accurate prediction of clinical outcome, capable of better evaluating datasets and giving significance to adjunctive prognostic factors, i.e. the outcome evaluation improves as the number of prognostic factors increases. Through this application to a real case study, we have shown how NPC methodology can offer a significant contribution to successful research in biomedical studies with several endpoints. Advantages of this approach are related to its flexibility in both handling any type of variables (categorical and quantitative with or without missing values) and simultaneously of taking into account the multivariate relation among those variables. Moreover, the underlying dependence relation structure is implicitly captured in a nonparametric way by the combination procedure. Hence, the researcher is not explicitly required to specify the dependence structure of response variables. This feature is of great importance especially for non-normal or categorical variables, in which dependence relations are too difficult to define and, if defined, are hard to cope with. In this study we have demonstrated that additional factors to those already known should be taken in account
Statistical Permutation Test Methodology
10 Epstein JI, Amin MB, Reuter VR, et al: The World Health Organization/International Society of Urological Pathology consensus classification of urothelial (transitional cell) neoplasms of the urinary bladder. Bladder Consensus Conference Committee. Am J Surg Pathol 1998;22:1435–1448. 11 Pesarin F, Salmaso L: Permutation Tests for Complex Data. Hoboken, John Wiley & Sons, 2010. 12 Beahrs OH: How can general surgery survive? Am Surg 1992;58:17–21. 13 Catto JW, Linkens DA, Abbod MF, et al: Artificial intelligence in predicting bladder cancer outcome: a comparison of neuro-fuzzy modeling and artificial neural networks. Clin Cancer Res 2003;9:4172–4177. 14 Burke HB, Goodman PH, Rosen DB, et al: Artificial neural networks improve the accuracy of cancer survival prediction. Cancer 1997;79: 857–862. 15 Frazier HA, Robertson JE, Dodge RK, Paulson DF: The value of pathologic factors in predicting cancer-specific survival among patients treated with radical cystectomy for transitional cell carcinoma of the bladder and prostate. Cancer 1993;71:3993–4001.
Urol Int 2015;94:262–269 DOI: 10.1159/000365292
16 Bassi P, Sacco E, De Marco V, et al: Prognostic accuracy of an artificial neural network in patients undergoing radical cystectomy for bladder cancer: a comparison with logistic regression analysis. BJU Int 2007; 99: 1007– 1012. 17 Solsona E, Iborra I, Dumont R, et al: Risk groups in patients with bladder cancer treated with radical cystectomy: statistical and clinical model improving homogeneity. J Urol 2005;174:1226–1230. 18 Qureshi KN, Naguib RN, Hamdy FC, et al: Neural network analysis of clinicopathological and molecular markers in bladder cancer. J Urol 2000;163:630–633. 19 Schwarzer G, Vach W, Schumacher M: On the misuses of artificial neural networks for prognostic and diagnostic classification in oncology. Stat Med 2000;19:541–561. 20 Pesarin F: Extending permutation conditional inference to unconditional one. Stat Methods Appt 2002;11:161–173.
269
Downloaded by: UCSF Library & CKM 169.230.243.252 - 4/16/2015 7:34:36 PM
4 Dalbagni G, Genega E, Hashibe M, et al: Cystectomy for bladder cancer: a contemporary series. J Urol 2001;165:1111–1116. 5 Madersbacher S, Hochreiter W, Burkhard F, et al: Radical cystectomy for bladder cancer today – a homogeneous series without neoadjuvant therapy. J Clin Oncol 2003;21:690–696. 6 Monzó Gardiner JI, Herranz Amo F, Díez Cordero JM, et al: Factores pronósticos en la supervivencia de los pacientes con carcinoma transicional de vejiga tratados con cistectomía radical. Actas Urol Esp 2009;33:249–257. 7 Shabsigh A, Bochner BH: Use of nomograms as predictive tools in bladder cancer. World J Urol 2006;24:489–498. 8 Finos L, Salmaso L: Weighted methods controlling the multiplicity when the number of variables is much higher than the number of observations. J Nonparametr Stat 2006; 18: 245–261. 9 Sobin LH, Greene FL: TNM classification: clarification of number of regional lymph nodes for pNo. Cancer 2001;92:452.
