Breast Cancer Research and Treatment 21: 47-53, 1992 © 1992 Kluwer Academic Publishers. Printed in the Netherlands.
Report
A demonstration that breast cancer recurrence can be predicted by neural network analysis Peter M. Ravdin, Gary M. Clark, Susan G. Hilsenbeck, Marilyn A. Owens*, Patricia Vendely*, M.R. Pandian*, and William L. McGuire Department of Medicine, Division of Oncology, University of Texas Health Science Center, San Antonio, Texas 78284-7884; and * Nichols Institute Reference Laboratories, San Juan Capistrano, California 92675
Key words: neural networks, breast cancer, prognosis, survival analysis, Cox regression
Summary Neural Network Analysis, a form of artificial intelligence, was successfully used to predict the clinical outcome of node-positive breast cancer patients. A Neural Network was trained to predict clinical outcome using prognostic information from 1008 patients. During training, the network received as input information tumor hormone receptor status, DNA index and S-phase determination by flow cytometry, tumor size, number of axillary lymph nodes involved with tumor, and age of the patient, as well as length of clinical followup, relapse status, and time of relapse. The ability of the trained Network to determine relapse probability was then validated in a separate set of 960 patients. The Neural Network was as powerful as Cox Regression Modeling in identifying breast cancer patients at high and low risk for relapse.
Introduction A problem often faced in clinical medicine is how to reach a conclusion about a patient's prognosis when presented with complex, and sometimes contradictory, clinical and prognostic information. The clinician usually makes decisions based on a simple dichotomization of variables into favorable and unfavorable classifications, and then rather subjectively weights the factors to reach an overall assessment of the patient's prognosis [1]. An example of this decision making process occurs when deciding whether a patient with primary breast cancer should receive adjuvant
therapy after surgery to remove the primary tumor. An important component of this decision is the assessment of the likelihood that the patient will suffer a recurrence of her disease, so that the risks and expected benefits of specific therapies can be compared. Currently, this assessment is made by considering such factors as menopausal status, whether regional lymph nodes are involved with tumor, the size of the tumor, estrogen receptor (ER) and progesterone receptor (PgR) status, and whether the proliferation rate of the tumor is high or low as measured by flow cytometry. Two weaknesses of this approach are: 1) information may be lost in the dichotomization of continuous variables, and 2) the subjective
Address for offprints: Dr. William L. McGuire, Medicine/Oncology,University of Texas Health Science Center, San Antonio TX 78282-7884, USA
48
PM Ravdin et al.
weighting of multiple factors does not lead to a quantitative assessment of risk of disease recurrence. More sophisticated quantitative techniques such as Cox Regression Modeling [2], recursive partitioning [3], and Neural Network Analysis [4] have the potential to improve this decision making process. Neural Network Analysis is a form of computer-simulated artificial intelligence that has actually outperformed other analytic techniques for such complex tasks as predicting protein secondary structure [5,6] and synthesizing spoken language [7]. Neural networks learn to distinguish different classes of events by being given information about an event and its eventual class or outcome [8]. During training, the neural network uses a powerful algorithm (known as the back propagation of errors) to continually modify its internal connections to optimize its prediction of class or outcome. This allows multiple layer neural networks to generate complex multidimensional prediction maps. In this paper, we show that neural networks can be trained to predict disease recurrence for patients with primary breast cancer, and we offer a preliminary comparison of the use of Neural Network Analysis and Cox Regression Modeling in establishing the prognosis of patients in a large breast cancer data base.
patients into a training set of 1008 patients and a validation set of 960 patients, based solely on whether the last digit of the sequentially assigned patient accession number was odd or even. The training set was used to construct prognostic models, and the validation set was used to test the ability of the resulting models to generalize to an independent set of patients.
Neural network models Neural networks were simulated using Nworks Professional II software (Neuralware, Pittsburgh, PA). All networks had three layers (input, hidden, and output) and used the hyperbolic tangent as a transfer function in the hidden and output layers. During training, internal connections were continuously modified using the back propagation of errors algorithm. The epoch size was 10 data presentations. No momentum term was used. In order to optimize the construction of neural network models, patients in the 1008 training set were further randomly assigned to a 508 patient "internal" training set used to train the models, or a 500 patient "internal" testing set used to evaluate the performance of the models, and to select the best model for later evaluation on the validation set.
Methods
Cox regression model
Patients and prognostic information
Cox Regression modeling was performed using the 2L program in the BMDP Statistical Software (Los Angeles, CA). The stepwise MPLR method was used. Exploratory data analyses of the training set were conducted to identify appropriate representations of the prognostic factors for inclusion in subsequent survival analyses. The various factors were considered alone, and as interaction terms with other variables, in order to identify combinations of terms that best fit the training data. Several models of near-identical performance on the training set were constructed. Of the models with the best performance in the training set, the simplest model was then used to
The data used in these analyses are from the Nichols Institute Oncology Research Network (Nichols Institute, San Juan Capistrano, CA). The patient population consists of 1968 patients with breast cancer who had undergone resection of the primary tumor, and at the time of surgery had axillary lymph nodes positive for cancer, but were without evidence of distant metastatic disease. ER and PgR determinations and DNA flow cytometry were performed at Nichols Institute on tumor specimens from each of these patients as previously described [9]. We separated the
Neural network analysis of relapse risk
49
Table 1. Summary of patient data: medians and (ranges).
Factor
Training Set
Test Set
Number of Patients Followup (months) % that relapsed DNA index S-phase (%) ER PgR Tumor Size (cm) Positive Nodes Patient Age
1008 16 (1-63) 13.6 1.45 (0.9-3.86) 6.5 (0.1-65.0) 61 (0-1282) 58 (0-2705) 2.5 (0.1-15) 3 (1-69) 60 (21-97)
960 16 (1-64) 11.3 1.43 (0.87-3.24) 6.8 (0.2-43.4) 65 (0-1698) 55 (0-3901) 2.5 (0.1-14.5) 3 (1-53) 60 (28-94)
make predictions of patient outcome in the validation data set.
training set, but poor performance on independent testing sets. Several of the prognostic factors were transformed before they were input into the neural
Results
Patient sets The medians and ranges for the prognostic variables are displayed in Table 1. There were no significant differences in the distributions of the variables in the training and validation sets. The median followup time was short, only 16 months.
Output Modifiable Connection Weights r - I
Output Layer
Neural network analysis The Neural Network used in the analysis is shown in Figure 1. It had eight input units, four units in the hidden layer, and one unit in the output layer. It was "trained" using the 1008 patient training set. Patients in the training set were randomly assigned to a 508 patient "internal" training set used to train the models, or a 500 patient "internal" testing set used to evaluate the performance of the models to select the best model for evaluation in the validation set. This "internal" training and testing was necessary because neural networks can be over-trained to recognize specific cases in a training set rather than learning general predictive characteristics. Over-training can lead to good performance on a
Hidden Layer
Input Layer
ER
to. tOextTL Pt.~gel PgR S-phase
# Nodes
F/U Time
Input Data
Figure I. This figure shows the basic architecture of the Neural Network that produced the indices used in this paper. The network had eight input units, four hidden layer units, and an output unit that was compared to the relapse status of the patient during training and produced the prognostic indices during testing.
50
PM Ravdin et al.
networks. This was done to lessen the impact of the few points with very high values by taking logarithms to the base 10 of some of the factors (ER, PgR, S-phase fraction, tumor size, and number of positive nodes). Another problem that needed to be addressed was the different lengths of followup of these patients, which ranged from 1 to 64 months. We felt that data from a non-relapsing patient that had been followed for five years should be given more weight than data from a non-relapsing patient that had been followed for only one year. Similarly, a patient that had relapsed in the first year should be given a different weight than a patient who had relapsed after five years. During training, this was handled by presenting a patient to the Neural Network once for each year of followup and providing the relapse status for that year of followup. This allowed the neural network to predict patient outcome at different followup times, and allowed it to search for possible complex, time-dependent interactions of the prognostic factors. This form of presentation of survival time information is analogous to the form used in life table survival analyses [10]. Table 2 shows the data as presented to the Neural Network. After neural net models were constructed using the "internal" training set of patients, they were validated on the "internal" test set. Each final model generated a prognostic index that was monotonically related to the predicted relapse rates. We then used the values of the prognostic index from the "internal" test set to create subsets of patients with differing predicted risks of
relapse at two years of followup. Several neural networks of near-identical performance were created.
Cox regression model The terms in the best Cox Regression Model created using the 1008 patient training set are shown in Table 3. The strongest terms were ER status and the number of positive lymph nodes. Other factors included in the model were tumor size, S-phase, and interaction terms between the number of positive nodes, and ER and S- phase.
Comparison of modeling techniques Both Neural Network Analysis and Cox Regression Modeling produced models that adequately fit the observed data. The strongest models developed using the training set were applied to the 960 patient validation set. Prognostic indices were determined by both Neural Network Analysis and Cox Regression Modeling, and global chi-square tests for goodness of fit were computed. Both models produced excellent fits to the observed data with the neural net model being marginally better than the Cox model (Neural Net chi-square=48.59, p
Report
A demonstration that breast cancer recurrence can be predicted by neural network analysis Peter M. Ravdin, Gary M. Clark, Susan G. Hilsenbeck, Marilyn A. Owens*, Patricia Vendely*, M.R. Pandian*, and William L. McGuire Department of Medicine, Division of Oncology, University of Texas Health Science Center, San Antonio, Texas 78284-7884; and * Nichols Institute Reference Laboratories, San Juan Capistrano, California 92675
Key words: neural networks, breast cancer, prognosis, survival analysis, Cox regression
Summary Neural Network Analysis, a form of artificial intelligence, was successfully used to predict the clinical outcome of node-positive breast cancer patients. A Neural Network was trained to predict clinical outcome using prognostic information from 1008 patients. During training, the network received as input information tumor hormone receptor status, DNA index and S-phase determination by flow cytometry, tumor size, number of axillary lymph nodes involved with tumor, and age of the patient, as well as length of clinical followup, relapse status, and time of relapse. The ability of the trained Network to determine relapse probability was then validated in a separate set of 960 patients. The Neural Network was as powerful as Cox Regression Modeling in identifying breast cancer patients at high and low risk for relapse.
Introduction A problem often faced in clinical medicine is how to reach a conclusion about a patient's prognosis when presented with complex, and sometimes contradictory, clinical and prognostic information. The clinician usually makes decisions based on a simple dichotomization of variables into favorable and unfavorable classifications, and then rather subjectively weights the factors to reach an overall assessment of the patient's prognosis [1]. An example of this decision making process occurs when deciding whether a patient with primary breast cancer should receive adjuvant
therapy after surgery to remove the primary tumor. An important component of this decision is the assessment of the likelihood that the patient will suffer a recurrence of her disease, so that the risks and expected benefits of specific therapies can be compared. Currently, this assessment is made by considering such factors as menopausal status, whether regional lymph nodes are involved with tumor, the size of the tumor, estrogen receptor (ER) and progesterone receptor (PgR) status, and whether the proliferation rate of the tumor is high or low as measured by flow cytometry. Two weaknesses of this approach are: 1) information may be lost in the dichotomization of continuous variables, and 2) the subjective
Address for offprints: Dr. William L. McGuire, Medicine/Oncology,University of Texas Health Science Center, San Antonio TX 78282-7884, USA
48
PM Ravdin et al.
weighting of multiple factors does not lead to a quantitative assessment of risk of disease recurrence. More sophisticated quantitative techniques such as Cox Regression Modeling [2], recursive partitioning [3], and Neural Network Analysis [4] have the potential to improve this decision making process. Neural Network Analysis is a form of computer-simulated artificial intelligence that has actually outperformed other analytic techniques for such complex tasks as predicting protein secondary structure [5,6] and synthesizing spoken language [7]. Neural networks learn to distinguish different classes of events by being given information about an event and its eventual class or outcome [8]. During training, the neural network uses a powerful algorithm (known as the back propagation of errors) to continually modify its internal connections to optimize its prediction of class or outcome. This allows multiple layer neural networks to generate complex multidimensional prediction maps. In this paper, we show that neural networks can be trained to predict disease recurrence for patients with primary breast cancer, and we offer a preliminary comparison of the use of Neural Network Analysis and Cox Regression Modeling in establishing the prognosis of patients in a large breast cancer data base.
patients into a training set of 1008 patients and a validation set of 960 patients, based solely on whether the last digit of the sequentially assigned patient accession number was odd or even. The training set was used to construct prognostic models, and the validation set was used to test the ability of the resulting models to generalize to an independent set of patients.
Neural network models Neural networks were simulated using Nworks Professional II software (Neuralware, Pittsburgh, PA). All networks had three layers (input, hidden, and output) and used the hyperbolic tangent as a transfer function in the hidden and output layers. During training, internal connections were continuously modified using the back propagation of errors algorithm. The epoch size was 10 data presentations. No momentum term was used. In order to optimize the construction of neural network models, patients in the 1008 training set were further randomly assigned to a 508 patient "internal" training set used to train the models, or a 500 patient "internal" testing set used to evaluate the performance of the models, and to select the best model for later evaluation on the validation set.
Methods
Cox regression model
Patients and prognostic information
Cox Regression modeling was performed using the 2L program in the BMDP Statistical Software (Los Angeles, CA). The stepwise MPLR method was used. Exploratory data analyses of the training set were conducted to identify appropriate representations of the prognostic factors for inclusion in subsequent survival analyses. The various factors were considered alone, and as interaction terms with other variables, in order to identify combinations of terms that best fit the training data. Several models of near-identical performance on the training set were constructed. Of the models with the best performance in the training set, the simplest model was then used to
The data used in these analyses are from the Nichols Institute Oncology Research Network (Nichols Institute, San Juan Capistrano, CA). The patient population consists of 1968 patients with breast cancer who had undergone resection of the primary tumor, and at the time of surgery had axillary lymph nodes positive for cancer, but were without evidence of distant metastatic disease. ER and PgR determinations and DNA flow cytometry were performed at Nichols Institute on tumor specimens from each of these patients as previously described [9]. We separated the
Neural network analysis of relapse risk
49
Table 1. Summary of patient data: medians and (ranges).
Factor
Training Set
Test Set
Number of Patients Followup (months) % that relapsed DNA index S-phase (%) ER PgR Tumor Size (cm) Positive Nodes Patient Age
1008 16 (1-63) 13.6 1.45 (0.9-3.86) 6.5 (0.1-65.0) 61 (0-1282) 58 (0-2705) 2.5 (0.1-15) 3 (1-69) 60 (21-97)
960 16 (1-64) 11.3 1.43 (0.87-3.24) 6.8 (0.2-43.4) 65 (0-1698) 55 (0-3901) 2.5 (0.1-14.5) 3 (1-53) 60 (28-94)
make predictions of patient outcome in the validation data set.
training set, but poor performance on independent testing sets. Several of the prognostic factors were transformed before they were input into the neural
Results
Patient sets The medians and ranges for the prognostic variables are displayed in Table 1. There were no significant differences in the distributions of the variables in the training and validation sets. The median followup time was short, only 16 months.
Output Modifiable Connection Weights r - I
Output Layer
Neural network analysis The Neural Network used in the analysis is shown in Figure 1. It had eight input units, four units in the hidden layer, and one unit in the output layer. It was "trained" using the 1008 patient training set. Patients in the training set were randomly assigned to a 508 patient "internal" training set used to train the models, or a 500 patient "internal" testing set used to evaluate the performance of the models to select the best model for evaluation in the validation set. This "internal" training and testing was necessary because neural networks can be over-trained to recognize specific cases in a training set rather than learning general predictive characteristics. Over-training can lead to good performance on a
Hidden Layer
Input Layer
ER
to. tOextTL Pt.~gel PgR S-phase
# Nodes
F/U Time
Input Data
Figure I. This figure shows the basic architecture of the Neural Network that produced the indices used in this paper. The network had eight input units, four hidden layer units, and an output unit that was compared to the relapse status of the patient during training and produced the prognostic indices during testing.
50
PM Ravdin et al.
networks. This was done to lessen the impact of the few points with very high values by taking logarithms to the base 10 of some of the factors (ER, PgR, S-phase fraction, tumor size, and number of positive nodes). Another problem that needed to be addressed was the different lengths of followup of these patients, which ranged from 1 to 64 months. We felt that data from a non-relapsing patient that had been followed for five years should be given more weight than data from a non-relapsing patient that had been followed for only one year. Similarly, a patient that had relapsed in the first year should be given a different weight than a patient who had relapsed after five years. During training, this was handled by presenting a patient to the Neural Network once for each year of followup and providing the relapse status for that year of followup. This allowed the neural network to predict patient outcome at different followup times, and allowed it to search for possible complex, time-dependent interactions of the prognostic factors. This form of presentation of survival time information is analogous to the form used in life table survival analyses [10]. Table 2 shows the data as presented to the Neural Network. After neural net models were constructed using the "internal" training set of patients, they were validated on the "internal" test set. Each final model generated a prognostic index that was monotonically related to the predicted relapse rates. We then used the values of the prognostic index from the "internal" test set to create subsets of patients with differing predicted risks of
relapse at two years of followup. Several neural networks of near-identical performance were created.
Cox regression model The terms in the best Cox Regression Model created using the 1008 patient training set are shown in Table 3. The strongest terms were ER status and the number of positive lymph nodes. Other factors included in the model were tumor size, S-phase, and interaction terms between the number of positive nodes, and ER and S- phase.
Comparison of modeling techniques Both Neural Network Analysis and Cox Regression Modeling produced models that adequately fit the observed data. The strongest models developed using the training set were applied to the 960 patient validation set. Prognostic indices were determined by both Neural Network Analysis and Cox Regression Modeling, and global chi-square tests for goodness of fit were computed. Both models produced excellent fits to the observed data with the neural net model being marginally better than the Cox model (Neural Net chi-square=48.59, p
