Ir J Med Sci DOI 10.1007/s11845-015-1291-8
ORIGINAL ARTICLE
Predicting prostate cancer: analysing the clinical efficacy of prostate cancer risk calculators in a referral population R. W. Foley1,2,3 • D. J. Lundon1,2,3 • K. Murphy4 • T. B. Murphy4,5 • D. J. Galvin1,3 • R. W. G. Watson1,2
Received: 12 February 2015 / Accepted: 29 March 2015 Ó Royal Academy of Medicine in Ireland 2015
Abstract Background The decision to proceed to biopsy for the diagnosis of prostate cancer in clinical practice is a difficult one. Prostate cancer risk calculators allow for a systematic approach to the use of patient information to predict a patient’s likelihood of prostate cancer. Aims In this paper, we validate the two leading prostate cancer risk calculators, the prostate cancer prevention trial (PCPT) and the European Randomized Study of Screening for Prostate Cancer (ERSPC) in an Irish population. Methods Data were collected for 337 men referred to one tertiary referral center in Ireland. Calibration analysis, ROC analysis and decision curve analysis were undertaken to ascertain the performance of the PCPT and the ERSPC risk calculators in this cohort. Results Of 337 consecutive biopsies, cancer was subsequently diagnosed in 146 men (43 %), 98 (67 %) of which were high grade. The AUC for the PCPT and ERSPC risk calculators were 0.68 and 0.66, respectively for the prediction of prostate cancer. Each calculator was sufficiently
& R. W. Foley [email protected] 1
UCD School of Medicine and Medical Science, University College Dublin, Belfield, Dublin 4, Ireland
2
UCD Conway Institute of Biomolecular and Biomedical Research, University College Dublin, Belfield, Dublin 4, Ireland
3
Department of Urology, Mater Misericordiae University Hospital, Eccles Street, Dublin 7, Ireland
4
UCD School of Mathematical Sciences, University College Dublin, Dublin 4, Ireland
5
Insight Centre for Data Analytics, University College Dublin, Dublin 4, Ireland
calibrated in this cohort. Decision curve analysis demonstrated a net benefit via the use of the PCPT and ERSPC risk calculators in the diagnosis of prostate cancer. Conclusions The PCPT and ERSPC risk calculators achieve a statistically significant prediction of prostate cancer in this Irish population. This study provides external validation for these calculators, and therefore these tools can be used to aid in clinical decision making. Keywords Biopsy Decision support techniques Prostatic neoplasm Risk
Introduction Prostate cancer (PCa) is the most common non-cutaneous cancer in men in Ireland [1]. This disease causes significant morbidity and mortality; every year, over 500 men in Ireland die from this disease [2]. The current standard of diagnosis for PCa is a transrectal ultrasound (TRUS) prostate biopsy. This procedure carries with it significant risks [3], including rare cases of biopsy-related mortality. It is important to assess the possible benefit of such a procedure before exposing a patient to these risks. The shared decision to proceed to biopsy of the prostate is based on the patient history, digital rectal examination (DRE) and a serum prostate-specific antigen (PSA) level following informative discussion with the patient and partners or carers. The NICE guidelines put this in context and the recommendations are echoed from the US College of Physicians [4]. Tools exist to provide a prediction of the likely outcome of a biopsy based on these and other weighted risk factors. The most thoroughly validated calculators to determine the risk of PCa on biopsy are the prostate cancer prevention trial
123
Ir J Med Sci
(PCPT) [5] and the European Randomized Study of Screening for Prostate Cancer (ERSPC) risk calculators [6]. These calculators are becoming increasingly important in clinical practice for the management of PCa, with recent European Association of Urology guidelines advocating the integration of a multivariate approach into PCa diagnosis [7]. The calculators were created in two distinct patient populations and each incorporates their own unique set of clinical parameters. The parameters included in the PCPT risk calculator are age, race, abnormality on DRE, family history of PCa, a previous negative biopsy and PSA level. The performance of the PCPT calculator in the participants with whom the calculator was developed gave an AUC value of 0.702 [5]. The ERSPC risk calculators consist of 6 steps and are available online [8]. Each of these risk calculators is tailored to a specific type of patient and calculates a man’s individual risk of PCa. Step 3?DRE which consists of PSA level, abnormality on DRE, DRE-based prostate volume estimate and step 4?DRE which consists of PSA level, abnormality on DRE, DRE-based prostate volume estimate and previous negative prostate biopsy were utilised in this study [9]. Step 3?DRE is applicable to men undergoing initial prostate biopsy, whilst step 4?DRE is applicable to men at repeat biopsy. The ERSPC steps 3?DRE and 4?DRE risk calculators were selected to allow for a meaningful comparison with the PCPT risk calculator. The steps 3?DRE and 4?DRE calculators incorporate DRE-based prostate volume estimation and not TRUS prostate volume. Because TRUS is not routinely performed prebiopsy, these steps of the ERSPC calculator are more readily applicable to the current Irish clinical setting. The performance of the ERSPC risk calculators have been validated in a number of cohorts, both North American and European, and have yielded AUC values between 0.70 and 0.80 [10–13]. The PCPT and ERSPC risk calculators were created from large-scale screening trials for PCa. A recent metaanalysis by the cochrane collaboration has found no benefit in PCa screening, and noted frequent harms associated with subsequent investigations, i.e., prostate biopsy [14]. Therefore, the decision on which patients to send for biopsy is a difficult one. Risk calculators for PCa prediction are important to aid with this decision. Before the possible introduction into clinical practice, these tools must be validated. As advocated by Trottier et al. [11], the disparity in performance of these calculators in different geographical regions ‘‘emphasizes the importance of nomogram validation in populations from different countries’’. This study aims to validate these risk calculators in a cohort of Irish men.
123
Materials and methods Patient population The patient population consisted of men referred to the Mater Misericordiae rapid access prostate cancer clinic (RAPCC) with an East of Ireland based catchment area. Ethical approval was awarded from the institution. We retrospectively collected the relevant information as described by the PCPT risk calculator and the ERSPC risk calculator in patients undergoing a TRUS biopsy. These men were referred to the RAPCC by their primary physician as per the National Cancer Control Programme guidelines [15]. The indications for referral according to these guidelines are an abnormal DRE or two abnormal PSA levels at 6-week intervals. An abnormal PSA is defined as a PSA [2.5 ng/ml if 40–49 years of age, [3.5 ng/ ml if 50–59 years of age, [4.5 ng/ml if 60–69 years of age and [6.5 ng/ml if 70–79 years of age. It is important to note that these age-related reference ranges for PSA have no clinical validity in that disease-free values are not found within the quoted ranges [16]. The patient population was analysed histologically for a positive PCa diagnosis following 12-core biopsy, and were subdivided according to Gleason grade, as defined by the International Society of Urological Pathology (ISUP) Consensus Conference [17].
Statistical analysis Basic statistical analysis of the study population’s characteristics was performed with the unpaired Student’s t test and the Wilcoxon Rank Sum test for continuous variables, while Pearson’s Chi-squared test was performed for categorical variables. Using the formula for the PCPT (v2.0) and the ERSPC (step 3?DRE and 4?DRE), which are available online [8, 18], each patient’s individual risk for PCa was calculated. To ascertain the value of DRE-based prostate volume for the ERSPC, each patient’s TRUS-estimated volume was converted into their respective categories, 25, 40 or 60 cm3. The prostate volume estimation for the ERSPC calculator is split into these three categories and allows physicians to potentially avail of this risk factor without having to send a patient for TRUS before biopsy. The risk for each calculator was then correlated with the outcome of prostate biopsy. Calibration plots were created which demonstrate the agreement between predicted probabilities of each calculator and the observed incidence of PCa. Significance values for goodness of fit were computed using the Hosmer–Lemeshow test. Receiver-operating characteristic analysis (sensitivity vs specificity plot) was utilized to demonstrate the discriminating ability of each calculator. Comparison of ROC curves took place via
Ir J Med Sci
difference in performance between the two calculators for the prediction of PCa (p [ 0.05). Decision curve analysis (Fig. 3) was utilized to characterize the clinical efficacy of the models. A model is of most clinical value if it has the highest net benefit across the full range of threshold probabilities at which a patient would choose to be biopsied [21]. The straight black line at y = 0 represents the decision curve for the strategy of performing a biopsy on no patients and the gray line represents the decision curve for the strategy of performing a biopsy on all patients. The net benefit for the PCPT and ERSPC calculators were comparable. Both calculators demonstrated a superior net benefit over the strategy of performing a biopsy in no patients and are superior to performing biopsy in all patients for a risk exceeding 36 and 38 %, respectively. Therefore, these calculators can be used by the clinician to facilitate the decision to proceed to prostate biopsy. The decision to biopsy a patient is supported if that individual’s risk is associated with a clinical net benefit, that is, a true result in excess of false results.
the DeLong method [19]. Decision curves, which plot the net benefit of each calculator compared to the net benefit of a strategy of performing biopsy on all patients or none, were formed. Net benefit is calculated by adding the ‘true positives’ and ‘true negatives’ and subtracting the ‘false positives’ and ‘false negatives’ for each risk percentage calculated. All statistical analysis was performed in the R statistics software package [20].
Results Of 337 patients undergoing TRUS biopsy, PCa was subsequently diagnosed in 146 men (43 %). Of these 146 cancer diagnoses, 98 (67 %) had high grade disease (CGleason 7). The characteristics of the study cohort are presented in Table 1. Figure 1 illustrates the results of calibration analysis. The PCPT calculator tended to underestimate a patient’s risk of PCa over its entire range of risk values, whereas the ERSPC calculator underestimated risk at the lower end of its predictions and overestimated PCa risk at the upper end. Both calculators were significantly miscalibrated according to the Hosmer–Lemeshow test (p \ 0.05). The discriminative ability of the PCPT and ERSPC to predict a positive cancer diagnosis was statistically significant (p \ 0.01) in this cohort (Fig. 2). The area under the receiver-operating characteristic curve (AUC) for the PCPT risk calculator was 0.681, while the AUC for the ERSPC risk calculator in predicting PCa was 0.662 (Fig. 1). Both calculators improved upon the AUC yielded by PSA alone (0.629), and this was statistically significant (p \ 0.01). However, there was so statistically significant
Table 1 Study cohort characteristics
Discussion A PSA of C4 (Hybritech calibration) has traditionally been used as a cut off for further investigation of possible PCa and in Ireland, a level of C3.5 is an indication for referral to a urologist for a specialist opinion. Results from PSA analytical calibration by Hybritech immunoassay technology are about 22 % lower than values from methods calibrated to WHO IRP 96/670 [16]. However, the use of PSA cut off values has been challenged with the current vogue in clinical urology seeking to abandon the use of
All
PCa
No PCa
p value
Patients
337
146 (43 %)
191 (57 %)
Mean age
62.89
64.89
61.32
\0.001*
Abnormal DRE
170
106
64
\0.001¥
Previous negative biopsy
77
31
46
0.5062¥
Family history of PCa
60
29
31
0.2548¥
Median PSA (range)
6 (0.1–1443)
7.03 (0.1–1443)
5.2 (0.2–65.8)
0.0004
No. Gleason score category Gleason 6
48
Gleason 7
50
Gleason 8
32
Gleason 9–10
16
PCa prostate cancer, DRE digital rectal exam * Student’s t test ¥
Chi-squared test
Wilcoxon rank sum test
123
Ir J Med Sci
Fig. 1 The relationship between the predicted probabilities of the PCPT risk calculator (a) and the ERSPC risk calculator (b) and the actual risk of prostate cancer. A perfect calculator is represented by the 45° line
Fig. 2 Receiver-operating characteristics curves demonstrating the performance of the PCPT and ERSPC risk calculators for a diagnosis of prostate cancer
Fig. 3 Decision curve analysis illustrating the clinical utility of the PCPT and ERSPC risk calculators in the diagnosis of prostate cancer. The area of the graph where the red (PCPT) and blue (ERSPC) line cross the black line corresponds to the risk values at which a net benefit is derived
these threshold values due to lack of specificity [16, 22]. Instead, it is important to take into account all of the potential risk factors for PCa before the decision to proceed to biopsy is made. The goal of PCa risk calculators is to allow for a logical approach to the use of these risk factors and to facilitate an informed decision. These calculators have the
123
potential to be introduced into clinical practice and have been shown to be effective, while also acceptable to both doctors and patients alike [23]. The application of risk calculators within PCa management is not limited to the decision for prostate biopsy. Step 6 of the ERSPC has been developed to estimate a patients risk of PCa within 4 years after initial PSA testing and can provide guidance as to whether a patient will need to be retested [24]. The PCPT and ERSPC risk calculators both demonstrated an acceptable calibration as evidence by their goodness of fit. The discriminative ability of the calculators (AUC values) was statistically significant in their prediction of PCa. Whilst the performance of the PCPT and ERSPC in the diagnosis of PCa was comparable on decision curve analysis and both were associated with a net benefit. These results support those of a recent study in a geographically distinct Irish cohort in the West of Ireland [25], in which PCPT was found to be more effective than the ERSPC. Therefore, these risk calculators have been validated in the Irish population and can be used in clinical settings to predict the outcome of a prostate biopsy. The PCPT risk calculator, based on these two validation studies, is currently the most suitable for use in an Irish population and can aid in the decision for prostate biopsy. Despite this study providing external validation for the PCPT and ERSPC calculators, it is important to mention that the AUC values in this Irish population are relatively low. Although the PCPT and ERSPC have an associated clinical net benefit in this cohort, they do not provide a benefit to the patient for risk values less than 36 %. This is a substantial shortcoming, as it is precisely these values in which a decision to proceed to TRUS biopsy of the prostate is most difficult. The relatively poor performance of these calculators highlights the difficulty in risk stratification of patients under investigation for PCa. Basic clinical parameters such as those utilized in the PCPT and ERSPC risk calculator allow for an analytical approach to the use of PCa risk factors; however, the correct strategy for
Ir J Med Sci
advising individual patients as to the hierarchy of clinical choices remains challenging. A strategy to improve upon the performance of the PCPT and ERSPC would be to use a risk calculator which has been built in the Irish population. Firstly, a large number of patients who have undergone biopsy, similar in number to those used to create the PCPT and ERSPC, i.e. [3000, must be collected and analyzed to create a sufficiently robust risk calculator. A risk calculator will perform best if it has a larger number of patients and their personal combination of risk factors from which to derive a new patient’s risk. The model will then have a greater chance of having ‘seen’ a new patient’s combination of risk factors before and consequently a greater chance of correctly predicting the new patient’s outcome. Secondly, the patients included in this new calculator must be collected from each of the 8 tertiary prostate cancer centers in Ireland to be most effective in predicting PCa on a national scale. Following the results of this study, the authors intend to build a risk calculator for the Irish population and this national risk stratification project is currently ongoing. There are a number of limitations to this study that we acknowledge, including the relatively small sample size. The possibility of a false negative biopsy result is an intrinsic limitation of this study. This risk has been reported in the literature as high as 24 % [26]. If we consider these false negative results, then flaws will exist in the creation of any risk assessment tool which stratifies patients according to the result of their prostate biopsy. Furthermore, patient risk stratification for PCa is also limited by potential variations in PSA levels from lab to lab secondary to a lack of PSA standardization [27]. This study has also assessed the performance of a calculator (the ERSPC), which includes a DRE-based volume estimate of the prostate. However, no standardized approach to volume estimation on DRE exists amongst clinicians. It is likely that a prospective study in which prostate volume on DRE was standardized amongst the participating urologists could increase the clinical applicability of this measurement as a risk factor.
Conclusion This study provides external validation of the PCPT and ERSPC risk calculators in a distinct population in the East of Ireland and these tools may be used to predict PCa in the Irish male. However, it is likely that more accurate risk stratification can be achieved by the use of a risk calculator that is more applicable to the Irish population. The creation of a novel calculator based on Irish men under the investigation for prostate cancer will allow for better calibration and may also allow for the discriminating
ability and clinical utility of these calculators to be improved upon. Acknowledgments RWG Watson and D. J. Lundon received funding from the Prostate Cancer Research Consortium (PCRC) under the Irish Cancer Society, the Urology Foundation and the Irish Research Council. RW Foley received funding through the UCD School of Medicine Intercalated MSc. Medical Science Scholarship Programme. Conflict of interest
None.
References 1. National Cancer Registry (2014) Cancer in Ireland 1994–2011: annual report of the National Cancer Registry 2014. http://www. ncri.ie/sites/ncri/files/pubs/annualreport2014.pdf. Accessed 2 Jul 2014 2. National Cancer Registry (2010) Cancer trends. No. 3. Recent trends in prostate cancer 3. Loeb S, Vellekoop A, Ahmed HU, Catto J, Emberton M, Nam R, Rosario DJ, Scattoni V, Lotan Y (2013) Systematic review of complications of prostate biopsy. Eur Urol 64:876–892 4. National Institute for Health and Care Excellence (2014) Prostate cancer: diagnosis and treatment, 1-recommendations. [CG175]. http://www.nice.org.uk/guidance/cg175. Accessed 9 Mar 2015 5. Thompson IM, Ankerst DP, Chi C, Goodman PJ, Tangen CM, Lucia MS, Feng Z, Parnes HL, Coltman CA (2006) Assessing prostate cancer risk: results from the Prostate Cancer Prevention Trial. J Natl Cancer Inst 98:529–534 6. Kranse R, Roobol M, Schro¨der FH (2008) A graphical device to represent the outcomes of a logistic regression analysis. Prostate 68:1674–1680 7. Heidenreich A, Abrahamsson P-A, Artibani W, Catto J, Montorsi F, Van Poppel H, Wirth M, Mottet N (2013) Early detection of prostate cancer: European Association of Urology recommendation. Eur Urol 64:347–354 8. SWOP (2014) SWOP: prostate cancer risk calculator (based on ERSPC). http://www.prostatecancer-riskcalculator.com/. Accessed 3 Jul 2014 9. Roobol MJ, Schro¨der FH, Hugosson J et al (2012) Importance of prostate volume in the European Randomised Study of Screening for Prostate Cancer (ERSPC) risk calculators: results from the prostate biopsy collaborative group. World J Urol 30:149–155 10. Zhu Y, Wang J-Y, Shen Y-J, Dai B, Ma C-G, Xiao W-J, Lin G-W, Yao X-D, Zhang S-L, Ye D-W (2012) External validation of the Prostate Cancer Prevention Trial and the European Randomized Study of Screening For Prostate Cancer risk calculators in a Chinese cohort. Asian J Androl 14:738–744 11. Trottier G, Roobol MJ, Lawrentschuk N et al (2011) Comparison of risk calculators from the Prostate Cancer Prevention Trial and the European Randomized Study of Screening For Prostate Cancer in a contemporary Canadian cohort. BJU Int 108:E237– E244 12. Van Vugt HA, Roobol MJ, Kranse R, Ma¨a¨tta¨nen L, Finne P, Hugosson J, Bangma CH, Schro¨der FH, Steyerberg EW (2011) Prediction of prostate cancer in unscreened men: external validation of a risk calculator. Eur J Cancer 47:903–909 13. Cavadas V, Oso´rio L, Sabell F, Teves F, Branco F, Silva-Ramos M (2010) Prostate cancer prevention trial and European randomized study of screening for prostate cancer risk calculators: a performance comparison in a contemporary screened cohort. Eur Urol 58:551–558
123
Ir J Med Sci 14. Ilic D, Neuberger MM, Djulbegovic M, Dahm P (2013) Screening for prostate cancer. Cochrane database Syst Rev 1:4720 15. Irish National Cancer Control Programme (2011) NCCP Prostate cancer referral guideline. http://www.healthlink.ie/Oncology/ NCCP. Prostate cancer referral guideline version 1.3 Jan 2011.pdf 16. Tormey WP (2014) The complexity of PSA interpretation in clinical practice. Surgeon 12:323–327 17. Epstein JI, Allsbrook WC, Amin MB, Egevad LL (2005) The 2005 International Society of Urological Pathology (ISUP) Consensus Conference on Gleason Grading of Prostatic Carcinoma. Am J Surg Pathol 29:1228–1242 18. PCPT (2014) Individualized risk assessment of prostate cancer (based on PCPT). http://deb.uthscsa.edu/URORiskCalc/Pages/ uroriskcalc.jsp. Accessed 3 Jul 2014 19. DeLong ER, DeLong DM, Clarke-Pearson DL (1988) Comparing the areas under two or more correlated receiver operating characteristic curves: a nonparametric approach. Biometrics 44:837–845 20. Robin X, Turck N, Hainard A, Tiberti N, Lisacek F, Sanchez J-C, Mu¨ller M (2011) pROC: an open-source package for R and S? to analyze and compare ROC curves. BMC Bioinform 12:77 21. Vickers A, Cronin A, Roobol M, Savage C, Peltola M, Pettersson K, Scardino PT, Schro¨der F, Lilja H (2010) Reducing unnecessary biopsy during prostate cancer screening using a four-kallikrein panel: an independent replication. J Clin Oncol 28:2493–2498
123
22. Thompson IM, Pauler DK, Goodman PJ et al (2004) Prevalence of prostate cancer among men with a prostate-specific antigen level B4.0 ng per milliliter. N Engl J Med 350:2239–2246 23. Van Vugt HA, Kranse R, Steyerberg EW, van der Poel HG, Busstra M, Kil P, Oomens EH, de Jong IJ, Bangma CH, Roobol MJ (2012) Prospective validation of a risk calculator which calculates the probability of a positive prostate biopsy in a contemporary clinical cohort. Eur J Cancer 48:1809–1815 24. Roobol MJ, Zhu X, Schro¨der FH, van Leenders GJLH, van Schaik RH, Bangma CH, Steyerberg EW (2013) A calculator for prostate cancer risk 4 years after an initially negative screen: findings from ERSPC Rotterdam. Eur Urol 63:627–633 25. Lundon DJ, Kelly BD, Foley R, Loeb S, Fitzpatrick JM, Watson RWG, Rogers E, Durkan GC, Walsh K (2014) Prostate cancer risk assessment tools in an unscreened population. World J Urol. doi:10.1007/s00345-014-1365-7 26. Roemeling S, Schro¨der FH (2008) Words of wisdom. Re: needle biopsies on autopsy prostates: sensitivity of cancer detection based on true prevalence. Eur Urol 53:663–664 27. Forde JC, Marignol L, Blake O, McDermott T, Grainger R, Crowley VE, Lynch TH (2012) Standardization of assay methods reduces variability of total PSA measurements: an Irish study. BJU Int 110:644–650
ORIGINAL ARTICLE
Predicting prostate cancer: analysing the clinical efficacy of prostate cancer risk calculators in a referral population R. W. Foley1,2,3 • D. J. Lundon1,2,3 • K. Murphy4 • T. B. Murphy4,5 • D. J. Galvin1,3 • R. W. G. Watson1,2
Received: 12 February 2015 / Accepted: 29 March 2015 Ó Royal Academy of Medicine in Ireland 2015
Abstract Background The decision to proceed to biopsy for the diagnosis of prostate cancer in clinical practice is a difficult one. Prostate cancer risk calculators allow for a systematic approach to the use of patient information to predict a patient’s likelihood of prostate cancer. Aims In this paper, we validate the two leading prostate cancer risk calculators, the prostate cancer prevention trial (PCPT) and the European Randomized Study of Screening for Prostate Cancer (ERSPC) in an Irish population. Methods Data were collected for 337 men referred to one tertiary referral center in Ireland. Calibration analysis, ROC analysis and decision curve analysis were undertaken to ascertain the performance of the PCPT and the ERSPC risk calculators in this cohort. Results Of 337 consecutive biopsies, cancer was subsequently diagnosed in 146 men (43 %), 98 (67 %) of which were high grade. The AUC for the PCPT and ERSPC risk calculators were 0.68 and 0.66, respectively for the prediction of prostate cancer. Each calculator was sufficiently
& R. W. Foley [email protected] 1
UCD School of Medicine and Medical Science, University College Dublin, Belfield, Dublin 4, Ireland
2
UCD Conway Institute of Biomolecular and Biomedical Research, University College Dublin, Belfield, Dublin 4, Ireland
3
Department of Urology, Mater Misericordiae University Hospital, Eccles Street, Dublin 7, Ireland
4
UCD School of Mathematical Sciences, University College Dublin, Dublin 4, Ireland
5
Insight Centre for Data Analytics, University College Dublin, Dublin 4, Ireland
calibrated in this cohort. Decision curve analysis demonstrated a net benefit via the use of the PCPT and ERSPC risk calculators in the diagnosis of prostate cancer. Conclusions The PCPT and ERSPC risk calculators achieve a statistically significant prediction of prostate cancer in this Irish population. This study provides external validation for these calculators, and therefore these tools can be used to aid in clinical decision making. Keywords Biopsy Decision support techniques Prostatic neoplasm Risk
Introduction Prostate cancer (PCa) is the most common non-cutaneous cancer in men in Ireland [1]. This disease causes significant morbidity and mortality; every year, over 500 men in Ireland die from this disease [2]. The current standard of diagnosis for PCa is a transrectal ultrasound (TRUS) prostate biopsy. This procedure carries with it significant risks [3], including rare cases of biopsy-related mortality. It is important to assess the possible benefit of such a procedure before exposing a patient to these risks. The shared decision to proceed to biopsy of the prostate is based on the patient history, digital rectal examination (DRE) and a serum prostate-specific antigen (PSA) level following informative discussion with the patient and partners or carers. The NICE guidelines put this in context and the recommendations are echoed from the US College of Physicians [4]. Tools exist to provide a prediction of the likely outcome of a biopsy based on these and other weighted risk factors. The most thoroughly validated calculators to determine the risk of PCa on biopsy are the prostate cancer prevention trial
123
Ir J Med Sci
(PCPT) [5] and the European Randomized Study of Screening for Prostate Cancer (ERSPC) risk calculators [6]. These calculators are becoming increasingly important in clinical practice for the management of PCa, with recent European Association of Urology guidelines advocating the integration of a multivariate approach into PCa diagnosis [7]. The calculators were created in two distinct patient populations and each incorporates their own unique set of clinical parameters. The parameters included in the PCPT risk calculator are age, race, abnormality on DRE, family history of PCa, a previous negative biopsy and PSA level. The performance of the PCPT calculator in the participants with whom the calculator was developed gave an AUC value of 0.702 [5]. The ERSPC risk calculators consist of 6 steps and are available online [8]. Each of these risk calculators is tailored to a specific type of patient and calculates a man’s individual risk of PCa. Step 3?DRE which consists of PSA level, abnormality on DRE, DRE-based prostate volume estimate and step 4?DRE which consists of PSA level, abnormality on DRE, DRE-based prostate volume estimate and previous negative prostate biopsy were utilised in this study [9]. Step 3?DRE is applicable to men undergoing initial prostate biopsy, whilst step 4?DRE is applicable to men at repeat biopsy. The ERSPC steps 3?DRE and 4?DRE risk calculators were selected to allow for a meaningful comparison with the PCPT risk calculator. The steps 3?DRE and 4?DRE calculators incorporate DRE-based prostate volume estimation and not TRUS prostate volume. Because TRUS is not routinely performed prebiopsy, these steps of the ERSPC calculator are more readily applicable to the current Irish clinical setting. The performance of the ERSPC risk calculators have been validated in a number of cohorts, both North American and European, and have yielded AUC values between 0.70 and 0.80 [10–13]. The PCPT and ERSPC risk calculators were created from large-scale screening trials for PCa. A recent metaanalysis by the cochrane collaboration has found no benefit in PCa screening, and noted frequent harms associated with subsequent investigations, i.e., prostate biopsy [14]. Therefore, the decision on which patients to send for biopsy is a difficult one. Risk calculators for PCa prediction are important to aid with this decision. Before the possible introduction into clinical practice, these tools must be validated. As advocated by Trottier et al. [11], the disparity in performance of these calculators in different geographical regions ‘‘emphasizes the importance of nomogram validation in populations from different countries’’. This study aims to validate these risk calculators in a cohort of Irish men.
123
Materials and methods Patient population The patient population consisted of men referred to the Mater Misericordiae rapid access prostate cancer clinic (RAPCC) with an East of Ireland based catchment area. Ethical approval was awarded from the institution. We retrospectively collected the relevant information as described by the PCPT risk calculator and the ERSPC risk calculator in patients undergoing a TRUS biopsy. These men were referred to the RAPCC by their primary physician as per the National Cancer Control Programme guidelines [15]. The indications for referral according to these guidelines are an abnormal DRE or two abnormal PSA levels at 6-week intervals. An abnormal PSA is defined as a PSA [2.5 ng/ml if 40–49 years of age, [3.5 ng/ ml if 50–59 years of age, [4.5 ng/ml if 60–69 years of age and [6.5 ng/ml if 70–79 years of age. It is important to note that these age-related reference ranges for PSA have no clinical validity in that disease-free values are not found within the quoted ranges [16]. The patient population was analysed histologically for a positive PCa diagnosis following 12-core biopsy, and were subdivided according to Gleason grade, as defined by the International Society of Urological Pathology (ISUP) Consensus Conference [17].
Statistical analysis Basic statistical analysis of the study population’s characteristics was performed with the unpaired Student’s t test and the Wilcoxon Rank Sum test for continuous variables, while Pearson’s Chi-squared test was performed for categorical variables. Using the formula for the PCPT (v2.0) and the ERSPC (step 3?DRE and 4?DRE), which are available online [8, 18], each patient’s individual risk for PCa was calculated. To ascertain the value of DRE-based prostate volume for the ERSPC, each patient’s TRUS-estimated volume was converted into their respective categories, 25, 40 or 60 cm3. The prostate volume estimation for the ERSPC calculator is split into these three categories and allows physicians to potentially avail of this risk factor without having to send a patient for TRUS before biopsy. The risk for each calculator was then correlated with the outcome of prostate biopsy. Calibration plots were created which demonstrate the agreement between predicted probabilities of each calculator and the observed incidence of PCa. Significance values for goodness of fit were computed using the Hosmer–Lemeshow test. Receiver-operating characteristic analysis (sensitivity vs specificity plot) was utilized to demonstrate the discriminating ability of each calculator. Comparison of ROC curves took place via
Ir J Med Sci
difference in performance between the two calculators for the prediction of PCa (p [ 0.05). Decision curve analysis (Fig. 3) was utilized to characterize the clinical efficacy of the models. A model is of most clinical value if it has the highest net benefit across the full range of threshold probabilities at which a patient would choose to be biopsied [21]. The straight black line at y = 0 represents the decision curve for the strategy of performing a biopsy on no patients and the gray line represents the decision curve for the strategy of performing a biopsy on all patients. The net benefit for the PCPT and ERSPC calculators were comparable. Both calculators demonstrated a superior net benefit over the strategy of performing a biopsy in no patients and are superior to performing biopsy in all patients for a risk exceeding 36 and 38 %, respectively. Therefore, these calculators can be used by the clinician to facilitate the decision to proceed to prostate biopsy. The decision to biopsy a patient is supported if that individual’s risk is associated with a clinical net benefit, that is, a true result in excess of false results.
the DeLong method [19]. Decision curves, which plot the net benefit of each calculator compared to the net benefit of a strategy of performing biopsy on all patients or none, were formed. Net benefit is calculated by adding the ‘true positives’ and ‘true negatives’ and subtracting the ‘false positives’ and ‘false negatives’ for each risk percentage calculated. All statistical analysis was performed in the R statistics software package [20].
Results Of 337 patients undergoing TRUS biopsy, PCa was subsequently diagnosed in 146 men (43 %). Of these 146 cancer diagnoses, 98 (67 %) had high grade disease (CGleason 7). The characteristics of the study cohort are presented in Table 1. Figure 1 illustrates the results of calibration analysis. The PCPT calculator tended to underestimate a patient’s risk of PCa over its entire range of risk values, whereas the ERSPC calculator underestimated risk at the lower end of its predictions and overestimated PCa risk at the upper end. Both calculators were significantly miscalibrated according to the Hosmer–Lemeshow test (p \ 0.05). The discriminative ability of the PCPT and ERSPC to predict a positive cancer diagnosis was statistically significant (p \ 0.01) in this cohort (Fig. 2). The area under the receiver-operating characteristic curve (AUC) for the PCPT risk calculator was 0.681, while the AUC for the ERSPC risk calculator in predicting PCa was 0.662 (Fig. 1). Both calculators improved upon the AUC yielded by PSA alone (0.629), and this was statistically significant (p \ 0.01). However, there was so statistically significant
Table 1 Study cohort characteristics
Discussion A PSA of C4 (Hybritech calibration) has traditionally been used as a cut off for further investigation of possible PCa and in Ireland, a level of C3.5 is an indication for referral to a urologist for a specialist opinion. Results from PSA analytical calibration by Hybritech immunoassay technology are about 22 % lower than values from methods calibrated to WHO IRP 96/670 [16]. However, the use of PSA cut off values has been challenged with the current vogue in clinical urology seeking to abandon the use of
All
PCa
No PCa
p value
Patients
337
146 (43 %)
191 (57 %)
Mean age
62.89
64.89
61.32
\0.001*
Abnormal DRE
170
106
64
\0.001¥
Previous negative biopsy
77
31
46
0.5062¥
Family history of PCa
60
29
31
0.2548¥
Median PSA (range)
6 (0.1–1443)
7.03 (0.1–1443)
5.2 (0.2–65.8)
0.0004
No. Gleason score category Gleason 6
48
Gleason 7
50
Gleason 8
32
Gleason 9–10
16
PCa prostate cancer, DRE digital rectal exam * Student’s t test ¥
Chi-squared test
Wilcoxon rank sum test
123
Ir J Med Sci
Fig. 1 The relationship between the predicted probabilities of the PCPT risk calculator (a) and the ERSPC risk calculator (b) and the actual risk of prostate cancer. A perfect calculator is represented by the 45° line
Fig. 2 Receiver-operating characteristics curves demonstrating the performance of the PCPT and ERSPC risk calculators for a diagnosis of prostate cancer
Fig. 3 Decision curve analysis illustrating the clinical utility of the PCPT and ERSPC risk calculators in the diagnosis of prostate cancer. The area of the graph where the red (PCPT) and blue (ERSPC) line cross the black line corresponds to the risk values at which a net benefit is derived
these threshold values due to lack of specificity [16, 22]. Instead, it is important to take into account all of the potential risk factors for PCa before the decision to proceed to biopsy is made. The goal of PCa risk calculators is to allow for a logical approach to the use of these risk factors and to facilitate an informed decision. These calculators have the
123
potential to be introduced into clinical practice and have been shown to be effective, while also acceptable to both doctors and patients alike [23]. The application of risk calculators within PCa management is not limited to the decision for prostate biopsy. Step 6 of the ERSPC has been developed to estimate a patients risk of PCa within 4 years after initial PSA testing and can provide guidance as to whether a patient will need to be retested [24]. The PCPT and ERSPC risk calculators both demonstrated an acceptable calibration as evidence by their goodness of fit. The discriminative ability of the calculators (AUC values) was statistically significant in their prediction of PCa. Whilst the performance of the PCPT and ERSPC in the diagnosis of PCa was comparable on decision curve analysis and both were associated with a net benefit. These results support those of a recent study in a geographically distinct Irish cohort in the West of Ireland [25], in which PCPT was found to be more effective than the ERSPC. Therefore, these risk calculators have been validated in the Irish population and can be used in clinical settings to predict the outcome of a prostate biopsy. The PCPT risk calculator, based on these two validation studies, is currently the most suitable for use in an Irish population and can aid in the decision for prostate biopsy. Despite this study providing external validation for the PCPT and ERSPC calculators, it is important to mention that the AUC values in this Irish population are relatively low. Although the PCPT and ERSPC have an associated clinical net benefit in this cohort, they do not provide a benefit to the patient for risk values less than 36 %. This is a substantial shortcoming, as it is precisely these values in which a decision to proceed to TRUS biopsy of the prostate is most difficult. The relatively poor performance of these calculators highlights the difficulty in risk stratification of patients under investigation for PCa. Basic clinical parameters such as those utilized in the PCPT and ERSPC risk calculator allow for an analytical approach to the use of PCa risk factors; however, the correct strategy for
Ir J Med Sci
advising individual patients as to the hierarchy of clinical choices remains challenging. A strategy to improve upon the performance of the PCPT and ERSPC would be to use a risk calculator which has been built in the Irish population. Firstly, a large number of patients who have undergone biopsy, similar in number to those used to create the PCPT and ERSPC, i.e. [3000, must be collected and analyzed to create a sufficiently robust risk calculator. A risk calculator will perform best if it has a larger number of patients and their personal combination of risk factors from which to derive a new patient’s risk. The model will then have a greater chance of having ‘seen’ a new patient’s combination of risk factors before and consequently a greater chance of correctly predicting the new patient’s outcome. Secondly, the patients included in this new calculator must be collected from each of the 8 tertiary prostate cancer centers in Ireland to be most effective in predicting PCa on a national scale. Following the results of this study, the authors intend to build a risk calculator for the Irish population and this national risk stratification project is currently ongoing. There are a number of limitations to this study that we acknowledge, including the relatively small sample size. The possibility of a false negative biopsy result is an intrinsic limitation of this study. This risk has been reported in the literature as high as 24 % [26]. If we consider these false negative results, then flaws will exist in the creation of any risk assessment tool which stratifies patients according to the result of their prostate biopsy. Furthermore, patient risk stratification for PCa is also limited by potential variations in PSA levels from lab to lab secondary to a lack of PSA standardization [27]. This study has also assessed the performance of a calculator (the ERSPC), which includes a DRE-based volume estimate of the prostate. However, no standardized approach to volume estimation on DRE exists amongst clinicians. It is likely that a prospective study in which prostate volume on DRE was standardized amongst the participating urologists could increase the clinical applicability of this measurement as a risk factor.
Conclusion This study provides external validation of the PCPT and ERSPC risk calculators in a distinct population in the East of Ireland and these tools may be used to predict PCa in the Irish male. However, it is likely that more accurate risk stratification can be achieved by the use of a risk calculator that is more applicable to the Irish population. The creation of a novel calculator based on Irish men under the investigation for prostate cancer will allow for better calibration and may also allow for the discriminating
ability and clinical utility of these calculators to be improved upon. Acknowledgments RWG Watson and D. J. Lundon received funding from the Prostate Cancer Research Consortium (PCRC) under the Irish Cancer Society, the Urology Foundation and the Irish Research Council. RW Foley received funding through the UCD School of Medicine Intercalated MSc. Medical Science Scholarship Programme. Conflict of interest
None.
References 1. National Cancer Registry (2014) Cancer in Ireland 1994–2011: annual report of the National Cancer Registry 2014. http://www. ncri.ie/sites/ncri/files/pubs/annualreport2014.pdf. Accessed 2 Jul 2014 2. National Cancer Registry (2010) Cancer trends. No. 3. Recent trends in prostate cancer 3. Loeb S, Vellekoop A, Ahmed HU, Catto J, Emberton M, Nam R, Rosario DJ, Scattoni V, Lotan Y (2013) Systematic review of complications of prostate biopsy. Eur Urol 64:876–892 4. National Institute for Health and Care Excellence (2014) Prostate cancer: diagnosis and treatment, 1-recommendations. [CG175]. http://www.nice.org.uk/guidance/cg175. Accessed 9 Mar 2015 5. Thompson IM, Ankerst DP, Chi C, Goodman PJ, Tangen CM, Lucia MS, Feng Z, Parnes HL, Coltman CA (2006) Assessing prostate cancer risk: results from the Prostate Cancer Prevention Trial. J Natl Cancer Inst 98:529–534 6. Kranse R, Roobol M, Schro¨der FH (2008) A graphical device to represent the outcomes of a logistic regression analysis. Prostate 68:1674–1680 7. Heidenreich A, Abrahamsson P-A, Artibani W, Catto J, Montorsi F, Van Poppel H, Wirth M, Mottet N (2013) Early detection of prostate cancer: European Association of Urology recommendation. Eur Urol 64:347–354 8. SWOP (2014) SWOP: prostate cancer risk calculator (based on ERSPC). http://www.prostatecancer-riskcalculator.com/. Accessed 3 Jul 2014 9. Roobol MJ, Schro¨der FH, Hugosson J et al (2012) Importance of prostate volume in the European Randomised Study of Screening for Prostate Cancer (ERSPC) risk calculators: results from the prostate biopsy collaborative group. World J Urol 30:149–155 10. Zhu Y, Wang J-Y, Shen Y-J, Dai B, Ma C-G, Xiao W-J, Lin G-W, Yao X-D, Zhang S-L, Ye D-W (2012) External validation of the Prostate Cancer Prevention Trial and the European Randomized Study of Screening For Prostate Cancer risk calculators in a Chinese cohort. Asian J Androl 14:738–744 11. Trottier G, Roobol MJ, Lawrentschuk N et al (2011) Comparison of risk calculators from the Prostate Cancer Prevention Trial and the European Randomized Study of Screening For Prostate Cancer in a contemporary Canadian cohort. BJU Int 108:E237– E244 12. Van Vugt HA, Roobol MJ, Kranse R, Ma¨a¨tta¨nen L, Finne P, Hugosson J, Bangma CH, Schro¨der FH, Steyerberg EW (2011) Prediction of prostate cancer in unscreened men: external validation of a risk calculator. Eur J Cancer 47:903–909 13. Cavadas V, Oso´rio L, Sabell F, Teves F, Branco F, Silva-Ramos M (2010) Prostate cancer prevention trial and European randomized study of screening for prostate cancer risk calculators: a performance comparison in a contemporary screened cohort. Eur Urol 58:551–558
123
Ir J Med Sci 14. Ilic D, Neuberger MM, Djulbegovic M, Dahm P (2013) Screening for prostate cancer. Cochrane database Syst Rev 1:4720 15. Irish National Cancer Control Programme (2011) NCCP Prostate cancer referral guideline. http://www.healthlink.ie/Oncology/ NCCP. Prostate cancer referral guideline version 1.3 Jan 2011.pdf 16. Tormey WP (2014) The complexity of PSA interpretation in clinical practice. Surgeon 12:323–327 17. Epstein JI, Allsbrook WC, Amin MB, Egevad LL (2005) The 2005 International Society of Urological Pathology (ISUP) Consensus Conference on Gleason Grading of Prostatic Carcinoma. Am J Surg Pathol 29:1228–1242 18. PCPT (2014) Individualized risk assessment of prostate cancer (based on PCPT). http://deb.uthscsa.edu/URORiskCalc/Pages/ uroriskcalc.jsp. Accessed 3 Jul 2014 19. DeLong ER, DeLong DM, Clarke-Pearson DL (1988) Comparing the areas under two or more correlated receiver operating characteristic curves: a nonparametric approach. Biometrics 44:837–845 20. Robin X, Turck N, Hainard A, Tiberti N, Lisacek F, Sanchez J-C, Mu¨ller M (2011) pROC: an open-source package for R and S? to analyze and compare ROC curves. BMC Bioinform 12:77 21. Vickers A, Cronin A, Roobol M, Savage C, Peltola M, Pettersson K, Scardino PT, Schro¨der F, Lilja H (2010) Reducing unnecessary biopsy during prostate cancer screening using a four-kallikrein panel: an independent replication. J Clin Oncol 28:2493–2498
123
22. Thompson IM, Pauler DK, Goodman PJ et al (2004) Prevalence of prostate cancer among men with a prostate-specific antigen level B4.0 ng per milliliter. N Engl J Med 350:2239–2246 23. Van Vugt HA, Kranse R, Steyerberg EW, van der Poel HG, Busstra M, Kil P, Oomens EH, de Jong IJ, Bangma CH, Roobol MJ (2012) Prospective validation of a risk calculator which calculates the probability of a positive prostate biopsy in a contemporary clinical cohort. Eur J Cancer 48:1809–1815 24. Roobol MJ, Zhu X, Schro¨der FH, van Leenders GJLH, van Schaik RH, Bangma CH, Steyerberg EW (2013) A calculator for prostate cancer risk 4 years after an initially negative screen: findings from ERSPC Rotterdam. Eur Urol 63:627–633 25. Lundon DJ, Kelly BD, Foley R, Loeb S, Fitzpatrick JM, Watson RWG, Rogers E, Durkan GC, Walsh K (2014) Prostate cancer risk assessment tools in an unscreened population. World J Urol. doi:10.1007/s00345-014-1365-7 26. Roemeling S, Schro¨der FH (2008) Words of wisdom. Re: needle biopsies on autopsy prostates: sensitivity of cancer detection based on true prevalence. Eur Urol 53:663–664 27. Forde JC, Marignol L, Blake O, McDermott T, Grainger R, Crowley VE, Lynch TH (2012) Standardization of assay methods reduces variability of total PSA measurements: an Irish study. BJU Int 110:644–650
