LETTERS TO THE EDITOR
European Journal of Cardio-Thoracic Surgery 49 (2016) 357–360
Antioxidants and K+ channel agonists versus hydrogen therapy during 1 ex vivo lung perfusion †
supplementing Steen solution™ with a combination of antioxidants and K+ channel agonists based on the above-discussed principles.
Mohamed S.A. Mohamed* Thoracic Transplantation Department, University Clinic Essen, Essen, Germany
REFERENCES
Received 17 April 2015; accepted 29 April 2015
[1] Haam S, Lee S, Paik HC, Park MS, Song JH, Lim BJ et al. The effects of hydrogen gas inhalation during ex vivo lung perfusion on donor lungs obtained after cardiac death. Eur J Cardiothorac Surg 2015;48:542–7. [2] Noda K, Shigemura N, Tanaka Y, Bhama J, D’Cunha J, Kobayashi H et al. Hydrogen preconditioning during ex vivo lung perfusion improves the quality of lung grafts in rats. Transplantation 2014;98:499–506. [3] Perry MM, Hui CK, Whiteman M et al. Hydrogen sulfide inhibits proliferation and release of IL-8 from human airway smooth muscle cells. Am J Respir Cell Mol Biol 2011;45:746–52. [4] George TJ, Arnaoutakis GJ, Beaty CA et al. Hydrogen sulfide decreases reactive oxygen in a model of lung transplantation. J Surg Res 2012;178:494–501. [5] Chatterjee S, Nieman GF, Christie JD, Fisher AB. Shear stress-related mechanosignaling with lung ischemia: lessons from basic research can inform lung transplantation. Am J Physiol Lung Cell Mol Physiol 2014;307:L668–80. [6] Barbagallo I, Marrazzo G, Frigiola A, Zappala A, Li Volti G. Role of carbon monoxide in vascular diseases. Curr Pharm Biotechnol 2012;13:787–96.
Several studies have confirmed the role of hydrogen to scavenge reactive oxygen species (ROSs) and to protect against ischaemic reperfusion injury (IRI) in the lung and the heart [1–4]. In a study, where rabbit lungs were exposed to ex vivo lung perfusion (EVLP), the infusion of H2S significantly reduced ROS activity within the grafts [4]. This resulted in a significant decrease of lipid peroxidation, epithelial apoptosis, airway smooth muscle cell (ASMC) proliferation and (interleukin) IL8 production [3, 4]. On the other hand, graft physiological parameters were improved in response to 2% hydrogen inhalation, while the graft is exposed to EVLP [1, 2]. During graft ischaemia and cold preservation, Na+/K+ adenosine 5’-triphosphatase is inhibited, leading to increased Na+ intracellular influx and K+ extracellular efflux. The inhibited activity of K+ channels leads to the failure to re-establish cell membrane hyperpolarization, leading to sustained cell membrane depolarization, impaired mitochondrial activity, enhanced activity of nicotinamide adenine dinucleotide phosphate hydrogen and Xanthine oxidases, leading to a significant increase in ROS production [5]. ROS stimulate ASMC proliferation, which may contribute to the development of pulmonary hypertension, and prime graft inflammasomes. The drop of the intracellular K+ levels activates the primed inflammasomes to activate caspase 1, which activates pro-IL1β and pro-IL18. While IL1β stimulates ASMC proliferation, both cytokines induce IL6. Through ROS scavenging by hydrogen, inflammasomes activation is attenuated, levels of IL1β, IL6 and IL8 are decreased, the graft oxygenation capacity and vascular resistance are improved [1, 2]. However, in the experiments of George et al. [4] the graft oxygenation capacity and vascular resistance did not differ between the hydrogen-treated and the non-hydrogen-treated groups. This apparent contradiction between the results of hydrogen therapy might be related to the administration of Na+ (as a bolus and infusion) during the EVLP model of George et al. [4], where donor blood was used as a perfusate. This might have potentiated the Na+ influx and the subsequent sustained membrane depolarization. On the contrary, the experiments conducted by Haam et al. [1] utilized a model of EVLP that follows the Toronto technique, using Perfadex and Steen solutions that contain physiological Na+ and K+ levels. This may highlight that the reconditioning of the lung graft is a complex procedure that requires multiple effectors that target various aspects simultaneously, where hydrogen therapy targets the reduction of ROS to eliminate their hazardous effects, and the effective EVLP technique improves other graft functional parameters. Nevertheless, hydrogen protects the lung graft against IRI as a potent antioxidant and through the up-regulation of heme-oxygenase-1 (HO-1) [1, 2]. HO-1 catalyzes carbon monoxide production, which is a potent antioxidant, activates guanylyl cyclase (which may explain the trend of increased cGMP observed by George et al.), and activates ATP-sensitive K+ channels and big conductance calcium-sensitive K+ channels [6]. The improved activity of K+ channels has been shown to recondition the graft, antagonize cell membrane depolarization, antagonize the opening of mitochondrial permeability transition pore during reperfusion, decrease ROS production and attenuate the inflammatory cytokine production. All of these would protect the graft against IRI and graft dysfunction [5]. Accordingly, a combination of antioxidants and K+ channel agonists may achieve at least a similar level of protection and may exhibit more significantly protective effects than those of the hydrogen therapy (future studies should be conducted to investigate this hypothesis). While hydrogen inhalation is applied together with an effective EVLP, in order to provide a ‘multitarget’ graft reconditioning procedure, a recent, theoretically described, EVLP protocol (Shehata protocol) introduced the notion of
* Corresponding author. Thoracic Transplantation Department, University Clinic Essen, Hufeland Straße 55, 45147 Essen, Germany. Tel: +49-201-7233779; fax: +49-201-7235471; e-mail: [email protected] (M.S.A. Mohamed). doi:10.1093/ejcts/ezv192 Advance Access publication 25 May 2015 †
The corresponding author of the original article [1] was invited to reply, but did not respond.
Knowingly repeating an incorrect and inefficient analysis is flawed logic Gary S. Collinsa,* and Yannick Le Manachb a Centre for Statistics in Medicine, Botnar Research Centre, University of Oxford, Oxford, UK b Departments of Anesthesia and Clinical Epidemiology and Biostatistics, Michael DeGroote School of Medicine, Faculty of Health Sciences, McMaster University and the Perioperative Research Group, Population Health Research Institute, Hamilton, Canada
Received 27 March 2015; accepted 29 April 2015 Keywords: EuroSCORE • risk score • calibration • reporting In their study, Garcia-Valentin et al. [1] evaluated the performance of EuroSCORE and EuroSCORE II in a cohort of 4034 patients undergoing cardiac surgery in Spain. While we applaud the authors for conducting this independent external validation, which is uncommon, we wish to highlight some methodological and reporting concerns so that future investigators do not replicate them. The two key components characterizing the performance of a prediction model are discrimination and calibration. Discrimination is the ability of the prediction model to differentiate between those who do and do not experience the outcome event (quantified by c-index which is equivalent to the area under the receiver operating characteristic curve). Calibration is the agreement between outcome predictions from the model and the observed outcomes. In the study by Garcia-Valentin et al., calibration was evaluated using the Hosmer–
© The Author 2015. Published by Oxford University Press on behalf of the European Association for Cardio-Thoracic Surgery. All rights reserved.
LETTERS TO THE EDITOR
Keywords: Ex vivo lung perfusion • Hydrogen lung preconditioning • Lung preservation • Lung transplantation • Graft dysfunction
Conflict of interest: none declared.
358
Letters to the Editor / European Journal of Cardio-Thoracic Surgery
Lemeshow test, which the authors correctly highlight as being problematic. The Hosmer–Lemeshow test has limited power to evaluate calibration, affected by sample size and grouping and gives no indication on the direction and magnitude of (mis)calibration [2]. Unfortunately, the authors disappointingly proceeded to use this test to judge calibration on the premise that this was used in the original model development study [3]. Repeating an incorrect and uninformative analysis on the grounds that it was done in the original study is flawed logic and if methodological concerns are raised, then alternative and correct analyses should be carried out. Knowingly repeating a flawed analysis even if concerns are raised, incorrectly supports and justifies its use by other investigators and the cycle is never broken. The study by Garcia-Valentin et al. raised concerns of the approach but only in the ‘Discussion’ section, which is often not read as closely as the ‘Methods’ or ‘Results’ section of a manuscript. In accordance with recent recommendations on the reporting of prediction model studies [2, 4], calibration should be assessed graphically by plotting predicted outcome probabilities (x-axis) against observed outcomes (y-axis) using a high-resolution smoothed (loess) line. The direction and magnitude of any miscalibration can then be examined across the entire probability range. The calibration plot can also be supplemented with a numerical quantification of calibration by examining the calibration slope and intercept and the 0.9 quantile of the absolute prediction error [5, 6]. We recommend investigators, peer reviewers and editors to read the recent guidance from the Transparent Reporting of a multivariable prediction model for Individual Prognosis or Diagnosis (TRIPOD) Initiative (www. tripod-statement.org). The TRIPOD reporting guideline for clinical prediction models, discusses key issues in the development and validation of a prediction model [4]. The TRIPOD guideline is similar to other well-known reporting guidelines (e.g. CONSORT, STROBE and PRISMA) designed to help authors, peer reviewers and journal editors in ensuring that the essential items describing the development or validation of a clinical prediction model are clearly reported. Accompanying the reporting guideline is an extensive Explanation and Elaboration article describing the rationale for the checklist item but also highlighting many methodological considerations when developing or validating a clinical prediction model [2].
REFERENCES [1] Garcia-Valentin A, Mestres CA, Bernabeu E, Bahamonde JA, Martin I, Rueda C et al. Validation and quality measurements for EuroSCORE and EuroSCORE II in the Spanish cardiac surgical population: a prospective, multicentre study. Eur J Cardiothorac Surg 2015; doi:10.1093/ejcts/ezv090. [2] Moons KGM, Altman DG, Reitsma JB, Ioannidis JPA, Macaskill P, Steyerberg EW et al. Transparent Reporting of a multivariable prediction model for Individual Prognosis Or Diagnosis (TRIPOD): Explanation and Elaboration. Ann Intern Med 2015;162:W1–W73. [3] Nashef SA, Roques F, Sharples LD, Nilsson J, Smith C, Goldstone AR et al. EuroSCORE II. Eur J Cardiothorac Surg 2012;41:734–44. [4] Collins GS, Reitsma JB, Altman DG, Moons KGM. Transparent Reporting of a multivariable prediction model for Individual Prognosis Or Diagnosis (TRIPOD): the TRIPOD statement. Ann Intern Med 2015;162:55–63. [5] Harrell FE Jr. Regression Modeling Strategies: with Applications to Linear Models, Logistic Regression and Survival Analysis. New York: Springer 2001. [6] Steyerberg EW. Clinical Prediction Models: a Practical Approach to Development, Validation, and Updating. New York: Springer 2009. * Corresponding author. Centre for Statistics in Medicine, Botnar Research Centre, University of Oxford, Windmill Road, Oxford OX3 7LD, UK. Tel: +44 (0) 1865 223460; e-mail: [email protected] (G.S. Collins). doi:10.1093/ejcts/ezv190 Advance Access publication 28 May 2015
Reply to Collins and Le Manach Antonio Garcia-Valentina,* and Carlos A. Mestresb a
Department of Cardiovascular Surgery, Hospital Clinic, Valencia, Spain Department of Cardiovascular Surgery, Hospital Clinic, University of Barcelona, Barcelona, Spain
b
Received 24 April 2015; accepted 29 April 2015
Keywords: Statistics • Cardiac surgery • Validation studies • Risk • Prospective studies • Clinical prediction rule Collins and Le Manach [1] have expressed concerns over the flawed methodology of our recent study [2]. These unfavourable comments are derived from the utilization of the Hosmer–Lemeshow calibration test which, in their opinion, results in intentional and inappropriate statistical approach. This test has been used for the evaluation of calibration in a well-known number of validation studies although recent consensus recommends other tests for that purpose. Unfortunately, these recommendations were not available at the time of the design of our project, as they date back to early 2015 [3]. Therefore, it may be inappropriate to impose this on authors, when this knowledge was not readily available. Concerns surrounding the Hosmer–Lemeshow test refer to the influence of sample size in results [4]. Recently, Lemeshow and coworkers set the rules to optimize statistical power and obtain meaningful results with the test by performing an adequate selection of the sample size and the number of groups in the analysis [5]. Novel calibration methods were considered during the design of our study although we found some advantages in the Hosmer–Lemeshow test. Readers are used to it by its wide utilization and this makes possible to easily compare results with previous studies in the same terms. Perhaps we failed to adequately describe this in our article and we apologize for this. Although we acknowledge these limitations, we do not share this negative opinion about the Hosmer–Lemeshow test. Consequently, we cannot accept the suggestion of intentional flaw that they intend to transmit. A good performance of the test was estimated during the calculation of sample size in the design phase, and this is the reason why it was decided to use it in our study. We were careful in our conclusions, conscious of the possible interpretations of our results and the limitations of the calibration test. These issues have been addressed in the ‘Discussion’ section, which is the place meant to do so according to the journal instructions. The reader understands that the sections ‘Methods’ and ‘Results’ refer to how to do things and which the results of doing something, regardless of outcomes, have been. Once again, the ‘Discussion’ section seems to be appropriate for all comments related to the topic. We are not in a position to discuss Collins and Le Manachs’ opinion about the actual amount of time readers dedicate to the ‘Discussion’ section, but it sounds appropriate that they should produce actual data to support their opinion, considering their strong scientific and methodological background. Our article underwent an exhaustive peer review process that involved 2 editors, 3 reviewers and 2 independent statisticians. Possible misinterpretations were reassessed, and no additional problems in our methodology detected. We thank Collins and Le Manach for reminding the community their recommendations, which we will consider. It is appropriate that explanations about unclear methods or data should be demanded. We understand the deep disappointment of Collins and Le Manach for what they consider a suboptimal methodology in our contribution. Scientific thinking should also stay away from radical ideas and disqualification, and should be respectful towards other thoughts that differ from one’s own.
REFERENCES [1] Collins G, Le Manach Y. Knowingly repeating an incorrect and inefficient analysis is flawed logic. Eur J Cardiothorac Surg 2016;49:357–8. [2] Garcia-Valentin A, Mestres CA, Bernabeu E, Bahamonde JA, Martin I, Rueda C et al. Validation and quality measurements for EuroSCORE and EuroSCORE II in the Spanish cardiac surgical population: a prospective, multicentre study. Eur J Cardiothorac Surg 2015; doi:10.1093/ ejcts/ezv090. [3] Moons KG, Altman DG, Reitsma JB, Ioannidis JP, Macaskill P, Steyerberg EW et al. Transparent Reporting of a multivariable prediction model for Individual Prognosis or Diagnosis (TRIPOD): explanation and elaboration. Ann Int Med 2015;162:W1–73. [4] Steyerberg E. Clinical Prediction Models, 1st edn. New York: Springer Science+Business Media, LLC, 2010. [5] Paul P, Pennell ML, Lemeshow S. Standardizing the power of the HosmerLemeshow goodness of fit test in large data sets. Stat Med 2013;32:67–80. * Corresponding author. Department of Cardiovascular Surgery, Hospital Clinic, Avda. Blasco Ibáñez 17, 46010 Valencia, Spain. Tel: +34-963987848; fax: +34-963987848; e-mail: [email protected] (A. García-Valentín). doi:10.1093/ejcts/ezv191 Advance Access publication 28 May 2015
European Journal of Cardio-Thoracic Surgery 49 (2016) 357–360
Antioxidants and K+ channel agonists versus hydrogen therapy during 1 ex vivo lung perfusion †
supplementing Steen solution™ with a combination of antioxidants and K+ channel agonists based on the above-discussed principles.
Mohamed S.A. Mohamed* Thoracic Transplantation Department, University Clinic Essen, Essen, Germany
REFERENCES
Received 17 April 2015; accepted 29 April 2015
[1] Haam S, Lee S, Paik HC, Park MS, Song JH, Lim BJ et al. The effects of hydrogen gas inhalation during ex vivo lung perfusion on donor lungs obtained after cardiac death. Eur J Cardiothorac Surg 2015;48:542–7. [2] Noda K, Shigemura N, Tanaka Y, Bhama J, D’Cunha J, Kobayashi H et al. Hydrogen preconditioning during ex vivo lung perfusion improves the quality of lung grafts in rats. Transplantation 2014;98:499–506. [3] Perry MM, Hui CK, Whiteman M et al. Hydrogen sulfide inhibits proliferation and release of IL-8 from human airway smooth muscle cells. Am J Respir Cell Mol Biol 2011;45:746–52. [4] George TJ, Arnaoutakis GJ, Beaty CA et al. Hydrogen sulfide decreases reactive oxygen in a model of lung transplantation. J Surg Res 2012;178:494–501. [5] Chatterjee S, Nieman GF, Christie JD, Fisher AB. Shear stress-related mechanosignaling with lung ischemia: lessons from basic research can inform lung transplantation. Am J Physiol Lung Cell Mol Physiol 2014;307:L668–80. [6] Barbagallo I, Marrazzo G, Frigiola A, Zappala A, Li Volti G. Role of carbon monoxide in vascular diseases. Curr Pharm Biotechnol 2012;13:787–96.
Several studies have confirmed the role of hydrogen to scavenge reactive oxygen species (ROSs) and to protect against ischaemic reperfusion injury (IRI) in the lung and the heart [1–4]. In a study, where rabbit lungs were exposed to ex vivo lung perfusion (EVLP), the infusion of H2S significantly reduced ROS activity within the grafts [4]. This resulted in a significant decrease of lipid peroxidation, epithelial apoptosis, airway smooth muscle cell (ASMC) proliferation and (interleukin) IL8 production [3, 4]. On the other hand, graft physiological parameters were improved in response to 2% hydrogen inhalation, while the graft is exposed to EVLP [1, 2]. During graft ischaemia and cold preservation, Na+/K+ adenosine 5’-triphosphatase is inhibited, leading to increased Na+ intracellular influx and K+ extracellular efflux. The inhibited activity of K+ channels leads to the failure to re-establish cell membrane hyperpolarization, leading to sustained cell membrane depolarization, impaired mitochondrial activity, enhanced activity of nicotinamide adenine dinucleotide phosphate hydrogen and Xanthine oxidases, leading to a significant increase in ROS production [5]. ROS stimulate ASMC proliferation, which may contribute to the development of pulmonary hypertension, and prime graft inflammasomes. The drop of the intracellular K+ levels activates the primed inflammasomes to activate caspase 1, which activates pro-IL1β and pro-IL18. While IL1β stimulates ASMC proliferation, both cytokines induce IL6. Through ROS scavenging by hydrogen, inflammasomes activation is attenuated, levels of IL1β, IL6 and IL8 are decreased, the graft oxygenation capacity and vascular resistance are improved [1, 2]. However, in the experiments of George et al. [4] the graft oxygenation capacity and vascular resistance did not differ between the hydrogen-treated and the non-hydrogen-treated groups. This apparent contradiction between the results of hydrogen therapy might be related to the administration of Na+ (as a bolus and infusion) during the EVLP model of George et al. [4], where donor blood was used as a perfusate. This might have potentiated the Na+ influx and the subsequent sustained membrane depolarization. On the contrary, the experiments conducted by Haam et al. [1] utilized a model of EVLP that follows the Toronto technique, using Perfadex and Steen solutions that contain physiological Na+ and K+ levels. This may highlight that the reconditioning of the lung graft is a complex procedure that requires multiple effectors that target various aspects simultaneously, where hydrogen therapy targets the reduction of ROS to eliminate their hazardous effects, and the effective EVLP technique improves other graft functional parameters. Nevertheless, hydrogen protects the lung graft against IRI as a potent antioxidant and through the up-regulation of heme-oxygenase-1 (HO-1) [1, 2]. HO-1 catalyzes carbon monoxide production, which is a potent antioxidant, activates guanylyl cyclase (which may explain the trend of increased cGMP observed by George et al.), and activates ATP-sensitive K+ channels and big conductance calcium-sensitive K+ channels [6]. The improved activity of K+ channels has been shown to recondition the graft, antagonize cell membrane depolarization, antagonize the opening of mitochondrial permeability transition pore during reperfusion, decrease ROS production and attenuate the inflammatory cytokine production. All of these would protect the graft against IRI and graft dysfunction [5]. Accordingly, a combination of antioxidants and K+ channel agonists may achieve at least a similar level of protection and may exhibit more significantly protective effects than those of the hydrogen therapy (future studies should be conducted to investigate this hypothesis). While hydrogen inhalation is applied together with an effective EVLP, in order to provide a ‘multitarget’ graft reconditioning procedure, a recent, theoretically described, EVLP protocol (Shehata protocol) introduced the notion of
* Corresponding author. Thoracic Transplantation Department, University Clinic Essen, Hufeland Straße 55, 45147 Essen, Germany. Tel: +49-201-7233779; fax: +49-201-7235471; e-mail: [email protected] (M.S.A. Mohamed). doi:10.1093/ejcts/ezv192 Advance Access publication 25 May 2015 †
The corresponding author of the original article [1] was invited to reply, but did not respond.
Knowingly repeating an incorrect and inefficient analysis is flawed logic Gary S. Collinsa,* and Yannick Le Manachb a Centre for Statistics in Medicine, Botnar Research Centre, University of Oxford, Oxford, UK b Departments of Anesthesia and Clinical Epidemiology and Biostatistics, Michael DeGroote School of Medicine, Faculty of Health Sciences, McMaster University and the Perioperative Research Group, Population Health Research Institute, Hamilton, Canada
Received 27 March 2015; accepted 29 April 2015 Keywords: EuroSCORE • risk score • calibration • reporting In their study, Garcia-Valentin et al. [1] evaluated the performance of EuroSCORE and EuroSCORE II in a cohort of 4034 patients undergoing cardiac surgery in Spain. While we applaud the authors for conducting this independent external validation, which is uncommon, we wish to highlight some methodological and reporting concerns so that future investigators do not replicate them. The two key components characterizing the performance of a prediction model are discrimination and calibration. Discrimination is the ability of the prediction model to differentiate between those who do and do not experience the outcome event (quantified by c-index which is equivalent to the area under the receiver operating characteristic curve). Calibration is the agreement between outcome predictions from the model and the observed outcomes. In the study by Garcia-Valentin et al., calibration was evaluated using the Hosmer–
© The Author 2015. Published by Oxford University Press on behalf of the European Association for Cardio-Thoracic Surgery. All rights reserved.
LETTERS TO THE EDITOR
Keywords: Ex vivo lung perfusion • Hydrogen lung preconditioning • Lung preservation • Lung transplantation • Graft dysfunction
Conflict of interest: none declared.
358
Letters to the Editor / European Journal of Cardio-Thoracic Surgery
Lemeshow test, which the authors correctly highlight as being problematic. The Hosmer–Lemeshow test has limited power to evaluate calibration, affected by sample size and grouping and gives no indication on the direction and magnitude of (mis)calibration [2]. Unfortunately, the authors disappointingly proceeded to use this test to judge calibration on the premise that this was used in the original model development study [3]. Repeating an incorrect and uninformative analysis on the grounds that it was done in the original study is flawed logic and if methodological concerns are raised, then alternative and correct analyses should be carried out. Knowingly repeating a flawed analysis even if concerns are raised, incorrectly supports and justifies its use by other investigators and the cycle is never broken. The study by Garcia-Valentin et al. raised concerns of the approach but only in the ‘Discussion’ section, which is often not read as closely as the ‘Methods’ or ‘Results’ section of a manuscript. In accordance with recent recommendations on the reporting of prediction model studies [2, 4], calibration should be assessed graphically by plotting predicted outcome probabilities (x-axis) against observed outcomes (y-axis) using a high-resolution smoothed (loess) line. The direction and magnitude of any miscalibration can then be examined across the entire probability range. The calibration plot can also be supplemented with a numerical quantification of calibration by examining the calibration slope and intercept and the 0.9 quantile of the absolute prediction error [5, 6]. We recommend investigators, peer reviewers and editors to read the recent guidance from the Transparent Reporting of a multivariable prediction model for Individual Prognosis or Diagnosis (TRIPOD) Initiative (www. tripod-statement.org). The TRIPOD reporting guideline for clinical prediction models, discusses key issues in the development and validation of a prediction model [4]. The TRIPOD guideline is similar to other well-known reporting guidelines (e.g. CONSORT, STROBE and PRISMA) designed to help authors, peer reviewers and journal editors in ensuring that the essential items describing the development or validation of a clinical prediction model are clearly reported. Accompanying the reporting guideline is an extensive Explanation and Elaboration article describing the rationale for the checklist item but also highlighting many methodological considerations when developing or validating a clinical prediction model [2].
REFERENCES [1] Garcia-Valentin A, Mestres CA, Bernabeu E, Bahamonde JA, Martin I, Rueda C et al. Validation and quality measurements for EuroSCORE and EuroSCORE II in the Spanish cardiac surgical population: a prospective, multicentre study. Eur J Cardiothorac Surg 2015; doi:10.1093/ejcts/ezv090. [2] Moons KGM, Altman DG, Reitsma JB, Ioannidis JPA, Macaskill P, Steyerberg EW et al. Transparent Reporting of a multivariable prediction model for Individual Prognosis Or Diagnosis (TRIPOD): Explanation and Elaboration. Ann Intern Med 2015;162:W1–W73. [3] Nashef SA, Roques F, Sharples LD, Nilsson J, Smith C, Goldstone AR et al. EuroSCORE II. Eur J Cardiothorac Surg 2012;41:734–44. [4] Collins GS, Reitsma JB, Altman DG, Moons KGM. Transparent Reporting of a multivariable prediction model for Individual Prognosis Or Diagnosis (TRIPOD): the TRIPOD statement. Ann Intern Med 2015;162:55–63. [5] Harrell FE Jr. Regression Modeling Strategies: with Applications to Linear Models, Logistic Regression and Survival Analysis. New York: Springer 2001. [6] Steyerberg EW. Clinical Prediction Models: a Practical Approach to Development, Validation, and Updating. New York: Springer 2009. * Corresponding author. Centre for Statistics in Medicine, Botnar Research Centre, University of Oxford, Windmill Road, Oxford OX3 7LD, UK. Tel: +44 (0) 1865 223460; e-mail: [email protected] (G.S. Collins). doi:10.1093/ejcts/ezv190 Advance Access publication 28 May 2015
Reply to Collins and Le Manach Antonio Garcia-Valentina,* and Carlos A. Mestresb a
Department of Cardiovascular Surgery, Hospital Clinic, Valencia, Spain Department of Cardiovascular Surgery, Hospital Clinic, University of Barcelona, Barcelona, Spain
b
Received 24 April 2015; accepted 29 April 2015
Keywords: Statistics • Cardiac surgery • Validation studies • Risk • Prospective studies • Clinical prediction rule Collins and Le Manach [1] have expressed concerns over the flawed methodology of our recent study [2]. These unfavourable comments are derived from the utilization of the Hosmer–Lemeshow calibration test which, in their opinion, results in intentional and inappropriate statistical approach. This test has been used for the evaluation of calibration in a well-known number of validation studies although recent consensus recommends other tests for that purpose. Unfortunately, these recommendations were not available at the time of the design of our project, as they date back to early 2015 [3]. Therefore, it may be inappropriate to impose this on authors, when this knowledge was not readily available. Concerns surrounding the Hosmer–Lemeshow test refer to the influence of sample size in results [4]. Recently, Lemeshow and coworkers set the rules to optimize statistical power and obtain meaningful results with the test by performing an adequate selection of the sample size and the number of groups in the analysis [5]. Novel calibration methods were considered during the design of our study although we found some advantages in the Hosmer–Lemeshow test. Readers are used to it by its wide utilization and this makes possible to easily compare results with previous studies in the same terms. Perhaps we failed to adequately describe this in our article and we apologize for this. Although we acknowledge these limitations, we do not share this negative opinion about the Hosmer–Lemeshow test. Consequently, we cannot accept the suggestion of intentional flaw that they intend to transmit. A good performance of the test was estimated during the calculation of sample size in the design phase, and this is the reason why it was decided to use it in our study. We were careful in our conclusions, conscious of the possible interpretations of our results and the limitations of the calibration test. These issues have been addressed in the ‘Discussion’ section, which is the place meant to do so according to the journal instructions. The reader understands that the sections ‘Methods’ and ‘Results’ refer to how to do things and which the results of doing something, regardless of outcomes, have been. Once again, the ‘Discussion’ section seems to be appropriate for all comments related to the topic. We are not in a position to discuss Collins and Le Manachs’ opinion about the actual amount of time readers dedicate to the ‘Discussion’ section, but it sounds appropriate that they should produce actual data to support their opinion, considering their strong scientific and methodological background. Our article underwent an exhaustive peer review process that involved 2 editors, 3 reviewers and 2 independent statisticians. Possible misinterpretations were reassessed, and no additional problems in our methodology detected. We thank Collins and Le Manach for reminding the community their recommendations, which we will consider. It is appropriate that explanations about unclear methods or data should be demanded. We understand the deep disappointment of Collins and Le Manach for what they consider a suboptimal methodology in our contribution. Scientific thinking should also stay away from radical ideas and disqualification, and should be respectful towards other thoughts that differ from one’s own.
REFERENCES [1] Collins G, Le Manach Y. Knowingly repeating an incorrect and inefficient analysis is flawed logic. Eur J Cardiothorac Surg 2016;49:357–8. [2] Garcia-Valentin A, Mestres CA, Bernabeu E, Bahamonde JA, Martin I, Rueda C et al. Validation and quality measurements for EuroSCORE and EuroSCORE II in the Spanish cardiac surgical population: a prospective, multicentre study. Eur J Cardiothorac Surg 2015; doi:10.1093/ ejcts/ezv090. [3] Moons KG, Altman DG, Reitsma JB, Ioannidis JP, Macaskill P, Steyerberg EW et al. Transparent Reporting of a multivariable prediction model for Individual Prognosis or Diagnosis (TRIPOD): explanation and elaboration. Ann Int Med 2015;162:W1–73. [4] Steyerberg E. Clinical Prediction Models, 1st edn. New York: Springer Science+Business Media, LLC, 2010. [5] Paul P, Pennell ML, Lemeshow S. Standardizing the power of the HosmerLemeshow goodness of fit test in large data sets. Stat Med 2013;32:67–80. * Corresponding author. Department of Cardiovascular Surgery, Hospital Clinic, Avda. Blasco Ibáñez 17, 46010 Valencia, Spain. Tel: +34-963987848; fax: +34-963987848; e-mail: [email protected] (A. García-Valentín). doi:10.1093/ejcts/ezv191 Advance Access publication 28 May 2015
