International Journal of
Radiation Oncology biology
physics
www.redjournal.org
Brief Report
Reporting of Uncertainty at the 2013 Annual Meeting of the American Society for Radiation Oncology W. Robert Lee, MD, MS, MEd Department of Radiation Oncology, Duke University School of Medicine, Durham, North Carolina Received Dec 16, 2013. Accepted for publication Dec 19, 2013.
Summary This brief analysis documents the fact that measures of uncertainty are not uniformly used in American Society for Radiation Oncology abstracts accepted for oral presentation.
Purpose: The annual meeting of the American Society for Radiation Oncology (ASTRO) is designed to disseminate new scientific findings and technical advances to professionals. Best practices of scientific dissemination require that some level of uncertainty (or imprecision) is provided. Methods and Materials: A total of 279 scientific abstracts were selected for oral presentation in a clinical session at the 2013 ASTRO Annual Meeting. A random sample of these abstracts was reviewed to determine whether a 95% confidence interval (95% CI) or analogous measure of precision was provided for time-to-event analyses. Results: A sample of 140 abstracts was reviewed. Of the 65 abstracts with Kaplan-Meier or cumulative incidence analyses, 6 included some measure of precision (6 of 65 Z 9%; 95% CI, 216). Of the 43 abstracts reporting ratios for time-to-event analyses (eg, hazard ratio, risk ratio), 22 included some measure of precision (22 of 43 Z 51%; 95% CI, 36-66). Conclusions: Measures of precision are not provided in a significant percentage of abstracts selected for oral presentation at the Annual Meeting of ASTRO. Ó 2014 Elsevier Inc.
“Medicine is a science of uncertainty and an art of probability” Sir William Osler
Introduction Throughout its history, one of the fundamental goals of the American Society for Radiation Oncology (ASTRO) has been to promote research and disseminate scientific results to the broader community. This dissemination has been accomplished primarily through publication of the International Journal of Radiation
Reprint requests to: W. Robert Lee, MD, MS, MEd, Department of Oncology, Duke University School of Medicine, DUMC, Box 3085, Durham, NC 27710. Tel: (919) 668-5640; E-mail: [email protected] Conflict of interest: none. Int J Radiation Oncol Biol Phys, Vol. 89, No. 1, pp. 222e223, 2014 0360-3016/$ - see front matter Ó 2014 Elsevier Inc. All rights reserved. http://dx.doi.org/10.1016/j.ijrobp.2013.12.031
Oncology, Biology, Physics and sponsorship of the annual scientific meeting. Clinical evidence, by its very nature, has inherent uncertainty. This uncertainty creates challenges of interpretation, implementation, and communication. Best practices of scientific dissemination require that some level of uncertainty (or imprecision) is provided to the audience, whether that audience is the patient, a reviewer, or a reader. This simple analysis was performed to determine the extent to which measures of uncertainty were provided within the abstracts accepted for oral presentation at the 2013 Annual Meeting of ASTRO (https:// www.astro.org/Meetings-and-Events/2013-Annual-Meeting/Index. aspx).
AcknowledgmentdThe author would like to acknowledge the work of Lynn Brown and Johanna VanArsdall of ASTRO for their assistance with this project.
Volume 89 Number 1 2014
Reporting uncertainty at ASTRO
223
Methods and Materials
Discussion
A brief proposal was submitted to the ASTRO Annual Meeting steering committee, and, after discussion and approval, ASTRO staff provided an Excel (Microsoft, Redmond, WA) spreadsheet containing information on all abstracts accepted for the 2013 Annual Meeting. Abstracts in nonclinical areas (eg, physics, biology) and posters were excluded, leaving 279 scientific abstracts selected for oral presentation at a clinical session. A random sample of these abstracts was produced by using the random number function in Excel software, and 140 abstracts were individually reviewed by the author. The primary objective of this review was to determine whether a 95% confidence interval (95% CI) or analogous measure of precision was provided for any time-to-event analyses. Results are provided as simple proportions, and the 95% CI was calculated using Excel software.
Communicating uncertainty (or imprecision) of clinical information is an important component of scientific dissemination. Failure to provide information on uncertainty can result in poor decisions on the part of clinicians and patients alike. Alternatively, optimizing the presentation of uncertainty can lead individuals to make better decisions. In this brief analysis of the clinical abstracts accepted for oral presentation at the 2013 ASTRO Annual Meeting, reporting of uncertainty in the form of 95% CI was far from uniform. In fact, less than 10% of the reviewed abstracts that reported KaplanMeier estimates included 95% CI.
Results A sampling of 140 abstracts were reviewed. Approximately onehalf of the abstracts reviewed (65 of 140; 46%) contained KaplanMeier or cumulative incidence analyses. Of these 65 abstracts, only 6 (9%; 95% CI, 2-16) included some measure of precision. Approximately one-third of the abstracts reviewed (43 of 140; 31%) reported ratios for time-to-event analyses (eg, hazard ratio, risk ratio). Of these 43 abstracts, only 22 (51%; 95% CI, 36-66) included some measure of precision.
Conclusions It is possible that the abstracts reviewed were a biased sample, but the abstracts were chosen at random. Furthermore, the level of reporting uncertainty in this analysis is similar to that in previous reports (1,2). Surely, we can do better.
References 1. Shakespeare TP, Holecek MJ. Should we be using confidence intervals when reporting results of oncology studies? Int J Radiat Oncol Biol Phys 1998;41:971-972. 2. Urbanic JJ, Lee WR. Confidence intervals and survival estimates: a systematic review of 3 oncology journals. Am J Clin Oncol 2006;29: 405-407.
Radiation Oncology biology
physics
www.redjournal.org
Brief Report
Reporting of Uncertainty at the 2013 Annual Meeting of the American Society for Radiation Oncology W. Robert Lee, MD, MS, MEd Department of Radiation Oncology, Duke University School of Medicine, Durham, North Carolina Received Dec 16, 2013. Accepted for publication Dec 19, 2013.
Summary This brief analysis documents the fact that measures of uncertainty are not uniformly used in American Society for Radiation Oncology abstracts accepted for oral presentation.
Purpose: The annual meeting of the American Society for Radiation Oncology (ASTRO) is designed to disseminate new scientific findings and technical advances to professionals. Best practices of scientific dissemination require that some level of uncertainty (or imprecision) is provided. Methods and Materials: A total of 279 scientific abstracts were selected for oral presentation in a clinical session at the 2013 ASTRO Annual Meeting. A random sample of these abstracts was reviewed to determine whether a 95% confidence interval (95% CI) or analogous measure of precision was provided for time-to-event analyses. Results: A sample of 140 abstracts was reviewed. Of the 65 abstracts with Kaplan-Meier or cumulative incidence analyses, 6 included some measure of precision (6 of 65 Z 9%; 95% CI, 216). Of the 43 abstracts reporting ratios for time-to-event analyses (eg, hazard ratio, risk ratio), 22 included some measure of precision (22 of 43 Z 51%; 95% CI, 36-66). Conclusions: Measures of precision are not provided in a significant percentage of abstracts selected for oral presentation at the Annual Meeting of ASTRO. Ó 2014 Elsevier Inc.
“Medicine is a science of uncertainty and an art of probability” Sir William Osler
Introduction Throughout its history, one of the fundamental goals of the American Society for Radiation Oncology (ASTRO) has been to promote research and disseminate scientific results to the broader community. This dissemination has been accomplished primarily through publication of the International Journal of Radiation
Reprint requests to: W. Robert Lee, MD, MS, MEd, Department of Oncology, Duke University School of Medicine, DUMC, Box 3085, Durham, NC 27710. Tel: (919) 668-5640; E-mail: [email protected] Conflict of interest: none. Int J Radiation Oncol Biol Phys, Vol. 89, No. 1, pp. 222e223, 2014 0360-3016/$ - see front matter Ó 2014 Elsevier Inc. All rights reserved. http://dx.doi.org/10.1016/j.ijrobp.2013.12.031
Oncology, Biology, Physics and sponsorship of the annual scientific meeting. Clinical evidence, by its very nature, has inherent uncertainty. This uncertainty creates challenges of interpretation, implementation, and communication. Best practices of scientific dissemination require that some level of uncertainty (or imprecision) is provided to the audience, whether that audience is the patient, a reviewer, or a reader. This simple analysis was performed to determine the extent to which measures of uncertainty were provided within the abstracts accepted for oral presentation at the 2013 Annual Meeting of ASTRO (https:// www.astro.org/Meetings-and-Events/2013-Annual-Meeting/Index. aspx).
AcknowledgmentdThe author would like to acknowledge the work of Lynn Brown and Johanna VanArsdall of ASTRO for their assistance with this project.
Volume 89 Number 1 2014
Reporting uncertainty at ASTRO
223
Methods and Materials
Discussion
A brief proposal was submitted to the ASTRO Annual Meeting steering committee, and, after discussion and approval, ASTRO staff provided an Excel (Microsoft, Redmond, WA) spreadsheet containing information on all abstracts accepted for the 2013 Annual Meeting. Abstracts in nonclinical areas (eg, physics, biology) and posters were excluded, leaving 279 scientific abstracts selected for oral presentation at a clinical session. A random sample of these abstracts was produced by using the random number function in Excel software, and 140 abstracts were individually reviewed by the author. The primary objective of this review was to determine whether a 95% confidence interval (95% CI) or analogous measure of precision was provided for any time-to-event analyses. Results are provided as simple proportions, and the 95% CI was calculated using Excel software.
Communicating uncertainty (or imprecision) of clinical information is an important component of scientific dissemination. Failure to provide information on uncertainty can result in poor decisions on the part of clinicians and patients alike. Alternatively, optimizing the presentation of uncertainty can lead individuals to make better decisions. In this brief analysis of the clinical abstracts accepted for oral presentation at the 2013 ASTRO Annual Meeting, reporting of uncertainty in the form of 95% CI was far from uniform. In fact, less than 10% of the reviewed abstracts that reported KaplanMeier estimates included 95% CI.
Results A sampling of 140 abstracts were reviewed. Approximately onehalf of the abstracts reviewed (65 of 140; 46%) contained KaplanMeier or cumulative incidence analyses. Of these 65 abstracts, only 6 (9%; 95% CI, 2-16) included some measure of precision. Approximately one-third of the abstracts reviewed (43 of 140; 31%) reported ratios for time-to-event analyses (eg, hazard ratio, risk ratio). Of these 43 abstracts, only 22 (51%; 95% CI, 36-66) included some measure of precision.
Conclusions It is possible that the abstracts reviewed were a biased sample, but the abstracts were chosen at random. Furthermore, the level of reporting uncertainty in this analysis is similar to that in previous reports (1,2). Surely, we can do better.
References 1. Shakespeare TP, Holecek MJ. Should we be using confidence intervals when reporting results of oncology studies? Int J Radiat Oncol Biol Phys 1998;41:971-972. 2. Urbanic JJ, Lee WR. Confidence intervals and survival estimates: a systematic review of 3 oncology journals. Am J Clin Oncol 2006;29: 405-407.
