Pediatr Radiol (2014) 44:940–941 DOI 10.1007/s00247-014-3080-3
RESEARCH FORUM
Comparative effectiveness research in pediatric radiology Robert C. Orth & Scott B. Cantor
Received: 17 December 2013 / Revised: 20 May 2014 / Accepted: 4 June 2014 # Springer-Verlag Berlin Heidelberg 2014
Abstract Comparative effectiveness research is designed to help patients, physicians and policymakers choose the best medical option among alternative diagnostic and interventional strategies. In this commentary, we provide a brief overview of comparative effectiveness research, discuss some of the challenges to applying it in pediatric radiology, and give several reasons behind the continued push for more comparative effectiveness research by governmental and other funding agencies. Keywords Pediatric radiology . Comparative effectiveness research . Cost-effectiveness analysis
The paradigm for radiology investigations has remained largely unchanged for decades: New technologies are invented, tested for accuracy and implemented clinically with sometimes little evidence supporting their potential benefit [1]. Fryback and Thornbury [2] proposed a six-level hierarchical model to classify assessment studies of diagnostic imaging efficacy (Table 1). Using this model, most diagnostic radiology research falls into levels 1 and 2, e.g. imaging resolution; test sensitivity and specificity [2, 3]. Studies evaluating the effects of imaging technologies on clinical diagnosis, therapeutic choices, patient outcomes and the value to society—i.e. levels 3, 4, 5, and 6—are less frequently performed. R. C. Orth (*) Edward B. Singleton Department of Pediatric Radiology, Texas Children’s Hospital, 6701 Fannin St., MC CC470.01, Houston, TX 77030-2399, USA e-mail: [email protected] S. B. Cantor Department of Health Services Research, The University of Texas MD Anderson Cancer Center, Houston, TX, USA
Comparative effectiveness research is the comparison of two or more strategies for preventing, treating, diagnosing and/or managing a health condition [4]. The purpose of comparative effectiveness research is to provide patients, physicians and other decision-makers with evidence-based information that will help them choose the best intervention for their specific situation [5]. The strategies compared must represent the best available alternatives and research should be performed in real-world settings. Several study methodologies may be used for comparative effectiveness research. Randomized controlled trials have a role but cannot be used to evaluate all imaging technologies because of cost, ethical considerations and the rapid pace of technological advances. Decision-analytic models can determine the effects of different imaging strategies on patient outcomes by simulating controlled trials, but these models depend on the quality of estimated test performance characteristics and other assumptions. Patient and provider behavioral analyses, quality improvement initiatives and informatics research are also used to conduct comparative effectiveness research. The challenges to performing comparative effectiveness research in pediatric radiology mirror many of the challenges to performing comparative effectiveness research in adult imaging, perhaps the most daunting of which is demonstrating a direct link between imaging test results and patient outcomes [6]. Diagnostic imaging is usually one step in a multistep process and differentiating the effects of diagnostic imaging from the effects of subsequent treatment is often difficult [7]. This challenge is exacerbated in pediatric radiology where interventional procedures and screening examinations, which tend to have more clearly defined endpoints and more direct linkage to outcomes than do diagnostic examinations, are less frequently performed than in adults. Innovations in imaging technology have led to tremendous advances in patient care. However, these same innovations
Pediatr Radiol (2014) 44:940–941
941
Table 1 A hierarchical model for assessment studies of diagnostic imaging efficacy [2, 3] Level of efficacy
Examples of endpoints for each level of efficacy
Level 1: technical efficacy Level 2: diagnostic accuracy efficacy Level 3: diagnostic thinking efficacy Level 4: therapeutic efficacy Level 5: patient outcome efficacy Level 6: societal efficacy
Imaging resolution Test sensitivity and specificity Pre- and post-test changes in subjectively determined outcome Effects of diagnostic on choice of therapy Value of test information including measures of morbidity, mortality, and quality-adjusted life years (QALYs) Cost-benefit and cost-effectiveness from the societal perspective
can contribute to rising health care costs. New imaging technologies are often adopted before being proven clinically effective, let alone cost effective. Almost invariably, new technologies are more expensive than those they replace. Cost-effectiveness analysis is a component of comparative effectiveness research that evaluates level 6 on the Fryback and Thornbury hierarchy, societal efficacy—the value of imaging technologies to society [2]. An example of costeffectiveness analysis in pediatric radiology is provided by Medina et al. [8], who investigated different diagnostic strategies for newborns with suspected occult spinal dysraphism. In this study, a decision-analytic model was constructed to compare the health outcomes and costs of MRI, ultrasound (US), radiographs and no imaging with close clinical followup. By modeling clinical outcomes and costs, the authors were able to show that the optimal strategy depends on the pretest probability of spinal dysraphism. For example, in high-risk newborns, MRI resulted in cost savings, whereas in low-risk infants, US or no imaging with close clinical follow-up was the preferred diagnostic strategy. If resources were limitless, we would not consider costs; we would choose imaging strategies solely on the basis of their clinical effectiveness. But resources are limited, and money spent on health care is not available for other priorities such as education, infrastructure and the environment. The United States spends more than any other country on health care as a percent of gross domestic product, nearly 18% in 2011. If the quality of U.S. health care reflected this cost, the price might be justifiable. Unfortunately, we have not been able to demonstrate better outcomes than nations that spend far less and, in many cases, our outcomes are poorer. For example, in 2011, the United States ranked 32nd in infant mortality and 26th in life expectancy among a list of developed nations while spending over than 50% more than the next highest-spending nation [9]. Too often, we are not getting a good value from our health care system. Cost-effectiveness analysis allows us to measure value by telling us how much benefit a treatment will provide relative to another and at what cost. The challenges to performing comparative effectiveness research in pediatric radiology are considerable but so are the potential rewards. By determining the most appropriate use of
imaging resources, we can improve patient care. The first step is to change our focus from determining whether we can effectively perform a given study to whether we should perform that study and what effect it will have on patient outcomes. We need to promote the methodologies of comparative effectiveness research by encouraging the application of these methods for presentation at our national and international meetings and for publication in our scientific journals. Successful comparative effectiveness research in pediatric radiology will require novel interdisciplinary collaborations and a continued focus on high-quality evidence-based research. Acknowledgments The authors wish to thank Markeda Wade in the Department of Scientific Publications at The University of Texas MD Anderson Cancer Center for editorial contributions that enhanced the quality of this manuscript. Conflicts of interest None
References 1. Pandharipande PV, Gazelle GS (2009) Comparative effectiveness research: what it means for radiology. Radiology 253:600–605 2. Fryback DG, Thornbury JR (1991) The efficacy of diagnostic imaging. Med Decis Making 11:88–94 3. Gazelle GS, Kessler L, Lee DWet al (2011) A framework for assessing the value of diagnostic imaging in the era of comparative effectiveness research. Radiology 261:692–698 4. Committee on Comparative Effectiveness Research Prioritization (2009) Report brief: initial national priorities for comparative effectiveness research. Institute of Medicine. http://www.iom.edu/ cerpriorities. Accessed 4 June 2014 5. Society for Medical Decision Making (2009) Background paper on comparative effectiveness research. http://www.smdm.org/news/ detail/smdm-board-background-paper-on-comparative-effectivenessresearch. Accessed 4 June 2014 6. Rawson JV (2011) Comparative effectiveness research in radiology: patients, physicians and policy makers. Acad Radiol 18:1067–1071 7. Drummond M, Griffin A, Tarricone R (2009) Economic evaluation for devices and drugs–same or different? Value Health 12:402–404 8. Medina LS, Crone K, Kuntz KM (2001) Newborns with suspected occult spinal dysraphism: a cost-effectiveness analysis of diagnostic strategies. Pediatrics 108:e101 9. (2013) OECD Health Data. http://www.oecd-ilibrary.org/socialissues-migration-health/health-key-tables-from-oecd_20758480. Accessed 3 December 2013
RESEARCH FORUM
Comparative effectiveness research in pediatric radiology Robert C. Orth & Scott B. Cantor
Received: 17 December 2013 / Revised: 20 May 2014 / Accepted: 4 June 2014 # Springer-Verlag Berlin Heidelberg 2014
Abstract Comparative effectiveness research is designed to help patients, physicians and policymakers choose the best medical option among alternative diagnostic and interventional strategies. In this commentary, we provide a brief overview of comparative effectiveness research, discuss some of the challenges to applying it in pediatric radiology, and give several reasons behind the continued push for more comparative effectiveness research by governmental and other funding agencies. Keywords Pediatric radiology . Comparative effectiveness research . Cost-effectiveness analysis
The paradigm for radiology investigations has remained largely unchanged for decades: New technologies are invented, tested for accuracy and implemented clinically with sometimes little evidence supporting their potential benefit [1]. Fryback and Thornbury [2] proposed a six-level hierarchical model to classify assessment studies of diagnostic imaging efficacy (Table 1). Using this model, most diagnostic radiology research falls into levels 1 and 2, e.g. imaging resolution; test sensitivity and specificity [2, 3]. Studies evaluating the effects of imaging technologies on clinical diagnosis, therapeutic choices, patient outcomes and the value to society—i.e. levels 3, 4, 5, and 6—are less frequently performed. R. C. Orth (*) Edward B. Singleton Department of Pediatric Radiology, Texas Children’s Hospital, 6701 Fannin St., MC CC470.01, Houston, TX 77030-2399, USA e-mail: [email protected] S. B. Cantor Department of Health Services Research, The University of Texas MD Anderson Cancer Center, Houston, TX, USA
Comparative effectiveness research is the comparison of two or more strategies for preventing, treating, diagnosing and/or managing a health condition [4]. The purpose of comparative effectiveness research is to provide patients, physicians and other decision-makers with evidence-based information that will help them choose the best intervention for their specific situation [5]. The strategies compared must represent the best available alternatives and research should be performed in real-world settings. Several study methodologies may be used for comparative effectiveness research. Randomized controlled trials have a role but cannot be used to evaluate all imaging technologies because of cost, ethical considerations and the rapid pace of technological advances. Decision-analytic models can determine the effects of different imaging strategies on patient outcomes by simulating controlled trials, but these models depend on the quality of estimated test performance characteristics and other assumptions. Patient and provider behavioral analyses, quality improvement initiatives and informatics research are also used to conduct comparative effectiveness research. The challenges to performing comparative effectiveness research in pediatric radiology mirror many of the challenges to performing comparative effectiveness research in adult imaging, perhaps the most daunting of which is demonstrating a direct link between imaging test results and patient outcomes [6]. Diagnostic imaging is usually one step in a multistep process and differentiating the effects of diagnostic imaging from the effects of subsequent treatment is often difficult [7]. This challenge is exacerbated in pediatric radiology where interventional procedures and screening examinations, which tend to have more clearly defined endpoints and more direct linkage to outcomes than do diagnostic examinations, are less frequently performed than in adults. Innovations in imaging technology have led to tremendous advances in patient care. However, these same innovations
Pediatr Radiol (2014) 44:940–941
941
Table 1 A hierarchical model for assessment studies of diagnostic imaging efficacy [2, 3] Level of efficacy
Examples of endpoints for each level of efficacy
Level 1: technical efficacy Level 2: diagnostic accuracy efficacy Level 3: diagnostic thinking efficacy Level 4: therapeutic efficacy Level 5: patient outcome efficacy Level 6: societal efficacy
Imaging resolution Test sensitivity and specificity Pre- and post-test changes in subjectively determined outcome Effects of diagnostic on choice of therapy Value of test information including measures of morbidity, mortality, and quality-adjusted life years (QALYs) Cost-benefit and cost-effectiveness from the societal perspective
can contribute to rising health care costs. New imaging technologies are often adopted before being proven clinically effective, let alone cost effective. Almost invariably, new technologies are more expensive than those they replace. Cost-effectiveness analysis is a component of comparative effectiveness research that evaluates level 6 on the Fryback and Thornbury hierarchy, societal efficacy—the value of imaging technologies to society [2]. An example of costeffectiveness analysis in pediatric radiology is provided by Medina et al. [8], who investigated different diagnostic strategies for newborns with suspected occult spinal dysraphism. In this study, a decision-analytic model was constructed to compare the health outcomes and costs of MRI, ultrasound (US), radiographs and no imaging with close clinical followup. By modeling clinical outcomes and costs, the authors were able to show that the optimal strategy depends on the pretest probability of spinal dysraphism. For example, in high-risk newborns, MRI resulted in cost savings, whereas in low-risk infants, US or no imaging with close clinical follow-up was the preferred diagnostic strategy. If resources were limitless, we would not consider costs; we would choose imaging strategies solely on the basis of their clinical effectiveness. But resources are limited, and money spent on health care is not available for other priorities such as education, infrastructure and the environment. The United States spends more than any other country on health care as a percent of gross domestic product, nearly 18% in 2011. If the quality of U.S. health care reflected this cost, the price might be justifiable. Unfortunately, we have not been able to demonstrate better outcomes than nations that spend far less and, in many cases, our outcomes are poorer. For example, in 2011, the United States ranked 32nd in infant mortality and 26th in life expectancy among a list of developed nations while spending over than 50% more than the next highest-spending nation [9]. Too often, we are not getting a good value from our health care system. Cost-effectiveness analysis allows us to measure value by telling us how much benefit a treatment will provide relative to another and at what cost. The challenges to performing comparative effectiveness research in pediatric radiology are considerable but so are the potential rewards. By determining the most appropriate use of
imaging resources, we can improve patient care. The first step is to change our focus from determining whether we can effectively perform a given study to whether we should perform that study and what effect it will have on patient outcomes. We need to promote the methodologies of comparative effectiveness research by encouraging the application of these methods for presentation at our national and international meetings and for publication in our scientific journals. Successful comparative effectiveness research in pediatric radiology will require novel interdisciplinary collaborations and a continued focus on high-quality evidence-based research. Acknowledgments The authors wish to thank Markeda Wade in the Department of Scientific Publications at The University of Texas MD Anderson Cancer Center for editorial contributions that enhanced the quality of this manuscript. Conflicts of interest None
References 1. Pandharipande PV, Gazelle GS (2009) Comparative effectiveness research: what it means for radiology. Radiology 253:600–605 2. Fryback DG, Thornbury JR (1991) The efficacy of diagnostic imaging. Med Decis Making 11:88–94 3. Gazelle GS, Kessler L, Lee DWet al (2011) A framework for assessing the value of diagnostic imaging in the era of comparative effectiveness research. Radiology 261:692–698 4. Committee on Comparative Effectiveness Research Prioritization (2009) Report brief: initial national priorities for comparative effectiveness research. Institute of Medicine. http://www.iom.edu/ cerpriorities. Accessed 4 June 2014 5. Society for Medical Decision Making (2009) Background paper on comparative effectiveness research. http://www.smdm.org/news/ detail/smdm-board-background-paper-on-comparative-effectivenessresearch. Accessed 4 June 2014 6. Rawson JV (2011) Comparative effectiveness research in radiology: patients, physicians and policy makers. Acad Radiol 18:1067–1071 7. Drummond M, Griffin A, Tarricone R (2009) Economic evaluation for devices and drugs–same or different? Value Health 12:402–404 8. Medina LS, Crone K, Kuntz KM (2001) Newborns with suspected occult spinal dysraphism: a cost-effectiveness analysis of diagnostic strategies. Pediatrics 108:e101 9. (2013) OECD Health Data. http://www.oecd-ilibrary.org/socialissues-migration-health/health-key-tables-from-oecd_20758480. Accessed 3 December 2013
