Original Article
Quality of Patient-Reported Outcome Reporting Across Cancer Randomized Controlled Trials According to the CONSORT Patient-Reported Outcome Extension: A Pooled Analysis of 557 Trials Fabio Efficace, PhD1; Peter Fayers, PhD2; Andrea Pusic, MD3; Yeliz Cemal, MD3; Jane Yanagawa, MD4; Marc Jacobs, MSc5; Andrea la Sala, PhD6; Valentina Cafaro, MSc1; Katie Whale, MSc7; Jonathan Rees, MD7; and Jane Blazeby, MD7; for the European Organization for Research and Treatment of Cancer Quality-of-Life Group (Patient-Reported Outcome Measurements Over Time in Oncology Registry)
BACKGROUND: The main objectives of this study were to identify the number of randomized controlled trials (RCTs) including a patient-reported outcome (PRO) endpoint across a wide range of cancer specialties and to evaluate the completeness of PRO reporting according to the Consolidated Standards of Reporting Trials (CONSORT) PRO extension. METHODS: RCTs with a PRO endpoint that had been performed across several cancer specialties and published between 2004 and 2013 were considered. Studies were evaluated on the basis of previously defined criteria, including the CONSORT PRO extension and the Cochrane Collaboration’s tool for assessing the risk of bias of RCTs. Analyses were also conducted by the type of PRO endpoint (primary vs secondary) and by the cancer disease site. RESULTS: A total of 56,696 potentially eligible records were scrutinized, and 557 RCTs with a PRO evaluation, enrolling 254,677 patients overall, were identified. PROs were most frequently used in RCTs of breast (n 5 123), lung (n 5 85), and colorectal cancer (n 5 66). Overall, PROs were secondary endpoints in 421 RCTs (76%). Four of 6 evaluated CONSORT PRO items were documented in less than 50% of the RCTs. The level of reporting was higher in RCTs with a PRO as a primary endpoint. The presence of a supplementary report was the only statistically significant factor associated with greater completeness of reporting for both RCTs with PROs as primary endpoints (b 5 .19, P 5 .001) and RCTs with PROs as secondary endpoints (b 5 .30, P < .001). CONCLUSIONS: Implementation of the CONSORT PRO extension is equally important across all cancer specialties. Its use can also contribute to revealing the robust PRO design of some studies, which might be obscured by poor outcome reporting. Cancer C 2015 American Cancer Society. 2015;121:3335-42. V KEYWORDS: cancer, clinical trials, Consolidated Standards of Reporting Trials (CONSORT), patient-reported outcomes, quality of life.
INTRODUCTION Major progress has been made in the treatment of cancer, with a remarkable number of new drugs approved between 2012 and 2013 by the US Food and Drug Administration.1 Many of these clinical achievements stem from randomized controlled trials (RCTs), which are the gold standard by which health care professionals and policymakers make decisions regarding treatment effectiveness.2 Because methodological rigor is essential to the conduct and reporting of RCTs, the initial version of the Consolidated Standards of Reporting Trials (CONSORT) statement was originally published in 19963 with the aim of facilitating
Corresponding author: Fabio Efficace, PhD, Data Center and Health Outcomes Research Unit, Italian Group for Adult Hematologic Diseases, Via Benevento 6, 00161 Rome, Italy; Fax: (011) 39 06 4402516; [email protected] 1 Data Center and Health Outcomes Research Unit, Italian Group for Adult Hematologic Diseases, Rome, Italy; 2Institute of Applied Health Sciences, University of Aberdeen, Aberdeen, United Kingdom; 3Department of Surgery, Memorial Sloan Kettering Cancer Center, New York, New York; 4Department of Surgery, David Geffen School of Medicine, University of California Los Angeles, Los Angeles, California; 5Department of Medical Psychology, Academic Medical Center/University of Amsterdam, Amsterdam, The Netherlands; 6Laboratory of Molecular and Cellular Immunology, Scientific Hospitalization and Care Institution San Raffaele Pisana, Rome, Italy; 7Bristol Centre for Surgical Research, School of Social and Community Medicine, University of Bristol, Bristol, United Kingdom
Fabio Efficace, Peter Fayers, and Jane Blazeby contributed to the study concept and design. All authors contributed to the acquisition of data, the analysis and interpretation of data, the drafting of the article, and the critical revision of the article for important intellectual content, and all authors had full access to all of the data (including statistical reports and tables) in the study and take responsibility for the integrity of the data and the accuracy of the data analysis. We thank Salvatore Soldati and Alessandro Perreca (Italian Group for Adult Hematologic Diseases) for their assistance with the data management and data analysis. Fabio Efficace and Jane Blazeby were part of the Consolidated Standards of Reporting Trials (CONSORT) patient-reported outcome (PRO) group that developed the CONSORT PRO extension. Additional supporting information may be found in the online version of this article. DOI: 10.1002/cncr.29489, Received: January 26, 2015; Revised: April 1, 2015; Accepted: April 27, 2015, Published online June 16, 2015 in Wiley Online Library (wileyonlinelibrary.com)
Cancer
September 15, 2015
3335
Original Article
adherence with key methodological aspects and improving the completeness of reporting of RCTs. The CONSORT statement is now endorsed by nearly all biomedical journals, and its use has been associated with improved reporting of trials.4 The impact of medical treatment on patients’ quality of life has become increasingly recognized as an important factor in the management of cancer, and this is reflected by the fact that more cancer RCTs have come to include quality of life or other types of patient-reported outcomes (PROs)5 as endpoints. The inclusion of PROs in drug development has great potential for providing invaluable information for better understanding the effectiveness of novel therapies.6 However, unlike assessments of traditional trial clinical endpoints (eg, progression-free or overall survival), the implementation of PROs in RCTs has specific methodological challenges.7-9 To increase the transparency and consistency of PRO reporting in RCTs, the CONSORT group published the CONSORT PRO extension in 2013.10 The extent to which PRO studies are able to fulfill their potential in generating patient-reported data that better inform treatment decisions also depends on the quality of their PRO reporting. Analyzing 124 RCTs in medical oncology with a PRO component published in select journals, a recent work has shown that adherence to the CONSORT PRO extension is suboptimal.11 However, it is not known how many cancer RCTs conducted more recently have actually included PROs because a systematic quantification of such studies across a wide spectrum of cancer diagnoses, regardless of journal publication, is lacking. Also, for the studies using a PRO, less is known about the use of PROs as either primary or secondary outcomes and about whether the pattern of PRO reporting is different in these 2 scenarios. To determine the effects of these recommendations on scientific publications,10 pre-CONSORT PRO benchmark data are necessary as a baseline for future comparisons, and we aimed to provide these data for specific cancer populations. To guide investigators when they are disclosing the results of RCTs, it is important to identify which items of the CONSORT PRO extension are most in need of improved reporting and to examine whether differences in PRO reporting exist among cancer specialties. The main objectives of this study were to identify the number of RCTs including a PRO endpoint across a wide range of cancer specialties and to evaluate the completeness of their PRO reporting according to the CONSORT PRO extension. The secondary objectives were to describe the level of reporting by the type of PRO end3336
point and cancer disease site and to investigate which factors were associated with a higher level of reporting. MATERIALS AND METHODS Search Strategy for Identification of Studies
A systematic literature search for studies meeting predefined criteria was performed with January 2004 as the starting date in the following databases: PubMed/MEDLINE, the Cochrane library, PsycINFO, and PsycARTICLES. The end of study inclusion ranged from February 2012 (for colorectal cancer) to December 2013 (for melanoma). Details of the literature search and searching strategies for each disease site included are available from the authors. Some of the cancer sites and studies analyzed in this overview have been discussed more extensively in earlier reports.12-14 The search was started in 2004 because work before that date had previously investigated the quality of PRO reporting for the main cancer sites.15 Only articles published in English were considered. Details on the retrieved abstracts and selection process were documented according to the Preferred Reporting Items for Systematic Reviews and MetaAnalyses guidelines.16 Criteria for Considering Studies Types of participants
Participants included adult patients diagnosed with bladder, brain (primary), breast, colorectal, esophagogastric, gynecological, head and neck, kidney, liver (primary), pancreatic, prostate, or testicular cancer, leukemia (and myelodysplastic syndromes), melanoma, or non–small cell lung cancer. No restriction was applied for the disease stage. Types of interventions
RCTs comparing conventional medical treatments were considered. Studies dealing with psychosocial interventions or complementary alternative medicine were excluded. Types of outcome measures examined
All studies including a PRO, either as a primary or secondary endpoint, were considered. Studies considering only satisfaction with care or evaluating adherence to therapy were not included. Types of studies
All RCTs enrolling at least 50 patients (combined arms) were included. If the same RCT had multiple publications, relevant data from all articles associated with it were Cancer
September 15, 2015
CONSORT PRO Extension in Cancer Trials/Efficace et al
combined to retrieve necessary data. Conference abstracts and case reports were excluded. Methods of Data Collection
Data were gathered through the Patient-Reported Outcome Measurements Over Time in Oncology registry (http://promotion.gimema.it/). This is an electronic password–protected database that systematically collects several types of information on published PRO RCTs in oncology. Several trained reviewers were involved in independently screening abstracts and reviewing all identified eligible studies that populate the database. Each reviewer was provided with a personal password to access the online system and completed a predefined electronic data extraction form (eDEF) for each RCT meeting the study criteria.13 Details on information contained in the eDEF have been previously reported.13 Each RCT was independently evaluated by at least 2 reviewers, and a senior reviewer was consulted to reconcile differences and achieve a consensus. A final eDEF was then validated and entered into the database. Type of Information Analyzed
Data analyzed for the purpose of this article included 1) basic RCT demographics (eg, international study, sample size, and disease type and stage), 2) completeness of reporting according to the CONSORT PRO extension,10 and 3) bias assessed with the Cochrane risk-of-bias tool.17 Data Analysis
The overall frequency distribution for each item of the CONSORT PRO extension was calculated for all studies and separately by the type of PRO endpoint (primary vs secondary). For descriptive purposes and to allow comparisons of future research with our findings, we also reported the level of reporting in RCTs conducted for 5 major cancer disease sites: breast, colorectal, gynecological, lung, and prostate cancer.18 For the purpose of this analysis, a CONSORT PRO checklist score was developed. Each item received a score of 1 or 0 (yes or no, respectively), and an adjusted CONSORT PRO checklist score (ie, the raw score divided by the number of applicable questions) was used to estimate the completeness of reporting. In univariate analyses, a linear regression model (with the adjusted checklist score as a dependent variable) was fitted. After we checked for possible high correlations among variables to be included in the analyses (multiple collinearity), a stepwise multivariate regression analysis was performed with the adjusted checklist score as a dependent variable. Cancer
September 15, 2015
Analyses were conducted separately for RCTs with PROs as primary outcomes and RCTs with PROs as secondary outcomes to facilitate data interpretation. In both analyses, the selection of factors potentially associated with a higher level of reporting was based on previous literature and on writing committee consensus.7,15 The following covariates were considered: industry support (yes vs no), disease stage (metastatic disease vs earlier stage disease), PRO difference between treatment arms (yes vs no), supplementary reports (yes vs no), disease site (breast cancer vs prostate cancer, breast cancer vs colorectal cancer, breast cancer vs non–small cell lung cancer, breast cancer vs gynecological cancer, and breast cancer vs other disease), international study (yes vs no), year of publication, and overall study sample size (continuous). Statistically significant variables were identified at the 5% level via Wald-type chi-square tests for linear regression. On the basis of previous data,13 relations between the risk of bias and PRO quality reporting were also examined. All statistical analyses were 2-sided and were performed with SAS 9.2 for Windows (SAS Institute, Inc, Cary, NC). RESULTS A total of 56,696 potentially eligible articles were identified, 557 RCTs with a PRO endpoint were included, and data were extracted for analysis. These 557 RCTs enrolled 254,677 patients (Supporting Fig. 1 [see online supporting information]). The majority of the trials were conducted in the United States (n 5 117 or 21%) and the United Kingdom (n 5 82 or 15%), with Canada, France, Germany, Italy, and the Netherlands accounting for at least 5% each. RCTs conducted with breast cancer patients most frequently included a PRO component (n 5 123). Other RCTs often reporting PRO data were conducted with patients with non–small cell lung cancer (n 5 85), colorectal cancer (n 5 66), and prostate cancer (n 5 65). PROs were primary endpoints in 136 RCTs (24%), and for these, 15 supplementary reports (11%) in addition to the initial RCT publication were identified. For the 421 RCTs (76%) with PROs as secondary outcomes, 114 supplementary reports (27%) providing additional PRO data were identified. Overall, 212 studies (38%) were conducted in a multinational setting, and the majority (n 5 338 or 61%) included patients with metastatic disease (Table 1). Completeness of PRO Reporting Across Cancer Specialties and by PRO Endpoint
The 2 most frequently reported CONSORT PRO items were 1) reporting in the abstract that a PRO was an 3337
Original Article TABLE 1. Number of RCTs Identified and General Characteristics (n 5 557)
Cancer Specialty Breast NSCLC Colorectal Prostate Gynecological Head and neck Esophagogastric Pancreatic Leukemia/myelodysplastic syndromes Brain (primary) Kidney Melanoma Liver (primary) Bladder Testicular
RCTs (n 5 557), No.
Patients Included (n 5 254,677), No.a
Metastatic/ Advanced Disease (n 5 338), No. (%)b
International Studies (n 5 212), No. (%)
PRO Endpoint: Primary/Secondary, No. (%)
123 85 66 65 50 40 29 24 21 14 13 10 9 5 3
83,312 28,878 36,367 22,071 24,991 9947 5132 8417 5839 3113 5847 7099 10,725 671 2268
52 (42)c 82 (96)d 44 (67) 34 (52) 39 (78)e 27 (68)f 20 (69)g 18 (75) N/A N/A 12 (92) 2 (20) 7 (78) 0 1 (33)
52 (42) 21 (25) 25 (38) 19 (29) 21 (42) 9 (23) 7 (24) 15 (63) 12 (57) 9 (64) 9 (69) 8 (80) 2 (22) 0 3 (100)
38 (31)/85 (69) 18 (21)/67 (79) 12 (18)/54 (82) 26 (40)/39 (60) 8 (16)/42 (84) 13 (32)/27 (68) 10 (34)/19 (66) 2 (8)/22 (92) 3 (14)/18 (86) 0 (0)/14 (100) 1 (8)/12 (92) 1 (10)/9 (90) 2 (22)/7 (78) 2 (40)/3 (60) 0 (0)/3 (100)
Abbreviations: N/A, not applicable; NSCLC, non–small cell lung cancer; PRO, patient-reported outcome; RCT, randomized controlled trial. a The overall number of patients included in the original RCT, regardless of those actually completing a PRO assessment. b This category also include studies enrolling at least some patients with advanced or metastatic disease. c Four studies (3%) were judged unclear. d Two studies (2%) were judged unclear. e One study (2%) was judged unclear. f Five studies (12%) were judged unclear. g One study (3%) was judged unclear.
outcome of the study (n 5 452 or 81%) and 2) reporting the use of well-validated PRO instruments (n 5 424 or 76%). Out of all 557 studies, 66% used either the Functional Assessment of Cancer Therapy or the European Organization for Research and Treatment of Cancer instruments (data not shown). The remaining 4 items, however, were documented in less than 50% of the RCTs, with less than one-third reporting a PRO hypothesis (n 5 93 or 17%), details on statistical approaches for dealing with missing PRO data (n 5 113 or 20%), and methods for PRO data collection (n 5 133 or 24%). Overall, less than 5% of RCTs documented all items of the CONSORT PRO extension. The level of reporting was statistically significant higher in RCTs with PRO as a primary endpoint in 4 (P1b, P2b, P6aa, and P20/21) of the 6 items (Table 2). A trend toward a greater completeness of reporting for RCTs with a PRO as a primary endpoint was found. For example, the percentages of RCTs addressing only 2 items at most, were 58% and 35% for RCTs with PROs as secondary and primary endpoints, respectively (Supporting Fig. 2 [see online supporting information]). Multivariate Analysis of Factors Associated With Greater Completeness of Reporting
The presence of a supplementary report was the only statistically significant factor associated with a higher level of 3338
reporting for both RCTs with PROs as primary endpoints (b 5 .194, P 5 .001) and RCTs with PROs as secondary endpoints (b 5 .305, P < .001). Also, the presence of a PRO difference between treatment arms was significantly associated with greater completeness of reporting (b 5 .044, P 5 .041) in RCTs with PROs as secondary endpoints (Supporting Table 1 [see online supporting information]). For descriptive purposes, in Figure 1, we report the level of reporting of RCTs with a PRO as a secondary endpoint by the presence of a supplementary publication. For all of the items, a greater level of reporting was evident for those RCTs with a supplementary publication. For example, although discussion of PRO-specific limitations and implications for generalizability and clinical practice was addressed in only 26% of RCTs without an additional report, this issue was reported in 81% of those RCTs with an accompanying PRO article. Level of PRO Reporting by Major Cancer Specialty and by RCT Risk of Bias
Items P1b and P6a were documented in more than 60% of studies, regardless of disease sites. Conversely, items P6aa and P12a were reported in less than one-third of studies, regardless of disease sites (Supporting Table 2 [see online supporting information]). Cancer
September 15, 2015
Cancer
September 15, 2015 Missing PRO data are a potential source of bias. A number of methods for dealing with missing data are available with different strengths and limitations, which should be described to facilitate interpretation. PRO-specific limitations may influence the generalizability of results and use in clinical practice.
Explicitly identifying PROs in the RCT abstract will facilitate the indexing and identification of studies to inform clinical care and evidence synthesis. PRO measures may be multidimensional and may assess the patient status at several time points during an RCT. A prespecified hypothesis reduces the risk of multiple statistical testing and selective reporting of PROs based on statistically significant results. This information will allow readers to assess the validity, reliability, and appropriateness of the PRO being used. Different methods of data collection could lead to a potential bias when one is interpreting outcomes.
Brief Explanatory Texta
96 (71) 52 (38)
424 (76) 133 (24)
254 (46)
82 (60)
24 (18)
38 (28)
93 (17)
113 (20)
129 (95)
452 (81)
172 (41)
89 (21)
81 (19)
328 (78)
55 (13)
323 (77)
Secondary Endpoint (n 5 421 or 76%), No. (%)
Quality of Patient-Reported Outcome Reporting Across Cancer Randomized Controlled Trials According to the CONSORT Patient-Reported Outcome Extension: A Pooled Analysis of 557 Trials Fabio Efficace, PhD1; Peter Fayers, PhD2; Andrea Pusic, MD3; Yeliz Cemal, MD3; Jane Yanagawa, MD4; Marc Jacobs, MSc5; Andrea la Sala, PhD6; Valentina Cafaro, MSc1; Katie Whale, MSc7; Jonathan Rees, MD7; and Jane Blazeby, MD7; for the European Organization for Research and Treatment of Cancer Quality-of-Life Group (Patient-Reported Outcome Measurements Over Time in Oncology Registry)
BACKGROUND: The main objectives of this study were to identify the number of randomized controlled trials (RCTs) including a patient-reported outcome (PRO) endpoint across a wide range of cancer specialties and to evaluate the completeness of PRO reporting according to the Consolidated Standards of Reporting Trials (CONSORT) PRO extension. METHODS: RCTs with a PRO endpoint that had been performed across several cancer specialties and published between 2004 and 2013 were considered. Studies were evaluated on the basis of previously defined criteria, including the CONSORT PRO extension and the Cochrane Collaboration’s tool for assessing the risk of bias of RCTs. Analyses were also conducted by the type of PRO endpoint (primary vs secondary) and by the cancer disease site. RESULTS: A total of 56,696 potentially eligible records were scrutinized, and 557 RCTs with a PRO evaluation, enrolling 254,677 patients overall, were identified. PROs were most frequently used in RCTs of breast (n 5 123), lung (n 5 85), and colorectal cancer (n 5 66). Overall, PROs were secondary endpoints in 421 RCTs (76%). Four of 6 evaluated CONSORT PRO items were documented in less than 50% of the RCTs. The level of reporting was higher in RCTs with a PRO as a primary endpoint. The presence of a supplementary report was the only statistically significant factor associated with greater completeness of reporting for both RCTs with PROs as primary endpoints (b 5 .19, P 5 .001) and RCTs with PROs as secondary endpoints (b 5 .30, P < .001). CONCLUSIONS: Implementation of the CONSORT PRO extension is equally important across all cancer specialties. Its use can also contribute to revealing the robust PRO design of some studies, which might be obscured by poor outcome reporting. Cancer C 2015 American Cancer Society. 2015;121:3335-42. V KEYWORDS: cancer, clinical trials, Consolidated Standards of Reporting Trials (CONSORT), patient-reported outcomes, quality of life.
INTRODUCTION Major progress has been made in the treatment of cancer, with a remarkable number of new drugs approved between 2012 and 2013 by the US Food and Drug Administration.1 Many of these clinical achievements stem from randomized controlled trials (RCTs), which are the gold standard by which health care professionals and policymakers make decisions regarding treatment effectiveness.2 Because methodological rigor is essential to the conduct and reporting of RCTs, the initial version of the Consolidated Standards of Reporting Trials (CONSORT) statement was originally published in 19963 with the aim of facilitating
Corresponding author: Fabio Efficace, PhD, Data Center and Health Outcomes Research Unit, Italian Group for Adult Hematologic Diseases, Via Benevento 6, 00161 Rome, Italy; Fax: (011) 39 06 4402516; [email protected] 1 Data Center and Health Outcomes Research Unit, Italian Group for Adult Hematologic Diseases, Rome, Italy; 2Institute of Applied Health Sciences, University of Aberdeen, Aberdeen, United Kingdom; 3Department of Surgery, Memorial Sloan Kettering Cancer Center, New York, New York; 4Department of Surgery, David Geffen School of Medicine, University of California Los Angeles, Los Angeles, California; 5Department of Medical Psychology, Academic Medical Center/University of Amsterdam, Amsterdam, The Netherlands; 6Laboratory of Molecular and Cellular Immunology, Scientific Hospitalization and Care Institution San Raffaele Pisana, Rome, Italy; 7Bristol Centre for Surgical Research, School of Social and Community Medicine, University of Bristol, Bristol, United Kingdom
Fabio Efficace, Peter Fayers, and Jane Blazeby contributed to the study concept and design. All authors contributed to the acquisition of data, the analysis and interpretation of data, the drafting of the article, and the critical revision of the article for important intellectual content, and all authors had full access to all of the data (including statistical reports and tables) in the study and take responsibility for the integrity of the data and the accuracy of the data analysis. We thank Salvatore Soldati and Alessandro Perreca (Italian Group for Adult Hematologic Diseases) for their assistance with the data management and data analysis. Fabio Efficace and Jane Blazeby were part of the Consolidated Standards of Reporting Trials (CONSORT) patient-reported outcome (PRO) group that developed the CONSORT PRO extension. Additional supporting information may be found in the online version of this article. DOI: 10.1002/cncr.29489, Received: January 26, 2015; Revised: April 1, 2015; Accepted: April 27, 2015, Published online June 16, 2015 in Wiley Online Library (wileyonlinelibrary.com)
Cancer
September 15, 2015
3335
Original Article
adherence with key methodological aspects and improving the completeness of reporting of RCTs. The CONSORT statement is now endorsed by nearly all biomedical journals, and its use has been associated with improved reporting of trials.4 The impact of medical treatment on patients’ quality of life has become increasingly recognized as an important factor in the management of cancer, and this is reflected by the fact that more cancer RCTs have come to include quality of life or other types of patient-reported outcomes (PROs)5 as endpoints. The inclusion of PROs in drug development has great potential for providing invaluable information for better understanding the effectiveness of novel therapies.6 However, unlike assessments of traditional trial clinical endpoints (eg, progression-free or overall survival), the implementation of PROs in RCTs has specific methodological challenges.7-9 To increase the transparency and consistency of PRO reporting in RCTs, the CONSORT group published the CONSORT PRO extension in 2013.10 The extent to which PRO studies are able to fulfill their potential in generating patient-reported data that better inform treatment decisions also depends on the quality of their PRO reporting. Analyzing 124 RCTs in medical oncology with a PRO component published in select journals, a recent work has shown that adherence to the CONSORT PRO extension is suboptimal.11 However, it is not known how many cancer RCTs conducted more recently have actually included PROs because a systematic quantification of such studies across a wide spectrum of cancer diagnoses, regardless of journal publication, is lacking. Also, for the studies using a PRO, less is known about the use of PROs as either primary or secondary outcomes and about whether the pattern of PRO reporting is different in these 2 scenarios. To determine the effects of these recommendations on scientific publications,10 pre-CONSORT PRO benchmark data are necessary as a baseline for future comparisons, and we aimed to provide these data for specific cancer populations. To guide investigators when they are disclosing the results of RCTs, it is important to identify which items of the CONSORT PRO extension are most in need of improved reporting and to examine whether differences in PRO reporting exist among cancer specialties. The main objectives of this study were to identify the number of RCTs including a PRO endpoint across a wide range of cancer specialties and to evaluate the completeness of their PRO reporting according to the CONSORT PRO extension. The secondary objectives were to describe the level of reporting by the type of PRO end3336
point and cancer disease site and to investigate which factors were associated with a higher level of reporting. MATERIALS AND METHODS Search Strategy for Identification of Studies
A systematic literature search for studies meeting predefined criteria was performed with January 2004 as the starting date in the following databases: PubMed/MEDLINE, the Cochrane library, PsycINFO, and PsycARTICLES. The end of study inclusion ranged from February 2012 (for colorectal cancer) to December 2013 (for melanoma). Details of the literature search and searching strategies for each disease site included are available from the authors. Some of the cancer sites and studies analyzed in this overview have been discussed more extensively in earlier reports.12-14 The search was started in 2004 because work before that date had previously investigated the quality of PRO reporting for the main cancer sites.15 Only articles published in English were considered. Details on the retrieved abstracts and selection process were documented according to the Preferred Reporting Items for Systematic Reviews and MetaAnalyses guidelines.16 Criteria for Considering Studies Types of participants
Participants included adult patients diagnosed with bladder, brain (primary), breast, colorectal, esophagogastric, gynecological, head and neck, kidney, liver (primary), pancreatic, prostate, or testicular cancer, leukemia (and myelodysplastic syndromes), melanoma, or non–small cell lung cancer. No restriction was applied for the disease stage. Types of interventions
RCTs comparing conventional medical treatments were considered. Studies dealing with psychosocial interventions or complementary alternative medicine were excluded. Types of outcome measures examined
All studies including a PRO, either as a primary or secondary endpoint, were considered. Studies considering only satisfaction with care or evaluating adherence to therapy were not included. Types of studies
All RCTs enrolling at least 50 patients (combined arms) were included. If the same RCT had multiple publications, relevant data from all articles associated with it were Cancer
September 15, 2015
CONSORT PRO Extension in Cancer Trials/Efficace et al
combined to retrieve necessary data. Conference abstracts and case reports were excluded. Methods of Data Collection
Data were gathered through the Patient-Reported Outcome Measurements Over Time in Oncology registry (http://promotion.gimema.it/). This is an electronic password–protected database that systematically collects several types of information on published PRO RCTs in oncology. Several trained reviewers were involved in independently screening abstracts and reviewing all identified eligible studies that populate the database. Each reviewer was provided with a personal password to access the online system and completed a predefined electronic data extraction form (eDEF) for each RCT meeting the study criteria.13 Details on information contained in the eDEF have been previously reported.13 Each RCT was independently evaluated by at least 2 reviewers, and a senior reviewer was consulted to reconcile differences and achieve a consensus. A final eDEF was then validated and entered into the database. Type of Information Analyzed
Data analyzed for the purpose of this article included 1) basic RCT demographics (eg, international study, sample size, and disease type and stage), 2) completeness of reporting according to the CONSORT PRO extension,10 and 3) bias assessed with the Cochrane risk-of-bias tool.17 Data Analysis
The overall frequency distribution for each item of the CONSORT PRO extension was calculated for all studies and separately by the type of PRO endpoint (primary vs secondary). For descriptive purposes and to allow comparisons of future research with our findings, we also reported the level of reporting in RCTs conducted for 5 major cancer disease sites: breast, colorectal, gynecological, lung, and prostate cancer.18 For the purpose of this analysis, a CONSORT PRO checklist score was developed. Each item received a score of 1 or 0 (yes or no, respectively), and an adjusted CONSORT PRO checklist score (ie, the raw score divided by the number of applicable questions) was used to estimate the completeness of reporting. In univariate analyses, a linear regression model (with the adjusted checklist score as a dependent variable) was fitted. After we checked for possible high correlations among variables to be included in the analyses (multiple collinearity), a stepwise multivariate regression analysis was performed with the adjusted checklist score as a dependent variable. Cancer
September 15, 2015
Analyses were conducted separately for RCTs with PROs as primary outcomes and RCTs with PROs as secondary outcomes to facilitate data interpretation. In both analyses, the selection of factors potentially associated with a higher level of reporting was based on previous literature and on writing committee consensus.7,15 The following covariates were considered: industry support (yes vs no), disease stage (metastatic disease vs earlier stage disease), PRO difference between treatment arms (yes vs no), supplementary reports (yes vs no), disease site (breast cancer vs prostate cancer, breast cancer vs colorectal cancer, breast cancer vs non–small cell lung cancer, breast cancer vs gynecological cancer, and breast cancer vs other disease), international study (yes vs no), year of publication, and overall study sample size (continuous). Statistically significant variables were identified at the 5% level via Wald-type chi-square tests for linear regression. On the basis of previous data,13 relations between the risk of bias and PRO quality reporting were also examined. All statistical analyses were 2-sided and were performed with SAS 9.2 for Windows (SAS Institute, Inc, Cary, NC). RESULTS A total of 56,696 potentially eligible articles were identified, 557 RCTs with a PRO endpoint were included, and data were extracted for analysis. These 557 RCTs enrolled 254,677 patients (Supporting Fig. 1 [see online supporting information]). The majority of the trials were conducted in the United States (n 5 117 or 21%) and the United Kingdom (n 5 82 or 15%), with Canada, France, Germany, Italy, and the Netherlands accounting for at least 5% each. RCTs conducted with breast cancer patients most frequently included a PRO component (n 5 123). Other RCTs often reporting PRO data were conducted with patients with non–small cell lung cancer (n 5 85), colorectal cancer (n 5 66), and prostate cancer (n 5 65). PROs were primary endpoints in 136 RCTs (24%), and for these, 15 supplementary reports (11%) in addition to the initial RCT publication were identified. For the 421 RCTs (76%) with PROs as secondary outcomes, 114 supplementary reports (27%) providing additional PRO data were identified. Overall, 212 studies (38%) were conducted in a multinational setting, and the majority (n 5 338 or 61%) included patients with metastatic disease (Table 1). Completeness of PRO Reporting Across Cancer Specialties and by PRO Endpoint
The 2 most frequently reported CONSORT PRO items were 1) reporting in the abstract that a PRO was an 3337
Original Article TABLE 1. Number of RCTs Identified and General Characteristics (n 5 557)
Cancer Specialty Breast NSCLC Colorectal Prostate Gynecological Head and neck Esophagogastric Pancreatic Leukemia/myelodysplastic syndromes Brain (primary) Kidney Melanoma Liver (primary) Bladder Testicular
RCTs (n 5 557), No.
Patients Included (n 5 254,677), No.a
Metastatic/ Advanced Disease (n 5 338), No. (%)b
International Studies (n 5 212), No. (%)
PRO Endpoint: Primary/Secondary, No. (%)
123 85 66 65 50 40 29 24 21 14 13 10 9 5 3
83,312 28,878 36,367 22,071 24,991 9947 5132 8417 5839 3113 5847 7099 10,725 671 2268
52 (42)c 82 (96)d 44 (67) 34 (52) 39 (78)e 27 (68)f 20 (69)g 18 (75) N/A N/A 12 (92) 2 (20) 7 (78) 0 1 (33)
52 (42) 21 (25) 25 (38) 19 (29) 21 (42) 9 (23) 7 (24) 15 (63) 12 (57) 9 (64) 9 (69) 8 (80) 2 (22) 0 3 (100)
38 (31)/85 (69) 18 (21)/67 (79) 12 (18)/54 (82) 26 (40)/39 (60) 8 (16)/42 (84) 13 (32)/27 (68) 10 (34)/19 (66) 2 (8)/22 (92) 3 (14)/18 (86) 0 (0)/14 (100) 1 (8)/12 (92) 1 (10)/9 (90) 2 (22)/7 (78) 2 (40)/3 (60) 0 (0)/3 (100)
Abbreviations: N/A, not applicable; NSCLC, non–small cell lung cancer; PRO, patient-reported outcome; RCT, randomized controlled trial. a The overall number of patients included in the original RCT, regardless of those actually completing a PRO assessment. b This category also include studies enrolling at least some patients with advanced or metastatic disease. c Four studies (3%) were judged unclear. d Two studies (2%) were judged unclear. e One study (2%) was judged unclear. f Five studies (12%) were judged unclear. g One study (3%) was judged unclear.
outcome of the study (n 5 452 or 81%) and 2) reporting the use of well-validated PRO instruments (n 5 424 or 76%). Out of all 557 studies, 66% used either the Functional Assessment of Cancer Therapy or the European Organization for Research and Treatment of Cancer instruments (data not shown). The remaining 4 items, however, were documented in less than 50% of the RCTs, with less than one-third reporting a PRO hypothesis (n 5 93 or 17%), details on statistical approaches for dealing with missing PRO data (n 5 113 or 20%), and methods for PRO data collection (n 5 133 or 24%). Overall, less than 5% of RCTs documented all items of the CONSORT PRO extension. The level of reporting was statistically significant higher in RCTs with PRO as a primary endpoint in 4 (P1b, P2b, P6aa, and P20/21) of the 6 items (Table 2). A trend toward a greater completeness of reporting for RCTs with a PRO as a primary endpoint was found. For example, the percentages of RCTs addressing only 2 items at most, were 58% and 35% for RCTs with PROs as secondary and primary endpoints, respectively (Supporting Fig. 2 [see online supporting information]). Multivariate Analysis of Factors Associated With Greater Completeness of Reporting
The presence of a supplementary report was the only statistically significant factor associated with a higher level of 3338
reporting for both RCTs with PROs as primary endpoints (b 5 .194, P 5 .001) and RCTs with PROs as secondary endpoints (b 5 .305, P < .001). Also, the presence of a PRO difference between treatment arms was significantly associated with greater completeness of reporting (b 5 .044, P 5 .041) in RCTs with PROs as secondary endpoints (Supporting Table 1 [see online supporting information]). For descriptive purposes, in Figure 1, we report the level of reporting of RCTs with a PRO as a secondary endpoint by the presence of a supplementary publication. For all of the items, a greater level of reporting was evident for those RCTs with a supplementary publication. For example, although discussion of PRO-specific limitations and implications for generalizability and clinical practice was addressed in only 26% of RCTs without an additional report, this issue was reported in 81% of those RCTs with an accompanying PRO article. Level of PRO Reporting by Major Cancer Specialty and by RCT Risk of Bias
Items P1b and P6a were documented in more than 60% of studies, regardless of disease sites. Conversely, items P6aa and P12a were reported in less than one-third of studies, regardless of disease sites (Supporting Table 2 [see online supporting information]). Cancer
September 15, 2015
Cancer
September 15, 2015 Missing PRO data are a potential source of bias. A number of methods for dealing with missing data are available with different strengths and limitations, which should be described to facilitate interpretation. PRO-specific limitations may influence the generalizability of results and use in clinical practice.
Explicitly identifying PROs in the RCT abstract will facilitate the indexing and identification of studies to inform clinical care and evidence synthesis. PRO measures may be multidimensional and may assess the patient status at several time points during an RCT. A prespecified hypothesis reduces the risk of multiple statistical testing and selective reporting of PROs based on statistically significant results. This information will allow readers to assess the validity, reliability, and appropriateness of the PRO being used. Different methods of data collection could lead to a potential bias when one is interpreting outcomes.
Brief Explanatory Texta
96 (71) 52 (38)
424 (76) 133 (24)
254 (46)
82 (60)
24 (18)
38 (28)
93 (17)
113 (20)
129 (95)
452 (81)
172 (41)
89 (21)
81 (19)
328 (78)
55 (13)
323 (77)
Secondary Endpoint (n 5 421 or 76%), No. (%)
