CLINICAL TRIALS
Letter to the Editor
Making data from clinical trials available
Berlin et al.1 and Fergusson and Hebert2 propose to control misleading associations found in clinical trial data by controlling access to the data. They do not trust others ransacking the data for these misleading associations. Their concern is understandable because probability statements based on ransacking randomized trials are less reliable and there is no generally accepted convention for keeping those statements separate from the more reliable confirmatory statistical analyses. Misleading associations are created any time treatment has an effect and a variate is not balanced. The variate could be anything from the color of subjects’ shoes to their sex. If randomization by chance placed all of one category of a variate in one group and a different category in the other and the results of the treatment showed no overlap between the two groups, then the two categories would correlate perfectly with the outcome due to the use of randomization rather than to a causal relationship. We dismiss both examples as unrealistic but for different reasons. In the case of the shoes, it is the implausibility of shoe color being causal. We ignore it and use the data. In the case of the sex, we would not trust the data because we would suspect that selection bias had occurred. We lose agreement when the plausibility of the variates lies between those extremes and they are imbalanced. Our current solutions have been to repeat the study enough times to expose the misleading associations due to chance and rely on randomization to prevent selection bias. A better alternative is to use minimization to avoid the misleading associations due to chance imbalances and to selection bias. Berger3 has laid out the key variates that a cheater would have to unbalance to cause selection bias. Minimization can prevent that action directly by keeping those variates balanced or indirectly by noting who was associated with the biased assignments.4 Using minimization with the recommended limiting p values keeps the results of the primary statistical analysis and ransacking distinctive. The p values generated in the primary analysis are labeled with a less-than sign since the secondary variates being balanced to some degree may have decreased it. The p values from the secondary analysis are labeled with a greater-than sign since multiple hypotheses will have been considered and so are less reliable.5
Clinical Trials 2014, Vol. 11(6) 685 Ó The Author(s) 2014 Reprints and permissions: sagepub.co.uk/journalsPermissions.nav DOI: 10.1177/1740774514547100 ctj.sagepub.com
When minimization is used to prevent selection bias, randomization should no longer be added to it because it weakens minimization’s ability to balance. That ability can be further enhanced with Zhao’s method of paying attention only to variates that are out of balance by a selected degree.6 Thus, replacing randomization with minimization will result in fewer misleading associations and those found by ransacking will be more appropriately discounted with distinctive labeling. This should lessen concerns of those who do not want to release the data from clinical trials. Therefore, a renewed debate on replacing randomization with minimization is desirable. Donald R Taves University of Washington, Seattle, WA, USA Corresponding author: Donald R Taves, University of Washington, 1959 NE Pacific Street, Seattle, WA 98195, USA. Email: [email protected] References 1. Berlin JA, Morris S, Rockhold FA, et al. Bumps and bridges on the road to responsible sharing of clinical trial data. Clin Trials 2014; 11: 7–12. 2. Fergusson DA and Hebert PC. Commentary on Berlin et al. Clin Trials 2014; 11: 13–4. 3. Berger VW. Selection bias and covariate imbalances in randomized clinical trials. Chichester: John Wiley & Sons, 2005. 4. Taves DR. Advantages of minimization over randomization: bias and analysis. Prospect Biol Med, in review (Also presented in the debate between Zhao W and Taves DR Session 10 Debating the Balance and Adjustment of Baseline Covariates, Society of Clinical Trials meeting in May 2014). 5. Taves DR. Rank-Minimization with a two-step analysis should replace randomization in clinical trials. J Clin Epidemiol 2012; 65: 3–6. 6. Zhao W, Hill MD and Palesch Y. Minimal sufficient balance—a new strategy to balance baseline covariates and preserve randomness of treatment allocation. Stat Methods Med Res. Epub ahead of print 26 January 2012. DOI: 10.1177/0962280212436447, http://smm.sagepub. com/content/early/2012/01/24/0962280212436447
Downloaded from ctj.sagepub.com at PURDUE UNIV LIBRARY TSS on March 13, 2015
Letter to the Editor
Making data from clinical trials available
Berlin et al.1 and Fergusson and Hebert2 propose to control misleading associations found in clinical trial data by controlling access to the data. They do not trust others ransacking the data for these misleading associations. Their concern is understandable because probability statements based on ransacking randomized trials are less reliable and there is no generally accepted convention for keeping those statements separate from the more reliable confirmatory statistical analyses. Misleading associations are created any time treatment has an effect and a variate is not balanced. The variate could be anything from the color of subjects’ shoes to their sex. If randomization by chance placed all of one category of a variate in one group and a different category in the other and the results of the treatment showed no overlap between the two groups, then the two categories would correlate perfectly with the outcome due to the use of randomization rather than to a causal relationship. We dismiss both examples as unrealistic but for different reasons. In the case of the shoes, it is the implausibility of shoe color being causal. We ignore it and use the data. In the case of the sex, we would not trust the data because we would suspect that selection bias had occurred. We lose agreement when the plausibility of the variates lies between those extremes and they are imbalanced. Our current solutions have been to repeat the study enough times to expose the misleading associations due to chance and rely on randomization to prevent selection bias. A better alternative is to use minimization to avoid the misleading associations due to chance imbalances and to selection bias. Berger3 has laid out the key variates that a cheater would have to unbalance to cause selection bias. Minimization can prevent that action directly by keeping those variates balanced or indirectly by noting who was associated with the biased assignments.4 Using minimization with the recommended limiting p values keeps the results of the primary statistical analysis and ransacking distinctive. The p values generated in the primary analysis are labeled with a less-than sign since the secondary variates being balanced to some degree may have decreased it. The p values from the secondary analysis are labeled with a greater-than sign since multiple hypotheses will have been considered and so are less reliable.5
Clinical Trials 2014, Vol. 11(6) 685 Ó The Author(s) 2014 Reprints and permissions: sagepub.co.uk/journalsPermissions.nav DOI: 10.1177/1740774514547100 ctj.sagepub.com
When minimization is used to prevent selection bias, randomization should no longer be added to it because it weakens minimization’s ability to balance. That ability can be further enhanced with Zhao’s method of paying attention only to variates that are out of balance by a selected degree.6 Thus, replacing randomization with minimization will result in fewer misleading associations and those found by ransacking will be more appropriately discounted with distinctive labeling. This should lessen concerns of those who do not want to release the data from clinical trials. Therefore, a renewed debate on replacing randomization with minimization is desirable. Donald R Taves University of Washington, Seattle, WA, USA Corresponding author: Donald R Taves, University of Washington, 1959 NE Pacific Street, Seattle, WA 98195, USA. Email: [email protected] References 1. Berlin JA, Morris S, Rockhold FA, et al. Bumps and bridges on the road to responsible sharing of clinical trial data. Clin Trials 2014; 11: 7–12. 2. Fergusson DA and Hebert PC. Commentary on Berlin et al. Clin Trials 2014; 11: 13–4. 3. Berger VW. Selection bias and covariate imbalances in randomized clinical trials. Chichester: John Wiley & Sons, 2005. 4. Taves DR. Advantages of minimization over randomization: bias and analysis. Prospect Biol Med, in review (Also presented in the debate between Zhao W and Taves DR Session 10 Debating the Balance and Adjustment of Baseline Covariates, Society of Clinical Trials meeting in May 2014). 5. Taves DR. Rank-Minimization with a two-step analysis should replace randomization in clinical trials. J Clin Epidemiol 2012; 65: 3–6. 6. Zhao W, Hill MD and Palesch Y. Minimal sufficient balance—a new strategy to balance baseline covariates and preserve randomness of treatment allocation. Stat Methods Med Res. Epub ahead of print 26 January 2012. DOI: 10.1177/0962280212436447, http://smm.sagepub. com/content/early/2012/01/24/0962280212436447
Downloaded from ctj.sagepub.com at PURDUE UNIV LIBRARY TSS on March 13, 2015
