Downloaded from http://ebmh.bmj.com/ on July 3, 2015 - Published by group.bmj.com
Evidence-Based Mental Health Online First, published on July 9, 2014 as 10.1136/eb-2014-101899 Statistics in pills
Missing outcome data in meta-analysis doi:10.1136/eb-2014-101899
Dimitris Mavridis,1,2 Anna Chaimani,1 Orestis Efthimiou,1 Georgia Salanti1 1 2
Department of Hygiene and Epidemiology, University of Ioannina School of Medicine, Ioannina, Greece Department of Primary Education, University of Ioannina, Ioannina, Greece
Correspondence to Dimitris Mavridis, [email protected]
ABSTRACT Missing outcome data is a common problem in mental health trials compromising the validity of results and leading to the loss of precision. The problem is accumulated in a meta-analysis of clinical trials with missing outcome data. MISSING DATA MECHANISMS The risk of bias due to missing outcome data depends on the reasons data are missing and more specifically on how the propensity for missing data depends on participants’ characteristics and outcomes.1 The Cochrane Collaboration regards missing outcome data as a major factor potentially affecting the credibility of a study. A popular taxonomy for missing data mechanisms distinguishes the following scenarios. ▸ Missing completely at random (MCAR): The propensity for missing outcome data is the same for all participants and is not related to participants’ characteristics or outcomes. ▸ Missing at random (MAR): The propensity for missing outcome data is related to participants’ observed characteristics but not to participants’ outcomes. ▸ Missing not at random (MNAR): The propensity for missing outcome data is related to participants’ outcomes. STATISTICAL METHODS TO ACCOUNT FOR MISSING OUTCOME DATA There is a plethora of simple and sophisticated approaches to account for missing outcome data in a trial. Common approaches include: ▸ Replacing missing values with the mean value of the observed data. ▸ Replacing a missing value with the last observed value (the so-called Last Observation Carried Forward, LOCF). This technique is routinely used in mental health trials. The aforementioned approaches underestimate SE of the treatment effect. ▸ Multiple imputation is a statistical technique for handling missing data in which all missing values are replaced using information from observed data. It gives valid results when missing values are MAR but not when they are MNAR. SYNTHESIS OF STUDIES WITH MISSING OUTCOME DATA Meta-analysts often assume that the missing outcome problem has been resolved at the trial level. Most clinical trials employ a complete
EvidCopyright Based Mental Health Augustauthor 2014 Vol (or 17 Notheir 3 Article
case analysis or suboptimal imputation techniques. The methods listed below can be used to account for missing outcome data in a single trial and in a meta-analysis. ▸ Complete case meta-analysis includes only those participants whose outcomes are known. This analysis gives unbiased results when missing data are MCAR but decreases precision and power. ▸ The best-case scenario assumes that all missing participants have a favourable outcome in the experimental group and poor outcomes in the control group; the worst-case scenario is the converse. The best-case/worst-case scenarios are often unrealistic and provide extreme results. ▸ A statistical model that relates missing data to observed data can be used to account for missing outcome values. Assumptions are needed about how much more likely is the outcome among participants who left the study compared with those who completed the study. Expert opinion is needed to quantify differences between treatment effects in completers and non-completers. A sensitivity analysis with increasingly stringent assumptions on how missing treatment effects are related to observed treatment effects is a sensible way to evaluate the robustness of results. ▸ Meta-analysis models that use informative missing parameters have been implemented in the metamiss command in STATA (http:// www.mrc-bsu.cam.ac.uk/software/stata-software/). Competing interests DM and GS received research funding from the European Research Council (IMMA 260559), AC and OE received funding from Greek national funds through the operational programme ‘Education and Lifelong Learning’ of the National Strategic Reference Framework (NSRF)–Research Funding Program: ARISTEIA. Investing in knowledge society through the European Social Fund.
REFERENCE 1.
Mavridis D, Chaimani A, Efthimiou O, et al. Addressing missing outcome data in meta-analysis. EBMH, 17.
employer) 2014. Produced by BMJ Publishing Group Ltd under licence. e4
Downloaded from http://ebmh.bmj.com/ on July 3, 2015 - Published by group.bmj.com
Missing outcome data in meta-analysis Dimitris Mavridis, Anna Chaimani, Orestis Efthimiou and Georgia Salanti Evid Based Mental Health published online July 9, 2014
Updated information and services can be found at: http://ebmh.bmj.com/content/early/2014/07/09/eb-2014-101899
These include:
Email alerting service
Receive free email alerts when new articles cite this article. Sign up in the box at the top right corner of the online article.
Notes
To request permissions go to: http://group.bmj.com/group/rights-licensing/permissions To order reprints go to: http://journals.bmj.com/cgi/reprintform To subscribe to BMJ go to: http://group.bmj.com/subscribe/
Evidence-Based Mental Health Online First, published on July 9, 2014 as 10.1136/eb-2014-101899 Statistics in pills
Missing outcome data in meta-analysis doi:10.1136/eb-2014-101899
Dimitris Mavridis,1,2 Anna Chaimani,1 Orestis Efthimiou,1 Georgia Salanti1 1 2
Department of Hygiene and Epidemiology, University of Ioannina School of Medicine, Ioannina, Greece Department of Primary Education, University of Ioannina, Ioannina, Greece
Correspondence to Dimitris Mavridis, [email protected]
ABSTRACT Missing outcome data is a common problem in mental health trials compromising the validity of results and leading to the loss of precision. The problem is accumulated in a meta-analysis of clinical trials with missing outcome data. MISSING DATA MECHANISMS The risk of bias due to missing outcome data depends on the reasons data are missing and more specifically on how the propensity for missing data depends on participants’ characteristics and outcomes.1 The Cochrane Collaboration regards missing outcome data as a major factor potentially affecting the credibility of a study. A popular taxonomy for missing data mechanisms distinguishes the following scenarios. ▸ Missing completely at random (MCAR): The propensity for missing outcome data is the same for all participants and is not related to participants’ characteristics or outcomes. ▸ Missing at random (MAR): The propensity for missing outcome data is related to participants’ observed characteristics but not to participants’ outcomes. ▸ Missing not at random (MNAR): The propensity for missing outcome data is related to participants’ outcomes. STATISTICAL METHODS TO ACCOUNT FOR MISSING OUTCOME DATA There is a plethora of simple and sophisticated approaches to account for missing outcome data in a trial. Common approaches include: ▸ Replacing missing values with the mean value of the observed data. ▸ Replacing a missing value with the last observed value (the so-called Last Observation Carried Forward, LOCF). This technique is routinely used in mental health trials. The aforementioned approaches underestimate SE of the treatment effect. ▸ Multiple imputation is a statistical technique for handling missing data in which all missing values are replaced using information from observed data. It gives valid results when missing values are MAR but not when they are MNAR. SYNTHESIS OF STUDIES WITH MISSING OUTCOME DATA Meta-analysts often assume that the missing outcome problem has been resolved at the trial level. Most clinical trials employ a complete
EvidCopyright Based Mental Health Augustauthor 2014 Vol (or 17 Notheir 3 Article
case analysis or suboptimal imputation techniques. The methods listed below can be used to account for missing outcome data in a single trial and in a meta-analysis. ▸ Complete case meta-analysis includes only those participants whose outcomes are known. This analysis gives unbiased results when missing data are MCAR but decreases precision and power. ▸ The best-case scenario assumes that all missing participants have a favourable outcome in the experimental group and poor outcomes in the control group; the worst-case scenario is the converse. The best-case/worst-case scenarios are often unrealistic and provide extreme results. ▸ A statistical model that relates missing data to observed data can be used to account for missing outcome values. Assumptions are needed about how much more likely is the outcome among participants who left the study compared with those who completed the study. Expert opinion is needed to quantify differences between treatment effects in completers and non-completers. A sensitivity analysis with increasingly stringent assumptions on how missing treatment effects are related to observed treatment effects is a sensible way to evaluate the robustness of results. ▸ Meta-analysis models that use informative missing parameters have been implemented in the metamiss command in STATA (http:// www.mrc-bsu.cam.ac.uk/software/stata-software/). Competing interests DM and GS received research funding from the European Research Council (IMMA 260559), AC and OE received funding from Greek national funds through the operational programme ‘Education and Lifelong Learning’ of the National Strategic Reference Framework (NSRF)–Research Funding Program: ARISTEIA. Investing in knowledge society through the European Social Fund.
REFERENCE 1.
Mavridis D, Chaimani A, Efthimiou O, et al. Addressing missing outcome data in meta-analysis. EBMH, 17.
employer) 2014. Produced by BMJ Publishing Group Ltd under licence. e4
Downloaded from http://ebmh.bmj.com/ on July 3, 2015 - Published by group.bmj.com
Missing outcome data in meta-analysis Dimitris Mavridis, Anna Chaimani, Orestis Efthimiou and Georgia Salanti Evid Based Mental Health published online July 9, 2014
Updated information and services can be found at: http://ebmh.bmj.com/content/early/2014/07/09/eb-2014-101899
These include:
Email alerting service
Receive free email alerts when new articles cite this article. Sign up in the box at the top right corner of the online article.
Notes
To request permissions go to: http://group.bmj.com/group/rights-licensing/permissions To order reprints go to: http://journals.bmj.com/cgi/reprintform To subscribe to BMJ go to: http://group.bmj.com/subscribe/
