This article was downloaded by: [Stony Brook University] On: 19 October 2014, At: 18:47 Publisher: Routledge Informa Ltd Registered in England and Wales Registered Number: 1072954 Registered office: Mortimer House, 37-41 Mortimer Street, London W1T 3JH, UK
Multivariate Behavioral Research Publication details, including instructions for authors and subscription information: http://www.tandfonline.com/loi/hmbr20
A New Procedure to Test Mediation With Missing Data Through Nonparametric Bootstrapping and Multiple Imputation a
Wei Wu & Fan Jia
a
a
University of Kansas Published online: 01 Oct 2013.
To cite this article: Wei Wu & Fan Jia (2013) A New Procedure to Test Mediation With Missing Data Through Nonparametric Bootstrapping and Multiple Imputation, Multivariate Behavioral Research, 48:5, 663-691, DOI: 10.1080/00273171.2013.816235 To link to this article: http://dx.doi.org/10.1080/00273171.2013.816235
PLEASE SCROLL DOWN FOR ARTICLE Taylor & Francis makes every effort to ensure the accuracy of all the information (the “Content”) contained in the publications on our platform. However, Taylor & Francis, our agents, and our licensors make no representations or warranties whatsoever as to the accuracy, completeness, or suitability for any purpose of the Content. Any opinions and views expressed in this publication are the opinions and views of the authors, and are not the views of or endorsed by Taylor & Francis. The accuracy of the Content should not be relied upon and should be independently verified with primary sources of information. Taylor and Francis shall not be liable for any losses, actions, claims, proceedings, demands, costs, expenses, damages, and other liabilities whatsoever or howsoever caused arising directly or
indirectly in connection with, in relation to or arising out of the use of the Content.
Downloaded by [Stony Brook University] at 18:47 19 October 2014
This article may be used for research, teaching, and private study purposes. Any substantial or systematic reproduction, redistribution, reselling, loan, sub-licensing, systematic supply, or distribution in any form to anyone is expressly forbidden. Terms & Conditions of access and use can be found at http://www.tandfonline.com/page/terms-and-conditions
Downloaded by [Stony Brook University] at 18:47 19 October 2014
Multivariate Behavioral Research, 48:663–691, 2013 Copyright © Taylor & Francis Group, LLC ISSN: 0027-3171 print/1532-7906 online DOI: 10.1080/00273171.2013.816235
A New Procedure to Test Mediation With Missing Data Through Nonparametric Bootstrapping and Multiple Imputation Wei Wu and Fan Jia University of Kansas
This article proposes a new procedure to test mediation with the presence of missing data by combining nonparametric bootstrapping with multiple imputation (MI). This procedure performs MI first and then bootstrapping for each imputed data set. The proposed procedure is more computationally efficient than the procedure that performs bootstrapping first and then MI for each bootstrap sample. The validity of the procedure is evaluated using a simulation study under different sample size, missing data mechanism, missing data proportion, and shape of distribution conditions. The result suggests that the proposed procedure performs comparably to the procedure that combines bootstrapping with full information maximum likelihood under most conditions. However, caution needs to be taken when using this procedure to handle missing not-at-random or nonnormal data.
Mediation is often of central interest to researchers in social and behavioral science research because it explains the mechanism for how and why an effect occurs. Mediation is said to occur when the effect of an independent variable (X) on a dependent variable (Y) is transmitted by a third variable (M; Baron & Kenny, 1986; Sobel, 1982). M is termed a mediator as it mediates the effect of X on Y. Figure 1 shows a mediation model with a single mediator. A mediating effect (also termed an indirect effect) can be quantified by and tested through
Correspondence concerning this article should be addressed to Wei Wu, Psychology Department, University of Kansas, 1415 Jayhawk Boulevard, Lawrence, KS 66045-7556. E-mail: [email protected]
663
Downloaded by [Stony Brook University] at 18:47 19 October 2014
664
WU AND JIA
FIGURE 1 A cross-sectional mediation model. X is an independent variable. Y is a dependent variable. M is a mediator. ©Y and ©M are residuals.
the product of the two direct effects (denoted by ab): (a) the effect of X on M (a effect) and (b) the effect of M on Y controlling for X (b effect). A well-known challenge in testing ab is that ab does not follow a normal distribution, even when both a and b are normally distributed. The nonnormality of the distribution of ab does not usually affect the point estimate of ab (the expected value of ab is equal to the product of the expected value of a and expected value of b). However, it would result in biased statistical inference if ab is treated as normally distributed. Two classes of solutions have been proposed in past literature to solve the problem: (a) derive the standard error of ab analytically based on an approximation of the true distribution of ab (Meeker, Cornwell, & Aroian, 1981) and (b) use a resampling approach such as bootstrapping or jackknife to derive an empirical sampling distribution (ESD) of ab (Bollen & Stine, 1990; MacKinnon, Lockwood, & Williams, 2004; Shrout & Bolger, 2002). Confidence intervals (CIs) then can be derived from the ESD for hypothesis testing of ab. Different resampling methods and CIs have been examined in past research. Accordingly, the optimal solution would thus be nonparametric bootstrapping with bias corrected CI (BCCI) because it offers the highest power to detect a mediating effect while keeping the Type I error rate within a robust region (MacKinnon et al., 2004). Almost all of the previous simulation studies on mediation analyses are limited to complete data. The presence of missing data raises another layer of challenge. In addition to accounting for the nonnormal distribution of ab, missing data need to be handled appropriately at the same time. A variety of advanced missing data techniques have emerged in the last two decades. Among them, full information maximum likelihood (FIML) and multiple imputation (MI) have been most popular (Graham, 2009; Rubin, 1987; Schafer & Graham, 2002). If bootstrapping is used to handle nonnormality of ab and one of the missing data techniques is used to handle missing data, a question naturally arises: What is an appropriate procedure to combine the two types of statistical techniques? This
Downloaded by [Stony Brook University] at 18:47 19 October 2014
TEST MEDIATION WITH MISSING DATA
665
article aims to provide insights into the question by (a) discussing the existing procedures to combine bootstrapping and missing data techniques, (b) proposing a new procedure to combine bootstrapping with MI, and (c) evaluating the new procedure using a simulation study. The remainder of the article is organized in the following manner: First, nonparametric bootstrapping and BCCI are briefly reviewed along with the missing data mechanisms and techniques. The existing procedures to test mediation with missing data are then introduced, followed by a new procedure to combine bootstrapping with MI. Next, an empirical example is used to illustrate the new procedure, followed by a simulation study that evaluates the performance of the new procedure in comparison with the one that combines bootstrapping with FIML. The article concludes with a discussion of the implications and limitations of the simulation study as well as providing general recommendations to researchers.
NONPARAMETRIC BOOTSTRAPPING AND BIAS-CORRECTED CONFIDENCE INTERVALS A nonparametric bootstrapping procedure starts with drawing a large number of samples (with the same sample size of the original data) with replacement, assuming that the original sample is the population (Efron, 1979; Efron & Tibshirani, 1994). Here, B is used to indicate the number of bootstrap samples. A general guideline to achieve a smooth and stable sampling distribution is for B D 1,000. A hypothesized model is then fit to each of the bootstrap samples, resulting in B point estimates of ™ (denoted by ™O b s), which form the ESD of ™. Here ™ represents a specific parameter, which would be ab for mediation. BCCI O 1 / and ™.p O 2/ can then be derived from the ESD (MacKinnon et al., 2004). If ™.p represent the percentiles corresponding to lower and upper limits of a BCCI, p1 and p2 , the cumulative probabilities associated with the percentiles are defined in Equations (1) and (2) (Carpenter & Bithell, 2000; DiCiccio & Efron, 1996; Efron & Tibshirani, 1994). BCCI adjusts for bias in the median of the ESD of ™, which might not agree with the point estimate of ™ from the original sample (denoted by ™O 0 ). This bias occurs due to asymmetry of the ESD. p1 D ˆ.2z0 C z’=2 /
(1)
p2 D ˆ.2z0 C z1
(2)
’=2/
where zO0 D ˆ 1 .p.™O b ™O 0 //. Here ˆ is the cumulative distribution function of the standard normal distribution. ’ is the significance level, which is .05 for a 95% CI. z’=2 and z1 ’=2 are
666
WU AND JIA
Downloaded by [Stony Brook University] at 18:47 19 October 2014
the 100 ’=2 and 100.1 ’=2/ percentiles in the z distribution, which would be the 2.5th and 97.5th percentiles for a 95% CI. z0 is the bias-correction factor. z0 can be estimated by the percentile in a z distribution corresponding to the percentage of ™O b s that are smaller than the ™O 0 . If there is no bias and the ESD is symmetric, 50% of the bootstrap samples would have a ™O b that is smaller than ™O 0 . As a result, z0 is 0 and no adjustment is imposed. On the other hand, if more than 50% of the ™O b s are smaller/larger than ™O 0 , the CI is then shifted to the right/left to correct for the bias.
MISSING DATA MECHANISMS AND TECHNIQUES Missing data can occur due to different causes (missing data mechanisms). Suppose a variable Y contains two components—observed .Yobs / and missing data .Ymi ss /. The mechanism behind Ymi ss belongs to one or a combination of three categories: (a) missing completely at random (MCAR; probability of missingness is not dependent on Y or any variables correlated with Y ), (b) missing at random (MAR; probability of missingness is dependent on some observed variable(s) but not Ymi ss ), and (c) missing not at random (MNAR; probability of missingness is dependent on Ymi ss ; Rubin, 1987). FIML and MI are designed to deal with MCAR and MAR data. FIML, often referred to as a direct maximum likelihood estimator, estimates a model by maximizing the log likelihood function of model parameters given observed data ll.™jyobs / (Arbuckle, 1996; Enders & Bandalos, 2001). FIML handles missing data and model estimation at the same time by taking into account missing data patterns in the computation of ll.™jyobs /. MI, in comparison, deals with missing data through filling in missing data values (Schafer & Olsen, 1998). To capture the uncertainty in predicting missing data given the observed information, MI generates multiple imputed data sets; each contains a set of plausible values for the missing data. The tested model is then fit to each imputed data set, and the point and standard error estimates from the analyses are pooled across imputations to produce the final point and standard error estimates for the model parameters. Both FIML and MI provide unbiased parameter estimates, assuming multivariate normality and MAR (Little & Rubin, 1987, 2002; Schafer, 1997). In fact, the parameter estimates from FIML and MI are expected to be identical given that (a) the same tested model and variables are included in both approaches and (b) the number of imputations in MI approaches infinity (Collins, Schafer, & Kam, 2001; Graham, Olchowski, & Gilreath, 2007; Schafer & Graham, 2002). However, the two methods have their own pros and cons in practice. FIML is easier to implement as it is a one-step approach and available in most of
Downloaded by [Stony Brook University] at 18:47 19 October 2014
TEST MEDIATION WITH MISSING DATA
667
the commercial software packages (e.g., Mplus, LISREL, EQS, AMOS, SAS PROC CALIS, and lavaan in R). MI is more difficult as it involves multiple steps and none of the software packages have fully automated them. Despite this disadvantage, there are benefits of using MI, specifically its flexibility. A variety of imputation algorithms and imputation models are available that might provide a better treatment of categorical variables and nonparametric relationships among the variables (Asparouhov & Muthén, 2010a; White, Royston, & Wood, 2011). In addition, MI creates complete data sets. As a result, the statistical methods that work only with complete data can be applied. Under certain circumstances, MI has to be adopted (Enders, 2010; Gottschall, West, & Enders, 2012). One circumstance occurs when there are item-level missing data. To illustrate, suppose that the X, Y , and M in a mediation model are scale scores, each of which is computed by averaging over a few items. If some item scores are missing, the best way to handle the missing data would be imputing them at the item level and then calculating the scale scores from the imputed item scores. By doing this, the observed information in the data set is fully utilized. FIML is inferior in this case as it relies on the analysis model in which the item scores are not directly included. A more thorough comparison between FIML and MI can be found in Allison (2012) and Enders (2010). Another circumstance occurs when weighted least square estimation methods are used. For instance, when there are categorical dependent variables in a model, WLSMV (a robust weighted least square estimator) is often used to estimate the model. WLSMV, however, uses pairwise deletion to handle missing data, which provides unbiased parameter estimates only for a special type of MAR data: missing data are dependent on observed independent variables but not observed dependent variables in the model (Asparouhov & Muthén, 2010b). To handle more general MAR data, a better solution would be using MI to impute missing data and then WLSMV to analyze the imputed data (Asparouhov, & Muthén, 2010a, 2010b).
PROCEDURES TO TEST MEDIATION WITH MISSING DATA Having introduced bootstrapping and missing data techniques, we now address the procedures to combine them. Processes have been available to combine bootstrapping with FIML or MI. Similar to mediation analyses with complete data, the goal of these procedures is to create an ESD of ™, assuming that the original sample is the population. The only difference is that the original sample now contains missing data. As a result, the target ESD should reflect two sources of uncertainty: uncertainty due to missing data and uncertainty due to sampling fluctuation had the data been complete. Significance tests can then proceed as in the complete data case after the ESD is derived. Note that the descriptions
668
WU AND JIA
here focus on establishing the ESD, which is composed of a large number of point estimates of ™ from bootstrap samples (™O b s). The point estimate of ™ from the original sample (™O 0 ) is obtained through FIML or MI without bootstrapping. BCCI is computed following Equations (1) and (2) using the obtained ™O 0 and ™O b s.
Downloaded by [Stony Brook University] at 18:47 19 October 2014
Procedure 1: FIML Nested Within Bootstrapping Nonparametric bootstrapping has long been combined with FIML to correct for the standard error bias associated with nonnormal distributions of observed variables with the presence of missing data (Enders, 2001). The same procedure can also be used to derive CIs for a mediating effect. This procedure involves two steps: 1. Bootstrap step: B (e.g., 1,000) bootstrap samples are generated using nonparametric bootstrapping. Because there are missing data in the original sample, the missing data will also be carried to the bootstrap samples. 2. FIML step: FIML is used to estimate a target mediation model for each of the bootstrapped samples. This step generates B point estimates of ™ (™O b s), which form the target ESD. Because FIML is nested within bootstrap in this procedure, it is denoted by BOOT(FIML). Mplus and EQS have automated this procedure and can output BCCI at the same time. Procedure 2: MI Nested Within Bootstrapping The second procedure is designed when MI is used to deal with missing data (Wang, Zhang, & Tong, 2012; Zhang & Wang, 2013). This procedure is denoted by BOOT(MI) and involves the following four steps (see also Figure 2(a) for a visual presentation). 1. Bootstrap step: This step is the same as the bootstrap step in Procedure 1. 2. MI step: For each of the B bootstrap samples, MI is used to obtain K (e.g., 100) imputed data sets. This step generates B K (e.g., 1,000 100 D 100,000) imputed data sets. 3. Analysis step: The tested model is fit to each of the B K imputed data sets. 4. Pooling step: For each bootstrap sample, ™O b is calculated by averaging the O across the K imputed data sets following Rubin’s (1987) rules. This ™s step results in B ™O b s, which form the target ESD.
Downloaded by [Stony Brook University] at 18:47 19 October 2014
(a) BOOT(MI)
(b) MI(BOOT) FIGURE 2 Graphic display of BOOT(MI) and MI(BOOT). The subscript K indicates the O are Kth imputed data sets and B indicates the Bth bootstrap sample. imp D imputation; ™s parameter estimates from the analyses (color figure available online).
669
Downloaded by [Stony Brook University] at 18:47 19 October 2014
670
WU AND JIA
Based on the aforementioned description, it is clear that Procedures 1 and 2 differ only in terms of whether ™O b s are obtained through FIML or MI. Because the two missing data techniques are expected to provide identical point estimates under the conditions mentioned earlier, they are expected to yield comparable ESDs of ™. BOOT(MI) offers an option for researchers to test mediation when MI is used to manage missing data. However, a major disadvantage of BOOT(MI) is that it is very computationally intensive as it requires analyzing a large number of data sets. For example, 100,000 data sets are to be analyzed in total with B D 1,000 and K D 100. Furthermore, it requires running MI for each bootstrap sample. If there are 1,000 bootstrapped samples, the MI needs to be run 1,000 times. This would impose a tremendous amount of computational burden. To mitigate this problem, we propose a new procedure to combine bootstrapping and MI as follows.
Procedure 3: Bootstrapping Nested Within MI This procedure runs MI first, followed by bootstrapping for each of the imputed data sets. This procedure is denoted as MI(BOOT). In what follows we describe the steps to implement the procedure and the rationale behind the procedure (see also Figure 2(b) for a visual presentation). 1. MI step: Use MI to obtain K imputed data sets. Each imputed data set contains one set of plausible values for the missing data, which is a random draw from the predictive distribution of missing data p.ymi ss jyobs ). 2. Bootstrap step: For each of the K imputed data sets, generate B bootstrap samples using nonparametric bootstrapping. This step yields B K bootstrap samples. 3. Analysis step: Fit the target model to each bootstrap sample, which results in B K point estimates of ™. The ™O b s from the same imputed data set form a distribution of ™ that is conditional on the imputed data set .t/ .p.™jyobs ; ymi ss //. Here, t indicates tth imputation; this distribution is identified as a conditional empirical sampling distribution. With K imputations, K conditional distributions are obtained. 4. Mixing step: This step raises the question as to how to best utilize the K conditional distributions for the statistical inference of ™. One possibility is to derive a CI (e.g., BCCI) from each of the conditional distributions and then aggregate the CIs. It is not clear, however, how to best pool the CIs as the conditional distributions are not normal in this case. We propose a solution to bypass the problem by mixing the conditional distributions. Recall that the goal of bootstrapping is to obtain the ESD of ™ assuming
TEST MEDIATION WITH MISSING DATA
671
Downloaded by [Stony Brook University] at 18:47 19 October 2014
that R the observed sample is the population p.™jyobs /. Because p.™jyobs / D p.™jyobs ; ymi ss /p.ymi ss jyobs /dymi ss , it can be approximated by mixing p.™jyobs ; ymi ss / over multiple sets of missing values drawn from the predictive distribution of missing data p.ymi ss jpobs / (Tanner & Wong, 1987, 2010). In other words, the B K estimates of ™ are mixed together to approximate p.™jyobs /. The BCCI can then be calculated using the B K ™O b s along with the ™O 0 (point estimate from the original sample) obtained through MI. This proposed procedure is different from previous procedures in several ways. First, it manages missing data first and then bootstraps. The uncertainty due to missing data is taken into account by MI and the uncertainty due to sampling fluctuation—had the data been complete—is accounted for by bootstrapping. In contrast, the previous procedures bootstrap first and handle missing data second. The two sources of uncertainty are simultaneously taken care of through bootstrapping. Second, the ESD of ™ from MI(BOOT) contains B K (e.g., 100,000) ™O b s instead of B (e.g., 1,000) ™O b s. More important, MI(BOOT) requires only 1 run of MI and 1,000 runs of bootstraps whereas BOOT(MI) requires 1,000 runs of the MI algorithm and 1 run of bootstrap. Because MI is much more computationally costly than bootstrap, MI(BOOT) runs more efficiently than BOOT(MI). For example, for the five variables shown in Figure 2, one MI procedure took 0.73 s whereas one bootstrap procedure took only 0.05 s to finish on a 2.3 GHz quad-core PC. This time difference can further increase when there are a large number of variables in the imputation model.
EMPIRICAL EXAMPLE Using an empirical example, we now demonstrate the three procedures: BOOT (FIML), BOOT(MI), and MI(BOOT). This example used data from Y. Yuan and MacKinnon (2009; retrieved from http://odin.mdacc.tmc.edu/yyuan/), which is originally from Elliot et al. (2007). This data set contains measures on five variables with n D 354: intervention group membership (binary, X), knowledge of benefit of healthy dietary pre- and postintervention (continuous), and healthy dietary behavior pre- and postintervention (continuous). Y. Yuan and MacKinnon tested a mediation model, which assumes that the intervention effect on the change in healthy dietary behavior (Y, post pre) is mediated by the change in understanding of the benefit (M, post pre). There were no missing data in the data set. To demonstrate the procedures for testing mediation with missing data, we simulated missing data on both M and Y. The missingness was determined by the intervention group membership.
Downloaded by [Stony Brook University] at 18:47 19 October 2014
672
WU AND JIA
Missingness was imposed in such a way that the participants in the control group had 30% probability of having missing data and the participants in the intervention group had a 10% chance of having missing data. The resulting data set contained 20.62% missing data in total. The three procedures were then applied to the data and BCCI was calculated for each of them. As a comparison, the BCCI from nonparametric bootstrapping with the complete data (denoted by BOOT(ML)) is also reported. For all of the procedures, 1,000 bootstrap samples and 100 imputed data sets were requested whenever MI was performed. The results are summarized in Table 1. It suggests that FIML and MI produced almost identical point estimates of ab given that the number of imputed data sets was 100. The BCs from the three bootstrapping procedures were all close to one another although their width and limits differed slightly. The computing times for each of the procedures were also recorded (see Table 1). BOOT(FIML) finished in about 1 s. MI(BOOT) took 103.38 s to finish (less than 2 min). In contrast, BOOT(MI) was done in 1,063.92 s, approximately 10 times that spent by MI(BOOT). In the following, a simulation study is conducted to evaluate the performance of MI(BOOT) in comparison with BOOT(FIML). We chose BOOT(FIML) to validate MI(BOOT) based on three considerations: (a) it is easiest to implement; (b) FIML often served as a benchmark to evaluate other missing data techniques; and (c) because BOOT(FIML) is expected to be comparable to BOOT(MI) under the aforementioned conditions, the results from this comparison should be generalizable to BOOT(MI) under those conditions. However, the three procedures may have differential performance under suboptimal conditions such as nonnormality and small sample sizes. In those cases, researchers are encouraged to compare MI(BOOT) directly with BOOT(MI).
TABLE 1 Point and Interval Estimates of the Indirect Effect in the Empirical Example from BOOT(FIML), MI(BOOT), BOOT(MI), and BOOT(ML)
Point estimate BCCIa Computing timeb
BOOT(FIML)
MI(BOOT)
BOOT(MI)
BOOT(ML)
0.068 [0.019, 0.162] 1s
0.067 [0.017, 0.152] 103.38 s
0.067 [0.017, 0.150] 1,063.92 s
0.056 [0.016, 0.135]
Multivariate Behavioral Research Publication details, including instructions for authors and subscription information: http://www.tandfonline.com/loi/hmbr20
A New Procedure to Test Mediation With Missing Data Through Nonparametric Bootstrapping and Multiple Imputation a
Wei Wu & Fan Jia
a
a
University of Kansas Published online: 01 Oct 2013.
To cite this article: Wei Wu & Fan Jia (2013) A New Procedure to Test Mediation With Missing Data Through Nonparametric Bootstrapping and Multiple Imputation, Multivariate Behavioral Research, 48:5, 663-691, DOI: 10.1080/00273171.2013.816235 To link to this article: http://dx.doi.org/10.1080/00273171.2013.816235
PLEASE SCROLL DOWN FOR ARTICLE Taylor & Francis makes every effort to ensure the accuracy of all the information (the “Content”) contained in the publications on our platform. However, Taylor & Francis, our agents, and our licensors make no representations or warranties whatsoever as to the accuracy, completeness, or suitability for any purpose of the Content. Any opinions and views expressed in this publication are the opinions and views of the authors, and are not the views of or endorsed by Taylor & Francis. The accuracy of the Content should not be relied upon and should be independently verified with primary sources of information. Taylor and Francis shall not be liable for any losses, actions, claims, proceedings, demands, costs, expenses, damages, and other liabilities whatsoever or howsoever caused arising directly or
indirectly in connection with, in relation to or arising out of the use of the Content.
Downloaded by [Stony Brook University] at 18:47 19 October 2014
This article may be used for research, teaching, and private study purposes. Any substantial or systematic reproduction, redistribution, reselling, loan, sub-licensing, systematic supply, or distribution in any form to anyone is expressly forbidden. Terms & Conditions of access and use can be found at http://www.tandfonline.com/page/terms-and-conditions
Downloaded by [Stony Brook University] at 18:47 19 October 2014
Multivariate Behavioral Research, 48:663–691, 2013 Copyright © Taylor & Francis Group, LLC ISSN: 0027-3171 print/1532-7906 online DOI: 10.1080/00273171.2013.816235
A New Procedure to Test Mediation With Missing Data Through Nonparametric Bootstrapping and Multiple Imputation Wei Wu and Fan Jia University of Kansas
This article proposes a new procedure to test mediation with the presence of missing data by combining nonparametric bootstrapping with multiple imputation (MI). This procedure performs MI first and then bootstrapping for each imputed data set. The proposed procedure is more computationally efficient than the procedure that performs bootstrapping first and then MI for each bootstrap sample. The validity of the procedure is evaluated using a simulation study under different sample size, missing data mechanism, missing data proportion, and shape of distribution conditions. The result suggests that the proposed procedure performs comparably to the procedure that combines bootstrapping with full information maximum likelihood under most conditions. However, caution needs to be taken when using this procedure to handle missing not-at-random or nonnormal data.
Mediation is often of central interest to researchers in social and behavioral science research because it explains the mechanism for how and why an effect occurs. Mediation is said to occur when the effect of an independent variable (X) on a dependent variable (Y) is transmitted by a third variable (M; Baron & Kenny, 1986; Sobel, 1982). M is termed a mediator as it mediates the effect of X on Y. Figure 1 shows a mediation model with a single mediator. A mediating effect (also termed an indirect effect) can be quantified by and tested through
Correspondence concerning this article should be addressed to Wei Wu, Psychology Department, University of Kansas, 1415 Jayhawk Boulevard, Lawrence, KS 66045-7556. E-mail: [email protected]
663
Downloaded by [Stony Brook University] at 18:47 19 October 2014
664
WU AND JIA
FIGURE 1 A cross-sectional mediation model. X is an independent variable. Y is a dependent variable. M is a mediator. ©Y and ©M are residuals.
the product of the two direct effects (denoted by ab): (a) the effect of X on M (a effect) and (b) the effect of M on Y controlling for X (b effect). A well-known challenge in testing ab is that ab does not follow a normal distribution, even when both a and b are normally distributed. The nonnormality of the distribution of ab does not usually affect the point estimate of ab (the expected value of ab is equal to the product of the expected value of a and expected value of b). However, it would result in biased statistical inference if ab is treated as normally distributed. Two classes of solutions have been proposed in past literature to solve the problem: (a) derive the standard error of ab analytically based on an approximation of the true distribution of ab (Meeker, Cornwell, & Aroian, 1981) and (b) use a resampling approach such as bootstrapping or jackknife to derive an empirical sampling distribution (ESD) of ab (Bollen & Stine, 1990; MacKinnon, Lockwood, & Williams, 2004; Shrout & Bolger, 2002). Confidence intervals (CIs) then can be derived from the ESD for hypothesis testing of ab. Different resampling methods and CIs have been examined in past research. Accordingly, the optimal solution would thus be nonparametric bootstrapping with bias corrected CI (BCCI) because it offers the highest power to detect a mediating effect while keeping the Type I error rate within a robust region (MacKinnon et al., 2004). Almost all of the previous simulation studies on mediation analyses are limited to complete data. The presence of missing data raises another layer of challenge. In addition to accounting for the nonnormal distribution of ab, missing data need to be handled appropriately at the same time. A variety of advanced missing data techniques have emerged in the last two decades. Among them, full information maximum likelihood (FIML) and multiple imputation (MI) have been most popular (Graham, 2009; Rubin, 1987; Schafer & Graham, 2002). If bootstrapping is used to handle nonnormality of ab and one of the missing data techniques is used to handle missing data, a question naturally arises: What is an appropriate procedure to combine the two types of statistical techniques? This
Downloaded by [Stony Brook University] at 18:47 19 October 2014
TEST MEDIATION WITH MISSING DATA
665
article aims to provide insights into the question by (a) discussing the existing procedures to combine bootstrapping and missing data techniques, (b) proposing a new procedure to combine bootstrapping with MI, and (c) evaluating the new procedure using a simulation study. The remainder of the article is organized in the following manner: First, nonparametric bootstrapping and BCCI are briefly reviewed along with the missing data mechanisms and techniques. The existing procedures to test mediation with missing data are then introduced, followed by a new procedure to combine bootstrapping with MI. Next, an empirical example is used to illustrate the new procedure, followed by a simulation study that evaluates the performance of the new procedure in comparison with the one that combines bootstrapping with FIML. The article concludes with a discussion of the implications and limitations of the simulation study as well as providing general recommendations to researchers.
NONPARAMETRIC BOOTSTRAPPING AND BIAS-CORRECTED CONFIDENCE INTERVALS A nonparametric bootstrapping procedure starts with drawing a large number of samples (with the same sample size of the original data) with replacement, assuming that the original sample is the population (Efron, 1979; Efron & Tibshirani, 1994). Here, B is used to indicate the number of bootstrap samples. A general guideline to achieve a smooth and stable sampling distribution is for B D 1,000. A hypothesized model is then fit to each of the bootstrap samples, resulting in B point estimates of ™ (denoted by ™O b s), which form the ESD of ™. Here ™ represents a specific parameter, which would be ab for mediation. BCCI O 1 / and ™.p O 2/ can then be derived from the ESD (MacKinnon et al., 2004). If ™.p represent the percentiles corresponding to lower and upper limits of a BCCI, p1 and p2 , the cumulative probabilities associated with the percentiles are defined in Equations (1) and (2) (Carpenter & Bithell, 2000; DiCiccio & Efron, 1996; Efron & Tibshirani, 1994). BCCI adjusts for bias in the median of the ESD of ™, which might not agree with the point estimate of ™ from the original sample (denoted by ™O 0 ). This bias occurs due to asymmetry of the ESD. p1 D ˆ.2z0 C z’=2 /
(1)
p2 D ˆ.2z0 C z1
(2)
’=2/
where zO0 D ˆ 1 .p.™O b ™O 0 //. Here ˆ is the cumulative distribution function of the standard normal distribution. ’ is the significance level, which is .05 for a 95% CI. z’=2 and z1 ’=2 are
666
WU AND JIA
Downloaded by [Stony Brook University] at 18:47 19 October 2014
the 100 ’=2 and 100.1 ’=2/ percentiles in the z distribution, which would be the 2.5th and 97.5th percentiles for a 95% CI. z0 is the bias-correction factor. z0 can be estimated by the percentile in a z distribution corresponding to the percentage of ™O b s that are smaller than the ™O 0 . If there is no bias and the ESD is symmetric, 50% of the bootstrap samples would have a ™O b that is smaller than ™O 0 . As a result, z0 is 0 and no adjustment is imposed. On the other hand, if more than 50% of the ™O b s are smaller/larger than ™O 0 , the CI is then shifted to the right/left to correct for the bias.
MISSING DATA MECHANISMS AND TECHNIQUES Missing data can occur due to different causes (missing data mechanisms). Suppose a variable Y contains two components—observed .Yobs / and missing data .Ymi ss /. The mechanism behind Ymi ss belongs to one or a combination of three categories: (a) missing completely at random (MCAR; probability of missingness is not dependent on Y or any variables correlated with Y ), (b) missing at random (MAR; probability of missingness is dependent on some observed variable(s) but not Ymi ss ), and (c) missing not at random (MNAR; probability of missingness is dependent on Ymi ss ; Rubin, 1987). FIML and MI are designed to deal with MCAR and MAR data. FIML, often referred to as a direct maximum likelihood estimator, estimates a model by maximizing the log likelihood function of model parameters given observed data ll.™jyobs / (Arbuckle, 1996; Enders & Bandalos, 2001). FIML handles missing data and model estimation at the same time by taking into account missing data patterns in the computation of ll.™jyobs /. MI, in comparison, deals with missing data through filling in missing data values (Schafer & Olsen, 1998). To capture the uncertainty in predicting missing data given the observed information, MI generates multiple imputed data sets; each contains a set of plausible values for the missing data. The tested model is then fit to each imputed data set, and the point and standard error estimates from the analyses are pooled across imputations to produce the final point and standard error estimates for the model parameters. Both FIML and MI provide unbiased parameter estimates, assuming multivariate normality and MAR (Little & Rubin, 1987, 2002; Schafer, 1997). In fact, the parameter estimates from FIML and MI are expected to be identical given that (a) the same tested model and variables are included in both approaches and (b) the number of imputations in MI approaches infinity (Collins, Schafer, & Kam, 2001; Graham, Olchowski, & Gilreath, 2007; Schafer & Graham, 2002). However, the two methods have their own pros and cons in practice. FIML is easier to implement as it is a one-step approach and available in most of
Downloaded by [Stony Brook University] at 18:47 19 October 2014
TEST MEDIATION WITH MISSING DATA
667
the commercial software packages (e.g., Mplus, LISREL, EQS, AMOS, SAS PROC CALIS, and lavaan in R). MI is more difficult as it involves multiple steps and none of the software packages have fully automated them. Despite this disadvantage, there are benefits of using MI, specifically its flexibility. A variety of imputation algorithms and imputation models are available that might provide a better treatment of categorical variables and nonparametric relationships among the variables (Asparouhov & Muthén, 2010a; White, Royston, & Wood, 2011). In addition, MI creates complete data sets. As a result, the statistical methods that work only with complete data can be applied. Under certain circumstances, MI has to be adopted (Enders, 2010; Gottschall, West, & Enders, 2012). One circumstance occurs when there are item-level missing data. To illustrate, suppose that the X, Y , and M in a mediation model are scale scores, each of which is computed by averaging over a few items. If some item scores are missing, the best way to handle the missing data would be imputing them at the item level and then calculating the scale scores from the imputed item scores. By doing this, the observed information in the data set is fully utilized. FIML is inferior in this case as it relies on the analysis model in which the item scores are not directly included. A more thorough comparison between FIML and MI can be found in Allison (2012) and Enders (2010). Another circumstance occurs when weighted least square estimation methods are used. For instance, when there are categorical dependent variables in a model, WLSMV (a robust weighted least square estimator) is often used to estimate the model. WLSMV, however, uses pairwise deletion to handle missing data, which provides unbiased parameter estimates only for a special type of MAR data: missing data are dependent on observed independent variables but not observed dependent variables in the model (Asparouhov & Muthén, 2010b). To handle more general MAR data, a better solution would be using MI to impute missing data and then WLSMV to analyze the imputed data (Asparouhov, & Muthén, 2010a, 2010b).
PROCEDURES TO TEST MEDIATION WITH MISSING DATA Having introduced bootstrapping and missing data techniques, we now address the procedures to combine them. Processes have been available to combine bootstrapping with FIML or MI. Similar to mediation analyses with complete data, the goal of these procedures is to create an ESD of ™, assuming that the original sample is the population. The only difference is that the original sample now contains missing data. As a result, the target ESD should reflect two sources of uncertainty: uncertainty due to missing data and uncertainty due to sampling fluctuation had the data been complete. Significance tests can then proceed as in the complete data case after the ESD is derived. Note that the descriptions
668
WU AND JIA
here focus on establishing the ESD, which is composed of a large number of point estimates of ™ from bootstrap samples (™O b s). The point estimate of ™ from the original sample (™O 0 ) is obtained through FIML or MI without bootstrapping. BCCI is computed following Equations (1) and (2) using the obtained ™O 0 and ™O b s.
Downloaded by [Stony Brook University] at 18:47 19 October 2014
Procedure 1: FIML Nested Within Bootstrapping Nonparametric bootstrapping has long been combined with FIML to correct for the standard error bias associated with nonnormal distributions of observed variables with the presence of missing data (Enders, 2001). The same procedure can also be used to derive CIs for a mediating effect. This procedure involves two steps: 1. Bootstrap step: B (e.g., 1,000) bootstrap samples are generated using nonparametric bootstrapping. Because there are missing data in the original sample, the missing data will also be carried to the bootstrap samples. 2. FIML step: FIML is used to estimate a target mediation model for each of the bootstrapped samples. This step generates B point estimates of ™ (™O b s), which form the target ESD. Because FIML is nested within bootstrap in this procedure, it is denoted by BOOT(FIML). Mplus and EQS have automated this procedure and can output BCCI at the same time. Procedure 2: MI Nested Within Bootstrapping The second procedure is designed when MI is used to deal with missing data (Wang, Zhang, & Tong, 2012; Zhang & Wang, 2013). This procedure is denoted by BOOT(MI) and involves the following four steps (see also Figure 2(a) for a visual presentation). 1. Bootstrap step: This step is the same as the bootstrap step in Procedure 1. 2. MI step: For each of the B bootstrap samples, MI is used to obtain K (e.g., 100) imputed data sets. This step generates B K (e.g., 1,000 100 D 100,000) imputed data sets. 3. Analysis step: The tested model is fit to each of the B K imputed data sets. 4. Pooling step: For each bootstrap sample, ™O b is calculated by averaging the O across the K imputed data sets following Rubin’s (1987) rules. This ™s step results in B ™O b s, which form the target ESD.
Downloaded by [Stony Brook University] at 18:47 19 October 2014
(a) BOOT(MI)
(b) MI(BOOT) FIGURE 2 Graphic display of BOOT(MI) and MI(BOOT). The subscript K indicates the O are Kth imputed data sets and B indicates the Bth bootstrap sample. imp D imputation; ™s parameter estimates from the analyses (color figure available online).
669
Downloaded by [Stony Brook University] at 18:47 19 October 2014
670
WU AND JIA
Based on the aforementioned description, it is clear that Procedures 1 and 2 differ only in terms of whether ™O b s are obtained through FIML or MI. Because the two missing data techniques are expected to provide identical point estimates under the conditions mentioned earlier, they are expected to yield comparable ESDs of ™. BOOT(MI) offers an option for researchers to test mediation when MI is used to manage missing data. However, a major disadvantage of BOOT(MI) is that it is very computationally intensive as it requires analyzing a large number of data sets. For example, 100,000 data sets are to be analyzed in total with B D 1,000 and K D 100. Furthermore, it requires running MI for each bootstrap sample. If there are 1,000 bootstrapped samples, the MI needs to be run 1,000 times. This would impose a tremendous amount of computational burden. To mitigate this problem, we propose a new procedure to combine bootstrapping and MI as follows.
Procedure 3: Bootstrapping Nested Within MI This procedure runs MI first, followed by bootstrapping for each of the imputed data sets. This procedure is denoted as MI(BOOT). In what follows we describe the steps to implement the procedure and the rationale behind the procedure (see also Figure 2(b) for a visual presentation). 1. MI step: Use MI to obtain K imputed data sets. Each imputed data set contains one set of plausible values for the missing data, which is a random draw from the predictive distribution of missing data p.ymi ss jyobs ). 2. Bootstrap step: For each of the K imputed data sets, generate B bootstrap samples using nonparametric bootstrapping. This step yields B K bootstrap samples. 3. Analysis step: Fit the target model to each bootstrap sample, which results in B K point estimates of ™. The ™O b s from the same imputed data set form a distribution of ™ that is conditional on the imputed data set .t/ .p.™jyobs ; ymi ss //. Here, t indicates tth imputation; this distribution is identified as a conditional empirical sampling distribution. With K imputations, K conditional distributions are obtained. 4. Mixing step: This step raises the question as to how to best utilize the K conditional distributions for the statistical inference of ™. One possibility is to derive a CI (e.g., BCCI) from each of the conditional distributions and then aggregate the CIs. It is not clear, however, how to best pool the CIs as the conditional distributions are not normal in this case. We propose a solution to bypass the problem by mixing the conditional distributions. Recall that the goal of bootstrapping is to obtain the ESD of ™ assuming
TEST MEDIATION WITH MISSING DATA
671
Downloaded by [Stony Brook University] at 18:47 19 October 2014
that R the observed sample is the population p.™jyobs /. Because p.™jyobs / D p.™jyobs ; ymi ss /p.ymi ss jyobs /dymi ss , it can be approximated by mixing p.™jyobs ; ymi ss / over multiple sets of missing values drawn from the predictive distribution of missing data p.ymi ss jpobs / (Tanner & Wong, 1987, 2010). In other words, the B K estimates of ™ are mixed together to approximate p.™jyobs /. The BCCI can then be calculated using the B K ™O b s along with the ™O 0 (point estimate from the original sample) obtained through MI. This proposed procedure is different from previous procedures in several ways. First, it manages missing data first and then bootstraps. The uncertainty due to missing data is taken into account by MI and the uncertainty due to sampling fluctuation—had the data been complete—is accounted for by bootstrapping. In contrast, the previous procedures bootstrap first and handle missing data second. The two sources of uncertainty are simultaneously taken care of through bootstrapping. Second, the ESD of ™ from MI(BOOT) contains B K (e.g., 100,000) ™O b s instead of B (e.g., 1,000) ™O b s. More important, MI(BOOT) requires only 1 run of MI and 1,000 runs of bootstraps whereas BOOT(MI) requires 1,000 runs of the MI algorithm and 1 run of bootstrap. Because MI is much more computationally costly than bootstrap, MI(BOOT) runs more efficiently than BOOT(MI). For example, for the five variables shown in Figure 2, one MI procedure took 0.73 s whereas one bootstrap procedure took only 0.05 s to finish on a 2.3 GHz quad-core PC. This time difference can further increase when there are a large number of variables in the imputation model.
EMPIRICAL EXAMPLE Using an empirical example, we now demonstrate the three procedures: BOOT (FIML), BOOT(MI), and MI(BOOT). This example used data from Y. Yuan and MacKinnon (2009; retrieved from http://odin.mdacc.tmc.edu/yyuan/), which is originally from Elliot et al. (2007). This data set contains measures on five variables with n D 354: intervention group membership (binary, X), knowledge of benefit of healthy dietary pre- and postintervention (continuous), and healthy dietary behavior pre- and postintervention (continuous). Y. Yuan and MacKinnon tested a mediation model, which assumes that the intervention effect on the change in healthy dietary behavior (Y, post pre) is mediated by the change in understanding of the benefit (M, post pre). There were no missing data in the data set. To demonstrate the procedures for testing mediation with missing data, we simulated missing data on both M and Y. The missingness was determined by the intervention group membership.
Downloaded by [Stony Brook University] at 18:47 19 October 2014
672
WU AND JIA
Missingness was imposed in such a way that the participants in the control group had 30% probability of having missing data and the participants in the intervention group had a 10% chance of having missing data. The resulting data set contained 20.62% missing data in total. The three procedures were then applied to the data and BCCI was calculated for each of them. As a comparison, the BCCI from nonparametric bootstrapping with the complete data (denoted by BOOT(ML)) is also reported. For all of the procedures, 1,000 bootstrap samples and 100 imputed data sets were requested whenever MI was performed. The results are summarized in Table 1. It suggests that FIML and MI produced almost identical point estimates of ab given that the number of imputed data sets was 100. The BCs from the three bootstrapping procedures were all close to one another although their width and limits differed slightly. The computing times for each of the procedures were also recorded (see Table 1). BOOT(FIML) finished in about 1 s. MI(BOOT) took 103.38 s to finish (less than 2 min). In contrast, BOOT(MI) was done in 1,063.92 s, approximately 10 times that spent by MI(BOOT). In the following, a simulation study is conducted to evaluate the performance of MI(BOOT) in comparison with BOOT(FIML). We chose BOOT(FIML) to validate MI(BOOT) based on three considerations: (a) it is easiest to implement; (b) FIML often served as a benchmark to evaluate other missing data techniques; and (c) because BOOT(FIML) is expected to be comparable to BOOT(MI) under the aforementioned conditions, the results from this comparison should be generalizable to BOOT(MI) under those conditions. However, the three procedures may have differential performance under suboptimal conditions such as nonnormality and small sample sizes. In those cases, researchers are encouraged to compare MI(BOOT) directly with BOOT(MI).
TABLE 1 Point and Interval Estimates of the Indirect Effect in the Empirical Example from BOOT(FIML), MI(BOOT), BOOT(MI), and BOOT(ML)
Point estimate BCCIa Computing timeb
BOOT(FIML)
MI(BOOT)
BOOT(MI)
BOOT(ML)
0.068 [0.019, 0.162] 1s
0.067 [0.017, 0.152] 103.38 s
0.067 [0.017, 0.150] 1,063.92 s
0.056 [0.016, 0.135]
