This article was downloaded by: [McGill University Library] On: 26 December 2012, At: 07:42 Publisher: Routledge Informa Ltd Registered in England and Wales Registered Number: 1072954 Registered office: Mortimer House, 37-41 Mortimer Street, London W1T 3JH, UK
Multivariate Behavioral Research Publication details, including instructions for authors and subscription information: http://www.tandfonline.com/loi/hmbr20
On the Equivalence of Factors and Components P.M. Bentler & Yutaka Kano Version of record first published: 10 Jun 2010.
To cite this article: P.M. Bentler & Yutaka Kano (1990): On the Equivalence of Factors and Components, Multivariate Behavioral Research, 25:1, 67-74 To link to this article: http://dx.doi.org/10.1207/s15327906mbr2501_8
PLEASE SCROLL DOWN FOR ARTICLE Full terms and conditions of use: http://www.tandfonline.com/page/termsand-conditions This article may be used for research, teaching, and private study purposes. Any substantial or systematic reproduction, redistribution, reselling, loan, sub-licensing, systematic supply, or distribution in any form to anyone is expressly forbidden. The publisher does not give any warranty express or implied or make any representation that the contents will be complete or accurate or up to date. The accuracy of any instructions, formulae, and drug doses should be independently verified with primary sources. The publisher shall not be liable for any loss, actions, claims, proceedings, demand, or costs or damages whatsoever or howsoever caused arising directly or indirectly in connection with or arising out of the use of this material.
Multivariate Behavioral Research, 25 (I ), 67-74 Copyright O 1990, Lawrence Erlbaurn Associates, Inc.
On the Equivalence of Factors and Components P. M. Bentler University of California, Los Angeles
Downloaded by [McGill University Library] at 07:42 26 December 2012
Yutaka Kano Osaka University
Debates on the virtues of common factor analysis versus principal component analysis, and their variations, go back about 50 years to the time of Thurstone and Hotelling. This is a debate that continues to be echoed in various scientific fields, for example, among references not cited by Velicer and Jackson (1990), Borgatta, Kercher, and Stull (1986) argue against the use of components, whereas Wilkinson (1 989) takes the opposite view. Velicer and Jackson do a commendable job of summarizing many of the issues that have been raised in this debate. We do not agree with their major conclusion "that there is little basis for preferring either component analysis or factor analysis," as we will explain. Then we prove a theorem that will help to clarify when component analysis and factor analysis are likely to yield similar results. Although similarity can thus be expected under certain conditions, a general equivalence of results by these methods under all conditions is not to be expected.
The Model Although the mathematics of the factor and component models are correctly described by Velicer and Jackson (1990), the major conceptual difference between the models needs further emphasis. Does one believe, in a particular application, that each variable to be analyzed is generated in part by a random error variate, and that these error variates are mutually uncorrelated? If so, the dimensionality of any meaningful model is greater than the dimensionality of the measured variables, and hence by definition a latent variable model, here, factor analysis, is called for. If one believes that some variables do not contain random errors, or that there are not as many random error variates as variables, This research was supported in part by USPHS grants DA00017 and DA01070, and by the Murata Overseas Scholarship Foundation. Address reprint requests to P. M. Bentler, Department of Psychology, UCLA, Los Angeles, CA 90024- 1563 or to Y. Kano, Department of Applied Mathematics, Faculty of Engineering Science, Osaka University, Toyonaka, Osaka 560, Japan.
JANUARY 1990
67
Downloaded by [McGill University Library] at 07:42 26 December 2012
P. Bentler and Y. Kano
then an alternative model, possibly component analysis under firther specifications, might be appropriate. In fact, it is our opinion that real data essentially always contain fully-dimensional random errors, that is, that a factor analysis model is essentially always preferable to a component model. Even though one may accept that factor analysis is the appropriate model to use in data analysis, this does not immediately imply that the model with a given number of factors is estimable or testable. For example, identification conditions may not be met. Furthermore, even if the model is testable, it may not be consistent with the data. Thus to apply factor analysis may be adifficult and not routine procedure. This is one of its virtues, as compared to components analysis, because the ability to reject a model is a fundamental aspect of data analysis using structural models (Bentler, 1989; Joreskog & Sorbom, 1988). In fact, the component model, or class of models as described by Velicer and Jackson, is not a model at all. As noted by Dunteman (1989, p. 56). "Principal components analysis is aprocedure to decompose the correlation matrix without regard to an underlying model." There is no hypothesis tested with the model. It is a nonfalsifiable procedure for analyzing data that can always be applied with any data set. Of course, there is a sampling theory for eigenvalues of a sample covariance matrix that can be used to test hypotheses about population roots of a covariance matrix (e.g., Anderson, 1963), but such hypotheses are typically hard to frame and not very informative about the structure of acorrelation matrix (discussed later in this article). In practical data analysis one can, of course, take the point of view that there is no single true model (De Leeuw, 1988). Such a point of view would deemphasize strict hypothesis testing and would suggest exploring alternative models that would, to a greater or lesser extent, yield parameter estimates that are stable and minimally biased. It would also be consistent with the use of fit indexes rather than probability values from a given goodness of fit test (Bentler. 1990; Mulaik et al., 1989). Within such a point of view, however. it still makes sense to us to define the class of models that might be of interest. We would argue that the error structure of component analysis is not appropriate to most real applications, and that the search for appropriate models should typically be limited to models with uncorrelated additive errors.
Invar-iancSeProperties Velicer and Jackson (1990) did not address component analysis versus factor analysis in the general case. They described these methods when applied to correlation matrices. What could one say about these methods as applied to covariance matrices, that is, to the same variables in a different metric'? One
MULTIVARIATE BEHAVIORAL RESEARCH
Downloaded by [McGill University Library] at 07:42 26 December 2012
P.Bentler and Y. Kano might hope that key aspects of any results are invariant with respect to such an arbitrary resealing. In the case of factor analysis, the effect of scale transformations is clear: the factor loadings are simply rescaled by the diagonal matrix involved. So, in Velicer and Jackson's Equation 1 , if one rescaled the variables q by a positive definite diagonal matrix D and analyzed D(q) instead of q, the factor loadings that result in theory, and are obtained with many methods of estimation, are DA instead of A. If the rescaled variables are analyzed by principal component analysis, on the other hand, no known relationship between the two sets of component results can be stated. Of course, image component analysis (their Equation 7) is scale invariant, a positive feature of this type of component analysis. A good discussion of some issues related to scale invariance is given by Cudeck (1989). A second aspect of invariance involves invariance of results with respect to a nonrandom sampling of subjects from the population of subjects. Although this is a complex topic, it is known that key features of a factor analysis solution (e.g., factor loadings, simple structure) can remain invariant under such selection (e.g., Meredith, 1964). This means that major conclusions will not be affected by such multivariate selection. In contrast, apparently no invariance results have been obtained for component solutions.
Estimation The computational economy, relatively speaking, of component analysis of a large number of variables makes the component procedure very attractive. It is certainly true that, historically speaking, the factor model has been much more expensive to estimate. As noted by Velicer and Jackson (1990), this situation has changed with the advent of noniterative estimators of the factor model (e.g., Hagglund, 1982; Bentler, 1982; Jennrich, 1986; Kano, 1989, in press). Unfortunately, these estimators are not well known or utilized. One may ask whether component analysis can be used to estimate some of the parameters of the factor analysis model (e.g., the factor loadings). As Velicer and Jackson's (1990) review makes clear, and as we will show below, sometimes this can be done. But in such applications the real interest remains on the factor model, and it should be recognized that component analysis is merely a computational convenience. Even Bartholomew's (1985) recent variant of component analysis is justified, essentially, on the basis of the factor analysis model (Bentler, 1985). Although in some applications there may be a genuine interest in the components, especially, principal components, these appear to be atypical applications.
JANUARY 1990
P. Bentler and Y. Kano
Ey~iivalenceof Component and Factor Analysis
Downloaded by [McGill University Library] at 07:42 26 December 2012
Many of the arguments for the use of component analysis are essentially practical. It has been found that results from application of both methods are quite similar. Velicer and Jackson (1 990) review some aspects of an analysis, such as the number of factors/components to retain for rotation, that are relevant to such similarity. Nonetheless, it appears that there are no results in the literature on precise conditions for any equivalence of outcome. We develop one such condition. Proposition Let a p-variate random variable Y have a factor analytic structure with a single common factor, that is, Y = at + E and Cov(Y) = C = aa' + Y.Let h be = 1 ). the largest eigenvalue of C, and o be the corresponding eigenvector (o'o Suppose a'a + ..,as p + os and that there exists a scalar yo > 0 such that yf, < y,, for all i, where y,is the i-th diagonal of 9'. Then (a) the correlation between the principal component o'Yand the common factor 5 converges to one a s p +m, (b) d h - + a a s p +w, that is, the principal component loading vector becomes equivalent to the factor loading vector.
By assumptions we have
and
(2)
Cw = (aa'+ Y)w = hw.
The squared correlation between o'Yand 5 can be written as
By using Equation 1, Equation 2 and the assumption that y,< w,,,
MULTIVARIATE BEHAVIORAL RESEARCH
P. Bentler and Y. Kano
This proves (a). It follows from Equation 2 that (XI = c x a , say. Then
-
Y)w = (a'@) x a
Downloaded by [McGill University Library] at 07:42 26 December 2012
with c = [al(hI- Y)-2a]-'i2.Substituting Equation 3 into Equation 2 means that ( a a ' + Y)(hI - Y)-la = h(hI - '4')-la, or
Let a = (a,, ..., ap)'. Assuming that a, # 0 and comparing the first elements of both-hand sides of Equation 4, we get
because Y/h+ 0 and (I - Y/h)-' - + Ias p
P
P
a?v,
= C i=l (A-v,)'
+w.
Multivariate Behavioral Research Publication details, including instructions for authors and subscription information: http://www.tandfonline.com/loi/hmbr20
On the Equivalence of Factors and Components P.M. Bentler & Yutaka Kano Version of record first published: 10 Jun 2010.
To cite this article: P.M. Bentler & Yutaka Kano (1990): On the Equivalence of Factors and Components, Multivariate Behavioral Research, 25:1, 67-74 To link to this article: http://dx.doi.org/10.1207/s15327906mbr2501_8
PLEASE SCROLL DOWN FOR ARTICLE Full terms and conditions of use: http://www.tandfonline.com/page/termsand-conditions This article may be used for research, teaching, and private study purposes. Any substantial or systematic reproduction, redistribution, reselling, loan, sub-licensing, systematic supply, or distribution in any form to anyone is expressly forbidden. The publisher does not give any warranty express or implied or make any representation that the contents will be complete or accurate or up to date. The accuracy of any instructions, formulae, and drug doses should be independently verified with primary sources. The publisher shall not be liable for any loss, actions, claims, proceedings, demand, or costs or damages whatsoever or howsoever caused arising directly or indirectly in connection with or arising out of the use of this material.
Multivariate Behavioral Research, 25 (I ), 67-74 Copyright O 1990, Lawrence Erlbaurn Associates, Inc.
On the Equivalence of Factors and Components P. M. Bentler University of California, Los Angeles
Downloaded by [McGill University Library] at 07:42 26 December 2012
Yutaka Kano Osaka University
Debates on the virtues of common factor analysis versus principal component analysis, and their variations, go back about 50 years to the time of Thurstone and Hotelling. This is a debate that continues to be echoed in various scientific fields, for example, among references not cited by Velicer and Jackson (1990), Borgatta, Kercher, and Stull (1986) argue against the use of components, whereas Wilkinson (1 989) takes the opposite view. Velicer and Jackson do a commendable job of summarizing many of the issues that have been raised in this debate. We do not agree with their major conclusion "that there is little basis for preferring either component analysis or factor analysis," as we will explain. Then we prove a theorem that will help to clarify when component analysis and factor analysis are likely to yield similar results. Although similarity can thus be expected under certain conditions, a general equivalence of results by these methods under all conditions is not to be expected.
The Model Although the mathematics of the factor and component models are correctly described by Velicer and Jackson (1990), the major conceptual difference between the models needs further emphasis. Does one believe, in a particular application, that each variable to be analyzed is generated in part by a random error variate, and that these error variates are mutually uncorrelated? If so, the dimensionality of any meaningful model is greater than the dimensionality of the measured variables, and hence by definition a latent variable model, here, factor analysis, is called for. If one believes that some variables do not contain random errors, or that there are not as many random error variates as variables, This research was supported in part by USPHS grants DA00017 and DA01070, and by the Murata Overseas Scholarship Foundation. Address reprint requests to P. M. Bentler, Department of Psychology, UCLA, Los Angeles, CA 90024- 1563 or to Y. Kano, Department of Applied Mathematics, Faculty of Engineering Science, Osaka University, Toyonaka, Osaka 560, Japan.
JANUARY 1990
67
Downloaded by [McGill University Library] at 07:42 26 December 2012
P. Bentler and Y. Kano
then an alternative model, possibly component analysis under firther specifications, might be appropriate. In fact, it is our opinion that real data essentially always contain fully-dimensional random errors, that is, that a factor analysis model is essentially always preferable to a component model. Even though one may accept that factor analysis is the appropriate model to use in data analysis, this does not immediately imply that the model with a given number of factors is estimable or testable. For example, identification conditions may not be met. Furthermore, even if the model is testable, it may not be consistent with the data. Thus to apply factor analysis may be adifficult and not routine procedure. This is one of its virtues, as compared to components analysis, because the ability to reject a model is a fundamental aspect of data analysis using structural models (Bentler, 1989; Joreskog & Sorbom, 1988). In fact, the component model, or class of models as described by Velicer and Jackson, is not a model at all. As noted by Dunteman (1989, p. 56). "Principal components analysis is aprocedure to decompose the correlation matrix without regard to an underlying model." There is no hypothesis tested with the model. It is a nonfalsifiable procedure for analyzing data that can always be applied with any data set. Of course, there is a sampling theory for eigenvalues of a sample covariance matrix that can be used to test hypotheses about population roots of a covariance matrix (e.g., Anderson, 1963), but such hypotheses are typically hard to frame and not very informative about the structure of acorrelation matrix (discussed later in this article). In practical data analysis one can, of course, take the point of view that there is no single true model (De Leeuw, 1988). Such a point of view would deemphasize strict hypothesis testing and would suggest exploring alternative models that would, to a greater or lesser extent, yield parameter estimates that are stable and minimally biased. It would also be consistent with the use of fit indexes rather than probability values from a given goodness of fit test (Bentler. 1990; Mulaik et al., 1989). Within such a point of view, however. it still makes sense to us to define the class of models that might be of interest. We would argue that the error structure of component analysis is not appropriate to most real applications, and that the search for appropriate models should typically be limited to models with uncorrelated additive errors.
Invar-iancSeProperties Velicer and Jackson (1990) did not address component analysis versus factor analysis in the general case. They described these methods when applied to correlation matrices. What could one say about these methods as applied to covariance matrices, that is, to the same variables in a different metric'? One
MULTIVARIATE BEHAVIORAL RESEARCH
Downloaded by [McGill University Library] at 07:42 26 December 2012
P.Bentler and Y. Kano might hope that key aspects of any results are invariant with respect to such an arbitrary resealing. In the case of factor analysis, the effect of scale transformations is clear: the factor loadings are simply rescaled by the diagonal matrix involved. So, in Velicer and Jackson's Equation 1 , if one rescaled the variables q by a positive definite diagonal matrix D and analyzed D(q) instead of q, the factor loadings that result in theory, and are obtained with many methods of estimation, are DA instead of A. If the rescaled variables are analyzed by principal component analysis, on the other hand, no known relationship between the two sets of component results can be stated. Of course, image component analysis (their Equation 7) is scale invariant, a positive feature of this type of component analysis. A good discussion of some issues related to scale invariance is given by Cudeck (1989). A second aspect of invariance involves invariance of results with respect to a nonrandom sampling of subjects from the population of subjects. Although this is a complex topic, it is known that key features of a factor analysis solution (e.g., factor loadings, simple structure) can remain invariant under such selection (e.g., Meredith, 1964). This means that major conclusions will not be affected by such multivariate selection. In contrast, apparently no invariance results have been obtained for component solutions.
Estimation The computational economy, relatively speaking, of component analysis of a large number of variables makes the component procedure very attractive. It is certainly true that, historically speaking, the factor model has been much more expensive to estimate. As noted by Velicer and Jackson (1990), this situation has changed with the advent of noniterative estimators of the factor model (e.g., Hagglund, 1982; Bentler, 1982; Jennrich, 1986; Kano, 1989, in press). Unfortunately, these estimators are not well known or utilized. One may ask whether component analysis can be used to estimate some of the parameters of the factor analysis model (e.g., the factor loadings). As Velicer and Jackson's (1990) review makes clear, and as we will show below, sometimes this can be done. But in such applications the real interest remains on the factor model, and it should be recognized that component analysis is merely a computational convenience. Even Bartholomew's (1985) recent variant of component analysis is justified, essentially, on the basis of the factor analysis model (Bentler, 1985). Although in some applications there may be a genuine interest in the components, especially, principal components, these appear to be atypical applications.
JANUARY 1990
P. Bentler and Y. Kano
Ey~iivalenceof Component and Factor Analysis
Downloaded by [McGill University Library] at 07:42 26 December 2012
Many of the arguments for the use of component analysis are essentially practical. It has been found that results from application of both methods are quite similar. Velicer and Jackson (1 990) review some aspects of an analysis, such as the number of factors/components to retain for rotation, that are relevant to such similarity. Nonetheless, it appears that there are no results in the literature on precise conditions for any equivalence of outcome. We develop one such condition. Proposition Let a p-variate random variable Y have a factor analytic structure with a single common factor, that is, Y = at + E and Cov(Y) = C = aa' + Y.Let h be = 1 ). the largest eigenvalue of C, and o be the corresponding eigenvector (o'o Suppose a'a + ..,as p + os and that there exists a scalar yo > 0 such that yf, < y,, for all i, where y,is the i-th diagonal of 9'. Then (a) the correlation between the principal component o'Yand the common factor 5 converges to one a s p +m, (b) d h - + a a s p +w, that is, the principal component loading vector becomes equivalent to the factor loading vector.
By assumptions we have
and
(2)
Cw = (aa'+ Y)w = hw.
The squared correlation between o'Yand 5 can be written as
By using Equation 1, Equation 2 and the assumption that y,< w,,,
MULTIVARIATE BEHAVIORAL RESEARCH
P. Bentler and Y. Kano
This proves (a). It follows from Equation 2 that (XI = c x a , say. Then
-
Y)w = (a'@) x a
Downloaded by [McGill University Library] at 07:42 26 December 2012
with c = [al(hI- Y)-2a]-'i2.Substituting Equation 3 into Equation 2 means that ( a a ' + Y)(hI - Y)-la = h(hI - '4')-la, or
Let a = (a,, ..., ap)'. Assuming that a, # 0 and comparing the first elements of both-hand sides of Equation 4, we get
because Y/h+ 0 and (I - Y/h)-' - + Ias p
P
P
a?v,
= C i=l (A-v,)'
+w.
