~r
.
MULTIDIMENSIONAL SCALING OF N SETS OF SIMILARITY MEASURES: A NONMETRIC INDIVIDUAL DIFFERENCES APPROACH’ VICTOR E. McGEE Psychology Department Dartmouth College ABSTRACT
Given N sets of similarity measures between pairs of stimulus objects a nonmetric multidimensional scaling procedure (known as CEMD) is proposed which allows one of four kinds of analysis to be performed on the data. The four types of solution are defined in terms of two binary decisions: (a) whether only one configuration is allowed for all N sets or whether each set is allowed to have its own configuration; and (b) whether the same monotonic transformation is applied to all N sets of input data or whether each set is allowed its own monotonic transformation. Where separate configurations are allowed the computer program seeks to keep these solutions as similar as possible and provides an index of the similarity of the configurations.
Consider a set of M similarity measures corresponding to M pairs of stimuli selected from a pool of k stimuli. (M might be k(k-I), or k(k-1)/2, or any reasonable subset of the complete set of pairs fork stimuli.) Denote such a set of similarity measures {S}. Obtain N such sets of similarity measures, either by replicating over one subject (or judge) N times or by using N different subjects (or judges). Denote these sets c11
S1={81};
sz={s2};
.....; S N = { 8 N }
.
The question is how should we analyse such data? Traditionally the N sets of measures would have been pooled in some way (summed, averaged, used in conjunction with the law of categorical judgment, etc.) and a compromise solution (coordinates for the k stimuli in some minimum dimensional r-space) for all the data would have resulted. Such an analysis would destroy individual differences. An alternative approach, which would preserve individual differences, involves the Eckart-Young procedure (Tucker & Messick, 1963). The data matrix, of order N times M, would be approximated by the product of an N times r matrix (indicating the individual differences) and an r times M matrix (indicating sets of similarity measures for ‘different points of view’). These types of analysis are known as ‘metric’ approaches, since the input data is regarded as ‘metric’ data and the solutions are ‘metric’ solutions. 1. This work was supported by an ONR grant (Nonr. 3897-06) to Dartmouth College. APRIL, 1968
233
By way of contrast there are a number of ‘nonmetric’ approaches that have recently been reported in the literature and which seem to offer important advantages over the ‘metric’ techniques (Shepard, 1962a, 196213; Kruskal, 1964a, 1964b ; McGee, 1966 ; Guttman, 1968 ; Lingoes & Guttman, 1967). Only the rank order of the similarities is important as input to the ‘nonmetric’ procedures but the solutions are again ‘metric’ solutions (Euclidean or any of the Minkowski metrics). Under these methods N sets of similarity measures could be pooled in some way so that a single minimum-dimensional solution would result. On the other hand it is also possible to handle each set of measures separately to obtain N minimum-dimensional solutions, one for each set. I n a sense this latter approach does not destroy individual differences but it may also be true that spurious differences result from separate analyses, as will be explained below. If we are looking for a ‘nonmetric’ individual differences procedure for N sets of similarity measures then we must ask whether there is something better than simply N separate analyses. In this paper a ‘nonmetric’ individual differences model is presented, based on the ‘elastic’ multidimensional scaling (EMD) procedure (McGee, 1966). The FORTRAN IV program for this individual differences model is known as CEMD-‘common’ elastic multidimensional scaling. Before proceeding to the statement of this model consider the following example.
AN EXAMPLE FROM THE REALM OF SPEECH-QUALITY PERCEPTION I n a study concerning speech-quality perception fifteen distortions of one recorded voice were arranged on tape in the form of 105 pairs and presented to many subjects to obtain many sets of similarity measures. Details of this study may be found in McGee (1964, 1965a). The first 100 subjects were required to repeat the whole paired comparison schedule so that for them there were two sets of similarity measures. In the earlier study a ‘metric’ individual differences model (the EckartrYoung procedure) was applied to one set of data for each of these 100 subjects. More recently many separate EMD analyses were conducted, resulting in an awareness of the problem posed in this paper. To illustrate the point consider just one subject, #11, and his two sets of similarity data, SI= {a1} and S2= {a2}. Figure 1 indicates the two-dimensional solutions that result from (a) two separate appli234
MULTIVARIATE BEHAVIORAL RESEARCH
cations of EMD, one for SI and one for Sa, and (b) one EMD solution based on the combined data, SI SZ.The differences between the two plots in Figure l (a) could be called intra-individual differences but it is clear that these differences are a compound of at least two components : (i) ‘true’ intra-individual differences, and (ii) artifacts of the EMD methodology itself. The single solution shown in Figure l ( b ) represents a compromise solution for the two sets of data and as such destroys any ‘true’ intra-individual differences. The experimenter finds himself in the position of not wanting to believe that the intra-individual differences are as great as indicated in Figure 1(a) since “the computer did not know that I wanted these solutions to be as similar as possible,” and also a little dissatisfied with the compromise solution in Figure 1(b) since there is no indication of the reliability (or reproducibility)
+
/-
ic
x
WORK f o r S e t S
1
= 2.05
WORK f o r S e t S z = 2 . 2 9
Fig. 1. Two-dimensional ‘minimum WORK’ solutions for 16 stimuli using ‘elastic’ multidimensional scaling procedure (EMD) ( a ) Two separate runs of EMD on the two sets of similarity measures for subject #11. The ‘dots’ show the solution for the first set, SI,and the ‘crosses’ show the solution for the second set, S,.
.
APRIL, 1968 ‘
235
WORK f o r s e t (SI +
s
) = 1.94 2
Fig. 1 (continued) (b) One run of EMD on the combined set of similarity measures for subject #11. This is the 2-space solution for set (S, +S,).
of the solution. Similarly, if we had two sets of similarity measures, S1and Sz, based on two different subjects, then two separate EMD solutions would show some spurious inter-individual differences and one compromise EMD solution would destroy any ‘true’ interindividual differences and would not suggest any index o f t h e repeatability (or reproducibility) of the perceptual space.
EMD METHODOLOGY AND
EXTENSION, CEMD
ITS
Details of the EMD methodology are available elsewhere (McGee, 196513; 1966) but a capsule summary is in order here. The set-of similarity measures, {a}, constitutes the input to a computer program known as EMD. A monotonic transformation of these measures is allowed : (8) 236
+
{dl
*
MULTIVARIATE BEHAVIORAL RESEARCH
The precise nature of this monotone transformation varies from iteration to iteration and is determined by means of an appropriate algorithm. What we want from the program is a set of coordinates for the k stimuli in some r-space (of known metric) such that the pair-wise distances in this r-space correspond most closely (in some sense) to the monotone transforms, {a}. Denote the matrix of coordinates X, of order k X r. Then for any given X-matrix we can compute the‘set of M pairwise distances, { d}, corresponding to the stimulus pairs of interest, and at every iteration we compare the set {d} with the set The criterion used in one form of EMD2 is known as WORK and is defined as follows:
{z}.
WORK = Z (1 - d/a)2 ,
121
where the summation is over all M pairs. This criterion is minimized iteratively by the method of steepest descent. In essence the method involves finding the minimum value of a nonlinear func$ion (WORK)in kr variables (the kr coordinates). Since the funcis nonlinear there are ‘local minima’ and this problem cannot be avoided entireIy. Now consider N sets of similarity measures: S1 through SN. ‘A natural extension of the WORK criterion would be WORK = 8 (WORK,)
131
i
where WORKi is the amount of WORK for the i-th set of similarity measures-fi = 1through N). There are two problems to face, however, and they are (i) whether or not to use the same monotonic transformation for each set of input measures, and (ii) whether or not to require the same set of coordinates for each set of similarity measures. Concerning the first problem we imagine each set of measures being transformed monotonically into &measures : .I
.
2
.
.
............. fN
-
: (6N)
{a,>
.
,where f, denotes the i-th monotonic function, The question is whether to require -2. A more general form of EMD methodology has been indicated in McGee (1967). See also Gregson & Russell (1967).
APRIL, 1968
237
f1= 92 = ....... = fN = f
or whether to allow, in general, for fl#f,#
........ #f,
.
Concerning the second problem we imagine each set of measures to be solved in the form of a coordinate matrix, Xi (i = 1through N), and the question is whether to require
XI=
xz = ........ = XN = x
or whether to allow, in. general, for
XI # xz#
same
Cell 1
Cell 2
different
Cell 3
Cell 4
MONOTONE TRANSFORM
........ # X N .
The cell 1 solution (same monotonic transformation for all sets of measures and one common solution matrix, X) is closest to the previous ‘one combined EMD solution’ (N sets of data combined and analysed by EMD), but it is not quite the same. The cell 4 solution (different monotonic transformations and different solu238
MULTIVARIATE BEHAVIORAL RESEARCH
tions for each set of measures) is ciosest to the previous 'Nsepal'ate EMD solutions' (one EMD solution for each set of measures), but there are important differences, including in particular the fact that under CEMD the N separate solutions are monitored so as to resemble each other more and more as the iterations proceed and an index of reproducibility is available. The cell 2 solution (same monotonic transformation but different solutions for each set of measures) is analogous to the imposition of the same transformation (for example, a linear transformation) on each set of similarity measures and then performing separate multidimensional scaling analyses on each set, with the important difference that again an index of reproducibility is available under the CEMD procedure. Cell 3 has no obvious previous analogue.
SOMEMATHEMATICALNOTES Cell 1 Solution. Given N sets of M measures we might average the measures for each pair to get one set of M averaged-measures. Applying the EMD procedure to these data we would get what has been called 'one combined EMD solution'. Using CEMD, cell 1, the NM measures are rank-ordered and a t each iteration in the numerical analysis a single monotonic transformation is applied to all NM measures. The WORK function [2] for one combined EMD solution would contain M terms, whereas, under CEMD, cell 1, the WORK function would contain NM terms. Since there is only one coordinate matrix, X, there is one set of configuration distances, {a}. In general, then, the cell 1 solution may be summarized as follows :
SI f : {SI}+
{a,}
Sz f : { & } +
{d,}
+cf.+ +cf.+
{d} {d}
f : {SN} + ( d ~ }+cf.+
{d}
..............................
SN
t
1' (same X )
(same f ) WORK=
N Z
i=l
[ S (1 - d/di)']
Cell 2 Solution. The N sets of M measures are combined and rankAPRIL, 1968
239
ordered and a t each iteration a single monotonic transformation,is determined for all NM measures. However, in this cell we allow separate solution matrices, X1 through XN.Thus we need to define a new WORK function and also the procedure by which the numerical analysis will monitor the separate solutions in order to make them as similar as possible. At each iteration there will be a set {ai} of transformed measures and a set {a,} of configuration distances (based on the coordinate matrix Xi) for the i-th set of measures, The WORK criterion for all N sets is defined as in equation [SI. In order to ensure that the N separate solutions, XI through XN,keep reasonably close to one another we define the centroid of the N solutions, as follows:
c41
r = (X,+X,+
.... + X , ) / N .
.
..
If we denote the general elements in X1, Xz,. .by xUl, xij2,. then the discrepancy between the separate solutions, XI through XN,and the centroid, I?, may be defined as follows : c51
where xu. is the general element in I?. Now as the iterations proceed we want to minimize both WORK and CI, and this gives rise to a ‘trade-off‘ problem, If the parameters are not chosen appropriately one of these criteria may be reduced a t the expense of the other. The following procedure was adopted in the CEMD program. Using the method of steepest descent to minimize a nonlinear function in kr variables involves changing the values of these variables a t each iteration in the following manner : C61
X’= X - STEP * Y
where X’ is the new coordinate matrix, X the old coordinate matrix, STEP is the step-size for proceeding down the steepest descent, and Y is the matrix of partial derivatives y i t h respect to each of the kr variables (coordinates). The stepsize is computed as
Dl
STEP = WORK / SSPDS
where SSPDS is the sum of squares of the partial derivatives in Y. Now under CEMD, cell 2, the function to be minimized is of the
form 240
MULTIVARIATE BEHAVIORAL RESEARCH
181
a’= WORK
+
+ C51).
(Le., C31
@
It has proven satisfactory in the minimization of a’ to change coordinates at each iteration according t o the formula: X’ = X - STEP1 * Yi - STEP2 * Y2 191 where ST.EPl = WORK / SSPDSl ; STEP2 = CJ / SSPDS2 ; Y1 = partial derivatives for minimizing WORK; Y2 = partial derivatives for minimizing CJ ; SSPDS, = sum of squares of elements in Y1 ; and SSPDS2 = sum of squares of elements in Y2. This procedure provides a balance between minimizing WORK and minimizing a. (It is a simple matter to weight one minimization more than the other, if this is desired for any reason.) The cell 2 solution may be summarized as follows:
SI f : {Si} + {a,} +-Cf.+ {ai} S2 f : (62)- {&} +cf.+ {d2}
..............................
S N
f : {SN}+
{&}
+cf.+
{a,}
t
t (different X’s)
(same f )
Equations [SI, [SI, [SI and [SI. The index is a measure of reproducibility of the N separate solutions. Cell 3 Solution. The N sets of M measures are retained as separate sets f o r the purposes of allowing a separate monotonic transformation to be applied t o each set at each iteration. This means simply that the algorithm for determining the monotonic transformation at each iteration is applied N times, once for each set of measures. Since only one solution is required there is only one coordinate matrix, X . The WORK function is formally identical to [3] and hew coordinates are obtained using [6] and [7]. The cell 3 solution may be summarized as follows :
Si fi : {Si}+ {di} t c f . +
{d}
S2 f2 : {62}+ {d2> + ~ f . +
{d}
..............................
SN fN : {
APRIL, 1968
8 ~3 }
(6,)
+Cf.+
{d}
1‘
t
(different f‘s)
(same X) 241
Equations [3], [61 and [7]. Cell 6 Solution. The N sets of M measures are retained as separate sets for the purpose of allowing a separate monotonic transformation for each set and N different coordinate matrices, XI through XN,are allowed. The function to be minimized is @’ (equation [SI) and new coordinates are obtained using equation [9]. The cell 4 solution may be summarized as follows: S1 f l : {Si}-+ S2 f n : {Sa}+
{d,} t-cf.+ Id2} c c f . +
{dl) {d2}
. . . . . . . . . . . . . . . . . . . . . . e . . . . . . .
SN fN : { 8 ~ } - + { b ~+Cf.+ }
{dN}
T
t
( d i f f e r e n t f‘s)
( d i f f e r e n tXs)
,Equations [SI, C51, [SI and [SI. The index @ is a measure of reproducibility of the N separate solutions.
APPLYING CEMD TO THE DATAFROM THE SPEECH QUALITYEXPERIMENT Consider the two sets of similarity measures, S1 and S2,for subject #11, as considered earlier. Figures 2, 3, 4 and 5 indicate the two-dimensional solutions under CEMD cell 1, cell 2, cell 3 and cell 4 respectively. Comparing Figures 1(b) and 2 it will be noticed that the combined EMD solution is not exactly the same as the cell 1 CEMD solution. However, are not great. Comparing Figures I (a) and 6 it be noticed that there is quite a considerable difference between two separate E D solutions and a cell 4 CEMD solution. The latter shows two solutions which agree much more closely than the former two solutions, as would be expected. Figure 3 indicates what is probably the preferred solution for two sets of data on the same subject. It seems reasonable to argue that a given individual would use the same ‘personal’ metric on both sets of judgments. In other words, the same monotonic function should be applied to the input measures in each set. Figure 4 indicates the cell 3 solution, which in this case is a nonpreferred analysis of the data sets. In general, if one subject is used repeatedly to obtain N sets 242
MULTIVARIATE BEHAVIORAL RESEAhCM
Fig. 2. The two-dimensional CEMD, Cell 1 solution for the two sets of measures (SIand S,) for subject #11. This should be compared with the combined EMD solution in Figure l ( b ) . Equation [3] interprets the quantity WORK,.
of similarity measures for the same set of M Stimulus pairs then it would seem appropriate to require the same monotonic transformation for all sets. The appropriate CEMD solution would therefore be either a cell 1 or a cell 2 solution. The cell 2 solution is preferable to the cell 1 solution in that intra-individual differences would not be obscured and some index of reproducibility could be obtained. If N different subjects are used to obtain N sets of similarity measures then it would seem appropriate to allow each individual to have his own ‘personal’ metric. The appropriate CEMD solution would therefore be either a cell 3 or a cell 4 solution. A cell 4 solution is preferable to a cell 3 solution in that inter-individual differences would not be obscured and some index of reproducibility (homogeneity) could be obtained. The index of reproducibility is admittedly arbitrary but can APRIL, 1968
243
Fig. 3. The two-dimensional CEMD, Cell 2 solution for the two sets of measures (SIand S,) for subject #11. Equations [SI, [SI and [lo] interpret the quantities WORK,, @ and R respectively.
be scaled so as t o provide an index between unity (perfect reproducibility or homogeheity) and zero (absence of any reproducibiL ity or homogeneity). In the EMD and CEMD programs the matrix of coordinates, X , is scaled at each iteration in such a way as t o make the root-mean-square-distance-from-the-origin equal to unity. Thus the index @ is constrained. In order to convert. this index to some other index between 0 and 1 we could either redefine @ initially or we could seek to discover the maximum size of the discrepancy, @, under a stated probqbility distribution of the k stimulus points in r-space. As a formal probability problem this is not straightforward but it is easy to simulate the situation. If we start with a uniform prior distribution of k points over a unit square (consideribg just the case of a 2-space for now), scale the coordinates so that the rms distance from the origin is unity, and repeat many times we can get an idea of the expected value of Q under random assignment of k points on N occasions. For the example 244
MULTIVARIATE BEHAVIORAL RESEARCH
WORK
1
=
3.16
WORKZ = 2.99
Fig. 4. The two-dimensional CEMD, Cell 3 solution for the two sets of measures (SIand S), for subject #11. Equation [3] interprets the quantity WORK,.
considered above (15 stimuli and N = 2) it was found that E (a) = 15.39. We might then define a new index of reproducibility as follows :
UOl
R = 1 - CI, / E (a).
Values of this index are shown in Figures 3 and 5. They are somewhat too close to unity in terms of our social conditioning about such indices but with practice it is possible to adapt!
DISCUSSION The use of CEMD has been illustrated for the case N = 2. For larger N, say N = 10, it is possible to use CEMD for the purpose of hierarchical clustering. Suppose we call for a cell 4 solution for N = 10 sets of measures. At each iteration there would be ten sepAPRIL, 1968
245
*k X
1
X
k
I
f
e
:\
xx
D
’r
iB
X
L
X
WORKl = 2 . 2 2
0
= . 9 2 (R = .94)
WORKZ = 2 . 6 3
Fig. 5. The two-dimensional CEMD, Cell 4 solution for the two sets of measures (SIand S), for subject #11. This should be compared with the separate EMD solutions in Figure l ( a ) . Equations [ a ] , [ 6 ] and [lo] interpret the quantities WORK, @ and R respectively.
arate coordinate matrices, XI through Xlo; ten sepayate WORK values, WORKl through WORK,,; and an index, @, indicating the closeness of the ten solutions. If we monitor the ten separate WORK values after the first I iterations (I = 5 or 10, say) there might be a natural separation of these values-some high and some low-so that it would be possible to divide the ten solutions into two groups according to their WORK values. Then each group would be treated separately for another I iteration with the possibility of a further split. In this way it is possible to perform hierarchical clustering of the N = 10 sets of measures. Details of this use of CEMD in connection with speech-quality perception are being prepared for publication elsewhere. The term ‘sets of similarity measures’ does not restrict the application of CEMD (or EMD) methodology to data obtained from successive interval scales. Any measure denoting relationship 246
MULTIVARIATE BEHAVIORAL RESEARCH
I
L
between two objects may be used-including correlation coeff icients, Cattell’s rp (profile similarity measure), Mahalanobis’s generalized D, overlap coefficients (as used in studies of word association), and so on. For a useful review of nonmetric methodologies in general and the types of measures to which they can be applied see Green, Carmane,.& Robinson (1967). More and more attention is being paid to the problem of reproducibility of multivariate structures under such headings as ‘parallel proportional profiles’ (Cattell, 1944 ; 1955; 1966), ‘factorial invariance’ (Meredith, 1964a, 1964b), ‘interbattery factor analysis’ (Tucker, 1958), ‘relations among m sets of measures’ (Horst, 1961), ‘simultaneous factoring of several Gramian matrices’ (Levin, 1966), and various others. These have all been concerned with the ‘metric’ models, and some are based on statistical models dealing with the sampling distribution of a variance-covariance matrix. Formal statistical tests have yet to be developed for most of the ‘nonmetric’ models but in many instances it seems to be more satisfying to deal with psychological data as merely rankorder data, and the absence of formal tests is not always felt as a serious problem. In a footnote to Cattell’s 1955 paper Burt comments ‘it seems desirable to determine the group factors separately and prove their similarity rather than to postulate their similarity and rotate them accordingly’ (p. 86). The argument of this paper would disagree with Burt’s position since separate determinations of the group factors would show differences due to both “true” differences and artifacts of the methodology. In other words, Burt‘s approach would seem to favor separate EMD analyses over one of the four CEMD analyses. Cattell’s approach is analogous to one of the CEMD analyses (cell 2 variety) since it allows the factor matrices to be different but attempts to make them as similar as possible. In general when the input data is liable to e k o r and when a monotonic transformation is allowed it should be recognized that a unique solution is not necessarily the ‘best’ solution and certainly not the only permissible solution. It is the philosophy of the EMD and CEMD approaches that there exists a set of permissible solutions for every set of input measures, and in the event of handling N sets of measures there is some degree of freedom about choosing permissible solutions for each set in such a way that N separate solutions look maximally similar? 3. The contributions of my research assistant, Joseph W. Serene, are gratefully acknowledged.
APRIL, 1968
e47
REFERENCES Cattell, R. B. Parallel proportional profiles and other principles for determining the choice of factors by rotation. Psychmetrika, 1944,9,267-283. Cattell, R. B. Factor rotation for proportional profiles: analytical solution and an example. British Journal of Statistical Psychology, 1955, 8, Pt. 11, 83-92. Cattell, R. B. & Coulter, M. A. Principles of behavioral taxonomy and the mathematical basis of the taxonome computer program. British J o u m l of Mathematical and Statistical Psychology, 1966, 19, Pt. 11, 237-269. Green, P. E., Carmone, F. J. & Robinson, P. J. Nonmetric scaling methods: an exposition and overview. Unpublished manuscript, August, 1967. Gregson, R. A. M. & Russell, P. N.A Note on a generating assumption in McGee’s multidimensional analysis of ‘elastic’ distances, British Journal of Mathematical and Statistical Psychology, 1967, 20, 239-242. Guttman, L. A general nonmetric technique for finding smallest Euclidean space for a configuration of points. Psychometrika, 1968 (in press). Horst, P. Relations among m sets of measures. Psychometrika, 1961, 26, 129-149. Kruskal, J. B. Multidimensional scaling by optimizing goodness of f i t to a non-metric hmothesis. Psychmetrika, 1964a, 89, 1-27. Kruskal, J. B. Non-metric multidimensional scaling: a numerical method. P w chomeh‘ka, 1964b, 29, 116-130. Levin, J. Simultaneous factor analysis of several Gramian matrices. Psychomtrika, 1966, $1, 413-419. Lingoes, J. C . & Guttman, L. Nonmetric factor analysis: a rank reducing alternative to linear factor analysis. Multivariute Behavioral Research, 1967, 2, 485-506. McGee, V . E. Semantic components of the quality of processed speech. Journal of Speech and Hearing Research, 1964, 7, 310-323. McGee,V. E. Determining perceptual spaces for the quality of filtered speech. Journal o f Speech & Hearing Research, 1965a, 8 , 23-38. McGee, V. E. Note on a n ‘elastic’ multidimensional scaling procedure. Perceptual and Motor Skills, 1966b, 21, 81-82. McGee, V. E. The multidimensional scaling of ‘elastic’ distances. British Journal of Mathematical and Statistical Psychology, 1966, 19, 181-196. McGee, V. E. A reply to some criticisms of elastic multidimensional scaling. British J o u m l of Mathematical and Statistical Psychology, 1967, PO, 243-247. Meredith, W. Notes on factorial invariance. Psychmetrika, 1964a, 89, 177-186. Meredith, W. Rotation to achieve factorial invariance. Psychmetrika, 1964b, 29, 187-206. Shepard, R. N. The analysis of proximities: multidimensional scaling with a n unknown distance function. I. Psychomet&ka, 1962a, 27, 125-140. Shepard, R. N. The analysis of proximities: multidimensional scaling with an unknown distance function. 11. Psychometrika, 1962b, 27,219-246. Tucker, L. R. An inter-battery method of factor analysis. Psychometrika, 1968, 23, 111-136. Tucker, L. R. & Messick, S. An individual differences model for multidimensional scaling. Psychumetrika, 1963, 28, 333-367.
248
MULTlVAR IATE BEHAVIORAL RESEARCH
.
MULTIDIMENSIONAL SCALING OF N SETS OF SIMILARITY MEASURES: A NONMETRIC INDIVIDUAL DIFFERENCES APPROACH’ VICTOR E. McGEE Psychology Department Dartmouth College ABSTRACT
Given N sets of similarity measures between pairs of stimulus objects a nonmetric multidimensional scaling procedure (known as CEMD) is proposed which allows one of four kinds of analysis to be performed on the data. The four types of solution are defined in terms of two binary decisions: (a) whether only one configuration is allowed for all N sets or whether each set is allowed to have its own configuration; and (b) whether the same monotonic transformation is applied to all N sets of input data or whether each set is allowed its own monotonic transformation. Where separate configurations are allowed the computer program seeks to keep these solutions as similar as possible and provides an index of the similarity of the configurations.
Consider a set of M similarity measures corresponding to M pairs of stimuli selected from a pool of k stimuli. (M might be k(k-I), or k(k-1)/2, or any reasonable subset of the complete set of pairs fork stimuli.) Denote such a set of similarity measures {S}. Obtain N such sets of similarity measures, either by replicating over one subject (or judge) N times or by using N different subjects (or judges). Denote these sets c11
S1={81};
sz={s2};
.....; S N = { 8 N }
.
The question is how should we analyse such data? Traditionally the N sets of measures would have been pooled in some way (summed, averaged, used in conjunction with the law of categorical judgment, etc.) and a compromise solution (coordinates for the k stimuli in some minimum dimensional r-space) for all the data would have resulted. Such an analysis would destroy individual differences. An alternative approach, which would preserve individual differences, involves the Eckart-Young procedure (Tucker & Messick, 1963). The data matrix, of order N times M, would be approximated by the product of an N times r matrix (indicating the individual differences) and an r times M matrix (indicating sets of similarity measures for ‘different points of view’). These types of analysis are known as ‘metric’ approaches, since the input data is regarded as ‘metric’ data and the solutions are ‘metric’ solutions. 1. This work was supported by an ONR grant (Nonr. 3897-06) to Dartmouth College. APRIL, 1968
233
By way of contrast there are a number of ‘nonmetric’ approaches that have recently been reported in the literature and which seem to offer important advantages over the ‘metric’ techniques (Shepard, 1962a, 196213; Kruskal, 1964a, 1964b ; McGee, 1966 ; Guttman, 1968 ; Lingoes & Guttman, 1967). Only the rank order of the similarities is important as input to the ‘nonmetric’ procedures but the solutions are again ‘metric’ solutions (Euclidean or any of the Minkowski metrics). Under these methods N sets of similarity measures could be pooled in some way so that a single minimum-dimensional solution would result. On the other hand it is also possible to handle each set of measures separately to obtain N minimum-dimensional solutions, one for each set. I n a sense this latter approach does not destroy individual differences but it may also be true that spurious differences result from separate analyses, as will be explained below. If we are looking for a ‘nonmetric’ individual differences procedure for N sets of similarity measures then we must ask whether there is something better than simply N separate analyses. In this paper a ‘nonmetric’ individual differences model is presented, based on the ‘elastic’ multidimensional scaling (EMD) procedure (McGee, 1966). The FORTRAN IV program for this individual differences model is known as CEMD-‘common’ elastic multidimensional scaling. Before proceeding to the statement of this model consider the following example.
AN EXAMPLE FROM THE REALM OF SPEECH-QUALITY PERCEPTION I n a study concerning speech-quality perception fifteen distortions of one recorded voice were arranged on tape in the form of 105 pairs and presented to many subjects to obtain many sets of similarity measures. Details of this study may be found in McGee (1964, 1965a). The first 100 subjects were required to repeat the whole paired comparison schedule so that for them there were two sets of similarity measures. In the earlier study a ‘metric’ individual differences model (the EckartrYoung procedure) was applied to one set of data for each of these 100 subjects. More recently many separate EMD analyses were conducted, resulting in an awareness of the problem posed in this paper. To illustrate the point consider just one subject, #11, and his two sets of similarity data, SI= {a1} and S2= {a2}. Figure 1 indicates the two-dimensional solutions that result from (a) two separate appli234
MULTIVARIATE BEHAVIORAL RESEARCH
cations of EMD, one for SI and one for Sa, and (b) one EMD solution based on the combined data, SI SZ.The differences between the two plots in Figure l (a) could be called intra-individual differences but it is clear that these differences are a compound of at least two components : (i) ‘true’ intra-individual differences, and (ii) artifacts of the EMD methodology itself. The single solution shown in Figure l ( b ) represents a compromise solution for the two sets of data and as such destroys any ‘true’ intra-individual differences. The experimenter finds himself in the position of not wanting to believe that the intra-individual differences are as great as indicated in Figure 1(a) since “the computer did not know that I wanted these solutions to be as similar as possible,” and also a little dissatisfied with the compromise solution in Figure 1(b) since there is no indication of the reliability (or reproducibility)
+
/-
ic
x
WORK f o r S e t S
1
= 2.05
WORK f o r S e t S z = 2 . 2 9
Fig. 1. Two-dimensional ‘minimum WORK’ solutions for 16 stimuli using ‘elastic’ multidimensional scaling procedure (EMD) ( a ) Two separate runs of EMD on the two sets of similarity measures for subject #11. The ‘dots’ show the solution for the first set, SI,and the ‘crosses’ show the solution for the second set, S,.
.
APRIL, 1968 ‘
235
WORK f o r s e t (SI +
s
) = 1.94 2
Fig. 1 (continued) (b) One run of EMD on the combined set of similarity measures for subject #11. This is the 2-space solution for set (S, +S,).
of the solution. Similarly, if we had two sets of similarity measures, S1and Sz, based on two different subjects, then two separate EMD solutions would show some spurious inter-individual differences and one compromise EMD solution would destroy any ‘true’ interindividual differences and would not suggest any index o f t h e repeatability (or reproducibility) of the perceptual space.
EMD METHODOLOGY AND
EXTENSION, CEMD
ITS
Details of the EMD methodology are available elsewhere (McGee, 196513; 1966) but a capsule summary is in order here. The set-of similarity measures, {a}, constitutes the input to a computer program known as EMD. A monotonic transformation of these measures is allowed : (8) 236
+
{dl
*
MULTIVARIATE BEHAVIORAL RESEARCH
The precise nature of this monotone transformation varies from iteration to iteration and is determined by means of an appropriate algorithm. What we want from the program is a set of coordinates for the k stimuli in some r-space (of known metric) such that the pair-wise distances in this r-space correspond most closely (in some sense) to the monotone transforms, {a}. Denote the matrix of coordinates X, of order k X r. Then for any given X-matrix we can compute the‘set of M pairwise distances, { d}, corresponding to the stimulus pairs of interest, and at every iteration we compare the set {d} with the set The criterion used in one form of EMD2 is known as WORK and is defined as follows:
{z}.
WORK = Z (1 - d/a)2 ,
121
where the summation is over all M pairs. This criterion is minimized iteratively by the method of steepest descent. In essence the method involves finding the minimum value of a nonlinear func$ion (WORK)in kr variables (the kr coordinates). Since the funcis nonlinear there are ‘local minima’ and this problem cannot be avoided entireIy. Now consider N sets of similarity measures: S1 through SN. ‘A natural extension of the WORK criterion would be WORK = 8 (WORK,)
131
i
where WORKi is the amount of WORK for the i-th set of similarity measures-fi = 1through N). There are two problems to face, however, and they are (i) whether or not to use the same monotonic transformation for each set of input measures, and (ii) whether or not to require the same set of coordinates for each set of similarity measures. Concerning the first problem we imagine each set of measures being transformed monotonically into &measures : .I
.
2
.
.
............. fN
-
: (6N)
{a,>
.
,where f, denotes the i-th monotonic function, The question is whether to require -2. A more general form of EMD methodology has been indicated in McGee (1967). See also Gregson & Russell (1967).
APRIL, 1968
237
f1= 92 = ....... = fN = f
or whether to allow, in general, for fl#f,#
........ #f,
.
Concerning the second problem we imagine each set of measures to be solved in the form of a coordinate matrix, Xi (i = 1through N), and the question is whether to require
XI=
xz = ........ = XN = x
or whether to allow, in. general, for
XI # xz#
same
Cell 1
Cell 2
different
Cell 3
Cell 4
MONOTONE TRANSFORM
........ # X N .
The cell 1 solution (same monotonic transformation for all sets of measures and one common solution matrix, X) is closest to the previous ‘one combined EMD solution’ (N sets of data combined and analysed by EMD), but it is not quite the same. The cell 4 solution (different monotonic transformations and different solu238
MULTIVARIATE BEHAVIORAL RESEARCH
tions for each set of measures) is ciosest to the previous 'Nsepal'ate EMD solutions' (one EMD solution for each set of measures), but there are important differences, including in particular the fact that under CEMD the N separate solutions are monitored so as to resemble each other more and more as the iterations proceed and an index of reproducibility is available. The cell 2 solution (same monotonic transformation but different solutions for each set of measures) is analogous to the imposition of the same transformation (for example, a linear transformation) on each set of similarity measures and then performing separate multidimensional scaling analyses on each set, with the important difference that again an index of reproducibility is available under the CEMD procedure. Cell 3 has no obvious previous analogue.
SOMEMATHEMATICALNOTES Cell 1 Solution. Given N sets of M measures we might average the measures for each pair to get one set of M averaged-measures. Applying the EMD procedure to these data we would get what has been called 'one combined EMD solution'. Using CEMD, cell 1, the NM measures are rank-ordered and a t each iteration in the numerical analysis a single monotonic transformation is applied to all NM measures. The WORK function [2] for one combined EMD solution would contain M terms, whereas, under CEMD, cell 1, the WORK function would contain NM terms. Since there is only one coordinate matrix, X, there is one set of configuration distances, {a}. In general, then, the cell 1 solution may be summarized as follows :
SI f : {SI}+
{a,}
Sz f : { & } +
{d,}
+cf.+ +cf.+
{d} {d}
f : {SN} + ( d ~ }+cf.+
{d}
..............................
SN
t
1' (same X )
(same f ) WORK=
N Z
i=l
[ S (1 - d/di)']
Cell 2 Solution. The N sets of M measures are combined and rankAPRIL, 1968
239
ordered and a t each iteration a single monotonic transformation,is determined for all NM measures. However, in this cell we allow separate solution matrices, X1 through XN.Thus we need to define a new WORK function and also the procedure by which the numerical analysis will monitor the separate solutions in order to make them as similar as possible. At each iteration there will be a set {ai} of transformed measures and a set {a,} of configuration distances (based on the coordinate matrix Xi) for the i-th set of measures, The WORK criterion for all N sets is defined as in equation [SI. In order to ensure that the N separate solutions, XI through XN,keep reasonably close to one another we define the centroid of the N solutions, as follows:
c41
r = (X,+X,+
.... + X , ) / N .
.
..
If we denote the general elements in X1, Xz,. .by xUl, xij2,. then the discrepancy between the separate solutions, XI through XN,and the centroid, I?, may be defined as follows : c51
where xu. is the general element in I?. Now as the iterations proceed we want to minimize both WORK and CI, and this gives rise to a ‘trade-off‘ problem, If the parameters are not chosen appropriately one of these criteria may be reduced a t the expense of the other. The following procedure was adopted in the CEMD program. Using the method of steepest descent to minimize a nonlinear function in kr variables involves changing the values of these variables a t each iteration in the following manner : C61
X’= X - STEP * Y
where X’ is the new coordinate matrix, X the old coordinate matrix, STEP is the step-size for proceeding down the steepest descent, and Y is the matrix of partial derivatives y i t h respect to each of the kr variables (coordinates). The stepsize is computed as
Dl
STEP = WORK / SSPDS
where SSPDS is the sum of squares of the partial derivatives in Y. Now under CEMD, cell 2, the function to be minimized is of the
form 240
MULTIVARIATE BEHAVIORAL RESEARCH
181
a’= WORK
+
+ C51).
(Le., C31
@
It has proven satisfactory in the minimization of a’ to change coordinates at each iteration according t o the formula: X’ = X - STEP1 * Yi - STEP2 * Y2 191 where ST.EPl = WORK / SSPDSl ; STEP2 = CJ / SSPDS2 ; Y1 = partial derivatives for minimizing WORK; Y2 = partial derivatives for minimizing CJ ; SSPDS, = sum of squares of elements in Y1 ; and SSPDS2 = sum of squares of elements in Y2. This procedure provides a balance between minimizing WORK and minimizing a. (It is a simple matter to weight one minimization more than the other, if this is desired for any reason.) The cell 2 solution may be summarized as follows:
SI f : {Si} + {a,} +-Cf.+ {ai} S2 f : (62)- {&} +cf.+ {d2}
..............................
S N
f : {SN}+
{&}
+cf.+
{a,}
t
t (different X’s)
(same f )
Equations [SI, [SI, [SI and [SI. The index is a measure of reproducibility of the N separate solutions. Cell 3 Solution. The N sets of M measures are retained as separate sets f o r the purposes of allowing a separate monotonic transformation to be applied t o each set at each iteration. This means simply that the algorithm for determining the monotonic transformation at each iteration is applied N times, once for each set of measures. Since only one solution is required there is only one coordinate matrix, X . The WORK function is formally identical to [3] and hew coordinates are obtained using [6] and [7]. The cell 3 solution may be summarized as follows :
Si fi : {Si}+ {di} t c f . +
{d}
S2 f2 : {62}+ {d2> + ~ f . +
{d}
..............................
SN fN : {
APRIL, 1968
8 ~3 }
(6,)
+Cf.+
{d}
1‘
t
(different f‘s)
(same X) 241
Equations [3], [61 and [7]. Cell 6 Solution. The N sets of M measures are retained as separate sets for the purpose of allowing a separate monotonic transformation for each set and N different coordinate matrices, XI through XN,are allowed. The function to be minimized is @’ (equation [SI) and new coordinates are obtained using equation [9]. The cell 4 solution may be summarized as follows: S1 f l : {Si}-+ S2 f n : {Sa}+
{d,} t-cf.+ Id2} c c f . +
{dl) {d2}
. . . . . . . . . . . . . . . . . . . . . . e . . . . . . .
SN fN : { 8 ~ } - + { b ~+Cf.+ }
{dN}
T
t
( d i f f e r e n t f‘s)
( d i f f e r e n tXs)
,Equations [SI, C51, [SI and [SI. The index @ is a measure of reproducibility of the N separate solutions.
APPLYING CEMD TO THE DATAFROM THE SPEECH QUALITYEXPERIMENT Consider the two sets of similarity measures, S1 and S2,for subject #11, as considered earlier. Figures 2, 3, 4 and 5 indicate the two-dimensional solutions under CEMD cell 1, cell 2, cell 3 and cell 4 respectively. Comparing Figures 1(b) and 2 it will be noticed that the combined EMD solution is not exactly the same as the cell 1 CEMD solution. However, are not great. Comparing Figures I (a) and 6 it be noticed that there is quite a considerable difference between two separate E D solutions and a cell 4 CEMD solution. The latter shows two solutions which agree much more closely than the former two solutions, as would be expected. Figure 3 indicates what is probably the preferred solution for two sets of data on the same subject. It seems reasonable to argue that a given individual would use the same ‘personal’ metric on both sets of judgments. In other words, the same monotonic function should be applied to the input measures in each set. Figure 4 indicates the cell 3 solution, which in this case is a nonpreferred analysis of the data sets. In general, if one subject is used repeatedly to obtain N sets 242
MULTIVARIATE BEHAVIORAL RESEAhCM
Fig. 2. The two-dimensional CEMD, Cell 1 solution for the two sets of measures (SIand S,) for subject #11. This should be compared with the combined EMD solution in Figure l ( b ) . Equation [3] interprets the quantity WORK,.
of similarity measures for the same set of M Stimulus pairs then it would seem appropriate to require the same monotonic transformation for all sets. The appropriate CEMD solution would therefore be either a cell 1 or a cell 2 solution. The cell 2 solution is preferable to the cell 1 solution in that intra-individual differences would not be obscured and some index of reproducibility could be obtained. If N different subjects are used to obtain N sets of similarity measures then it would seem appropriate to allow each individual to have his own ‘personal’ metric. The appropriate CEMD solution would therefore be either a cell 3 or a cell 4 solution. A cell 4 solution is preferable to a cell 3 solution in that inter-individual differences would not be obscured and some index of reproducibility (homogeneity) could be obtained. The index of reproducibility is admittedly arbitrary but can APRIL, 1968
243
Fig. 3. The two-dimensional CEMD, Cell 2 solution for the two sets of measures (SIand S,) for subject #11. Equations [SI, [SI and [lo] interpret the quantities WORK,, @ and R respectively.
be scaled so as t o provide an index between unity (perfect reproducibility or homogeheity) and zero (absence of any reproducibiL ity or homogeneity). In the EMD and CEMD programs the matrix of coordinates, X , is scaled at each iteration in such a way as t o make the root-mean-square-distance-from-the-origin equal to unity. Thus the index @ is constrained. In order to convert. this index to some other index between 0 and 1 we could either redefine @ initially or we could seek to discover the maximum size of the discrepancy, @, under a stated probqbility distribution of the k stimulus points in r-space. As a formal probability problem this is not straightforward but it is easy to simulate the situation. If we start with a uniform prior distribution of k points over a unit square (consideribg just the case of a 2-space for now), scale the coordinates so that the rms distance from the origin is unity, and repeat many times we can get an idea of the expected value of Q under random assignment of k points on N occasions. For the example 244
MULTIVARIATE BEHAVIORAL RESEARCH
WORK
1
=
3.16
WORKZ = 2.99
Fig. 4. The two-dimensional CEMD, Cell 3 solution for the two sets of measures (SIand S), for subject #11. Equation [3] interprets the quantity WORK,.
considered above (15 stimuli and N = 2) it was found that E (a) = 15.39. We might then define a new index of reproducibility as follows :
UOl
R = 1 - CI, / E (a).
Values of this index are shown in Figures 3 and 5. They are somewhat too close to unity in terms of our social conditioning about such indices but with practice it is possible to adapt!
DISCUSSION The use of CEMD has been illustrated for the case N = 2. For larger N, say N = 10, it is possible to use CEMD for the purpose of hierarchical clustering. Suppose we call for a cell 4 solution for N = 10 sets of measures. At each iteration there would be ten sepAPRIL, 1968
245
*k X
1
X
k
I
f
e
:\
xx
D
’r
iB
X
L
X
WORKl = 2 . 2 2
0
= . 9 2 (R = .94)
WORKZ = 2 . 6 3
Fig. 5. The two-dimensional CEMD, Cell 4 solution for the two sets of measures (SIand S), for subject #11. This should be compared with the separate EMD solutions in Figure l ( a ) . Equations [ a ] , [ 6 ] and [lo] interpret the quantities WORK, @ and R respectively.
arate coordinate matrices, XI through Xlo; ten sepayate WORK values, WORKl through WORK,,; and an index, @, indicating the closeness of the ten solutions. If we monitor the ten separate WORK values after the first I iterations (I = 5 or 10, say) there might be a natural separation of these values-some high and some low-so that it would be possible to divide the ten solutions into two groups according to their WORK values. Then each group would be treated separately for another I iteration with the possibility of a further split. In this way it is possible to perform hierarchical clustering of the N = 10 sets of measures. Details of this use of CEMD in connection with speech-quality perception are being prepared for publication elsewhere. The term ‘sets of similarity measures’ does not restrict the application of CEMD (or EMD) methodology to data obtained from successive interval scales. Any measure denoting relationship 246
MULTIVARIATE BEHAVIORAL RESEARCH
I
L
between two objects may be used-including correlation coeff icients, Cattell’s rp (profile similarity measure), Mahalanobis’s generalized D, overlap coefficients (as used in studies of word association), and so on. For a useful review of nonmetric methodologies in general and the types of measures to which they can be applied see Green, Carmane,.& Robinson (1967). More and more attention is being paid to the problem of reproducibility of multivariate structures under such headings as ‘parallel proportional profiles’ (Cattell, 1944 ; 1955; 1966), ‘factorial invariance’ (Meredith, 1964a, 1964b), ‘interbattery factor analysis’ (Tucker, 1958), ‘relations among m sets of measures’ (Horst, 1961), ‘simultaneous factoring of several Gramian matrices’ (Levin, 1966), and various others. These have all been concerned with the ‘metric’ models, and some are based on statistical models dealing with the sampling distribution of a variance-covariance matrix. Formal statistical tests have yet to be developed for most of the ‘nonmetric’ models but in many instances it seems to be more satisfying to deal with psychological data as merely rankorder data, and the absence of formal tests is not always felt as a serious problem. In a footnote to Cattell’s 1955 paper Burt comments ‘it seems desirable to determine the group factors separately and prove their similarity rather than to postulate their similarity and rotate them accordingly’ (p. 86). The argument of this paper would disagree with Burt’s position since separate determinations of the group factors would show differences due to both “true” differences and artifacts of the methodology. In other words, Burt‘s approach would seem to favor separate EMD analyses over one of the four CEMD analyses. Cattell’s approach is analogous to one of the CEMD analyses (cell 2 variety) since it allows the factor matrices to be different but attempts to make them as similar as possible. In general when the input data is liable to e k o r and when a monotonic transformation is allowed it should be recognized that a unique solution is not necessarily the ‘best’ solution and certainly not the only permissible solution. It is the philosophy of the EMD and CEMD approaches that there exists a set of permissible solutions for every set of input measures, and in the event of handling N sets of measures there is some degree of freedom about choosing permissible solutions for each set in such a way that N separate solutions look maximally similar? 3. The contributions of my research assistant, Joseph W. Serene, are gratefully acknowledged.
APRIL, 1968
e47
REFERENCES Cattell, R. B. Parallel proportional profiles and other principles for determining the choice of factors by rotation. Psychmetrika, 1944,9,267-283. Cattell, R. B. Factor rotation for proportional profiles: analytical solution and an example. British Journal of Statistical Psychology, 1955, 8, Pt. 11, 83-92. Cattell, R. B. & Coulter, M. A. Principles of behavioral taxonomy and the mathematical basis of the taxonome computer program. British J o u m l of Mathematical and Statistical Psychology, 1966, 19, Pt. 11, 237-269. Green, P. E., Carmone, F. J. & Robinson, P. J. Nonmetric scaling methods: an exposition and overview. Unpublished manuscript, August, 1967. Gregson, R. A. M. & Russell, P. N.A Note on a generating assumption in McGee’s multidimensional analysis of ‘elastic’ distances, British Journal of Mathematical and Statistical Psychology, 1967, 20, 239-242. Guttman, L. A general nonmetric technique for finding smallest Euclidean space for a configuration of points. Psychometrika, 1968 (in press). Horst, P. Relations among m sets of measures. Psychometrika, 1961, 26, 129-149. Kruskal, J. B. Multidimensional scaling by optimizing goodness of f i t to a non-metric hmothesis. Psychmetrika, 1964a, 89, 1-27. Kruskal, J. B. Non-metric multidimensional scaling: a numerical method. P w chomeh‘ka, 1964b, 29, 116-130. Levin, J. Simultaneous factor analysis of several Gramian matrices. Psychomtrika, 1966, $1, 413-419. Lingoes, J. C . & Guttman, L. Nonmetric factor analysis: a rank reducing alternative to linear factor analysis. Multivariute Behavioral Research, 1967, 2, 485-506. McGee, V . E. Semantic components of the quality of processed speech. Journal of Speech and Hearing Research, 1964, 7, 310-323. McGee,V. E. Determining perceptual spaces for the quality of filtered speech. Journal o f Speech & Hearing Research, 1965a, 8 , 23-38. McGee, V. E. Note on a n ‘elastic’ multidimensional scaling procedure. Perceptual and Motor Skills, 1966b, 21, 81-82. McGee, V. E. The multidimensional scaling of ‘elastic’ distances. British Journal of Mathematical and Statistical Psychology, 1966, 19, 181-196. McGee, V. E. A reply to some criticisms of elastic multidimensional scaling. British J o u m l of Mathematical and Statistical Psychology, 1967, PO, 243-247. Meredith, W. Notes on factorial invariance. Psychmetrika, 1964a, 89, 177-186. Meredith, W. Rotation to achieve factorial invariance. Psychmetrika, 1964b, 29, 187-206. Shepard, R. N. The analysis of proximities: multidimensional scaling with a n unknown distance function. I. Psychomet&ka, 1962a, 27, 125-140. Shepard, R. N. The analysis of proximities: multidimensional scaling with an unknown distance function. 11. Psychometrika, 1962b, 27,219-246. Tucker, L. R. An inter-battery method of factor analysis. Psychometrika, 1968, 23, 111-136. Tucker, L. R. & Messick, S. An individual differences model for multidimensional scaling. Psychumetrika, 1963, 28, 333-367.
248
MULTlVAR IATE BEHAVIORAL RESEARCH
