1 / 15

Classical Scaling

Classical Scaling. 기계학습 연구실 박성인 2010.02.25. Contents. Introduction Finding Coordinates in Classical Scaling A Numerical Example for Classical Scaling Choosing a Different Origin Advanced Topics. Introduction.

marlis
Download Presentation

Classical Scaling

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. Classical Scaling 기계학습 연구실 박성인 2010.02.25

  2. Contents • Introduction • Finding Coordinates in Classical Scaling • A Numerical Example for Classical Scaling • Choosing a Different Origin • Advanced Topics

  3. Introduction • The basic idea of classical scaling is to assume that the dissimilarities are distances and then find coordinates that explain them • Classical scaling uses the same procedure (In Section 7.9) but operates on squared dissimilarities instead of • This method is popular because it gives an analytical solution, requiring no iterations

  4. Finding Coordinates in Classical Scaling • (from 7.5 Section) • cis the vector with the diagonal elements of XX’ • Multiplying the left and the right sides by the centering matrix and by the factor -1/2 • Double centering • The centering around B can be removed because X is column centered, and hence so is B

  5. Double Centering • A way of normalizing the data matrix • Types • Row centering • Each row of the row-centered data matrix sums to zero • Column centering • Each column of the column-centered data matrix sums to zero • Double centering • Consists of the successive application of both row centering and column centering • Example A = AB = (Column centering) ABA = (Double centering) B = BA = (Row centering)

  6. Finding Coordinates in Classical Scaling • To find the MDS coordinates from B, we factor B by eigendecompositionQΛQ′ • The procedure for classical scaling • Compute the matrix of squared dissimilarities • Apply double centering to this matrix • Compute the eigendecomposition of QΛQ′ • Let the matrix of the first meigenvaluesgreater than zero be and the first m columns of Q. Then, the coordinate matrix of classical scaling is given by • If happens to be a Euclidean distance matrix, then classical scaling finds the coordinates up to a rotation

  7. Finding Coordinates in Classical Scaling • In step 4, negative eigenvaluescan occur but not if is a Euclidean distance matrix (see Chapter 19) • In classical scaling, the negative eigenvalues (and its eigenvectors) are simply ignored as error • Classical scaling minimizes the loss function (called Strain) • A nice property of classical scaling is that the dimensions are nested • MDS by minimizing Stress does not give nested solutions

  8. A numerical Example for Classical Scaling • Using the faces data from Table 4.4

  9. Choosing a Different Origin • X is constructed so that its columns sum to zero • The origin of configuration X coincides with the center of gravity of its points (centriod) • However, choosing this origin is not necessarily the best choice • Example • Some objects may be less familiar to the respondents, and thus lead to less reliable distance estimates than others • It is wiser to pick as an origin a point that is based more on the points associated with less error • For general solution, consider picking some arbitrary point s as the new origin, with restriction that s lies in the space of the other points

  10. Choosing a Different Origin • In terms of algebra, point s should lie in the row space of X • The coordinate vector of s is a weighted sum of the rows of X, s’ = w’X • w’ is an m-element row vector of weights • With s as the new origin, the point coordinates • If the weight vector w is chosen such that w’1 = 1, then is a projector • If B=XX’, one obtains after projecting X to a new origin s

  11. Choosing a Different Origin • If one chooses a particular object i as the origin • w’=[0,…,1,…,0] • the 1 is in the ith position • If one picks the centroid as the origin, then w’ = [1/n,…,1/n] • Another choice is to pick the weights in w so that they reflect the reliability of the objects • Unreliable elements should have a weight close to zero and reliable elements a high value • The origin will be attracted more towards the reliable points

  12. Advanced Topics • The linear constraints imposed on X require X = YC • Yis an n x rmatrix of r external variables • C are weights to be optimized by classical scaling • The method of computing C • P is orthonormal(P’P =I) • The Strain loss function used by classical scaling

  13. Advanced Topics • Example • We reanalyze the constrained MDS of the facial expression data in Section 10.3 • The external constraint matrix Y is defined as in Table 10.2 • The optimal weight matrix C obtained by constrained classical scaling is

  14. Advanced Topics • Example • To get the coordinates X • We compute X = YC • This solution does not differ much from the constrained MDS solution in 10.4 • The main difference lies in the location of point 8

  15. Advanced Topics • Classical scaling can even be used to study or discover the “intrinsic geometry” of highly nonlinear structures contained in high-dimensional spaces • To study such geometries and to unroll them into low-dimensional Euclidean geometries, Tenenbaum, De Silva, and Langford(2000) first define the radius of a small neighborhood • Applying classical scaling to dissimilarities generated in such a way from nonlinear structures allowed Tenenbaum et al.(2000) to unroll these structures successfully

More Related