Low Dimensional Representations And Multidimensional Scaling (MDS ) (Sec 10.14)

1. Low Dimensional Representations And Multidimensional Scaling (MDS) (Sec 10.14) • Given n points (objects) x1, …, xn . No class labels • Suppose only the similarities between the n objects are provided • Goal is to represent these n objects in some low dimensional space in such a way that the distances between points in that space corresponds to the dissimilarities in the original space • If an accurate representation can be found in 2 or 3 dimensions than we can visualize the structure of the data • Find a configuration of points y1, …, ynfor which the n(n-1) distances dij are as close as possible to the original similarities; this is called Multidimensional scaling • Two cases • Meaningful to talk about the distances between given n points • Only rank order among similarities are meaningful

2. Distances Between Given Points is Meaningful

3. Criterion Functions • Sum of squared error functions • Since they only involve distances between points, they are invariant to rigid body motions of the configuration • Criterion functions have been normalized so their minimum values are invariant to dilations of the sample points

4. Finding the Optimum Configuration • Use gradient-descent procedure to find an optimal configuration y1, …, yn

5. Example 20 iterations with Jef

6. Nonmetric Multidimensional Scaling • Numerical values of dissimilarities are not as important as their rank order • Monotonicityconstraint: rank order of dij = rank order of ij • The degree to which dij satisfy the monotonicy constraint is measured by • Normalize to prevent it from being collapsed