Local and global mappings of topology representing networks

1. Local and global mappings of topology representing networks Agnes Vathy-Fogarassy , Janos Abonyi InS, Vol.179, 2009, pp. 3791–3803. Presenter : Wei-Shen Tai 2009/10/13

2. Outline • Introduction • Vector quantization • Competitive Hebbian Learning • Topology representing network based mapping algorithms • Neural Gas (NG), Topology Representing Network (TRN) • Mapping vs. Dimension Reduction • Analysis of the Topology Representing Network based mapping methods • Distance preservation and neighborhood preservation • Conclusion • Comments

3. Motivation • Combine vector quantization and mapping methods in order to visualize the data structure in a low-dimensional vector space. 3-D data structure Vector quantization Vector quantization & mapping

4. Objective • Topology Representing Network Map (TRN Map) • TRN obtains the graph of Topology Representing Network. • MDS based on graph (geodesic) distances to visualize representing node in 2-dimension vector space.

5. Vector Quantization and Competitive Hebbian Learning • Vector Quantization • A large set of points (vectors) are divided into groups. Each group is represented by its centroid point, as in k-means and some other clustering algorithms. • Competitive Hebbian Learning • For each input signal x connects the two closest (measured by Euclidean distance) centers by an edge.

6. NG & TRN • Neural Gas • Neighborhood ranking reference vector wj.for input xi • Update wj according to the distance ranking. • Topology Representing Network • NG was used for clustering purpose in conjunction with the Hebbian learning. Reference vector Input cluster

7. TRN Map • Normalize the input data set X. • Create the Topology Representing Network of X by the use of the TRN algorithm . • If M(D) is not connected, connect these subgraphs. ** this process is necessary for building a full connected graph. • Calculate the geodesic distances between all pairs wi;wj M(D). • Map the graph M(D) into a 2-dimensional vector space with MDS based on the graph distances of M(D). • Create component planes for the resulting TRN Map based on the values of wi M(D). 3-D data structure Vector quantization Vector quantization & mapping

8. Mapping vs. Dimension Reduction • Mapping • Represents the data structure of input data in a map with lower dimension. However, it cannot guarantee the consistency between data space and map space, such as CGS, NG and SOM. • Dimension Reduction • Attributes of inputs are transformed into fewer representative variables by statistical function or the characteristic of geodesic distance can be preserved by objective function. Those methods can fully present the original data structure in coordinates, such as PCA, SM and MDS. • Dimension Reduction can be regarded as a mapping method.

9. Mapping quality data • Distance preservation • MDS stress function • Sammon stress function • Neighborhood preservation • Trustworthiness (data) • k=3, green and blue • Continuity (map) • k=3, gray and navy map

10. Analysis of TRN mapping methods

11. Analysis of TRN mapping methods

12. Conclusions • Mapping based on the TRN • MDS is a global reconstructiontechnique, hence it is less sensitive to the number k-nearest neighbors and the number of codebook vectors. • Metric mapping based algorithms minimize the stress functions directly, hence their performance is the best in distance perseveration.

13. Comments • Advantage • This paper provides four quality index to evaluate distance preservation and neighborhood preservation. • Drawback • MDS can apply metric (distance) and non-metric (ranking) to preserve the pairwisedistance and rank ordering among data objects. Nevertheless, the mapping result of original data set via MDS is not compared to the other methods in this paper. • (Neighborhood Preservation) NP based methods should outperform than (Distance Preservation) DP NP based methods in two neighborhood preservation index. However, it seems unreasonable that a different result happened in optical recognition of handwritten digits. • Application • Dimension reduction and visualization for high- dimension data.