Self Organized Maps SOM

1. Self Organized Maps (SOM) © Dragoljub Pokrajac 2003

2. Idea of Self Organized Maps Neurons are spatially positioned For each pair of neurons a symmetric distance is defined Hence, for each neuron its neighborhood is defined

3. Q:How the Neurons Are Specified? A:Each neuron is specified by its state The neuron state is N-dimensional vector s Q:What the inputs of the network? A: The inputs are N-dimensional vectors Q: How the network is defined? A: The network is specified by the number of neurons and by the neighborhood function ?. Q: What is the weight between the two neurons? It is equal to ?(i,j)

4. Training of Self-Organized Maps Training can occur in two modes Incremental mode In incremental mode, each training example appears at the network only once Batch mode The same training example appears at the input several times Training occurs in epochs

5. Incremental Training of Self-Organized Maps For each training example x 1. Find the closest neuron Neuron I which state sI is closest in Euclidian sense to x

6. 2.Adjust the state of the closest neuron to become closer to the input vector

7. 3. Propagate the change to the neighboring neurons and adjust their states sj?sj+ ? (I,j) * ?sI, j?I

8. How to define neighborhood function? “Box” neighborhood:

9. Euclidean Neighborhood

10. Euclidean Neighborhood- “Sombrero”

11. What Topologies are Possible? Rectangular grid Hexagonal Random

12. Training in Batch Mode Two-phase training Each phase occurs through a number of epochs Phase 1: ordering Phase 2: tuning

13. Ordering Phase In this phase: We start with large neighborhood (big K) Large learning rate We gradually shrink the neighborhood and decrease the learning rate

14. Tuning Phase We perform given number of epochs to finely tune the states We fix the neighborhood and slowly decrease learning rate

15. Applications of Self-Organized Maps Learning Distributions and clustering (see demosm2) Linear Vector Quantization (a classification method)