Ongoing software project, not “theory” Encapsulated internals & interfaces Today:

1. …

2. Ongoing software project, not “theory” • Encapsulated internals & interfaces • Today: • Details of module internals • Details of architecture & signaling/feedback • Single, clean, simple inputs • (26 slides) • Not yet: • time • noise • robustness • multiple/partial hypotheses

3. One “compressor”: • Generic memory unit • Learns about low-dim structure in high-dim data • Converts live data between low-dim   high-dim • Hierarchy of compressors: • Each learns from compressed & • combined output of those below • Bi-directional (feedback)

4. Quantizing & representing high-dim input Compressor Internals Compressing old: Matching to previous compression Bi-directional mapping P = p1 + p2 + … Probability estimation

5. X • “Point” = position, weight, radius Quantizing & representing high-dim input • Two point-clouds: mapping vs. learning (sync occasionally) Online updates: 3. Move it to absorb new point, preserving center of mass 2. Choose the lightest 1. Find 3 closest cloud-points 4. Increase weight 5. Update radius 6. (prune lightweight points) Result: Point-cloud approximates input cloud, with roughly equal weight per point

6. Compressing high to low (ISOMAP) 1. Find local distances in high-dim space 2. Create long-range distances from shortest piecewise path (“geodesic”) 3. Link “islands” until all Dij defined 4. Diagonalize F(Dij) to get low-dim cloud (arbitrary coordinates)

7. The low-dim mapping is not always unique… Keeping new maps consistent with old ones …so rotate & stretch new cloud to minimize distance from old one (SVD) Rotated new cloud Old cloud New cloud

8. 1. Find new point’s closest 4-5 neighbors Mapping new points using point-clouds W2 2. Express it as their center-of-mass (SVD) W1 = W3 W4 3. Construct low-dim output from corresponding neighbors & weights W2 W1 = 4. Also works mapping lowhigh W3 W4

9. P = p1 + p2 + … P = p1 + p2 + … Each point is center of gaussian Prob. Estimation Ri Pi = exp ( -0.5 r2 / R2) / (RD Ptot) “Probability” of test point is sum over local gaussians Probability =“Closeness” to manifold = how much to trust this point … use it later in mixing estimates.

10. Creating forward output Compressors interacting Feedback mixed back in Settling

11. Creating output • Map from high to low dim • Expose result to all Compressors above • Re-map output backwards to high dim • Expose as feedback to Compressors below

12. Mix feedback into output 1. Average feedback from above 2. Get probabilities of feedback and own output P P 3. Create weighted mixture of them

13. Updating and settling 1. Expose mixture as updated output, and map downward as updated feedback 2. Iterate a few times to settle --- done with description of system --

14. General simulation results 3-layer hierarchy with 2-1 convergence Input is 9x6 “pixel” space with random illumination Display low-dim output in 2-D Ä color

15. Simple 1-dim illumination How does each module map the input space? ?

17. Toroidal 1-dim illumination How does each module map the circular input space? = ?

19. 2-dim spot illumination How does each module map the 2-D input space? ?

21. “Hallucinating” spots driven from above • Force activity at a single • location in top module 2. Let feedback move down 3. Look at what lower modules think input ought to be ? ? ?

23. 2-dim clustered spots (left & right) How does each module map the 2-D input space? ?

25. Next steps Architecture • Time • Reference problem • Reference platform • Integration method • Separate streams for transforms vs. objects • Get people involved! Algorithms • Noise • Multiple hypotheses • Distributed representation • “neurons” • Better quantization, mapping, robustness