1 / 32

Neural Wiring Optimization Paradigm

Neural Wiring Optimization Paradigm. Christopher Cherniak & Zekeria Mokhtarzada  Committee for Philosophy & the Sciences University of Maryland www.glue.umd.edu/~cherniak/  Network optimization in brain "Save wire": connection minimization Steiner tree

leane
Download Presentation

Neural Wiring Optimization Paradigm

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. Neural Wiring Optimization Paradigm Christopher Cherniak & Zekeria Mokhtarzada  Committee for Philosophy & the Sciences University of Maryland www.glue.umd.edu/~cherniak/  Network optimization in brain "Save wire": connection minimization Steiner tree Component placement optimization [QAP] NP-complete problems: How does Nature solve them? Physics  Optimization  Neural Structure

  2. Two-way street between: philosophy & empirical computational neuroanatomy. • Bounded-resource philosophical framework: "We don't have God's brain." • Human computational resources are critically constrained; in particular, neural connectivity. • Hence, strong pressure to optimize use of brain's limited neuro-wiring. Generative rule of neural structure: "Save wire." • Combinatorial network optimization theory. • 1. Neuron Arbor Optimization. "Neuron arbors act like flowing water." • Tree-optimization concept: Steiner tree. • Locally, for isolated "Y-tree". • (a) Fluid-dynamic model: • At junction, trunk costs more than branches. Fits fluid-dynamical model: relation of diameters of branches to trunk minimizes internal wall-drag of laminar flow • [ t 3 = b13 + b23 ]. • (b) Fluid-static model: • Then at local junction, for given trunk & branch weights, cost-minimizing branch angle is via vector-mechanical tug-of-war ("triangle of forces law") • [ cos θ = (t 2 - b12 - b22) / 2b1b2 ]. • Best fit is for cost = total volume (vs surface area, length, signal-delay). • Globally, for multi-junction tree. • Local optimization large-scale optimization. • = combinatorial problem ("NP-hard"). • E.g., for 12 terminals, 6 x 108 alternative topologies must be exhaustively searched(!) • Observed trees cost ~5% more than volume-minimal trees, across all topologies: • For planar dendrites (rabbit retina ganglion, amacrine; cat ganglion), axons (mouse thalamus). • Also nonliving trees, e.g., river drainage networks.

  3. 2. Component Placement Optimization [qap]: Given: connections among components. Find: layout of components, on 2-d surface, that minimizes total cost of interconnections (e.g., wirelength). NP-hard: For n-component system, n! alternative layouts must be exhaustively searched. "Brain as ultimate microchip" = organizing principle, at multiple hierarchical levels, of nervous system anatomy. "Why brain is in head" of invertebrates & vertebrates: minimizes total nerve connection costs to & from brain. Nematode C. elegans nervous system is first ever fully mapped. The actual layout of 11 ganglia is the wirelength-minimizing one [87,803 um] -- out of 40 million possibilities(!) Similarly, CPO predicts actual layout of 40 functional areas of cat cerebral cortex. To best-in-a-billion optimality level. To computational limits of detectability [ 40! = 1047 layouts ]. "Best of all possible brains," a predictive success story.

  4. 3. Mechanisms of NP-hard neural optimization. Blind trial & error exhaustive search for, e.g., minimum-wiring layout of a 50 component system would require more than age of Universe. Generally, exact solutions are computationally intractable. So, quick & dirty approximate/probabilistic heuristics. In particular, optimization "for free, directly from physics". (i) E.g., "instant neuron arbors, just add water": i.e., arbor optimization via fluid dynamics. (ii) Layout optimization (e.g., for roundworm ganglia) via "mesh of springs" force-directed placement simulation. Also, simple genetic algorithm performs well for worm ganglia, and for cortex areas. Functional role of "best of all possible brains" wiring optimization? Perhaps an economical means of complex structure generation that is transmissible thru limited-capacity "genomic bottleneck". So: Physics -> Optimization -> Neuroanatomy A neural optimization paradigm is a structuralist position, postulating innate abstract internal structure. (Vs tabula rasa, empty-organism, no structure in hardware.) Continental rationalist; but for brain rather than mind. Non-Genomic Nativism: Hardwired; however, not via genome, but by exploiting basic physical processes -- complex biological structure as self-organizing. Metaphilosophy: Holistic, vs compartmentalized, model of interrelation of philosophy & science. www.glue.umd.edu/~cherniak/

  5. Arbor Optimization Minimal Spanning Tree Steiner Tree Minimal spanning tree (A) and Steiner tree (B) for 5 nodes on a plane. Steiner tree is shorter, but much more computationally costly to construct (NP-hard). Steiner tree applies to neuron arbors; with cost measure as arbor volume, not tree length.

  6. Neuron arbor junction (cat retina ganglion cell dendrite). (a) Branch and trunk diameters conform to t3 = b13 + b23, a fluid-dynamic model for minimum internal walldrag of pumped flow (laminar regime). (b) In turn, angle θ conforms to the "triangle of forces" law, a cosine function of the diameters: cos θ = (t2 - b12 - b22) / 2b1b2 . This yields the minimum volume for a Y-tree junction. "Neuron junctions act like flowing water.”

  7. The simplest network optimization problem: Find shortest distance between 2 points. -- Via vector-mechanics.

  8. Minimizing cost of 1-junction network. Via tug-of-war "Triangle of Forces" cos law of vector mechanics.

  9. Even a simple 5-terminal tree has 15 alternative possible topologies, or connecting patterns. Arbor optimization requires not just (a) best embedding of an arbor, but also (b) exhaustive search of all possible topologies.

  10. Optimal topols Actual topols Vol err: 2.6% Surf err: 27.2% Leng err: 60.6% Optimization analysis of 5-terminal subtree of rabbit retina ganglion cell dendrite. Dendritic arbors best fit minimum-volume model.

  11. Complex biological structure arising "for free, directly from physics". -- "Instant arbors, just add water." In each case, from micron to meter scale, actual structure is within a few percent of minimum-vol configuration shown.

  12. Elm twig. Arbor includes multiple junctions.

  13. Lichtenberg Figure, a planar high-voltage discharge pattern. This nonliving structure also conforms to the min-vol arbor model.

  14. Mouse thalamus axon arbor. (A) Actual multi-junction tree in broken lines, optimal embedding of actual topol (wrt vol) in solid lines; Vol err: 2.2%. (B) "Best of all possible topologies" (wrt vol); Vol err: 2.5%. Only 10 of the 10,395 alternative topologies here have lower total vol costs, when optimally embedded, than the actual topol. And, topol search yields little improvement, compared to embedding.

  15. 9-terminal arbor of mouse thalamus axon: Distribution of vol costs of all 135,135 possible topologies, each optimally embedded. Histogram shows usual pattern for natural arbors, living and nonliving -- more costly topologies are more common, cheapest ones are rarest. The most costly optimally embedded "pessimal" layouts have only about 12% greater vol than cheapest one. Hence, for optimization, "topology does not matter".

  16. 1 2 3 3 1 2 Component Placement Optimization A B (Cost: 5) (Cost: 4) Component placement optimization: Connection cost-minimization. Total system is 1-d array of components 1 - 3 (each dimensionless, with 0 length-cost). (A) a suboptimal layout (cost: 5), vs (B) globally optimal layout (cost: 4).

  17. 1.3 mm PHANRNGDOLAVNRVVCaVCpPADRLU Head Tail C. elegans adult hermaphrodite nematode. Nervous system contains 11 ganglia.

  18. 50 um C. Elegans ganglia: their body locations and schematized shapes.

  19. Cell bodies of all neurons in head of C. elegans. (White et al, 1986) Neuron process tracts in head of C. elegans.

  20. AINL & AINR, interneurons with cell bodies in Lateral ganglion in head. Every synapse is shown; all are in Ring.

  21. Complete ganglion-level connectivity map for C. elegans nervous system. Each horizontal microline represents one of its 302 neurons. Horiz scaling: ~ 100x. This actual ganglion layout requires the least total connection length of all ~40 million alternative orderings.

  22. Zoom-in: PH,AN,DO = Ganglia + = Cell body = Sensor ~ = Muscle = Chemical Synapse = Electrical Synapse

  23. Frequency Layout Wirecost (mm) Actual Distribution of wirecosts (total wirelength) of all possible layouts of ganglia of C. elegans. 10,000 bin histogram of 39,916,800 alternative orderings. Least costly and most costly layouts are rarest. The last-place "pessimal" layout requires ~ 4 times as much total connection fiber as the actual optimal one.

  24. Generation: 16 8 11 10 7 5 3 12 6 4 2 1 220947.75----------------------------------------- 6 12 7 10 11 3 5 8 4 2 1 168993.75------------------------------------- 6 12 7 10 11 3 5 8 4 2 1 168993.75------------------------------------- 1 11 10 7 5 3 12 6 4 8 2 144033.50-------------------------- 8 11 10 7 5 3 12 6 4 2 1 220947.75----------------------------------------- 8 11 10 7 5 3 12 6 4 1 2 221697.75----------------------------------------- 6 12 7 10 11 3 5 2 4 1 8 124148.75----------------- 1 11 10 7 5 3 12 6 4 8 2 144033.50-------------------------- 8 11 10 7 12 3 5 6 4 2 1 221238.75----------------------------------------- 8 11 10 7 5 3 12 6 4 1 2 221697.75----------------------------------------- 185673.30========================================= Generation: 17 12 6 7 10 11 3 5 2 4 1 8 124964.75----------------- 6 12 7 10 11 3 5 8 4 2 1 168993.75------------------------------------- 6 12 7 10 11 3 5 8 4 2 1 168993.75------------------------------------- 1 11 10 7 5 3 12 6 4 8 2 144033.50-------------------------- 6 12 7 10 11 3 5 8 4 2 1 168993.75------------------------------------- 8 11 10 7 5 3 12 6 4 2 1 220947.75----------------------------------------- 6 12 7 10 11 3 5 2 4 1 8 124148.75----------------- 1 11 10 7 5 3 12 6 4 8 2 144033.50-------------------------- 1 11 10 7 5 3 12 6 4 8 2 144033.50-------------------------- 1 11 10 7 5 3 12 6 4 8 2 144033.50-------------------------- 155317.66=============================== Generation: 18 12 6 7 10 11 3 5 2 4 1 8 124964.75----------------- 4 11 10 7 5 3 12 6 1 8 2 150274.00----------------------------- 6 12 7 10 11 3 5 2 4 1 8 124148.75----------------- 1 11 10 7 5 3 12 6 4 8 2 144033.50-------------------------- 1 11 10 7 5 3 12 4 6 8 2 144277.00-------------------------- 1 11 10 7 5 3 12 6 4 8 2 144033.50-------------------------- 6 12 7 10 11 3 5 2 4 1 8 124148.75----------------- 1 11 10 7 5 3 12 6 4 8 2 144033.50-------------------------- 1 11 10 7 5 3 12 6 4 8 2 144033.50-------------------------- 1 11 10 7 5 3 12 6 4 8 2 144033.50-------------------------- 138798.08======================== Generation: 19 12 6 7 10 11 3 5 2 4 1 8 124964.75----------------- 6 12 7 10 11 3 5 2 4 1 8 124148.75----------------- 6 12 7 10 11 3 5 2 4 1 8 124148.75----------------- 1 6 12 7 10 11 3 5 2 4 8 117829.75-------------- 6 12 7 10 11 3 2 5 4 1 8 125990.75------------------ 12 6 7 10 11 3 5 1 4 2 8 125909.75------------------ 6 12 7 10 11 3 5 2 4 1 8 124148.75----------------- 6 12 7 10 11 3 5 2 4 1 8 124148.75----------------- 1 11 10 7 5 3 12 6 4 8 2 144033.50-------------------------- 1 11 10 7 5 3 12 6 4 8 2 144033.50-------------------------- 127935.70=================== GenAlg, a simple genetic algorithm, rapidly and reliably finds the optimal (minimum wirelength) layout of C. elegans ganglia among 11! alternatives. Here, population size is only 10.

  25. Mean Wirecost (um) Generation Number GenAlg rapidly finds the optimal (minimum wirelength) layout of C. elegans ganglia among 11! alternatives. Here, in only 150 generations.

  26. Tensarama, a force-directed placement algorithm for optimizing layout of C. elegans ganglia. This "mesh of springs" vector-mechanical energy-minimization simulation represents each of the worm's ~ 1,000 connections acting upon the movable ganglia PH, AN, etc. The key feature of Tensarama performance for the actual worm connectivity matrix is its low susceptibility to local minima traps. -- Unlike Tensarama performance for small modifications of the actual connectivity matrix ("butterfly effect"), and unlike FDP algorithms in general for circuit design. Here Tensarama is trapped with a “killer” connectivity matrix that differs from the actual matrix by only one less connection.

  27. Layout Wirecost (mm) Actual Layout Adjacency Cost Adjacency Rule conformance, vs total wirecost, of 100,000 C. elegans ganglion layouts. ["If components a & b are connected, then a & b are adjacent."] Generally, the Adjacency Rule is not an effective heuristic to good wirecost. However, the small set of layouts best fitting the Adjacency Rule (points at far left) behave markedly differently: they correspond closely to the best wirecost layouts.

  28. Edge Core Parcellation of functional areas of macaque cerebral cortex. Component placement optimization analysis of layout of 17 core areas (white) of visual cortex, along with immediately contiguous edge areas (dark gray). Reported interconnections among core areas are indicated by lighter straight lines. Rostral is to right. In a connection cost analysis, this actual layout of the core visual system ranks in the top one-millionth of all alternative layouts.

  29. Cerebral cortex of cat. Placement of 39 interconnected functional areas of visual, auditory, somatosensory systems. Exhaustive search of samples of alternative layouts suggests this actual layout ranks in top 100 billionth of all possible layouts wrt Adjcost. -- "Best of all possible brains"?

  30. References Cherniak, C (1995) Neural component placement. Trends in Neurosciences, 18: 522-527. Cherniak, C., Changizi, M., & Kang, D. (1999) Large-scale optimization of neuron arbors. Physical Review E, 59: 6001-6009. Cherniak, C, Mokhtarzada, Z, Rodriguez, R, & Changizi, K (2004) Global optimization of cerebral cortex layout. Proc Nat Acad Sci, 101: 1081-1086. Cherniak, C (2005) Innateness and brain-wiring optimization. In: A Zilhao, ed, Evolution, Rationality and Cognition (Routledge) 103-112. www.glue.umd.edu/~cherniak/

More Related