Cognitive Computing via Synaptronics and Supercomputing

1. Cognitive Computing via Synaptronics and Supercomputing

2. "The information that comes from deep in the evolutionary past we call genetics. The information passed along from hundreds of years ago we call culture. The information passed along from decades ago we call family, and the information offered months ago we call education. But it is all information that flows through us. The brain is adapted to the river of knowledge and exists only as a creature in that river. Our thoughts are profoundly molded by this long historic flow, and none of us exists, self-made, in isolation from it."

3. Inflection Point 1: Neuroscience has matured 1414 pages

4. Inflection Point 2: Supercomputing meets Brain

5. Mammalian-scale simulation in near real-time? Computation Memory Communication

6. Cat 763 x 106 6.1 x 1012 Monkey 2 x 109 20 x 1012 Mouse 16 x 106 128 x 109 Rat 56 x 106 448 x 109 Human 22 x 109 220 x 1012 Almaden BG/L December, 2006 Watson BG/L April, 2007 WatsonShaheen BG/P March, 2009 LLNL Dawn BG/P May, 2009 BlueGene Meets Brain N: S: New results for SC09 Latest simulations achieve unprecedented scale of 109 neurons and 1013 synapses

7. Inflection Point 3: Nanotechnology meets Brain

8. Novel non-von Neumann Architectures are necessary Data from Todd Hylton Brain can be realized in modern electronics

9. Turning Back the Clock Digital, synchronous conventional, 5GHz(compare Power 6, 2008) Digital, semi-synchronous, 5 MHz(compare IBM PC/8088, 1978) Digital-Analog, asynchronous, clockless(compare the Brain) Digital, asynchronous, 100 kHz(compare ENIAC, 1946) Commandment:Do what is necessary, when it is necessary, and only that which is necessary.

10. Dharmendra S ModhaIBM Research – Almaden Raghavendra SinghIBM Research – India Network Architecture of the White Matter Pathways in the Macaque BrainPNAS (July 2010)

11. The connection model • Cortex has evolved such that it is organized into areas with distinct structural and functional properties • Primary sensory areas • Association areas • Motor areas • The white matter (myelinated nerve cell) underneath the outer covering of gray matter (nerve cell bodies), interconnects different regions of the central nervous system and carries nerve impulses between neuron • Model each area as a node and each connection as an edge in a graph • Analysis and Visualization of the brain • Wire length minimization • Organizational model that suggest the flow of information from input of sensory signals to the eventual output by motor neurons • Use model to simulate dynamics in the simulator

12. CoCoMac: Connectivity data on the Macaque brain Rolf Kotter, Klass Stephen, 2000 413 literature reports 7007 brain sites 8003 mapping details 2508 tracer injections 39748 connection details

13. AP84-TE PG91b-IT BP82-46 SP89a-46 FV91-V4 FV91-TF RV99-TF FV91-V1 RV99-CA1 BR98-CA1 FV91-TH BR98-TH כ כ = = = = PFCd 9 PCi IPL PF#1 7b 24c כ כ = = = = = CCa 24c 24 24d CMAr Divergent Nomenclature and Abundant Conflicts

24. Bundling Algorithm by Holten, 2006

28. Kaiser, Hilgetag, 2006

29. Notable Collations

30. Notable Collations

32. Cingulum Bundle

33. Uncinate Fasciculus

34. C, C, C, C, C, and C Complete Cortex, Thalamus, Basal Ganglia Comprehensive Includes every study in CoCoMac Consistent Every connection can be tracked back Concise 6,877 areas to 383 Coherent Unified hierarchical parcellation Colossal 3 times larger than previous network wetware to software

35. Aggregate Statistics

36. Brain is small-world SCC: 351 areas, 6,491 connections

37. “Organized Complexity” – Weaver, 1948

38. Degree Distribution Consistent with Exponential

39. Prefrontal Cortex is Topologically Central

41. Brain is small-world, Core is “tiny”-world! Core contains only 32% of vertices yet 88% of all edges originate or terminate in the core

42. Core contains correlated-anti-correlated networksand may be a key to consciousness Fox, Snyder, Vincent, Corbetta, Van Essen, and Raichle, 2005

43. Inter-chip Connectivity

44. Rent’s Rule • Rent's rule pertains to the organization of computing logic, specifically the relationship between the number of external signal connections (C) to a logic block with the number of logic gates (N) in the logic block • E.F. Rent observed a power-law relationship in the 1960’s - the law has been shown to hold true for small circuits upto mainframe computers 0 ≤ p≤1 is the Rent parameter and k is the Rent coefficient. • Intrinsically it’s a surface area (wire) to volume (number of nodes) relationship • Represents a cost-efficient solution to the challenge of embedding a high dimensional functional interconnect topology in a relatively low dimensional physical space with economical wiring costs • Circuits with greater logical capacity have higher values of Rent parameter • Microprocessor (0.45), Gates Arrays (0.5), High speed Computers (0.63) • For 2D layouts p> 0.5 implies that wires must grow longer as circuit size increases; global connections dominate over local connections for large p • The relative contribution of wiring to layout area will grow with the size of circuit to allow space for a greater number of wires to pass between adjacent nodes, increasing the node-to-node spacing • Allometric scaling • Gray (physical) to white (logical) matter scaling - Zhang & Sejnowski

45. Rent’s Rule • High value of p • Topological dimensionality of network greater than 3, i.e., greater than the dimensionality of the Euclidean space in which the network is embedded • Communication is a significant factor of power and space • Tradeoff between wiring costs and greater logical capacity. • Rewiring the network so as to reduce its topological dimension results in loss of functional modularity

47. “white matter is nature’s finest masterpiece” Nicolaus Steno, 1669

48. Owing both to limitations in hardware and architecture, these (convential) machines are of limited utility in complex, real-world environments, which demand an intelligence that has not yet been captured in an algorithmic-computational paradigm. As compared to biological systems for example, today’s programmable machines are less efficient by a factor of one million to one billion in complex, real-world environments. The SyNAPSE program seeks to break the programmable machine paradigm and define a new path forward for creating useful, intelligent machines. The vision for the anticipated DARPA SyNAPSE program is the enabling of electronic neuromorphic machine technology that is scalable to biological levels. Programmable machines are limited not only by their computational capacity, but also by an architecture requiring (human-derived) algorithms to both describe and process information from their environment. In contrast, biological neural systems (e.g., brains) autonomously process information in complex environments by automatically learning relevant and probabilistically stable features and associations. Since real world systems are always many body problems with infinite combinatorial complexity, neuromorphic electronic machines would be preferable in a host of applications—but useful and practical implementations do not yet exist.