1 / 13

History of Neural Computing

History of Neural Computing. McCulloch - Pitts 1943 - showed that a ”neural network” with simple logical units computes any computable function - beginning of Neural Computing, Artificial Intelligence, and Automaton Theory Wiener 1948

ash
Download Presentation

History of Neural Computing

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. History of Neural Computing • McCulloch - Pitts 1943 - showed that a ”neural network” with simple logical units computes any computable function - beginning of Neural Computing, Artificial Intelligence, and Automaton Theory • Wiener 1948 - Cybernetics, first time statistical mechanics model for computing - - compare Hopfield 1982 • Hebb 1949 - physiological learning rule based on the synaptic modification, Hebbian learning - - repeated synaptic activity strenghen synaptic response

  2. Marvin Minsky 1954 - ”Neural - anlog system & brain model” Ph.D. thesis at Princeton - An article ”Toward AI” in 1961, a chapter ”Neural Computing” - The book ”Computation: Finite and infinite machines” transform McCulloch - Pitts results into Automaton theory • Gabor 1954 - nonlinear adaptive filter • Taylor 1956 - associative memory -> learning matrix - also early works for correlation matrix memory (Anderson 1972, Kohonen 1972, Nakano 1972)

  3. PERCEPTRON

  4. PERCEPTRON • Rosenblatt 1958 - a new method for supervised learning ”perceptron convergence theorem” • Widrow - Hoff 1960 - LMS-algorithm for learning Adaline • Widrow 1962 - Madaline: leyered neural networks • Amari 1967 - stochastic gradient method • Nilsson 1965 - linearly separable sets During golden era of Perceptrons, in 60’s, it was believed that they solve all problems.

  5. PERCEPTRON • Minsky - Papert 1969 - the book ”Perceptrons” - showed mathematically the restrictions of 1-leyer perceptrons - they doubted that more leyers do not bring essentially more power Neural Network research went into ”HALT” state • The research was low about ten years - reasons: low computing power psychologically math results • research was continued in neurosciences and in psychology

  6. Self-Organizing Maps • This reseach was continued during ”Perceptron-halt” • von der Malsburg 1973 - first demonstration of self-organization - first paper was inspired by topological maps in brain • Grossberg 1980 - a new form of self-organization; ART • Kohonen 1982 - 1 and 2 dimensional lattice, different to von der Malsburg - nowadays a benchmark SOM

  7. Self-Organizing Maps

  8. Hopfield networks • Hopfield 1982 - formulation of an energy function for understanding how attraction network work - popular in 80’s: feedback Neural Net = Hopfield Net - no neurophysiologically adequate, but interesting since information could be stored into a stable net • Paper triggered a new era of Neural Networks • Paper caused much controversy, there were similar ideas in the literature: Cragg-Tamperley (1954), Cowan (1967), Grossberg (1967)

  9. New rise of NN • Kirkpatrick - Gelatt - Vecchi 1983 - Simulated annealing for combinatorial optimization problem - idea from statistical mechanics model for cooling in crystal formation • Ackley - Hinton - Sejnowski 1985 - Bolzmann machine, first succeeded realization of multileyer network --> earlier psychological barrier was broken • Barto - Sutton - Anderson 1983 - reinforcement learning, balance of a broomstick

  10. MULTILEYER PERCEPTRON

  11. (Error) Back Propagation • Problem in multileyer perceptron network: How to update the weights? • Rumelhart - Hinton - Williams 1986 - The book: Parallel Distributed Processing - back propagation algorithm solve problem - most popular learning algorithm for MLP’s • found also by Parker 1985, LeCun 1985 • earlier by Werbos 1974 (Bryson-Ho 1969)

  12. MULTILEYER PERCEPTRON

  13. Latest additions • Broomhead - Lowe 1988 - Radial basis functions (RBF) - input leyer : nonlinear hidden leyer : linear output leyer - link neural networks to numerical analysis • Linsker 1988 - self organization in perceptual networks - triggered again interest of information theorists • Bell - Sejnowski 1995 - blind source separation

More Related