1 / 21

Neural Networks

Neural Networks. Joost N. Kok Universiteit Leiden. Neural Networks. Book: Introduction to the theory of Neural Computation by Hertz, Krogh, Palmer Website: www.liacs.nl/home/joost/nn08.htm Additional Material: Powerpoint Sheets, Journal Articles, practical exercises

ksouth
Download Presentation

Neural Networks

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. Neural Networks Joost N. Kok Universiteit Leiden

  2. Neural Networks • Book: Introduction to the theory of Neural Computation by Hertz, Krogh, Palmer • Website: www.liacs.nl/home/joost/nn08.htm • Additional Material: Powerpoint Sheets, Journal Articles, practical exercises • Other recommended books: Haykin, Bishop

  3. The Von Neumann architecture

  4. The Hungarian-born mathematician, John von Neumann (1903-1957)

  5. The biological architecture

  6. Biological computers Five distinguishing properties: • Highly parallel • Robust and fault tolerant • Adaptive • Deals with fuzzy, noisy information • Small, compact

  7. Graceful Degradation performance damage

  8. Neurons

  9. Brain consists of 100000000000 (1011) neurons

  10. Neural activity out in

  11. Artificial Neuron

  12. 1 f(x) = 1 + e -x/a Input-output function • nonlinear function: a  0 f(e) a   e

  13. wAB A B Artificial Connections (Synapses) • wAB • The weight of the connection from neuron A to neuron B

  14. Supervised Networks

  15. Example

  16. Real Neurons • Nonlinear Summation • Sequences of pulses • No fixed time-delay

  17. History • 1943: McCulloch and Pitts: artificial neuron • 1960: Rosenblatt: perceptrons • 1969: Minsky and Papert: XOR problem • 197?: Associative content-addressable memory • 198?: Hopfield Networks, Boltzmann Networks • 1985: Backpropagation learning rule • 2000+ : Spiking networks, Support Vector Machines

  18. Hopfield McCulloch Pitts Papert Minsky Boltzmann Hebb

  19. Issues • Neurocomputing vs. Neuroscience • Types of Learning: • Supervised • Unsupervised • Reinforcement

  20. Research Questions • Design: what is best architecture? • Learning: find good algorithms • Analysis: what is the power of networks? • Implementation: how should the network be implemented?

More Related