1 / 12

Introduction to Neural Networks

Introduction to Neural Networks. Freek Stulp. Overview. Biological Background Artificial Neuron Classes of Neural Networks Perceptrons Multi-Layered Feed-Forward Networks Recurrent Networks Conclusion. Biological Background. Neuron consists of: Cell body Dendrites Axon Synapses.

berget
Download Presentation

Introduction to Neural Networks

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. Introduction to Neural Networks Freek Stulp

  2. Overview • Biological Background • Artificial Neuron • Classes of Neural Networks • Perceptrons • Multi-Layered Feed-Forward Networks • Recurrent Networks • Conclusion

  3. Biological Background • Neuron consists of: • Cell body • Dendrites • Axon • Synapses • Neural activation : • Throught dendrites/axon • Synapses have different strengths

  4. ini= SajWji ai= g(ini) Artificial Neuron Input links (dendrites) Unit (cell body) Output links (axon) aj Wji ai

  5. Ij Wj O W0 -1 W1 in= SajWj a = g(in) a1 a W2 a2 { 0, in<0 g(in) = 1, in>0 a = g(-W0 + W1a1 + W2a2) Class I: Perceptron

  6. Perceptrons can learn mappings from inputs I to outputs O by changing weights W Training setD: Inputs: I0, I1 ... In Targets: T0, T1 ...Tn Example: boolean ORD: Output O of network is not necessary equal to T! Learning in Perceptrons

  7. Learning in Perceptrons • Error often defined as: E(W) = 1/2SdD(td-od)2 • Go towards the minimum error! • Update rules: • wi = wi + Dwi • Dwi = -hdE/dwi • dE/dwi =d/dwi 1/2SdD(td-od)2=SdD(td-od)iid • This is called gradient descent i

  8. Feed-forward: Output links only connected to input links in the next layer • Multiple layers: • hidden layer(s) Input Hidden Output Class II: Multi-layer Feed-forward Networks Complex non-linear functions can be represented

  9. For output layer, weight updating similar to perceptrons. Problem: What are the errors in the hidden layer? Backpropagation Algorithm For each hidden layer (from output to input): For each unit in the layer determine how much it contributed to the errors in the previous layer. Adapt the weight according to this contribution This is also gradient descent Learning in MLFF Networks

  10. Input Hidden Output Class III: Recurrent Networks No restrictions on connections Behaviour more difficult to predict/ understand

  11. Inspiration from biology, though artificial brains are still very far away. Perceptrons too simple for most problems. MLFF Networks good as function approximators. Many of your articles use these networks! Recurrent networks complex but useful too. Conclusion

  12. Literature • Artificial Intelligence: A Modern Approach • Stuart Russel and Peter Norvig • Machine Learning • Tom M. Mitchell

More Related