1 / 29

Introduction to Artificial Neural Networks

Introduction to Artificial Neural Networks. Andrew L. Nelson Visiting Research Faculty University of South Florida. Overview. References Introduction History Biologically Inspired Applications The Perceptron Activation Functions Hidden Layer Networks Training with BP Examples.

ping
Download Presentation

Introduction to Artificial Neural Networks

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. Introduction to Artificial Neural Networks Andrew L. Nelson Visiting Research Faculty University of South Florida

  2. Overview • References • Introduction • History • Biologically Inspired • Applications • The Perceptron • Activation Functions • Hidden Layer Networks • Training with BP • Examples • Outline to the left • Current topic in red • Introduction • History and Origins • Biologically Inspired • Applications • Perceptron • Activation Functions • Hidden Layer Networks • Training with BP • Examples Neural Networks

  3. References • References • Introduction • History • Biologically Inspired • Applications • The Perceptron • Activation Functions • Hidden Layer Networks • Training with BP • Examples • W. S. McCulloch, W. Pitts, "A logical calculus of the ideas immanent in nervous activity," Bulletin of Mathematical Biophysics, vol. 5 pp. 115-133, 1943. • J. L. McClelland, D. E. Rumelhart, Parallel Distributed Processing: Explorations in the Microstructure of Cognition, The MIT Press, 1986. • C. M. Bishop, Neural Networks for Pattern Recognition, Oxford University Press, 1995. Neural Networks

  4. Introduction • References • Introduction • History • Biologically Inspired • Applications • The Perceptron • Activation Functions • Hidden Layer Networks • Training with BP • Examples • Artificial Neural Networks (ANN) • Connectionist computation • Parallel distributed processing • Computational models • Biologically Inspired computational models • Machine Learning • Artificial intelligence Neural Networks

  5. History • References • Introduction • History • Biologically Inspired • Applications • The Perceptron • Activation Functions • Hidden Layer Networks • Training with BP • Examples • McCulloch and Pitts introduced the Perceptron in 1943. • Simplified model of a biological neuron • Fell out of favor in the late 1960's • (Minsky and Papert) • Perceptron limitations • Resurgence in the mid 1980's • Nonlinear Neuron Functions • Back-propagation training Neural Networks

  6. Summary of Applications • References • Introduction • History • Biologically Inspired • Applications • The Perceptron • Activation Functions • Hidden Layer Networks • Training with BP • Examples • Function approximation • Pattern recognition • Signal processing • Modeling • Control • Machine learning Neural Networks

  7. Biologically Inspired • References • Introduction • History • Biologically Inspired • Applications • The Perceptron • Activation Functions • Hidden Layer Networks • Training with BP • Examples • Electro-chemical signals • Threshold output firing Neural Networks

  8. The Perceptron • References • Introduction • History • Biologically Inspired • Applications • The Perceptron • Activation Functions • Hidden Layer Networks • Training with BP • Examples • Binary classifier functions • Threshold activation function Neural Networks

  9. The Perceptron: Threshold Activation Function • References • Introduction • History • Biologically Inspired • Applications • The Perceptron • Activation Functions • Hidden Layer Networks • Training with BP • Examples • Binary classifier functions • Threshold activation function Neural Networks

  10. Linear Activation functions • References • Introduction • History • Biologically Inspired • Applications • The Perceptron • Activation Functions • Hidden Layer Networks • Training with BP • Examples • Output is scaled sum of inputs Neural Networks

  11. Nonlinear Activation Functions • References • Introduction • History • Biologically Inspired • Applications • The Perceptron • Activation Functions • Hidden Layer Networks • Training with BP • Examples • Sigmoid Neuron unit function Neural Networks

  12. Layered Networks • References • Introduction • History • Biologically Inspired • Applications • The Perceptron • Activation Functions • Hidden Layer Networks • Training with BP • Examples . Neural Networks

  13. SISO Single Hidden Layer Network • References • Introduction • History • Biologically Inspired • Applications • The Perceptron • Activation Functions • Hidden Layer Networks • Training with BP • Examples • Can represent and single input single output functions: y = f(x) Neural Networks

  14. Training Data Set • References • Introduction • History • Biologically Inspired • Applications • The Perceptron • Activation Functions • Hidden Layer Networks • Training with BP • Examples Adjust weights (w) to learn a given target function: y = f(x) Given a set of training data X→Y Neural Networks

  15. Training Weights: Error Back-Propagation (BP) • References • Introduction • History • Biologically Inspired • Applications • The Perceptron • Activation Functions • Hidden Layer Networks • Training with BP • Examples • Weight update formula: Neural Networks

  16. Error Back-Propagation (BP) • References • Introduction • History • Biologically Inspired • Applications • The Perceptron • Activation Functions • Hidden Layer Networks • Training with BP • Examples Training error term: e Neural Networks

  17. BP Formulation • References • Introduction • History • Biologically Inspired • Applications • The Perceptron • Activation Functions • Hidden Layer Networks • Training with BP • Examples Neural Networks

  18. BP Formulation • References • Introduction • History • Biologically Inspired • Applications • The Perceptron • Activation Functions • Hidden Layer Networks • Training with BP • Examples Neural Networks

  19. BP Formulation • References • Introduction • History • Biologically Inspired • Applications • The Perceptron • Activation Functions • Hidden Layer Networks • Training with BP • Examples Neural Networks

  20. BP Formulation • References • Introduction • History • Biologically Inspired • Applications • The Perceptron • Activation Functions • Hidden Layer Networks • Training with BP • Examples Neural Networks

  21. BP Formulation • References • Introduction • History • Biologically Inspired • Applications • The Perceptron • Activation Functions • Hidden Layer Networks • Training with BP • Examples Neural Networks

  22. BP Formulation • References • Introduction • History • Biologically Inspired • Applications • The Perceptron • Activation Functions • Hidden Layer Networks • Training with BP • Examples Neural Networks

  23. BP Formulation • References • Introduction • History • Biologically Inspired • Applications • The Perceptron • Activation Functions • Hidden Layer Networks • Training with BP • Examples Neural Networks

  24. BP Formulation • References • Introduction • History • Biologically Inspired • Applications • The Perceptron • Activation Functions • Hidden Layer Networks • Training with BP • Examples Neural Networks

  25. Example: The XOR problem: • References • Introduction • History • Biologically Inspired • Applications • The Perceptron • Activation Functions • Hidden Layer Networks • Training with BP • Examples • Single hidden layer: 3 Sigmoid neurons • 2 inputs, 1 output Desired I/O table (XOR): Neural Networks

  26. Example: The XOR problem: • References • Introduction • History • Biologically Inspired • Applications • The Perceptron • Activation Functions • Hidden Layer Networks • Training with BP • Examples • Training error over epoch Neural Networks

  27. Example: The XOR problem: • References • Introduction • History • Biologically Inspired • Applications • The Perceptron • Activation Functions • Hidden Layer Networks • Training with BP • Examples initial_weights = 0.0654 0.2017 0.0769 0.1782 0.0243 0.0806 0.0174 0.1270 0.0599 0.1184 0.1335 0.0737 0.1511 final_weights = 4.6970 -4.6585 2.0932 5.5168 -5.7073 -3.2338 -0.1886 1.6164 -0.1929 -6.8066 6.8477 -1.6886 4.1531 Neural Networks

  28. Example: The XOR problem: • References • Introduction • History • Biologically Inspired • Applications • The Perceptron • Activation Functions • Hidden Layer Networks • Training with BP • Examples Mapping produced by the trained neural net: Neural Networks

  29. Example: Overtraining • References • Introduction • History • Biologically Inspired • Applications • The Perceptron • Activation Functions • Hidden Layer Networks • Training with BP • Examples • Single hidden layer: 10 Sigmoid neurons • 1 input, 1 output Neural Networks

More Related