1 / 20

Neural Networks

Neural Networks. Introduction. Artificial Neural Networks (ANN) Connectionist computation Parallel distributed processing Biologically Inspired computational models Machine Learning Artificial intelligence

urania
Download Presentation

Neural Networks

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. Neural Networks

  2. Introduction • Artificial Neural Networks (ANN) • Connectionist computation • Parallel distributed processing • Biologically Inspired computational models • Machine Learning • Artificial intelligence "the study and design of intelligent agents" where an intelligent agent is a system that perceives its environment and takes actions that maximize its chances of success.

  3. History • McCulloch and Pitts introduced the Perceptron in 1943. • Simplified model of a biological neuron • The drawback in the late 1960's • (Minsky and Papert) • Perceptron limitations • The solution in the mid 1980's • Multi-layer perceptron • Back-propagation training

  4. Summary of Applications • Function approximation • Pattern recognition/Classification • Signal processing • Modeling • Control • Machine learning

  5. Biologically Inspired. • Electro-chemical signals • Threshold output firing • Human brain: About 100 billion (1011) neurons and • 100 trillion (1014) synapses

  6. The Perceptron • Sum of Weighted Inputs. • Threshold activation function

  7. Activation Function • The sigmoid function: Logsig(Matlab)

  8. Activation Function • The tanH function: tansig (Matlab)

  9. The multi layer perceptron (MLP) W2 W3 W1 f f f f f f ... zin f f f zout ... ... ... f f f 1 1 1 Y2 X2 Y1 X3 Y3 X1 Y0 F1 F2 F3 zin zout W2 W1 W3 1 1 1

  10. The multi layer perceptron (MLP) Y2 X2 Y1 X3 Y3 X1 Y0 F1 F2 F3 zin zout W2 W1 W3 1 1 1

  11. Supervised Learning • Learning a function from supervised training data. A set of Input vectors Zinand corresponding desired output vectors Zout. • The performance function

  12. Supervised LearningGradient descent backpropagation • The Back Propagation Error Algorithm

  13. BPE learning. f S Y1 X1 F zin zout W2 W1 f S 1 ... f S ... ... f S 1

  14. Neural Networks 0 Collect data. 1 Create the network. 2 Configure the network. 3 Initialize the weights. 4 Train the network. 5 Validate the network. 6 Use the network.

  15. Collect data. Lack of information in the traning data. The main problem ! • As few neurons in the hidden layer as posible. • Only use the network in working points represented in the traningdata. • Use validation and test data. • Normalize inputs/targets to fall in the range [-1,1] or have zero mean and unity variance

  16. Create the network.Configure the network.Initialize the weights. f S f S ... f S ... ... f S Only one hidden layer. 1 Number of neurons in the hidden layer

  17. Train the network.Validate the network. Dividing the Data into three subsets. Training set (fx. 70%) Validation set (fx. 15%) Test set (fx. 15%) trainlm: Levenberg-Marquardt trainbr: Bayesian Regularization trainbfg: BFGS Quasi-Newton trainrp: Resilient Backpropagation trainscg: Scaled Conjugate Gradient traincgb: Conjugate Gradient with Powell/Beale Restarts traincgf: Fletcher-Powell Conjugate Gradient traincgp: Polak-Ribiére Conjugate Gradient trainoss: One Step Secant traingdx: Variable Learning Rate Gradient Descent traingdm: Gradient Descent with Momentom traingd: Gradient Descent Number of iterations.

  18. Other types of Neural networks The RCE net: Only for classification. o X1 x o x o x x x o o x x x o o o x x x o o X2

  19. Other types of Neural networks The RCE net: Only for classification. o X1 x o l x o x l x x S o ... l o x ... x x l o o o x x x o o X2

  20. Parzen Estimator Y X G S G Xin Yout / ... G S ... G Yout x x x x x x x x Xin

More Related