1 / 18

Back-propagation network (BPN)

Back-propagation network (BPN). Student : Dah-Sheng Lee Professor: Hahn-Ming Lee Date:20 September 2003. Outline. What is a Neural Network ? Artificial Neural Network (ANN) property Back-propagation network (BPN) Reference. What is a Neural Network ?.

ramona-smith
Download Presentation

Back-propagation network (BPN)

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. Back-propagation network (BPN) Student : Dah-Sheng Lee Professor: Hahn-Ming Lee Date:20 September 2003

  2. Outline • What is a Neural Network ? • Artificial Neural Network (ANN) property • Back-propagation network (BPN) • Reference

  3. What is a Neural Network ? • Neural Networks are a different paradigm for computing: • von Neumann machines are based on the processing/memory abstraction of human information processing. • neural networks are based on the parallel architecture of animal brains. • Neural networks are a form of multiprocessor computer system, with • simple processing elements • a high degree of interconnection • simple scalar messages • adaptive interaction between elements

  4. What is a Neural Network ? (cont…) A biological neuron may have as many as 10,000 different inputs, and may send its output (the presence or absence of a short-duration spike) to many other neurons. Neurons are wired up in a 3-dimensional pattern. Real brains, however, are orders of magnitude more complex than any artificial neural network so far considered.

  5. 神經樹突（dendrites）(dendrites） 神經突觸（synapses）（synapses） 神經軸突（axon） 神經核（soma）

  6. What is a Neural Network ? (cont…) The dendrites are extensions of a neuron which connect to other neurons to form a neural network, while synapses are a gateway which connects to dendrites that come from other neurons. A biological neuron may thus be connected to other neurons as well as accepting connections from other neurons, and so we have the basis of a network. Through those connections, electrical pulses are transmitted, and information is carried in the timing and the frequency with which these pulses are emitted. So, our neuron receives information from other neurons, processes it and then relays this information to other neurons.

  7. Artificial Neural Network (ANN) property • ANN運作所需的範例資料有 • Training example • Testing example • 待推案例 • ANN characteristics • Input : training set,testing set • output • Processing Element(PE)

  8. Artificial Neural Network (ANN) property (cont…) • ANN Type • Supervised learning :Perceptron, BPN, PNN,LVQ,CPN • Unsupervised learning :SOM, ART • Associate learning :Hopfield, Bidirectional Associative Memory(BAM), Hopfield-Tank • Optimization application :HTN, ANN(Annealed Neural Network)

  9. X1 X2 Y ‧ ‧ ‧ Xn X1 X2 Y ‧ ‧ ‧ Xn X1 X2 Y ‧ ‧ ‧ Xn Artificial Neural Network (ANN) property (cont…) • ANN structure • One way feedforward • Two way feedforward • Feedback

  10. Artificial Neural Network (ANN) property (cont…) • ANN Basic Model • Processing Element(PE) : summation fc.,Activity fc.,transfer fc. • Input layer : [x1.....xi](training set,testing set) • Hidden layer : present PE's internal relationship • Output layer : normalize output,competitive output,competitive learning • Network : Learning,Recalling • Weights : connecting between layers

  11. Artificial Neural Network (ANN) property (cont…) • Transfer Function Type • Discrete Type • Perceptron/step fc. • Signum fc. • Signum 0 fc. • Hopfield-Tank fc. • BAM fc. • Linear Type • threshold line fc. • straight linear fc. • Nonlinear Type • sigmoid fc. • Hyperbolic tangent fc.

  12. Artificial Neural Network (ANN) property (cont…) (Single-layer Perceptrons)

  13. Back-propagation network (BPN) The network model “BPN” is • Supervised learning • Feedforward • Multilayer Perceptrons (Special case: no hidden layer)

  14. (Multilayer Perceprons)

  15. Back-propagation network (BPN)Training algorithm • Step 1: Initialize the network synaptic weights to small random value. • Step 2:Form the set of training input/output pairs, present an input pattern and calculate the network response. • Step 3: The desire network response is compared with the actual output of the network, and by using 1* and 2* all the local errors can be computed • Step 4:The weight of the network are update according to 3* • Step 5:Until the network reaches a predetermined level of accuracy in producing the adequate response for all the training pattern, continue step 2 through 4

  16. Back-propagation network (BPN)advantage algorithm • Backpropagation Learning Algorithm with Momentum Updating • Batch Updating • Search-Then-Converge Method • Batch Updating with Variable Learning Rate • etc…

  17. Reference • “Principles of Neuroncomputing for Science and Engineer” Fredric M. Ham; Ivica Kostanic; McGRAW-HILL INTERNATIONAL EDITION, 2001 • “類神經網路模式應用與實作” 8th 葉怡成 儒林出版社, 2003

More Related