1 / 80

Computacion inteligente

Computacion inteligente. The perceptron. Agenda. History Perceptron structure Perceptron as a two-class classifier The Perceptron as a Logic Operator Perceptron Learning Perceptron Convergence Theorem Examples Perceptron Limitation. History. History.

lel
Download Presentation

Computacion inteligente

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. Computacion inteligente The perceptron

  2. Agenda • History • Perceptron structure • Perceptron as a two-class classifier • The Perceptron as a Logic Operator • Perceptron Learning • Perceptron Convergence Theorem • Examples • Perceptron Limitation

  3. History

  4. History • 1943 The perceptron was derived from a biological brain neuron model introduced by Mc Culloch & Pitts • Artificial neural networks in principle could compute any arithmetic and logical functions. • McCulloch and Pitts network cannot learn. • 1949 Hebb proposed a learning law that explained how a network of neurons learned

  5. History • 1957 Rosenblatt developed Perceptron, a hardware neural net for character recognition • Rosenblatt designed the perceptron with a view toward explaining & modeling pattern recognition abilities of biological visual systems • 1958 Rosenblatt was credited with the perceptron learni learning algorithm

  6. Rosenblatt’s elementary perceptron • A two-class problem that consists of determining whether the input pattern is a “p” or not

  7. History • 1969Minsky & Papert - perceptron limitation • Famous “XOR” problem • Death of ANN • 1974-1986 Various people solve the problems with perceptrons: • Algorithms for training feedforward multilayered perceptrons • Error Back Propagation (Rumelhart et al 1986) • Re-emergence of ANN

  8. Perceptron structure

  9. Perceptron structure

  10. Perceptron structure • Input patterns represented by vector x • wiis a modifiable weight associated with an incoming signal xi • The weighted sum of the inputs is applied to the hard limiter, with a threshold value b The threshold b = w0 can be viewed as the connection weight between the output unit and a dummy incoming signal x0 = 1

  11. Perceptron structure • The weighted sum of the inputs is applied to the hard limiter, which produces an output equal to +1 if its input is positive, and -1 if it is negative. • the hard limiteris

  12. Perceptron structure • the hard limitercan be • sgn(x): a sign function, or, hardlims(x) • step(x): a step function, or, hardlim(x) Matlab toolbox

  13. Perceptron mathematical model • Neuron output • Vector form b included

  14. Perceptron as a two-class classifier

  15. Perceptron as a two-class classifier • The aim of the perceptron is to classify inputs, x1, x2, . . ., xn, into one of two classes, say A1 and A2. • The input patterns belong to one of two classes. This can only happen when they are linearly separable

  16. Two-input perceptron • The decisionboundary is determined by

  17. x2 x1 Example of Perceptron Decision Boundary w = (w1, w2, b) = (1, -1, 0)

  18. x2 x1 Example of Perceptron Decision Boundary w = (w1, w2, b) = (1, -1, 0) wT. x = 0 => 1. x1 - 1. x2 + 0.1 = 0 => x1 - x2 = 0 => x1 = x2

  19. x2 x1 Example of Perceptron Decision Boundary w = (w1, w2, b) = (1, -1, 0) wT. x = 0 => 1. x1 - 1. x2 + 0.1 = 0 => x1 - x2 = 0 => x1 = x2 This is the equation for the decision boundary

  20. wT. x = 0 x2 x1 Example of Perceptron Decision Boundary w = (w1, w2, b) = (1, -1, 0) This is the equation for decision region -1 wT. x < 0 => x1 - x2 < 0 => x1 < x2

  21. wT. x = 0 x2 x1 Example of Perceptron Decision Boundary w = (w1, w2, b) = (1, -1, 0) wT. x > 0 => x1 - x2 > 0 => x1 > x2 This is the equation for decision region +1

  22. Example • Decision Boundary for

  23. Three-input perceptron • The decisionboundary is determined by

  24. n-input perceptron • In the case of an elementary perceptron, the n-dimensional space is divided by a hyperplane into two decision regions. • The hyperplane is defined by the linearlyseparable function:

  25. Example Matrix with 64 inputs shown

  26. The Perceptron as a Logic Operator

  27. x 2 1 0 0 x Truth Table for Logical AND 1 0 1 0 1 x y x & y 0 0 0 1 1 0 1 1 output inputs The Perceptron as a Logic Operator • The Perceptron as a Logic AND

  28. x +2 if sum < 0 : 0 else : 1 y +2 2x+3y-3 0 1 -3 0 Truth Table for Logical AND 0 1 sum output inputs weights x y x & y 0 0 0 1 1 0 1 1 output inputs The Perceptron as a Logic AND

  29. x 2 1 x Truth Table for Logical OR 1 0 1 x y x | y 0 0 0 1 0 1 1 1 0 1 1 1 output inputs The Perceptron as a Logic Operator • The Perceptron as a Logic OR

  30. x +2 if sum < 0 : 0 else : 1 y +2 2x+2y-1 1 -1 Truth Table for Logical OR sum output inputs weights x y x | y 0 0 0 1 0 1 1 1 0 1 1 1 output inputs The Perceptron as a Logic OR

  31. The Perceptron as a Logic Operator • Exercise • Show that a perceptron neuron with the followings weights and bias implements the OR gate

  32. Perceptron Learning

  33. Learning in natural neural networks • At the neural level the learning happens by • Changing of the synaptic strengths, • Eliminating some synapses, and building new ones. • This procedure is called the Learning rule

  34. Learning in artificial neural networks • By learning rule we mean a procedure for modifying the weights and biases of a network. • The purpose of the learning rule is to train the network to perform some task. This procedure may also referred to as a training algorithm.

  35. Learning Paradigms • Supervised learning • Unsupervised learning • Reinforcement learning • Classification • Control • Function approximation • Associative memory • Clustering • Control

  36. Supervised Learning • The learning rule is provided with • a set of examples (the training set) of proper network behavior xk :Input to the network tk : Correct (target) output

  37. Supervised Learning • As the inputs are applied to the network, the network outputs are compared to the targets ak :Output of the network tk :correct (target) output

  38. Supervised Learning • The learning rule is then used to adjust the weights and biases of the network in order to move the network outputs closer to the targets. • wnew: Updated Weights • wold: Initial weights weights adjusted according to error

  39. Supervised Learning weights adjusted according to error

  40. Perceptron Training • How can we train a perceptron for a classification task? • We try to find suitable values for the weights in such a way that the training examples are correctly classified. • Geometrically, we try to find a hyper-plane that separates the examples of the two classes

  41. Perceptron Training • Rosenblatt (1958) devised a learning algorithm for artificial neurons • start with a training set (example inputs & corresponding desired outputs) • train the network to recognize the examples in the training set (by adjusting the weights on the connections) • once trained, the network can be applied to new examples

  42. Perceptron learning algorithm • Set the weights on the connections with random values. • Iterate through the training set, comparing the output of the network with the desired output for each example. • If all the examples were handled correctly, then DONE. • Otherwise, update the weights for each incorrect example: If output = -1 but should be 1 If output = 1 but should be -1 • GO TO 2

  43. Variables and parameters • Training setS of examples {x,t} • x is an input vector and • t the desired target vector • Example: Logical AND • S = {(0,0),0}, {(0,1),0}, {(1,0),0}, {(1,1),1}

  44. Variables and parameters • Variables and parameters at iteration k of the learning algorithm: • Input vector: x(k) = [+1, x1(k), x2(k), …, xm(k)]T • Weight vector: w(k) = [b(k), w1(k), w2(k), …, wm(k)]T • Bias: b(k) • Actual response: y(k) • Desired response: t(k) • Learning rate parameter:  > 0

  45. Perceptron Learning Rule • If at iteration k, the actual output is y(k) and the desired output is t(k), then the error is given by: • wherek = 1, 2, 3, . . . • Iteration k here refers to the k-th training example presented to the perceptron.

  46. Perceptron Learning Rule • Perceptron Learning Rule • If the error, e(k), is positive, we need to increase perceptron output y(k), but if it is negative, we need to decrease y(k).

  47. Perceptron’s training algorithm • Step 1: Initialisation • Set initial weights w1, w2,…, wm and threshold b to random numbers. • Step 2: Activation • Activate the perceptron by applying inputs x1(k), x2(k),…, xm(k). Calculate the actual output at iteration k = 1

  48. Perceptron’s training algorithm • Step 3: Weight training • Update the weights of the perceptron • Step 4: Iteration • Increase iteration k by one, go back to Step 2 and repeat the process until convergence.

  49. Perceptron Convergence Theorem

  50. Perceptron Convergence Theorem • The perceptron learning algorithm will always find weights to classify the inputs if such a set of weights exists. Minsky & Papert showed such weights exist if and only if the problem is linearly separable key reason for interest in perceptrons

More Related