350 likes | 534 Views
Nanjing University of Science & Technology. Pattern Recognition: Statistical and Neural. Lonnie C. Ludeman Lecture 19 Oct 26, 2005. Lecture 19 Topics. 1. Structures of Optimal Statistical Classifiers 2. Neural Network History 3. Biological Neural Networks
E N D
Nanjing University of Science & Technology Pattern Recognition:Statistical and Neural Lonnie C. Ludeman Lecture 19 Oct 26, 2005
Lecture 19 Topics 1. Structures of Optimal Statistical Classifiers 2. Neural Network History 3. Biological Neural Networks 4. Modified McColloch Pitts Model – Example 5. Artificial Neural Element - Definition
Structures from Statistical Decision Theory and Perceptron Algorithm (a). Two Class Likelihood Ratio test (b) Structures to Motivate Neural Networks (c) Two Class General Linear (d). N Class General minimum P(error) (e) N Class Gaussian
Decision Rule if l(x) > T decide x from C1 < T decide x from C2 = T decide x randomly between C1 and C2
C1 if f(x) > T < C2 Decision Rule
C1 if f(x) > T < C2 Decision Rule
if p(x | Ck) > p(x | Cj ) for all j ≠ k decide Ck Decision Rule
These Structures will be seen to be similar to the structures used in designs with Neural Networks
Important Events in Neural Networks History 1943McColloch Pitts Model 1958Perceptron Algorithm- Rosenblatt 1960-62ADALINE – Widrow and Hoff 1969Minsky and Papert-Limitations 1980’sGrossberg , Hopfield, and Rumelhart – Backpropagation algorithms, Adaptive Resonance Theory Period of Revival 1990’s Maturation of Field
IMPORTANT BOOKS 1990Artificial Neural Systems- Jacek M. Zurada 1992Neural Networks and Fuzzy Systems- Bart Kosco 1994 Neural Networks: A Comprehensive Foundation- Simon Haykin
Neural Networks Biological (Real) Mathematical (Artificial) How do you tell them apart ??? Squeeze them !!! Excite them !!!
Modified McColloch Pitts Neuronal Model (Threshold Logic Unit (TLU) )
Question What can we do with a Modified McColloch-Pitts Model ??? Answer Surprisingly we can model all logical expressions !!!
Implementation of Logical AND and OR T = n - 1/2 T = 1/2
Implementation of Logical NOT -1/2 Since we can model logical AND, OR and NOT we can model alllogical expressions
Example – Implementation of a given logic expression Given: Implement f(x) using Modified McColloch-Pitts Neurons
Solution: -1/2 2.5 -1/2 T=1/2 -1/2 3.5 -1 -1/2
Artificial Neural Element (ANE) Node net Input Vector
Artificial Neural Element Mathematical Model net = w1x1 + w2x2 + …+ wnxn + wn+1 = wTx Linear Operation on x y = f(net) = f( wTx) Nonlinear Operation on x Nonlinear Activation Function
Artificial Neural Element Nodal Representation Vector Notation
Hyperbolic Tangent Activation Function Definition Equivalent Form f(net)
Non Monitonic Activation Functions can be very Useful Examples: Potential Function ApproachRadial Basis Functions
Summary Lecture 19 1. Structures of Optimal Statistical Classifiers 2. Neural Network History 3. Biological Neural Networks 4. Modified McColloch Pitts Model – Example 5. Artificial Neural Element - Definition