Introduction to Neural Networks: A New Era in Computing

1. INTRODUCTION TO NEURAL NETWORKS

2. A new sort of computer • What are (everyday) computer systems good at... and not so good at?

3. Neural networks to the rescue… • Neural network:information processing paradigm inspired by biological nervous systems, such as our brain • Structure: large number of highly interconnected processing elements (neurons) working together • Like people, they learn from experience (by example)

4. What is NN? “Data processing system consisting of a large number of simple, highly interconnected processing elements (artificial neurons) in an architecture inspired by the structure of the cerebral cortex of the brain” (Tsoukalas & Uhrig, 1997).

5. Inspiration from Neurobiology Human Biological Neuron

6. Inspiration from Neurobiology Signal Processing • A neuron: many-inputs / one-output unit • output can be excited or not excited • incoming signals from other neurons determine if the neuron shall excite ("fire") • Output subject to attenuation in the synapses, which are junction parts of the neuron

7. Inspiration from Neurobiology Artificial Neuron The components of a basic artificial neuron Four basic components of a human biological neuron

8. Inspiration from Neurobiology • Neural networks are configured for a specific application, such as pattern recognition or data classification, through a learning process • In a biological system, learning involves adjustments to the synaptic connections between neurons  same for artificial neural networks (ANNs)

9. Inspiration from Neurobiology NN General Architecture • NN deals with training samples belonging to known classes and finding a generalized classifier to predict the class for any new samples. Hidden layer Input layer Output layer Attribute1 Attribute2 Attribute3 NN general architecture

10. Where can neural network systems help… • when we can't formulate an algorithmic solution. • when we can get lots of examples of the behavior we require. ‘learning from experience’ • when we need to pick out the structure from existing data.

11. History • 1943 McCulloch-Pitts neurons • 1949 Hebb’s law • 1958 Perceptron (Rosenblatt) • 1960 Adaline, better learning rule (Widrow, Huff) • 1969 Limitations (Minsky, Papert) • 1972 Kohonen nets, associative memory

12. History • 1977 Brain State in a Box (Anderson) • 1982 Hopfield net, constraint satisfaction • 1985 ART (Carpenter, Grossfield) • 1986 Backpropagation (Rumelhart, Hinton, McClelland) • 1988 Neocognitron, character recognition (Fukushima)

13. Characterizations • Architecture – a pattern of connections between neurons • Learning Algorithm – a method of determining the connection weights • Activation Function

14. Problem Domains • Storing and recalling patterns • Classifying patterns • Mapping inputs onto outputs • Grouping similar patterns • Finding solutions to constrained optimization problems

15. 10 01 1 1 1 1 00 00 10 Input patterns 1 1 00 Coronary Disease Input layer Output layer ST OP 01 00 10 1 1 Neural Sorted Net 00 10 1 1 patterns . 1 1 00 Problem Domains

16. 10 10 01 00 00 00 1 1 1 1 1 1 Problem Domains

17. Features • Neurons can generalize novel input stimuli • Neurons are fault tolerant and can sustain damage

18. Who is interested?... • Electrical Engineers – signal processing, control theory • Computer Engineers – robotics • Computer Scientists – artificial intelligence, pattern recognition • Mathematicians – modelling tool when explicit relationships are unknown

19. ANN Applications • Signal processing • Pattern recognition, e.g. handwritten characters or face identification. • Diagnosis or mapping symptoms to a medical case. • Speech recognition • Human Emotion Detection • Educational Loan Forecasting

20. 1 1 1 1 20 20 37 37 10 10 1 1 ANN Applications Abdominal Pain Prediction Intensity Duration Pain Male Temp Age WBC Pain adjustable weights Diverticulitis Pancreatitis Appendicitis Ulcer Pain Obstruction Cholecystitis Duodenal Non-specific Small Bowel Perforated 0 0 0 1 0 0 0

21. ANN Applications Voice Recognition

22. ANN Applications Educational Loan Forecasting System