An Introduction to Neural Networks

1. An Introduction to Neural Networks Presented by Scott Lichtor

2. Motivation I found for Neural Networks • Pavlov’s dog • Simple->Complex • Learning

3. Overview • Basics of the Nervous System • Neurons • Synapses • Action Potentials • Neural Networks • Abstract Neurons • More Complicated Neurons • Learning • Supervised • Unsupervised • Reinforcement • Conclusion

4. Basics of the Nervous System • The nervous system coordinates the actions of an animal • Body parts send messages to the brain • Brain sends messages to body parts • The basic unit of the nervous system is the neuron

5. Neurons • Receive messages at the dendrites • Message is sent quickly down the axon using electrical impulses • What happens when the signal reaches the end of the axon? Image taken from img460.imageshack.us

6. Synapses • Chemical Synapses • Slow • Strong • Can be transmitted over long distances Image taken from http://www.airlinesafety.com/editorials

7. Synapses • Electrical Synapses • Very fast • Fade quickly Image taken from wikipedia.org

8. Action Potentials • Action potentials are shocks to a particular neuron • The shock travels along the affected neuron • Then, the action potential is transmitted from the affected neuron to the neurons connected to it • The shock is transmitted to its destination in the same fashion

9. Abstract Neurons • So biological neurons can be used to send modified messages from place to place • Can be used to accomplish very complex tasks using relatively simple parts • Can neurons represent other things/be used for other objectives?

10. Abstract Neurons • Neurons can represent neuron-like things • Inputs -> Processes -> Outputs Image taken from http://3.bp.blogspot.com/

11. Abstract Neurons • Can “train” the neurons • Neurons fire (output 1) under certain patterns • Don’t fire (output 0) under other patterns • Firing rule: if an outcome doesn’t fit in either pattern, it fires if it has more in common with the first set, and doesn’t fire if it has more in common with the second set. • If there’s a tie, the neuron may fire, or it may not

12. Abstract Neurons • Example • A neuron takes three inputs (X1, X2, X3) • The neuron is trained to output 1 if the inputs are 111 or 101 • Trained to output 0 if the inputs are 000 or 001 • Before firing rule: • After firing rule:

13. Abstract Neurons • The abstract neuron model can be used for pattern recognition • Example: determine whether a ‘T’ or ‘H’ is displayed • Can we model more complicated processes with neurons?

14. More Complicated Neurons • McCulloch and Pitts model • Difference from previous model: inputs are weighted. • Add weighted inputs together: if the sum is greater than a threshold, then the neuron fires Image taken from http://www.doc.ic.ac.uk/~nd/surprise_96/journal/vol4/cs11/report.html

15. More Complicated Neurons • Mathematically: neuron fires if • X1W1 + X2W2 + X3W3 + ... > T

16. Examples AND Gate XOR Gate Image taken from http://www.heatonresearch.com

17. More Complicated Neurons • New model is very adaptable/powerful • Input weights and threshold can be changed so the neuron responds differently/more accurately to a situation • Pavlov’s dog • Various algorithms adapt neurons and neural networks to situations • Delta rule (feed-forward networks) • Back-error projection (feedback networks)

18. Learning • For the network to adapt, it must learn. • There are three types of learning used with neural networks: • Supervised learning • Unsupervised learning • Reinforcement learning

19. Supervised Learning • In supervised learning, the system learns using test data given from an external teacher • The test data tells the system what outputs result from certain inputs • The system tries to match the response of the test data, i.e. minimize the error between the neural network outputs and the test data outputs given the same inputs Image taken from http://www.learnartificialneuralnetworks.com

20. Unsupervised Learning • In unsupervised learning, the network is given no output data • Instead, the network is given just input data • The goal of the network, then, is to group the input data • Example: mortgage requests • The network is given credit ratings, size of mortgage, interest rate, etc. • The network groups the data; probably into accept and deny

21. Reinforcement Learning • Network performs actions on the input data • The environment grades the network (good or bad) • The network makes adjustments accordingly • Middle ground between supervised and unsupervised learning

22. Conclusion • The learning aspect of neural networks makes their applications astounding • For computers, one has to know how to solve a particular problem • Neural networks can solve problems that one doesn’t know how to solve

23. Conclusion • Just some of the uses: sales forecasting, stock market prediction, customer research, modeling and diagnosing the cardiovascular system, “Instant Physician”, interpretation of multi-meaning Chinese words, facial recognition, etc. etc. etc. • Something I found interesting: the interconnectedness of different subjects

24. Sources • http://www.doc.ic.ac.uk/~nd/surprise_96/journal/vol4/cs11/report.html • http://www.learnartificialneuralnetworks.com/ • http://www.ryerson.ca/~dgrimsha/courses/cps721/unsupervised.html • http://www.willamette.edu/~gorr/classes/cs449/intro.html • http://www.statsoft.com/textbook/stneunet.html • http://www.wikipedia.org