1 / 19

Neural Networks

Neural Networks. William Lai Chris Rowlett. What are Neural Networks?. A type of program that is completely different from functional programming. Consists of units that carry out simple computations linked together to perform a function

kachina
Download Presentation

Neural Networks

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. Neural Networks William Lai Chris Rowlett

  2. What are Neural Networks? • A type of program that is completely different from functional programming. • Consists of units that carry out simple computations linked together to perform a function • Modeled after the decision making process of the biological network of neurons in the brain

  3. The Biology of Neural Networks • Neural Networks are models of neuron clusters in the brain • Each Neuron has a: • Dendrites • Axon • Terminal buds • Synapse • Action potential is passed down the axon, which causes the release of neurotransmitters

  4. Types of Neural Networks:General • Supervised • During training, error is determined by subtracting output from actual value • Unsupervised • Nothing is known of results • Used to classify complicated data • Nonlearning • Optimization

  5. Types of Neural Networks:Specific • Perceptrons • A subset of feed-forward networks, containing only one input layer, one output layer, and each input unit links to only output units • Feed-forward networks • a.k.a. Directed Acyclic Graphs • Each unit only links to units in subsequent layers • Allows for hidden layers • Recurrent networks • Not very well understood • Units can link to units in the same layer or even previous layers • Example: The Brain

  6. Neural Net Capabilities • Neural Nets can do anything a normal digital computer can do (such as perform basic or complex computations) • Functional Approximations/Mapping • Classification • Good at ignoring ‘noise’

  7. Neural Net Limitations • Problems similar to Y=1/X between (0,1) on the open interval • (Pseudo)-random number predictors • Factoring integers or determining prime numbers • Decryption

  8. History of Neural Networks • McColloch and Pitts (1943) • Co-wrote first paper on possible model for a neuron • Widrow Hoff (1959) • Developed MADALINE and ADALINE • MADALINE was the first neural network to try to solve a real world problem • Eliminates echo in phone lines • vonNeumann architecture took over for about 20 years (60’s-80’s)

  9. Early Applications • Checkers (Samuel, 1952) • At first, played very poorly as a novice • With practice games, eventually beat its author • ADALINE (Widrow and Hoff, 1959) • Recognizes binary patterns in streaming data • MADALINE (same) • Multiple ADAptive LINear Elements • Uses an adaptive filter that eliminates echoes on phone lines

  10. Modern Practical Applications • Pattern recognition, including • Handwriting Deciphering • Voice Understanding • “Predictability of High-Dissipation Auroral Activity” • Image analysis • Finding tanks hiding in trees (cheating) • Material Classification • "A real-time system for the characterization of sheep feeding phases from acoustic signals of jaw sounds"

  11. How Do Neural Networks Relate to Artificial Intelligence? • Neural networks are usually geared towards some application, so they represent the practical action aspect of AI • Since neural networks are modeled after human brains, they are an imitation of human action. However, than can be taught to act rationally instead. • Neural networks can modify their own weights and learn.

  12. The Future of Neural Networks • Pulsed neural networks • The AI behind a good Go playing agent • Increased speed through the making of chips • robots that can see, feel, and predict the world around them • improved stock prediction • common usage of self-driving cars • Applications involving the Human Genome • Project self-diagnosis of medical problems using neural networks

  13. Past Difficulties • Single-layer approach limited applications • Converting Widrow-Hoff Technique for use with multiple layers • Use of poorly chosen and derived learning function • High expectations and early failures led to loss of funding

  14. Recurring Difficulties • Cheating • Exactly what a neural net is doing to get its solutions is unknown and therefore, it can cheat to find the solution as opposed to find a reliable algorithm • Memorization • Overfitting without generalization

  15. Describing Neural Net Units • All units have input values, aj • All input values are weighted, as in each aj is multiplied by the link’s weight, Wj,i • All weighted inputs are summed, generating ini • The unit’s activation function is called on ini, generating the activation value ai • The activation value is output to every destination of the current unit’s links.

  16. OR XOR Perceptrons • Single layer neural networks • Require linearly separable functions • Guarantees the one solution

  17. Back-Propagation • Back-propagation uses a special function to divide the error of the outputs to all the weights of the network • The result is a slow-learning method for solving many real world problems

  18. Organic vs. Artificial • Computer cycle times are in the order of nanoseconds while neurons take milliseconds • Computers compute the results of each neuron sequentially, while all neurons in the brain fire simultaneously every cycle • Result: massive parallelism makes brains a billion times faster than computers, even though computer bits can cycle a million times faster than neurons

  19. Questions?

More Related