1 / 16

Artificial Neural Networks

Explore the world of artificial neural networks for supervised learning, from the basics of k-Nearest Neighbor to advanced Neural Network Training. Discover how ANNs mimic human neurons and the connection with modern AI systems. Learn about the Perceptron algorithm, backpropagation, and handling non-linearly separable data.

Download Presentation

Artificial Neural Networks

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript


  1. Artificial Neural Networks • Artificial Neural Networks are (among other things) another technique for supervised learning k-Nearest Neighbor Decision Tree Neural Network Training Data Test Data Classification

  2. Human neuron • Dendrites pick up signals from other neurons • When signals from dendrites reach a threshold, a signal is sent down axon to synapse

  3. Connection with AI • Most modern AI: • “Systems that act rationally” • Implementing neurons in a computer • “Systems that think like humans” • Why artificial neural networks then? • “Universal” function fitter • Potential for massive parallelism • Some amount of fault-tolerance • Trainable by inductive learning, like other supervised learning techniques

  4. Perceptron Example 1 = malignant 0 = benign # of tumors w1 = -0.1 Output Unit w2 = 0.9 Avg area Avg density w3 = 0.1 Input Units

  5. The Perceptron: Input Units • Input units: features in original problem • If numeric, often scaled between –1 and 1 • If discrete, often create one input node for each category • Can also assign values for a single node (imposes ordering)

  6. The Perceptron: Weights • Weights: Represent importance of each input unit • Combined with input units to feed output units • The output unit receives as input:

  7. The Perceptron: Output Unit • The output unit uses an activation function to decide what the correct output is • Sample activation function:

  8. Simplifying the threshold • Managing the threshold is cumbersome • Incorporate as a “virtual” weight

  9. How to learn the right weights? • Need to redefine perceptron • “Step function” no good – need something differentiable • Replace with sigmoid approximation

  10. Sigmoid function • Good approximation to step function • As binfinity,sigmoid  step • We’ll just take b = 1 for simplicity

  11. Computing weights • Think of as a gradient descent method, where weights are variables and trying to minimize error:

  12. The Perceptron Learning Rule: How do we compute weights?

  13. Can appropriate weights always be found? • ONLY IF data is linearly separable

  14. What if data is not linearly separable? Neural Network. O • Each hidden unit is a perceptron • The output unit is another perceptron with hidden units as input Vj

  15. Backpropagation: How do we compute weights?

  16. Neural Networks and machine learning issues • Neural networks can represent any training set, if enough hidden units are used • How long do they take to train? • How much memory? • Does backprop find the best set of weights? • How to deal with overfitting? • How to interpret results?

More Related