1 / 15

Artificial Intelligence 10. Neural Networks

Understand linear regression and neural networks for predictive modeling, along with optimization techniques like gradient descent and back propagation. Enhance your knowledge on approximating functions using neural networks and handling overfitting issues.

mjohn
Download Presentation

Artificial Intelligence 10. Neural Networks

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. Artificial Intelligence10. Neural Networks Japan Advanced Institute of Science and Technology (JAIST) Yoshimasa Tsuruoka

  2. Outline • Regression • Linear regression • Gradient descent • Neural networks • Back propagation • Lecture slides • http://www.jaist.ac.jp/~tsuruoka/lectures/

  3. Linear regression • Input: vector • Output: numerical value • Example • Predict the level of comfortableness from temperature and humidity

  4. Optimizing the weight vector • Minimize the sum of squared errors

  5. Gradient descent • Move in the direction of the negative gradient

  6. Optimizing the weight vector • Squared errors summed over the whole training samples • Squared error on a particular sample n • Stochastic gradient computed from samplen

  7. Neural networks • Two-layer neural network Hidden Layer Input Activation Output Input Output

  8. Activation function • Transforms the activation level of a unit into an output

  9. Optimizing the weight vector • Error w.r.t. a particular samplen • Gradient First layer Second layer

  10. Gradient • Second layer Error

  11. Gradient • First layer

  12. Gradient • In summary, Error in the first layer

  13. Back propagation • Backward propagation of errors The same technique can be applied to neural networks with more than one layer of hidden units

  14. Neural networks • Capacity of approximating an arbitrary function • Prone to overfitting • The error function is not convex • Gradient descent can only give you local minima

  15. Questionnaires • Lecture codeI2152

More Related