1 / 33

Deep Learning Tutorial

Deep Learning Tutorial. Mitesh M. Khapra IBM Research India (Ideas and material borrowed from Richard Socher’s tutorial @ ML Summer School 2014 Yoshua Bengio’s tutorial @ ML Summer School 2014 & Hugo Larochelle’s lecture videos & slides). Roadmap. What? Why? How? Where?.

aristotle-thornton
Download Presentation

Deep Learning Tutorial

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. Deep Learning Tutorial Mitesh M. Khapra IBM Research India (Ideas and material borrowed from Richard Socher’s tutorial @ ML Summer School 2014 YoshuaBengio’s tutorial @ ML Summer School 2014 & Hugo Larochelle’s lecture videos & slides)

  2. Roadmap • What? • Why? • How? • Where?

  3. Roadmap • What are Deep Neural Networks? • Why? • How? • Where?

  4. Roadmap • What are Deep Neural Networks? • Why should I be interested in Deep Learning? • How? • Where?

  5. Roadmap • What are Deep Neural Networks? • Why should I be interested in Deep Learning? • How do I make a Deep Neural Network work? • Where?

  6. Roadmap • What are Deep Neural Networks? • Why should I be interested in Deep Learning? • How do I make train a Deep Neural Network work? • Where?

  7. Roadmap • What are Deep Neural Networks? • Why should I be interested in Deep Learning? • How do I train a Deep Neural Network? • Where can I find additional material?

  8. the what?

  9. A typical machine learning example data label number of positive words, number of negative words, length of review, author name, bag of words, etc. feature vector feature extraction

  10. next A typical machine learning example data label

  11. So, where does deep learning fit in? • Machine Learning • hand crafted features • optimize weights to improve prediction • Representation Learning • automaticallylearn features • Deep Learning • automaticallylearn multiple levels of features From RicharSocher’s tutorial @ ML Summer School, Lisbon

  12. back The basic building block single artificial neuron

  13. Okay, so what can I use it for? • For binary classification problems by treating • Works when data is linearly separable (image from Hugo Larochelles’sslides)

  14. What are its limitations? • Fails when data is not linearly separable…. (images from Hugo Larochelles’sslides) • …unless the input is suitably transformed

  15. A neural network for XOR Wait…., are you telling me that I will always have to meditate on the data and then decide the transformation/network ? No, definitely not. The XOR example is only to give the intuition. The key takeaway is that by adding more layers you can make the data separable. A multi-layered neural network Lets spend some more time in understanding this ….

  16. (graphs from Pascal Vincent’s slides) Capacity of a multi-layer network

  17. Capacity of a multi-layer network (image from Pascal Vincent’s slides)

  18. Capacity of a multi-layer network In particular, we can find a separator for the XOR problem (images from from Pascal Vincent’s slides) • Universal Approximation Theorem (Hornik, 1991) : • “a single hidden layer neural network with a linear output unit can approximate any continuous function arbitrary well, given enough hidden units”

  19. Lets take a minute here… If “a single hidden layer neural network” is enough then why go deeper? Hand-crafted featuresrepresentations Automatically learned featuresrepresentations … … …

  20. Multiple layers = multiple levels of features But why would I be interested in learning multiple levels of representations ? Lets see where the motivation comes from…

  21. The brain analogy Layer 1 representation nose Layer 2 representation mouth eyes face Layer 3 representation (idea from Hugo Larochelle’s slides)

  22. YAWN!!!! Enough With the Brain Tampering Just tell me Why should I be interested In Deep Learning?(“Show Me the Money”)

  23. the why?

  24. (from Y. Bengio’s MLSS 2014 slides) Used in a wide variety of applications

  25. Industrial Scale Success Stories Speech Recognition Object Recognition Face Recognition Cross Language Learning Machine Translation Text Analytics Disclaimer: Some nodes and edges may be missing due to limited public knowledge Dramatic improvements reported in some cases

  26. (from Y. Bengio’s MLSS 2014 slides) Some more success stories

  27. Let me see if I understand this correctly… • Speech Recognition, Machine Translation, etc. are more than 50 years old • Single artificial neurons have been around for more than 50 years No, even deep neural networks have been around for many, many years but prior to 2006 training deep nets was unsuccessful 50+ years?

  28. (from Y. Bengio’s MLSS 2014 slides) So what has changed since 2006? • New methods for unsupervised pre-training have been developed • More efficient parameter estimation methods • Better understanding of model regularization • Faster machines and more data help DL more than other algorithms

  29. the how?

  30. recap single artificial neuron

  31. Switching to slides corresponding to lecture 2 from Hugo Larochelle’s course http://info.usherbrooke.ca/hlarochelle/neural_networks/content.html

  32. the where?

  33. Some pointers to additional material • http://deeplearning.net/ • http://info.usherbrooke.ca/hlarochelle/neural_networks/content.html

More Related