1 / 13

Comp 3503 / 5013 Dynamic Neural Networks

Comp 3503 / 5013 Dynamic Neural Networks. Daniel L. Silver March, 2014. Outline. Hopfield Networks Boltzman Machines Mean Field Theory Restricted Boltzman Machines (RBM). Dynamic Neural Networks. See handout for image of spider, beer and dog

swain
Download Presentation

Comp 3503 / 5013 Dynamic Neural Networks

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. Comp 3503 / 5013Dynamic Neural Networks Daniel L. Silver March, 2014

  2. Outline • Hopfield Networks • Boltzman Machines • Mean Field Theory • Restricted BoltzmanMachines (RBM)

  3. Dynamic Neural Networks • See handout for image of spider, beer and dog • The search for a model or hypothesis can be considered the relaxation of a dynamic system into a state of equilibrium • This is the nature of most physical systems • Pool of water • Air in a room • Mathematics is that of thermal-dynamics • Quote from John Von Neumann

  4. Hopfield Networks • See hand out

  5. Hopfield Networks • Hopfield Network video intro • http://www.youtube.com/watch?v=gfPUWwBkXZY • http://faculty.etsu.edu/knisleyj/neural/ • Try these Applets: • http://lcn.epfl.ch/tutorial/english/hopfield/html/index.html • http://www.cbu.edu/~pong/ai/hopfield/hopfieldapplet.html

  6. Hopfield Networks Basics with Geoff Hinton: • Introduction to Hopfield Nets • http://www.youtube.com/watch?v=YB3-Hn-inHI • Storage capacity of Hopfield Nets • http://www.youtube.com/watch?v=O1rPQlKQBLQ

  7. Hopfield Networks Advanced concepts with Geoff Hinton: • Hopfield nets with hidden units • http://www.youtube.com/watch?v=bOpddsa4BPI • Necker Cube • http://www.cs.cf.ac.uk/Dave/JAVA/boltzman/Necker.html • Adding noise to improve search • http://www.youtube.com/watch?v=kVgT2Eaa6KA

  8. Boltzman Machine • See Handout • http://www.scholarpedia.org/article/Boltzmann_machine Basics with Geoff Hinton • Modeling binary data • http://www.youtube.com/watch?v=MKdvJst8a6k • BM Learning Algorithm • http://www.youtube.com/watch?v=QgrFsnHFeig

  9. Restricted Boltzman Machine • RBM Overview: • http://blog.echen.me/2011/07/18/introduction-to-restricted-boltzmann-machines/ • RBM Details and Code: • http://www.deeplearning.net/tutorial/rbm.html

  10. Restricted Boltzman Machine • Geoff Hinton on RBMS: • RBMs and Constrastive Divergence Algorithm • http://www.youtube.com/watch?v=fJjkHAuW0Yk • An example of RBM Learning • http://www.youtube.com/watch?v=Ivj7jymShN0 • RBMs applied to Collaborative Filtering • http://www.youtube.com/watch?v=laVC6WFIXjg

  11. Deep Learning Architectures Deep Belief RBM Networks with Geoff Hinton: • Learning layers of features by stacking RBMs • http://www.youtube.com/watch?v=VRuQf3DjmfM • Discriminative fine-tuning in DBN • http://www.youtube.com/watch?v=-I2pgcH02QM • What happens during fine-tuning? • http://www.youtube.com/watch?v=yxMeeySrfDs

  12. Deep Learning Architectures • Modeling real-value data (G.Hinton) • http://www.youtube.com/watch?v=jzMahqXfM7I • Deep Learning of Representations (Y. Bengio) • http://www.youtube.com/watch?v=4xsVFLnHC_0 • Deep Belief Convolution Network (Javascript) • Runs well under Google Chrome • https://www.jetpac.com/deepbelief

  13. Additional References • Courseracourse – Neural Networks fro Machine Learning: • https://class.coursera.org/neuralnets-2012-001/lecture • ML: Hottest Tech Trend in next 3-5 Years • http://www.youtube.com/watch?v=b4zr9Zx5WiE

More Related