1 / 21

A machine learning perspective on neural networks and learning tools

A machine learning perspective on neural networks and learning tools. Tom Schaul. Overview. PyBrain: training artificial neural networks for classification, (sequence) prediction and control Neural networks Modular structure Available architectures Training Supervised learning

nita
Download Presentation

A machine learning perspective on neural networks and learning tools

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. A machine learning perspective on neural networks and learning tools Tom Schaul

  2. Overview PyBrain: training artificial neural networks for classification, (sequence) prediction and control • Neural networks • Modular structure • Available architectures • Training • Supervised learning • Optimization • Reinforcement learning (RL) 4th FACETS CodeJam Workshop - Tom Schaul - PyBrain

  3. Disclaimer • Only version 0.3, you may encounter • inconsistencies • bugs • undocumented “features” • But growing • 10+ contributors • 100+ followers (github, mailing list) • 1000+ downloads 4th FACETS CodeJam Workshop - Tom Schaul - PyBrain

  4. (Our) Neural Networks No spikes Continuous activations Discrete time steps 4th FACETS CodeJam Workshop - Tom Schaul - PyBrain

  5. Parameters Derivatives Network Structure: Modules output error output derivatives Module parameters input input error 4th FACETS CodeJam Workshop - Tom Schaul - PyBrain

  6. Network Structure: Connections output output output output error output error output error Module Module Module FullConnection input input input input error input error input error 4th FACETS CodeJam Workshop - Tom Schaul - PyBrain

  7. Module Module Module Module Module Module Network Structure:Graphs, Recurrency, Nesting 4th FACETS CodeJam Workshop - Tom Schaul - PyBrain

  8. Network Components: Modules • Module types • layers of neurons • additive or multiplicative • sigmoidal squashing functions • stochastic outputs • gate units • memory cells (e.g. LSTM cells) • … 4th FACETS CodeJam Workshop - Tom Schaul - PyBrain

  9. Network Components: Connections • Connection • Fully connected or sparse • Time-recurrent • Weight-sharing • may contain parameters • … 4th FACETS CodeJam Workshop - Tom Schaul - PyBrain

  10. Network Architectures • Feed-forward networks, including • Deep Belief Nets • Restricted Boltzmann Machines (RBM) • Recurrent networks, including • Reservoirs (Echo State networks) • Bidirectional networks • Long Short-Term Memory (LSTM) architectures • Multi-Dimensional Recurrent Networks (MDRNN) • Custom-designed topologies 4th FACETS CodeJam Workshop - Tom Schaul - PyBrain

  11. Overview • Neural networks • Modular structure • Available architectures • Training • Supervised learning • Optimization • Reinforcement learning (RL) 4th FACETS CodeJam Workshop - Tom Schaul - PyBrain

  12. Parameters Derivatives Training: Supervised Learning compare to target output error output derivatives Backpropagation gradientupdate onparameters Module parameters input input error 4th FACETS CodeJam Workshop - Tom Schaul - PyBrain

  13. Parameters Training: Black-box Optimization BlackBoxOptimizer update parameters fitness parameters Black box fitness function based on e.g. MSE, accuracy, rewards multiple fitness values: multi-objective optimization 4th FACETS CodeJam Workshop - Tom Schaul - PyBrain

  14. Optimization Algorithms (Stochastic) Hill-climbing Particle Swarm Optimization (PSO) (Natural) Evolution Strategies (ES) Covariance Matrix Adaptation (CMA) Genetic Algorithms (GA) Co-evolution Multi-Objective Optimization (NSGA-II) … 4th FACETS CodeJam Workshop - Tom Schaul - PyBrain

  15. Training: Reinforcement Learning Environment Environment state action Task Experiment reward observation action Agent 4th FACETS CodeJam Workshop - Tom Schaul - PyBrain

  16. DataSet Module Learner Explorer reward observation action RL: Agents, Learners, Exploration LearningAgent 4th FACETS CodeJam Workshop - Tom Schaul - PyBrain

  17. RL: Learning Algorithms and Exploration • Value-based RL • Q-Learning, SARSA • Fitted-Q Iteration • Policy Gradient RL • REINFORCE • Natural Actor-Critic • Exploration methods • Epsilon-Greedy • Boltzmann • State-Dependent Exploration 4th FACETS CodeJam Workshop - Tom Schaul - PyBrain

  18. RL: Environments and Tasks 2D Mazes (MDP / POMDP) Pole balancing 3D environments (ODE, FlexCube) Board games (e.g. Atari-Go, Pente) 4th FACETS CodeJam Workshop - Tom Schaul - PyBrain

  19. Also in PyBrain • Unsupervised learning and preprocessing • Support Vector Machines (through LIBSVM) • Tools • Plotting / Visualization • netCDF support • XML read/write support • arac: fast C version 4th FACETS CodeJam Workshop - Tom Schaul - PyBrain

  20. References • Source download, documentation www.pybrain.org • Mailing list (200+ members) groups.google.com/group/pybrain • Feature requests github.com/pybrain/pybrain/issues • CitationT. Schaul, J. Bayer, D. Wierstra, Y. Sun, M. Felder, F. Sehnke, T. Rückstieß and J. Schmidhuber. PyBrain. Journal of Machine Learning Research, 2010. 4th FACETS CodeJam Workshop - Tom Schaul - PyBrain

  21. Acknowledgements Justin Bayer Martin Felder Thomas Rückstiess Frank Sehnke DaanWierstra and many more who contributed code, suggestions, bug fixes … … and you for your attention! 4th FACETS CodeJam Workshop - Tom Schaul - PyBrain

More Related