A machine learning perspective on neural networks and learning tools
Download
1 / 21

A machine learning perspective on neural networks and learning tools - PowerPoint PPT Presentation


  • 265 Views
  • Uploaded on

A machine learning perspective on neural networks and learning tools. Tom Schaul. Overview. PyBrain: training artificial neural networks for classification, (sequence) prediction and control Neural networks Modular structure Available architectures Training Supervised learning

loader
I am the owner, or an agent authorized to act on behalf of the owner, of the copyrighted work described.
capcha
Download Presentation

PowerPoint Slideshow about 'A machine learning perspective on neural networks and learning tools' - nita


An Image/Link below is provided (as is) to download presentation

Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author.While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server.


- - - - - - - - - - - - - - - - - - - - - - - - - - E N D - - - - - - - - - - - - - - - - - - - - - - - - - -
Presentation Transcript

Overview
Overview learning tools

PyBrain: training artificial neural networks for classification, (sequence) prediction and control

  • Neural networks

    • Modular structure

    • Available architectures

  • Training

    • Supervised learning

    • Optimization

    • Reinforcement learning (RL)

4th FACETS CodeJam Workshop - Tom Schaul - PyBrain


Disclaimer
Disclaimer learning tools

  • Only version 0.3, you may encounter

    • inconsistencies

    • bugs

    • undocumented “features”

  • But growing

    • 10+ contributors

    • 100+ followers (github, mailing list)

    • 1000+ downloads

4th FACETS CodeJam Workshop - Tom Schaul - PyBrain


Our neural networks
(Our) Neural Networks learning tools

No spikes

Continuous activations

Discrete time steps

4th FACETS CodeJam Workshop - Tom Schaul - PyBrain


Network structure modules

Parameters learning tools

Derivatives

Network Structure: Modules

output error

output

derivatives

Module

parameters

input

input error

4th FACETS CodeJam Workshop - Tom Schaul - PyBrain


Network structure connections
Network Structure: Connections learning tools

output

output

output

output error

output error

output error

Module

Module

Module

FullConnection

input

input

input

input error

input error

input error

4th FACETS CodeJam Workshop - Tom Schaul - PyBrain


Network structure graphs recurrency nesting

Module learning tools

Module

Module

Module

Module

Module

Network Structure:Graphs, Recurrency, Nesting

4th FACETS CodeJam Workshop - Tom Schaul - PyBrain


Network components modules
Network Components: Modules learning tools

  • Module types

    • layers of neurons

      • additive or multiplicative

      • sigmoidal squashing functions

      • stochastic outputs

    • gate units

    • memory cells (e.g. LSTM cells)

4th FACETS CodeJam Workshop - Tom Schaul - PyBrain


Network components connections
Network Components: Connections learning tools

  • Connection

    • Fully connected or sparse

    • Time-recurrent

    • Weight-sharing

    • may contain parameters

4th FACETS CodeJam Workshop - Tom Schaul - PyBrain


Network architectures
Network Architectures learning tools

  • Feed-forward networks, including

    • Deep Belief Nets

    • Restricted Boltzmann Machines (RBM)

  • Recurrent networks, including

    • Reservoirs (Echo State networks)

    • Bidirectional networks

    • Long Short-Term Memory (LSTM) architectures

    • Multi-Dimensional Recurrent Networks (MDRNN)

  • Custom-designed topologies

4th FACETS CodeJam Workshop - Tom Schaul - PyBrain


Overview1
Overview learning tools

  • Neural networks

    • Modular structure

    • Available architectures

  • Training

    • Supervised learning

    • Optimization

    • Reinforcement learning (RL)

4th FACETS CodeJam Workshop - Tom Schaul - PyBrain


Training supervised learning

Parameters learning tools

Derivatives

Training: Supervised Learning

compare to target

output error

output

derivatives

Backpropagation

gradientupdate onparameters

Module

parameters

input

input error

4th FACETS CodeJam Workshop - Tom Schaul - PyBrain


Training black box optimization

Parameters learning tools

Training: Black-box Optimization

BlackBoxOptimizer

update parameters

fitness

parameters

Black box

fitness function based on e.g. MSE, accuracy, rewards

multiple fitness values: multi-objective optimization

4th FACETS CodeJam Workshop - Tom Schaul - PyBrain


Optimization algorithms
Optimization Algorithms learning tools

(Stochastic) Hill-climbing

Particle Swarm Optimization (PSO)

(Natural) Evolution Strategies (ES)

Covariance Matrix Adaptation (CMA)

Genetic Algorithms (GA)

Co-evolution

Multi-Objective Optimization (NSGA-II)

4th FACETS CodeJam Workshop - Tom Schaul - PyBrain


Training reinforcement learning
Training: Reinforcement Learning learning tools

Environment

Environment

state

action

Task

Experiment

reward

observation

action

Agent

4th FACETS CodeJam Workshop - Tom Schaul - PyBrain


Rl agents learners exploration

DataSet learning tools

Module

Learner

Explorer

reward

observation

action

RL: Agents, Learners, Exploration

LearningAgent

4th FACETS CodeJam Workshop - Tom Schaul - PyBrain


Rl learning algorithms and exploration
RL: Learning Algorithms learning toolsand Exploration

  • Value-based RL

    • Q-Learning, SARSA

    • Fitted-Q Iteration

  • Policy Gradient RL

    • REINFORCE

    • Natural Actor-Critic

  • Exploration methods

    • Epsilon-Greedy

    • Boltzmann

    • State-Dependent Exploration

4th FACETS CodeJam Workshop - Tom Schaul - PyBrain


Rl environments and tasks
RL: Environments and Tasks learning tools

2D Mazes (MDP / POMDP)

Pole balancing

3D environments (ODE, FlexCube)

Board games (e.g. Atari-Go, Pente)

4th FACETS CodeJam Workshop - Tom Schaul - PyBrain


Also in pybrain
Also in PyBrain learning tools

  • Unsupervised learning and preprocessing

  • Support Vector Machines (through LIBSVM)

  • Tools

    • Plotting / Visualization

    • netCDF support

    • XML read/write support

  • arac: fast C version

4th FACETS CodeJam Workshop - Tom Schaul - PyBrain


References
References learning tools

  • Source download, documentation

    www.pybrain.org

  • Mailing list (200+ members)

    groups.google.com/group/pybrain

  • Feature requests

    github.com/pybrain/pybrain/issues

  • CitationT. Schaul, J. Bayer, D. Wierstra, Y. Sun, M. Felder, F. Sehnke, T. Rückstieß and J. Schmidhuber. PyBrain. Journal of Machine Learning Research, 2010.

4th FACETS CodeJam Workshop - Tom Schaul - PyBrain


Acknowledgements
Acknowledgements learning tools

Justin Bayer

Martin Felder

Thomas Rückstiess

Frank Sehnke

DaanWierstra

and many more who contributed code, suggestions, bug fixes …

… and you for your attention!

4th FACETS CodeJam Workshop - Tom Schaul - PyBrain