A machine learning perspective on neural networks and learning tools
Download
1 / 21

A machine learning perspective on neural networks and learning tools - PowerPoint PPT Presentation


  • 259 Views
  • Uploaded on

A machine learning perspective on neural networks and learning tools. Tom Schaul. Overview. PyBrain: training artificial neural networks for classification, (sequence) prediction and control Neural networks Modular structure Available architectures Training Supervised learning

loader
I am the owner, or an agent authorized to act on behalf of the owner, of the copyrighted work described.
capcha
Download Presentation

PowerPoint Slideshow about ' A machine learning perspective on neural networks and learning tools' - nita


An Image/Link below is provided (as is) to download presentation

Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author.While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server.


- - - - - - - - - - - - - - - - - - - - - - - - - - E N D - - - - - - - - - - - - - - - - - - - - - - - - - -
Presentation Transcript

Overview
Overview learning tools

PyBrain: training artificial neural networks for classification, (sequence) prediction and control

  • Neural networks

    • Modular structure

    • Available architectures

  • Training

    • Supervised learning

    • Optimization

    • Reinforcement learning (RL)

4th FACETS CodeJam Workshop - Tom Schaul - PyBrain


Disclaimer
Disclaimer learning tools

  • Only version 0.3, you may encounter

    • inconsistencies

    • bugs

    • undocumented “features”

  • But growing

    • 10+ contributors

    • 100+ followers (github, mailing list)

    • 1000+ downloads

4th FACETS CodeJam Workshop - Tom Schaul - PyBrain


Our neural networks
(Our) Neural Networks learning tools

No spikes

Continuous activations

Discrete time steps

4th FACETS CodeJam Workshop - Tom Schaul - PyBrain


Network structure modules

Parameters learning tools

Derivatives

Network Structure: Modules

output error

output

derivatives

Module

parameters

input

input error

4th FACETS CodeJam Workshop - Tom Schaul - PyBrain


Network structure connections
Network Structure: Connections learning tools

output

output

output

output error

output error

output error

Module

Module

Module

FullConnection

input

input

input

input error

input error

input error

4th FACETS CodeJam Workshop - Tom Schaul - PyBrain


Network structure graphs recurrency nesting

Module learning tools

Module

Module

Module

Module

Module

Network Structure:Graphs, Recurrency, Nesting

4th FACETS CodeJam Workshop - Tom Schaul - PyBrain


Network components modules
Network Components: Modules learning tools

  • Module types

    • layers of neurons

      • additive or multiplicative

      • sigmoidal squashing functions

      • stochastic outputs

    • gate units

    • memory cells (e.g. LSTM cells)

4th FACETS CodeJam Workshop - Tom Schaul - PyBrain


Network components connections
Network Components: Connections learning tools

  • Connection

    • Fully connected or sparse

    • Time-recurrent

    • Weight-sharing

    • may contain parameters

4th FACETS CodeJam Workshop - Tom Schaul - PyBrain


Network architectures
Network Architectures learning tools

  • Feed-forward networks, including

    • Deep Belief Nets

    • Restricted Boltzmann Machines (RBM)

  • Recurrent networks, including

    • Reservoirs (Echo State networks)

    • Bidirectional networks

    • Long Short-Term Memory (LSTM) architectures

    • Multi-Dimensional Recurrent Networks (MDRNN)

  • Custom-designed topologies

4th FACETS CodeJam Workshop - Tom Schaul - PyBrain


Overview1
Overview learning tools

  • Neural networks

    • Modular structure

    • Available architectures

  • Training

    • Supervised learning

    • Optimization

    • Reinforcement learning (RL)

4th FACETS CodeJam Workshop - Tom Schaul - PyBrain


Training supervised learning

Parameters learning tools

Derivatives

Training: Supervised Learning

compare to target

output error

output

derivatives

Backpropagation

gradientupdate onparameters

Module

parameters

input

input error

4th FACETS CodeJam Workshop - Tom Schaul - PyBrain


Training black box optimization

Parameters learning tools

Training: Black-box Optimization

BlackBoxOptimizer

update parameters

fitness

parameters

Black box

fitness function based on e.g. MSE, accuracy, rewards

multiple fitness values: multi-objective optimization

4th FACETS CodeJam Workshop - Tom Schaul - PyBrain


Optimization algorithms
Optimization Algorithms learning tools

(Stochastic) Hill-climbing

Particle Swarm Optimization (PSO)

(Natural) Evolution Strategies (ES)

Covariance Matrix Adaptation (CMA)

Genetic Algorithms (GA)

Co-evolution

Multi-Objective Optimization (NSGA-II)

4th FACETS CodeJam Workshop - Tom Schaul - PyBrain


Training reinforcement learning
Training: Reinforcement Learning learning tools

Environment

Environment

state

action

Task

Experiment

reward

observation

action

Agent

4th FACETS CodeJam Workshop - Tom Schaul - PyBrain


Rl agents learners exploration

DataSet learning tools

Module

Learner

Explorer

reward

observation

action

RL: Agents, Learners, Exploration

LearningAgent

4th FACETS CodeJam Workshop - Tom Schaul - PyBrain


Rl learning algorithms and exploration
RL: Learning Algorithms learning toolsand Exploration

  • Value-based RL

    • Q-Learning, SARSA

    • Fitted-Q Iteration

  • Policy Gradient RL

    • REINFORCE

    • Natural Actor-Critic

  • Exploration methods

    • Epsilon-Greedy

    • Boltzmann

    • State-Dependent Exploration

4th FACETS CodeJam Workshop - Tom Schaul - PyBrain


Rl environments and tasks
RL: Environments and Tasks learning tools

2D Mazes (MDP / POMDP)

Pole balancing

3D environments (ODE, FlexCube)

Board games (e.g. Atari-Go, Pente)

4th FACETS CodeJam Workshop - Tom Schaul - PyBrain


Also in pybrain
Also in PyBrain learning tools

  • Unsupervised learning and preprocessing

  • Support Vector Machines (through LIBSVM)

  • Tools

    • Plotting / Visualization

    • netCDF support

    • XML read/write support

  • arac: fast C version

4th FACETS CodeJam Workshop - Tom Schaul - PyBrain


References
References learning tools

  • Source download, documentation

    www.pybrain.org

  • Mailing list (200+ members)

    groups.google.com/group/pybrain

  • Feature requests

    github.com/pybrain/pybrain/issues

  • CitationT. Schaul, J. Bayer, D. Wierstra, Y. Sun, M. Felder, F. Sehnke, T. Rückstieß and J. Schmidhuber. PyBrain. Journal of Machine Learning Research, 2010.

4th FACETS CodeJam Workshop - Tom Schaul - PyBrain


Acknowledgements
Acknowledgements learning tools

Justin Bayer

Martin Felder

Thomas Rückstiess

Frank Sehnke

DaanWierstra

and many more who contributed code, suggestions, bug fixes …

… and you for your attention!

4th FACETS CodeJam Workshop - Tom Schaul - PyBrain


ad