A machine learning perspective on neural networks and learning tools
This presentation is the property of its rightful owner.
Sponsored Links
1 / 21

A machine learning perspective on neural networks and learning tools PowerPoint PPT Presentation


  • 166 Views
  • Uploaded on
  • Presentation posted in: General

A machine learning perspective on neural networks and learning tools. Tom Schaul. Overview. PyBrain: training artificial neural networks for classification, (sequence) prediction and control Neural networks Modular structure Available architectures Training Supervised learning

Download Presentation

A machine learning perspective on neural networks and learning tools

An Image/Link below is provided (as is) to download presentation

Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author.While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server.


- - - - - - - - - - - - - - - - - - - - - - - - - - E N D - - - - - - - - - - - - - - - - - - - - - - - - - -

Presentation Transcript


A machine learning perspective on neural networks and learning tools

A machine learning perspective on neural networks and learning tools

Tom Schaul


Overview

Overview

PyBrain: training artificial neural networks for classification, (sequence) prediction and control

  • Neural networks

    • Modular structure

    • Available architectures

  • Training

    • Supervised learning

    • Optimization

    • Reinforcement learning (RL)

4th FACETS CodeJam Workshop - Tom Schaul - PyBrain


Disclaimer

Disclaimer

  • Only version 0.3, you may encounter

    • inconsistencies

    • bugs

    • undocumented “features”

  • But growing

    • 10+ contributors

    • 100+ followers (github, mailing list)

    • 1000+ downloads

4th FACETS CodeJam Workshop - Tom Schaul - PyBrain


Our neural networks

(Our) Neural Networks

No spikes

Continuous activations

Discrete time steps

4th FACETS CodeJam Workshop - Tom Schaul - PyBrain


Network structure modules

Parameters

Derivatives

Network Structure: Modules

output error

output

derivatives

Module

parameters

input

input error

4th FACETS CodeJam Workshop - Tom Schaul - PyBrain


Network structure connections

Network Structure: Connections

output

output

output

output error

output error

output error

Module

Module

Module

FullConnection

input

input

input

input error

input error

input error

4th FACETS CodeJam Workshop - Tom Schaul - PyBrain


Network structure graphs recurrency nesting

Module

Module

Module

Module

Module

Module

Network Structure:Graphs, Recurrency, Nesting

4th FACETS CodeJam Workshop - Tom Schaul - PyBrain


Network components modules

Network Components: Modules

  • Module types

    • layers of neurons

      • additive or multiplicative

      • sigmoidal squashing functions

      • stochastic outputs

    • gate units

    • memory cells (e.g. LSTM cells)

4th FACETS CodeJam Workshop - Tom Schaul - PyBrain


Network components connections

Network Components: Connections

  • Connection

    • Fully connected or sparse

    • Time-recurrent

    • Weight-sharing

    • may contain parameters

4th FACETS CodeJam Workshop - Tom Schaul - PyBrain


Network architectures

Network Architectures

  • Feed-forward networks, including

    • Deep Belief Nets

    • Restricted Boltzmann Machines (RBM)

  • Recurrent networks, including

    • Reservoirs (Echo State networks)

    • Bidirectional networks

    • Long Short-Term Memory (LSTM) architectures

    • Multi-Dimensional Recurrent Networks (MDRNN)

  • Custom-designed topologies

4th FACETS CodeJam Workshop - Tom Schaul - PyBrain


Overview1

Overview

  • Neural networks

    • Modular structure

    • Available architectures

  • Training

    • Supervised learning

    • Optimization

    • Reinforcement learning (RL)

4th FACETS CodeJam Workshop - Tom Schaul - PyBrain


Training supervised learning

Parameters

Derivatives

Training: Supervised Learning

compare to target

output error

output

derivatives

Backpropagation

gradientupdate onparameters

Module

parameters

input

input error

4th FACETS CodeJam Workshop - Tom Schaul - PyBrain


Training black box optimization

Parameters

Training: Black-box Optimization

BlackBoxOptimizer

update parameters

fitness

parameters

Black box

fitness function based on e.g. MSE, accuracy, rewards

multiple fitness values: multi-objective optimization

4th FACETS CodeJam Workshop - Tom Schaul - PyBrain


Optimization algorithms

Optimization Algorithms

(Stochastic) Hill-climbing

Particle Swarm Optimization (PSO)

(Natural) Evolution Strategies (ES)

Covariance Matrix Adaptation (CMA)

Genetic Algorithms (GA)

Co-evolution

Multi-Objective Optimization (NSGA-II)

4th FACETS CodeJam Workshop - Tom Schaul - PyBrain


Training reinforcement learning

Training: Reinforcement Learning

Environment

Environment

state

action

Task

Experiment

reward

observation

action

Agent

4th FACETS CodeJam Workshop - Tom Schaul - PyBrain


Rl agents learners exploration

DataSet

Module

Learner

Explorer

reward

observation

action

RL: Agents, Learners, Exploration

LearningAgent

4th FACETS CodeJam Workshop - Tom Schaul - PyBrain


Rl learning algorithms and exploration

RL: Learning Algorithms and Exploration

  • Value-based RL

    • Q-Learning, SARSA

    • Fitted-Q Iteration

  • Policy Gradient RL

    • REINFORCE

    • Natural Actor-Critic

  • Exploration methods

    • Epsilon-Greedy

    • Boltzmann

    • State-Dependent Exploration

4th FACETS CodeJam Workshop - Tom Schaul - PyBrain


Rl environments and tasks

RL: Environments and Tasks

2D Mazes (MDP / POMDP)

Pole balancing

3D environments (ODE, FlexCube)

Board games (e.g. Atari-Go, Pente)

4th FACETS CodeJam Workshop - Tom Schaul - PyBrain


Also in pybrain

Also in PyBrain

  • Unsupervised learning and preprocessing

  • Support Vector Machines (through LIBSVM)

  • Tools

    • Plotting / Visualization

    • netCDF support

    • XML read/write support

  • arac: fast C version

4th FACETS CodeJam Workshop - Tom Schaul - PyBrain


References

References

  • Source download, documentation

    www.pybrain.org

  • Mailing list (200+ members)

    groups.google.com/group/pybrain

  • Feature requests

    github.com/pybrain/pybrain/issues

  • CitationT. Schaul, J. Bayer, D. Wierstra, Y. Sun, M. Felder, F. Sehnke, T. Rückstieß and J. Schmidhuber. PyBrain. Journal of Machine Learning Research, 2010.

4th FACETS CodeJam Workshop - Tom Schaul - PyBrain


Acknowledgements

Acknowledgements

Justin Bayer

Martin Felder

Thomas Rückstiess

Frank Sehnke

DaanWierstra

and many more who contributed code, suggestions, bug fixes …

… and you for your attention!

4th FACETS CodeJam Workshop - Tom Schaul - PyBrain


  • Login