A machine learning perspective on neural networks and learning tools

Download Presentation

A machine learning perspective on neural networks and learning tools

Loading in 2 Seconds...

- 200 Views
- Uploaded on
- Presentation posted in: General

A machine learning perspective on neural networks and learning tools

Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author.While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server.

- - - - - - - - - - - - - - - - - - - - - - - - - - E N D - - - - - - - - - - - - - - - - - - - - - - - - - -

A machine learning perspective on neural networks and learning tools

Tom Schaul

PyBrain: training artificial neural networks for classification, (sequence) prediction and control

- Neural networks
- Modular structure
- Available architectures

- Training
- Supervised learning
- Optimization
- Reinforcement learning (RL)

4th FACETS CodeJam Workshop - Tom Schaul - PyBrain

- Only version 0.3, you may encounter
- inconsistencies
- bugs
- undocumented “features”

- But growing
- 10+ contributors
- 100+ followers (github, mailing list)
- 1000+ downloads

4th FACETS CodeJam Workshop - Tom Schaul - PyBrain

No spikes

Continuous activations

Discrete time steps

4th FACETS CodeJam Workshop - Tom Schaul - PyBrain

Parameters

Derivatives

output error

output

derivatives

Module

parameters

input

input error

4th FACETS CodeJam Workshop - Tom Schaul - PyBrain

output

output

output

output error

output error

output error

Module

Module

Module

FullConnection

input

input

input

input error

input error

input error

4th FACETS CodeJam Workshop - Tom Schaul - PyBrain

Module

Module

Module

Module

Module

Module

4th FACETS CodeJam Workshop - Tom Schaul - PyBrain

- Module types
- layers of neurons
- additive or multiplicative
- sigmoidal squashing functions
- stochastic outputs

- gate units
- memory cells (e.g. LSTM cells)
- …

- layers of neurons

4th FACETS CodeJam Workshop - Tom Schaul - PyBrain

- Connection
- Fully connected or sparse
- Time-recurrent
- Weight-sharing
- may contain parameters
- …

4th FACETS CodeJam Workshop - Tom Schaul - PyBrain

- Feed-forward networks, including
- Deep Belief Nets
- Restricted Boltzmann Machines (RBM)

- Recurrent networks, including
- Reservoirs (Echo State networks)
- Bidirectional networks
- Long Short-Term Memory (LSTM) architectures
- Multi-Dimensional Recurrent Networks (MDRNN)

- Custom-designed topologies

4th FACETS CodeJam Workshop - Tom Schaul - PyBrain

- Neural networks
- Modular structure
- Available architectures

- Training
- Supervised learning
- Optimization
- Reinforcement learning (RL)

4th FACETS CodeJam Workshop - Tom Schaul - PyBrain

Parameters

Derivatives

compare to target

output error

output

derivatives

Backpropagation

gradientupdate onparameters

Module

parameters

input

input error

4th FACETS CodeJam Workshop - Tom Schaul - PyBrain

Parameters

BlackBoxOptimizer

update parameters

fitness

parameters

Black box

fitness function based on e.g. MSE, accuracy, rewards

multiple fitness values: multi-objective optimization

4th FACETS CodeJam Workshop - Tom Schaul - PyBrain

(Stochastic) Hill-climbing

Particle Swarm Optimization (PSO)

(Natural) Evolution Strategies (ES)

Covariance Matrix Adaptation (CMA)

Genetic Algorithms (GA)

Co-evolution

Multi-Objective Optimization (NSGA-II)

…

4th FACETS CodeJam Workshop - Tom Schaul - PyBrain

Environment

Environment

state

action

Task

Experiment

reward

observation

action

Agent

4th FACETS CodeJam Workshop - Tom Schaul - PyBrain

DataSet

Module

Learner

Explorer

reward

observation

action

LearningAgent

4th FACETS CodeJam Workshop - Tom Schaul - PyBrain

- Value-based RL
- Q-Learning, SARSA
- Fitted-Q Iteration

- Policy Gradient RL
- REINFORCE
- Natural Actor-Critic

- Exploration methods
- Epsilon-Greedy
- Boltzmann
- State-Dependent Exploration

4th FACETS CodeJam Workshop - Tom Schaul - PyBrain

2D Mazes (MDP / POMDP)

Pole balancing

3D environments (ODE, FlexCube)

Board games (e.g. Atari-Go, Pente)

4th FACETS CodeJam Workshop - Tom Schaul - PyBrain

- Unsupervised learning and preprocessing
- Support Vector Machines (through LIBSVM)
- Tools
- Plotting / Visualization
- netCDF support
- XML read/write support

- arac: fast C version

4th FACETS CodeJam Workshop - Tom Schaul - PyBrain

- Source download, documentation
www.pybrain.org

- Mailing list (200+ members)
groups.google.com/group/pybrain

- Feature requests
github.com/pybrain/pybrain/issues

- CitationT. Schaul, J. Bayer, D. Wierstra, Y. Sun, M. Felder, F. Sehnke, T. Rückstieß and J. Schmidhuber. PyBrain. Journal of Machine Learning Research, 2010.

4th FACETS CodeJam Workshop - Tom Schaul - PyBrain

Justin Bayer

Martin Felder

Thomas Rückstiess

Frank Sehnke

DaanWierstra

and many more who contributed code, suggestions, bug fixes …

… and you for your attention!

4th FACETS CodeJam Workshop - Tom Schaul - PyBrain