Neuromodulation in artificial neural networks
This presentation is the property of its rightful owner.
Sponsored Links
1 / 35

Neuromodulation in Artificial Neural Networks PowerPoint PPT Presentation

  • Uploaded on
  • Presentation posted in: General

Neuromodulation in Artificial Neural Networks. Andy Philippides. Outline. Biological background GasNet background Why increased evolvability? 3 hypotheses from neuromodulation Hypothesis 1: Different temporal scales Hypothesis 2: Spatial embedding Hypothesis 3: Modulation

Download Presentation

Neuromodulation in Artificial Neural Networks

An Image/Link below is provided (as is) to download presentation

Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author.While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server.

- - - - - - - - - - - - - - - - - - - - - - - - - - E N D - - - - - - - - - - - - - - - - - - - - - - - - - -

Presentation Transcript

Neuromodulation in artificial neural networks

Neuromodulation in Artificial Neural Networks

Andy Philippides



  • Biological background

  • GasNet background

  • Why increased evolvability? 3 hypotheses from neuromodulation

    • Hypothesis 1: Different temporal scales

    • Hypothesis 2: Spatial embedding

    • Hypothesis 3: Modulation

  • Different temporal scales (esp. GasNets vs CTRNNs)

  • Coupled networks through spatial embedding

  • Conclusions and caveats

Classical neurotransmission

Point-to-point transmission at synapses

Short temporal scale

Overriding metaphor is electrical nodes connected by wires


  • Inspiration for standard connectionist ANN

We now know it s not quite that simple

We now know it’s not quite that simple

  • Single neurons are highly complex

  • Brains involve many interacting dynamical systems

  • Discovery of gaseous diffusible neurotransmitters (NO,CO,H2S – all highly toxic!)

  • Allow interactions between synaptically unconnected neurons

  • Gases are neuromodulatory:

“Any communication between neurons caused by the release of a chemical that is either not fast, or not point-to-point or not simply excitation or inhibition” (Katz, 1999)

Neuromodulation by nitric oxide no

Freely diffusing therefore can act as a volume signal

Acts over wide range of temporal scales (ms to years)

Involved in learning, vasodilation, modulation of sensory input etc

Modulatory: up-regulates/down-regulates neuron gains (sometimes both at the same neuron)

Highly reactive so cannot measure directly

To understand its function must model it: combination of mathematical modelling and more abstract GasNet model

Neuromodulation by nitric oxide (NO)

No in the locust mushroom body

NO in the locust Mushroom Body

Mushroom Body is the seat of associative learning

Made up of ~50k parallel tubular Kenyon Cells (KCs)

Concentric organisation: NO-synthesising KCs surround NO-receptive KCS


How does this morphology affect the no signals

How does this morphology affect the NO signals?

Single KCs are ineffective but NO from multiple KCs summates: serves as a low-pass filter (also seen in evolved GasNets)

Signal variability reduced by segregated organisation

Inspiration for new form of ann gasnets

Inspiration for new form of ANN: GasNets

Positive and negative electrical connections + diffusing modulatory gas

Node emits gas due to high electrical or chemical activity

2 distinct but linked signalling systems: electrical and chemical

Space and time crucial, local processes


Diffusion in gasnets 1

Diffusion in GasNets (1)



1. Gas cloud centred on emitter with a (variable) radius of action

2. Gas concentration at receiver dependent on distance to emitter i.e. Conc_Receiver = f(Distance) Conc_Emitter

Space nodes exist in is crucial to action of gas

Diffusion in gasnets 2

Diffusion in GasNets (2)

3. Gas concentrations at emitter rise and fall over time at a (1-10 times) slower rate than electrical changes

4. Gas alters slope of transfer functions of receivers



Rate of rise = 3

Ojt = tanh[kjt(ΣwijOit-1 + Ij) + bj]

Chemical and Electrical on different temporal scales

Gasnets evolvability

GasNets evolvability

  • GasNet controllers developed by evolutionary search

  • Experiments focussed on the speed of evolution of different networks which is taken as a proxy for evolvability

  • Idea is that networks that are evolvable will also be adaptable to new environments and suitable for the generation of autonomous behaviour

  • Evidence of faster (evolutionary) search for GasNet controllers on a number of robot (and other) tasks

  • New variants developed which further improve evolvability (twice and 10 times as fast)

  • Analysis of overlap of chemical/electrical connections shows looser coupling for new variants

Why is evolvability increased

Why is evolvability increased?

No explanation for GasNet evolvability in terms of fitness landscape properties (eg neutrality, epistasis etc)

Return to neuromodulation definition:

“… communication … that is either not fast, or not point-to-point or not simply excitation or inhibition”

Suggests 3 (linked) hypotheses:

1. Action of gas over different (and separate) temporal scales

2. Spatial embedding of network: serves to (flexibly) couple 2 interacting signalling systems? Other effects?

3. Modulatory effects

Gasnet vs instant gasnet

GasNet vs ‘Instant’ GasNet

Test a new variant where rise/fall of gas is on the same temporal scale as electrical connections

Instant Plexus worse (roughly twice as bad) than original on robot and CPG task (Philippides, Cherian)

Seemed to be bad at tuning behaviour to get highest fitnesses

‘Tuning time’ of 467 generations compared to 100 for standard

Ctrnn vs gasnet

CTRNN vs GasNet

If availability of different temporal scales is useful, if a network’s electrical connections can be ‘slow’, is it equivalent to a GasNet?

  • Eg in a CTRNN (leaky integrator), time-scale of electrical activation set by t

  • = 1, nodes are integrate-and-fire, as in the GasNet

  • > 1, nodes integrate activity over time, as the gas does

Work by Chris Buckley suggests the explicit separation of time-scales, as in nervous systems could be key

Sven magg gasnet vs ctrnn 2

Sven Magg: GasNet vs CTRNN (2)

  • Compared CTRNN and GasNets on 2 CPG tasks

  • On 4:1 GasNets were better and quicker than CTRNNs

  • Lots of variants of connection scheme etc

Neuromodulation in artificial neural networks

  • However on 7:5 pattern GasNets were worse

  • Why the differences for different patterns?

Similar to work of Gary McHale evolving gaits in quadrupeds and bipeds: GasNets to be as good as and more ‘general’ than CTRNNs

Different temporal dynamics

Different temporal dynamics

  • Patterns have a different balance of on-off ones and zeroes

  • CTRNNs have the same on and off dynamics

  • Combination of modulation and temporal dynamics inherent in GasNets lead to different on-off dynamics

Sven magg gasnet vs ctrnn 3

Sven Magg: GasNet vs CTRNN (3)

Next used Beer’s diamond catching, ball avoiding robot: a reactive task? If timing unimportant will GasNets be bad?

  • CTRNNs better but so are integrate and fire neurons

  • However, NoGas worse than GasNet…

  • By stripping down elements of GasNets, addition of weights helped network NOT timing elements

  • Is it simply timing that aids GasNet evolvability?

Gasnet vs ctrnn 4 mark pratley

GasNet vs CTRNN (4): Mark Pratley

  • Range of (simulated) robot walking tasks (pattern generation)

  • Used GasNet CTRNN: slope of transfer function affected by gas:

  • Non-spatial: extent of gas effect determined by weighted connections (both elec. and chem. nets fully connected)

  • CTRNN with gas outperforms standard: modulation is key?

  • But different performance of GasNet variants suggests it is not all temporal dynamics and modulation: spatial embedding?

Temporal scales summary

Temporal scales summary

  • Different temporal scales aid evolvability and temporal adaptibility

  • In both real and artificial nervous systems, slow scales can be used to filter noisy inputs

  • Can match world time-scales longer than those inherent in the network, but it’s harder to tune mechanisms

  • Explicit separation of different mechanisms’ time-scales may be useful

  • High transfer slopes can easily lead to oscillatory networks

  • Certainly differences in GasNets and CTRNNs which may be due to the differences in on-off dynamics

Evolvability and coupling

Evolvability and coupling

  • Original GasNets had tight coupling between electrical and chemical processes as ‘connections’ were specified over the same space

  • New versions allow for more flexibly coupled processes with distinct characteristics (electrical, chemical)

  • May allow non-destructive “tuning” of functionality of one system against the other (Conrad, 90)

What if we alter the spatial coupling by changing gas diffusion

What if we alter the (spatial) coupling by changing gas diffusion?

1. Gas spreads through the entire network (more destructive interference?) – [Red line]

2. Different 2d spaces for chemical and electrical (less coupling?) –[Green line]

Median run lengths over 20 runs:

Plexus = 512, Whole = 1130, Decoupled = 638)

Summary: Whole is like the original GasNet.

No significant difference between Plexus and Decoupled

Modularity peter fine

Modularity (Peter Fine)

Also, changing the spatial embedding of the network leads to differences in their modularity



Here the more connected network evolved faster

Mark pratley does coupling change through evolution

Mark Pratley: Does coupling change through evolution?

Examined the correlation of electrical and chemical weight matrices during evolution

Most runs inconclusive (though definite clusters)

However, 1 (/100) run produced the following graph of coupling against evolutionary time

An accident? Or proof that coupling has an effect?

Also, constraining the coupling to different degrees affects fitness depending on degree: unconstrained and 15% best



  • Different and explicitly separate temporal scales seems useful

  • CTRNNs vs GasNets: no clear winner – especially if weights are used but there are differences – maybe due to different off-on temporal dynamics

  • Flexible coupling of 2 systems aids evolvability

  • Some evidence of coupling changing during the course of evolution and changing the outcome of evolutionary search

  • What about modulation? Intuitively, seems powerful to have 2 different modes of connection: additive and multiplicative

  • Maybe the combination of 2 signalling channels: fast, additive electrical connections and slow, modulatory/mutliplicative gases allows independent behaviour on different time-scales and rich adaptive dynamics in general (cf Buckley and scale)

Future work

Future work

Spatially Embedded Complex Systems Engineering (SECSE) will look in more detail at issues raised (Patricia Vargas, Lionel Barnett, Xiaobing Hu, Dave Harris (SPRU))

  • Better measures of coupling

  • Measures for on-line dynamics in networks

  • Systematic investigations of impact on evolvability of different temporal scales, spatial embedding and modulation

Visually guided insect learning algorithms to develop more complex GasNets

  • GasNets and learning

  • More complex architectures (eg spiking network Dan Bush)



  • Phil Husbands

  • Michael O’Shea

  • Sven Magg

  • Peter Fine

  • Mark Pratley

  • Philip Cherian

  • Chris Buckley

  • Tom Smith

  • Complex Networks Workshop (which led to SECSE)

  • EPSRC/BBSRC funded work

Plexus gasnet

Original spatial distribution based on a single neuron

Plexus has a uniform distribution

Gas cloud centred on a genetically specified position in the network plane

Plexus GasNet

Receptor gasnet

Receptor GasNet

  • In nervous systems, only neurons with one of the receptors for NO will be affected

  • Each node may have quantities of receptors (none, medium, maximum)

  • Modulation depends on amount of receptor

Gas concentration

Receptor concentration

Stability analysis

0 < k<= 1

Stability Analysis

Transfer function is tanh(kx). For a single neuron with +ve recurrency stability of fixed points determined by |k sech2(kx)| >1

ie Stability and number of fixed points governed by |k|

Neuromodulation in artificial neural networks

k > 1

-1<= k < 0

Neuromodulation in artificial neural networks

k < -1

Neuromodulation in artificial neural networks

Leads to various ‘interesting’ dynamics for 2 neurons with different ‘phases’ of stable/semi-stable behaviour

cf Buckley: showed gas allows ‘re-setting’ of network from a stable to a unstable state

Conclusions mark pratley

Conclusions: Mark Pratley

Tanh activation function performs better than sigmoidal

Top group tanh variants, bottom group sigmoids

Adding a function modifying variable k can give improvement,

However effect is dependent on whether gas is used and which activation function

Neuromodulation in artificial neural networks

Gas increases fitness by increasing sensitivity of neurons thus increasing leg oscillation size

Type of gas needed depends on range of k’s allowed

Transplanting neurons evolved with gas into NoGas and vice versa show electrical system is robust

Electric system performs leg movement with a smooth phase-locked horizontal and vertical rhythm, while gas tunes the size of oscillations.

  • Login