Adapt an event passing protocol for reducing delivery costs in scatter gather parallel processes
This presentation is the property of its rightful owner.
Sponsored Links
1 / 19

ADaPT: An Event-Passing Protocol for Reducing Delivery Costs in Scatter-Gather Parallel Processes PowerPoint PPT Presentation


  • 67 Views
  • Uploaded on
  • Presentation posted in: General

ADaPT: An Event-Passing Protocol for Reducing Delivery Costs in Scatter-Gather Parallel Processes. Outline. ADaPT: An Event-Passing Protocol For Reducing Delivery Costs in Scatter-Gather Parallel Processes.  Motivation.  Motivation.  Established Techniques.  ADaPT.

Download Presentation

ADaPT: An Event-Passing Protocol for Reducing Delivery Costs in Scatter-Gather Parallel Processes

An Image/Link below is provided (as is) to download presentation

Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author.While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server.


- - - - - - - - - - - - - - - - - - - - - - - - - - E N D - - - - - - - - - - - - - - - - - - - - - - - - - -

Presentation Transcript


Adapt an event passing protocol for reducing delivery costs in scatter gather parallel processes

ADaPT: An Event-Passing Protocol for Reducing Delivery Costs in Scatter-Gather Parallel Processes


Outline

Outline

ADaPT: An Event-Passing Protocol

For Reducing Delivery Costs in

Scatter-Gather Parallel Processes

 Motivation

 Motivation

 Established Techniques

 ADaPT

 Performance Comparison

 Conclusions


Motivation

Motivation

What is the Laboratory for Neural Dynamics?

  • A computational-science section of the Center for Neural Engineering

  • Part of a National Science Foundation engineering research center dedicated to biomimetic microelectronic systems

  • Combines computational electrophysiology, engineering, pharmacology, and other disciplines

  • Integrates empirically-measured, realistic, and biologically-inspired synaptic models for the purposeof temporal signals processing


Motivation1

Motivation

The Dynamic Synapse

  • Biologically-inspired rather than realistic

  • Computationally-complex and non-linear

  • Signals processing application was originally a proof of concept

  • Now a synergistic field for the Center

Postsynapse

Neuron

Presynapse

Na+

Figure 1: Electro-chemical synaptic transmission

Ca2+

Ca2+

Feedback

threshold

threshold

Action Potential input

Glutamate release

Synaptic Potential Summation

Action Potential output


Motivation2

A

AP

AP

AP

EPSP

EPSP

EPSP

EPSP

EPSP

EPSP

EPSP

EPSP

EPSP

EPSP

EPSP

EPSP

Motivation

Dynamic Synapse Neural Networks

LAYER 1

  • Classical NN structure

  • Increased synaptic functionality

  • Parameter trainingvia genetic algorithms

Array of K input neurons

LAYER 2

3xK Pre-synaptic Matrix

3xK Post-synaptic Matrix

K

Output Neurons

3

2

1

AP

AP

A

AP

AP

Feedback Modulation

Captured Sound

K length filter bank

Microphone Array

Figure 2: 3xK 2-Layer DSNN Single Word Classifier.


Outline1

Outline

ADaPT: An Event-Passing Protocol

For Reducing Delivery Costs in

Scatter-Gather Parallel Processes

 Motivation

 Motivation

Established Techniques

 ADaPT

 Performance Comparison

 Conclusions


Established techniques

Established Techniques

Master

Scatter-Gather

Computation Time

  • Naïve Approach

  • Multiple SequentialScatter-Gathers

  • With uniform computation time, exhibits decent parallelism

  • With variable computation times,significant idle time

Worker n -2

Worker n -1

Worker 1

Worker 2

Worker 3

Worker 4

Worker 5

Worker 6

Worker n

Master

Worker n -2

Worker n -1

Worker 1

Worker 2

Worker 3

Worker 4

Worker 5

Worker 6

Worker n

Master

Worker n -2

Worker n -1

Worker 1

Worker 2

Worker 3

Worker 4

Worker 5

Worker 6

Worker n

Master

Figure 3: Multi-phase evaluation of 3n genomes by n workers using naïve scatter-gathering.


Established techniques1

Established Techniques

A More Efficient Mapping

Master

Computation Time

  • Asynchronous scattering

  • Reduced idle time for workers

  • Closer to optimal time to solution

  • Dynamic allocation of resources

    • More difficult

Worker n -2

Worker n -1

Worker 1

Worker 2

Worker 3

Worker 4

Worker 5

Worker 6

Worker n

Master

Figure 4: Multi-phase evaluation of 3n genomes by n workers using a more efficient mapping.


Outline2

Outline

ADaPT: An Event-Passing Protocol

For Reducing Delivery Costs in

Scatter-Gather Parallel Processes

 Motivation

 Motivation

 Established Techniques

 ADaPT

 Performance Comparison

 Conclusions


Adapt

ADaPT

Adaptive Data-parallel Publish/Subscribe Transport Protocol

  • Publish/Subscribe

    • Worker-centric, i.e. processes subscribe to the master

    • Data is transported (published) to workers as events

    • Unsubscription is possible

  • Two-phase adaptive protocol

    • Learning phase: request-reply, monitoring of time between requests

    • Aggressive phase: events are pushed to workers at regular intervals


Outline3

Outline

ADaPT: An Event-Passing Protocol

For Reducing Delivery Costs in

Scatter-Gather Parallel Processes

 Motivation

 Motivation

 Established Techniques

 ADaPT

 Performance Comparison

 Conclusions


Performance comparison

Performance Comparison

Message-passing costs for MPI scatters

  • Two protocols

    • Aggressive & Conservative

  • Scatter/Gathers in most implementations use conservative protocols

  • Analysis due to Gropp, et. al.

C(MPI Scatter) = (# pop.)[3s + r(n+3e)]

Where n = event payloade = envelope

r = network bandwidth

s = latency

Equation 1: Computation time cost in of scatters

In MPI.


Performance comparison1

Performance Comparison

Computational Costs for Multiple scatters in MPI

  • Our assumption is a normally distributed population of compute times

  • An ideal ordering of computations would be sortedby compute time

  • How much idle time is present?

C(Computation) = (# pop.)(avg. compute time) + (# workers)(avg. compute time)

Equation 2: Computation time of a normally-distributed population using scatters in MPI.

Figure 5: Graph of sorted compute times of anormal distribution illustrating idle time.


Performance comparison2

Performance Comparison

Message-Passing costs for ADaPT

  • Three different costs of event-passing in ADaPT:

    • Subscription

    • Learning Phase

    • Aggressive Phase

C(subscription) = (# workers) x [s + re]

C(learning) = (# samples) x [2s + r(n+2e)]

C(aggressive) = (# pop - # samples) x

[s + r(n+e)]

Note: we assume control events to be

of size e

Equation 3: Event-passing costs of ADaPT.


Performance comparison3

Performance Comparison

Unsubscribe Costs for ADaPT

  • An unsubscribe occurs when a worker’s event buffer is in danger of overflowing

  • With ADaPT, an overflowoccurs when a worker receivesm-1 events triggering computetimes greater than the estimatedaverage (assuming a worker buffers m events)

  • Conservatively, we have decidedthat workers should clear theirbuffers before resubscribing

- We used a Monte Carlo simulation (details in paper) to determine E,the % pop with compute times > than the estimated mean given error as a function of % pop. sampled

P(unsubscribe) =

E*Pop C m-1

Pop C m-1

C(unsubscribe) = P(unsubscribe) x

[2(s+re) + (m-1)(avg. compute+ Δ)]

Equation 4: Costs of worker unsubscription in ADaPT.


Performance comparison4

Performance Comparison

Analysis

(# pop - # samples)2re + (# samples)(avg. compute time) >(# pop / m) x P(unsubscribe) x

[2re + (m-1)(avg. compute time)]

  • Which protocol is more appropriate?

  • For simplicity of comparisonwe will drop the latency termand assume the number of samples to be equal to thenumber of workers

    • i.e. each worker’s firstcomputation is monitored

Equation 5: Cost comparison of MPI vs. ADaPT.

Figure 6: Graph of inequality in Equation 5.


Outline4

Outline

ADaPT: An Event-Passing Protocol

For Reducing Delivery Costs in

Scatter-Gather Parallel Processes

 Motivation

 Motivation

 Established Techniques

 ADaPT

 Performance Comparison

 Conclusions


Conclusions

Conclusions

What have we shown?

  • ADaPT is useful when multiple scattering of data must occur due to natural aggregation

    • An example is the training of the DSNN using genetic algorithms

  • Worker-centric approach for reduced processor idle time

  • Unsubscription is expensive but can be avoided withgreater event-buffering capabilities

  • ADaPT exploits an event pattern which emerges fromthe application of a well-known architectural pattern


Adapt an event passing protocol for reducing delivery costs in scatter gather parallel processes

Thank You


  • Login