adapt an event passing protocol for reducing delivery costs in scatter gather parallel processes
Download
Skip this Video
Download Presentation
ADaPT: An Event-Passing Protocol for Reducing Delivery Costs in Scatter-Gather Parallel Processes

Loading in 2 Seconds...

play fullscreen
1 / 19

ADaPT: An Event-Passing Protocol for Reducing Delivery Costs in Scatter-Gather Parallel Processes - PowerPoint PPT Presentation


  • 97 Views
  • Uploaded on

ADaPT: An Event-Passing Protocol for Reducing Delivery Costs in Scatter-Gather Parallel Processes. Outline. ADaPT: An Event-Passing Protocol For Reducing Delivery Costs in Scatter-Gather Parallel Processes.  Motivation.  Motivation.  Established Techniques.  ADaPT.

loader
I am the owner, or an agent authorized to act on behalf of the owner, of the copyrighted work described.
capcha
Download Presentation

PowerPoint Slideshow about ' ADaPT: An Event-Passing Protocol for Reducing Delivery Costs in Scatter-Gather Parallel Processes' - cooper


An Image/Link below is provided (as is) to download presentation

Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author.While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server.


- - - - - - - - - - - - - - - - - - - - - - - - - - E N D - - - - - - - - - - - - - - - - - - - - - - - - - -
Presentation Transcript
outline
Outline

ADaPT: An Event-Passing Protocol

For Reducing Delivery Costs in

Scatter-Gather Parallel Processes

 Motivation

 Motivation

 Established Techniques

 ADaPT

 Performance Comparison

 Conclusions

motivation
Motivation

What is the Laboratory for Neural Dynamics?

  • A computational-science section of the Center for Neural Engineering
  • Part of a National Science Foundation engineering research center dedicated to biomimetic microelectronic systems
  • Combines computational electrophysiology, engineering, pharmacology, and other disciplines
  • Integrates empirically-measured, realistic, and biologically-inspired synaptic models for the purposeof temporal signals processing
motivation1
Motivation

The Dynamic Synapse

  • Biologically-inspired rather than realistic
  • Computationally-complex and non-linear
  • Signals processing application was originally a proof of concept
  • Now a synergistic field for the Center

Postsynapse

Neuron

Presynapse

Na+

Figure 1: Electro-chemical synaptic transmission

Ca2+

Ca2+

Feedback

threshold

threshold

Action Potential input

Glutamate release

Synaptic Potential Summation

Action Potential output

motivation2

A

AP

AP

AP

EPSP

EPSP

EPSP

EPSP

EPSP

EPSP

EPSP

EPSP

EPSP

EPSP

EPSP

EPSP

Motivation

Dynamic Synapse Neural Networks

LAYER 1

  • Classical NN structure
  • Increased synaptic functionality
  • Parameter trainingvia genetic algorithms

Array of K input neurons

LAYER 2

3xK Pre-synaptic Matrix

3xK Post-synaptic Matrix

K

Output Neurons

3

2

1

AP

AP

A

AP

AP

Feedback Modulation

Captured Sound

K length filter bank

Microphone Array

Figure 2: 3xK 2-Layer DSNN Single Word Classifier.

outline1
Outline

ADaPT: An Event-Passing Protocol

For Reducing Delivery Costs in

Scatter-Gather Parallel Processes

 Motivation

 Motivation

Established Techniques

 ADaPT

 Performance Comparison

 Conclusions

established techniques
Established Techniques

Master

Scatter-Gather

Computation Time

  • Naïve Approach
  • Multiple SequentialScatter-Gathers
  • With uniform computation time, exhibits decent parallelism
  • With variable computation times,significant idle time

Worker n -2

Worker n -1

Worker 1

Worker 2

Worker 3

Worker 4

Worker 5

Worker 6

Worker n

Master

Worker n -2

Worker n -1

Worker 1

Worker 2

Worker 3

Worker 4

Worker 5

Worker 6

Worker n

Master

Worker n -2

Worker n -1

Worker 1

Worker 2

Worker 3

Worker 4

Worker 5

Worker 6

Worker n

Master

Figure 3: Multi-phase evaluation of 3n genomes by n workers using naïve scatter-gathering.

established techniques1
Established Techniques

A More Efficient Mapping

Master

Computation Time

  • Asynchronous scattering
  • Reduced idle time for workers
  • Closer to optimal time to solution
  • Dynamic allocation of resources
    • More difficult

Worker n -2

Worker n -1

Worker 1

Worker 2

Worker 3

Worker 4

Worker 5

Worker 6

Worker n

Master

Figure 4: Multi-phase evaluation of 3n genomes by n workers using a more efficient mapping.

outline2
Outline

ADaPT: An Event-Passing Protocol

For Reducing Delivery Costs in

Scatter-Gather Parallel Processes

 Motivation

 Motivation

 Established Techniques

 ADaPT

 Performance Comparison

 Conclusions

adapt
ADaPT

Adaptive Data-parallel Publish/Subscribe Transport Protocol

  • Publish/Subscribe
    • Worker-centric, i.e. processes subscribe to the master
    • Data is transported (published) to workers as events
    • Unsubscription is possible
  • Two-phase adaptive protocol
    • Learning phase: request-reply, monitoring of time between requests
    • Aggressive phase: events are pushed to workers at regular intervals
outline3
Outline

ADaPT: An Event-Passing Protocol

For Reducing Delivery Costs in

Scatter-Gather Parallel Processes

 Motivation

 Motivation

 Established Techniques

 ADaPT

 Performance Comparison

 Conclusions

performance comparison
Performance Comparison

Message-passing costs for MPI scatters

  • Two protocols
    • Aggressive & Conservative
  • Scatter/Gathers in most implementations use conservative protocols
  • Analysis due to Gropp, et. al.

C(MPI Scatter) = (# pop.)[3s + r(n+3e)]

Where n = event payload e = envelope

r = network bandwidth

s = latency

Equation 1: Computation time cost in of scatters

In MPI.

performance comparison1
Performance Comparison

Computational Costs for Multiple scatters in MPI

  • Our assumption is a normally distributed population of compute times
  • An ideal ordering of computations would be sortedby compute time
  • How much idle time is present?

C(Computation) = (# pop.)(avg. compute time) + (# workers)(avg. compute time)

Equation 2: Computation time of a normally-distributed population using scatters in MPI.

Figure 5: Graph of sorted compute times of anormal distribution illustrating idle time.

performance comparison2
Performance Comparison

Message-Passing costs for ADaPT

  • Three different costs of event-passing in ADaPT:
    • Subscription
    • Learning Phase
    • Aggressive Phase

C(subscription) = (# workers) x [s + re]

C(learning) = (# samples) x [2s + r(n+2e)]

C(aggressive) = (# pop - # samples) x

[s + r(n+e)]

Note: we assume control events to be

of size e

Equation 3: Event-passing costs of ADaPT.

performance comparison3
Performance Comparison

Unsubscribe Costs for ADaPT

  • An unsubscribe occurs when a worker’s event buffer is in danger of overflowing
  • With ADaPT, an overflowoccurs when a worker receivesm-1 events triggering computetimes greater than the estimatedaverage (assuming a worker buffers m events)
  • Conservatively, we have decidedthat workers should clear theirbuffers before resubscribing

- We used a Monte Carlo simulation (details in paper) to determine E,the % pop with compute times > than the estimated mean given error as a function of % pop. sampled

P(unsubscribe) =

E*Pop C m-1

Pop C m-1

C(unsubscribe) = P(unsubscribe) x

[2(s+re) + (m-1)(avg. compute+ Δ)]

Equation 4: Costs of worker unsubscription in ADaPT.

performance comparison4
Performance Comparison

Analysis

(# pop - # samples)2re + (# samples)(avg. compute time) >(# pop / m) x P(unsubscribe) x

[2re + (m-1)(avg. compute time)]

  • Which protocol is more appropriate?
  • For simplicity of comparisonwe will drop the latency termand assume the number of samples to be equal to thenumber of workers
    • i.e. each worker’s firstcomputation is monitored

Equation 5: Cost comparison of MPI vs. ADaPT.

Figure 6: Graph of inequality in Equation 5.

outline4
Outline

ADaPT: An Event-Passing Protocol

For Reducing Delivery Costs in

Scatter-Gather Parallel Processes

 Motivation

 Motivation

 Established Techniques

 ADaPT

 Performance Comparison

 Conclusions

conclusions
Conclusions

What have we shown?

  • ADaPT is useful when multiple scattering of data must occur due to natural aggregation
    • An example is the training of the DSNN using genetic algorithms
  • Worker-centric approach for reduced processor idle time
  • Unsubscription is expensive but can be avoided withgreater event-buffering capabilities
  • ADaPT exploits an event pattern which emerges fromthe application of a well-known architectural pattern
ad