Language extensions for speculative parallelism
This presentation is the property of its rightful owner.
Sponsored Links
1 / 20

Language extensions for speculative parallelism PowerPoint PPT Presentation


  • 84 Views
  • Uploaded on
  • Presentation posted in: General

Language extensions for speculative parallelism. Kapil Vaswani (Microsoft Research, India) Prakash Prabhu (Princeton University) G. Ramalingam (Microsoft Research, India). Speculation. Take a risk in anticipation of gain Widely used to optimize performance Caching and pre-fetching

Download Presentation

Language extensions for speculative parallelism

An Image/Link below is provided (as is) to download presentation

Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author.While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server.


- - - - - - - - - - - - - - - - - - - - - - - - - - E N D - - - - - - - - - - - - - - - - - - - - - - - - - -

Presentation Transcript


Language extensions for speculative parallelism

Language extensions for speculative parallelism

Kapil Vaswani (Microsoft Research, India)

Prakash Prabhu (Princeton University)

G. Ramalingam (Microsoft Research, India)


Speculation

Speculation

  • Take a risk in anticipation of gain

  • Widely used to optimize performance

    • Caching and pre-fetching

    • Branch prediction, speculative code motion in compilers

  • Speculative parallelism

    • Doing work in parallel anticipating later use

    • Software transactions and futures

  • Focus of this talk

    • Value speculation based parallelization


Value speculation

Value speculation

Producer

Consumer

value T

Consumer

Producer

Prediction function

Correct

value T

=

Consumer

Speculative

value T’


Lexical analysis

Lexical analysis

<html><p><a href=“http://research.microsoft.com”>Microsoft Research</a></p></html>

h

Accounts for a large fraction of execution time in browsers, word processing apps

<

>

t

m

l

Lexical analyzer

<html>, <p>, <a, href, =, http://research.microsoft.com, >,

Microsoft Research, </a>, </p>, </html>


Huffman decoding

Huffman decoding

1.00

0

1

010000011110010…

01000011110010…

0.60

0.40

00

01

10

11

Huffman decoder

0.30

d, 0.20

a, 0.30

010

011

e, 0.20

baaeead…

c, 0.15

b, 0.15

pos = 0;

while (pos < inputBuffer.size) {

bitsRead = decode(inputBuffer, pos);

pos = pos + bitsRead;

}


What s in common

What’s in common?

  • “Sequential” loops

    • Cross iteration dependencies

    • Not directly amenable parallelization

  • Not hard to find 

    • Game playing

    • Planning and scheduling

    • Data flow analysis

    • Dynamic programming

    • Machine learning


Speculative huffman decoding

Speculative Huffman decoding

0011000010…

0110011100…

k

ith chunk

A A E E A D

.… 0 1 0 0 0 0 1 1 1 1 0 0 1 0

0 1 0 0 0 0 1 1 1 1 0 0 1 0

B A C E A D


Speculative lexical analysis bodik et al

Speculative lexical analysis (Bodik et al)

<HTML><p>…

… </HTML>

s1

k

ith chunk

s1

sik

k

s1


Performance

Performance


Language extensions

Language extensions


Why language extensions

Why language extensions?

  • Careful speculation requires thought

    • Prediction functions domain specific

    • Not always possible to automate

  • Writing correct speculative code non-trivial

    • Spawning and scheduling threads

    • Check predictions, cancel and re-invoke consumers

    • Check conflicts between producers and consumers

    • Deal with exceptions

  • Let programmer specify speculation


Speculative parallel for

Speculative parallel for

Parallel.For (int from, int to,

void delegate(Int32) work)

Speculation.Parallel.For(int from, int to,

T delegate (Int32, T) work,

T delegate (Int32) predictor)


Speculative parallel for1

Speculative parallel for

from

from + 1

from + 2

to

Original loop

work

work

work

work

T

T

T

from

from + 1

from + 2

to

pred

pred

pred

pred

Speculative parallel for

T

T

T

T

work

work

work

work

=

T

T

T

T


Example speculative lexical analysis

Example: speculative lexical analysis

int overlap = 20;

intchunk_size = total_size / p;

Speculation.Parallel.For(0, p,

(i, state) =>

{

return LexicalAnalysis(i*chunk_size,

(i + 1) * chunk_size,

state);

},

(i) =>

{

if (i == 0)

return START_STATE;

else

return LexicalAnalysis(

(i – 1) * chunk_size – overlap,

(i – 1) * chunk_size,

START_STATE);

)

}

);

Loop iteration

Prediction function


Speculative parallel for2

Speculative parallel for

  • Thread creation and scheduling

  • Check predictions

    • If incorrect, cancels and re-run next iteration

    • One misprediction does not imply that all subsequent iterations are wrong!

  • Deal with speculative exceptions


Safety

Safety

  • Safe variant

    • Guarantees sequential semantics

    • Increased sequential cost

  • Unsafe variant

    • Responsibility of managing side-effects left to the programmer

    • A type system that guarantees safety


Speculative task

Speculative task

Stand-alone speculative construct


Example speculative dfs

Example: speculative DFS

intSpeculativeDFS (Heap h, int position, intsearchValue) {

if (h.ValueAt(position) == searchValue)

return position;

Speculate (

() =>

{

return SpeculativeDFS(h, 2 * position + 1, searchValue);

},

(found) =>

{

if (found != NOT_FOUND)

return found;

else

return SpeculativeDFS(h, 2 * position + 2, searchValue);

}

() =>

{

return NOT_FOUND;

}

);

}

Producer

Consumer

Prediction function


Summary

Summary

  • Speculation is an interesting algorithm design and programming idiom

  • A library that supports speculative constructs

  • On the lookout for more speculative patterns and idioms


Questions

Questions?


  • Login