slide1 n.
Download
Skip this Video
Loading SlideShow in 5 Seconds..
From C IEL to Firmament & DIOS a heavenly tale of not just clouds PowerPoint Presentation
Download Presentation
From C IEL to Firmament & DIOS a heavenly tale of not just clouds

Loading in 2 Seconds...

play fullscreen
1 / 45

From C IEL to Firmament & DIOS a heavenly tale of not just clouds - PowerPoint PPT Presentation


  • 58 Views
  • Uploaded on

From C IEL to Firmament & DIOS a heavenly tale of not just clouds. Joint work with:. Steven Hand Anil Madhavapeddy Chris Smowton Steven Smith Derek Murray (MSR-SVC). Disclaimer . Recap: C IEL. [NSDI 2011]. A. B. G. M. M. M. R. R. R. Dynamic task graphs. G. a. M. M. T. x.

loader
I am the owner, or an agent authorized to act on behalf of the owner, of the copyrighted work described.
capcha
Download Presentation

From C IEL to Firmament & DIOS a heavenly tale of not just clouds


An Image/Link below is provided (as is) to download presentation

Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author.While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server.


- - - - - - - - - - - - - - - - - - - - - - - - - - E N D - - - - - - - - - - - - - - - - - - - - - - - - - -
Presentation Transcript
slide1

From CIEL to Firmament & DIOSa heavenly tale of not just clouds

Joint work with:

Steven HandAnil MadhavapeddyChris SmowtonSteven Smith

Derek Murray (MSR-SVC)

slide3

Recap: CIEL

[NSDI 2011]

slide4

A

B

G

M

M

M

R

R

R

dynamic task graphs
Dynamic task graphs

G

a

M

M

T

x

b

R

R

Allow tasks to spawn more tasks

slide6

Experiment from D. Murray,

A distributed execution engine supporting

data-dependent control flow.

PhD thesis, University of Cambridge,

2011.

slide7

[interlude]

polyglot CIEL

slide8

polyglot CIEL

[unpublished]

slide9

CIEL

Saving state – options

heavyweight

lightweight

VM migration

BLCR

(processcheckpoint.)

Serializablecontinuations

Haskellmonads

hardware / OS level

application level

slide11

Binomial options pricing

Experiment from D. Murray, C. Smowton, M. Schwarzkopf, S. Smith, A. Madhavapeddy. A polyglot approach to cloud programming. Unpublished, 2011.

slide12

What‘s next?!

many-core clustersheterogeneity

slide13

timespin on unmodified CIEL

41.6x

rel. overhead

5.1x

less is better

1.3x

seconds

1.04x

number of cores

slide15

...

Host OS

...

Hardware

User code

CIEL

Programming Model

1st class exec.

2nd

Skywriting

Execution Engine

Master

W0

W1

Wn

...

...

slide16

DIOS

...

User code

CIEL

Programming Model

1st class exec.

2nd

Skywriting

Firmament: Coordination Engine

...

Hardware

slide17

Firmament

multi-scaleheterogeneity-aware

slide21

Google trace, machine specs

CPU cores (normalized)

Total RAM (normalized)

slide24

Firmament

Cluster knowledge base

  • historic task resource usage historic task performance info machine information

Efficient runtime

[Storage? Networking? Transfer management?]

slide25

Firmament

It’s real!

  • ~2k LOC, basic tests run

ToDo (aka WIP):

  • knowledge base design & impl. scheduling algorithms interface to CIEL
slide26

DIOS

...

User code

CIEL

Programming Model

1st class exec.

2nd

Skywriting

Firmament: Coordination Engine

...

Hardware

slide27

DIOS

topology-awareinterference-awarelightweight OS

slide28

Heterogeneity [again!]

Many-core => intra-machine communication = important!

slide29

Intel Core i7-2600K @ 3.40GHz (native)

48-core AMD Opteron 6168

(native)

(Xen)

Joint work with Steven Smith, Anil Madhavapeddy, and Chris Smowton; cf. “The case for reconfigurable I/O“ (RESoLVE 2012)

slide30

Intel Xeon E5620 @ 2.40GHz (native)

Different physicalcore

Hyperthread

Joint work with Steven Smith, Anil Madhavapeddy, and Chris Smowton; cf http://fable.io

slide31

Intel Core i7-2600K @ 3.40GHz (native)

Different physicalcore

Hyperthread

Joint work with Steven Smith, Anil Madhavapeddy, and Chris Smowton; cf http://fable.io

slide32

AMD Opteron 6168@ 1.9 GHz (native)

Same MCM, same socket

Different MCM,different socket,2-hop Hypertransport

Joint work with Steven Smith, Anil Madhavapeddy, and Chris Smowton; cf http://fable.io

slide33

Topology-awareness

OS responsibility? Yes.

General case = hard!

Workload-awareness helps!

slide36

Interference

#include <results>

slide37

Lightweight

Make the OS do exactly (and just) what is needed.

Dedicate resources instead of sharing them.

slide38

Resourcemultiplexing

Process mgmt

Isolation

I/O mgmt

Concurrency primitives

Multi-threading

Pre-emption

Locking

Filesystem

Shell

Standard libs

slide40

DIOS

...

Scheduling

Firmament: Coordination Engine

...

Program

slide41

DIOS

Pieces exist

  • currently combining ;-)

WIP:

  • interference experiments related work reading group starting point? (Linux or Xen?)
binomial options pricing
Binomial options pricing

800k (EC2)

800k (MC)

400k (EC2)

400k (MC)

200k (EC2)

higheris better

200k (MC)

slide45

Redis example

Numbers and experiment by Sören Bleikertz: http://openfoo.org/blog/redis-native-xen.html