rex replication at the speed of multi core n.
Download
Skip this Video
Loading SlideShow in 5 Seconds..
Rex: Replication at the Speed of Multi-core PowerPoint Presentation
Download Presentation
Rex: Replication at the Speed of Multi-core

Loading in 2 Seconds...

play fullscreen
1 / 35

Rex: Replication at the Speed of Multi-core - PowerPoint PPT Presentation


  • 72 Views
  • Uploaded on

Rex: Replication at the Speed of Multi-core. Zhenyu Guo, Chuntao Hong , Dong Zhou*, Mao Yang, Lidong Zhou, Li Zhuang Microsoft Research CMU*. Tension between Replication and Multi-core. Most applications are multi-threaded But, to replicate, you can only use single-thread

loader
I am the owner, or an agent authorized to act on behalf of the owner, of the copyrighted work described.
capcha
Download Presentation

PowerPoint Slideshow about 'Rex: Replication at the Speed of Multi-core' - robert-page


An Image/Link below is provided (as is) to download presentation

Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author.While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server.


- - - - - - - - - - - - - - - - - - - - - - - - - - E N D - - - - - - - - - - - - - - - - - - - - - - - - - -
Presentation Transcript
rex replication at the speed of multi core

Rex: Replication at the Speed of Multi-core

Zhenyu Guo, Chuntao Hong, Dong Zhou*, Mao Yang,

Lidong Zhou, Li Zhuang

Microsoft Research CMU*

tension between replication and multi core
Tension between Replication and Multi-core
  • Most applications are multi-threaded
  • But, to replicate, you can only use single-thread

Sacrifices performance for replication

Database

Key-value Stores

Replication

Multi-core

Lock Server

File Server

outline
Outline
  • Motivation
  • System Overview
  • Implementation
  • Evaluation
state machine replication
State Machine Replication
  • To replicate a service:
    • Model as deterministic state machine
    • Order requests with consensus protocol
    • Execute with single-thread

Server

Server

requests

Server

Server

Server

Server

Server

Sequential Execution Consistent States

Consensus

Server

Server

Server

Server

Inconsistent States

Parallel Execution

Server

Server

Server

Server

Multi-core

why multi thread breaks state machine replication
Why Multi-thread Breaks State Machine Replication
  • Non-deterministic decisions: locking order, etc…
  • Replicas make decisions independently

Performance

Consistency

Server 1

Server 2

rex execute agree follow
Rex: Execute-Agree-Follow

Secondary

Primary

Traces

Traces

Traces

Secondary

Consensus

Execute

Agree

Follow

programming with rex
Programming With Rex
  • Model app as RexRSM
  • Use Rex to make non-deterministic decisions
      • RexLocks, RexCond, …
      • RexTimeStamp, RexRand, etc.
outline1
Outline
  • Motivation
  • System Overview
  • Implementation
  • Evaluation
normal execution primary
Normal Execution: Primary

request 1

1

Trace:

(t1, 1, request 1)

Causal edge((t1, 3)->(t2, 2))

(t1, 4, reply 1)

...

lockA

request 2

2

1

unlockA

3

lockA

2

reply 1

unlockA

4

3

reply 2

4

Primary

normal execution secondary
Normal Execution: Secondary

request 2

1

request 1

(t1, 1, request 1)

Causal edge((t1, 3)->(t2, 2))

(t1, 4, reply 1)

...

1

lockA

2

lockA

2

unlockA

3

unlockA

waited event

3

reply 1

4

reply 2

4

Secondary

primary failover
Primary Failover
  • Primary
    • restart from checkpoint
    • rejoin
  • Secondary
    • upgrade to primary
    • switch replay -> record

Committed

Uncommitted

Crash

unique challenges integrating replication and record replay
Unique Challenges: Integrating Replication and Record/Replay
  • Inconsistency cut
  • “Holes” in logs
  • Causal edge pruning
  • Hybrid execution
the inconsistent cut problem
The Inconsistent Cut Problem
  • Collects logs at each thread asynchronously
  • Inconsistent cut contains destination nodes without source node
  • Problem: not be able to follow
solving inconsistent cut problem
Solving Inconsistent Cut Problem
  • Define consensus on last consistent cut
  • Drop C1-C2 when primary fail
  • Reply only when reply contained in a committed consistent cut

Use vector clock to track

outline2
Outline
  • Motivation
  • System Overview
  • Implementation
  • Evaluation
experiment setup
Experiment Setup
  • Real-world Applications
  • Micro-benchmark: for lock contention ratio
  • Servers: 12-core, 24-thread, 10GE network
performance overview
Performance Overview
  • Rex scales as nonreplicated
  • <24% overhead
leveldb in detail
LevelDB in Detail

overhead drops with more threads to schedule

Waited events grows with # threads, so does overhead

# cores

lock conflict ratio
Lock Conflict Ratio

Overhead < 15%

summary
Summary
  • Rex: execute-agree-follow
  • Applied to six real-world applications
  • Preserves scalability and low overhead
dealing with data races
Dealing with Data Races
  • Reply logging & compare
  • Resource version checking
  • Lock-free data structures: NATIVE_EXEC
  • Experience shows that getting rid of data races is doable
workloads
Workloads
  • Thumbnail:
    • 1 pic per request
  • K-V stores:
    • 1M pairs
    • 16 byte key, 100 byte value
    • 10% write
  • File system:
    • 16KB random requests
    • 20% write
  • Xlock:
    • 90% lease renew
    • 100B – 5KB file
request granularity
Request Granularity

10% computation in locks

1% conflict ratio

improving performance causal edge pruning with vector clock
Improving Performance: Causal Edge Pruning with Vector Clock
  • More causal edges, more overhead
  • Causal edge pruning: trades primary performance for secondary

Reduces 58% ~ 99% causal edges

correctness
Correctness
  • Correctness guaranteed by:
    • Captures all non-determinism with Rex
    • Consensus on traces
    • Agreed trace is a continuous sequence (no holes)
inconsistent cut why i s it bad
Inconsistent Cut: Why Is It Bad?

Trace: t1 unlock -> t2 lock -> t2 unlock -> t3 lock reply: 0

Replay: t1 unlock -> t3 lock -> t3 unlock -> t2 lock reply: 1

Should we reply 0 or 1?

inconsistent cut solving the reply problem
Inconsistent Cut: Solving the Reply Problem
  • Reply only when reply and all its dependencies are committed
  • Use a vector clock to detect