Cs4231 parallel and distributed algorithms ay 2006 2007 semester 2
Download
1 / 27

CS4231 Parallel and Distributed Algorithms AY 2006/2007 Semester 2 - PowerPoint PPT Presentation


  • 167 Views
  • Updated On :

CS4231 Parallel and Distributed Algorithms AY 2006/2007 Semester 2. Lecture 6 Instructor: Haifeng YU. Review of Last Lecture. Chapter 9 “Global Snapshot” What is our goal? Formalizing consistent global snapshot Protocol for capturing a consistent global snapshot. Today’s Roadmap.

Related searches for CS4231 Parallel and Distributed Algorithms AY 2006/2007 Semester 2

loader
I am the owner, or an agent authorized to act on behalf of the owner, of the copyrighted work described.
capcha
Download Presentation

PowerPoint Slideshow about 'CS4231 Parallel and Distributed Algorithms AY 2006/2007 Semester 2' - santos


An Image/Link below is provided (as is) to download presentation

Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author.While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server.


- - - - - - - - - - - - - - - - - - - - - - - - - - E N D - - - - - - - - - - - - - - - - - - - - - - - - - -
Presentation Transcript
Cs4231 parallel and distributed algorithms ay 2006 2007 semester 2 l.jpg

CS4231Parallel and Distributed AlgorithmsAY 2006/2007 Semester 2

Lecture 6

Instructor: Haifeng YU


Review of last lecture l.jpg
Review of Last Lecture

  • Chapter 9 “Global Snapshot”

  • What is our goal?

  • Formalizing consistent global snapshot

  • Protocol for capturing a consistent global snapshot

CS4231 Parallel and Distributed Algorithms AY2006/2007 Semester 2


Today s roadmap l.jpg
Today’s Roadmap

  • Chapter 12 “Message Ordering”

  • FIFO ordering

    • Already discussed last lecture

  • Causal ordering and its application

  • Total ordering and its application

CS4231 Parallel and Distributed Algorithms AY2006/2007 Semester 2


A internet chat room example l.jpg
A Internet Chat Room Example

Alice (process1)

Hi Bob, I just introduced you to Cary

Introducing Bob to Cary

Bob (process2)

Hi Cary, this is Bob

Cary (process3)

???

Sorry, I don’t know any Bob.

The notion of “causal order”.

CS4231 Parallel and Distributed Algorithms AY2006/2007 Semester 2


Formalizing causal order l.jpg
Formalizing Causal Order

  • If a “send” event s1 caused a “send” event s2

    • Causal order requires the corresponding receive event r1 be before r2 (r1 and r2 on the same process)

    • But how do we know whether s1 caused s2?

  • Based on “Happened-Before” relation

    • This is the only ordering observable to external users

  • If a “send” event s1 happens before a “send” event s2

    • Then s1 may have caused s2

    • Let’s be pessimistic and be safe -- Assume s1 indeed caused s2

  • Causal order: If s1 happens before s2, and r1 and r2 are on the same process, then r1 must be before r2

CS4231 Parallel and Distributed Algorithms AY2006/2007 Semester 2


How to ensure causal ordering intuition l.jpg
How to Ensure Causal Ordering – Intuition

Message1 A to B

Message1 A to C

PS:

Message1 A to C

Alice (process1)

Bob (process2)

Message1 B to C

PS:

Message1 A to C

Message1 A to B

Cary (process3)

CS4231 Parallel and Distributed Algorithms AY2006/2007 Semester 2


How to ensure causal ordering protocol l.jpg
How to Ensure Causal Ordering – Protocol

  • Each process maintains n*n matrix M

    • M[i,j] denotes that number of messages sent from i to j, as known by the local process

  • If process i send a message to process j

    • On process i: M[i,j]++

    • Piggyback M on the message

  • Upon process j receiving a message from process i with matrix M’ piggybacked

    • If M[k,j]  M’[k,j] for all k except k = i, AND M[i,j] == M’[i,j] -1,

      then deliver message and M = pairwise-max(M,M’)

    • Else delay message

CS4231 Parallel and Distributed Algorithms AY2006/2007 Semester 2


Example run for the protocol l.jpg
Example Run for the Protocol

(0,0,1)

(0,0,0)

(0,0,0)

(0,1,1)

(0,0,0)

(0,0,0)

Alice (process1)

(0,1,1)

(0,0,1)

(0,0,0)

can deliver

Bob (process2)

Message1 B to C

(0,0,0)

(0,0,0)

(0,0,0)

(0,1,1)

(0,0,0)

(0,0,0)

can’t deliver

Cary (process3)

(0,0,0)

(0,0,0)

(0,0,0)

(0,1,1)

(0,0,1)

(0,0,0)

CS4231 Parallel and Distributed Algorithms AY2006/2007 Semester 2


Correctness proof of the protocol l.jpg
Correctness Proof of the Protocol

  • (The textbook does not have an explicit correctness proof.)

  • s1, s2, r1, r2, where s1 happened before s2

    • We want to prove that r1 is never delivered after r2

  • Consider the case where s1 and s2 are on different processes

  • W.l.o.g., assume s1 on process 1, s2 on process 2, r1 and r2 on process 3

  • We focus on the first 3 elements of the 3rd row of the matrices

CS4231 Parallel and Distributed Algorithms AY2006/2007 Semester 2


Correctness proof of the protocol10 l.jpg
Correctness Proof of the Protocol

x1

y1

0

x2

y2

0

x2  x1

y2  y1

Alice (process1)

x4  x3  x2

x3

y3

0

x4

y4

0

y4 > y3  y2

Bob (process2)

x1

y1

0

x4

y4

0

Cary (process3)

CS4231 Parallel and Distributed Algorithms AY2006/2007 Semester 2


Correctness proof of the protocol continued l.jpg
Correctness Proof of the Protocol (continued)

  • Prove by contradiction: Assume Red delivered before Blue

    • After delivering Red: 3rd column of the matrix on process 3 will be

    • The values in the matrix never decreases: Blue can never be delivered

 x4

 y4

0

x4  x1

x1

y1

0

x4

y4

0

y4 > y1

Cary (process3)

CS4231 Parallel and Distributed Algorithms AY2006/2007 Semester 2


Need more correctness proof l.jpg
Need More Correctness Proof

  • What we proved so far:

    • If s1 happened before s2, then r1 is never delivered after r2 is delivered

    • We don’t know if r1 and r2 will be delivered at all!

  • We will now prove that

    • At any given time, there must be one message that can be delivered to the process – induction will take care of the other messages

CS4231 Parallel and Distributed Algorithms AY2006/2007 Semester 2


More correctness proof l.jpg
More Correctness Proof

  • Consider a process and its corresponding column in the matrix

  • If there is any message (from process i) undelivered to the process, then there must be messages whose matrix has the column

x_1

x_2

x_n

?

?

x_i + 1

?

call such message as successor messages

CS4231 Parallel and Distributed Algorithms AY2006/2007 Semester 2


More correctness proof14 l.jpg
More Correctness Proof

  • Consider the non-empty set of all successor messages and their corresponding “send” events

    • There must be a send event s that does not have any other send events that happened before s

    • Call the corresponding message first successor message

  • Claim: Any first successor message can be delivered

    • W.l.o.g, assume the message is from process 1

x_1

x_2

x_n

x_1 + 1

y_2

y_n

need to show x_i  y_i for i between 2 and n

W.l.o.g, assume i = 2

The matrix column on the receiver

The matrix column in the message

CS4231 Parallel and Distributed Algorithms AY2006/2007 Semester 2


More correctness proof15 l.jpg
More Correctness Proof

  • Prove by contradiction: Suppose y_2 > x_2

process1

this is not a first successor message then!

process2

?

x_2

?

x_1 + 1

y_2

y_n

The last delivered message from process 2

must be another un-delivered message

Receiver process

CS4231 Parallel and Distributed Algorithms AY2006/2007 Semester 2


Summary of correctness proof l.jpg
Summary of Correctness Proof

  • All messages will eventually be delivered

  • The delivery order satisfies causal order

CS4231 Parallel and Distributed Algorithms AY2006/2007 Semester 2


Causal ordering of broadcast messages l.jpg
Causal Ordering of Broadcast Messages

  • Broadcast: Every message is sent to all people (including the sender itself)

    • Modeled as n point-to-point messages

  • Application: Internet chat room

  • For same reason as before, we may need to ensure causal order among messages

    • Each process uses the previous protocol

CS4231 Parallel and Distributed Algorithms AY2006/2007 Semester 2


A simpler protocol for causal ordering broadcast l.jpg
A Simpler Protocol for Causal Ordering Broadcast

  • Each process maintains a message log for all messages delivered

    • When sending a message, append the message to log and send the whole log

    • Upon receiving a log, scan the log sequentially, for any message not seen before: Append the message to local log and deliver it

  • Can be used for point-to-point as well

    • But inefficient..

CS4231 Parallel and Distributed Algorithms AY2006/2007 Semester 2


Total ordering of broadcast messages l.jpg
Total Ordering of Broadcast Messages

  • All messages delivered to all processes in exactly the same order

    • Also called atomic broadcast

  • Total ordering does not imply causal ordering

  • Causal ordering does not imply total ordering

  • Thus…there is something called “causal total order”

CS4231 Parallel and Distributed Algorithms AY2006/2007 Semester 2


Application internet chat room l.jpg
Application: Internet Chat Room

  • We want to assign numbers to people have said

    • The number have to be consistent across all users

[0] Alice: Welcome!

[1] Bob: Hello, Alice

[2] Alice: Let’s try to prove P=NP

[3] Bob: OK, how do we start?

….

….

….

[1712] Bob: I am confused, what were we trying to prove?

[1713] Alice: Please refer to message [2]

CS4231 Parallel and Distributed Algorithms AY2006/2007 Semester 2


Application replicated state machine l.jpg
Application: Replicated State Machine

  • State machine can be an abstract of many things

    • E.g., A database

    • Typically deterministic

  • Replication state machine

    • Multiple copies of the same state machine for performance / availability

  • Each “instruction” is broadcast to all state machines

    • The state machines must execution the “instruction” in exactly the same order!

CS4231 Parallel and Distributed Algorithms AY2006/2007 Semester 2


Using a coordinator for total order broadcast l.jpg
Using a Coordinator for Total Order Broadcast

  • A special process is assigned as the coordinator

  • To broadcast a message

    • Send a message to the coordinator

    • Coordinator assigns a sequence number to the message

    • Coordinator forward the message to all processes with the sequence number

    • Messages delivered according to sequence number order

  • Problem:

    • Coordinator is performance bottleneck

    • What if coordinator fails?

CS4231 Parallel and Distributed Algorithms AY2006/2007 Semester 2


Skeen s algorithm for total order broadcast l.jpg
Skeen’s Algorithm for Total Order Broadcast

pick the max clock value received as message number

process 1

  • Each process maintains

    • Logical clock and a message buffer for undelivered messages

  • A message in the buffer is delivered / removed if

    • All messages in the buffer have been assigned numbers

    • This message has the smallest number

broadcast message

acknowledge

process 2

notify message number

process 3

reply with current logical clock value

put msg in buffer

CS4231 Parallel and Distributed Algorithms AY2006/2007 Semester 2


Correctness proof for skeen s algorithm l.jpg
Correctness Proof for Skeen’s Algorithm

pick the max clock value received as message number

process 1

  • Claim: All messages will be assigned message numbers

  • Claim: All messages will be delivered

  • Claim: If message A has a number smaller than B, then B is delivered after A – Prove by contradiction (continue on next slide)

broadcast message

acknowledge

process 2

notify message number

process 3

reply with current logical clock value

put msg in buffer

CS4231 Parallel and Distributed Algorithms AY2006/2007 Semester 2


Correctness proof for skeen s algorithm continued l.jpg
Correctness Proof for Skeen’s Algorithm -- Continued

  • Suppose A is delivered on process 3 after B.

  • A must be placed in buffer after B is delivered

  • A must have a number larger than B – Contradiction.

process 3

B placed in buffer

B’s number received and B delivered

A must be placed in buffer after B delivered

reply for A

key: Process 3’s logical clock now must be larger than B’s number

CS4231 Parallel and Distributed Algorithms AY2006/2007 Semester 2


Summary l.jpg
Summary

  • Chapter 12 “Message Ordering”

  • FIFO ordering for point-to-point messages

    • Already discussed last lecture

  • Causal ordering for point-to-point messages

    • Applications

    • Protocol to ensure causal ordering

  • Causal ordering for broadcast messages

    • Protocol

  • Total ordering for broadcast messages

    • Application

    • Skeen’s algorithm

CS4231 Parallel and Distributed Algorithms AY2006/2007 Semester 2


Homework assignment l.jpg
Homework Assignment

  • Page 206, Problem 12.3. For each pair (x, y) of the three pairs (i.e., C1 & C2, C1 & C3, C2 & C3)

    • Prove that x=y, OR

    • Prove x implies y, and a counter example that satisfies y but not x, OR

    • Prove y implies x, and a counter example that satisfies x but not y, OR

    • A counter example that satisfies y but not x, and a second counter example that satisfies x but not y

  • Think about why the protocol on slide 18 will deliver all messages in causal order

  • For the protocol on slide 22, think about whether the resulting total order satisfy causal order? Why?

  • Homework due a week from today

  • Read Chapter 13

CS4231 Parallel and Distributed Algorithms AY2006/2007 Semester 2