Efficient Message Ordering in Dynamic Networks - PODC 1996 Talk Slides

1. Efficient Message Ordering in Dynamic Networks PODC 1996 talk slides Idit Keidar and Danny Dolev The Hebrew University Transis Project

2. The System Model • Asynchronous • Network partitions and site failures • Transport layer (Transis) provides: • Locally ordered multicast within connected components • Membership

3. An Algorithm for Totally Ordering Messages • A connected majority can always make progress • Previous protocols sometimes block • All processes may initiate messages, that may eventually be ordered • A majority orders messages within two rounds

4. COReL - Consistent Object Replication Layer Application totally ordered messages multicast messages COReL locally ordered messages send messages membership Transis physical network

5. Reliable Multicast • Logging all messages • ACKing logged messages • Retransmission upon recovery • Gossip-like dissemination

6. Message Ordering • Transis orders messages within each network component • This order is not atomic • COReL uses this order within a primary component: Members of a primary component P totally order a message once it was ACKed by all the members of P

7. The Colors Model • Red: • No knowledge of order • Messages from minority • Yellow: • Message from primary • Green: • Totally ordered • All members of primary marked as yellow (ACKed) Message Queue (MQ)

8. 6 5 4 4 3 3 2 2 1 1 Knowledge Levels in MQ Process s partitioned from p, q and r 5 4 3 x 2 2 1 1 s p q r

9. Invariants of MQ • FIFO and Causal Question is ordered before answer • No changes in green • Agreed green prefix • Yellow If p marked m as green in P, and q knows of P: • q has m as yellow or green • q and p have the same messages before m

10. Recovery:(upon membership change) • Retransmission of missing messages • Green and yellow messages from the last primary component are ordered before concurrent red messages • Agreeing upon MQ • Preserving the invariants • If majority: establish new primary

11. Recovery Majority? Attempt All Attempted? Commit All Committed? Establish Establishing a New Primary If a membership change occurs - restart the Recovery

12. 6 5 y 4 4 4 x 3 3 3 2 2 2 2 1 1 1 1 During Recovery - Need to Know the Last Primary q, r and s become connected BEFORE RECOVERY AFTER y x 4 3 2 1 Last Primary {p, q, r} {q, r, s} {s}

13. Numbering the Primary • How can we determine which primary is latest? • Use two variables: • LAST COMMITTED PRIMARY • Used to know which is later • Unique number • LAST ATTEMPTED PRIMARY • Guarantees uniqueness of LAST COMMITTED PRIMARY

14. A Majority Can Always Make Progress When a majority becomes connected: • All members recover state of last primary • The members establish a new primary • The members of an established primary order messages ACKed by all of them

15. Conclusions • COReL orders messages within connected primary components in two communication rounds • Extends local order determined by Transis within primary components • Messages from minorities diffuse through the system and eventually become ordered

16. Thanks to ... • Yair Amir, Dalia Malki, Catriel Beeri