Fall 2012 CISG 5500 Advanced Operating Systems

1. Chapter 5 Theoretical Foundations Fall 2012CISG 5500Advanced Operating Systems Donghyun (David) Kim Department of Mathematics and Computer ScienceNorth Carolina Central University Some slides are in courtesy of Dr. Erciyes, CSU San Marcos / Dr. Prabhakaran, UT-Dallas

2. Absence of a global clock (system-wide common clock) • Difficult to decide which event occurred first inside a distributed system • Absence of shared memory • No individual process has an up-to-date state of the entire system • A view of a system is coherent if all the observations of different processes are made at the same physical time • A process in a distributed system can be obtain a coherent but partial view of the system or a complete but incoherent view of the system. • A complete view is also referred to as a global state. Inherent Limitation of Distributed Systems

3. $500 $200 $450 $200 Impact of Lack of Global View $500 $250

4. Lamport’s Logical Clocks

5. Space-time Diagram Space Internal Events Messages Lamport’s Logical Clocks – cont’ Time

6. Conditions satisfied: • is clock in Process . • If ab in process , • Let a: sending message m in ; b : receiving message m in ; then, . • Implementation Rules: • R1: (d > 0); clock is updated between two successive events. • R2: ; (d > 0); When receives a message m with a time stamp ( assigned by , the sender; , a being the event of sending message m). • A reasonable value for d is 1 Lamport’s Logical Clocks – cont’

7. Example of Logical Clocks (5=max(4+1,2+1) (7=max(6+1,4+1) (6) (1) (3) (2) (4) Lamport’s Logical Clocks – cont’ (7=max(6+1,4+1)) (4) (1) (3=max(2+1,2+1)) (2)

8. Ensure if ab, then C(a) < C(b). However, reverse is not true! • Reason: each clock can independently advance due to the occurrence of local events in a process! (1) (3) (2) (4) (6) (5) (7) Limitation of Lamport’s Logical Clocks (7) (4) (1) (3) (2)

9. Keep track of transitive dependencies among processes for recovery purposes. • is a “vector” clock at process whose entries are the “assumed”/”best guess” clock values of different processes. • is the best guess of for ’s clock for • Vector clock rules: • for successive events in • For all k, , when a message m with time stamp is received by from . Vector Clocks

10. Equal: iff • Not equal: iff • Less than or equal: iff • Not less than or equal: iff • Less than: iff and • Not less than: iff ( and ) • Concurrent: iff and (2,0,0) (1,0,0) (3,4,1) (0,1,0) (2,3,1) (2,4,1) (2,2,0) Vector Clocks – cont’ (0,0,2) (0,0,1)

11. Events a and b are causally related if or . Otherwise, these events are concurrent. • In the system of vector clocks, iff . • Casual ordering of messages • If , then every recipient of both messages must receive before . (Otherwise, postpones to process until is received) (2) (1) Causal Ordering of Messages

12. BSS: Birman-Schiper-Stephenson Protocol • Broadcast based: a message sent is received by all other processes. • Deliver a message to a process only if the message preceding it immediately, has been delivered to the process. Otherwise, buffer the message. • Accomplished by using a vector accompanying the message. BSS Protocol for Causal Ordering of Messages

13. Process increments the vector time , timestamps, and broadcasts the message m. denotes the number of messages preceding m. • A process receives m. mis delivered when: • for all k in , n is the total number of processes. Delayed message are queued in a sorted manner. • Concurrent messages are ordered by time of receipt. • When mis delivered at , updated according Rule 2 of vector clocks. • has received all ’s messages preceding m. • has received all other messages received by before sending m. BSS Protocol for Causal Ordering of Messages – cont’

14. (0,0,0) (buffer) (0,0,1) (0,1,1) deliver from buffer (0,0,1) (0,1,1) BSS Protocol for Causal Ordering of Messages – cont’ (0,0,1) (0,1,1)

15. In distributed systems, recording a coherent (consistent) global state is challenging due to the absence of a global clock and shared memory. Global State (1) C1: Empty $500 $200 Global State (2) C2: Empty A B C1: transfer $50 Global State (3) Global State $450 $200 C1: Empty C2: Empty A B $450 $250 C2: Empty A B

16. (e.g.,) Global state of A is recorded in (1) and not in (2). • State of B, C1, and C2 are recorded in (2) • Extra amount of $50 will appear in global state • Reason: A’s state recorded before sending message and C1’s state after sending message. Global State (2) Global State (1) C1: transfer $50 C1: Empty Recording Global State $450 $500 $200 $200 C2: Empty C2: Empty A B A B

17. Inconsistent global state if , where • nis number of messages sent by A along channel before A’s state was recorded • is number of messages sent by Aalong the channel before channel’s state was recorded. • Consistent global state: Global State (2) Global State (1) Recording Global State – cont’ C1: transfer $50 n = 0 C1: Empty $450 $500 $200 $200 C2: Empty C2: Empty A B A B

18. Similarly, for consistency • : number of messages received along channel before B’s state recording • m: number of messages received along channel by B before channel’s state was recorded. Global State (3) Global State (2) C1: transfer $50 C1: Empty Recording Global State – cont’ $450 $450 $250 $200 C2: Empty C2: Empty A B A B

19. Also, , as in no system no. of messages sent along the channel be less than that received • Hence, • Consistent global state should satisfy the above equation. • Consistent global state: • Channel state: sequence of messages sent before recording sender’s state, excluding the messages received before receiver’s state was recorded. • Only transit messages are recorded in the channel state. Recording Global State – cont’

20. : message M sent from to • : message Mreceived by from • : time of event x • : local state at • is in iff • is in iff • : set of messages sent/recorded at and NOT received/recorded at Recording Global State – cont’

21. : set of messages NOT sent/recorded at and received/recorded at • Global State, • Consistent Global State, AND for all i in n, is null. • Transitless global state, AND for all iin n, is null. Recording Global State – cont’

22. Strongly consistent global state: consistent and transitless, i.e., all send and the corresponding receive events are recorded in all . Recording Global State – cont’

23. Distributed algorithm to capture a consistent global state. Communication channels assumed to be FIFO. • Uses a marker to initiate the algorithm. Marker sort of dummy message, with no effect on the functions of processes. • Sending Marker by P: • P records its state. • For each outgoing channel C, P sends a marker on Cbefore Psends further messages along C. • Receiving Marker by Q: • If Qhas NOT recorded its state: (a). Record the state of Cas an empty sequence. (b) SEND marker (use above rule). • Else (Qhas recorded state before): Record the state of Cas sequence of messages received along C, after Q’s state was recorded and before Qreceived the marker. • FIFO channel condition + markers help in satisfying consistency condition. (What happens if the FIFO assumption is removed?) Chandy-Lamport Algorithm

24. Initiation of marker can be done by any process, with its own unique marker: <process id, sequence number>. • Several processes can initiate state recording by sending markers. Concurrent sending of markers allowed. • One possible way to collect global state: all processes send the recorded state information to the initiator of marker. Initiator process can sum up the global state. Chandy-LamportAlgorithm – cont’

25. Example: • Channel state example: sent to at , sent to at , …. Send Marker Send Marker Record channel state Record channel state Record channel state Chandy-LamportAlgorithm – cont’