Synchronization II

1. Synchronization II • We will be looking at synchronization in two kinds of systems: • Shared memory systems • Mutual exclusion • Semaphores • Message passing systems • Time in distributed systems • Mutual exclusion • Election in distributed systems

2. Shared Memory Coordination • We will be looking at processes coordination using shared memory and busy waiting. • So we don't send messages but read and write shared variables. • When we need to wait, we loop and don't context switch. • This can be wasteful of resources if we must wait a long time.

3. Shared Memory Coordination • Context switching primitives normally use busy waiting in their implementation. • Mutual Exclusion • Consider adding one to a shared variable V. • When compiled onto many machines get three instructions • load r1  V • add r1  r1+1 • store r1  V

4. Mutual Exclusion • Assume V is initially 10 and one process begins the 3 instruction sequence. • After the first instruction context switch to another process. • Registers are of course saved. • New process does all three instructions. • Context switch back. • Registers are of course restored. • First process finishes. • V has been incremented twice but has only reached 11.

5. Mutual Exclusion • The problem is that the 3 instruction sequence must be atomic, i.e. cannot be interleaved with another execution of these instructions • That is, one execution excludes the possibility of another. So they must exclude each other, i.e. we must have mutual exclusion. • This was a race condition. • Hard bugs to find since non-deterministic. • Can in general involve more than two processes

6. Mutual Exclusion • The portion of code that requires mutual exclusion is often called a critical section. • One approach is to prevent context switching. • We can do this for the kernel of a uniprocessor. • Mask interrupts. • Not feasible for user mode processes. • Not feasible for multiprocessors.

7. Mutual Exclusion • Critical Section Problem is to implement: loop trying-part critical-section releasing-part non-critical section end loop • So that when many processes execute this you never have more than one in the critical section. • That is you must write the trying-part and the releasing-part.

8. Mutual Exclusion • Trivial solution. • Let releasing-part be simply "halt”. • This shows we need to specify the problem better. • Additional requirement: • Assume that if a process begins execution of its critical section and no other process enters the critical section, then the first process will eventually exit the critical section.

9. Mutual Exclusion • Then the requirement is "If a process is executing its trying part, then some process will eventually enter the critical section". • Software-only solutions to CS problem. • We assume the existence of atomic loads and stores. • Only up to word length (i.e. not a whole page). • We start with the case of two processes. • Easy if we want tasks to alternate in CS and you know which one goes first in CS.

10. Mutual Exclusion shared int turn = 1 loop loop while (turn=2) while (turn=1) CS CS turn=2 turn=1 NCS NCS

11. Mutual Exclusion • But always alternating does not satisfy the additional requirement above. • Let NCS for process 1 be an infinite loop (or a halt). • We will get to a point when process 2 is in its trying part but turn=1 and turn will not change. • So some process enters its trying part but neither process will enter the CS.

12. Mutual Exclusion • The first solution that worked was discovered by a mathematician named Dekker. • Now we will use turn only to resolve disputes.

13. Dekker’s Algorithm /* Variables are global and shared */ for (; ;) { // process 1 - an infinite loop to show it enters // CS more than once. Turn is initially 1. p1wants = 1; while (p2wants == 1) { if (turn == 2) { p1wants = 0; while (turn == 2) {/* Empty loop */} p1wants = 1; } } critical_section(); turn = 2; p1wants = 0; noncritical_section(); }

14. Dekker’s Algorithm /* Variables are global and shared */ for (; ;) { // process 2 - an infinite loop to show it enters // CS more than once. Turn is initially 1. p2wants = 1; while (p1wants == 1) { if (turn == 1) { p2wants = 0; while (turn == 1) {/* Empty loop */} p2wants = 1; } } critical_section(); turn = 1; p2wants = 0; noncritical_section(); }

15. Mutual Exclusion • The winner-to-be just loops waiting for the loser to give up and then goes into the CS. • The loser-to-be: • Gives up. • Waits to see that the winner has finished. • Starts over (knowing it will win). • Dijkstra extended Dekker's solution for > 2 processes. • Others improved the fairness of Dijkstra's algorithm.

16. Mutual Exclusion • These complicated methods remained the simplest known until 1981 when Peterson found a much simpler method. • Keep Dekker's idea of using turn only to resolve disputes, but drop the complicated then body of the if.

17. Peterson’s Algorithm /* Variables are global and shared */ for (; ;) { // process 1 - an infinite loop to show it enters // CS more than once. p1wants = 1; turn = 2; while (p2wants && turn == 2) {/* empty loop */} critical_section(); p1wants = 0; noncritical_section(); }

18. Peterson’s Algorithm /* Variables are global and shared */ for (; ;) { // process 2 - an infinite loop to show it enters // CS more than once. p2wants = 1; turn = 1; while (p1wants && turn == 1) {/* empty loop */} critical_section(); p2wants = 0; noncritical_section(); }

19. Semaphores • Trying and release often called entry and exit, or wait and signal, or down and up, or P and V (the latter are from Dutch words since Dijkstra is Dutch). • Let’s try to formalize the entry and exit parts. • To get mutual exclusion we need to ensure that no more than one task can pass through P until a V has occurred. The idea is to keep trying to walk through the gate and when you succeed atomically closed the gate behind you so that no one else can enter.

20. Semaphores • Definition (not an implementation): • Let S be an enumerated type with values closed and open (like a gate). P(S) is while S = closed S  closed • The failed test and the assignment are a single atomic action.

21. Semaphores P(S) is label: {[ --begin atomic part if S = open S  closed else }] --end atomic part goto label V(S) is S  open

22. Semaphores • Note that this P and V (not yet implemented) can be used to solve the critical section problem very easily. • The entry part is P(S). • The exit part is V(S). • Note that Dekker and Peterson do not give us a P and V since each process has a unique entry and a unique exit. • S is called a (binary) semaphore.

23. Semaphores • To implement binary semaphores we need some help from our hardware friends. Boolean in out X TestAndSet(X) is oldx  X X  true return oldx • Note that the name is a good one. This function tests the value of X and sets it (i.e. sets it true; reset is to set false).

24. Semaphores • Now P/V for binary semaphores is trivial. • S is Boolean variable (false is open, true is closed). P(S) is while (TestAndSet(S)) V(S) is S  false • This works fine no matter how many processes are involved.

25. Counting Semaphores • Now want to consider permitting a bounded number of processors into what might be called a semi-critical section. loop P(S) SCS // at most k processes can be here // simultaneously V(S) NCS • A semaphore S with this property is called a counting semaphore.

26. Counting Semaphores • If k=1, we get a binary semaphore so counting semaphore generalizes to binary semaphore. • How can we implement a counting semaphore given binary semaphores? • S is a nonnegative integer. • Initialize S to k, the max number allowed in SCS. • Use k=1 to get binary semaphore (hence the name binary). • We only ask for: • Limit of k in SCS (analogue of mutual exclusion). • Progress: If process enters P and < k in SCS, a process will enter the SCS.

27. Counting Semaphores • We do not ask for fairness, and don't assume it (for the binary semaphore) either. binary semaphore q // initially open binary semaphore r // initially closed integer NS; // might be negative, keeps value of S P(S) is V(S) is P(q) P(q) NS-- NS++ if NS < 0 if NS <= 0 V(r) P(r) else V(q) V(q)

28. Distributed System Coordination • No shared memory so: • No semaphores • No test-and-set • No fetch-and-add • Basic "rules of the game" • The relevant information is scattered around the system • Each process makes decisions based on local information • Try to avoid a single point of failure • No common clock supplied by hardware

29. Distributed System Coordination • Our first goal is to deal with the last rule. • Some common time is needed (e.g. make). • Hardware doesn't supply it. • So software must. • For isolated uniprocessor all that really matters is that the clock advances monotonically (never goes backward). • This says that if one time is greater than another the second happened later than the first.

30. Distributed System Coordination • For isolated multiple processor system need the clocks to agree so if one processor marks one file as later than a second processor marks a second file, the second marking really did come after the first. • Processor has a timer that interrupts periodically. The interrupt is called a clock tick. • Software keeps a count of # of ticks since a known time.

31. Distributed System Coordination • Initialized at boot time • Sadly the hardware used to generate clock ticks isn't perfect and some run slightly faster than others. This causes clock skew. • Clocks that agree with each other (i.e. are consistent, see below) are called logical clocks. • If the system must interact with the real world (say a human), it is important that the system time is at least close to the real time. • Clocks that agree with real time are called physical clocks.

32. Logical Clocks • What does it mean for clocks to agree with each other? • For logical clocks we only care that if one event happens before another, the first occurs at an earlier time on the local clock. • We say a happens before b, written ab if one of the following holds: • Events a and b are in the same process, and a occurs before b. • Event a is the sending of message M and b is the receiving of M.

33. Logical Clocks • Transitivity, i.e. a  c and c  b. • We say a happens after b if b happens before a. • We say a and b are concurrent if neither a  b nor b  a. • We require for logical clocks that a  b implies C(a) < C(b), where C(x) is the time at which x occurs. That is C(x) is the value of the local clock when event x occurs.

34. Logical Clocks • How do we ensure this rule (a  b implies C(a) < C(b))? We start by using the local clock value, but need some fixes. • Condition 1 is almost satisfied. If a process stays on a processor, the clock value will never decrease. So we need two fixes. • If a process moves from one processor to another, must save the clock value on the first and make sure that the second is no earlier (indeed it should probably be at lease one tick later).

35. Logical Clocks • If two events occur rapidly (say two message sends), make sure that there is at lease one clock tick between. • Condition 2 is not satisfied. The clocks on different processors can definitely differ a little so a message might arrive before it was sent. • The fix is to record in the message the time of sending and when it arrives, set the local clock to max(local time, message send time + 1) • The above is Lamport's algorithm for synchronizing logical clocks.

36. Physical Clocks • What is time in the physical world. • Earth revolves about the sun once per year • Earth rotates on its axis once per day • 1/24 day is hour, etc. for minute and second • But earth "day" isn't exactly the same all the time so now we use atomic clocks to define one second to be the time for a cesium 133 atom to make 9,192,631,770 "transitions”. • We don't care about any of this really. For us the right time is that broadcast by the NIST on WWV (short wave radio) or via GEOS satellite.

37. Physical Clocks • How do we get the times on machines to agree with NIST? • Send a message to to NIST (or a surrogate) asking for the right time and change yours to that one. • How often? • If you know: • the max drift rate D (each machine drifts D from reality) • the max error you are willing to tolerate E • can calculate how long L you can wait between updates. • L = E / 2D

38. Physical Clocks • If only one machine drifts (other is NIST) L = E/D. • How should we change the time? • It is bad to have big jumps and bad to go backwards. • So make an adjustment in how much you add at each clock tick. • How do you send message and get reply in zero (or even just "known in advance") time? • You can't. See how long it took for reply to come back, subtract service time at NIST and divide by 2. Add this to the time NIST returns.

39. Physical Clocks • What if you can't reach a machine known to be perfect (or within D)? • Ask "around" for time. • Take average. • Broadcast result. • Eliminate outliers. • Try to contact nodes who can contact NIST or nodes who can contact nodes who can contact NIST.

40. Mutual Exclusion • Try to do mutual exclusion without shared memory. • Centralized approach • Pick a process as a coordinator (mutual-exclusion-server) • To get access to Critical Section send message to coordinator and await reply. • When you leave CS send message to coordinator. • When coordinator gets a message requesting CS it • Replies if the CS is free • Enter requesters name into waiting queue

41. Mutual Exclusion • When coordinator gets a message announcing departure from CS • Removes head entry from list of waiters and replies to it • The simplest solution and perhaps the best • Distributed solution • When you want to get into CS • Send request mesasge to everyone (except yourself) • Include timestamp (logical clock!) • Wait until you receive OK from everyone • When receive request ...

42. Mutual Exclusion • If you are not in CS and don't want to be, say OK • If you are in CS, put requester's name on list • If you are not in CS but want to be, • If your TS is lower, put name on list • If your TS is higher, send OK • When you leave CS, send OK to all on your list • Token Passing solution • Form logical ring • Pass token around ring • When you have the token you can enter CS (hold on to the token until you exit)

43. Mutual Exclusion • Comparison • Centralized is best • Distributed of theoretical interest • Token passing good if hardware is ring based (e.g. a token ring LAN)

44. Election • How do you get the coordinator for centralized algorithm? • Bully Algorithm • Used initially and in general when any process notices that the process it thinks is the coordinator is not responding. • Processes are numbered (e.g. by their addresses or names or whatever). • The algorithm below determines the highest numbered process, which then becomes the elected member.

45. Election • Called the bully algorithm because the biggest wins. • When a process wants to hold an election • It sends an election message to all higher numbered processes. • If any respond, original process gives up. • If none respond, it has won and announces election result to all processors (called coordinator message in book). • When you receive an election message: • Send OK.

46. Election • Start an election yourself. • When a process comes back up, it starts an election. • Ring algorithm (again biggest number wins). • Form a (logical) ring. • When a process wants a new leader: • Send election message to next process (in ring) include its process # in message. • If message not ack'ed, send to next in ring.

47. Election • When message returns: • Election done. • Send result message around the ring. • When request message arrives: • Add your # to message and send to next process. • If concurrent elections occur, same result for all.