Understanding Ordering Constraints and Invariants in Multithreaded Environments

1. CPS110: Ordering constraints Landon Cox January 21, 2009

2. Too much milk solution leave noteLandon while (noteMelissa){ do nothing } if (noMilk){ buy milk; } remove noteLandon leave noteMelissa if (no noteLandon){ if (noMilk){ buy milk; } } remove noteMelissa

3. Too much milk “w/o waiting”? lock () if (noNote && noMilk){ leave note “at store” unlock () buy milk lock () remove note unlock () } else { unlock () } lock () if (noNote && noMilk){ leave note “at store” unlock () buy milk lock () remove note unlock () } else { unlock () } Not holding lock Only hold lock while handling shared resource.

4. Review of invariants • What is an invariant? • A “consistent/sane state” • Something that is “always” true • When can an invariant be broken? • Can only be broken while lock is held • And only by thread holding the lock • Really a “public” invariant • What is the data’s state in when the lock is free • Like having a room tidy before guests arrive • Hold a lock whenever manipulating shared data

5. More on invariants • What about reading shared data? • Still must hold lock • Else another thread could break invariant • (Thread A prints Q as Thread B enqueues) • Hold a lock whenever reasoning about shared data

6. Intro to ordering constraints • Say you want dequeue to wait while the queue is empty • Can we just busy-wait? • No! • Still holding lock dequeue () { lock (qLock); element=NULL; while (head==NULL) {} // remove head element=head->next; head->next=NULL; unlock (qLock); return element; }

7. Release lock before spinning? dequeue () { lock (qLock); element=NULL; unlock (qLock); while (head==NULL) {} lock (qLock); // remove head element=head->next; head->next=NULL; unlock (qLock); return element; } What can go wrong? Another dequeuer could “steal” our element Head might be NULL when we try to remove entry

8. One more try dequeue () { lock (qLock); element=NULL; while (head==NULL) { unlock (qLock); lock (qLock); } // remove head element=head->next; head->next=NULL; unlock (qLock); return element; } • Does it work? • Seems ok • Why? • ShS protected • What’s wrong? • Busy-waiting • Wasteful

9. Ideal solution • Would like dequeueing thread to “sleep” • Add self to “waiting list” • Enqueuer can wake up when Q is non-empty • Problem: what to do with the lock? • Why can’t dequeueing thread sleep with lock? • Enqueuer would never be able to add

10. Release the lock before sleep? enqueue () { acquire lock find tail of queue add new element if (dequeuer waiting){ remove from wait list wake up dequeuer } release lock } dequeue () { acquire lock … if (queue empty) { release lock add self to wait list sleep } … release lock } Does this work?

11. Release the lock before sleep? enqueue () { acquire lock find tail of queue add new element if (dequeuer waiting){ remove from wait list wake up dequeuer } release lock } dequeue () { acquire lock … if (queue empty) { release lock add self to wait list sleep } … release lock } 1 2 3 Thread can sleep forever Other problems? Wait list is shared and unprotected. (bad idea)

12. Release the lock before sleep? enqueue () { acquire lock find tail of queue add new element if (dequeuer waiting){ remove from wait list wake up dequeuer } release lock } dequeue () { acquire lock … if (queue empty) { add self to wait list release lock sleep } … release lock }

13. Release the lock before sleep? enqueue () { acquire lock find tail of queue add new element if (dequeuer waiting){ remove from wait list wake up dequeuer } release lock } dequeue () { acquire lock … if (queue empty) { add self to wait list release lock sleep } … release lock } 1 2 3 Problem: missed wake-up Note: this can be fixed, but it’s messy

14. Two types of synchronization • As before we need to raise the level of abstraction • Mutual exclusion • One thread doing something at a time • Use locks • Ordering constraints • Describes “before-after” relationships • One thread waits for another • Use monitors

15. Course administration • Project 0 • 3/12 have perfect scores • If you are struggling with P0, think about why • C++ language/Linux environment issues? • Coding before thinking? • Not reading the spec closely enough? • Time management (i.e. not starting early enough)? • Group management? • Fix any non-language issues before Project 1 • (language issues will get fixed with practice) • Project 0 questions?

16. Course administration • Project 1 will be posted today (due February 18th) • Project 1 data structures • STL (less coding , more learning) • Roll-your-own (less learning, more coding) • Either is acceptable • Reminder: office hours (come talk to us!) • Me: 1-250 on Fridays in D304 • Ryan: 1:30-2:30 on Tu, 430-530 on Th in North Bldg • Niko: 7-9 on Wednesdays in von der Heyden • Matt: 415-615 on Tu on 3rd floor of LSRC • Extra hours near project deadlines • Other questions or concerns?

17. Monitor synchronization • Mutual exclusion • One thread doing something at a time • Use locks • Ordering constraints • Describes “before-after” relationships • One thread waits for another • Monitor: a lock + its condition variable

18. Locks and condition variables • Condition variables • Lets threads sleep inside a critical section • Internal atomic actions (for now, by definition) • CV State = queue of waiting threads + one lock // begin atomic release lock put thread on wait queue go to sleep // end atomic

19. Condition variable operations wait (){ release lock put thread on wait queue go to sleep // after wake up acquire lock } Atomic Lock always held Lock usually held Lock usually held Lock always held signal (){ wakeup one waiter (if any) } Atomic broadcast (){ wakeup all waiters (if any) } Atomic

20. CVs and invariants • User programs • Ok to leave invariants violated before wait? • No: wait can release the lock • Larger rule about returning from wait • Lock may have changed hands • State can change between wait entry and return • Don’t make assumptions about shared state

21. Multi-threaded queue enqueue () { acquire lock find tail of queue add new element signal (lock, CV) release lock } dequeue () { acquire lock if (queue empty) { wait (lock, CV) } remove item from queue release lock return removed item } What if “queue empty” takes more than one instruction? Any problems with the “if” statement in dequeue?

22. Multi-threaded queue enqueue () { acquire lock find tail of queue add new element signal (lock, CV) release lock } dequeue () { acquire lock if (queue empty) { // begin atomic wait release lock sleep and wait // end atomic wait re-acquire lock } remove item from queue release lock return removed item }

23. Multi-threaded queue enqueue () { acquire lock find tail of queue add new element signal (lock, CV) release lock } dequeue () { acquire lock if (queue empty) { // begin atomic wait release lock sleep and wait // end atomic wait re-acquire lock } remove item from queue release lock return removed item } 1 2 4 dequeue () { acquire lock … return removed item } 3

24. Multi-threaded queue enqueue () { acquire lock find tail of queue add new element signal (lock, CV) release lock } dequeue () { acquire lock if (queue empty) { // begin atomic wait release lock sleep and wait // end atomic wait re-acquire lock } remove item from queue release lock return removed item } How to solve?

25. Multi-threaded queue The “condition” in condition variable enqueue () { acquire lock find tail of queue add new element signal (lock, CV) release lock } dequeue () { acquire lock while (queue empty) { wait (lock, CV) } remove item from queue release lock return removed item } Solve with a while loop (“loop before you leap”) You can now do the P1 disk scheduler …

26. Mesa vs. Hoare monitors • So far, we’ve described Mesa monitors • After waking up, no special priority • Threads have to recheck condition • Hoare semantics are “simpler” • But complicate implementation

27. Hoare semantics • Condition guaranteed true after wakeup • i.e. no need to loop • Waiter acquires lock before any threads run • (since lock protects condition) • Including the signaler! • Signaler must give up lock and signal atomically • What would this mean for invariants? • All invariants must be established before signal • We will use Mesa semantics

28. Tips for using monitors • List the shared data needed for problem • Figure out the locks • 1 lock per group of shared data • Bracket code that uses shared data with lock/unlock • Think about before-after conditions • 1 condition variable per type of condition • CV’s lock should protect data used to evaluate condition • Call wait when you need to wait on a condition • Loop to re-check condition when wait returns • Call signal when condition changes • Ensure invariants are established when lock is not held (unlock, wait)

29. Producer-consumer problem • Producer makes something consumer wants • Goal: avoid lock-step (direct hand-off) Soda drinker (consumer) Delivery person (producer) Problem: everyone’s time is wasted.

30. Producer-consumer problem • Instead, use a fixed-size, shared buffer • Producer puts in (must wait when full) • Consumer takes out (must wait when empty) • Must synchronize access to buffer • Examples • Unix pipes: cpp | cc1 | cc2 | as • Interaction between hardware devices/buffers • Project 1 disk scheduler

31. Use a soda machine as a buffer Soda drinker (consumer) Delivery person (producer) Vending machine (buffer)

32. Solving producer-consumer • What are the variables/shared state? • Soda machine buffer • Number of sodas in machine (≤ MaxSodas) • Locks? • 1 to protect all shared state (sodaLock) • Mutual exclusion? • Only one thread can manipulate machine at a time • Ordering constraints? • Consumer must wait if machine is empty (CV hasSoda) • Producer must wait if machine is full (CV hasRoom)

33. Producer-consumer code consumer () { lock (sodaLock) while (numSodas == 0) { wait (sodaLock,hasSoda) } take soda out of machine signal (hasRoom) unlock (sodaLock) } producer () { lock (sodaLock) while(numSodas==MaxSodas){ wait (sodaLock, hasRoom) } add soda to machine signal (hasSoda) unlock (sodaLock) }

34. Variations: looping producer producer () { lock (sodaLock) while (1) { while(numSodas==MaxSodas){ wait (sodaLock, hasRoom) } add soda to machine signal (hasSoda) } unlock (sodaLock) } • Producer • Infinite loop ok? • Why/why not? • Release lock in wait call

35. Variations: resting producer producer () { lock (sodaLock) while (1) { sleep (1 hour) while(numSodas==MaxSodas){ wait (sodaLock, hasRoom) } add soda to machine signal (hasSoda) } unlock (sodaLock) } • Producer • Sleep ok? • Why/why not? • Shouldn’t hold locks during a slow operation

36. Variations: one CV? consumer () { lock (sodaLock) while (numSodas == 0) { wait (sodaLock,hasRorS) } take soda out of machine signal (hasRorS) unlock (sodaLock) } producer () { lock (sodaLock) while(numSodas==MaxSodas){ wait (sodaLock,hasRorS) } add soda to machine signal (hasRorS) unlock (sodaLock) } Two producers, two consumers: who consumes a signal? ProducerA and ConsumerB wait while ConsumerC signals?

37. Variations: one CV? consumer () { lock (sodaLock) while (numSodas == 0) { wait (sodaLock,hasRorS) } take soda out of machine signal (hasRorS) unlock (sodaLock) } producer () { lock (sodaLock) while(numSodas==MaxSodas){ wait (sodaLock,hasRorS) } add soda to machine signal (hasRorS) unlock (sodaLock) } Is it possible to have a producer and consumer both waiting? max=1, cA and cB wait, pC adds/signals, pD waits, cA wakes

38. Variations: one CV? consumer () { lock (sodaLock) while (numSodas == 0) { wait (sodaLock,hasRorS) } take soda out of machine signal (hasRorS) unlock (sodaLock) } producer () { lock (sodaLock) while(numSodas==MaxSodas){ wait (sodaLock,hasRorS) } add soda to machine signal (hasRorS) unlock (sodaLock) } How can we make the one CV solution work?

39. Variations: one CV? consumer () { lock (sodaLock) while (numSodas == 0) { wait (sodaLock,hasRorS) } take soda out of machine broadcast (hasRorS) unlock (sodaLock) } producer () { lock (sodaLock) while(numSodas==MaxSodas){ wait (sodaLock,hasRorS) } add soda to machine broadcast (hasRorS) unlock (sodaLock) } Use broadcast instead of signal

40. Broadcast vs signal • Can I always use broadcast instead of signal? • Yes, assuming threads recheck condition • Why might I use signal instead? • Efficiency (spurious wakeups) • May wakeup threads for no good reason • Next class: reader/writer locks