Concurrent access to Objects and variables

1. Concurrent access to Objects and variables Distributed Software Engineering C:\unocourses\4350\slides\DefiningThreads

2. Concurrent access • With multi-threaded programming, threads can access same data, controls need to be in place to ensure that their access has the desired effects and the data remains consistent. • Problems expected: • Current thread get preempted halfway its execution for other thread to execute. • While thread is preempted, other thread access same data (object’s instance variables) and performs changes in data that was or was being manipulated by preempted thread. Distributed Software Engineering C:\unocourses\4350\slides\DefiningThreads

3. Designing solutions • Make operations atomic (to avoid race conditions: two threads sending same message to same object at the same time and behavior of code depends on who wins) • Do not allow a thread to be suspended in the middle of an operation while giving access to another thread to perform operations on same data. • Warning: these solutions create their own set of new problems. • Hard news: we need the solutions proposed but need to deal with the problems created by them. Distributed Software Engineering C:\unocourses\4350\slides\DefiningThreads

4. Multi-threaded programming properties • Safety : nothing bad ever happens • Safety failure:Unintended behavior at run time • Desired goal: all object states are consistent. • Liveness: anything ever happens • Liveness failure: things just stop running • Desired goal: all threads have their chance to run to expected completion. • These properties need to be balanced • But: better do nothing (liveness failure) that something that leads to a safety failure • On the other hand: there are many situations where is better to give an incorrect answer than not one at all. • Warning: some of the easiest thing to improve liveness property can destroy safety property and viceversa. • Get them both right is the challenge in Multi-threaded programming. Distributed Software Engineering C:\unocourses\4350\slides\DefiningThreads

5. Multi-threaded software properties • On top of these two fundamental multi-threaded programming properties, we have the following two software eng. Properties: • Performance The extend to which activities execute soon and quickly. • Reusability The utility of objects and classes across multiple contexts. Distributed Software Engineering C:\unocourses\4350\slides\DefiningThreads

6. Programming gets harder • Most multi-threaded safety matters cannot be checked automatically • Must rely on the programmer and design being implemented: • Strong hint: design up-front. • Design vs. code ratio: 10 – 1 • There are many methodologies to prove manually that designs are safe. We will cover some during this course. • But most importantly: safety properties relies strongly on the software engineering practices by the programmers. Distributed Software Engineering C:\unocourses\4350\slides\DefiningThreads

7. Maintaining object’s consistency • Sometime hard to completely establish what is legal and meaningful for a class • Better route: establish conceptual-level invariants. (Example: volume for water tank must be between zero and capacity) • Object’s consistency follows preservation of their invariants: an object is consistent if all fields obey their invariants. • To uncover invariants: • Clearly understand object’s properties • Constraints on object’s representation • Note: safe objects may occasionally enter transiently inconsistent states, in the mist of methods; but they never attempt to initialize new actions while inconsistent. Distributed Software Engineering C:\unocourses\4350\slides\DefiningThreads

8. Importance of Object’s invariants • One reason for being more careful about invariants in multi-threaded programming: it is much easier to break them than in sequential programming. • Remainder: the need of protection against inconsistency also arises in sequential programming: • Processing exceptions • Making callbacks • Self-calls between methods in a class. Distributed Software Engineering C:\unocourses\4350\slides\DefiningThreads

9. Maintaining consistency: Exclusion • Guaranteeing atomicity • Failures of maintaining atomicity: • Read/Write conflicts • Write/Write conflicts • Impossible to predict the consequences of actions when objects are inconsistent. Distributed Software Engineering C:\unocourses\4350\slides\DefiningThreads

10. Liveness • Goal: every activity eventually progresses towards completion • But an activity may fail (even transiently) for any number of possibly interrelated reason Distributed Software Engineering C:\unocourses\4350\slides\DefiningThreads

11. Liveness failures • Locking • Waiting for a condition • Input from another device • CPU contention • Failure due to exception, error or fault. • But: • Temporal blocking of progress is expected, but this blocking time cannot be unbounded. Distributed Software Engineering C:\unocourses\4350\slides\DefiningThreads

12. Potential permanent liveness failures • Deadlock : circular dependencies • Thread A hold a lock for object X, and needs to acquire and lock object Y • Thread B already locks object Y, and needs to acquire lock for object X. • Missed signals: thread started waiting after given notification to wake up was given. • Nested monitor lockouts: waiting thread holds lock needed by another thread attempting to wake it up. • Livelock: continuously retried action continuously fails. Distributed Software Engineering C:\unocourses\4350\slides\DefiningThreads

13. Potential permanent liveness failures • Starvation • Resource exhaustion • Distributed failure: a remote server machine connected by a socket crashes or otherwise becomes inaccessible. Distributed Software Engineering C:\unocourses\4350\slides\DefiningThreads

14. Performance • Throughput: number of ops/unit of time • Latency: time elapsed between issuing a message and its execution • Capacity: number of simultaneous activities • Efficiency: throughput/(#comp resources) • Scalability: rate at which latency and throughput improve when resources are added to system. • Degradation: rate at which latency or throughput worsens as more clients, activities, operations are added without adding resources. Distributed Software Engineering C:\unocourses\4350\slides\DefiningThreads

15. Efficiency trade-off • Poorer efficiency for better latency, and scalability • Locks • Monitors • Threads • Context-switching • Scheduling • Locality • Algorithmics • All the above reduce efficiency due to overhead and contention that can slow down program. Distributed Software Engineering C:\unocourses\4350\slides\DefiningThreads

16. Strategies for safety in objects • Immutability • Avoid state changes. • Synchronization • Dynamically ensuring exclusive access. • Containment • Structurally ensuring exclusive access. • Variants: • Stateless methods • Partial synchronization • Managed ownership Distributed Software Engineering C:\unocourses\4350\slides\DefiningThreads

17. Immutable Objects • To avoid synchronization, do not change the state of the objects. • Programs are easier to understand • But, these programs are unable to handle user interactions, cooperating threads, etc. • But we can at least search for: selected immutability. Distributed Software Engineering C:\unocourses\4350\slides\DefiningThreads

18. Simplest immutable objects • Do not have state at all. • Their methods are stateless; they are given data through parameters. Such data is primitive type data or object reference which are not updated by the method. Class StatelessAdder { public int add(int a, int b) { return a + b;} } • Methods are always safe and live Distributed Software Engineering C:\unocourses\4350\slides\DefiningThreads

19. Simplest immutable objects • This safety and liveness also hold for classes with only final fields. class ImmutableAdder { public ImmutableAdder (int a) { offset = a;} public int addOffset(int b) { return offset + b; } private final int offset; } Distributed Software Engineering C:\unocourses\4350\slides\DefiningThreads

20. Immutability in API’s • ADT’s implementing values: colors, strings. These classes are called Value containers. • Can create a numeric class where the methods never alter their fields but construct and return values as the result of the methods. • Design different classes supporting different usages: provide immutable and updateable versions: String, StringBuffer. • Protect objects via copying, when is rarely done and is cheap. Distributed Software Engineering C:\unocourses\4350\slides\DefiningThreads

21. Immutability for sharing • Immutability is a useful technique for sharing values and avoid costs associated from exclusion in multi-threaded programming. (MTP) • Instances of many utility classes in MTP are intrinsically immutable and shared by many other objects. class Relay { public Relay ( Server s) { server = s;} public void execute() { server.execute();} } Distributed Software Engineering C:\unocourses\4350\slides\DefiningThreads

22. Constructing immutable objects • Fields must be final • Do not allow fields to be accessed until construction is completed. (must lock) • Field values must be copied (or can use references if values are immutable themselves) • Constructors depending on the construction of other objects must relay on locking. Distributed Software Engineering C:\unocourses\4350\slides\DefiningThreads

23. Immutability variant: stateless methods • A mutable object can provide services via strictly stateless methods, ie, have no bearing on the object’s state. • Stateless methods can be used as if they where methods of an immutable object. • Stateless methods often make copies of arguments and results. • Copying only works if it is to be used locally. • Java does not provide pass-by-copy parameter passing mechanism, so it is left up to the programmer. Distributed Software Engineering C:\unocourses\4350\slides\DefiningThreads

24. Basic concepts: atomic operations (atomicity). • Atomic operations can't be interrupted (divided) • In java assigment to int, boolean is atomic • But assignment to double or long is not atomic long x; • thread 1 may attempt to execute: x = 0x0123456789abcdef • thread 2 may attempt to execute : x = 0; • possible results: 0x0123456789abcdef; 0x0123456700000000; 0x0000000089abcdef; 0x0000000000000000; Distributed Software Engineering C:\unocourses\4350\slides\DefiningThreads

25. 64-bit assignment • 64-bit assignment is effectively implemented as: x.high = 0x01234567 x.low = 0x89abcdef; • You can be preempted between the assignment operations, unless you lock. Distributed Software Engineering C:\unocourses\4350\slides\DefiningThreads

26. Basic concepts: synchronization • Mechanism to provide locking or exclusion by other threads while a thread is executing some common code. • Mechanisms to assure that multiple threads: • Start execution at the same time and run concurrently ("condition variables" or "events"). • Do not run simultaneously when accessing the same object ("monitors"). • Do not run simultaneously when accessing the same code ("critical sections"). • The synchronizedkeyword is essential in implementing synchronization (but is poorly designed.) • No timeout, so deadlock detection is impossible. • There is no way to determine if an object is already lock due to synchronization • If an object attempts to lock an already lock object, it is placed in a list waiting to acquire the lock, but no thread can be removed from that list by client. Distributed Software Engineering C:\unocourses\4350\slides\DefiningThreads

27. Example of need to exclude public class Even { /** * ensure: nVal() % 2 == 0 */ public void next() { ++n; ++n; } public int nVal(){ return n; } private int n = 0; } Typical behavior: This code might work well most of the time, and possibly only rarely will exhibit safety failure. When it does: hard to debug, as the inconsistency is usually manifested in other unrelated objectstate. Solution: Declare next() synchronized. This excludes access to this code by one thread when other thread is executing it: becomes atomic. Distributed Software Engineering C:\unocourses\4350\slides\DefiningThreads

28. Locks and Objects • Every object instance has ONE LOCK and only one. • Scalar fields can only be locked via their enclosing object instance. • Methods and blocks of code can be marked as synchronized; fields are not. • Arrays are objects as is Class object. • Locking an array of objects does not automatically lock all its elements. • There are NO constructs for simultaneously locking multiple objects in a single atomic op. Distributed Software Engineering C:\unocourses\4350\slides\DefiningThreads

29. Lock ownership • An object’s lock is held (owned) by the thread executing synchronized code of the object. • Thus, same thread can call other synchronized methods of the same object. Distributed Software Engineering C:\unocourses\4350\slides\DefiningThreads

30. Block synchronization • A block of code can be synchronized, but it takes an argument on which object to block. Very commonly the object is this synchronized(obj) { //code to lock } Semantics: • No other thread can execute block concurrently. • No other synchronized method or block of code can be executed by any other thread on same object. • Block synchronization can be done on any object • Use it for protecting access to objects with no synchronized methods. • For synchronizing a series of accesses to an object. Distributed Software Engineering C:\unocourses\4350\slides\DefiningThreads

31. Synchronization and inheritance • The synchronized keyword is not part of the method signature • It is NOT a method modifier • It is not inheritable • Methods in interfaces cannot be specified synchronized • Constructors can not be synchronized either, but block synchronization can be used within. • Synchronized instance methods in subclasses employ the same lock as those in the superclass. • Synchronizing an inner class method is independent of its outer class. • Inner classes can use a block with the instance of the outer lock as the object to lock. Distributed Software Engineering C:\unocourses\4350\slides\DefiningThreads

32. Acquiring and releasing lock • Lock is acquired on entry to method or block. • Lock is released on exit of synchronized code, including exit due to an exception. • Locks are per-thread not per invocation of method. • A thread hitting synchronized accesses such code if: • No thread has the lock on the object for such code. OR • Same thread already possesses the lock of such object. • Therefore, a thread possessing an object’s lock can make recursive calls without being locked out. • a synchronized method or block obeys these rules only with respect to other synchronized methods or blocks on the same object. • Methods not synchronized still can execute at any time by any other thread even if synchronized code is executing. • Any thread locked out due to a synchronized code will be place on “hold in a queue” • When the object’s lock is released, there is no guarantee on which thread will acquire the lock next. No fairness, but indeterminancy. Distributed Software Engineering C:\unocourses\4350\slides\DefiningThreads

33. Static fields or methods and synchronization • Object synchronization does not affect static class fields or methods • Static class fields or methods are synchronized with the lock of the Class object for that class. • When synchronizing a static method it acquires a different lock than the lock for any object of that class; but all static methods will use same lock. • A static block can be used against the lock of the object Class for that class. • Example: having class C • synchronized (C.class) {….} Distributed Software Engineering C:\unocourses\4350\slides\DefiningThreads

34. Fully synchronized objects • Safest (but might not be the best) multi-threaded strategy is to restrict attention to fully synchronized objects: • All public methods are synchronized • There are no public fields or other encapsulation violations • All methods terminate • All fields are initialized to a consistent state in the constructors • State of object is consistent (obey invariants) at both the beginning and end of each method, even in the presence of exceptions. • But this use costs and may lead to deadlocks. Distributed Software Engineering C:\unocourses\4350\slides\DefiningThreads

35. Code execution exclusion via synchronization • With synchronized code declaration, two or more threads are guarantee not to interfere with each other on the same object. • If an object has synchronized methods and two threads invoke such methods simultaneously on the same object, synchronization guarantees exclusion but it does not guarantee order of execution. Distributed Software Engineering C:\unocourses\4350\slides\DefiningThreads

36. Constructors and Synchronization • Constructors are not synchronized because it is executed only when creating an object, which can only happen in one thread executing the creation. • Constructors cannot be synchronized. Distributed Software Engineering C:\unocourses\4350\slides\DefiningThreads

37. Synchronization • Always lock during updates to objects. • Always lock during access of possibly updated object fields. • Never lock when invoking methods on other objects. Distributed Software Engineering C:\unocourses\4350\slides\DefiningThreads

38. Basic concepts: semaphores A semaphore is any object that two threads can use to synchronize with one another. Distributed Software Engineering C:\unocourses\4350\slides\DefiningThreads

39. The mutex (mutual-exclusion semaphore) • The mutex is the key to a lock • Though it is sometimes called a “lock.” • Ownership is the critical concept • To cross a synchronized statement, a thread must have the key, otherwise it blocks (is suspended). • Only one thread can have the key (own the mutex) at a time. • Every Object contains an internal mutex: Object mutex = new Object(); synchronized( mutex ) { // guarded code is here. } Distributed Software Engineering C:\unocourses\4350\slides\DefiningThreads

40. Monitors and airplane bathrooms • A monitor is a body of code (not necessarily contiguous), access to which is guarded by a single mutex. • Every object has its own monitor (and its own mutex). • Think “airplane bathroom” • Only one person (thread) can be in it at a time • Locking the door acquires the associated mutex. • You can't leave without unlocking the door. • Other people must line up outside the door if somebody's in there. • Acquisition is not necessarily FIFO order. Distributed Software Engineering C:\unocourses\4350\slides\DefiningThreads

41. Synchronization with individual locks • Enter the monitor by passing over the synchronized keyword: I.e. acquiring an object’s lock, • Object has a synchronized method and a client invoked such method • Client is executing a synchronized block on an object • Entering the monitor does not restrict access to objects used inside the monitor—it just prevents other threads from entering the monitor. long field; Object lock = new Object(); synchronized(lock) { field = new_value; } Distributed Software Engineering C:\unocourses\4350\slides\DefiningThreads

42. Method-level synchronization • The monitor is associated with the object, not the code. • Two threads can happily access the same synchronized code at the same time, provided that different objects receive the request. • E.g. Two threads can enqueue to different queues at the same time, but they cannot simultaneously access the same queue: • Same as synchronized(this) class Queue{ public synchronized void enqueue(Object o){ /*…*/ } public synchronized Object dequeue(){ /*…*/ } Distributed Software Engineering C:\unocourses\4350\slides\DefiningThreads

43. He came in the Bathroom Window • The Bathroom can have several doors class Bathroom_window{ private double guard_this; public synchronized void ringo(double value){ guard_this = value; } public double george(){// WRONG! return guard_this; // needs synchronization } } Distributed Software Engineering C:\unocourses\4350\slides\DefiningThreads

44. Constructors can’t be synchronized, so always have back doors class Unpredictable{ private final int x; private final int y; public Unpredictable(int init_x, int init_y){ new Thread(){ public void run(){ System.out.println(“x=“ + x + “ y=“ + y); } }.start(); x = init_x; y = init_y; } } • Putting the thread-creation code at the bottom doesn’t help (the optimizer might move it). Distributed Software Engineering C:\unocourses\4350\slides\DefiningThreads

45. Locking the constructor’s back door class Predictable{ private final int x; private final int y; public Predictable(int init_x, int init_y){ synchronized( this ){ new Thread(){ public void run(){ synchronized( Predictable.this){ System.out.println(“x=“+x+“ y=“+y); } } }.start(); x = init_x; y = init_y; } } } Distributed Software Engineering C:\unocourses\4350\slides\DefiningThreads

46. Synchronization isn’t cheap class Synch { synchronized int locking ( int a, int b ) { return a + b;} int not_locking ( int a, int b ) { return a + b;} static public void main(String[] arguments){ Synch tester = new Synch(); double start = new Date().get Time(); for(long i = 1000000; --i >= 0 ;) tester.locking(0,0); double end = new Date().getTime(); double locking_time = end - start; // repeat for not_locking } } Distributed Software Engineering C:\unocourses\4350\slides\DefiningThreads

47. Synchronization isn’t cheap • % java -verbose:gc Synch • Pass 0: Time lost: 234 ms. 121.39% increase • Pass 1: Time lost: 139 ms. 149.29% increase • Pass 2: Time lost: 156 ms. 155.52% increase • Pass 3: Time lost: 157 ms. 155.87% increase • Pass 4: Time lost: 157 ms. 155.87% increase • Pass 5: Time lost: 155 ms. 154.96% increase • Pass 6: Time lost: 156 ms. 155.52% increase • Pass 7: Time lost: 3,891 ms. 1,484.70% increase • Pass 8: Time lost: 4,407 ms. 1,668.33% increase • 200MHz Pentium, NT4/SP3, JDK 1.2.1, HotSpot 1.0fcs, E • • Contention in last two passes (Java Hotspot can’t use • atomic-bit-test-and-set). Distributed Software Engineering C:\unocourses\4350\slides\DefiningThreads

48. Synchronization isn’t cheap BUT • The cost of poor design is always higher than the cost of synchronization. • Pick a fast algorithm. • Overhead can be insignificant when the synchronized method is doing a time-consuming operation. • But in OO systems, small synchronized methods often chain to small synchronized methods. Distributed Software Engineering C:\unocourses\4350\slides\DefiningThreads

49. Avoiding synchronization • Reentrant code doesn’t need to be synchronized. • Code that uses only local variables and arguments (no static variables, no fields in the class). • Atomic operations do not need to be synchronized, but beware of reordering. • Assignment to all non-64-bit things, including booleans and references are usually safe, but sequence not preserved. • Must be declared volatile, but volatile might not work. • Assignment to volatile doubles and floats should be atomic (but most JVMs don’t do it). • Code may be reordered, so assignment to several atomic variables must be synchronized. • Sequence of volatile operations should be preserved, but usually isn’t. Distributed Software Engineering C:\unocourses\4350\slides\DefiningThreads

50. Avoiding synchronization • Synchronize the smallest block possible to minimize the odds of contention. • Method-level synchronization should be avoided in very-high-performance systems. • Don’t synchronize the methods of classes that are called only from one thread. • Use Collection-style synchronization decorators when you need synchronized behavior. • Collection c = new ArrayList(); • c = Collections.synchronizedCollection(c); Distributed Software Engineering C:\unocourses\4350\slides\DefiningThreads