Multithreaded Programming using Java Threads

1. Multithreaded Programming using Java Threads Rajkumar Buyya Cloud Computing and Distributed Systems (CLOUDS) Laboratory Dept. of Computer Science and Software Engineering University of Melbourne, Australia http://www.cloubus.org/~raj or http://www.buyya.com Lecture: Christian Vecchiola csve@unimelb.edu.au

2. Agenda • Introduction • Defining Threads • Java Threading API • Priorities • Synchronization • Assignment 1: • Multi-Threaded Math Server • Advanced Topics: • Concurrency Models: master/worker, pipeline, peer processing • Multithreading Vs multiprocessing

3. Introduction • Modern Systems • Multiple applications run concurrently games web & email office automation multimedsa pictures Multitasking

4. Introduction • Modern Applications (I) • Applications perform many tasks at once.. Background printing GUI rendering Application core logic

5. Introduction • Modern Applications (II) Video Streaming Favorities, Share, Comments Posting

6. Introduction • Modern Applications (III) Process Request Client 1 Process Request Client N Web/FTP server Process Request Client 2 Client 1 Client 2 Multithreading Client N

7. Introduction • Modern Applications & Systems • Operating System Level • Multitasking: multiple applications running at once • Application Level • Multithreading: multiple operations performed at the same time • Bottom Line: • Illusion of concurrency Today’s focus

8. Defining Threads • Threads • Thread: set of instructions that are executed sequentially within an program. public class SingleThread { public static void main(String[] args) { …… … for (inti=0; i<args.Length; i++) { if (args[i].equals("-r") == true) { SingleThread.executeOptionR(args); } } } private static void executeOptionR(String[] args) { …… … } } One Thread

9. Defining Threads • Multithreading • Multithreading: execution of multiple threads within the same process space. public class MultiThread { public static void main(String[] args) { …… … for (inti=0; i<args.Length; i++) { if (args[i].equals("-op1“) == true) { Thread t1 = new Thread(Op1); t1.start(); } if (args[i].equals("-op2“) == true) { Thread t2 = new Thread(Op2); t2.start(); } } } } public class Op1 implements Runnable { public void run() { …… … } } public class Op2 implements Runnable { public void run() { …… … } } Thread Thread Thread

10. Defining Threads • Hierarchy Parallelism Code Granularity Large grain - task level Program Medium grain – control level Function (thread) Fine grain – data level Loop (Compiler) Very fine grain – instruction level Pipeline hardware Sockets / MPI PVM Task i-1 Task i Task i+1 func1(…) { … … } func1(…) { … … } func1(…) { … … } Threads Compilers a(0)=… b(0)=… a(0)=… b(0)=… a(0)=… b(0)=… CPU + X load

11. Defining Threads • Threads and Processes • Each application that is executed on modern operating systems contains at least one thread. Single-threaded Process Address space • When a process starts a default (main) thread is created. • This threads lasts for the entire execution of the process. • It executes the code that is loaded in the process space sequentially. Main Thread Execution Timeline

12. Defining Threads • Threads and Processes Multithreaded Process • When a process starts a default (main) thread is created. • From this thread other threads can be created. • Each of these threads can run for different periods • Each of these threads can create new threads • All the threads have access to a shared memory space. Common address space Main Thread Thread Thread Execution Timeline Thread Thread Multiple Execution Streams

13. Defining Threads • Why to use threads? • Originally conceived to simplify and improve algorithms programming • Major applications: • GUI programming • Throughput computing algoritmo process independent components that can be performed concurrently threads

14. Defining Threads • Applications • Threads are mostly used to perform: • Parallelism and concurrent execution of independent tasks / operations. • Implementation of reactive user interfaces. • Non blocking I/O operations. • Asynchronous behavior. • Timer and alarms implementation.

15. Defining Threads • Example: Web/FTP Server Main Thread while <running> { <wait for request> <create a new worker thread> <start the thread> } <request 2> <request N> <request 1> Web/FTP server Execution Timeline Worker Thread Worker Thread Worker Thread

16. Defining Threads • Summing Up • A piece of code that run in concurrent with other threads. • Each thread is a statically ordered sequence of instructions. • Threads are being extensively used express concurrency on both single and multiprocessors machines. • Programming a task having multiple threads of control – Multithreading or Multithreaded Programming.

17. Java Threading API • Overview • Objects and classes for thread management • java.lang.Thread • java.lang.Runnable • java.lang.ThreadGroup • Language constructs for synchronization • synchronized keyword • Object.[wait(), notify(), notifyAll()] • Advanced features: • thread-safe collections

18. Java Threading API • Thread Creation • Extends the java.lang.Thread class • Implement the java.lang.Runnableinterface Method 2 import java.lang.Runnable; public class Worker implements Runnable { public Worker() { …… } // main method of the thread public void run() { …… } public static void main(String[] args) { Worker wk = new Worker(); Thread thread = new Thread(wk); thread.start(); } } Method 1 import java.lang.Thread; public class Worker extends Thread { public Worker() { …… } // main method of the thread public void run() { …… } public static void main(String[] args) { Worker thread = new Worker(); thread.start(); } }

19. Java Threading API • Thread Creation • Extending java.lang.Thread • Objects are threads • Define a new class that extends Thread • Implement the run method. • Advantages: • Easy to create and start… • Disadvantages: • The class must inherit from Thread

20. Java Threading API • Thread Creation • Extending java.lang.Thread import java.lang.Thread; public class Worker extends Thread { public Worker() { …… } // main method of the thread public void run() { System.out.println("This class extends Thread"); } public static void main(String[] args) { Worker thread = new Worker(); thread.start(); } }

21. Java Threading API • Thread Creation • Implementing java.lang.Runnable • Objects are not threads • Define a new class that implements Runnable • Implement the run method. • Advantages: • The class can inherit from any type… • Disadvantages: • The setup requires more work…

22. Java Threading API • Thread Creation • Implementing java.lang.Runnable import java.lang.Runnable; public class Worker implements Runnable { public Worker() { …… } // main method of the thread public void run() { System.out.println("This class implement Runnable"); } public static void main(String[] args) { Worker worker = new Worker(); Thread thread = new Thread(worker); thread.start(); } }

23. Java Threading API • Thread Life Cycle Management I/O completion new suspend() timer expire / interrupted start() blocked ready notify() sleeping dispatch sleep() waiting suspend() block on I/O running wait() stop() dead completion

24. Java Threading API • Example: 3 Threads Program ThreadTest.java C.java A.java B.java public class C extends Thread { public void run() { for(inti=0; i<5; i++) { System.out.println(“\tFrom Thread C i=“+i); } System.out.println(“Exit from Thread C i=“+i); } } public class ThreadTest { public void main(String[] args) { new A().start(); new B().start(); new C().start(); } } public class A extends Thread { public void run() { for(inti=0; i<5; i++) { System.out.println(“\tFrom Thread A i=“+i); } System.out.println(“Exit from Thread A i=“+i); } } public class B extends Thread { public void run() { for(inti=0; i<5; i++) { System.out.println(“\tFrom Thread B i=“+i); } System.out.println(“Exit from Thread B i=“+i); } }

25. Java Threading API • Example: 3 Threads Program Run 1 Run 2 [raj@mundroo]raj: java ThreadTest From Thread A: i= 1 From Thread A: i= 2 From Thread A: i= 3 From Thread A: i= 4 From Thread A: i= 5 Exit from Thread A From Thread C: k= 1 From Thread C: k= 2 From Thread C: k= 3 From Thread C: k= 4 From Thread C: k= 5 Exit from Thread C From Thread B: j= 1 From Thread B: j= 2 From Thread B: j= 3 From Thread B: j= 4 From Thread B: j= 5 Exit from Thread B [raj@mundroo]raj: java ThreadTest From Thread A: i= 1 From Thread A: i= 2 From Thread A: i= 3 From Thread A: i= 4 From Thread A: i= 5 From Thread C: k= 1 From Thread C: k= 2 From Thread C: k= 3 From Thread C: k= 4 From Thread C: k= 5 Exit from Thread C From Thread B: j= 1 From Thread B: j= 2 From Thread B: j= 3 From Thread B: j= 4 From Thread B: j= 5 Exit from Thread B Exit from Thread A

26. Java Threading API • Thread Priorities • Java allows developer to set the priority with which it will be executed by the JVM. • By default each thread is assigned the NORM_PRIORITY and threads are scheduled in FCFS mode. • There is a range of values for priorities: • MIN_PRIORITY = 1 • NORM_PRIORITY = 5 • MAX_PRIORITY = 10

27. Java Threading API • Thread Priorities • Let’s modify the previous example… ThreadPriorities.java public class ThreadPriorities { public void main(String[] args) { A threadA = new A(); B threadB = new B(); C threadC = new C(); threadA.setPriority(Thread.MAX_PRIORITY); threadB.setPriority(threadA.getPriority()+1); threadC.setPriority(Thread.MIN_PRIORITY); System.out.println(“Started Thread A…"); threadA.start(); System.out.println("Started Thread B…"); threadB.start(); System.out.println("Started Thread A…"); threadC.start(); System.out.println("End of main Thread"); } }

28. Java Threading API • Thread Priorities • Priority values are… • This means.. • The number of available priorities depends on • The implementation of the JVM • The hosting OS • The mapping of java Threads to physical OS threads • Only the ordering of priority values is guaranteed INDICATIVE VALUES!

29. Java Threading API • Other operations • Identification • get/set Name and Id • isActive / isInterrupted • activeCount / currentThread • StackTrace access • Direct operations • suspend / resume / interrupt • start / stop / sleep • Synchronization • join • holdsLock • … more

30. Synchronization • Overview • Required when accessing shared resources • Prevents... • ... spurious write/reads • ... races on data • ... inconsistency of data • Ensures that... • ... only the allowed number of threads (generally one) access the data at the same time. • Requires primitives at language level

31. Synchronization • Overview • Examples: • Printer (two person jobs cannot be printed at the same time). • Simultaneous operations on your bank account. • Can the following operations be done • at the same time on the same account? • - Deposit() • - Withdraw() • - Enquire()

32. Synchronization • Scenario: online bank Internet Bank Server Bank Operator 1 Client 1 Bank Local Area Network Client 2 Bank Operator M Client N Bank Database

33. Synchronization • Scenario: online bank • Assumption • The internet bank server processes each request with a different thread • Case study • A customer operates on account X • An operator updates account X • (Alternatively: two or more customers/operators update the same account)

34. Synchronization • Scenario: online bank • Problem! • If two threads access the same bank account the final value of the account could be unpredictable! • In particular the value could be: • Not valid • Or an amount that does not map to any operation performed • This happens because there is no exclusive access to the bank account data from each thread.

35. Synchronization • Scenario: online bank • Synchronization allows to solve the problem: • It allow each thread to have exclusive access • It makes the accesses of threads sequential • It preserves the consistency of data • But… • It does not enforce an ordering for accessing the data from each thread

36. Synchronization • Synchronization API • Java provides different facilities for synchronization • synchronized • keyword (method modifier) • transforms the entire scope of the method a protected region that is executed atomically • used for simple monitors and binary semaphores • Object.[wait, notify, notifyAll] • “special methods” • implement the primitives wait and signal (Dijkstra) • used for integer semaphore, and more complex synchronization patterns

37. Synchronization • Scenario: online bank • Solution (naïve) • The threads operating on the bank account share the same Account instance • The account class is “synchronized” • deposit() • enquire() • withdraw() are “modified” with the synchronized keyword.

38. Synchronization • Scenario: online bank <balance> + deposit(int amount) + withdraw(int amount) + enquire() Account <account-ref:account> op: account.withdraw(50); <account-ref:account> op: account.enquire(); <account-ref:account> op: account.deposit(1000) Shared Instance Thread Thread Thread Client Request 1 Client Request 2 Client Request N

39. Synchronization • Scenario: online bank • Account class Account.java public class Account { int balance; public synchronized void deposit(int amount) { this.balance += amount; } public synchronized void withdraw(int amount) this.balance -= amount; } public synchronized void enquire( ) { return this.balance; } }

40. Synchronization • Scenario: online bank • Putting all together.. … public class InternetBankingSystem { public void main(String[] args) { String id = args[0]; Account account = getAccountFromDB(id); // perform three different operations on the same account // asynchronously WithdrawThread w = new WithdrawThread(account, 50); DepositThread d = new DepositThread(account, 10000); EnquireThread e = new EnquireThread(account); w.start(); d.start(); e.start(); } }

41. Synchronization • Scenario: online bank • Threads… WithdrawThread.java import Account; public class WithdrawThread extends Thread { private Account account; public withdrawThread(Account acc, intamnt) { this.account = acc; this.amount = amnt; } public void run() { this.account.withdraw(this.amount); } }

42. Synchronization • Object.[wait, notify, notifyAll] • These methods are “special”: • The JVM ensure that they are executed atomically • They have a specific implementation • When to use them? • They can be used to emulate synchronized • They can implement different synchronization patterns • They require more work and a better understanding

43. Assignment 1 • Multi-Threaded Math Server sin(5.0) ? Process Request Client 1 Process Request Client 2 Process Request Client N double y = sqrt(2.0); double y = cos(3.0); double y = sin(5.0); MathServer Client 1 cos(3.0) ? Client 2 • Available operations: • - sin(double x) • - cos(double x) • sqrt(double x) • etc, etc... sqrt(2.0) ? Client N

44. Assignment 1 • Multi-Threaded Math Server • Hints: • Use Sockets for communication… • Use one thread for each request… • Implement a very simple protocol (i.e.): • Request: <op> <value> • Response: <result>

45. Advanced Topics • Overview • Concurrency Models: • The master/worker model • The peer model • A thread pipeline • Multithreading vs Multiprocessing

46. Advanced Topics • Master/Worker Model • 1 Master thread • Coordinates the work • Creates / controls worker thread • Entry point of the system • N Slave threads • Depend upon on master • Performthe execution of the task • Notify (return results to) the master on completion

47. Advanced Topics • Master/Worker Model Resources Program Workers Input/ Request Files Master <Main Thread> <Task Z> <Task X> <Task Y> Databases Thread Thread Thread Thread Disks Special Devices

48. Advanced Topics • Peer Model • All threads are “equal” • The system is completely decentralized • All the threads can process each task • Each of the threads.. • picks up the input • processes it • There is no coordinator thread

49. Advanced Topics • Master/Worker Model Resources Program Peers Input/ Request Files <Task Y> <Task X> <Task Z> Databases Thread Thread Thread Input/ Request Disks Special Devices Input/ Request

50. Advanced Topics • Pipeline Model • Processing happens in “stages” • The system features a sequential behavior • Each thread processes a request and passes it to the next thread in the chain • Each thread represents a stage of the processing • The number of concurrent requests processed is equal to the number of stages of the pipeline