Threading and Parallel Programming Constructs

1. Threading and Parallel Programming Constructs OSLAB Sun-Sook Kim 2010.11.05

2. contents • Synchronization • Critical Sections • Deadlock • Synchronization Primitives • Messages • Flow Control-based Concept • Implementation-dependent Threading Feature

3. Synchronization • Coordinate thread execution and management • Enforcing mechanism used to impose constraint on the order of execution of thread • Controls relative order of thread execution • Resolves any conflict among threads • Mutual exclusion • Exist only one thread block in a critical section • Condition synchronization • Specifies some specific conditions

4. Synchronization <Shared Data Synchronization, Where Data d Is Protected by Synchronization Operation> Ti Tj . . . . . . ti Synchronization Operation to Enter tj A section contains shared data or Critical Section d Shared Data tk Synchronization Operation to Leave tl T=f(t) d=f(t)=s(…, ti,tj,tk, tl,…) <Generic Representation of Synchronization Block inside Source Code>

5. Operational Flow of Threads for an Application Operational Flow of Thread Implementation Source Code T1 … . . . Parallel Code Block or a section needs multithread synchronization . . . Parallel Code Block . . T2 Tm . . . Perform synchronization operations using parallel constructs Bi T1 T2 Tn . . . Perform synchronization operations using parallel constructs Bj T1…p

6. Critical Sections • A section of a code block • Shared dependency variables reside • Shared variables have dependency among multiple threads . . . . . . . .. . . . . . T2 ImplementationIssue of Critical Section Tm . . . T1 Perform mutually exclusive and Do not use critical sections simultaneously Minimize the size

7. Deadlock Thread is blocked waiting on a resource of another thread that will never become available Ti Ti Tj Ti Tj ta tj ti li wi = f(td) tb Section Required by thread Ti wj = f(td) rj ri td ti td tc Self-deadlock Recursive Deadlock Lock-ordering Deadlock

8. Synchronization Primitives • Semaphores • First set of software-oriented primitives to accomplish mutual exclusion of parallel process synchronization P : “potential delay”, “wait“ V : “barrier removal”, ”release” semaphore s s.sem=1 begin T:<non-critical section> P(s) <critical section> V(s) Goto T end P(s) →atomic {sem=sem-1; temp=sem} if(temp<0) • {thread T blocked and enlists on a waiting list for s} • V(s) →atomic {sem=sem+1; temp=sem} • if(temp<=0) • {Release one thread from the waiting list for s}

9. Synchronization Primitives • Locks : single thread handles a lock at one instance <acquire/release lock L> • Mutexes : simplest lock an implementation can use. • Recursive locks : recursive until without causing the thread to deadlock • Read-write locks : limit the write access to only one thread • Spin Locks : non-blocking lock , thread context switching with queuing technique, FIFO

10. Messages • Special method of communication to transfer information or a signal from one domain to another Tj Tj Process-Process Messaging Ti Ti Intra-Process Messaging Inter-Process Messaging PM PN <Massage Passing Model>

11. Messages Microprocessor Register Use of hardware system components according to the size of messages Caches(L0,L1<L2=LLC>) DMA Hardware / Processor Also Known as I/O Processor, external to main processor <System Components Associated with Size of Messages>

12. Flow Control-based Concept Fence : prevent movement across a boundary Application • Barrier : • logical computational point in the control flow of operation • Guarantees that no threads proceed beyond an execution logical point until all threads have arrived at that logical point Software Memory Model Fence Mechanism • Guarantees completeness • Halt all post-fence memory operation Hardware Memory Model Multi-core / Multi-processor System

13. Implementation-dependent Threading Feature • Functionalities and features of threads in different environments are very similar • But semantics could be different • Different implementations may causeapplications to fail in unexpected ways Win 64 • thread Win 32 • thread POSIX thread