Sharing Data Safely: Interprocess Synchronization and Communication

1. Sharing Data Safely: Interprocess Synchronization and Communication

2. In These Notes . . . • Sharing data safely • When multiple threads/processes interact in a system, new species of bugs arise • Compiler tries to save time by not reloading values which it doesn’t realize may have changed • Switching between threads can lead to trying to operate upon partially updated variables/data structures • We must design the system to prevent or avoid them

3. Volatile Data • Compilers assume that variables in memory do not change spontaneously, and optimize based on that belief • Don’t reload a variable from memory if you haven’t stored something there since the last time you read from it • Read variable from memory into register (faster access) • Write back to memory at end of the procedure, or before a procedure call • This optimization can fail • Example: reading from input port, polling for key press • while (SW_0) ; will read from SW_0 once and reuse that value • Will generate an infinite loop triggered by SW_0 being true • Variables for which it fails • Memory-mapped peripheral register – register changes on its own • Global variables modified by an ISR – ISR changes the variable • Global variables in a multithreaded application – another thread or ISR changes the variable

4. The Volatile Directive • Need to tell compiler which variables may change outside of their control • Use volatile keyword to force compiler to reload these vars from memory for each use volatile unsigned int num_ints; • Pointer to a volatile int volatile int * var; // or int volatile * var; • Now each C source read of a variable (e.g. status register) will result in a assembly language move instruction • Good explanation in Nigel Jones’ “Volatile,” Embedded Systems Programming July 2001

5. Cooperation and Sharing Information • Program consists of one or more threads/processes • Any two threads/processes are either independent or cooperating • Cooperation enables • Improved performance by overlapping activities or working in parallel • Better program structure (easier to develop and debug) • Easy sharing of information • Two methods to share information • Shared memory • Message passing

6. Shared Memory • Is practical when communication cost is low • Low-end embedded systems have no memory protection support • Threads can access the data directly – e.g. global variables • (Who needs seatbelts or airbags!) • UNIX and high-end embedded systems have memory protection support • Impossible to see other processes’ memory space by default • E.g. virtual memory • Establish a mapping between process’s address space to a named memory object which can be shared across processes • POSIX Threads (pthreads) API is a standard for workstation programming

7. Message Passing • Most useful when communication cost is high • Often used for distributed systems • Producer process generates message, consumer process receives it • Each process must be able to name other process • Consumer is assumed to have an infinite receive queue • Bounded queue complicates the programming • OS manages messages • Mailbox is a queue with only one entry

8. The Shared Data Problem • Often we want to split work between ISR and the task code • Some variables must be shared to transfer information • Problem results from task code using shared data non-atomically • An atomic part of a program is non-interruptible • A critical section(group of instructions) in a program must be executed atomically for correct program behavior • “If, while non-atomically reading (writing) a shared data object, a task can be preempted by code which writes (reads or writes) that shared data object, then a race condition exists” • get_ticks() returns a long, formed by concatenating variable tchi and register tc • If an interrupt occurs in get_ticks, we may get old value of tchi and new value of tc • This is a race condition volatile unsigned int tchi=0; #pragma INTERRUPT tc_isr void tc_isr(void) { tchi++; } unsigned long get_ticks(){ unsigned long temp; temp = tchi; temp <<= 16; temp += tc; return temp; } 3 1 2 4 5

9. Critical section: A non-re-entrant piece of code that can only be executed by one process at a time. Some synchronization mechanism is required at the entry and exit of the critical section to ensure exclusive use. Re-entrant Code: Code which can have multiple simultaneous, interleaved, or nested invocations which will not interfere with each other. This is important for parallel processing, recursive functions or subroutines, and interrupt handling. If invocations must share data, the code is non-reentrant. (e.g. using global variable, not restoring all relevant processor state (e.g. flags)) If each invocation has its own data, the code is reentrant. (e.g. using own stack frame and restoring all relevant processor state) Race condition: Anomalous behavior due to unexpected critical dependence on the relative timing of events. Result of get_ticks() example depends on the relative timing of the read and write operations. Critical Sections Lead to Race Conditions

10. Long Integer long int ct; void f1() { ct++; } void f2() { if (ct==0x00010000) /* … */ } ; void f1() add.w #0001H,_ct adcf.w _ct+2 rts ; void f2() cmp.w #0,_ct jnz unequal cmp.w #1,_ct+2 jnz unequal ; equal code here jmp cont unequal: ; unequal code here cont: • What if f2() starts running after the f1’s add.w (resulting in a carry) but before the adcf.w? • Race condition due to non-atomic operation • Data structures • Large variables

11. Size field is modified by both enqueue and dequeue functions Does compiler generate code which is atomic? The original unoptimized code is very inefficient, but the optimized version is good ; Enqueue ; q->Size++; mov.w -2[FB],A0 ; q mov.w -2[FB],A1 ; q mov.w 0024H[A0],0024H[A1] add.w #0001H,0024H[A1] ; Dequeue ; q->Size--; mov.w -3[FB],A0 ; q mov.w -3[FB],A1 ; q mov.w 0024H[A0],0024H[A1] sub.w #0001H,0024H[A1] ; Optimized Enqueue ; q->Size++; mov.w -2[FB],A1 ; q add.w #0001H,0024H[A1] ; Optimized Dequeue ; q->Size--; mov.w -3[FB],A1 ; q sub.w #0001H,0024H[A1] Is Queue Access Atomic for Serial Example?

12. Solution 1 – Disable Interrupts • Disable interrupts during critical section • Renesas syntax -> • Problems • You must determine where the critical sections are, not the compiler (it’s not smart enough) • Disabling interrupts increases the response time for other interrupts • What if interrupts were already disabled when we called get_ticks? • Need to restore the interrupt masking to previous value #define ENABLE_INTS {_asm(" FSET I");} #define DISABLE_INTS {_asm(" FCLR I");} unsigned long get_ticks(){ unsigned long temp; DISABLE_INTS; temp = tchi; temp <<= 16; temp += tc; ENABLE_INTS; return temp; }

13. Are Interrupts Currently Enabled? #define I_MASK (0x0040) #define GET_INT_STATUS(x) {_asm(" STC FLG,$$[FB]",x); x &= I_MASK;} #define ENABLE_INTS {_asm(" FSET I");} #define DISABLE_INTS {_asm(" FCLR I");} unsigned long get_ticks(){ unsigned long temp, iflg; GET_INT_STATUS(iflg); DISABLE_INTS; temp = tchi; temp <<= 16; temp += tc; if (iflg) ENABLE_INTS; return temp; } • FLG’s I flag (bit 6) • Enables/disables interrupts • Section 1.4 of ESM • Need to examine flag register, but how? • Not memory-mapped • Can’t access with BTST • Solution • STC: Store from control register (ESM) • Use a macro (CLPM98) to copy the flag bit into a variable iflg in our code (we copy the whole register, then mask out the other bits) – nifty feature! • Later use that variable iflg to determine whether to re-enable interrupts

14. Solution 2 – Disable the Scheduler • If no ISR shares this data with the thread, can disable scheduler, keeping it from switching to another thread • Interrupts are still enabled • Counter-productive • We added the scheduler to provide efficient processor sharing • This defeats the purpose of the scheduler!

15. Solution 3 – Use Semaphore to Control Access • Operating system typically offers mutual exclusion support through semaphores • Provide mutually exclusive access to a shared resource • Signal occurrence of events • Link resumption of threads to semaphore events • Allow tasks to synchronize their activities • Behavior • Thread requests semaphore to enter critical section • If semaphore available (non-zero), thread enters critical section and OS updates semaphore state (sets to zero or decrements) • If semaphore unavailable (zero), OS moves thread to waiting queue • When a semaphore becomes available, OS moves the thread waiting on it to the ready queue • After critical section, thread releases semaphore

16. Semaphore Operations Provided by OS • Creation/initialization • Take/Wait/Pend/P • Often includes time-out parameter. Wait returns error code, allowing calling task to decide how to deal with lack of semaphore. • Release/Signal/Post/V • If no task is waiting on semaphore, increment its value • If any tasks are waiting on this semaphore, move the highest priority (or longest-waiting) task to the Ready queue • Two types of Semaphores • Binary (0 and 1) • Only one thread can access shared resource at a time • Counting (0 through N) • Up to N devices can access shared resource at a time

17. Using Semaphores • Rules and Overview • We create a semaphore to guard a shared resource to maintain data integrity • We must get permission to access the resource • We must release that permission when done • Semaphore operations • Take (P) the semaphore before (down, pend) • Release (V) it after (up, post) • Value of semaphore indicates number of units of resource available for use • Use a binary semaphore (1 or 0) to control access to a specific resource • P: wait until semaphore is free, then take it (down) • If semaphore is free, take it and continue executing • Otherwise put calling thread into waiting state • V: release the semaphore (up) • If a task is waiting for this semaphore, move that task to the ready queue long int counter; void f1() { Take(counter_sem); counter++; Release(counter_sem); } void f2() { Take(counter_sem); counter++; Release(counter_sem); }

18. Implementing Semaphores • Semaphores are implemented with OS function or macro • Fundamental issue: need to access guard variable atomically • Option 1: disable interrupts before accessing semaphore • Easy to implement • Needs no special support • Delays processor unnecessarily • Becomes a large problem for multiprocessors • Option 2: use special read-modify-write assembler instruction • Test-and-set • Read a memory location and, if the value is 0, set it to 1 and return true. Otherwise, return false • M16C: BTSTS dest (Bit test and set) • Z <= 1 if dest == 0 (“return value is Z flag”), else Z <= 0 • C <= 1 if dest != 0, else C <= 0 • dest <= 1 • BTSTC: Bit test and clear • Fetch-and-increment • Return the current value of a memory location and increment the value in memory by 1 • Compare-and-swap • Compare the value of a memory location with an old value, and if not the same, replace with a new value

19. Solutions to Shared Data Problem • Disable interrupts • Only method if ISR and task share data • Fast – single instruction, typically • Greedy – slows down response time for all other threads • Disable scheduler • Poor performance if no kernel used • Use OS-provided semaphore • Some slowdown, but only significantly affects threads using them • Need OS

20. uC/OS-II Intertask Comm. and Synchronization • OS provides three mechanisms • Semaphores • Message Mailboxes • Message Queues • Common points • Only tasks (not ISRs) can wait on events • Multiple tasks can wait on the same event • When event occurs, highest-priority task runs first • Can specify a time-out period • Implemented with Event Control Blocks (stored in an OS_EVENT type struct) • OSEventType: semaphore, mailbox, or queue • OSEventPtr: pointer, used only for mailbox or queue • OSEventCnt: counter used for semaphore (up to 65535) • OSEventTbl/OSEventGrp: indicate tasks waiting on event

21. Semaphores • Uses • Signal when something has happened • Provide mutually exclusive access to a shared resource • Control access to a set of identical counted resources • Enable this service by defining OS_SEM_EN to 1 in os_cfg.h • Student exercise: Are these semaphores implemented by disabling interrupts or using test-and-set instructions? • Creating a semaphore before its first use • OSSemCreate() in os_sem.c • Argument: initial value • Binary semaphore: set to 1 • Counting semaphore for N shared resources: set to N • Return value: pointer to ECB • Is null if no ECBs left • Use this pointer to refer to semaphore (as “handle”) • Cannot delete or free up ECB for semaphore!

22. Using Semaphores • Taking a semaphore • OSSemPend() in os_sem.c • Arguments: • pointer to semaphore ECB, • time-out value (16 bits) • Time-out value of 0 indicates task will never time out! • error code pointer • Points to place to put error code if something fails • Doesn’t return (task blocks) until message available, or time out expires • So, your code should check error code pointer to determine if it timed out • Signaling a semaphore • OSSemPost() in os_sem.c • Arguments: pointer to semaphore ECB • Highest priority task waiting on semaphore will be added to ready list (and perhaps run immediately)

23. Example: uC/OSII Demo • Tasks • Task 1 • Flashes red LED • Displays count of loop iterations on LCD top line • Task 2 • Flashes green LED • Task 3 • Flashes yellow LED • Displays count of loop iterations on LCD bottom line • LCD problem • Tasks 1 and 3 use the LCD • Task 1 is higher priority than Task 3, so it can preempt task 3 • This preemption could occur within Task 3’s access to the LCD • If this happens, displayed text will be garbled (or controller IC may even hang) • Fix this by sequentializing access (enforcing mutual exclusion) • itoa() problem • itoa() function converts an integer to a text string (char array) • Space for only one string is allocated, so itoa is non-reentrant • Various fixes possible. • Here, just protecting it with LCD semaphore to ensure mutual exclusion • Could also change itoa() so that calling function must allocate space.

24. Semaphore Solution // define semaphore variable as global OS_EVENT * LCD_Sem; // create semaphore in main(), check for failure LCD_Sem = OSSemCreate(1); if (!LCD_Sem) { DisplayString(LCD_LINE1, "Sem Fail"); while (1); } // Task 1 OSSemPend(LCD_Sem, 10000, &err); if (err == OS_NO_ERR) { DisplayString(LCD_LINE1, p); err = OSSemPost(LCD_Sem); } else … // failure code // Task 3 OSSemPend(LCD_Sem, 10000, &err); if (err == OS_NO_ERR) { DisplayString(LCD_LINE2, p); err = OSSemPost(LCD_Sem); } else … // failure code

25. More On Semaphores • Checking the status of a semaphore • OSSemQuery() in os_sem.c • Copies the contents of the ECB into your own data structure • Useful for debugging

26. Mailboxes • Use • Allows task or ISR to send one pointer-size variable to another task • Only one item – no additional buffering • Usually pass a pointer to data structure with actual information • Enable this service by defining OS_MBOX_EN to 1 in os_cfg.h • Must create mailbox before using it • OSMboxCreate() • Argument: optional pointer to message (if Mbox must be full after creation), or else NULL • Send a message to a mailbox • OSMboxPost() • Arguments • Pointer to mailbox ECB • Pointer to message (or, a pointer-sized variable) • Wait for message to arrive at mailbox • OSMboxPend() • Arguments • Pointer to mailbox ECB • Time-out value (16 bits) • Pointer to space for writing error code • Doesn’t return until message available, or time out expires • Be sure to check error code!

27. More on Mailboxes • ISR should get message from mailbox without blocking • OSMboxAccept() • Argument: pointer to mailbox ECB • Returns mailbox contents (pointer) if not empty, void if mailbox was empty • OSMboxQuery() • Copies the contents of the ECB into your own data structure • Useful for debugging

28. Mailbox Example • Enhance demonstration • Task 2 will detect when switch S3 is pressed and tell Task 4 via an mbox • Add Task 4 • Task 4 blocks on the mbox (runs only when something posts a message in the mbox) • When it gets the message, will change a global direction variable • Task 3 uses this direction variable for counting

29. Mailbox Solution // define mailbox variable as global OS_EVENT * Switch3_Mbox; int T3CountDir = 1; // task 3 counts up initially #define SWITCH3_PRESSED (12345) main() { … // create mailbox, check for failure Switch3_Mbox = OSMboxCreate(NULL); if (!Switch3_Mbox) { DisplayString(LCD_LINE1, "MboxFail"); while (1); } // create task 4 OSTaskCreate(Task4,NULL,TOS(Task4Stk), 8); } // end of main Task3() { … counter += T3CountDir; // count in specified direction … } Task4() { … message = (INT16U) OSMboxPend(Switch3_Mbox, 0, &err); // never time out if (err == OS_NO_ERR) { // got a message if (message == SWITCH3_PRESSED) { T3CountDir *= -1; } … }

30. Message Queues • Provides a queue of pointers • Allows tasks or ISRs to send pointer-sized variables to another task with buffering (several can accumulate) • Enable this service by defining OS_Q_EN to 1 in os_cfg.h • Define number of queues as OS_MAX_QS in os_cfg.h • Queue is implemented as circular buffer • Must create queue first • OSQCreate() • Arguments • start: array of pointers • size: number of pointers in array start • User must define array pointed to by start • Returns pointer to ECB • Cannot delete the queue

31. More on Message Queues • Waiting on a message queue • OSQPend() • Arguments • Pointer to queue ECB • Time-out value • Pointer to error code • Returns the message, or NULL • Doesn’t return until message available, or time out expires • Be sure to check error code after function returns • Posting a message in a queue • OSQPost • Arguments • Pointer to queue ECB • Pointer to enqueue • Posts to tail of queue (FIFO) • OSQPostFront • Like OSQPost, but bypasses rest of queue and posts to head (stack)

32. Even More on Message Queues • Getting a message without blocking • OSQAccept() • Argument: pointer to queue ECB • Returns NULL if queue empty, or pointer if not empty • Also dequeues returnd pointer value

33. Message Queue Example • Enhance demonstration further • Task 2 will • Detect when switch S2 is pressed, • Sample the ADC several times • Send those readings to task 1 via a message queue • Task 1 will display those readings on the LCD rather than count • Task 1 will wait for a message in the ADC_Q • If it gets one, it will display it on the LCD and try again (with a slight delay, to make it visible to humans) • If it doesn’t before its period expires, it will display the counter value instead • See code for details

34. A needs resources X and Y B needs resources X and Y Sequence leading to deadlock A requests and gets (locks) X context switch B locks Y B requests X, doesn’t get it, leading to… context switch A can’t get Y B can’t get X Deadlock A() { lock(X); lock(Y); …. unlock(Y); unlock(X); } B() { lock(Y); lock(X); …. unlock(X); unlock(Y); } ContextSwitch B never gets X

35. Deadlock (Cont'd) • Deadlock: A situation where two or more processes are unable to proceed because each is waiting for one of the others to do something. • Livelock: When two or more processes continuously change their state in response to changes in the other process(es) without doing any useful work. This is similar to deadlock in that no progress is made but differs in that neither process is blocked or waiting for anything. • Deadlock can occur whenever multiple parties are competing for exclusive access to multiple resources -- what can be done? • Deadlock prevention • Deadlock avoidance • Deadlock detection and recovery

36. Deadlock Prevention • Deny one of the four necessary conditions • Make resources sharable • No mutual exclusion • Processes MUST request ALL resources at the same time. • Either all at start or release all before requesting more • “Hold and wait for” not allowed • Poor resource utilization and possible starvation • If process requests a resource which is unavailable • It must release all resources it currently holds and try again later • Allow preemption • Leads loss of work • Impose an ordering on resource types. • Process requests resources in a pre-defined order • No circular wait • This can be too restrictive

37. More Deadlock Strategies • Avoidance • Allow necessary conditions to occur, but use algorithms to predict deadlock and refuse resource requests which could lead to deadlock – Called Banker’s Algorithm • Running this algorithm on all resource requests eats up compute time • Detection and Recovery • Check for circular wait periodically. If detected, terminate all deadlocked processes (extreme solution but very common) • Checking for circular wait is expensive • Terminating all deadlocked processes might not be appropriate