CMP320 Operating Systems Lecture 07, 08 Operating System Concepts

1. CMP320Operating SystemsLecture 07, 08Operating System Concepts October 05 2009 Arif Butt • Note: Some slides and/or pictures are adapted from Lecture slides / Books of • Dr Mansoor Sarwar. • Dr Kubiatowicz. • Dr P. Bhat. • Dr Hank Levy. • Dr Indranil Gupta. • Text Book - OS Concepts by Silberschatz, Galvin, and Gagne. • Ref Books • Operating Systems Design and Implementation by Tanenbaum. • Operating Systems by William Stalling. • Operating Systems by Colin Ritchie.

2. Today’s Agenda • Review of previous lecture • Cooperating Processes • Producer Consumer Problem • IPC • Message Passing System • Direct Communication • Indirect Communication • Shared Memory System • Using pipes and FIFOS on the shell • Using pipe() system call in UNIX programs • Using mknod() system call in UNIX programs • Using shmget() system call in UNIX programs CMP320 PUCIT Arif Butt

3. Cooperating Processes • Independent process is a process that cannot affect or cannot be affected by the execution of another process. A process that does not share data with another process is independent • Cooperating process is a process that can affect or can be affected by the execution of another process. A process that share data with other process is a cooperating process • Advantages of Cooperating processes: • Information sharing • Computation speed up • Modularity • Convenience CMP320 PUCIT Arif Butt

4. Inter Process Communication • IPC is a mechanism for processes to communicate and to synchronize their actions. There are two fundamental models of IPC: • Shared Memory – A region of memory that is shared by cooperating processes is established. Processes can then exchange information by reading and writing data to the shared region • Messaging system – Communication takes place by means of messages exchanged between the cooperating processes. Processes communicate with each other without resorting to shared variables. (pipes, FIFO, sockets, message queues, semaphores, signals) CMP320 PUCIT Arif Butt

5. Inter Process Communication CMP320 PUCIT Arif Butt

6. Producer-Consumer Problem Empty Pool Consumer Producer Full Pool CMP320 PUCIT Arif Butt

7. Producer-Consumer Problem (cont…) • A producer process produces information that is consumed by a consumer process • To allow producer and consumer run concurrently we must have a buffer that can be filled by the producer and emptied by the consumer. Buffer can be bounded or unbounded • Unbounded Buffer • Places no practical limit on the size of the buffer • The consumer may have to wait for new items (if the buffer is empty), but the producer can always produce new items • Bounded Buffer • Assumes a fixed size buffer • The consumer must wait if the buffer is empty, and the producer must wait if the buffer is full CMP320 PUCIT Arif Butt

8. Solution 1-Bounded Buffer Problem #define BUFFER_SIZE 5 typedef struct{ ---- } item; item buffer[BUFFER_SIZE]; int in = 0; //points to location where next item will be placed, will be used by producer process int out = 0;//points to location from where item is to be consumed, will be used by consumer process Producer Process item nextProduced; while(1) { nextProduced = getItem(); while(((in + 1)%BUFFER_SIZE) == out) ; //do nothing buffer[in] = nextProduced; in = (in + 1) % BUFFER_SIZE; } Consumer Process item nextConsumed; while(1) { while(in == out) ; //do nothing nextConsumed = buffer[out]; out = (out + 1) % BUFFER_SIZE; } What is the problem with this solution? CMP320 PUCIT Arif Butt

9. Solution 2 - Bounded Buffer Problem #define BUFFER_SIZE 5 typedef struct{ ---- } item; item buffer[BUFFER_SIZE]; int in = 0; //points to location where next item will be placed, will be used by producer process int out = 0;//points to location from where item is to be consumed, will be used by consumer process int ctr = 0; Can you do it without using the ctr variable? Producer Process item nextProduced; while(1) { nextProduced = getItem(); while(ctr == BUFFER_SIZE) ; //do nothing buffer[in] = nextProduced; in = (in + 1) % BUFFER_SIZE; ctr++; } Consumer Process item nextConsumed; while(1) { while(ctr == 0) ; //do nothing nextConsumed = buffer[out]; out = (out + 1) % BUFFER_SIZE; ctr--; } CMP320 PUCIT Arif Butt

10. Messaging System • Message passing provides a mechanism to allow processes to communicate and to synchronize their actions without sharing the same address space • It is particularly useful in a distributed environment, where the communicating processes may reside on different computers connected by a network • For example; a chat program used on the www • Message Passing is one solution to the communication requirement. The added bonus is: It works with shared memory and with distributed systems CMP320 PUCIT Arif Butt

11. Messaging System (cont…) • A message passing facility provides at least two operations: • send(message) – message size fixed or variable • receive(message) • If P and Q wish to communicate, they need to: • establish a communicationlink between them • exchange messages via send/receive CMP320 PUCIT Arif Butt

12. Messaging System (cont…) • Direct Communication • Processes must name each other explicitly: • send (P, message) – send a message to process P • receive(Q, message) – receive a message from process Q • Send primitive includes a specific identifier of the destination process • Receive primitive could know ahead of time which process a message is expected • Receive primitive could use source parameter to return a value when the receive operation has been performed • Example is a letter mailed via Pakistan Postal Service • Properties of communication link in Direct Comm.: • A Link is established automatically • A link is associated with exactly one pair of communicating processes • Between each pair there exists exactly one link CMP320 PUCIT Arif Butt

13. Messaging System (cont…) • Indirect Communication • Messages are directed and received from mailboxes (also referred to as ports) • Each mailbox has a unique id • Processes can communicate only if they share a mailbox • We can think of a mailbox as an abstract object into which a message can be placed in or removed from • Properties of communication link in Indirect Comm.: • Link established only if processes share a common mailbox. • A link may be associated with many processes • Each pair of processes may share several communication links CMP320 PUCIT Arif Butt

14. Messaging System (cont…) Indirect Communication (cont…) • Operations • create a new mailbox. • send and receive messages through mailbox. • destroy a mailbox. • Primitives are defined as: send(A, message) – send a message to mailbox A receive(A, message) – receive a message from mailbox A • Mailbox Sharing • P1, P2, and P3 share mailbox A • P1, sends; P2 and P3 receive • Who gets the message? • Solutions • Allow a link to be associated with at most two processes. • Allow only one process at a time to execute a receive operation. • Allow the system to select arbitrarily the receiver. Sender is notified who the receiver was. CMP320 PUCIT Arif Butt

15. Indirect Process Communication (…) CMP320 PUCIT Arif Butt

16. General Message Format CMP320 PUCIT Arif Butt

17. Messaging System (cont…) Synchronization • Message passing may be either blocking or non-blocking • Blocking is considered synchronous • Blocking send. The sending process is blocked until the message is received by the receiving process or by the mailbox. (Producer process blocks when the buffer is full) • Blocking receive. The receiver blocks until a message is available. (Consumer process blocks when the buffer is empty) • Non-blocking is considered asynchronous • Non-blocking send. The sending process sends the message and resumes operation • Non-blocking receive. The receiver receives either a valid message or a null CMP320 PUCIT Arif Butt

18. Messaging System (cont…) Buffering • Whether communication is direct or indirect, messages exchanged by communicating processes reside in a temporary queue. Such queues can be implemented in three ways: • Zero Capacity. The queue has a length of zero. Thus the link cannot have any messages waiting in it. The sender must block until the recipient receives the message • Bounded Capacity. The queue has finite length n. Sender must wait if link is full • Unbounded Capacity. The queue is of infinite length. So any number of messages can wait in it. Thus the sender never blocks CMP320 PUCIT Arif Butt

19. UNIX / LINUX IPC Tools • Pipes are used for communication between related processes (parent-child-sibling) on the same UNIX system. P1 P2 Pipe UNIX/Linux System CMP320 PUCIT Arif Butt

20. UNIX / LINUX IPC Tools • Named pipes (FIFO) are used for communication between related or unrelated processes on the same UNIX system. P1 P2 p1 p2 FIFO UNIX/Linux System CMP320 PUCIT Arif Butt

21. UNIX / LINUX IPC Tools • Socketsare used for communication between related or unrelated processes on the same or different UNIX systems. P1 P2 Network Connection Socket Socket Computer 1 Computer 2 CMP320 PUCIT Arif Butt

22. USING PIPESINCMD LINE & PROGRAMMING CMP320 PUCIT Arif Butt

23. Cmd Line use of Pipes • The UNIX system allows stdout of a command to be connected to stdin of another command using the pipe operator |. • cmd1 | cmd2 | cmd3 | … | cmdN • Stdout of cmd1 is connected to stdin of cmd2, stdout of cmd2 is connected to stdin of cmd3, … and stdout of cmdN-1 is connected to stdin of cmdN. cmd2 cmd1 cmdN pipe pipe pipe CMP320 PUCIT Arif Butt

24. Cmd Line Use of Pipes • Example 1. Write a command that displays the contents of /etc/ directory, one page at a time. $ ls –l /etc/ 1> temp $ less 0< temp This will display the contents of file temp on screen one page at a time. After this we need to remove the temp file as well. $ rm temp So we needed three commands to accomplish the task, and the command sequence is also extremely slow because file read and write operations are involved. Lets use the pipe symbol $ ls –l /etc/ | less The net effect of this command is the same. It does not use a disk to connect standard output of ls –l to standard input of less because pipe is implemented in the main memory. CMP320 PUCIT Arif Butt

25. Cmd Line Use of Pipes (cont…) • Example 2. Write a command that will sort the contents of file students.txt and displays those contents on screen after removing duplication if any. $ sort 0< students.txt 1> stds_sorted $ uniq stds_sorted $ rm stds_sorted Lets use the pipe symbol $ sort students.txt | uniq CMP320 PUCIT Arif Butt

26. Cmd Line Use of Pipes (cont…) • Pipes and redirection operators alone cannot be used to redirect stdout of a command to a file as well as another command in a single command • tee command copies stdin to stdout and at the same time copies it to one or more files • $ tee tout1.txt tout2.txt • tee command is used to redirect the stdout of a command to one or more files, as well as to another command • cmd1 | tee file1 … fileN | cmd2 • Stdout of cmd1 is connected to stdin of tee, and tee sends its input to files file1, … fileN and as stdin of cmd2 • $ who | tee who.txt CMP320 PUCIT Arif Butt

27. System Call - pipe() Call Fails when: • Opening a pipe is equivalent to opening two files, so at least two slots need to be empty in the PPFDT, otherwise, the call will fail. • Buffer space not available in the kernel. • File table is full. CMP320 PUCIT Arif Butt

28. System Call - pipe() Child Process will send a string to Parent Process and Parent Process will print that on screen. fork P2 P1 parent child Child writes in the pipe. It blocks if the pipe is full. Parent read from the pipe. It blocks if the pipe is empty. pipe Read end Write end • Process P1 will create a pipe (having two ends, a read end and a write end). • Then the Parent Process P1 will create a child process P2. • There are two descriptors associated with the pipe. The writer process (child) will use the write descriptor of the pipe. The reader process (parent) will use the read descriptor of the pipe. • The child process will write data onto the pipe and the parent process will read that data at the read end and will display it on the screen. CMP320 PUCIT Arif Butt

29. Sample Program using pipe() /* pipe1.c (Parent creates pipe, forks a child, child writes into pipe, and parent reads from pipe).*/ #include <stdio.h> #include <stdlib.h> #include <string.h> #include <sys/types.h> #include <sys/wait.h> int main(){ intpipefd[2], cpid, n, rc, nr; char *msg = "Hello, world!\n“; char * buf[1024]; rc = pipe(pipefd); //create a pipe if (rc == -1) {//if pipe call fails printf(“\n Pipe failed"); exit(1);} cpid = fork(); //if pipe call is successful we create a child if (cpid == -1) { printf(“\n Fork failed"); //if fork fails exit(1); } CMP320 PUCIT Arif Butt

30. Sample Program using pipe() (cont…) if (cpid == 0) { // Child’s Code (child is the writer process) write(pipefd[1], msg, strlen(msg)); exit(0); } else { // Parent’s Code (parent is the reader process) n = strlen(msg); nr = read(pipefd[0], buf, n); rc = write(1, buf, nr);//use write sc to display the data //read from the pipe on the screen printf("Good work child!\n"); return(0); } } CMP320 PUCIT Arif Butt

31. USING FIFOSINCMD LINE & PROGRAMMING CMP320 PUCIT Arif Butt

32. Named Pipes / FIFOs • First In First Out are special file types in UNIX used for communication between related or unrelated processes on a computer. • Processes communicating with pipes must be related to each other through a common ancestor process that you execute, processes communicating with FIFOs do not have to have this kind of relationship. They can be independently executed programs on a system. • A pipe is a main memory buffer and has no name, while FIFO is created on disk and has a name like a file name. • So like files and unlike a pipe, a FIFO has to be created and opened before using it for communication. • We can use mkfifo command to create a FIFO file in a shell session. We can use mknod() system call or mkfifo() library call to create a FIFO in a process. • UNIX system calls read(), write(), close(), etc. can be used with FIFOs. CMP320 PUCIT Arif Butt

33. Named Pipes / FIFOs • Two common uses of FIFOs are: • Used by shell commands to pass data from one shell pipeline to another, without creating temporary files • In client server applications, FIFOs are used to pass data between a server process and client processes CMP320 PUCIT Arif Butt

34. Cmd Line Use of FIFOs • Example. Write a command that will count the number of lines in the man page of ls. $ man ls | wc –l 239 $ • Example. Write a command that will count only the number of lines containing string “ls” in the man page of ls. $ man ls | grep ls | wc –l 9 $ CMP320 PUCIT Arif Butt

35. Cmd Line Use of FIFOs (cont…) $ mkfifo fifo1 $ prog3 0< fifo1 & $ prog1 0< infile | tee fifo1 | prog2 • First line will create an empty FIFO named fifo1. • In second line, prog3 is trying to read its input from fifo1. Since fifo1 is empty therefore prog3 will block at the read operation. More so the & shows that it will run in the back ground. • In third line, prog1 will read its input from infile and send its o/p to tee utility and tee will do two things: • Write the o/p to fifo1 as well as to the stdout which will be sent to prog3 • Through the pipe will give the o/p to prog2 $ mkfifo fifo1 man ls 1> ls.dat $ cat 0< fifo1 | grep ls | wc –l & [1] 21108 $ sort 0< ls.dat | tee fifo1 | wc –l 9 239 wc -l grep fifo1 Pipe sort tee infile Pipe wc -l CMP320 PUCIT Arif Butt

36. Client Server Comm with FIFOs Server will open a well known FIFO, which it will use to read the request of client. Each client will open a client FIFO and send its name to the server process, so now the server knows via which client FIFO it can send reply to a particular client CMP320 PUCIT Arif Butt

37. System Call - mknod() Call Fails when: • File with the given name already exists • Pathname too long • A component in the pathname not searchable or non existent • Destination directory is read only • Not enough memory space available CMP320 PUCIT Arif Butt

38. Library Call - mkfifo() Call Fails when: • File with the given name already exists • Pathname too long • A component in the pathname not searchable or non existent • Destination directory is read only • Not enough memory space available CMP320 PUCIT Arif Butt

39. Example - FIFO Reading Writing FIFO1 Server Client FIFO2 Reading Writing Display Screen CMP320 PUCIT Arif Butt

40. Client - Server Code //myheader.h #include <stdio.h> #include <stdlib.h> #include <string.h> #include <sys/types.h> #include <sys/stat.h> #include <sys/errno.h> #define FIFO1 "/tmp/fifo1" #define FIFO2 "/tmp/fifo2" #define PERMS 0666 #define MESSAGE1 "Hello, world!\n" #define MESSAGE2 "Hello, class!\n“ extern int errno; CMP320 PUCIT Arif Butt

41. Server Code - 1 #include “myheader.h” main(){ char buff[1024]; int readfd, writefd, n, size; /*create fifo1 using mknod system call*/ if (mknod (FIFO1, S_IFIFO | PERMS, 0) < 0) { perror ("mknod FIFO1"); exit (1); } /*create fifo2 using mkfifo library call. if mkfifo() fails in creating FIFO2, before exiting we must remove FIFO1 from file system*/ if (mkfifo(FIFO2, PERMS) < 0) { unlink (FIFO1); perror("mknod FIFO2"); exit (1); } CMP320 PUCIT Arif Butt

42. Server Code - 2 if ((readfd = open(FIFO1, 0)) < 0) {//open FIFO1 for reading perror ("open FIFO1"); exit (1); } if ((writefd = open(FIFO2, 1)) < 0){//open FIFO2 for writing perror ("open FIFO2"); exit (1); } size = strlen(MESSAGE1) + 1; /*if client has yet not run, the server process will block on following read. As soon as some client executes and writes some msg in FIFO1, the server process will automatically unblock.*/ if ((n = read(readfd, buff, size)) < 0) { perror ("server read"); exit (1); } if (write (1, buff, n) < n) {//write on screen perror("server write1"); exit (1); } size = strlen(MESSAGE2) + 1; if(write(writefd,MESSAGE2,size) != size){//write on FIFO2 perror ("server write2"); exit (1); } close (readfd); close (writefd); } CMP320 PUCIT Arif Butt

43. Client Code - 1 #include “myheader.h” main() { char buff[BUFSIZ]; int readfd, writefd, n, size;  // open FIFO1 for writing if ((writefd = open(FIFO1, 1)) < 0) { perror ("client open FIFO1"); exit (1); } // open FIFO2 for reading if ((readfd = open(FIFO2, 0)) < 0) { perror ("client open FIFO2"); exit (1); } size = strlen(MESSAGE1) + 1; // write message in FIFO1 if (write(writefd, MESSAGE1, size) != size) { perror ("client write1"); exit (1); } CMP320 PUCIT Arif Butt

44. Client Code - 2 /*if server has yet not run, the client process will block on following read. As soon as the server writes some msg in FIFO2, the client process will automatically unblock.*/ if ((n = read(readfd, buff, size)) < 0) { perror ("client read");exit (1); } else if (write(1, buff, n) != n) { perror ("client write2");exit (1); } close(readfd);close(writefd); /* Remove FIFOs now that we are done using them */ if (unlink (FIFO1) < 0) { perror("client unlink FIFO1"); exit (1); } if (unlink (FIFO2) < 0) { perror("client unlink FIFO2"); exit (1); } exit (0); } CMP320 PUCIT Arif Butt

45. FIFO-Running the Code Use any editor to type your program and then to compile use gcc compiler: $ gcc server.c –o server $ gcc client.c –o client $ server & [1] 432056 $ client Hello, World! Hello, Class! $ CMP320 PUCIT Arif Butt

46. Shared Memory CMP320 PUCIT Arif Butt

47. Shared Memory System • Shared memory allows two or more processes to share a given region of memory. • This is the fastest form of IPC because the data does not need to be copied between the client and server. • The only trick in using shared memory is synchronizing access to a given region among multiple processes. • If the server is placing data into a shared memory region, the client shouldn't try to access the data until the sever is done. • Often semaphores are used to synchronize shared memory access. We can use record locking as well. CMP320 PUCIT Arif Butt

48. Code Data Heap Stack Shared Code Data Heap Stack Shared Shared Memory System (cont…) Data 2 Stack 1 Heap 1 • In shared memory we declare a given section in the memory as one that will be used simultaneously by several processes. • Typically a shared memory region resides in the address space of the process creating the shared memory segment. • Other processes that wish to communicate using this shared memory segment must attach it to their address space. Code 1 Stack 2 Data 1 Prog 2 Logical Address Space 2 Prog 1 Logical Address Space 1 Heap 2 Code 2 Shared CMP320 PUCIT Arif Butt

49. System Call shmget() • A process must first create a shared memory segment using shmget() system call (SHared Memory GET). • On success, shmget() returns a shared memory identifier that is used in subsequent shared memory functions. • On failure, it returns -1. • int segment_id = shmget(IPC_PRIVATE, 4096, S_IRUSR | S_IWUSR); CMP320 PUCIT Arif Butt

50. System Call shmat() • Processes that wish to access a shared memory segment must attach it to their address space using the shmat() system call. (SHared Memory ATach) • On success, shmat() returns a pointer to the first byte of shared memory. • On failure, it returns -1. • char * shared_memory = (char*) shmat(segment_id, NULL, 0); CMP320 PUCIT Arif Butt