Inter-Process Communication (IPC)

Inter-Process Communication(IPC) Operating Systems

Multiprocessing • Multiple programs run simultaneously • Share resources on the machine • CPU context switches between processes • That are not blocked i.e., waiting for I/O • I/O could be from/to another process • Processes executing concurrently may be • Independent processes • It cannot affect or be affected by another process • Cooperating processes • It can affect or be affected by another process

Motivation for process cooperation • There are several reasons for processes to cooperate • Information sharing • Share data or files • Computation speedup • Ability to use multiple CPU cores • Modularity • Construct system as collection of loosely coupled programs • Permits each program to change independent of the other • Convenience • Copy-paste data in different formats from one application to another.

Inter-Process Communication (IPC) • Cooperating process use Inter-process Communication (IPC) to exchange data and information • There are two fundamental strategies for IPC that are supported by almost all OS • Shared memory • A shared region of memory is established via OS • Processes can exchange information using shared memory • Message passing • Processes send messages to each other for IPC • Messages may go over the network or use special message-passing infrastructure of the OS.

Shared Memory Text, RO data, and Data One process typically creates a shared memory segment via OS calls. Shared Memory Attach/map Text, RO data, and Data Heap Stack Space Shared Memory Virtual memory segments that are mapped to the shared memory Heap Text, RO data, and Data Attach/map Stack Space Shared Memory Heap Stack Space Other processes attach or memory-map (mmap) shared segment into their virtual address space to read/write to the shared memory.

Message Passing Process uses syscalls to send data as message Process uses syscalls to receive data as message Process uses syscalls to send data as message Process uses syscalls to receive data as message Operating System Operating System Network • Processes communicate and coordinate by sending and receiving messages. • Messages contain data to be exchanged. • Communication maybe direct (peer-to-peer) or indirect (through an intermediate shared mail box or address like IP-address) • Communication maybe be synchronous (processes wait until communication is complete) or asynchronous.

Shared-memory vs. Message Passing

Shared memory API • Linux supports following APIs for shared memory operations: • shmget: Allocate a shared memory segment and obtain key (a integer value) for further operations. • shmat: Attach or map shared memory segment into the processes virtual memory space and obtain a pointer for further memory operations. • shmdt: Detach or un-map shared memory segment. • shmctl: Perform various control operations on shared memory segment, including marking shared memory for deletion.

shmget • shmget allocates a shared memory segment • intshmget(int key, size_t size, intshm_flags) • Same key value is used by all processes. A special key value of IPC_PRIVATE is handy to share memory between child processes. • The size of memory to be allocated is in bytes. Note that the OS will round it up to be multiple of virtual memory (VM) page size (typically 4K). • The shm_flags can be • 0 (zero): Get existing shared memory segment with keykey. If segment is not found, then shmget returns -1 indicating error. • IPC_CREATE: Use existing shared memory segment with keykey. If segment does not exist, then create a new one. • IPC_CREATE | IPC_EXCL: Create memory segment with keykey. If one already exists then return with -1 indicating error. • The flags must include S_IRUSR | S_IWUSR flags to enable read & write permissions for user creating the shared memory • The flags may also include flags to enable read & write permissions for users in your group and others (rest of the world). See man pages for various flags. • Return value: A non-negative key value (integer) for use with other shared memory system calls.

shmat • This syscall attaches the shared memory segment identified by a given key into the address space (aka virtual memory space) of the calling process. • void* shmat(intshmKey, const void *addr, intshm_flag) • The shmKey value must be a valid key returned by shmgetsyscall. • The addr parameter is typically NULL permitting the system to choose a suitable unused address to map the shared memory segment. • The shm_flag can be: • 0 (zero): Map for reading and writing operations • SHM_RDONLY: Read only (there is no concept of write only for shared memory) • Return value: The method returns pointer to shared memory location. • The pointer must be suitably type-casted for further use.

shmdt • This system call detaches (or un-maps) an attached (or mapped) shared memory segment • intshmdt(const void* shmaddr) • The shmaddr parameter is the pointer returned by shmatsyscall. • Return value: On success the return value is 0 (zero). On errors, the return value is -1 and errno is set to indicate cause of error. • Notes: • After a fork the child inherits the attached shared memory segments. • After exec all attached shared memory segments are detached from the calling process • Upon exit (or process termination) the OS detaches all attached shared memory segments.

shmctl • Control shared memory segment. This system call is used to mark a shared memory segment for deletion. • intshmctl(intshmKey, intcmd, structshmid_ds*) • shmKey is the value returned by shmget. • cmd has several options (see man page). The IPC_RMID is used to mark segment to be destroyed. The segment is actually destroyed only after the last process detaches it. • Return value: This call returns 0 (zero) on success. On errors it returns -1 and errno is set to indicate cause of error.

A shared memory example #include<iostream> #include<array> #include<sys/ipc.h> #include<sys/shm.h> #include<sys/stat.h> // Shared memory key #define SHM_KEY 12345 #define MEM_SIZE 1024 // Write data into shared memory int main(intargc, char *argv[]) { constintshm_key = shmget(SHM_KEY, MEM_SIZE, S_IRUSR | S_IWUSR | IPC_CREAT); int *sharedMem = reinterpret_cast<int*>(shmat(shm_key, NULL, 0)); for(int i = 0; (i < 25); i++) { *sharedMem = i; sleep(1); } shmdt(sharedMem); shmctl(shm_key, IPC_RMID, NULL); return 0; } #include<iostream> #include<array> #include<sys/ipc.h> #include<sys/shm.h> #include<sys/stat.h> // Shared memory key #define SHM_KEY 12345 #define MEM_SIZE 1024 // Read data from shared memory int main(intargc, char *argv[]) { constintshm_key = shmget(SHM_KEY, MEM_SIZE, S_IRUSR); int *sharedMem = reinterpret_cast<int*>(shmat(shm_key, NULL, SHM_RDONLY)); for(int i = 0; (i < 25); i++) { std::cout << *sharedMem << std::endl; sleep(1); } shmdt(sharedMem); return 0; }

Tips with shared memory • Shared memory is a flat memory region • The OS does not control the type of data stored in it. • Use classes that does not have any virtual methods (aka C structures). • Strings in virtual memory must be dealt with as C-strings. Use intermediate C++ strings for ease: char* c_str = reinterpret_cast<char*>(stmat(shm_key, NULL, 0)); // Copy c-string to C++ string for manipulation std::string str(c_str, MEM_SIZE); // Manipulate c++ string // Copy data from c++ string back to c-string str += ‘\0’; // Ensure trailing ‘\0’ for c-string std::copy(str.begin(), str.end(), c_str); • It is up to the cooperating processes to appropriately manage shared memory • This can often lead to race conditions that we will explore later on in this course

A shared memory example with strings #include<sys/shm.h> #include<sys/stat.h> // Shared memory key #define SHM_KEY 12345 #define MEM_SIZE 1024 // Write string into shared memory int main(intargc, char *argv[]) { constintshm_key = shmget(SHM_KEY, MEM_SIZE, S_IRUSR | S_IWUSR | IPC_CREAT); char *cStr = reinterpret_cast<char*>(shmat(shm_key, NULL, 0)); for(int i = 0; (i < 25); i++) { std::ostringstreammsg; msg << "This is test #" << i << "..."; std::string str = msg.str() + ‘\0’; std::copy(str.begin(),str.end(),cStr); sleep(1); } shmdt(sharedMem); shmctl(shm_key, IPC_RMID, NULL); return 0; } #include<iostream> #include<sys/shm.h> #include<sys/stat.h> // Shared memory key #define SHM_KEY 12345 #define MEM_SIZE 1024 // Read data from shared memory int main(intargc, char *argv[]) { constintshm_key = shmget(SHM_KEY, MEM_SIZE, 0); char *cStr = reinterpret_cast<char*>(shmat(shm_key, NULL, SHM_RDONLY)); for(int i = 0; (i < 25); i++) { std::string str(cStr, MEM_SIZE); std::cout << str << std::endl; sleep(1); } shmdt(sharedMem); return 0; }

Message passing API • Linux supports following APIs for message passing operations: • Note: These work only on the same macine! • msgget: Allocate message queue and obtain key (a integer value) for further operations. • msgsnd: Send message to a given message queue. • msgrecv: Read (and remove) message from a given message queue. • msgctl: Perform various control operations on a message queue, including deleting the queue immediately.

msgget • msgetreturns message queue identifier associated with a given key • intmsgget(int key, intmsg_flags) • Same key value is used by all processes to operate on a given queue. A special key value of IPC_PRIVATE is handy to create new key. • The msg_flags can be • 0 (zero): Get existing queue for key. If queue is not found, then msgmget returns -1 indicating error. • IPC_CREATE: Use existing queue with keykey. If queue does not exist, then create a new one. • IPC_CREATE | IPC_EXCL: Create queue with keykey. If one already exists then return with -1 indicating error. • The flags must include S_IRUSR | S_IWUSR flags to enable read & write permissions for user creating the shared memory • The flags may also include flags to enable read & write permissions for users in your group (S_IRGRP | S_IWGRP) and others (S_IROTH | S_IWOTH). • Return value: A non-negative key value (integer) for use with other message queue system calls.

msgsnd • Send a message to a given message queue • The calling process must have write permissions • intmsgsnd(intqKey, const void* data, size_tdataSize, intmsg_flag) • qKey is the value returned by msgget. • data is pointer to the data to be written. This must be in the form: structMessage { longintmy_msg_type; // User-defined message type char data[<size>]; }; • dataSize is number of bytes to be written • msg_flag can be • 0 (zero): Block until space is available in queue • IPC_NOWAIT: Exit with error (sets errno) if queue is full • Return value: On success the return value is 0 (zero). On errors, the return value is -1 and errno is set to indicate cause of error.

msgrcv • Receive a message from a given message queue • The calling process must have write permissions • intmsgrcv(intqKey, void* data, size_tdataSize, longmsgtyp, intmsg_flag) • qKey is the value returned by msgget. • data is pointer to storage location in the form: structMessage { longintmy_msg_type; char data[<size>]; }; • dataSize is maximum number of bytes that can be read • msgtype is (user-defined) message type to be read. • msg_flag can be • 0 (zero): Block until a message is available in queue • IPC_NOWAIT: Exit with error (sets errno) if queue is full • Return value: On success the return value is 0 (zero). On errors, the return value is -1 and errno is set to indicate cause of error.

msgctl • Control message queue operations. This system call is used to delete a message queue. • intmsgctol(intqKey, intcmd, structmsqid_ds*) • qKey is the value returned by msgget. • cmd has several options (see man page). The IPC_RMID is used to delete the queue. The queue is deleted immediately. • Return value: This call returns 0 (zero) on success. On errors it returns -1 and errno is set to indicate cause of error.

Message Passing Example: Common header file Msg.h #include<iostream> #include<cstdio> #include<cstring> #include<sys/msg.h> #include<sys/stat.h> #define MSG_SIZE 124 #define MSG_TYPE 42 structMessageType { longintmy_msg_type; chardata[MSG_SIZE]; };

Message Passing Example #include"Msg.h" intmain() { constintmsqid = msgget(123456, S_IRUSR | S_IWUSR | S_IROTH | IPC_CREAT); // Check to ensure msqid is not -1! MessageTypemsg; for(int i = 0; (i < 25); i++) { msg.my_msg_type = MSG_TYPE; sprintf(msg.data, "Testing-%d", i); msgsnd(msqid, &msg, MSG_SIZE, 0); sleep(1); } msgctl(msqid, IPC_RMID, NULL); return 0; } #include"Msg.h" intmain() { constintmsqid = msgget(123456, S_IRUSR ); // Check to ensure msqid is not -1! MessageTypemsg; while (msgrcv(msqid, &msg, MSG_SIZE, MSG_TYPE, 0) != -1){ std::cout << msg.data << std::endl; } return 0; }

ipcs • Linux provides a utility command called ipcs that can be used to list various IPC entries • raodm@cse381-f12:~/$ ipcs • ------ Shared Memory Segments -------- • key shmid owner perms bytes nattch status • 0x00003039 2293760 raodm 600 1024 1 • ------ Semaphore Arrays -------- • key semid owner perms nsems • ------ Message Queues -------- • key msqid owner perms used-bytes messages • 0x0001e240 393217 raodm 604 248 2 • raodm@cse381-f12:~/$

Permissions • The IPC entries are similar to files on Linux • Permissions to access them are controlled by the system’s permission management layer • On Linux & Unix machines permissions are organized into 3 categories: • User: The user who created an entry • Group: The set of users belonging to the user’s groups. • System administrator creates groups and adds users to various groups. • Setting group privileges permits other users in a group to access and use shared resources such as files & IPC entries • Others: Rest of the users of the system

Permissions • Most Linux & Unix commands display permissions as either numbers or letters for user, group, & others in the following forms: • As a set of at least 9 letters letters • 3 chars for user, group, & others: rwxrwxrwx • Where r is for read permission, w is for write permission, and x is for execute permission. If a permission is not available then a - is displayed • Example: rw-r--r--, user can read & write, group and others can only read • As octal 3 octal digits • Each digit for user, group, & others in this order • Octal numbers have 3 bits, each bit corresponds to: • 001: (Lowest-bit) Execute privilege • 010: (Middle bit) Write Access • 100: (Highest-bit) Read Access • Accordingly, rw-r--r--maps to 0644.

Pipes • Pipe is used for IPC and acts a conduit for data • Pipes were one of the first IPC mechanisms in Unix • There are two types of pipes • Anonymous or ordinary pipes • Exist only when communicating processes are running • Are unidirectional (can read or write but not both) • Need two pipes for bidirectional communication • Always limited to processes on the same machine • Named pipes (aka FIFOs) • Exist independent of communicating processes • Are unidirectional in Linux but bidirectional in Windows™ • In Unix FIFOs carry stream data • On Windows™ Named pipes can also be used for • Message passing • Be used across machines in the same workgroup

Working with ordinary pipes • Ordinary (or anonymous) pipes are the most frequently used constructs in operating systems • The system call pipe(intfd[]) is used to create a pair of pipe handles • First file descriptor (fd[0]) is for read-end of the pipe • Second file descriptor (fd[1])is for write-end of the pipe • The file descriptors are preserved even after a call to fork and execvp. • This permits the pipes to be used to communicate between parent and child. • The file descriptors are similar to standard input and standard output streams of a process • This permits the use of the dup2(intoldfd, intnewfd) system call to duplicate or tie (or copule) file descriptors together.

Typical use of pipes: An Overview • Ordinary (or anonymous) pipes are typically used in the following manner: • Parent process creates one or more pipes • The number depends on the number of streams of data to be exchanged. • Parent forks to create a child process • Now both the child and parent have all the pipe file descriptors • Parent Process operations: • Closes relevant end of the pipe(s). For instance if parent is sending data to the child, then parent will close read-end of the corresponding pipe. • Wrap file descriptors into I/O stream for convenience • Child process operations (concurrently with parent): • Closes relevant end of the pipe(s). For instance if child is reading data, then child will close the write-end of the corresponding pipe. • Child may perform additional file descriptor manipulations via dup2() system call. • Child then uses the pipe(s) or run execvp to run a different program

Simple Pipe (Part 1/2) #include<iostream> #include<cstring> #include<ext/stdio_filebuf.h> #include<unistd.h> #include<sys/wait.h> #define READ 0 #define WRITE 1 void child(intpipefd[]); void parent(intpipefd[]); int main() { intpipefd[2]; pipe(pipefd); // check return to ensure success pid_tpid = fork(); if (pid == 0) { child(pipefd); } else { parent(pipefd); intexitCode = 0; waitpid(pid, &exitCode, 0); } return 0; }

Simple Pipe (Part 2/2) void parent(intpipefd[]) { close(pipefd[READ]); std::filebuf *fb = new __gnu_cxx::stdio_filebuf<char>(pipefd[WRITE], std::ios::out, 1); std::ostream *os = new std::ostream(fb); for(int i=0; (i<10); i++) { (*os) << "Testing #" << i << std::endl; } deleteos; close(pipefd[WRITE]); } void child(constintpipefd[]) { close(pipefd[WRITE]); std::filebuf *fb = new __gnu_cxx::stdio_filebuf<char>(pipefd[READ], std::ios::in, 1); std::istream *is = new std::istream(fb); std::string line; while (!is->eof() && is->good()) { getline(*is, line); std::cout << "* " << line << std::endl; } delete is; close(pipefd[READ]); }

Multi-process pipe (1/2) #include<iostream> #include<cstring> #include<ext/stdio_filebuf.h> #include<unistd.h> #include<sys/wait.h> #define READ 0 #define WRITE 1 voidrun_ps(intpipefd) { // Tie std::cout to write // end of pipe dup2(pipefd, WRITE); charcmd[3] = "ps"; char arg0[5] = "-fea"; char *argv[] = { cmd, arg0, NULL }; execvp(cmd, argv); } voidrun_grep(intpipefd) { // Tie std::cin of grep // to read end of pipe dup2(pipefd, READ); charcmd[5] = "grep"; char arg0[6] = "emacs"; char *argv[] = { cmd, arg0, NULL }; execvp(cmd, argv); }

Multi-process Pipe (2/2) // Accomplish "ps -fea | grepemacs" int main(intargc, char *argv[]) { intpipefd[2]; pipe(pipefd); // check return to ensure success pid_t pid2, pid1 = fork(); if (pid1 == 0) { // Child process #1 close(pipefd[1]); run_grep(pipefd[0]); } else { // Fork and run "grepemacs" (in parent) pid2 = fork(); if (pid2 == 0) { // Child process #2 close(pipefd[0]); run_ps(pipefd[1]); } else { // In parent process. // Wait for ps -fea command to finish intexitCode = 0; waitpid(pid2, &exitCode, 0); // Close write-end of input stream // to grep Otherwise // grep will *not* terminate! close(pipefd[1]); // Wait for grep command to finish waitpid(pid1, &exitCode, 0); } } return 0; }

Common use of pipes • The shell provides a powerful and convenient mechanism using pipes • To connect output of a program as input to another program • ps –fea|grepemacs • Pipes are also used to redirect output to a file • ps –fea | grepemacs> output.txt • Pipes are used to redirect input from a file • ps –fea > out.txt • grepemacs< out.txt

Inter-Process Communication (IPC)