Inter-process Communication

Inter-process Communication Pipes

Signals provide a simple way of communicating between processes, but the amount of information passed is limited to a signal number How do we pass more useful information?

> cmd1 | cmd2 pipe standard input standard output Pipes connect a data flow from one process to another

Process Pipes A pipe is a mechanism for interprocess communication; data written to the pipe by one process can be read by another process. The data is handled in a first-in, first-out (FIFO) order. The pipe has no name, so it can only be used by the process that created it and by descendents that inherit the file descriptors on fork( ).

A pipe has to be open at both ends simultaneously. If you read from a pipe file that doesn't have any processes writing to it (perhaps because they have all closed the file, or exited), the read returns end-of-file. Using normal blocking reads however, the read will block if the pipe is empty. Writing to a pipe that doesn't have a reading process is treated as an error condition; it generates a SIGPIPE signal, and fails with error code EPIPE if the signal is handled or blocked.

Pipes do not allow file positioning. Both reading and writing operations happen sequentially; reading from the beginning of the file and writing at the end.

A common use of pipes is to send data to or receive data from a program being run as a subprocess.

The pipe call is a low-level call. #include <unistd.h> int pipe(int fd[2]); The pipe call The pipe call fills in two file descriptors fd[0] is the file descriptor for reading from the pipe fd[1] is the file descriptor for writing to the pipe It is easy to remember which comes first if you remember that 0 is standard in and 1 is standard out. returns 0 if successful -1 if the call fails

The following code fragment creates an un-named pipe: Declare the file descriptors, then pass them to the pipe command. int fd[2]; if (pipe(fd) == -1) perror(“Failed to create pipe…”);

Using pipe In typical use, a process creates a pipe just before it forks one or more child processes The pipe is then used for communication either between the parent or child processes, or between two sibling processes.

#include <stdio.h> #include <string.h> #include <unistd.h> #include <sys/types.h> #define BUFSIZE 10 int main ( ) { char bufin[BUFSIZE] = "empty"; char bufout[ ] = "hello"; int bytesin; pid_t childpid; int fd[2]; Example

if(pipe(fd) == -1) { perror("Failed to create the pipe..."); exit (1); } bytesin = strlen(bufin); childpid = fork( ); if (childpid == -1) { perror("Failed to fork child process..."); exit (1); } Create the pipe Create a child process

parent fd[1] if (childpid > 0) // parent code { write(fd[1], bufout, strlen(bufout)+1); } else // child code { bytesin = read(fd[0], bufin, BUFSIZE); } fprintf(stderr, "[%ld]:my bufin is (%.*s), my bufout is (%s)\n", (long)getpid(), bytesin, bufin, bufout); return 0; pipe hello fd[0] child empty pipes/ex6.1

Note that reads from pipes are not atomic! That is, there is no guarantee that the read will get all of the data put into the pipe by the write!

Using a pipe with a new program The next logical step in our discussion is to have the child process exec a new program, and then use the pipe to communicate between these two different programs.

Create a Pipe fork () starting consumer pipe myPipe[0] Example child process exec ( ) open pipe & read until EOF write to pipe hello …

This program is the “Consumer”. It reads text data from a pipe and send it to standard-out a character at a time int main (int argc, char *argv[]) { FILE *stream; int c, fd; printf ( “Consumer started …\n” ); sscanf ( argv[1], “%d”, &fd ); stream = fdopen ( fd, “r” ); printf ( “Consumer … pipe opened.\n” ); while ((c=fgetc(stream) != EOF) putchar ( c ); fclose(stream); } Example fdopen open takes a file descriptor and opens it on a stream. The consumer gets the file descriptor as one of its arguments. Here we convert it back to an int and store it in fd.

this is the “Producer” program. The first thing it does is create a pipe. int main ( ) { pid_t pid; int myPipe[2]; char fd[1024]; if (pipe (myPipe)) { printf(Pipe failed…\n”); }

after creating the pipe, fork a child process. pid = fork( ); if (pid == (pid_t)0) { sprintf ( fd, “%d”, myPipe[0] ); (void) execl ( “./consumer”, “consumer”, fd, (char *) 0 ); return 0; } else if (pid < (pid_t)0) { printf(“Fork failed…\n”); return 1; } else { writeToPipe(myPipe[1]); printf(“Producer … write to pipe complete.\n”); return 0; } } if this is the child process, format the pipe’s file descriptor into a string, and then pass it as a parameter to the exec call, invoking the consumer process. the fork failed… if this is the parent process, write some text data to the pipe using the writeToPipe function created in the previous example.

Communications buffers such as pipes can be empty if all of the information previously written has been read. The empty buffer is not an end-of-file condition. Rather, it reflects the asynchronous nature of inter-process communication. A read call will normally block, waiting for data to become available. However, a read on a pipe that has the other end closed for writing will not block, but will return a zero. This allows the reading process to detect the pipe equivalent to an end-of-file condition and react accordingly.

Pipelines …

What does these commands do? ls –l > myfile.txt sort –n +4 < myfile.txt Sort is a filter. It normally takes input from stdin and outputs to stdout. In this case we redirected its standard input to come from myfile.txt Redirect standard output of the ls command to the file myfile.txt

We can achieve the same effect by using a pipe. This eliminates the intermediate file myfile.txt ls –l | sort –n +4

We can achieve the same effect by using a pipe. This eliminates the intermediate file myfile.txt ls –l | sort –n +4 sort file descriptor table pipe read standard out standard err 0 1 2 1 sort 2 0 pipe 1 ls file descriptor table standard in pipe write standard err 0 1 2 ls 2 0

dup and dup2 Duplicates this file descriptor. The new file descriptor is guaranteed to have the lowest available file descriptor. #include <unistd.h> int dup(int fd); int dup2(int fd1, int fd2); Duplicates this file descriptor on fd2. If the file on fd2 is open, it is closed first and then the duplicate is made.

0 int main ( ) { pid_t childpid; int fd[2]; if ((pipe(fd) == -1) || ((childpid = fork( )) == -1)) { perror("Failed to set up pipeline..."); return (1); } 1 2 parent Do it in a Program … parent file descriptor table standard in standard out standard err 0 1 2

0 1 int main ( ) { pid_t childpid; int fd[2]; if ((pipe(fd) == -1) || ((childpid = fork( )) == -1)) { perror("Failed to set up pipeline..."); return (1); } 2 parent 3 4 Do it in a Program … pipe parent file descriptor table standard in standard out standard err pipe read pipe write 0 1 2 3 4 fd[0] fd[1]

0 1 int main ( ) { pid_t childpid; int fd[2]; if ((pipe(fd) == -1) || ((childpid = fork( )) == -1)) { perror("Failed to set up pipeline..."); return (1); } 2 parent 3 4 pipe 4 3 child 2 0 1 parent file descriptor table child file descriptor table standard in standard out standard err pipe read pipe write standard in standard out standard err pipe read pipe write 0 1 2 3 4 0 1 2 3 4 fd[0] fd[0] fd[1] fd[1]

if (childpid == 0) { if (dup2(fd[1], STDOUT_FILENO) == -1) perror ("Failed to redirect stdout of ls"); else if ((close(fd[0]) == -1) || (close(fd[1]) == -1)) perror ("Failed to close extra file descriptors"); else { execl("/bin/ls","ls", "-l", NULL); perror("Failed to exec ls ..."); } return (1); } Standard out is first closed, then file descriptor fd[1] is duplicated on the file descriptor for stdout. 0 child file descriptor table after dup2 call 1 2 parent standard in pipe write standard err pipe read pipe write 0 1 2 3 4 3 4 pipe fd[0] 1 4 3 fd[1] child 2 0

if (childpid == 0) { if (dup2(fd[1], STDOUT_FILENO) == -1) perror ("Failed to redirect stdout of ls"); else if ((close(fd[0]) == -1) || (close(fd[1]) == -1)) perror ("Failed to close extra file descriptors"); else { execl("/bin/ls","ls", "-l", NULL); perror("Failed to exec ls ..."); } return (1); } 0 child file descriptor table after dup2 call 1 2 parent standard in pipe write standard err 0 1 2 3 4 pipe 1 child 2 0

if(dup2(fd[0], STDIN_FILENO) == -1) perror("Failed to redirect stdin of sort..."); else if ((close(fd[0] == -1) || close(fd[1]) == -1)) perror("Failed to close extra file descriptors"); else { execl("/usr/bin/sort", "sort", "-n", "+4", NULL); perror("Failed to exec sort"); } return 1; } 1 2 parent 0 3 4 parent file descriptor table after dup2 call pipe pipe read standard out standard err pipe read pipe write 0 1 2 3 4 child 2 0 fd[0] 1 fd[1]

if(dup2(fd[0], STDIN_FILENO) == -1) perror("Failed to redirect stdin of sort..."); else if ((close(fd[0] == -1) || close(fd[1]) == -1)) perror("Failed to close extra file descriptors"); else { execl("/usr/bin/sort", "sort", "-n", "+4", NULL); perror("Failed to exec sort"); } return 1; } 1 2 parent 0 parent file descriptor table after dup2 call pipe pipe read standard out standard err 0 1 2 child 2 0 1

if (childpid == 0) { if (dup2(fd[1], STDOUT_FILENO) == -1) perror ("Failed to redirect stdout of ls"); else if ((close(fd[0]) == -1) || (close(fd[1]) == -1)) perror ("Failed to close extra file descriptors"); else { execl("/bin/ls","ls", "-l", NULL); perror("Failed to exec ls ..."); } return (1); } 0 child file descriptor table after dup2 call 1 2 parent standard in pipe write standard err 0 1 2 3 4 pipe 1 child ls writes to stdout 2 0

if(dup2(fd[0], STDIN_FILENO) == -1) perror("Failed to redirect stdin of sort..."); else if ((close(fd[0] == -1) || close(fd[1]) == -1)) perror("Failed to close extra file descriptors"); else { execl("/usr/bin/sort", "sort", "-n", "+4", NULL); perror("Failed to exec sort"); } return 1; } 1 2 parent 0 sort reads from stdin parent file descriptor table after dup2 call pipe pipe read standard out standard err 0 1 2 child 2 0 1

Named Pipes Also referred to as FIFOs A special type of file that exists in the file system Behaves like the un-named pipes we have been studying

mkfifo call #include <sys/types.h> #include <sys/stst.h> int mkfifo(const char *filename, mode_t mode); returns 0 if successful -1 if not permissions

#include <unistd.h> #include <stdlib.h> #include <stdio.h> #include <sys/types.h> #include <sys/stat.h> int main () { int res = mkfifo(“/tmp/myFifo”, 0777); if ( res == 0 ) printf(FIFO created …\n”); exit (0); } Test it out … This program creates the named pipe, myFifo

try: ls -l /tmp/myFifo we see that the named pipe is there. cat < /tmp/myFifo nothing happens, because there is no data in the pipe. The call blocks until some data appears echo “abracadabra” > /tmp/myFifo nothing happens, because there is no process reading the other end of the pipe. The call blocks. cat < /tmp/myFifo & echo “abracadabra” > /tmp/myFifo the cat call initially blocks because there is no data to read. When echo makes data available, the cat command reads the data and prints it. It then exits because the pipe will have been closed. Subsequent reads return a 0, indicating end of file.

a FIFO exists as a named file, not an open file descriptor, and so must be opened like a regular file. The main restriction is that you cannot open a FIFO for reading and writing (O_RDWR mode). If you want to pass data in two directions, use two FIFOs. Opening a FIFO O_NONBLOCK mode has a special meaning on FIFO files. There are four possible combinations of O_RDONLY, O_WRONLY and O_NONBLOCK:

open (const char *path, O_RDONLY); The open call will block until a process opens the same FIFO for writing. open (const char *path, O_RDONLY | O_NONBLOCK); The call will succeed and return immediately, even if the FIFO has not been opened for writing by any process. open (const char *path, O_WRONLY); The call will block until a process opens the same FIFO for reading. open (const char *path, O_WRONLY | NON_BLOCK); The call will return immediately. If no process has the FIFO open for reading open will return an erro, -1, and the FIFO won’t be opened.

Reading and Writing FIFOs defined in limits.h A read on an empty blocking FIFO will wait until some data is available to be read. Conversely, a read on a non-blocking FIFO with no data will return 0 bytes. A write on a blocking FIFO will wait until the data can be written. A write on a FIFO that can’t accept all of the bytes will either: Fail if the request is for PIPE_BUF bytes or less and the data can’t be written Write part of the data if the request is for more than PIPE_BUF bytes, returning the number of bytes written.

Atomic Writes If you have multiple processes writing to a named pipe, you need to guarantee that each write is “atomic”, or data from different processes may be interleaved. To ensure that a write is atomic, make all of your write requests to a blocking FIFO, with the number of bytes written each time less than PIPE_BUF.

Inter-process Communication