Inter-Process Communication

Inter-Process Communication Vivek Pai / Kai Li Princeton University

Mechanics • Last few slides from last time lost for good • Reading assignment – OSDI 3.0-3.2 • Read ahead? 4.0, 4.1, 4.3, 4.6.3 • Gedankenexperiment first • Message-passing next

The select( ) Function • int select(int nfds, fd_set *readfds, fd_set *writefds, fd_set *errorfds, struct timeval *timeout) • fd_set is a bit vector – bit position is file descriptor # • Most of these parameters are input/output

What Does select( ) Give You? The select( ) function indicates which of the specified file descriptors is ready for reading, ready for writing, or has an error condition pending. If the specified condition is false for all of the specified file descriptors, select( ) blocks, up to the specified timeout interval, until the specified condition is true for at least one of the specified file descriptors.

Note the Legalese/Precision The select( ) function indicates which of the specified file descriptors is ready for reading, ready for writing, or has an error condition pending. If the specified condition is false for all of the specified file descriptors, select() blocks, up to the specified timeout interval, until the specified condition is true for at least one of the specified file descriptors.

Here’s An Important Clue The select( ) function indicates which of the specified file descriptors is ready for reading, ready for writing, or has an error condition pending. If the specified condition is false for all of the specified file descriptors, select() blocks, up to the specified timeout interval, until the specified condition is true for at least one of the specified file descriptors.

What Does This Imply? Output bit it set if input bit was set AND action can be performed If input bit was set AND action can be performed what is output bit? When is this condition evaluated?

A Closer Look at Reading for i = 0; i < nfds; i++ if (input bit not set) continue; if (file descriptor doesn’t exist) return error; output bit = TestReadable( i ); Where does interrupt enable/disable go?

for i = 0; i < nfds; i++ if (input bit not set) continue; if (fd doesn’t exist) return error; outbit = TestReadable(i); if (outbit) anyset= TRUE; end for if (anyset) return answer if (timeout specified) set timer condition_wait(selectflag) if (timeout) return nothing go back to for… A Closer Look at Reading

And Now, Back to Our Regularly-Scheduled Program…

What Does select( ) Teach Us? • You cannot disable interrupts for long • Exact answers may be difficult • May even be practically impossible • Flexible guarantees are helpful in this regard • Seemingly straightforward function is complex • Programmer needs to be aware of behavior • Looping on select( ) eliminates “deficiency”

Big Picture Sender Receiver Process Process

Message Passing API • Generic API • send( dest, msg ), recv( src, msg ) • What should the “dest” and “src” be? • pid • file: e.g. a pipe • port: network address, pid, etc • no src: receive any message • src combines both specific and any • What should “msg” be? • Need both buffer and size for a variable sized message

Issues • Asynchronous or synchronous? • How are links established? • Can a link be associated with more than two processes? • How many links can there be between any pair? • What is the size of a message? • Is a link unidirectional or bidirectional? • Direct vs. indirect • Exceptions such as message losses

Send • Synchronous • Will not return until data is out of its source memory • Block on full buffer • Asynchronous • Return as soon as initiating its hardware • Completion • Require applications to check status • Notify or signal the application • Block on full buffer

Receive • Asynchronous • Return data if there is a message • Return status if there is no message (probe) • Synchronous • Return data if there is a message • Block on empty buffer

Handler Interface • Handler( src, msg, func ) • Execute upon a message arrival • Which one is more powerful? • Recv with a thread can emulate a Handler • Handler can be used to emulate Recv by using Monitor • Pros and Cons • Handlers are better for event-based applications (no need to think about threads), but concurrent execution requires more thought • Recv with threads require thread context switches but can run concurrently

Buffering • No buffering • Sender must wait until the receiver receives the message • Rendezvous on each message • Bounded buffer • Finite size • Sender blocks when the buffer is full • Use mesa-monitor to solve the problem • Unbounded buffer • “Infinite” size • Sender never blocks

A single buffer at the receiver More than one process may send messages to the receiver To receive from a specific sender, it requires searching through the whole buffer A buffer at each sender A sender may send messages to multiple receivers To get a message, it also requires searching through the whole buffer Direct Communication … …

Indirect Communication • Use a “mailbox” to allow many-to-many communication • Requires open/close a mailbox before using it • Where should the buffer be? • A buffer and its mutex/ conditions should be at the mailbox • Fixed sized messages? • Not necessarily - can break a large message into packets • Are there any differences between a mailbox and a pipe? • A mailbox allows many to many communication • A pipe implies one sender and one receiver

Using Message-Passing • What is message-passing for? • Communication across address spaces • Communication across protection domains • Synchronization • Use a mailbox to communicate between a process/thread and an interrupt handler: fake a sender Receive Keyboard mbox

Producer-Consumer Problem with Message Passing Producer(){ while (1) { … produce item … send( consumer, item); } } Consumer(){ while (1) { recv( producer, item ); … consume item … } }

Exception: Process Termination • P waits for a message from Q, but Q has terminated • Problem: P may be blocked forever • Solution: • P checks once a while • Catch the exception and informs P • Send ack message • P sends a message to Q, but Q has terminated • Problem: P has no buffer and will be blocked forever • Solution: Check Q’s state and cleanup?

Exception: Lost Messages • Detection • Acknowledge each message sent • Timeout on the sender side • Retransmission • Sequence number for each message • Process pair for a channel • Physically link level • Retransmit a message on timeout • Retransmit a message on out-of-sequence acknowledgement • Remove duplication messages on the receiver side • Is NoAck a good idea?

Ever Wonder How The Web Works? • Your browser (client) is on one machine • The server is somewhere else far away • Far implies a “long” time to communicate • When you send a message, where does it go? • To the server, of course • But how ?

Inter-Process Communication