mpi communications l.
Skip this Video
Loading SlideShow in 5 Seconds..
MPI Communications PowerPoint Presentation
Download Presentation
MPI Communications

Loading in 2 Seconds...

play fullscreen
1 / 22

MPI Communications - PowerPoint PPT Presentation

  • Uploaded on

MPI Communications. Point to Point Collective Communication Data Packaging. Point-to-Point Communication Send and Receive. MPI_Send/MPI_Recv provide point-to-point communication synchronization protocol is not fully specified. what are possibilities?. Send and Receive Synchronization.

I am the owner, or an agent authorized to act on behalf of the owner, of the copyrighted work described.
Download Presentation

PowerPoint Slideshow about 'MPI Communications' - valora

An Image/Link below is provided (as is) to download presentation

Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author.While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server.

- - - - - - - - - - - - - - - - - - - - - - - - - - E N D - - - - - - - - - - - - - - - - - - - - - - - - - -
Presentation Transcript
mpi communications

MPI Communications

Point to Point

Collective Communication

Data Packaging

point to point communication send and receive
Point-to-Point CommunicationSend and Receive
  • MPI_Send/MPI_Recv provide point-to-point communication
    • synchronization protocol is not fully specified.
      • what are possibilities?
send and receive synchronization
Send and Receive Synchronization
  • Fully Synchronized (Rendezvous)
    • Send and Receive complete simultaneously
      • whichever code reaches the Send/Receive first waits
    • provides synchronization point (up to network delays)
  • Buffered
    • Receive must wait until message is received
    • Send completes when message is moved to buffer clearing memory of message for reuse
send and receive synchronization4
Send and Receive Synchronization
  • Asynchronous
    • Sending process may proceed immediately
      • does not need to wait until message is copied to buffer
      • must check for completion before using message memory
    • Receiving process may proceed immediately
      • will not have message to use until it is received
      • must check for completion before using message
mpi send and receive
MPI Send and Receive
  • MPI_Send/MPI_Recv are synchronous, but buffering is unspecified
    • MPI_Recv suspends until message is received
    • MPI_Send may be fully synchronous or may be buffered
      • implementation dependent
  • Variations allow synchronous or buffering to be specified
asynchronous send and receive
Asynchronous Send and Receive
  • MPI_Isend() / MPI_Irecv() are non-blocking. Control returns to program after call is made.
  • Syntax is the same as for Send and Recv, except a MPI_Request* parameter is added to Isend and replaces the MPI_Status* for receive.
detecting completion
Detecting Completion
  • MPI_Wait(&request, &status)
    • requestmatches request on Isend or Irecv
    • status returns status equivalent to

status for Recv when complete

    • Blocks for send until message is buffered or sent so message variable is free
    • Blocks for receive until message is received and ready
detecting completion8
Detecting Completion
  • MPI_Test(&request, flag, &status)
    • request,status as for MPI_Wait
    • does not block
    • flag indicates whether message is sent/received
    • enables code which can repeatedly check for communication completion
collective communications
Collective Communications
  • One to Many (Broadcast, Scatter)
  • Many to One (Reduce, Gather)
  • Many to Many (All Reduce, Allgather)
  • A selected processor sends to all other processors in the communicator
  • Any type of message can be sent
  • Size of message should be known by all (it could be broadcast first)
  • Can be optimized within system for any given architecture
mpi bcast syntax
MPI_Bcast() Syntax

MPI_Bcast(mess, count, MPI_INT,


mess pointer to message buffer

count number of items sent

MPI_INT type of item sent

Note: count and type should be the same

on all processors

root sending processor

MPI_COMM_WORLD communicator within which

broadcast takes place

mpi barrier


MPI_COMM_WORLD communicator within which

broadcast takes place

provides for barrier synchronization without message of broadcast

  • All Processors send to a single processor, the reverse of broadcast
  • Information must be combined at receiver
  • Several combining functions available
mpi reduce syntax
MPI_Reduce() syntax

MPI_Reduce(&dataIn, &result, count,



dataIn data sent from each processor

resultstores result of combining operation

countnumber of items in each of dataIn, result

MPI_DOUBLEdata type for dataIn, result

MPI_SUMcombining operation

rootrank of processor receiving data


mpi reduce
  • Data and result may be arrays -- combining operation applied element-by-element
  • Illegal to alias dataIn and result
    • avoids large overhead in function definition
mpi scatter
  • Spreads array to all processors
  • Source is an array on the sending processor
  • Each receiver, including sender, gets a piece of the array corresponding to their rank in the communicator
mpi gather
  • Opposite of Scatter
  • Values on all processors (in the communicator) are collected into an array on the receiver
  • Array locations correspond to ranks of processors
many to many communications
Many to Many Communications
  • MPI_Allreduce
    • Syntax like reduce, except no root parameter
    • All nodes get result
  • MPI_Allgather
    • Syntax like gather, except no root parameter
    • All nodes get resulting array
  • Underneath -- virtual butterfly network
data packaging
Data packaging
  • Needed to combine irregular, non-contiguous data into single message
  • pack -- unpack, explicitly pack data into a buffer, send, unpack data from buffer
  • Derived data types, MPI heterogeneous data types which can be sent as a message
mpi pack syntax
MPI_Pack() syntax

MPI_Pack(Aptr, count, MPI_DOUBLE,

buffer, size, &pos, MPI_COMM_WORLD);

Aptr pointer to data to pack

countnumber of items to pack

type of items

bufferbuffer being packed

sizesize of buffer (in bytes)

posposition in buffer (in bytes), updated


mpi unpack
  • reverses operation of MPI_Pack()

MPI_Unpack(buffer, size, &pos, Aptr, count, MPI_DOUBLE, MPI_COMM_WORLD);