Protocols

The MPI standard is designed to give an MPI implementation freedom in chosing exactly how and when data is moved from one process to another.

In this implementation, the information is divided into two parts: the message envelope, containing information about the message (such as tag, communicator, and length), and the data. The envelope is relatively small, and is delivered to the destination immediately.See the comments on resource restrictions later in this paper. The data may or may not be delivered at once. In fact, there are four possibilities that we will allow for in our implemenation:

Short

The data is delivered within the message envelope.

Eager

The data is delivered without waiting for the receiver to request it

Rendezvous

The data is not delivered until the receiver requests it

Get

The data is copied by the receiver using shared memory or remote memory operations.

Within the eager and rendezvous protocols, the method by which the data is delivered once requested may be blocking or nonblocking. In a nonblocking transfer, once the data is requested, the sender can call a system service routine to begin the transfer and then return to the user without waiting for the transfer to complete (until a later wait call). This is similar to the MPI nonblocking operations (MPI_Isend() MPI_Wait()).

No single choice of protocol is optimal. In many applications, a combination of these protocols is appropriate. For example, for messages of intermediate length, the eager protocol may offer the best combination of performance and reliability, while for very long data, only the rendezvous (or get) protocols maintain correctness. The ADI implementation design here supports using multiple protocols.

To simplify the implementation, only contiguous data areas are handled. The more general MPI datatypes are handled by packing and unpacking data into contiguous buffers, as described in [(ref nextgen_adi)].