The Second-Generation ADI
for the MPICH Implementation
of MPIWilliam Gropp
Ewing Lusknumberdate

In this paper we describe an abstract device interface (ADI) that may be used to efficiently implement the Message Passing Interface (MPI). After experience with a first-generation ADI that made certain assumptions about the devices and tradeoffs in the design, it has become clear that, particularly on systems with low-latency communication, the first-generation ADI design imposes too much additional latency. In addition, the first-generation design is awkward for heterogeneous systems, complex for noncontiguous messaging, and inadequate at error handling. The design in this note describes a new ADI that provides lower latency in common cases and is still easy to implement, while retaining many opportunities for customization to any advanced capabilities that the underlying hardware may support.


Contents
  • Introduction
  • Principles
  • The Contiguous Routines
  • Sending
  • Buffered Sends
  • Receiving
  • Summary of Blocking Routines
  • Nonblocking Routines
  • Summary of Nonblocking Routines
  • Message Completion
  • Freeing requests
  • Shared Data Structures
  • Noncontiguous Operations
  • Pack and Unpack
  • Implementation of Noncontiguous Operations
  • Other Functions
  • Queue Information
  • Miscellaneous Functions and Values
  • Context Management
  • Support for MPI Attributes
  • Collective Routines
  • Changes from the First-Generation ADI
  • Special Considerations
  • Structure of MPI_Request
  • Freed Requests
  • Thread Safety
  • Cancelling a Message
  • Error Handling
  • Heterogeneous Support
  • Examples
  • ADI Bindings
  • Acknowledgments
  • Bibliography