1 / 5

Rethinking Backward Compatibility in MPI Library: Introducing MPI_Count for Handling Large Counts

The Backward Compatibility Working Group debates the implementation of MPI_Count to address counts exceeding 2^32, sparking mixed sentiments. Users question the necessity and implications on existing MPI APIs due to potential incompatibilities with 32/64-bit libraries and performance trade-offs. Various solutions are proposed, including introducing new functions, updating existing ones, or relying on C auto-promotion. The community struggles to reach a consensus, with concerns about confusion, backward compatibility, and future-proofing the MPI library.

neil
Download Presentation

Rethinking Backward Compatibility in MPI Library: Introducing MPI_Count for Handling Large Counts

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. Backward Compatibility WG

  2. The Big Issue:Counts Larger Than 232 • Counts are expressed as “int” / “INTEGER” • Usually limited to 232 • Propose a new type: MPI_Count • Can be larger than an int / INTEGER • “Mixed sentiments” within the Forum • Is it useful? Or just lazy? MPI_SEND(void *buf, int count, …) MPI_SEND(void *buf, MPI_Count count, …)

  3. Do we need MPI_Count? YES NO Very few users “Syntactic sugar” for making a datatype Affects many, many MPI API functions Could just fix MPI_GET_COUNT Potential incompatibilities (32 and 64 bit MPI libraries) • Some users have asked for it • Trivially send large msgs. • No need to make a datatype • POSIX went to size_t • Why not MPI? • Think about the future: • Bigger RAM makes 232 relevant • Coalescing off-node msgs. • More cores, larger MPI jobs

  4. Solutions proposed • Do nothing • Tell users “use datatypes” • New, duplicate functions • E.g., MPI_SEND_LARGE • Use MPI_Count only for new MPI-3 routines • Change C bindings • Rely on C auto-promotion • Only fix MPI IO functions • Where MPI_BYTE is used ✖ ✖ ✖ ✖ ✔ Ouch!! Confusing to users Not forward-looking Bad for Fortran Might be ok…?

  5. Solutions proposed Forum has hated every proposal What about sizes, tags, etc.? • Only fix “query” functions • E.g., MPI_GET_COUNT_LONG • Create a system for API versioning • Update all functions to use MPI_Count • Update all functions to use MPI_Count, MPI_Tag, MPI_Size, … ✖ ✖ ✔ ✖ Might be ok…? Ouch!!

More Related