Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Missing UCX RMA support for 8/16bit MPI atomics #86

Open
lundgren87 opened this issue Apr 12, 2022 · 0 comments
Open

Missing UCX RMA support for 8/16bit MPI atomics #86

lundgren87 opened this issue Apr 12, 2022 · 0 comments

Comments

@lundgren87
Copy link
Member

lundgren87 commented Apr 12, 2022

The ArgoDSM atomics currently make use of atomic MPI operations of size 8, 16, 32 and 64 bits. The choice of MPI datatype is made through the below function or its unsigned or float equivalent.

/**
* @brief Returns an MPI integer type that exactly matches in size the argument given
*
* @param size The size of the datatype to be returned
* @return An MPI datatype with MPI_Type_size == size
*/
static MPI_Datatype fitting_mpi_int(std::size_t size) {
MPI_Datatype t_type;
using namespace argo;
switch (size) {
case 1:
t_type = MPI_INT8_T;
break;
case 2:
t_type = MPI_INT16_T;
break;
case 4:
t_type = MPI_INT32_T;
break;
case 8:
t_type = MPI_INT64_T;
break;
default:
throw std::invalid_argument(
"Invalid size (must be either 1, 2, 4 or 8)");
break;
}
return t_type;
}

OpenMPI (and most definitely MPICH as it is bundled) nowdays pushes Infiniband users towards using UCX. However, UCX does not provide full RMA support for 8 and 16 bit atomics, instead falling back to active messaging for these (if supported at all).
UCX Documentation
Related issue

Some of the ArgoDSM backend tests (atomicXchgAll, atomicXchgOne) currently fail with "unsupported datatype" when forcing the selection of the UCX osc module or when other alternatives are disabled. For both performance (avoiding active communication) and compatibility reasons, perhaps it would be better to perform at least a properly aligned 32-bit atomic operation instead of 8/16?

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

1 participant