Any plans to port to latest Horovod (0.24) ?

[ASDen]

First thanks for the great effort here, Horovod really needs a good gradient compression library! Today's cloud environments really needs this and this is why the official DDP started also supporting gradient compression algorithms....FP16 is not enough anymore

However, I can see only an old version is supported, any plans / roadmap to port to newer versions ?

Thanks,

[hangxu0304]

Hi. Generally, we only plan to support the major versions of Horovod. Is there any specific feature or compatibility issue in Horovod 0.24?

[ASDen]

yeah it just doesn't work with 0.24 :)
Most of this is because a lot has changed in torch/optimizer.py in horovod, and currently grace just copies a modified old version of it, which doesn't work

[hangxu0304]

Thanks for letting us know. We are working on a more elegant way to integrate with Horovod. Hopefully, it will ease the compatibility issue. Stay tuned.