TPU Example / Support

[ryanwongsa]

❓ Questions/Help/Support

I would like to use Ignite with TPUs and was wondering if there are any examples currently available which currently use TPUs with Ignite or does Ignite have some limitations which would not allow for easy compatibility with TPUs.

[vfdev-5]

@ryanwongsa unfortunately, there is no yet examples of using Ignite with TPUs.

I think there is no limitations on that as PyTorch xla API is rather similar to GPUs. But, yes, it will be good idea to add some example with Colab link.

[vfdev-5]

@ryanwongsa please, checkout added example or notebook in Colab and feel free to give us any feedback.

[ryanwongsa]

Thanks.

I am not sure if it is suitable for including in an example notebook but there are currently Kaggle kernels which use TPU training on 8 cores, such as bert-multi-lingual-tpu-training-8-cores and I was wondering what the best approach would be to do it in a similar way with Ignite.

The torch_xla ParallelLoader is also used on top of a Dataloader which might cause issues too but I am not sure.

[vfdev-5]

@ryanwongsa let me check this kernel with Ignite.