New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

NVIDIA DALI pipelines to facilitate direct loading of data into GPU memory. #54

Open

negin513 opened this issue Jul 15, 2024 · 0 comments

Contributor

negin513 commented Jul 15, 2024

I have recently learned about NVIDIA DALI (Data Loading Library) that can be used for streaming data directly.

This can be potentially used for data streaming of our files directly to GPUs to avoid any CPU-GPU data transfer latency.

This discussion in Xarray Repository is relevant: pydata/xarray#9249

@weiji14 suggested that this can be integrated as a dedicated BackendEntrypoint as an alternative to the kvikio engine and I like this idea.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment