Does iouring do DMA?

[MartyMcFlyInTheSky]

Forgive my naive terminology, I'm quite new to the topic:

So I worked with iouring a little bit, but I always wondered what optimizations are in place to make DMA transfers, either from other hosts (RDMA) or from disk to memory? Will iouring make use of potential DMA capabilities if I e.g. ask the interface to offload some disk data to a specific virtual memory location in userspace?

I hope this even makes sense.

[axboe]

Not sure I follow the question at all... For normal read/write operation, whether DMA is used or not is entirely up to the subsystem that is hosting the IO. Let's say it's a read of a file on an nvme device, yes DMA is certainly used, as PIO would be very slow. But io_uring doesn't know or care, it simply asks for "please put data from location X into memory Y". For normal buffered IO, Y is a page cache location, kernel memory. For O_DIRECT IO, Y is a userspace address, so the device generally DMA's directly into the userspace page.

[MartyMcFlyInTheSky]

Thanks for your answer @axboe . So am I understanding correctly all I can do from the userspace side with the ioruing api is specify O_DIRECT and let the kernel do the inference on whether DMA is possible or not? I thought there were mechanisms in the hand of a programmer that you can use to initiate DMA transfers directly. I'm just trying to figure out if I would need to step down into the kernel somehow to enable DMA or ioruing does that for me. Also maybe I should mention that I'm very interested in the RDMA capabilities of iouring as well. If you say that ioruing does not care about those intrinsics, who does? Will the kernel make sure that RDMA is used whenever possible (e.g. given a supported network card, bus etc..)? This would be the best case for me. But maybe RDMA is a whole different can of worms?

To give some context: I would like to give our database the capability to retrieve data from a storage backend regardless if the storage backend is a local disk or another machine entirely. If I could use ioruing for both (DMA disk reads, RDMA reads from other network nodes) that would significantly simplify everything for me.

[MartyMcFlyInTheSky]

I'm still interested in this for our application. We really would like to enable RDMA for exchange of big binary data over ethernet(RoCE)/infiniband, but it seems that at least for InfiniBand one needs to use a specialized library (libibverbs) to induce the connection. From the surface it looks thought like something the liburing or its kernel-side implementation might be able to abstract that away, so my question is: does it?