tune CUDA kernels automatically #206
Conversation
|
Trying this out on my real example, it seems like this adds quite a lot of overhead, so the optimal thread size should probably be determined at kernel creation time, not every time it is invoked. |
|
Turns out that this overhead came from having to switch from a static to a dynamic workgroupsize, so I am thinking that it might be better to have a separate API for determining the optimal workgroupsize for a kernel. |
|
Well, that was embarassing... Turns out the weird performance issues I was seeing came from me accidentally capping the CUDA memory limit way to low. 🤦 With that fixed, this does actually give a nice speedup even if I change to a dynamic workgroupsize. |
|
bors try |
Co-authored-by: Valentin Churavy <vchuravy@users.noreply.github.com>
simeonschaub
changed the title
simeonschaub
force-pushed
the
sds/autotune
branch
from
f051d95
to
f484b67
Compare
|
Ok, I think I am fairly happy with this now. If you agree with the way this works, it should be good to go from my side. |
|
bors r+ |
|
Mind also tagging a release with this? |
JuliaGPU/KernelAbstractions.jl#206 added the ability to automatically tune the workgroupsize of CUDA kernels. This PR stops hardcoding a default workgroupsize and lets KernelAbstractions handle that. This does change the workgroupsize from being statically sized to being dynamically sized, but in my testing, even with fairly small workgroupsizes, that didn't really make a difference.
This is still quite rough around the edges, but I am putting this up for feedback. This automatically splits up the threads over leading dimensions of the ndrange for better performance if the first dimension is small.