A100 Performance and comments #921
Replies: 1 comment 1 reply
-
|
Still unable to scale the frames with mass (30+) vCPU and 100gb GPU. Not sure where the limitations are... |
Beta Was this translation helpful? Give feedback.
-
|
it's the onnx runtime? i think it has its limits. and doesnt scale like that. because of the io and swaps between cpu and gpu all the time. try running without masking, without any post proces and on the 128pixel first, does that scale? things you could try now Windows users: Make sure that "Hardware-accelerated GPU scheduling" is enabled in your Windows Graphics settings. resource reading material to understand the arcitecture and why it's probably limited https://onnxruntime.ai/docs/performance/device-tensor.html |
Beta Was this translation helpful? Give feedback.
-
Dear RoopUnleashed lovers.
All my props to the author. What a legend.
just wanted to let you know that TensorDock has Windows servers and GPUs... im currently running on an A100 80GB lol. Just to test...
As I notice, the bottleneck becomes the CPUs at this level of compute.
I'm still getting nominal frames 5 to 30 with post GPEN on an A100, 256bit.
Any wise words or tips how to skew the workload to GPU vs CPU?
fyi - @C0untFloyd Thank you for all your work! It's honestly the best solution out there.
Just to mention - ffmpeg doesnt auto install, one must rename the folder it downloads that includes I believe versioning or something.. Once you rename the folder to simply "ffmpeg" things can proceed.
Beta Was this translation helpful? Give feedback.
All reactions