GPU is not being used properly in the latest version 20240930

[muta1725]

RTX4060
Python 3.10.11
CUDA Toolkit 12.1
Torch 2.5.0+cu121

Version 20240927 works fine, but version 20240930 is very slow to process even tiny models.
Normal when processed using the CPU, but this problem occurs when using the GPU.

Specifically, processing a 120-second audio file takes 160 seconds with any model.

There are no errors, so I’m not sure what the cause could be. Is anyone else experiencing a similar issue? Any help would be appreciated.

Normally, the GPU utilization stays at 100% during execution.

[glangford]

One possibility is the introduction of SDPA in 20240930, I see this torch issue raised here for example

• Severe SDPA Performance Regression 2.5.0-RC1 pytorch/pytorch#135778

SDPA commit

• 27f9713

..and that CuDNN backend for SDPA is listed as Beta. Maybe test with torch < 2.5?

[muta1725]

I didn't know that such an issue had been raised.
I lowered the torch version to 2.4.1 to try it out and it stabilized.
Thank you very much.