vae.encode in AutoencoderKLCogVideoX consumes too much GPU memory

[peki12345]

vae.encode in AutoencoderKLCogVideoX consumes too much GPU memory, I don't know exactly how much it will consume, but my 24GB GPU cannot run successfully. I want to know if there is a way to optimize this?
from diffusers.models import AutoencoderKLCogVideoX
vae = AutoencoderKLCogVideoX.from_pretrained("THUDM/CogVideoX-2b", subfolder="vae", torch_dtype=torch.float16).to("cuda")
frames = torch.rand(torch.Size([1, 49, 3, 720, 480])).half().cuda()
frames = frames.permute(0, 2, 1, 3, 4)
frames = vae.scaling_factor * frames
latents = vae.encode(frames).sample
I get "CUDA out of memory" using the above code.

[peki12345]

I just couldn't execute this code even with 80GB of GPU memory, so I think this may be a problem with my code? So how can I use this 3D VAE to encode my video?

[a-r-r-o-w]

Working on adding tiled encoding for lora training here btw: #9302. Without it, memory consumption is indeed too high

[tolgacangoz]

It is said that ~34 GB is needed for encoding. Could you try with with torch.no_grad():?
Additionally, encoding could be tiled as done for decoding.

[peki12345]

It is said that ~34 GB is needed for encoding. Could you try with with torch.no_grad():? Additionally, encoding can be tiled as done for decoding.

Thank you very much, I overlooked 'with torch. no_grad():', now the GPU memory consumption is normal (34GB). I will try the tiled strategy.