C/C++ Implementation of UNet2DConditionModel

[btakita]

I'm seeing a performance issue (~30-120ms creating a numpy Array totaling 4096 bytes) & high CPU usage from UNet2DConditionModel. Am I doing something wrong or is there a C/C++ implementation for this class? I have not been able to find one. If a fast implementation does not exist, do you have a bounty program? Thanks!

    def load_model_unet(
        self, latent_batch_model: Tensor, pe_batch: Tensor
    ) -> UNet2DConditionModel:
        return self.unet.model(
            latent_batch_model,
            self.timesteps,
            encoder_hidden_states=pe_batch,
        )

[asomoza]

If you're not using the GPU shouldn't the high cpu usage expected?

There's a library for using stable diffusion and Flux with c++ called stable-diffusion.cpp but it's not diffusers based so it's not the same class like you're asking and we don't have an official diffusers clone in c++.

We don't have a bounty program, if you find the cause of the issue you're experimenting, you can just open an issue with it and if it's really a problem with the library we will fix it or if we don't have the bandwidth at the moment, we will open it to the community if they want to help and fix it faster.

[btakita]

I am using the GPU. For video streaming, it seems that there is high CPU usage. The GPU operations in the pipeline are fast but UNet20ConditionModel has a bunch of python with iterations to denoise. So I suspect that this is the cause. Note that it's 30-120ms, so I can see why it wouldn't be a problem in most cases such as batch processing. But for video streaming with realtime constraints, it starts to be an issue.

Thank you for the info on stable-diffusion.cpp & the process to open an issue. I'll go ahead & do so.

[asomoza]

oh I understand now, I would love to see a demo of it when you have a PoC. ccing @a-r-r-o-w because I think I saw something related in another issue.