How to count number of activated parameters?

[daeunni]

This is just a simple question. (I am new to here 🥲)
I am using train_dreambooth_lora.py, how can I count the number of activated parameters?

[a-r-r-o-w]

I haven't tested but to count number of trainable parameters for which gradients are activated, we can do something like:

num_trainable = sum([param.numel() for param in model.parameters() if param.requires_grad])

You can add this after the weights of vae, text_encoder and unet are frozen (see this).

[daeunni]

Thank you so much. Really helpful!