fix: lora+: include lr in optimizer kwargs

[kallewoof]

Playing with the LoRA+ optimizer, I realized that the underlying optimizer will complain about the required positional lr argument if it is not included in the optimizer kwargs.

In the original implementation, this was actually already the case, but somehow this fell out along the way.

[BenjaminBossan]

Thanks for fixing this issue. But isn't the better fix to add lr to the kwargs? I.e.

- lr = kwargs["lr"]
+ kwargs["lr"] = lr

Then we can leave lr in the function signature, which is more user friendly than raising a KeyError down the line.

[kallewoof]

Ah, to make it a forced argument to create_loraplus_optimizer. Yes.

[HuggingFaceDocBuilderDev]

The docs for this PR live here. All of your documentation changes will be reflected on that endpoint. The docs are available until 30 days after the last update.

[BenjaminBossan]

Thanks for fixing this oversight.