Decrease memory usage of merge_and_unload

[snarayan21]

Addresses #1939

[snarayan21]

@BenjaminBossan Would be great to get your approval here!

[BenjaminBossan]

I checked the PEFT code on why we merge this way. Turns out we used to do it in-place but this PR changed it: #1372. IMO merging should not affect the error reported there, so I think we can undo the change, but I wanted to provide the necessary context.

Could you please also update the other merge functions?

peft/src/peft/tuners/lora/layer.py

Line 671 in ba75bb1

base_layer.weight.data = base_layer.weight.data + self.get_delta_weight(active_adapter)

peft/src/peft/tuners/lora/layer.py

Line 927 in ba75bb1

base_layer.weight.data = base_layer.weight.data + delta_weight

Thanks.

I plan to make a release soon, aiming for tomorrow. Let's get this fix ready before the release.

[snarayan21]

@BenjaminBossan thanks for the pointers, found some additional places where this change is also needed.
And yeah since #1372 was a training time error making this fix for merging adapters shouldn't be applicable.

Hope this looks good to you.

[snarayan21]

A release tomorrow with this fix would be ideal.

[HuggingFaceDocBuilderDev]

The docs for this PR live here. All of your documentation changes will be reflected on that endpoint. The docs are available until 30 days after the last update.

[BenjaminBossan]

Strange that Windows CI is consistently failing. Hard to imagine that this is caused by the PR but I'm investigating further (but don't have a Windows machine myself). Hopefully this is just a temporary issue.

[snarayan21]

Yeah quite weird:

The directory name is invalid: 'C:\\Users\\RUNNER~1\\AppData\\Local\\Temp\\tmpv612vppk\\model.safetensors'

[BenjaminBossan]

Okay, so the same Windows errors as in #1947, which has no code changes.

[BenjaminBossan]

Thanks for restoring the more memory efficient merging behavior. A release tomorrow should be possible.