FIX Multiple adapters and modules_to_save #1615
Merged
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Resolves #1574.
Previously, we had the bug that if we had multiple adapters, some with
modules_to_save
and others without, when trying to switch to an adapterwithout
modules_to_save
, theModulesToSaveWrapper
would raise an errorbecause it cannot find that adapter. Now, when it detects this, it is
just disabled (so it uses the original weight).
Moreover, we had the issue that when we were using classes such as
PeftModelForSequenceClassification
, we implicitly added the classifierlayers to
model.modules_to_save
. However, this would only add a newModulesToSaveWrapper
instance for the first adapter being initialized.When initializing a 2nd adapter via
add_adapter
, this information wasignored. To fix this, I now update the
peft_config.modules_to_save
toexplicitly add the classifier layers. This is a departure from how this
worked previously, but I'm couldn't find a better way to ensure that
this bug was fixed (LMK if you have a suggestion).
Finally, there was a bug in
add_weighted_adapters
when we were mergingmultiple adapters with
modules_to_save
. Previously, when we calledmodel.add_weighted_adapter
, the LoRA weights were merged and a newModulesToSaveWrapper
was added for the new adapter based on the firstLoraConfig
of the two adapters. ThisModulesToSaveWrapper
is just a copyof the original weights. Thus, when we switch to the newly merged
adapter, we just use the original weights for
modules_to_save
. Thisdoesn't make a lot of sense and is probably surprising for the user.
Now, we raise an error when we detect this to alert the user to this
fact.
Note that when only one of the adapters to be merged has a
modules_to_save
, this does not raise an error, instead that module isbeing used.
Edit: If this is merged, we should add a note to the next release notes
about possible changes, as this results in a different model for the same
config (though it's a rather specific edge case). We should clarify that the
previous behavior was erroneous and new one is correct.