Update docs for X-LoRA and some bugfixes

[EricLBuehler]

Refs #1966. Adds some stuff to the docs, but the main focus is on the loading bugfix.

I ran:

ruff format
make quality
make style
pytest tests/test_xlora.py -v

With:

> ruff --version
ruff 0.5.7
> pytest --version
pytest 8.3.2

Looks like some tests are still failing, I'll take a look!

[review-notebook-app]

Check out this pull request on 

See visual diffs & provide feedback on Jupyter Notebooks.

---

Powered by ReviewNB

[HuggingFaceDocBuilderDev]

The docs for this PR live here. All of your documentation changes will be reflected on that endpoint. The docs are available until 30 days after the last update.

[BenjaminBossan]

Thanks for the updates, nice that you could solve the loading issue.

Regarding the notebook: I added some comments but I would suggest to leave it out of the PR until it's finished. Also, WDYT about adding all the X-LoRA examples into a single directory examples/xlora? This may require updating some links.

It would be also great if we had add a test to ensure that the loading functionality works. For reference, this is what I used to test it (based on your notebook):

import torch
from peft.tuners.xlora.config import XLoraConfig
from peft.utils.peft_types import PeftType, TaskType
from peft import get_peft_model
from safetensors.torch import load_file
from transformers import AutoModelForCausalLM

torch.manual_seed(123)

model_name= 'HuggingFaceH4/zephyr-7b-beta'
model = AutoModelForCausalLM.from_pretrained(model_name)
model.config.use_cache = False

adapters = [
    "lamm-mit/Zephyr_Bioinspired",
    "lamm-mit/Zephyr_CoT",
]
adapters = {str(i): file_name for i, file_name in enumerate(adapters)}

peft_config = XLoraConfig(
    task_type=TaskType.CAUSAL_LM,
    peft_type=PeftType.XLORA,
    hidden_size=model.config.hidden_size,
    adapters=adapters,
    xlora_depth=8,
    xlora_size=2048,
    layerwise_scalings=True,
    xlora_dropout_p=0.2,
)
model = get_peft_model(model, peft_config)

sd = load_file("examples/xlora_train/zephyr_bioinspired_adapter_model-fixed.safetensors")
w0 = model.base_model.model.model.layers[0].self_attn.q_proj.lora_A["0"].weight
w1 = sd['base_model.model.model.layers.0.self_attn.q_proj.lora_A.weight']

assert torch.allclose(w0, w1)

Of course, for a unit test, it should use a smaller model and ideally the adapter should be loaded from a HF-internal repo like https://huggingface.co/peft-internal-testing (LMK if you need access).

I think the only CI failures are from spurious network errors

Yes, sometimes there can be timeouts, leading to failed tests. If you spot that, feel free to re-start the failing tests.

[EricLBuehler]

Regarding the notebook: I added some comments but I would suggest to leave it out of the PR until it's finished. Also, WDYT about adding all the X-LoRA examples into a single directory examples/xlora? This may require updating some links.

I have added this, I agree it makes more sense.

It would be also great if we had add a test to ensure that the loading functionality works.
Of course, for a unit test, it should use a smaller model and ideally the adapter should be loaded from a HF-internal repo like https://huggingface.co/peft-internal-testing (LMK if you need access).

That sounds great. I'll create a dummy adapter on a smaller model, as we only made X-LoRA Zephyr and Gemma 2 7b models. To upload the adapter, if you could grant access to https://huggingface.co/peft-internal-testing it would be great! Afterward, I can create the unit test.

[BenjaminBossan]

LMK when this is ready for review.

[EricLBuehler]

I think this should be ready for review! I don't plan any more changes.

[BenjaminBossan]

Thanks for these fixes and the improved docs. LGTM.

[EricLBuehler]

Thanks!

[benjamin-marie]

The code in the documentation (https://huggingface.co/docs/peft/main/en/package_reference/xlora) is deprecated.
It calls prepare_model_for_int8_training and passes "use_flash_attention_2" when loading the model.

[EricLBuehler]

Hey @benjamin-marie! Thanks for pointing this out, I'll add a PR to fix this.