support Cambricon MLUs device

[huismiling]

Currently,Transformers and Accelerate have supported cambricon mlu (huggingface/transformers#29627, huggingface/accelerate#2552).

This PR enables users to leverage the cambricon mlu for training and inference of peft models.

[younesbelkada]

Hi @huismiling
Thanks so much for adding this ! I left one comment with respect to backward compatibility, what do you think?

[huismiling]

@BenjaminBossan @younesbelkada Thanks for your work. Will this be Ok ?

[HuggingFaceDocBuilderDev]

The docs for this PR live here. All of your documentation changes will be reflected on that endpoint. The docs are available until 30 days after the last update.

[BenjaminBossan]

@huismiling Could you please run make style on your PR?

[huismiling]

@BenjaminBossan make style is done.

[BenjaminBossan]

LGTM, thanks. @younesbelkada feel free to merge if you also approve.

Out of curiosity @huismiling, did you manage to run PEFT successfully on Cambricon MLUs?

[younesbelkada]

Thanks for adding the support for Cambricon MLU devices in PEFT !

[huismiling]

@BenjaminBossan Currently, I work for Cambricon. It is pleasure to @me for MLUs issues.

[BenjaminBossan]

Currently, I work for Cambricon.

So you could run the test suite successfully on MLUs?

[huismiling]

yeah, I ran pytest tests in peft dir. This is the results.
= 88 failed, 4284 passed, 1479 skipped, 1 xfailed, 2980 warnings in 6218.07s (1:43:38) =

There are some assertion errors. Most of errors is like the following. This difference should be acceptable.
E + where False = <built-in method allclose of type object at 0x7f3752f76840>(tensor([[-3.0688e+00, -4.7594e-02],\n [-1.2797e+01, -2.7268e-06]], device='mlu:0'), tensor([[-3.0634e+00, -4.7853e-02],\n [-1.2790e+01, -2.7268e-06]], device='mlu:0',\n grad_fn=<LogSoftmaxBackward0>), atol=1e-05, rtol=1e-05)

[BenjaminBossan]

Okay, so it's mostly about tolerances. If you manage to find more acceptable tolerance values, feel free to open a PR. Ideally, we would have all tests passing, so that we can more easily detect regressions.