Add DeBERTa model

[w5688414]

PR types

New features

• deepset/deberta-v3-large-squad2
• microsoft/deberta-v2-xlarge
• microsoft/deberta-v3-base
• microsoft/deberta-v3-large
• microsoft/deberta-base

PR changes

Description

borrow from previous PR: #5414

import numpy as np
import paddle
import torch
from paddlenlp.transformers import DebertaV2Tokenizer

def test_precision(model_name):
    pp_model = PaddleDebertaModel.from_pretrained(model_name)
    # pp_model = PaddleDebertaModel.from_pretrained(model_name.split('/')[-1])
    

    hf_model = HuggingfaceModel.from_pretrained(model_name)
    input_ids = np.random.randint(1, 1000, size=(2, 10))
    pp_inputs = paddle.to_tensor(input_ids)
    hf_inputs = torch.tensor(input_ids)
    pp_model.eval()
    hf_model.eval()
    with paddle.no_grad():
        pp_output = pp_model(pp_inputs, output_hidden_states=True, return_dict=True)
    with torch.no_grad():
        hf_output = hf_model(hf_inputs, output_hidden_states=True)

    if "start_logits" in hf_output.keys():
        for key in ['start_logits', 'end_logits']:
            diff = abs(hf_output[key].detach().numpy() - pp_output[key].numpy())
            print(f"{key} max diff: {np.max(diff)}, min diff: {np.min(diff)}")

    for i in range(pp_model.config.num_hidden_layers + 1):
        diff = abs(hf_output["hidden_states"][i].detach().numpy() - pp_output["hidden_states"][i].numpy())

        print(f"layer {i} max diff: {np.max(diff)}, min diff: {np.min(diff)}")


from transformers import AutoModelForQuestionAnswering as HuggingfaceModel
from paddlenlp.transformers import DebertaV2ForQuestionAnswering as PaddleDebertaModel
model_name = "deepset/deberta-v3-large-squad2"
test_precision(model_name)

output is:

start_logits max diff: 5.0067901611328125e-06, min diff: 1.862645149230957e-08
end_logits max diff: 3.3080577850341797e-06, min diff: 8.940696716308594e-08
layer 0 max diff: 9.5367431640625e-07, min diff: 0.0
layer 1 max diff: 2.86102294921875e-06, min diff: 0.0
layer 2 max diff: 4.291534423828125e-06, min diff: 0.0
layer 3 max diff: 7.152557373046875e-06, min diff: 0.0
layer 4 max diff: 5.7220458984375e-06, min diff: 0.0
layer 5 max diff: 6.198883056640625e-06, min diff: 0.0
layer 6 max diff: 8.106231689453125e-06, min diff: 0.0
layer 7 max diff: 6.67572021484375e-06, min diff: 0.0
layer 8 max diff: 6.198883056640625e-06, min diff: 0.0
layer 9 max diff: 8.106231689453125e-06, min diff: 0.0
layer 10 max diff: 1.0728836059570312e-05, min diff: 0.0
layer 11 max diff: 9.775161743164062e-06, min diff: 0.0
layer 12 max diff: 1.1086463928222656e-05, min diff: 0.0
layer 13 max diff: 9.298324584960938e-06, min diff: 0.0
layer 14 max diff: 8.106231689453125e-06, min diff: 0.0
layer 15 max diff: 1.3113021850585938e-05, min diff: 0.0
layer 16 max diff: 1.2874603271484375e-05, min diff: 0.0
layer 17 max diff: 3.4332275390625e-05, min diff: 0.0
layer 18 max diff: 1.9073486328125e-05, min diff: 0.0
layer 19 max diff: 1.1682510375976562e-05, min diff: 0.0
layer 20 max diff: 1.52587890625e-05, min diff: 0.0
layer 21 max diff: 2.384185791015625e-05, min diff: 0.0
layer 22 max diff: 2.5510787963867188e-05, min diff: 0.0
layer 23 max diff: 3.337860107421875e-05, min diff: 0.0
layer 24 max diff: 1.71661376953125e-05, min diff: 0.0

其中模型的参数是fp16，会产生一些微小的差别，是由于torch是基于fp32加载的（变成fp16会报错，有算子不支持），paddle是基于fp16加载的，计算出来的结果会稍有不同

加入文档：

跟huggingface的源代码有如下两个区别：

• XSoftmax和XDropout， XSoftmax缺少_softmax_backward_data算子，Paddle无法直接实现反向，已提issue：_softmax_backward_data Paddle#63387

这两个算子不影响推理，可能会影响训练对齐。

[paddle-bot]

Thanks for your contribution!

[codecov]

Codecov Report

Attention: Patch coverage is 72.99509% with 495 lines in your changes are missing coverage. Please review.

Project coverage is 55.23%. Comparing base (7b493a8) to head (78af468).
Report is 9 commits behind head on develop.

Files	Patch %	Lines
paddlenlp/transformers/deberta_v2/modeling.py	65.83%	234 Missing ⚠️
paddlenlp/transformers/deberta/modeling.py	76.26%	155 Missing ⚠️
paddlenlp/transformers/deberta_v2/tokenizer.py	62.17%	101 Missing ⚠️
paddlenlp/transformers/deberta/tokenizer.py	97.36%	4 Missing ⚠️
paddlenlp/transformers/deberta_v2/configuration.py	97.36%	1 Missing ⚠️

Additional details and impacted files

@@             Coverage Diff             @@
##           develop    #8227      +/-   ##
===========================================
+ Coverage    55.15%   55.23%   +0.08%     
===========================================
  Files          601      609       +8     
  Lines        91764    94218    +2454     
===========================================
+ Hits         50611    52040    +1429     
- Misses       41153    42178    +1025     

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

[seetimee]

请问能加入中文版的deberta吗？

[w5688414]

中文哪个版本？

[seetimee]

好像只有二郎神的v2

[w5688414]

可以给出对应的hf的deberta链接

[seetimee]

https://huggingface.co/IDEA-CCNL/Erlangshen-DeBERTa-v2-710M-Chinese

[w5688414]

欢迎开发者贡献:

PaddleNLP/paddlenlp/transformers/deberta_v2/modeling.py

Line 787 in f27fdb4

def _get_name_mappings(cls, config):