Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

【PaddlePaddle Hackathon】57、在 PaddleNLP 中完善XLNet模型下游任务代码,新增DistilBert预训练权重 #1073

Closed
TCChenlong opened this issue Sep 23, 2021 · 1 comment
Labels

Comments

@TCChenlong
Copy link

(此 ISSUE 为 PaddlePaddle Hackathon 活动的任务 ISSUE,更多详见PaddlePaddle Hackathon

【任务说明】

  • 任务标题:完善GPT模型下游任务代码,新增GPT预训练权重

  • 技术标签:深度学习框架,预训练语言模型

  • 任务难度:中等

  • 详细描述:

  1. 在PaddleNLP的XLNet模型代码中,新增 XLNetLMHeadModel,XLNetForMultipleChoice 和 XLNetForQuestionAnswering 这三个类。参考 https://github.com/huggingface/transformers/blob/master/src/transformers/models/xlnet/modeling_xlnet.py
  2. 新增 distilbert-base-multilingual-cased 和 sshleifer/tiny-distilbert-base-uncased-finetuned-sst-2-english 两个模型参数权重。权重参考:https://huggingface.co/models,转换文档:[如何贡献模型](http://wiki.baidu.com/[https:/paddlenlp.readthedocs.io/zh/latest/community/contribute_models/index.html)

【提交内容】

  • 任务 PR 到 PaddleNLP(代码+注释)+ 项目单测文件

  • 任务 PR 到 PaddleNLP(贡献模型权重+权重转换代码)

【技术要求】

  • 了解PaddleNLP
  • 了解Hugging-Face transformers
  • 了解预训练语言模型
@github-actions
Copy link

This issue is stale because it has been open for 60 days with no activity. 当前issue 60天内无活动,被标记为stale。

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Projects
None yet
Development

No branches or pull requests

4 participants