Instructions

TencentPretrain supports a wide range of pre-training models and downstream tasks. This section shows the comprehensive use-cases when using the TencentPretrain. In many cases, we use BERT model to demonstrate how to use TencentPretrain by default.

• Preprocess the data
• Pretrain the model
• Pretraining model examples
• Finetune the tasks
• Tokenization and vocabulary
• Scripts
• DeepSpeed Support