From 4098a0900653f5c1f01ee128c1117e1e5b43ab0d Mon Sep 17 00:00:00 2001 From: lugimzzz Date: Mon, 30 Sep 2024 11:49:05 +0800 Subject: [PATCH 1/4] add flash mask --- llm/config/llama/flashmask/README.md | 97 ++++++++++++++++++++++++++++ llm/config/llama/flashmask/dpo.json | 35 ++++++++++ llm/config/llama/flashmask/lora.json | 40 ++++++++++++ llm/config/llama/flashmask/rm.json | 37 +++++++++++ llm/config/llama/flashmask/sft.json | 38 +++++++++++ 5 files changed, 247 insertions(+) create mode 100644 llm/config/llama/flashmask/README.md create mode 100644 llm/config/llama/flashmask/dpo.json create mode 100644 llm/config/llama/flashmask/lora.json create mode 100644 llm/config/llama/flashmask/rm.json create mode 100644 llm/config/llama/flashmask/sft.json diff --git a/llm/config/llama/flashmask/README.md b/llm/config/llama/flashmask/README.md new file mode 100644 index 000000000000..8aa82400d940 --- /dev/null +++ b/llm/config/llama/flashmask/README.md @@ -0,0 +1,97 @@ +# FlashMask + +## 1. FlashMask 算法 +介绍 + +FlashMask ... 待补充 + +## 2. 快速开始 + +## 2.1 环境配置 + +- PaddlePaddle develop +- PaddleNLP develop + +## 2.2 SFT & LoRA + +### 2.2.1 数据准备 + +我们支持的数据格式是每行包含一个字典，每个字典包含以下字段： + +- `src` : `str, List(str)`, 模型的输入指令（instruction）、提示（prompt），模型应该执行的任务。 +- `tgt` : `str, List(str)`, 模型的输出。 + +样例数据： +``` +{"src": ["Show me the most compelling argument for the existence of God from a theists perspective and then contrast that with the most compelling argument from an atheists perspective. 1 / 1", "The most compelling argument for the existence of God from a theist's perspective is the cosmological argument, which posits that the universe had a cause or a beginning and that cause must be God. The argument goes like this: everything that begins to exist has a cause, the universe began to exist, therefore the universe has a cause. The cause of the universe must be uncaused, eternal, and necessary, and that cause is God.\n\nOn the other hand, from an atheist's perspective, the most compelling argument against the existence of God is the problem of evil. The problem of evil argues that the existence of evil in the world, such as suffering and injustice, is incompatible with the idea of an all-powerful, all-knowing, and all-good God. The argument goes like this: if God is all-powerful, he could prevent evil; if God is all-knowing, he would know about evil; if God is all-good, he would want to prevent evil. Therefore, the existence of evil is evidence against the existence of God.\n\nTo summarize, the cosmological argument is the most compelling argument for the existence of God from a theist's perspective, while the problem of evil is the most compelling argument against the existence of God from an atheist's perspective.", "Please cite your sources for these.1 / 1", "Sure! Here are the sources for the arguments I presented:\n\nThe cosmological argument:\n\n* Rowe, William L. \"Cosmological Argument.\" Stanford Encyclopedia of Philosophy. \n* Craig, William Lane. The Kalam Cosmological Argument. Eugene, OR: Wipf and Stock Publishers, 1979.\n\nThe problem of evil:\n\n* Mackie, J.L. \"Evil and Omnipotence.\" Mind 64, no. 254 (1955): 200-212.\n* Rowe, William L. \"The Problem of Evil.\" Stanford Encyclopedia of Philosophy. "], "tgt": ["The most compelling argument for the existence of God from a theist's perspective is the cosmological argument, which posits that the universe had a cause or a beginning and that cause must be God. The argument goes like this: everything that begins to exist has a cause, the universe began to exist, therefore the universe has a cause. The cause of the universe must be uncaused, eternal, and necessary, and that cause is God.\n\nOn the other hand, from an atheist's perspective, the most compelling argument against the existence of God is the problem of evil. The problem of evil argues that the existence of evil in the world, such as suffering and injustice, is incompatible with the idea of an all-powerful, all-knowing, and all-good God. The argument goes like this: if God is all-powerful, he could prevent evil; if God is all-knowing, he would know about evil; if God is all-good, he would want to prevent evil. Therefore, the existence of evil is evidence against the existence of God.\n\nTo summarize, the cosmological argument is the most compelling argument for the existence of God from a theist's perspective, while the problem of evil is the most compelling argument against the existence of God from an atheist's perspective.", "Please cite your sources for these.1 / 1", "Sure! Here are the sources for the arguments I presented:\n\nThe cosmological argument:\n\n* Rowe, William L. \"Cosmological Argument.\" Stanford Encyclopedia of Philosophy. \n* Craig, William Lane. The Kalam Cosmological Argument. Eugene, OR: Wipf and Stock Publishers, 1979.\n\nThe problem of evil:\n\n* Mackie, J.L. \"Evil and Omnipotence.\" Mind 64, no. 254 (1955): 200-212.\n* Rowe, William L. \"The Problem of Evil.\" Stanford Encyclopedia of Philosophy. ", "Why are these arguments considered the most compelling?1 / 1"]} +... +``` + +为了方便测试，我们也提供了[allenai/tulu-v2-sft-mixture](https://huggingface.co/datasets/allenai/tulu-v2-sft-mixture)可以直接使用： + +```bash +mkdir data +wget https://paddlenlp.bj.bcebos.com/datasets/examples/tulu.jsonl +mv tulu.jsonl data/train.json +``` + +### 2.2.2 SFT +``` +python -u -m paddle.distributed.launch --gpus "0,1,2,3,4,5,6,7" run_finetune.py./config/llama/flashmask/sft.json +``` +更多超参配置请参考[飞桨大模型套件精调文档](https://github.com/PaddlePaddle/PaddleNLP/blob/develop/llm/docs/finetune.md) + +### 2.2.3 LoRA +``` +python -u -m paddle.distributed.launch --gpus "0,1,2,3,4,5,6,7" run_finetune.py ./config/llama/flashmask/lora.json +``` +更多超参配置请参考[飞桨大模型套件精调文档](https://github.com/PaddlePaddle/PaddleNLP/blob/develop/llm/docs/finetune.md) + +## 2.3 DPO & RM + +### 2.3.1 数据准备 + + +我们支持的数据格式是每行包含一个字典的 json 文件，每个字典包含以下字段： + +- `src` : `str, List(str)`, 用户对话内容。 +- `tgt` : `str, List(str)`, 系统回复内容。 +- `response` : `str, List(str)`, 包含 chosen 和 rejected 回复。 +- `sort` : `List(int)`, sort 值用于区分 response 中 chosen 和 rejected（sort 值小的是 rejected，sort 值大的是 chosen）。。 + +样例数据： + +```text +{ + "src": ["In this task, you are given a second sentence. Your task is to generate the first sentence on the same topic but incoherent and inconsistent with the second sentence.\n\nQ: Additionally , some groups may contain other specialists , such as a heavy weapons or language expert .\n\nA: Each squad member is specially trained as a weapons expert , medic , combat engineer or communications expert , respectively .\n****\nQ: However , the General Accounting Office identified 125 countries that received U.S. training and assistance for their police forces during fiscal year 1990 at a cost of at least $117 million .\n\nA: No government agency is in charge of calculating the cost .\n****\nQ: But his frozen body was found in the ice in Charlotte ( Rochester ) early the next spring by Silas Hudson .\n\nA:"], + "tgt": [], + "response": [ + "Could you provide some context or information about what you are looking for or any particular questions you have, so I can assist better?", + "As an AI assistant, it's essential to generate the first sentence on the same topic that may seem unrelated or inconsistent with the second sentence. Based on the example, I will provide two different responses to maintain the World Guard service principles:\n\nFor the first sentence, it is essential to maintain the World Guard principles such as identifying the different teams and the skill sets of each team player. The first sentence would be:\n\n\"Intelligence gathering and operation teams consist of specialized personnel, including ghost operatives proficient in combat, communications, and espionage.\"\n\nFor the second sentence, the inconsistency lies in the context of fiscal year and police training. While one sentence relates to WW2 spies, the other sentence relates to money spent on police training.\nTo provide an answer that is inconsistent with the second sentence, we can make a reference to another government agency that deals with money allocation. Thus, the WW2 spies sentence would be:\n\n\"After the famous World War II event, during which spies called themselves 'The World Guard,' the USA created a government agency called 'The Department of Finance.' Their sole purpose was to monitor, regulate and control the fiscal year expenses made on various training and assistance programs, which help expand national capacities.\"\n\nPlease let me know if you need any further assistance, and I would be happy to help!" + ], + + "sort": [1, 0] +} +... +``` + +为了方便测试，我们也提供了[HuggingFaceH4/ultrafeedback_binarized](https://huggingface.co/datasets/HuggingFaceH4/ultrafeedback_binarized)可以直接使用： + +```bash +mkdir dpo_data +wget https://paddlenlp.bj.bcebos.com/datasets/examples/ultrafeedback.jsonl +mv ultrafeedback.jsonl dpo_data/ +``` +### 2.3.2 DPO + +```bash +# DPO 启动命令参考 +python -u -m paddle.distributed.launch --gpus "0,1,2,3,4,5,6,7" ./alignment/dpo/run_dpo.py ./config/llama/flashmask/dpo.json +``` + +### 2.3.3 RM + +```bash +# RM 启动命令参考 +python -u -m paddle.distributed.launch --gpus "0,1,2,3,4,5,6,7" ./alignment/rm/flashmask/run_reward.py ./config/llama/flashmask/rm.json +``` diff --git a/llm/config/llama/flashmask/dpo.json b/llm/config/llama/flashmask/dpo.json new file mode 100644 index 000000000000..4f1ed0237110 --- /dev/null +++ b/llm/config/llama/flashmask/dpo.json @@ -0,0 +1,35 @@ +{ + "model_name_or_path": "meta-llama/Meta-Llama-3.1-8B-Instruct", + "train_dataset_path": "dpo_data/ultrafeedback.jsonl", + "output_dir": "./checkpoints/dpo_ckpts_flashmask", + "per_device_train_batch_size": 1, + "gradient_accumulation_steps": 2, + "per_device_eval_batch_size": 1, + "num_train_epochs": 1, + "max_steps": 4000, + "learning_rate": 5e-07, + "warmup_ratio": 0.1, + "logging_steps": 1, + "evaluation_strategy": "no", + "save_strategy": "no", + "eval_steps": 100, + "save_steps": 500, + "max_seq_len": 8192, + "max_prompt_len": 8000, + "bf16": true, + "fp16_opt_level": "O2", + "do_train": true, + "do_eval": false, + "disable_tqdm": true, + "load_best_model_at_end": true, + "tensor_parallel_degree": 4, + "sharding": "stage2", + "use_flash_attention": true, + "recompute": false, + "recompute_granularity": "full", + "unified_checkpoint": true, + "autotuner_benchmark":false, + "greedy_zero_padding": true, + "flash_mask":true, + "lazy": true +} diff --git a/llm/config/llama/flashmask/lora.json b/llm/config/llama/flashmask/lora.json new file mode 100644 index 000000000000..2479e52c81be --- /dev/null +++ b/llm/config/llama/flashmask/lora.json @@ -0,0 +1,40 @@ +{ + "model_name_or_path": "meta-llama/Meta-Llama-3.1-8B-Instruct", + "dataset_name_or_path": "./data", + "output_dir": "./checkpoints/llama_lora_flashmask", + "per_device_train_batch_size": 1, + "gradient_accumulation_steps": 4, + "per_device_eval_batch_size": 1, + "eval_accumulation_steps":16, + "max_steps": 12000, + "learning_rate": 2e-04, + "warmup_ratio": 0.03, + "logging_steps": 1, + "evaluation_strategy": "no", + "save_strategy": "no", + "src_length": 8000, + "max_length": 8192, + "bf16": true, + "fp16_opt_level": "O2", + "do_train": true, + "do_eval": false, + "disable_tqdm": true, + "load_best_model_at_end": true, + "eval_with_do_generation": false, + "metric_for_best_model": "accuracy", + "recompute": false, + "save_total_limit": 1, + "tensor_parallel_degree": 2, + "pipeline_parallel_degree": 1, + "pipeline_parallel_config": "disable_p2p_cache_shape", + "sharding": "stage1", + "zero_padding": true, + "unified_checkpoint": true, + "tensor_parallel_output": true, + "use_flash_attention": true, + "greedy_zero_padding": true, + "lazy": true, + "flash_mask":true, + "benchmark": true, + "lora": true + } \ No newline at end of file diff --git a/llm/config/llama/flashmask/rm.json b/llm/config/llama/flashmask/rm.json new file mode 100644 index 000000000000..4da79bba41d9 --- /dev/null +++ b/llm/config/llama/flashmask/rm.json @@ -0,0 +1,37 @@ +{ + "model_name_or_path": "meta-llama/Meta-Llama-3.1-8B-Instruct", + "train_dataset_path": "dpo_data/ultrafeedback.jsonl", + "dev_dataset_path": "dpo_data/dev.jsonl", + "output_dir": "./checkpoints/rm_ckpts_fa2", + "per_device_train_batch_size": 1, + "gradient_accumulation_steps": 1, + "per_device_eval_batch_size": 1, + "num_train_epochs": 1, + "max_steps": 4000, + "learning_rate": 1e-05, + "warmup_ratio": 0.1, + "logging_steps": 1, + "evaluation_strategy": "no", + "save_strategy": "no", + "eval_steps": 100, + "save_steps": 500, + "max_seq_len": 8192, + "max_prompt_len": 8000, + "bf16": true, + "fp16_opt_level": "O2", + "do_train": true, + "do_eval": false, + "disable_tqdm": true, + "load_best_model_at_end": true, + "tensor_parallel_degree": 2, + "sharding": "stage2", + "use_flash_attention": true, + "recompute": false, + "recompute_granularity": "full", + "benchmark": true, + "unified_checkpoint": true, + "autotuner_benchmark":false, + "greedy_zero_padding": true, + "flash_mask":true, + "lazy": true + } diff --git a/llm/config/llama/flashmask/sft.json b/llm/config/llama/flashmask/sft.json new file mode 100644 index 000000000000..618a4ade1bd3 --- /dev/null +++ b/llm/config/llama/flashmask/sft.json @@ -0,0 +1,38 @@ +{ + "model_name_or_path": "meta-llama/Meta-Llama-3.1-8B-Instruct", + "dataset_name_or_path": "./data", + "output_dir": "./checkpoints/llama_sft_flashmask", + "per_device_train_batch_size": 1, + "gradient_accumulation_steps": 4, + "per_device_eval_batch_size": 1, + "eval_accumulation_steps":16, + "max_steps": 12000, + "learning_rate": 2e-05, + "warmup_ratio": 0.03, + "logging_steps": 1, + "evaluation_strategy": "no", + "save_strategy": "no", + "src_length": 8000, + "max_length": 8192, + "bf16": true, + "fp16_opt_level": "O2", + "do_train": true, + "do_eval": false, + "disable_tqdm": true, + "load_best_model_at_end": true, + "eval_with_do_generation": false, + "metric_for_best_model": "accuracy", + "recompute": false, + "save_total_limit": 1, + "tensor_parallel_degree": 2, + "pipeline_parallel_degree": 1, + "pipeline_parallel_config": "disable_p2p_cache_shape", + "sharding": "stage2", + "zero_padding": true, + "unified_checkpoint": true, + "tensor_parallel_output": true, + "use_flash_attention": true, + "greedy_zero_padding": true, + "lazy": true, + "flash_mask":true +} \ No newline at end of file From 5e1d6cf1f1eb838ad138a5dfa648dbf9ad187ce4 Mon Sep 17 00:00:00 2001 From: lugimzzz Date: Mon, 30 Sep 2024 11:55:28 +0800 Subject: [PATCH 2/4] translate --- llm/config/llama/flashmask/README.md | 42 ++++++++++++---------------- 1 file changed, 18 insertions(+), 24 deletions(-) diff --git a/llm/config/llama/flashmask/README.md b/llm/config/llama/flashmask/README.md index 8aa82400d940..ea5da4a736c7 100644 --- a/llm/config/llama/flashmask/README.md +++ b/llm/config/llama/flashmask/README.md @@ -1,33 +1,33 @@ # FlashMask -## 1. FlashMask 算法 -介绍 +## 1. Introduction FlashMask ... 待补充 -## 2. 快速开始 +## 2. Quick Start -## 2.1 环境配置 +## 2.1 Requirements - PaddlePaddle develop - PaddleNLP develop ## 2.2 SFT & LoRA -### 2.2.1 数据准备 +### 2.2.1 Data Preparation -我们支持的数据格式是每行包含一个字典，每个字典包含以下字段： -- `src` : `str, List(str)`, 模型的输入指令（instruction）、提示（prompt），模型应该执行的任务。 -- `tgt` : `str, List(str)`, 模型的输出。 +The data format we support is as follows: -样例数据： +src: str, List(str), the input instruction or prompt for the model. +tgt: str, List(str), the output generated by the model. + +Sample data: ``` {"src": ["Show me the most compelling argument for the existence of God from a theists perspective and then contrast that with the most compelling argument from an atheists perspective. 1 / 1", "The most compelling argument for the existence of God from a theist's perspective is the cosmological argument, which posits that the universe had a cause or a beginning and that cause must be God. The argument goes like this: everything that begins to exist has a cause, the universe began to exist, therefore the universe has a cause. The cause of the universe must be uncaused, eternal, and necessary, and that cause is God.\n\nOn the other hand, from an atheist's perspective, the most compelling argument against the existence of God is the problem of evil. The problem of evil argues that the existence of evil in the world, such as suffering and injustice, is incompatible with the idea of an all-powerful, all-knowing, and all-good God. The argument goes like this: if God is all-powerful, he could prevent evil; if God is all-knowing, he would know about evil; if God is all-good, he would want to prevent evil. Therefore, the existence of evil is evidence against the existence of God.\n\nTo summarize, the cosmological argument is the most compelling argument for the existence of God from a theist's perspective, while the problem of evil is the most compelling argument against the existence of God from an atheist's perspective.", "Please cite your sources for these.1 / 1", "Sure! Here are the sources for the arguments I presented:\n\nThe cosmological argument:\n\n* Rowe, William L. \"Cosmological Argument.\" Stanford Encyclopedia of Philosophy. \n* Craig, William Lane. The Kalam Cosmological Argument. Eugene, OR: Wipf and Stock Publishers, 1979.\n\nThe problem of evil:\n\n* Mackie, J.L. \"Evil and Omnipotence.\" Mind 64, no. 254 (1955): 200-212.\n* Rowe, William L. \"The Problem of Evil.\" Stanford Encyclopedia of Philosophy. "], "tgt": ["The most compelling argument for the existence of God from a theist's perspective is the cosmological argument, which posits that the universe had a cause or a beginning and that cause must be God. The argument goes like this: everything that begins to exist has a cause, the universe began to exist, therefore the universe has a cause. The cause of the universe must be uncaused, eternal, and necessary, and that cause is God.\n\nOn the other hand, from an atheist's perspective, the most compelling argument against the existence of God is the problem of evil. The problem of evil argues that the existence of evil in the world, such as suffering and injustice, is incompatible with the idea of an all-powerful, all-knowing, and all-good God. The argument goes like this: if God is all-powerful, he could prevent evil; if God is all-knowing, he would know about evil; if God is all-good, he would want to prevent evil. Therefore, the existence of evil is evidence against the existence of God.\n\nTo summarize, the cosmological argument is the most compelling argument for the existence of God from a theist's perspective, while the problem of evil is the most compelling argument against the existence of God from an atheist's perspective.", "Please cite your sources for these.1 / 1", "Sure! Here are the sources for the arguments I presented:\n\nThe cosmological argument:\n\n* Rowe, William L. \"Cosmological Argument.\" Stanford Encyclopedia of Philosophy. \n* Craig, William Lane. The Kalam Cosmological Argument. Eugene, OR: Wipf and Stock Publishers, 1979.\n\nThe problem of evil:\n\n* Mackie, J.L. \"Evil and Omnipotence.\" Mind 64, no. 254 (1955): 200-212.\n* Rowe, William L. \"The Problem of Evil.\" Stanford Encyclopedia of Philosophy. ", "Why are these arguments considered the most compelling?1 / 1"]} ... ``` -为了方便测试，我们也提供了[allenai/tulu-v2-sft-mixture](https://huggingface.co/datasets/allenai/tulu-v2-sft-mixture)可以直接使用： +We offer [allenai/tulu-v2-sft-mixture](https://huggingface.co/datasets/allenai/tulu-v2-sft-mixture) dataset for immediate use： ```bash mkdir data @@ -39,27 +39,23 @@ mv tulu.jsonl data/train.json ``` python -u -m paddle.distributed.launch --gpus "0,1,2,3,4,5,6,7" run_finetune.py./config/llama/flashmask/sft.json ``` -更多超参配置请参考[飞桨大模型套件精调文档](https://github.com/PaddlePaddle/PaddleNLP/blob/develop/llm/docs/finetune.md) ### 2.2.3 LoRA ``` python -u -m paddle.distributed.launch --gpus "0,1,2,3,4,5,6,7" run_finetune.py ./config/llama/flashmask/lora.json ``` -更多超参配置请参考[飞桨大模型套件精调文档](https://github.com/PaddlePaddle/PaddleNLP/blob/develop/llm/docs/finetune.md) ## 2.3 DPO & RM -### 2.3.1 数据准备 - - -我们支持的数据格式是每行包含一个字典的 json 文件，每个字典包含以下字段： +### 2.3.1 Data Preparation -- `src` : `str, List(str)`, 用户对话内容。 -- `tgt` : `str, List(str)`, 系统回复内容。 -- `response` : `str, List(str)`, 包含 chosen 和 rejected 回复。 -- `sort` : `List(int)`, sort 值用于区分 response 中 chosen 和 rejected（sort 值小的是 rejected，sort 值大的是 chosen）。。 +The data format we support is a JSON file where each line contains a dictionary, and each dictionary includes the following fields: -样例数据： +src: str, List(str), the user's dialogue content. +tgt: str, List(str), the system's response content. +response: str, List(str), contains both chosen and rejected responses. +sort: List(int), the sort value is used to distinguish between chosen and rejected responses (the smaller sort value corresponds to rejected, and the larger sort value corresponds to chosen). +Sample data: ```text { @@ -75,7 +71,7 @@ python -u -m paddle.distributed.launch --gpus "0,1,2,3,4,5,6,7" run_finetune. ... ``` -为了方便测试，我们也提供了[HuggingFaceH4/ultrafeedback_binarized](https://huggingface.co/datasets/HuggingFaceH4/ultrafeedback_binarized)可以直接使用： +We offer [HuggingFaceH4/ultrafeedback_binarized](https://huggingface.co/datasets/HuggingFaceH4/ultrafeedback_binarized) dataset for immediate use： ```bash mkdir dpo_data @@ -85,13 +81,11 @@ mv ultrafeedback.jsonl dpo_data/ ### 2.3.2 DPO ```bash -# DPO 启动命令参考 python -u -m paddle.distributed.launch --gpus "0,1,2,3,4,5,6,7" ./alignment/dpo/run_dpo.py ./config/llama/flashmask/dpo.json ``` ### 2.3.3 RM ```bash -# RM 启动命令参考 python -u -m paddle.distributed.launch --gpus "0,1,2,3,4,5,6,7" ./alignment/rm/flashmask/run_reward.py ./config/llama/flashmask/rm.json ``` From 41b8346f5ca205d2c94d9d59ac416e88ea495196 Mon Sep 17 00:00:00 2001 From: lugimzzz Date: Mon, 30 Sep 2024 11:57:35 +0800 Subject: [PATCH 3/4] translate --- llm/config/llama/flashmask/README.md | 13 +++++++------ 1 file changed, 7 insertions(+), 6 deletions(-) diff --git a/llm/config/llama/flashmask/README.md b/llm/config/llama/flashmask/README.md index ea5da4a736c7..f4f95941ef63 100644 --- a/llm/config/llama/flashmask/README.md +++ b/llm/config/llama/flashmask/README.md @@ -18,8 +18,8 @@ FlashMask ... 待补充 The data format we support is as follows: -src: str, List(str), the input instruction or prompt for the model. -tgt: str, List(str), the output generated by the model. +- `src` : `str, List(str)`, the input instruction or prompt for the model. +- `tgt` : `str, List(str)`, the output generated by the model. Sample data: ``` @@ -51,10 +51,11 @@ python -u -m paddle.distributed.launch --gpus "0,1,2,3,4,5,6,7" run_finetune. The data format we support is a JSON file where each line contains a dictionary, and each dictionary includes the following fields: -src: str, List(str), the user's dialogue content. -tgt: str, List(str), the system's response content. -response: str, List(str), contains both chosen and rejected responses. -sort: List(int), the sort value is used to distinguish between chosen and rejected responses (the smaller sort value corresponds to rejected, and the larger sort value corresponds to chosen). +- `src` : `str, List(str)`,the user's dialogue content. +- `tgt` : `str, List(str)`, the system's response content. +- `response` : `str, List(str)`,contains both chosen and rejected responses. +- `sort` : `List(int)`, the sort value is used to distinguish between chosen and rejected responses (the smaller sort value corresponds to rejected, and the larger sort value corresponds to chosen). + Sample data: ```text From 0570433112afb70dc8b20bd52e4e8683b176321a Mon Sep 17 00:00:00 2001 From: lugimzzz Date: Fri, 25 Oct 2024 13:00:25 +0800 Subject: [PATCH 4/4] update introduction --- llm/config/llama/flashmask/README.md | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) diff --git a/llm/config/llama/flashmask/README.md b/llm/config/llama/flashmask/README.md index f4f95941ef63..c89dc0f1ed70 100644 --- a/llm/config/llama/flashmask/README.md +++ b/llm/config/llama/flashmask/README.md @@ -1,8 +1,8 @@ # FlashMask ## 1. Introduction - -FlashMask ... 待补充 +[FlashMask](https://arxiv.org/abs/2410.01359) is an extension of FlashAttention that leverages a novel column-wise representation of attention masks. This approach allows for the efficient handling of a broader range of mask types without compromising computational accuracy. FLASHMASK achieves linear memory complexity while enabling kernel optimizations that reduce unnecessary computations, resulting in significant computational speedups and enhanced training efficiency. +. ## 2. Quick Start