-
Notifications
You must be signed in to change notification settings - Fork 3k
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
[Readme] Add flash mask #9219
Merged
Merged
[Readme] Add flash mask #9219
Changes from all commits
Commits
Show all changes
4 commits
Select commit
Hold shift + click to select a range
File filter
Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
There are no files selected for viewing
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,92 @@ | ||
# FlashMask | ||
|
||
## 1. Introduction | ||
[FlashMask](https://arxiv.org/abs/2410.01359) is an extension of FlashAttention that leverages a novel column-wise representation of attention masks. This approach allows for the efficient handling of a broader range of mask types without compromising computational accuracy. FLASHMASK achieves linear memory complexity while enabling kernel optimizations that reduce unnecessary computations, resulting in significant computational speedups and enhanced training efficiency. | ||
. | ||
|
||
## 2. Quick Start | ||
|
||
## 2.1 Requirements | ||
|
||
- PaddlePaddle develop | ||
- PaddleNLP develop | ||
|
||
## 2.2 SFT & LoRA | ||
|
||
### 2.2.1 Data Preparation | ||
|
||
|
||
The data format we support is as follows: | ||
|
||
- `src` : `str, List(str)`, the input instruction or prompt for the model. | ||
- `tgt` : `str, List(str)`, the output generated by the model. | ||
|
||
Sample data: | ||
``` | ||
{"src": ["Show me the most compelling argument for the existence of God from a theists perspective and then contrast that with the most compelling argument from an atheists perspective. 1 / 1", "The most compelling argument for the existence of God from a theist's perspective is the cosmological argument, which posits that the universe had a cause or a beginning and that cause must be God. The argument goes like this: everything that begins to exist has a cause, the universe began to exist, therefore the universe has a cause. The cause of the universe must be uncaused, eternal, and necessary, and that cause is God.\n\nOn the other hand, from an atheist's perspective, the most compelling argument against the existence of God is the problem of evil. The problem of evil argues that the existence of evil in the world, such as suffering and injustice, is incompatible with the idea of an all-powerful, all-knowing, and all-good God. The argument goes like this: if God is all-powerful, he could prevent evil; if God is all-knowing, he would know about evil; if God is all-good, he would want to prevent evil. Therefore, the existence of evil is evidence against the existence of God.\n\nTo summarize, the cosmological argument is the most compelling argument for the existence of God from a theist's perspective, while the problem of evil is the most compelling argument against the existence of God from an atheist's perspective.", "Please cite your sources for these.1 / 1", "Sure! Here are the sources for the arguments I presented:\n\nThe cosmological argument:\n\n* Rowe, William L. \"Cosmological Argument.\" Stanford Encyclopedia of Philosophy. <https://plato.stanford.edu/entries/cosmological-argument/>\n* Craig, William Lane. The Kalam Cosmological Argument. Eugene, OR: Wipf and Stock Publishers, 1979.\n\nThe problem of evil:\n\n* Mackie, J.L. \"Evil and Omnipotence.\" Mind 64, no. 254 (1955): 200-212.\n* Rowe, William L. \"The Problem of Evil.\" Stanford Encyclopedia of Philosophy. <https://plato.stanford.edu/entries/evil/>"], "tgt": ["The most compelling argument for the existence of God from a theist's perspective is the cosmological argument, which posits that the universe had a cause or a beginning and that cause must be God. The argument goes like this: everything that begins to exist has a cause, the universe began to exist, therefore the universe has a cause. The cause of the universe must be uncaused, eternal, and necessary, and that cause is God.\n\nOn the other hand, from an atheist's perspective, the most compelling argument against the existence of God is the problem of evil. The problem of evil argues that the existence of evil in the world, such as suffering and injustice, is incompatible with the idea of an all-powerful, all-knowing, and all-good God. The argument goes like this: if God is all-powerful, he could prevent evil; if God is all-knowing, he would know about evil; if God is all-good, he would want to prevent evil. Therefore, the existence of evil is evidence against the existence of God.\n\nTo summarize, the cosmological argument is the most compelling argument for the existence of God from a theist's perspective, while the problem of evil is the most compelling argument against the existence of God from an atheist's perspective.", "Please cite your sources for these.1 / 1", "Sure! Here are the sources for the arguments I presented:\n\nThe cosmological argument:\n\n* Rowe, William L. \"Cosmological Argument.\" Stanford Encyclopedia of Philosophy. <https://plato.stanford.edu/entries/cosmological-argument/>\n* Craig, William Lane. The Kalam Cosmological Argument. Eugene, OR: Wipf and Stock Publishers, 1979.\n\nThe problem of evil:\n\n* Mackie, J.L. \"Evil and Omnipotence.\" Mind 64, no. 254 (1955): 200-212.\n* Rowe, William L. \"The Problem of Evil.\" Stanford Encyclopedia of Philosophy. <https://plato.stanford.edu/entries/evil/>", "Why are these arguments considered the most compelling?1 / 1"]} | ||
... | ||
``` | ||
|
||
We offer [allenai/tulu-v2-sft-mixture](https://huggingface.co/datasets/allenai/tulu-v2-sft-mixture) dataset for immediate use: | ||
|
||
```bash | ||
mkdir data | ||
wget https://paddlenlp.bj.bcebos.com/datasets/examples/tulu.jsonl | ||
mv tulu.jsonl data/train.json | ||
``` | ||
|
||
### 2.2.2 SFT | ||
``` | ||
python -u -m paddle.distributed.launch --gpus "0,1,2,3,4,5,6,7" run_finetune.py./config/llama/flashmask/sft.json | ||
``` | ||
|
||
### 2.2.3 LoRA | ||
``` | ||
python -u -m paddle.distributed.launch --gpus "0,1,2,3,4,5,6,7" run_finetune.py ./config/llama/flashmask/lora.json | ||
``` | ||
|
||
## 2.3 DPO & RM | ||
|
||
### 2.3.1 Data Preparation | ||
|
||
The data format we support is a JSON file where each line contains a dictionary, and each dictionary includes the following fields: | ||
|
||
- `src` : `str, List(str)`,the user's dialogue content. | ||
- `tgt` : `str, List(str)`, the system's response content. | ||
- `response` : `str, List(str)`,contains both chosen and rejected responses. | ||
- `sort` : `List(int)`, the sort value is used to distinguish between chosen and rejected responses (the smaller sort value corresponds to rejected, and the larger sort value corresponds to chosen). | ||
|
||
Sample data: | ||
|
||
```text | ||
{ | ||
"src": ["In this task, you are given a second sentence. Your task is to generate the first sentence on the same topic but incoherent and inconsistent with the second sentence.\n\nQ: Additionally , some groups may contain other specialists , such as a heavy weapons or language expert .\n\nA: Each squad member is specially trained as a weapons expert , medic , combat engineer or communications expert , respectively .\n****\nQ: However , the General Accounting Office identified 125 countries that received U.S. training and assistance for their police forces during fiscal year 1990 at a cost of at least $117 million .\n\nA: No government agency is in charge of calculating the cost .\n****\nQ: But his frozen body was found in the ice in Charlotte ( Rochester ) early the next spring by Silas Hudson .\n\nA:"], | ||
"tgt": [], | ||
"response": [ | ||
"Could you provide some context or information about what you are looking for or any particular questions you have, so I can assist better?", | ||
"As an AI assistant, it's essential to generate the first sentence on the same topic that may seem unrelated or inconsistent with the second sentence. Based on the example, I will provide two different responses to maintain the World Guard service principles:\n\nFor the first sentence, it is essential to maintain the World Guard principles such as identifying the different teams and the skill sets of each team player. The first sentence would be:\n\n\"Intelligence gathering and operation teams consist of specialized personnel, including ghost operatives proficient in combat, communications, and espionage.\"\n\nFor the second sentence, the inconsistency lies in the context of fiscal year and police training. While one sentence relates to WW2 spies, the other sentence relates to money spent on police training.\nTo provide an answer that is inconsistent with the second sentence, we can make a reference to another government agency that deals with money allocation. Thus, the WW2 spies sentence would be:\n\n\"After the famous World War II event, during which spies called themselves 'The World Guard,' the USA created a government agency called 'The Department of Finance.' Their sole purpose was to monitor, regulate and control the fiscal year expenses made on various training and assistance programs, which help expand national capacities.\"\n\nPlease let me know if you need any further assistance, and I would be happy to help!" | ||
], | ||
|
||
"sort": [1, 0] | ||
} | ||
... | ||
``` | ||
|
||
We offer [HuggingFaceH4/ultrafeedback_binarized](https://huggingface.co/datasets/HuggingFaceH4/ultrafeedback_binarized) dataset for immediate use: | ||
|
||
```bash | ||
mkdir dpo_data | ||
wget https://paddlenlp.bj.bcebos.com/datasets/examples/ultrafeedback.jsonl | ||
mv ultrafeedback.jsonl dpo_data/ | ||
``` | ||
### 2.3.2 DPO | ||
|
||
```bash | ||
python -u -m paddle.distributed.launch --gpus "0,1,2,3,4,5,6,7" ./alignment/dpo/run_dpo.py ./config/llama/flashmask/dpo.json | ||
``` | ||
|
||
### 2.3.3 RM | ||
|
||
```bash | ||
python -u -m paddle.distributed.launch --gpus "0,1,2,3,4,5,6,7" ./alignment/rm/flashmask/run_reward.py ./config/llama/flashmask/rm.json | ||
``` |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,35 @@ | ||
{ | ||
"model_name_or_path": "meta-llama/Meta-Llama-3.1-8B-Instruct", | ||
"train_dataset_path": "dpo_data/ultrafeedback.jsonl", | ||
"output_dir": "./checkpoints/dpo_ckpts_flashmask", | ||
"per_device_train_batch_size": 1, | ||
"gradient_accumulation_steps": 2, | ||
"per_device_eval_batch_size": 1, | ||
"num_train_epochs": 1, | ||
"max_steps": 4000, | ||
"learning_rate": 5e-07, | ||
"warmup_ratio": 0.1, | ||
"logging_steps": 1, | ||
"evaluation_strategy": "no", | ||
"save_strategy": "no", | ||
"eval_steps": 100, | ||
"save_steps": 500, | ||
"max_seq_len": 8192, | ||
"max_prompt_len": 8000, | ||
"bf16": true, | ||
"fp16_opt_level": "O2", | ||
"do_train": true, | ||
"do_eval": false, | ||
"disable_tqdm": true, | ||
"load_best_model_at_end": true, | ||
"tensor_parallel_degree": 4, | ||
"sharding": "stage2", | ||
"use_flash_attention": true, | ||
"recompute": false, | ||
"recompute_granularity": "full", | ||
"unified_checkpoint": true, | ||
"autotuner_benchmark":false, | ||
"greedy_zero_padding": true, | ||
"flash_mask":true, | ||
"lazy": true | ||
} |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,40 @@ | ||
{ | ||
"model_name_or_path": "meta-llama/Meta-Llama-3.1-8B-Instruct", | ||
"dataset_name_or_path": "./data", | ||
"output_dir": "./checkpoints/llama_lora_flashmask", | ||
"per_device_train_batch_size": 1, | ||
"gradient_accumulation_steps": 4, | ||
"per_device_eval_batch_size": 1, | ||
"eval_accumulation_steps":16, | ||
"max_steps": 12000, | ||
"learning_rate": 2e-04, | ||
"warmup_ratio": 0.03, | ||
"logging_steps": 1, | ||
"evaluation_strategy": "no", | ||
"save_strategy": "no", | ||
"src_length": 8000, | ||
"max_length": 8192, | ||
"bf16": true, | ||
"fp16_opt_level": "O2", | ||
"do_train": true, | ||
"do_eval": false, | ||
"disable_tqdm": true, | ||
"load_best_model_at_end": true, | ||
"eval_with_do_generation": false, | ||
"metric_for_best_model": "accuracy", | ||
"recompute": false, | ||
"save_total_limit": 1, | ||
"tensor_parallel_degree": 2, | ||
"pipeline_parallel_degree": 1, | ||
"pipeline_parallel_config": "disable_p2p_cache_shape", | ||
"sharding": "stage1", | ||
"zero_padding": true, | ||
"unified_checkpoint": true, | ||
"tensor_parallel_output": true, | ||
"use_flash_attention": true, | ||
"greedy_zero_padding": true, | ||
"lazy": true, | ||
"flash_mask":true, | ||
"benchmark": true, | ||
"lora": true | ||
} |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,37 @@ | ||
{ | ||
"model_name_or_path": "meta-llama/Meta-Llama-3.1-8B-Instruct", | ||
"train_dataset_path": "dpo_data/ultrafeedback.jsonl", | ||
"dev_dataset_path": "dpo_data/dev.jsonl", | ||
"output_dir": "./checkpoints/rm_ckpts_fa2", | ||
"per_device_train_batch_size": 1, | ||
"gradient_accumulation_steps": 1, | ||
"per_device_eval_batch_size": 1, | ||
"num_train_epochs": 1, | ||
"max_steps": 4000, | ||
"learning_rate": 1e-05, | ||
"warmup_ratio": 0.1, | ||
"logging_steps": 1, | ||
"evaluation_strategy": "no", | ||
"save_strategy": "no", | ||
"eval_steps": 100, | ||
"save_steps": 500, | ||
"max_seq_len": 8192, | ||
"max_prompt_len": 8000, | ||
"bf16": true, | ||
"fp16_opt_level": "O2", | ||
"do_train": true, | ||
"do_eval": false, | ||
"disable_tqdm": true, | ||
"load_best_model_at_end": true, | ||
"tensor_parallel_degree": 2, | ||
"sharding": "stage2", | ||
"use_flash_attention": true, | ||
"recompute": false, | ||
"recompute_granularity": "full", | ||
"benchmark": true, | ||
"unified_checkpoint": true, | ||
"autotuner_benchmark":false, | ||
"greedy_zero_padding": true, | ||
"flash_mask":true, | ||
"lazy": true | ||
} |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,38 @@ | ||
{ | ||
"model_name_or_path": "meta-llama/Meta-Llama-3.1-8B-Instruct", | ||
"dataset_name_or_path": "./data", | ||
"output_dir": "./checkpoints/llama_sft_flashmask", | ||
"per_device_train_batch_size": 1, | ||
"gradient_accumulation_steps": 4, | ||
"per_device_eval_batch_size": 1, | ||
"eval_accumulation_steps":16, | ||
"max_steps": 12000, | ||
"learning_rate": 2e-05, | ||
"warmup_ratio": 0.03, | ||
"logging_steps": 1, | ||
"evaluation_strategy": "no", | ||
"save_strategy": "no", | ||
"src_length": 8000, | ||
"max_length": 8192, | ||
"bf16": true, | ||
"fp16_opt_level": "O2", | ||
"do_train": true, | ||
"do_eval": false, | ||
"disable_tqdm": true, | ||
"load_best_model_at_end": true, | ||
"eval_with_do_generation": false, | ||
"metric_for_best_model": "accuracy", | ||
"recompute": false, | ||
"save_total_limit": 1, | ||
"tensor_parallel_degree": 2, | ||
"pipeline_parallel_degree": 1, | ||
"pipeline_parallel_config": "disable_p2p_cache_shape", | ||
"sharding": "stage2", | ||
"zero_padding": true, | ||
"unified_checkpoint": true, | ||
"tensor_parallel_output": true, | ||
"use_flash_attention": true, | ||
"greedy_zero_padding": true, | ||
"lazy": true, | ||
"flash_mask":true | ||
} |
Oops, something went wrong.
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
建议放到llm/readme里,能力列表里增加一列,说明flashmask支持情况