[Feature] Add MaskFormer #2789
There are no files selected for viewing
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
|
|
@@ -26,7 +26,6 @@ val_dataset: | |
| - type: Normalize | ||
| mode: val | ||
|
|
||
| optimizer: | ||
| type: SGD | ||
| momentum: 0.9 | ||
|
|
||
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,17 @@ | ||
| # Per-Pixel Classification is Not All You Need for Semantic Segmentation | ||
|
|
||
| ## Reference | ||
|
|
||
| > Cheng, Bowen, Alex Schwing, and Alexander Kirillov. "Per-pixel classification is not all you need for semantic segmentation." Advances in Neural Information Processing Systems 34 (2021): 17864-17875. | ||
|
|
||
| ## Performance | ||
|
|
||
| ### ADE20k | ||
|
|
||
| | Model | Backbone | Resolution | Training Iters | mIoU | mIoU (flip) | mIoU (ms+flip) | Links | | ||
| |:-:|:-:|:-:|:-:|:-:|:-:|:-:|:-:| | ||
| |Maskformer-tiny|SwinTransformer|512x512|160000|47.6|-|-|[model](https://bj.bcebos.com/paddleseg/dygraph/ade20k/maskformer_ade20k_swin_tiny/model.pdparams) \| [log](https://bj.bcebos.com/paddleseg/dygraph/ade20k/maskformer_ade20k_swin_tiny/train.log) \| [vdl](https://www.paddlepaddle.org.cn/paddle/visualdl/service/app/scalar?id=e59773eaad87f677837add5ff110441e)| | ||
|
|
||
| * Maskformer support different backbone including tiny, small, base and large. Due to long training time, the accuracy result is not provided. | ||
|
|
||
| * Maskformer-Base and Maskformer-Large need to be evaled with multi-scale and flip by default. |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,76 @@ | ||
| batch_size: 4 | ||
| iters: 160000 | ||
|
|
||
| train_dataset: | ||
| type: MaskedADE20K | ||
| dataset_root: data/ADEChallengeData2016/ | ||
| transforms: | ||
| - type: ResizeByShort | ||
| short_size: [320, 384, 448, 512, 576, 640, 704, 768, 832, 896, 960, 1024, 1088, 1152, 1216, 1280, 1344] | ||
| max_size: 2560 | ||
| - type: RandomPaddingCrop | ||
| crop_size: [640, 640] | ||
| - type: RandomDistort | ||
| brightness_range: 0.125 | ||
| brightness_prob: 1.0 | ||
| contrast_range: 0.5 | ||
| contrast_prob: 1.0 | ||
| saturation_range: 0.5 | ||
| saturation_prob: 1.0 | ||
| hue_range: 18 | ||
| hue_prob: 1.0 | ||
| - type: RandomHorizontalFlip | ||
| - type: Padding | ||
| target_size: [640, 640] | ||
| im_padding_value: 128 | ||
| - type: Normalize | ||
| mean: [0.485, 0.456, 0.406] | ||
| std: [0.229, 0.224, 0.225] | ||
|
|
||
|
|
||
| val_dataset: | ||
| type: MaskedADE20K | ||
| dataset_root: data/ADEChallengeData2016/ | ||
| transforms: | ||
| - type: ResizeByShort | ||
| short_size: 512 | ||
| - type: Normalize | ||
| mean: [0.485, 0.456, 0.406] | ||
| std: [0.229, 0.224, 0.225] | ||
| mode: val | ||
|
|
||
| model: | ||
| type: MaskFormer | ||
| num_classes: 150 | ||
| backbone: | ||
| type: SwinTransformer_base_patch4_window7_384_maskformer | ||
| pretrained: https://bj.bcebos.com/paddleseg/paddleseg/dygraph/ade20k/maskformer_ade20k_swin_base/pretrain/model.pdparams | ||
|
There was a problem hiding this comment. pretrained还是针对整个maskformer模型，不是针对backbone啊 There was a problem hiding this comment. 不是，这个是只有swin的模型，训练日志里加载的参数数量也可以看出来 |
||
|
|
||
| optimizer: | ||
| type: AdamW | ||
| weight_decay: 0.01 | ||
| custom_cfg: | ||
| - name: backbone | ||
| lr_mult: 1.0 | ||
| - name: norm | ||
| weight_decay_mult: 0.0 | ||
| - name: relative_position_bias_table | ||
| weight_decay_mult: 0.0 | ||
| grad_clip_cfg: | ||
| name: ClipGradByNorm | ||
| clip_norm: 0.01 | ||
|
|
||
| lr_scheduler: | ||
| type: PolynomialDecay | ||
| warmup_iters: 1500 | ||
| warmup_start_lr: 6.0e-11 | ||
| learning_rate: 6.0e-05 | ||
| end_lr: 0 | ||
| power: 0.9 | ||
|
|
||
| loss: | ||
| types: | ||
| - type: MaskFormerLoss | ||
| num_classes: 150 | ||
| eos_coef: 0.1 | ||
| coef: [1] | ||
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,75 @@ | ||
| batch_size: 4 | ||
| iters: 160000 | ||
|
|
||
| train_dataset: | ||
| type: MaskedADE20K | ||
| dataset_root: data/ADEChallengeData2016/ | ||
| transforms: | ||
| - type: ResizeByShort | ||
| short_size: [320, 384, 448, 512, 576, 640, 704, 768, 832, 896, 960, 1024, 1088, 1152, 1216, 1280, 1344] | ||
| max_size: 2560 | ||
| - type: RandomPaddingCrop | ||
| crop_size: [640, 640] | ||
| - type: RandomDistort | ||
| brightness_range: 0.125 | ||
| brightness_prob: 1.0 | ||
| contrast_range: 0.5 | ||
| contrast_prob: 1.0 | ||
| saturation_range: 0.5 | ||
| saturation_prob: 1.0 | ||
| hue_range: 18 | ||
| hue_prob: 1.0 | ||
| - type: RandomHorizontalFlip | ||
| - type: Padding | ||
| target_size: [640, 640] | ||
| im_padding_value: 128 | ||
| - type: Normalize | ||
| mean: [0.485, 0.456, 0.406] | ||
| std: [0.229, 0.224, 0.225] | ||
|
|
||
| val_dataset: | ||
| type: MaskedADE20K | ||
| dataset_root: data/ADEChallengeData2016/ | ||
| transforms: | ||
| - type: ResizeByShort | ||
| short_size: 512 | ||
| - type: Normalize | ||
| mean: [0.485, 0.456, 0.406] | ||
| std: [0.229, 0.224, 0.225] | ||
| mode: val | ||
|
|
||
| model: | ||
| type: MaskFormer | ||
| num_classes: 150 | ||
| backbone: | ||
| type: SwinTransformer_large_patch4_window7_384_maskformer | ||
| pretrained: https://bj.bcebos.com/paddleseg/paddleseg/dygraph/ade20k/maskformer_ade20k_swin_large/pretrain/model.pdparams | ||
|
|
||
| optimizer: | ||
| type: AdamW | ||
| weight_decay: 0.01 | ||
| custom_cfg: | ||
| - name: backbone | ||
| lr_mult: 1.0 | ||
| - name: norm | ||
| weight_decay_mult: 0.0 | ||
| - name: relative_position_bias_table | ||
| weight_decay_mult: 0.0 | ||
| grad_clip_cfg: | ||
| name: ClipGradByNorm | ||
| clip_norm: 0.01 | ||
|
|
||
| lr_scheduler: | ||
| type: PolynomialDecay | ||
| warmup_iters: 1500 | ||
| warmup_start_lr: 6.0e-11 | ||
| learning_rate: 6.0e-05 | ||
| end_lr: 0 | ||
| power: 0.9 | ||
|
|
||
| loss: | ||
| types: | ||
| - type: MaskFormerLoss | ||
| num_classes: 150 | ||
| eos_coef: 0.1 | ||
| coef: [1] |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,75 @@ | ||
| batch_size: 4 | ||
| iters: 160000 | ||
|
|
||
| train_dataset: | ||
| type: MaskedADE20K | ||
| dataset_root: data/ADEChallengeData2016/ | ||
| transforms: | ||
| - type: ResizeByShort | ||
| short_size: [256, 307, 358, 409, 460, 512, 563, 614, 665, 716, 768, 819, 870, 921, 972, 1024] | ||
| max_size: 2048 | ||
| - type: RandomPaddingCrop | ||
| crop_size: [512, 512] | ||
| - type: RandomDistort | ||
| brightness_range: 0.125 | ||
| brightness_prob: 1.0 | ||
| contrast_range: 0.5 | ||
| contrast_prob: 1.0 | ||
| saturation_range: 0.5 | ||
| saturation_prob: 1.0 | ||
| hue_range: 18 | ||
| hue_prob: 1.0 | ||
| - type: RandomHorizontalFlip | ||
| - type: Padding | ||
| target_size: [512, 512] | ||
| im_padding_value: 128 | ||
| - type: Normalize | ||
| mean: [0.485, 0.456, 0.406] | ||
| std: [0.229, 0.224, 0.225] | ||
|
|
||
| val_dataset: | ||
| type: MaskedADE20K | ||
| dataset_root: data/ADEChallengeData2016/ | ||
| transforms: | ||
| - type: ResizeByShort | ||
| short_size: 512 | ||
| - type: Normalize | ||
| mean: [0.485, 0.456, 0.406] | ||
| std: [0.229, 0.224, 0.225] | ||
| mode: val | ||
|
|
||
| model: | ||
| type: MaskFormer | ||
| num_classes: 150 | ||
| backbone: | ||
| type: SwinTransformer_small_patch4_window7_224_maskformer | ||
| pretrained: https://bj.bcebos.com/paddleseg/paddleseg/dygraph/ade20k/maskformer_ade20k_swin_small/pretrain/model.pdparams | ||
|
|
||
| optimizer: | ||
| type: AdamW | ||
| weight_decay: 0.01 | ||
| custom_cfg: | ||
| - name: backbone | ||
| lr_mult: 1.0 | ||
| - name: norm | ||
| weight_decay_mult: 0.0 | ||
| - name: relative_position_bias_table | ||
| weight_decay_mult: 0.0 | ||
| grad_clip_cfg: | ||
| name: ClipGradByNorm | ||
| clip_norm: 0.01 | ||
|
|
||
| lr_scheduler: | ||
| type: PolynomialDecay | ||
| warmup_iters: 1500 | ||
| warmup_start_lr: 6.0e-11 | ||
| learning_rate: 6.0e-05 | ||
| end_lr: 0 | ||
| power: 0.9 | ||
|
|
||
| loss: | ||
| types: | ||
| - type: MaskFormerLoss | ||
| num_classes: 150 | ||
| eos_coef: 0.1 | ||
| coef: [1] |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,75 @@ | ||
| batch_size: 2 | ||
| iters: 160000 | ||
|
|
||
| train_dataset: | ||
| type: MaskedADE20K | ||
| dataset_root: data/ADEChallengeData2016/ | ||
| transforms: | ||
| - type: ResizeByShort | ||
| short_size: [256, 307, 358, 409, 460, 512, 563, 614, 665, 716, 768, 819, 870, 921, 972, 1024] | ||
| max_size: 2048 | ||
| - type: RandomPaddingCrop | ||
| crop_size: [512, 512] | ||
| - type: RandomDistort | ||
| brightness_range: 0.125 | ||
| brightness_prob: 1.0 | ||
| contrast_range: 0.5 | ||
| contrast_prob: 1.0 | ||
| saturation_range: 0.5 | ||
| saturation_prob: 1.0 | ||
| hue_range: 18 | ||
| hue_prob: 1.0 | ||
| - type: RandomHorizontalFlip | ||
| - type: Padding | ||
| target_size: [512, 512] | ||
| im_padding_value: 128 | ||
| - type: Normalize | ||
| mean: [0.485, 0.456, 0.406] | ||
| std: [0.229, 0.224, 0.225] | ||
|
|
||
| val_dataset: | ||
| type: MaskedADE20K | ||
| dataset_root: data/ADEChallengeData2016/ | ||
| transforms: | ||
| - type: ResizeByShort | ||
| short_size: 512 | ||
| - type: Normalize | ||
| mean: [0.485, 0.456, 0.406] | ||
| std: [0.229, 0.224, 0.225] | ||
| mode: val | ||
|
|
||
| model: | ||
| type: MaskFormer | ||
| num_classes: 150 | ||
| backbone: | ||
| type: SwinTransformer_tiny_patch4_window7_224_maskformer | ||
| pretrained: https://bj.bcebos.com/paddleseg/paddleseg/dygraph/ade20k/maskformer_ade20k_swin_tiny/pretrain/model.pdparams | ||
|
|
||
| optimizer: | ||
| type: AdamW | ||
| weight_decay: 0.01 | ||
| custom_cfg: | ||
| - name: backbone | ||
| lr_mult: 1.0 | ||
| - name: norm | ||
| weight_decay_mult: 0.0 | ||
| - name: relative_position_bias_table | ||
| weight_decay_mult: 0.0 | ||
| grad_clip_cfg: | ||
| name: ClipGradByNorm | ||
| clip_norm: 0.01 | ||
|
|
||
| lr_scheduler: | ||
| type: PolynomialDecay | ||
| warmup_iters: 1500 | ||
| warmup_start_lr: 6.0e-11 | ||
| learning_rate: 6.0e-05 | ||
| end_lr: 0 | ||
| power: 0.9 | ||
|
|
||
| loss: | ||
| types: | ||
| - type: MaskFormerLoss | ||
| num_classes: 150 | ||
| eos_coef: 0.1 | ||
| coef: [1] |
Oops, something went wrong.
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
前面已经有RandomPaddingCrop，不需要padding了吧。
如果四个yml文件只是model不一样，一个yml文件作为base，其中三个yml文件可以使__base__包含。
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
已删除，除了model还有训练数据处理不一样，目前已经修改为最大化继承。