[Bug] mask_rcnn got all loss=0.000 for custom COCO dataset with RLE instance segmentation encoding

[kaixin-bai]

Prerequisite

• I have searched Issues and Discussions but cannot get the expected help.
• I have read the FAQ documentation but cannot get the expected help.
• The bug has not been fixed in the latest version (master) or latest version (3.x).

Task

I'm using the official example scripts/configs for the officially supported tasks/models/datasets.

Branch

master branch https://github.com/open-mmlab/mmdetection

Environment

mmdet                   2.25.2

Reproduces the problem - code sample

I build a custom COCO dataset with 1 class, and the instance segmentation is RLE encoding with "iscrowd:1":

"annotations": [
        {
            "category_id": 1,
            "image_id": 900,
            "iscrowd": 1,
            "segmentation": {
                "counts": [
                    2043410,
                    6,
                    1,
                    11,
                    .......... (too many numbers, ignore with this line)
                    9,
                    974843
                ],
                "size": [
                    1544,
                    2064
                ]
            },
            "bbox": [
                1323.0,
                684.0,
                110.0,
                288.0
            ],
            "area": 23775.0,
            "id": 4611
        },

I also check the validation of my dataset with pycocotools, the visualization shows that there's no problem for my dataset:

Reproduces the problem - command or script

(mmdet) kb@gpu01:/data-r10/kb/Projects/SynDataGen/mmdetection$ python3 tools/train.py configs/mask_rcnn/mask_rcnn_r50_caffe_fpn_mstrain-poly_1x_mps1texture_coco.py 

Reproduces the problem - error message

/data-r10/kb/Projects/SynDataGen/mmdetection/mmdet/utils/setup_env.py:39: UserWarning: Setting OMP_NUM_THREADS environment variable for each process to be 1 in default, to avoid your system being overloaded, please further tune the variable for optimal performance in your application as needed.
  f'Setting OMP_NUM_THREADS environment variable for each process '
/data-r10/kb/Projects/SynDataGen/mmdetection/mmdet/utils/setup_env.py:49: UserWarning: Setting MKL_NUM_THREADS environment variable for each process to be 1 in default, to avoid your system being overloaded, please further tune the variable for optimal performance in your application as needed.
  f'Setting MKL_NUM_THREADS environment variable for each process '
2022-11-02 21:24:52,354 - mmdet - INFO - Environment info:
------------------------------------------------------------
sys.platform: linux
Python: 3.6.13 |Anaconda, Inc.| (default, Jun  4 2021, 14:25:59) [GCC 7.5.0]
CUDA available: True
GPU 0: NVIDIA GeForce GTX 1080 Ti
CUDA_HOME: None
GCC: gcc (Ubuntu 7.5.0-3ubuntu1~18.04) 7.5.0
PyTorch: 1.10.1+cu102
PyTorch compiling details: PyTorch built with:
  - GCC 7.3
  - C++ Version: 201402
  - Intel(R) Math Kernel Library Version 2020.0.0 Product Build 20191122 for Intel(R) 64 architecture applications
  - Intel(R) MKL-DNN v2.2.3 (Git Hash 7336ca9f055cf1bfa13efb658fe15dc9b41f0740)
  - OpenMP 201511 (a.k.a. OpenMP 4.5)
  - LAPACK is enabled (usually provided by MKL)
  - NNPACK is enabled
  - CPU capability usage: AVX512
  - CUDA Runtime 10.2
  - NVCC architecture flags: -gencode;arch=compute_37,code=sm_37;-gencode;arch=compute_50,code=sm_50;-gencode;arch=compute_60,code=sm_60;-gencode;arch=compute_70,code=sm_70
  - CuDNN 7.6.5
  - Magma 2.5.2
  - Build settings: BLAS_INFO=mkl, BUILD_TYPE=Release, CUDA_VERSION=10.2, CUDNN_VERSION=7.6.5, CXX_COMPILER=/opt/rh/devtoolset-7/root/usr/bin/c++, CXX_FLAGS= -Wno-deprecated -fvisibility-inlines-hidden -DUSE_PTHREADPOOL -fopenmp -DNDEBUG -DUSE_KINETO -DUSE_FBGEMM -DUSE_QNNPACK -DUSE_PYTORCH_QNNPACK -DUSE_XNNPACK -DSYMBOLICATE_MOBILE_DEBUG_HANDLE -DEDGE_PROFILER_USE_KINETO -O2 -fPIC -Wno-narrowing -Wall -Wextra -Werror=return-type -Wno-missing-field-initializers -Wno-type-limits -Wno-array-bounds -Wno-unknown-pragmas -Wno-sign-compare -Wno-unused-parameter -Wno-unused-variable -Wno-unused-function -Wno-unused-result -Wno-unused-local-typedefs -Wno-strict-overflow -Wno-strict-aliasing -Wno-error=deprecated-declarations -Wno-stringop-overflow -Wno-psabi -Wno-error=pedantic -Wno-error=redundant-decls -Wno-error=old-style-cast -fdiagnostics-color=always -faligned-new -Wno-unused-but-set-variable -Wno-maybe-uninitialized -fno-math-errno -fno-trapping-math -Werror=format -Wno-stringop-overflow, LAPACK_INFO=mkl, PERF_WITH_AVX=1, PERF_WITH_AVX2=1, PERF_WITH_AVX512=1, TORCH_VERSION=1.10.1, USE_CUDA=ON, USE_CUDNN=ON, USE_EXCEPTION_PTR=1, USE_GFLAGS=OFF, USE_GLOG=OFF, USE_MKL=ON, USE_MKLDNN=ON, USE_MPI=OFF, USE_NCCL=ON, USE_NNPACK=ON, USE_OPENMP=ON, 

TorchVision: 0.11.2+cu102
OpenCV: 4.6.0
MMCV: 1.6.2
MMCV Compiler: GCC 7.3
MMCV CUDA Compiler: 10.2
MMDetection: 2.25.2+9d3e162
------------------------------------------------------------

2022-11-02 21:24:53,575 - mmdet - INFO - Distributed training: False
2022-11-02 21:24:54,769 - mmdet - INFO - Config:
model = dict(
    type='MaskRCNN',
    backbone=dict(
        type='ResNet',
        depth=50,
        num_stages=4,
        out_indices=(0, 1, 2, 3),
        frozen_stages=1,
        norm_cfg=dict(type='BN', requires_grad=False),
        norm_eval=True,
        style='caffe',
        init_cfg=dict(
            type='Pretrained',
            checkpoint='open-mmlab://detectron2/resnet50_caffe')),
    neck=dict(
        type='FPN',
        in_channels=[256, 512, 1024, 2048],
        out_channels=256,
        num_outs=5),
    rpn_head=dict(
        type='RPNHead',
        in_channels=256,
        feat_channels=256,
        anchor_generator=dict(
            type='AnchorGenerator',
            scales=[8],
            ratios=[0.5, 1.0, 2.0],
            strides=[4, 8, 16, 32, 64]),
        bbox_coder=dict(
            type='DeltaXYWHBBoxCoder',
            target_means=[0.0, 0.0, 0.0, 0.0],
            target_stds=[1.0, 1.0, 1.0, 1.0]),
        loss_cls=dict(
            type='CrossEntropyLoss', use_sigmoid=True, loss_weight=1.0),
        loss_bbox=dict(type='L1Loss', loss_weight=1.0)),
    roi_head=dict(
        type='StandardRoIHead',
        bbox_roi_extractor=dict(
            type='SingleRoIExtractor',
            roi_layer=dict(type='RoIAlign', output_size=7, sampling_ratio=0),
            out_channels=256,
            featmap_strides=[4, 8, 16, 32]),
        bbox_head=dict(
            type='Shared2FCBBoxHead',
            in_channels=256,
            fc_out_channels=1024,
            roi_feat_size=7,
            num_classes=1,
            bbox_coder=dict(
                type='DeltaXYWHBBoxCoder',
                target_means=[0.0, 0.0, 0.0, 0.0],
                target_stds=[0.1, 0.1, 0.2, 0.2]),
            reg_class_agnostic=False,
            loss_cls=dict(
                type='CrossEntropyLoss', use_sigmoid=False, loss_weight=1.0),
            loss_bbox=dict(type='L1Loss', loss_weight=1.0)),
        mask_roi_extractor=dict(
            type='SingleRoIExtractor',
            roi_layer=dict(type='RoIAlign', output_size=14, sampling_ratio=0),
            out_channels=256,
            featmap_strides=[4, 8, 16, 32]),
        mask_head=dict(
            type='FCNMaskHead',
            num_convs=4,
            in_channels=256,
            conv_out_channels=256,
            num_classes=1,
            loss_mask=dict(
                type='CrossEntropyLoss', use_mask=True, loss_weight=1.0))),
    train_cfg=dict(
        rpn=dict(
            assigner=dict(
                type='MaxIoUAssigner',
                pos_iou_thr=0.7,
                neg_iou_thr=0.3,
                min_pos_iou=0.3,
                match_low_quality=True,
                ignore_iof_thr=-1),
            sampler=dict(
                type='RandomSampler',
                num=256,
                pos_fraction=0.5,
                neg_pos_ub=-1,
                add_gt_as_proposals=False),
            allowed_border=-1,
            pos_weight=-1,
            debug=False),
        rpn_proposal=dict(
            nms_pre=2000,
            max_per_img=1000,
            nms=dict(type='nms', iou_threshold=0.7),
            min_bbox_size=0),
        rcnn=dict(
            assigner=dict(
                type='MaxIoUAssigner',
                pos_iou_thr=0.5,
                neg_iou_thr=0.5,
                min_pos_iou=0.5,
                match_low_quality=True,
                ignore_iof_thr=-1),
            sampler=dict(
                type='RandomSampler',
                num=512,
                pos_fraction=0.25,
                neg_pos_ub=-1,
                add_gt_as_proposals=True),
            mask_size=28,
            pos_weight=-1,
            debug=False)),
    test_cfg=dict(
        rpn=dict(
            nms_pre=1000,
            max_per_img=1000,
            nms=dict(type='nms', iou_threshold=0.7),
            min_bbox_size=0),
        rcnn=dict(
            score_thr=0.05,
            nms=dict(type='nms', iou_threshold=0.5),
            max_per_img=100,
            mask_thr_binary=0.5)))
dataset_type = 'CocoDataset'
data_root = '/data-r10/kb/Projects/SynDataGen/datasets/mps1/'
img_norm_cfg = dict(
    mean=[103.53, 116.28, 123.675], std=[1.0, 1.0, 1.0], to_rgb=False)
train_pipeline = [
    dict(type='LoadImageFromFile'),
    dict(
        type='LoadAnnotations',
        with_bbox=True,
        with_mask=True,
        poly2mask=False),
    dict(
        type='Resize',
        img_scale=[(1333, 640), (1333, 672), (1333, 704), (1333, 736),
                   (1333, 768), (1333, 800)],
        multiscale_mode='value',
        keep_ratio=True),
    dict(type='RandomFlip', flip_ratio=0.5),
    dict(
        type='Normalize',
        mean=[103.53, 116.28, 123.675],
        std=[1.0, 1.0, 1.0],
        to_rgb=False),
    dict(type='Pad', size_divisor=32),
    dict(type='DefaultFormatBundle'),
    dict(type='Collect', keys=['img', 'gt_bboxes', 'gt_labels', 'gt_masks'])
]
test_pipeline = [
    dict(type='LoadImageFromFile'),
    dict(
        type='MultiScaleFlipAug',
        img_scale=(1333, 800),
        flip=False,
        transforms=[
            dict(type='Resize', keep_ratio=True),
            dict(type='RandomFlip'),
            dict(
                type='Normalize',
                mean=[103.53, 116.28, 123.675],
                std=[1.0, 1.0, 1.0],
                to_rgb=False),
            dict(type='Pad', size_divisor=32),
            dict(type='ImageToTensor', keys=['img']),
            dict(type='Collect', keys=['img'])
        ])
]
data = dict(
    samples_per_gpu=2,
    workers_per_gpu=2,
    train=dict(
        type='CocoDataset',
        ann_file='/data-r10/kb/Projects/SynDataGen/datasets/mps1/train.json',
        img_prefix=
        '/data-r10/kb/Projects/SynDataGen/datasets/mps1/train_texture/',
        pipeline=[
            dict(type='LoadImageFromFile'),
            dict(
                type='LoadAnnotations',
                with_bbox=True,
                with_mask=True,
                poly2mask=False),
            dict(
                type='Resize',
                img_scale=[(1333, 640), (1333, 672), (1333, 704), (1333, 736),
                           (1333, 768), (1333, 800)],
                multiscale_mode='value',
                keep_ratio=True),
            dict(type='RandomFlip', flip_ratio=0.5),
            dict(
                type='Normalize',
                mean=[103.53, 116.28, 123.675],
                std=[1.0, 1.0, 1.0],
                to_rgb=False),
            dict(type='Pad', size_divisor=32),
            dict(type='DefaultFormatBundle'),
            dict(
                type='Collect',
                keys=['img', 'gt_bboxes', 'gt_labels', 'gt_masks'])
        ]),
    val=dict(
        type='CocoDataset',
        ann_file='/data-r10/kb/Projects/SynDataGen/datasets/mps1/val.json',
        img_prefix=
        '/data-r10/kb/Projects/SynDataGen/datasets/mps1/val_texture/',
        pipeline=[
            dict(type='LoadImageFromFile'),
            dict(
                type='MultiScaleFlipAug',
                img_scale=(1333, 800),
                flip=False,
                transforms=[
                    dict(type='Resize', keep_ratio=True),
                    dict(type='RandomFlip'),
                    dict(
                        type='Normalize',
                        mean=[103.53, 116.28, 123.675],
                        std=[1.0, 1.0, 1.0],
                        to_rgb=False),
                    dict(type='Pad', size_divisor=32),
                    dict(type='ImageToTensor', keys=['img']),
                    dict(type='Collect', keys=['img'])
                ])
        ]),
    test=dict(
        type='CocoDataset',
        ann_file='/data-r10/kb/Projects/SynDataGen/datasets/mps1/json.json',
        img_prefix=
        '/data-r10/kb/Projects/SynDataGen/datasets/mps1/val_texture/',
        pipeline=[
            dict(type='LoadImageFromFile'),
            dict(
                type='MultiScaleFlipAug',
                img_scale=(1333, 800),
                flip=False,
                transforms=[
                    dict(type='Resize', keep_ratio=True),
                    dict(type='RandomFlip'),
                    dict(
                        type='Normalize',
                        mean=[103.53, 116.28, 123.675],
                        std=[1.0, 1.0, 1.0],
                        to_rgb=False),
                    dict(type='Pad', size_divisor=32),
                    dict(type='ImageToTensor', keys=['img']),
                    dict(type='Collect', keys=['img'])
                ])
        ]))
evaluation = dict(metric=['bbox', 'segm'])
optimizer = dict(type='SGD', lr=0.02, momentum=0.9, weight_decay=0.0001)
optimizer_config = dict(grad_clip=None)
lr_config = dict(
    policy='step',
    warmup='linear',
    warmup_iters=500,
    warmup_ratio=0.001,
    step=[8, 11])
runner = dict(type='EpochBasedRunner', max_epochs=12)
checkpoint_config = dict(interval=1)
log_config = dict(
    interval=1,
    hooks=[dict(type='TextLoggerHook'),
           dict(type='TensorboardLoggerHook')])
custom_hooks = [dict(type='NumClassCheckHook')]
dist_params = dict(backend='nccl')
log_level = 'INFO'
load_from = None
resume_from = None
workflow = [('train', 1)]
opencv_num_threads = 0
mp_start_method = 'fork'
auto_scale_lr = dict(enable=False, base_batch_size=16)
work_dir = './work_dirs/mask_rcnn_r50_caffe_fpn_mstrain-poly_1x_mps1texture_coco'
auto_resume = False
gpu_ids = [0]

2022-11-02 21:24:54,769 - mmdet - INFO - Set random seed to 695735717, deterministic: False
2022-11-02 21:24:56,621 - mmdet - INFO - initialize ResNet with init_cfg {'type': 'Pretrained', 'checkpoint': 'open-mmlab://detectron2/resnet50_caffe'}
2022-11-02 21:24:56,622 - mmcv - INFO - load model from: open-mmlab://detectron2/resnet50_caffe
2022-11-02 21:24:56,622 - mmcv - INFO - load checkpoint from openmmlab path: open-mmlab://detectron2/resnet50_caffe
2022-11-02 21:24:57,748 - mmcv - WARNING - The model and loaded state dict do not match exactly

unexpected key in source state_dict: conv1.bias

2022-11-02 21:24:59,876 - mmdet - INFO - initialize FPN with init_cfg {'type': 'Xavier', 'layer': 'Conv2d', 'distribution': 'uniform'}
2022-11-02 21:25:00,280 - mmdet - INFO - initialize RPNHead with init_cfg {'type': 'Normal', 'layer': 'Conv2d', 'std': 0.01}
2022-11-02 21:25:00,328 - mmdet - INFO - initialize Shared2FCBBoxHead with init_cfg [{'type': 'Normal', 'std': 0.01, 'override': {'name': 'fc_cls'}}, {'type': 'Normal', 'std': 0.001, 'override': {'name': 'fc_reg'}}, {'type': 'Xavier', 'distribution': 'uniform', 'override': [{'name': 'shared_fcs'}, {'name': 'cls_fcs'}, {'name': 'reg_fcs'}]}]
loading annotations into memory...
Done (t=6.19s)
creating index...
index created!
2022-11-02 21:25:11,138 - mmdet - INFO - Automatic scaling of learning rate (LR) has been disabled.
loading annotations into memory...
Done (t=1.61s)
creating index...
index created!
2022-11-02 21:25:12,764 - mmdet - INFO - Start running, host: kb@ar-gpu01, work_dir: /data-r10/kb/Projects/SynDataGen/mmdetection/work_dirs/mask_rcnn_r50_caffe_fpn_mstrain-poly_1x_mps1texture_coco
2022-11-02 21:25:12,764 - mmdet - INFO - Hooks will be executed in the following order:
before_run:
(VERY_HIGH   ) StepLrUpdaterHook                  
(NORMAL      ) CheckpointHook                     
(LOW         ) EvalHook                           
(VERY_LOW    ) TextLoggerHook                     
(VERY_LOW    ) TensorboardLoggerHook              
 -------------------- 
before_train_epoch:
(VERY_HIGH   ) StepLrUpdaterHook                  
(NORMAL      ) NumClassCheckHook                  
(LOW         ) IterTimerHook                      
(LOW         ) EvalHook                           
(VERY_LOW    ) TextLoggerHook                     
(VERY_LOW    ) TensorboardLoggerHook              
 -------------------- 
before_train_iter:
(VERY_HIGH   ) StepLrUpdaterHook                  
(LOW         ) IterTimerHook                      
(LOW         ) EvalHook                           
 -------------------- 
after_train_iter:
(ABOVE_NORMAL) OptimizerHook                      
(NORMAL      ) CheckpointHook                     
(LOW         ) IterTimerHook                      
(LOW         ) EvalHook                           
(VERY_LOW    ) TextLoggerHook                     
(VERY_LOW    ) TensorboardLoggerHook              
 -------------------- 
after_train_epoch:
(NORMAL      ) CheckpointHook                     
(LOW         ) EvalHook                           
(VERY_LOW    ) TextLoggerHook                     
(VERY_LOW    ) TensorboardLoggerHook              
 -------------------- 
before_val_epoch:
(NORMAL      ) NumClassCheckHook                  
(LOW         ) IterTimerHook                      
(VERY_LOW    ) TextLoggerHook                     
(VERY_LOW    ) TensorboardLoggerHook              
 -------------------- 
before_val_iter:
(LOW         ) IterTimerHook                      
 -------------------- 
after_val_iter:
(LOW         ) IterTimerHook                      
 -------------------- 
after_val_epoch:
(VERY_LOW    ) TextLoggerHook                     
(VERY_LOW    ) TensorboardLoggerHook              
 -------------------- 
after_run:
(VERY_LOW    ) TextLoggerHook                     
(VERY_LOW    ) TensorboardLoggerHook              
 -------------------- 
2022-11-02 21:25:12,764 - mmdet - INFO - workflow: [('train', 1)], max: 12 epochs
2022-11-02 21:25:12,765 - mmdet - INFO - Checkpoints will be saved to /data-r10/kb/Projects/SynDataGen/mmdetection/work_dirs/mask_rcnn_r50_caffe_fpn_mstrain-poly_1x_mps1texture_coco by HardDiskBackend.
2022-11-02 21:25:15,527 - mmdet - INFO - Epoch [1][1/450]	lr: 2.000e-05, eta: 3:56:26, time: 2.628, data_time: 2.292, memory: 2781, loss_rpn_cls: 0.6497, loss_rpn_bbox: 0.0000, loss_cls: 0.5660, acc: 75.5859, loss_bbox: 0.0000, loss_mask: 0.0000, loss: 1.2157
2022-11-02 21:25:15,836 - mmdet - INFO - Epoch [1][2/450]	lr: 5.996e-05, eta: 2:10:26, time: 0.272, data_time: 0.080, memory: 2781, loss_rpn_cls: 0.6493, loss_rpn_bbox: 0.0000, loss_cls: 0.5279, acc: 80.9570, loss_bbox: 0.0000, loss_mask: 0.0000, loss: 1.1772
2022-11-02 21:25:16,121 - mmdet - INFO - Epoch [1][3/450]	lr: 9.992e-05, eta: 1:35:51, time: 0.297, data_time: 0.116, memory: 2781, loss_rpn_cls: 0.6466, loss_rpn_bbox: 0.0000, loss_cls: 0.3724, acc: 96.4844, loss_bbox: 0.0000, loss_mask: 0.0000, loss: 1.0189
2022-11-02 21:25:16,434 - mmdet - INFO - Epoch [1][4/450]	lr: 1.399e-04, eta: 1:18:19, time: 0.287, data_time: 0.103, memory: 2875, loss_rpn_cls: 0.6418, loss_rpn_bbox: 0.0000, loss_cls: 0.1563, acc: 99.9023, loss_bbox: 0.0000, loss_mask: 0.0000, loss: 0.7981
2022-11-02 21:25:16,730 - mmdet - INFO - Epoch [1][5/450]	lr: 1.798e-04, eta: 1:08:20, time: 0.316, data_time: 0.130, memory: 2875, loss_rpn_cls: 0.6305, loss_rpn_bbox: 0.0000, loss_cls: 0.0428, acc: 100.0000, loss_bbox: 0.0000, loss_mask: 0.0000, loss: 0.6733
2022-11-02 21:25:17,000 - mmdet - INFO - Epoch [1][6/450]	lr: 2.198e-04, eta: 1:01:05, time: 0.277, data_time: 0.109, memory: 2875, loss_rpn_cls: 0.6223, loss_rpn_bbox: 0.0000, loss_cls: 0.0132, acc: 100.0000, loss_bbox: 0.0000, loss_mask: 0.0000, loss: 0.6354
2022-11-02 21:25:17,312 - mmdet - INFO - Epoch [1][7/450]	lr: 2.598e-04, eta: 0:56:10, time: 0.297, data_time: 0.101, memory: 2875, loss_rpn_cls: 0.5971, loss_rpn_bbox: 0.0000, loss_cls: 0.0055, acc: 100.0000, loss_bbox: 0.0000, loss_mask: 0.0000, loss: 0.6026
2022-11-02 21:25:17,598 - mmdet - INFO - Epoch [1][8/450]	lr: 2.997e-04, eta: 0:52:27, time: 0.295, data_time: 0.117, memory: 2875, loss_rpn_cls: 0.5806, loss_rpn_bbox: 0.0000, loss_cls: 0.0030, acc: 100.0000, loss_bbox: 0.0000, loss_mask: 0.0000, loss: 0.5836
2022-11-02 21:25:17,941 - mmdet - INFO - Epoch [1][9/450]	lr: 3.397e-04, eta: 0:49:45, time: 0.314, data_time: 0.108, memory: 3051, loss_rpn_cls: 0.5443, loss_rpn_bbox: 0.0000, loss_cls: 0.0018, acc: 100.0000, loss_bbox: 0.0000, loss_mask: 0.0000, loss: 0.5460
2022-11-02 21:25:18,271 - mmdet - INFO - Epoch [1][10/450]	lr: 3.796e-04, eta: 0:47:47, time: 0.337, data_time: 0.137, memory: 3051, loss_rpn_cls: 0.5179, loss_rpn_bbox: 0.0000, loss_cls: 0.0007, acc: 100.0000, loss_bbox: 0.0000, loss_mask: 0.0000, loss: 0.5186
2022-11-02 21:25:18,619 - mmdet - INFO - Epoch [1][11/450]	lr: 4.196e-04, eta: 0:46:13, time: 0.341, data_time: 0.133, memory: 3051, loss_rpn_cls: 0.4778, loss_rpn_bbox: 0.0000, loss_cls: 0.0004, acc: 100.0000, loss_bbox: 0.0000, loss_mask: 0.0000, loss: 0.4782
2022-11-02 21:25:18,961 - mmdet - INFO - Epoch [1][12/450]	lr: 4.596e-04, eta: 0:44:58, time: 0.348, data_time: 0.137, memory: 3051, loss_rpn_cls: 0.4473, loss_rpn_bbox: 0.0000, loss_cls: 0.0002, acc: 100.0000, loss_bbox: 0.0000, loss_mask: 0.0000, loss: 0.4475
2022-11-02 21:25:19,262 - mmdet - INFO - Epoch [1][13/450]	lr: 4.995e-04, eta: 0:43:40, time: 0.314, data_time: 0.132, memory: 3051, loss_rpn_cls: 0.3984, loss_rpn_bbox: 0.0000, loss_cls: 0.0001, acc: 100.0000, loss_bbox: 0.0000, loss_mask: 0.0000, loss: 0.3985
2022-11-02 21:25:19,600 - mmdet - INFO - Epoch [1][14/450]	lr: 5.395e-04, eta: 0:42:35, time: 0.319, data_time: 0.117, memory: 3051, loss_rpn_cls: 0.3340, loss_rpn_bbox: 0.0000, loss_cls: 0.0000, acc: 100.0000, loss_bbox: 0.0000, loss_mask: 0.0000, loss: 0.3341
2022-11-02 21:25:19,899 - mmdet - INFO - Epoch [1][15/450]	lr: 5.794e-04, eta: 0:41:41, time: 0.327, data_time: 0.137, memory: 3051, loss_rpn_cls: 0.2874, loss_rpn_bbox: 0.0000, loss_cls: 0.0000, acc: 100.0000, loss_bbox: 0.0000, loss_mask: 0.0000, loss: 0.2874
2022-11-02 21:25:20,180 - mmdet - INFO - Epoch [1][16/450]	lr: 6.194e-04, eta: 0:40:39, time: 0.282, data_time: 0.107, memory: 3051, loss_rpn_cls: 0.2382, loss_rpn_bbox: 0.0000, loss_cls: 0.0000, acc: 100.0000, loss_bbox: 0.0000, loss_mask: 0.0000, loss: 0.2383
2022-11-02 21:25:20,486 - mmdet - INFO - Epoch [1][17/450]	lr: 6.594e-04, eta: 0:39:50, time: 0.297, data_time: 0.107, memory: 3051, loss_rpn_cls: 0.2009, loss_rpn_bbox: 0.0000, loss_cls: 0.0000, acc: 100.0000, loss_bbox: 0.0000, loss_mask: 0.0000, loss: 0.2009
2022-11-02 21:25:20,818 - mmdet - INFO - Epoch [1][18/450]	lr: 6.993e-04, eta: 0:39:12, time: 0.319, data_time: 0.116, memory: 3051, loss_rpn_cls: 0.1716, loss_rpn_bbox: 0.0000, loss_cls: 0.0000, acc: 100.0000, loss_bbox: 0.0000, loss_mask: 0.0000, loss: 0.1716
2022-11-02 21:25:21,151 - mmdet - INFO - Epoch [1][19/450]	lr: 7.393e-04, eta: 0:38:40, time: 0.326, data_time: 0.130, memory: 3051, loss_rpn_cls: 0.1310, loss_rpn_bbox: 0.0000, loss_cls: 0.0000, acc: 100.0000, loss_bbox: 0.0000, loss_mask: 0.0000, loss: 0.1310
2022-11-02 21:25:21,418 - mmdet - INFO - Epoch [1][20/450]	lr: 7.792e-04, eta: 0:38:06, time: 0.309, data_time: 0.137, memory: 3051, loss_rpn_cls: 0.0997, loss_rpn_bbox: 0.0000, loss_cls: 0.0000, acc: 100.0000, loss_bbox: 0.0000, loss_mask: 0.0000, loss: 0.0997
2022-11-02 21:25:21,745 - mmdet - INFO - Epoch [1][21/450]	lr: 8.192e-04, eta: 0:37:32, time: 0.292, data_time: 0.094, memory: 3051, loss_rpn_cls: 0.0846, loss_rpn_bbox: 0.0000, loss_cls: 0.0000, acc: 100.0000, loss_bbox: 0.0000, loss_mask: 0.0000, loss: 0.0846
2022-11-02 21:25:22,047 - mmdet - INFO - Epoch [1][22/450]	lr: 8.592e-04, eta: 0:37:08, time: 0.324, data_time: 0.130, memory: 3051, loss_rpn_cls: 0.0786, loss_rpn_bbox: 0.0000, loss_cls: 0.0000, acc: 100.0000, loss_bbox: 0.0000, loss_mask: 0.0000, loss: 0.0786

 .............

2022-11-02 21:37:28,074 - mmdet - INFO - Epoch [1][109/450]	lr: 4.336e-03, eta: 0:30:03, time: 0.318, data_time: 0.131, memory: 3056, loss_rpn_cls: 0.0000, loss_rpn_bbox: 0.0000, loss_cls: 0.0000, acc: 100.0000, loss_bbox: 0.0000, loss_mask: 0.0000, loss: 0.0000
2022-11-02 21:37:28,426 - mmdet - INFO - Epoch [1][110/450]	lr: 4.376e-03, eta: 0:30:03, time: 0.341, data_time: 0.118, memory: 3056, loss_rpn_cls: 0.0000, loss_rpn_bbox: 0.0000, loss_cls: 0.0000, acc: 100.0000, loss_bbox: 0.0000, loss_mask: 0.0000, loss: 0.0000
2022-11-02 21:37:28,760 - mmdet - INFO - Epoch [1][111/450]	lr: 4.416e-03, eta: 0:30:03, time: 0.347, data_time: 0.140, memory: 3056, loss_rpn_cls: 0.0000, loss_rpn_bbox: 0.0000, loss_cls: 0.0000, acc: 100.0000, loss_bbox: 0.0000, loss_mask: 0.0000, loss: 0.0000
2022-11-02 21:37:29,066 - mmdet - INFO - Epoch [1][112/450]	lr: 4.456e-03, eta: 0:30:01, time: 0.314, data_time: 0.119, memory: 3056, loss_rpn_cls: 0.0000, loss_rpn_bbox: 0.0000, loss_cls: 0.0000, acc: 100.0000, loss_bbox: 0.0000, loss_mask: 0.0000, loss: 0.0000
2022-11-02 21:37:29,395 - mmdet - INFO - Epoch [1][113/450]	lr: 4.496e-03, eta: 0:29:59, time: 0.308, data_time: 0.109, memory: 3056, loss_rpn_cls: 0.0000, loss_rpn_bbox: 0.0000, loss_cls: 0.0000, acc: 100.0000, loss_bbox: 0.0000, loss_mask: 0.0000, loss: 0.0000
2022-11-02 21:37:29,687 - mmdet - INFO - Epoch [1][114/450]	lr: 4.535e-03, eta: 0:29:58, time: 0.314, data_time: 0.131, memory: 3056, loss_rpn_cls: 0.0000, loss_rpn_bbox: 0.0000, loss_cls: 0.0000, acc: 100.0000, loss_bbox: 0.0000, loss_mask: 0.0000, loss: 0.0000
2022-11-02 21:37:30,018 - mmdet - INFO - Epoch [1][115/450]	lr: 4.575e-03, eta: 0:29:56, time: 0.309, data_time: 0.109, memory: 3056, loss_rpn_cls: 0.0000, loss_rpn_bbox: 0.0000, loss_cls: 0.0000, acc: 100.0000, loss_bbox: 0.0000, loss_mask: 0.0000, loss: 0.0000
2022-11-02 21:37:30,303 - mmdet - INFO - Epoch [1][116/450]	lr: 4.615e-03, eta: 0:29:54, time: 0.313, data_time: 0.131, memory: 3056, loss_rpn_cls: 0.0000, loss_rpn_bbox: 0.0000, loss_cls: 0.0000, acc: 100.0000, loss_bbox: 0.0000, loss_mask: 0.0000, loss: 0.0000
2022-11-02 21:37:30,610 - mmdet - INFO - Epoch [1][117/450]	lr: 4.655e-03, eta: 0:29:52, time: 0.293, data_time: 0.102, memory: 3056, loss_rpn_cls: 0.0000, loss_rpn_bbox: 0.0000, loss_cls: 0.0000, acc: 100.0000, loss_bbox: 0.0000, loss_mask: 0.0000, loss: 0.0000
2022-11-02 21:37:30,875 - mmdet - INFO - Epoch [1][118/450]	lr: 4.695e-03, eta: 0:29:49, time: 0.286, data_time: 0.116, memory: 3056, loss_rpn_cls: 0.0000, loss_rpn_bbox: 0.0000, loss_cls: 0.0000, acc: 100.0000, loss_bbox: 0.0000, loss_mask: 0.0000, loss: 0.0000
2022-11-02 21:37:31,182 - mmdet - INFO - Epoch [1][119/450]	lr: 4.735e-03, eta: 0:29:47, time: 0.287, data_time: 0.095, memory: 3056, loss_rpn_cls: 0.0000, loss_rpn_bbox: 0.0000, loss_cls: 0.0000, acc: 100.0000, loss_bbox: 0.0000, loss_mask: 0.0000, loss: 0.0000
2022-11-02 21:37:31,471 - mmdet - INFO - Epoch [1][120/450]	lr: 4.775e-03, eta: 0:29:44, time: 0.297, data_time: 0.117, memory: 3056, loss_rpn_cls: 0.0000, loss_rpn_bbox: 0.0000, loss_cls: 0.0000, acc: 100.0000, loss_bbox: 0.0000, loss_mask: 0.0000, loss: 0.0000
2022-11-02 21:37:31,809 - mmdet - INFO - Epoch [1][121/450]	lr: 4.815e-03, eta: 0:29:43, time: 0.311, data_time: 0.108, memory: 3056, loss_rpn_cls: 0.0001, loss_rpn_bbox: 0.0000, loss_cls: 0.0000, acc: 100.0000, loss_bbox: 0.0000, loss_mask: 0.0000, loss: 0.0001
2022-11-02 21:37:32,088 - mmdet - INFO - Epoch [1][122/450]	lr: 4.855e-03, eta: 0:29:41, time: 0.311, data_time: 0.137, memory: 3056, loss_rpn_cls: 0.0001, loss_rpn_bbox: 0.0000, loss_cls: 0.0000, acc: 100.0000, loss_bbox: 0.0000, loss_mask: 0.0000, loss: 0.0001

Additional information

1. Does mmdetection support rle for mask_rcnn?
2. Why is the loss all zero?

[BIGWangYuDong]

1. mmdetection support load rle type
2. You can first debug your code that whether gt_mask is normal, Then, you can have a look that because acc is 100, maybe you meet overfit problem. which means the network have fully detect training dataset.

[kaixin-bai]

pycocotools has problems for RLE support, do not use iscrowd=1, change it to 0

[Huxufeng666]

thank you very much

[kaixin-bai]

In https://github.com/cocodataset/cocoapi/blob/master/PythonAPI/pycocotools/cocoeval.py#L109 you could find that:

gt['ignore'] = 'iscrowd' in gt and gt['iscrowd']

That means if the parameter iscrowd is True, the instances in your dataset would anyway be ignored.

If i fix the problem in pycocotools, my dataset works fine in paddledetection(this package also need to be modified for support of RLE and iscrowd), but still cannot run with mmdetection.

[sasniper007]

pycocotools has problems for RLE support, do not use iscrowd=1, change it to 0

In annotations/xxx.json, iscrowd=1 can be changed to 0 for RLE. It works well!