ISIC-2024

4th place for ISIC 2024 - Skin Cancer Detection Challenge

Summary

1. Environment

• Ubuntu 22.04 LTS
• CUDA 12.1
• Nvidia Driver Version: 535.161.07
• Python 3.10.13
• GPU: 40GB, RAM: 64GB

conda create -n venv python=3.10.13
conda activate venv
conda install pytorch=2.1.2 torchvision=0.16.2 torchaudio=2.1.2 pytorch-cuda=12.1 -c pytorch -c nvidia
pip install -r requirements.txt

2. Data preparation

• Download competition dataset and extract to folder ./dataset/isic_2024
• Download isic 2020 dataset and extract to folder ./dataset/isic_2020
• Download isic 2019 dataset and extract to folder ./dataset/isic_2019
• Download isic 2018 dataset and extract to folder ./dataset/isic_2018
• Download PAD UFES 20 dataset and extract to folder ./dataset/PAD-UFES-20
• dataset structure should be ./dataset/dataset_structure.txt
• Then run following scripts

cd src/prepare
python split_2024.py
python split_2020.py
python split_2019.py
python split_2018.py
python split_pad_ufes_20.py
python prepare_isic_2024_tabular_v2.py

3. Train models

3.1 Image pipeline

3.1.1 Multi-label classification exp0

• 4 multi-label classification models trained with ISIC 2024+2020+2019 + PAD UFES, validated with ISIC 2024

cd src/image_exp0
### For Leaderboard Prize
python train.py --cfg configs/swin_tiny_224.yaml && python predict_oof.py --cfg configs/swin_tiny_224.yaml
python train.py --cfg configs/convnextv2_base_128.yaml && python predict_oof.py --cfg configs/convnextv2_base_128.yaml
python train.py --cfg configs/convnextv2_large_64.yaml && python predict_oof.py --cfg configs/convnextv2_large_64.yaml
python train.py --cfg configs/coatnet_rmlp_1_224.yaml && python predict_oof.py --cfg configs/coatnet_rmlp_1_224.yaml
python train_ema.py --cfg configs/swin_tiny_224.yaml
python train_ema.py --cfg configs/convnextv2_base_128.yaml
python train_ema.py --cfg configs/convnextv2_large_64.yaml
python train_ema.py --cfg configs/coatnet_rmlp_1_224.yaml

### For Secondary Prizes
python train.py --cfg configs/vit_tiny_224.yaml && python predict_oof.py --cfg configs/vit_tiny_224.yaml
python train_ema.py --cfg configs/vit_tiny_224.yaml

• Result

Backbone	Image size	CV pAUC
swin_tiny	224	0.1609
convnextv2_base	128	0.1641
convnextv2_large	64	0.1642
coatnet_rmlp_1	224	0.1617
vit_tiny	224	0.1640

3.1.2 Lesion segmentation

• To create masks for aux models (segmentation + multi-label classification) exp1, I trained 3 models with ISIC 2018 data

cd src/lesion_segmentation
python train.py --cfg configs/eb5_unet++.yaml
python train.py --cfg configs/eb7_unet++.yaml
python train.py --cfg configs/mit_b5_fpn.yaml

• Result

Backbone	Decoder	Image size	IoU
efficientnet-b5	Unet++	512	0.827
efficientnet-b7	Unet++	256	0.829
mit-b5	FPN	512	0.843

• Then run following scripts to create mask

python predict_2024.py && python predict_2020.py && python predict_2019.py && python predict_pad_ufes.py

3.1.3 Multi-task classification + segmentation

• 3 multi-task segmentation + classification models trained with ISIC 2024+2020+2019 + PAD UFES, validated with ISIC 2024. For the submission, I only used the prediction from the classification task.

cd src/image_exp1_aux
### For Leaderboard Prize
python train.py --cfg configs/eb3_224.yaml && python predict_oof.py --cfg configs/eb3_224.yaml
python train.py --cfg configs/mit_b0_384.yaml && python predict_oof.py --cfg configs/mit_b0_384.yaml
python train.py --cfg configs/mit_b5_224.yaml && python predict_oof.py --cfg configs/mit_b5_224.yaml
python train_ema.py --cfg configs/eb3_224.yaml
python train_ema.py --cfg configs/mit_b0_384.yaml
python train_ema.py --cfg configs/mit_b5_224.yaml

### For Secondary Prizes
python train.py --cfg configs/mit_b0_224.yaml && python predict_oof.py --cfg configs/mit_b0_224.yaml
python train_ema.py --cfg configs/mit_b0_224.yaml

• Result

Backbone	Decoder	Image size	CV pAUC
efficientnet-b3	Unet	224	0.1638
mit-b0	FPN	384	0.1671
mit-b5	FPN	224	0.1656
mit-b0	FPN	224	0.1660

3.1.4 Multi-label classification exp3

• 3 multi-label classification models trained only with ISIC 2024 data

cd src/image_exp3
python train.py --cfg configs/vit_tiny_384.yaml && python predict_oof.py --cfg configs/vit_tiny_384.yaml
python train.py --cfg configs/swin_tiny_256.yaml && python predict_oof.py --cfg configs/swin_tiny_256.yaml
python train.py --cfg configs/convnextv2_tiny_288.yaml && python predict_oof.py --cfg configs/convnextv2_tiny_288.yaml
python train_ema.py --cfg configs/vit_tiny_384.yaml
python train_ema.py --cfg configs/swin_tiny_256.yaml
python train_ema.py --cfg configs/convnextv2_tiny_288.yaml

• Result

Backbone	Image size	CV pAUC
vit_tiny	384	0.1688
swin_tiny	256	0.1655
convnextv2_tiny	288	0.1645

3.2 Tabular pipeline

I used 3 models: LightGBM, CatBoost, XGBoost. I combined 10 features from the image pipeline and all the features from the amazing tabular notebook

cd src/tabular
python train_tab_meta_feat.py
### For Leaderboard Prize
python train_10_model_feat.py
### For Secondary Prizes
python train_2_model_feat.py

• Result

	LGB pAUC	CB pAUC	XGB pAUC	Gmean pAUC
meta feat	0.17806	0.17498	0.17954	0.17879
10model feat	0.18650	0.18669	0.18647	0.18703
2model feat	0.18406	0.18378	0.18328	0.18438

4.FINAL SUBMISSION

	PublicLB	PrivateLB	Prize
Sub1-GPU: 0.2*(meta only) + 0.8*(10model feat)	0.18229	0.17225	4th place leaderboard prize
Sub2-CPU: 0.2*(meta only) + 0.8*(2model feat)	0.18094	0.17011	Top-15 retrieval sensitivity prize