Train CIFAR10,CIFAR100 with Pytorch-lightning

Measure Out-of-Distribution Detection using several methods with pytorch-lightning. Methods include ODIN etc.

Requirements

• setup/requirements.txt

torch 1.5.1
torchvision 0.6.1
pytorch-lightning 0.9.0rc5
tqdm
argparse
pytablewriter
seaborn
enum34
scipy
cffi
sklearn

• install requirements using pip

pip3 install -r setup/requirements.txt

How to train models

After you have cloned the repository, you can train each models with datasets cifar10, cifar100, SVHN. Trainable models are VGG, Resnet, WideResnet, Densenet-BC, Densenet.

python train.py

How to measure Out-of-Distribution Detection

After you train models, run main.py with several arguments.

# in-distribution : CIFAR10, out-distribution : LSUN, model : Densenet-BC
python main.py --in_dataset="CIFAR10" --out_dataset="LSUN" --nn="Densetnet_BC"

Detection results

• Densenet-BC

Methods	In-dist	Out-dist	FPR at TPR 95%	Detection Error	AUROC	AUPR In	AUPR Out
Baseline	CIFAR	LSUN	38.4%	21.5%	94.5%	95.7%	93.2%
ODIN	CIFAR	LSUN	18.1%	11.5%	97.0%	97.4%	96.6%