Energy-based Out-of-distribution Detection (Energy OOD)

This repository is the official implementation of Energy-based Out-of-distribution Detection by Weitang Liu, Xiaoyun Wang, John Owens and Yixuan Li. This method is an effective and easy OOD detector with and without fine-tuning. Our code is implemented with courtesy of Outlier-Exposure. If you have any code related questions, such as this issue and this issue, we highly recommened to check the couterpart in Outlier-Exposure.

Pretrained Models and Datasets

Pretrained models are provided in folder

./CIFAR/snapshots/

Please download the datasets in folder

./data/

Testing and Fine-tuning

run energy score testing for cifar10 WRN

bash run.sh energy 0

run energy score testing for cifar100 WRN

bash run.sh energy 1

run energy score training and testing for cifar10 WRN

bash run.sh energy_ft 0

run energy score training and testing for cifar100 WRN

bash run.sh energy_ft 1

Results

Our model achieves the following average performance on 6 OOD datasets:

1. MSP vs energy score with and without fine-tuned on CIFAR-10

Model name	FPR95
Softmax score	51.04%
Energy score (ours)	33.01%
Softmax score with fine-tune	8.53%
Energy score with fine-tune (ours)	3.32%

2. CIFAR-10 (in-distribution) vs SVHN (out-of-distribution) Score Distributions

3. Performance among different baselines for WideResNet

CIFAR-10:

Model name	FPR95
Softmax score	51.04%
Energy score (ours)	33.01%
ODIN	35.71%
Mahalanobis	37.08%
Outlier Exposure	8.53%
Energy score with fine-tune (ours)	3.32%

CIFAR-100:

Model name	FPR95
Softmax score	80.41%
Energy score (ours)	73.60%
ODIN	74.64%
Mahalanobis	54.64%
Outlier Exposure	58.10%
Energy score with fine-tune (ours)	47.55%

Outlier Datasets

These experiments make use of numerous outlier datasets. Links for less common datasets are as follows, 80 Million Tiny Images Textures, Places365, LSUN-C, LSUN-R, iSUN and SVHN.

Citation

 @article{liu2020energy,
      title={Energy-based Out-of-distribution Detection},
      author={Liu, Weitang and Wang, Xiaoyun and Owens, John and Li, Yixuan},
      journal={Advances in Neural Information Processing Systems},
      year={2020}
 }