nn_curvature

This project empirically shows neural network training molds feature maps to expand areas near decision boundaries. For details, see our preprint Neural networks learn to magnify areas near decision boundaries.

Galaries

Description	Figures
Volume element of a single hidden layer (250 hidden units) architecture trained on sinusoid binary classification task
Volume element of a 3-hidden-layer (8 hidden units each) architecture trained on sinusoid binary classification task
Volume element of linear interpolation between two MNIST digits (1 and 6) using a single hidden layer (2000 hidden units) architecture
Volume element of interpolated digits in a plane spanned by three MNIST digits (5, 6, and 1)

Setup

To setup the environment, at root, run

conda env create -f environment.yml
conda activate curvature

Run Experiments

To run experiments, we first need to specify paths for data, model temp files, and visualizations in config.toml. Different experiments can have separate folders to store final results. One example is

['exp']
data_dir='data/'
result_dir='img/'
model_dir='model/'

Next specify the tag argument in each bash script before running so that it reads the coresponding paths in config.toml. To run cifar, for instance, at root:

bash commands/train_cifar.sh

Acknowledgement

If you find this code useful, please consider citing our preprint:

@article{zv2023neural,
  title={Neural networks learn to magnify areas near decision boundaries}, 
  author={Zavatone-Veth, Jacob A. and Yang, Sheng and Rubinfien, Julian A. and Pehlevan, Cengiz},
  year={2023},
  journal={arXiv preprint},
  eprint={2301.11375},
  archivePrefix={arXiv},
  primaryClass={cs.LG}
}