Skip to content

Ἀνατομή is a PyTorch library to analyze representation of neural networks

License

Notifications You must be signed in to change notification settings

moskomule/anatome

Folders and files

NameName
Last commit message
Last commit date

Latest commit

 

History

86 Commits
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 

Repository files navigation

anatome

Ἀνατομή is a PyTorch library to analyze internal representation of neural networks

This project is under active development and the codebase is subject to change.

v0.0.5 introduces significant changes to distance.

Installation

anatome requires

Python>=3.9.0
PyTorch>=1.10
torchvision>=0.11

After the installation of PyTorch, install anatome as follows:

pip install -U git+https://github.com/moskomule/anatome

Available Tools

Representation Similarity

To measure the similarity of learned representation, anatome.SimilarityHook is a useful tool. Currently, the following methods are implemented.

import torch
from torchvision.models import resnet18
from anatome import Distance

random_model = resnet18()
learned_model = resnet18(pretrained=True)
distance = Distance(random_model, learned_model, method='pwcca')
with torch.no_grad():
    distance.forward(torch.randn(256, 3, 224, 224))

# resize if necessary by specifying `size`
distance.between("layer3.0.conv1", "layer3.0.conv1", size=8)

Loss Landscape Visualization

from torch.nn import functional as F
from torchvision.models import resnet18
from anatome import landscape2d

x, y, z = landscape2d(resnet18(),
                      data,
                      F.cross_entropy,
                      x_range=(-1, 1),
                      y_range=(-1, 1),
                      step_size=0.1)
imshow(z)

Fourier Analysis

  • Yin et al. NeurIPS 2019 etc.,
from torch.nn import functional as F
from torchvision.models import resnet18
from anatome import fourier_map

map = fourier_map(resnet18(),
                  data,
                  F.cross_entropy,
                  norm=4)
imshow(map)

Citation

If you use this implementation in your research, please cite as:

@software{hataya2020anatome,
    author={Ryuichiro Hataya},
    title={anatome, a PyTorch library to analyze internal representation of neural networks},
    url={https://github.com/moskomule/anatome},
    year={2020}
}