Skip to content

Unofficial PyTorch implementation of Fastformer based on paper "Fastformer: Additive Attention Can Be All You Need"."

License

Notifications You must be signed in to change notification settings

wilile26811249/Fastformer-PyTorch

Folders and files

NameName
Last commit message
Last commit date

Latest commit

 

History

8 Commits
 
 
 
 
 
 
 
 
 
 

Repository files navigation

Fastformer-PyTorch

Unofficial PyTorch implementation of Fastformer based on paper Fastformer: Additive Attention Can Be All You Need.

Network Architecture image from the paper

Usage :

import torch
from Fastformer import Fastformer

model = Fastformer(dim = 3, decode_dim = 8)
x = torch.randn(4, 6, 3)
mask = torch.ones(1, 8).bool()
result = model(x, mask)
print(result.size())

Citation :

@misc{wu2021fastformer,
    title={Fastformer: Additive Attention Can Be All You Need},
    author={Chuhan Wu, Fangzhao Wu, Tao Qi and Yongfeng Huang},
    year={2021},
    eprint={2108.09084v2},
    archivePrefix={arXiv},
    primaryClass={cs.CV}
}

If this implement have any problem please let me know, thank you.

About

Unofficial PyTorch implementation of Fastformer based on paper "Fastformer: Additive Attention Can Be All You Need"."

Resources

License

Stars

Watchers

Forks

Releases

No releases published

Packages

No packages published

Languages