Reason for doubling length in both LongConv and FlashFFTConv implementations

[nichoffs]

I noticed that, in flashfftconv_long_convs.py, the LongConv length is doubled during initialization.

self.L = L*2

Similarly, the FlashFFTConv class also has double the actual input length:

self.flashfftconv = FlashFFTConv(2048, dtype=torch.bfloat16)

Does this allow for bi-directional support or does it have something to do with wraparound during the convolution? What's the reason for the doubling?

Thanks!

[DanFu09]

These are to create a causal convolution. If you make the kernel length equal to 2L, you get a bidirectional convolution. If the kernel is length L, it’s called a circular convolution. FlashFFTConv can do any of these by setting the kernel length as you’d like. This blog has more breakdown: https://hazyresearch.stanford.edu/blog/2023-12-11-conv-tutorial

…

On Fri, Dec 22, 2023 at 11:12 AM N T Hoffs ***@***.***> wrote: I noticed that, in flashfftconv_long_convs.py, the LongConv length is doubled during initialization. self.L = L*2 Similarly, the FlashFFTConv class also has double the actual input length: self.flashfftconv = FlashFFTConv(2048, dtype=torch.bfloat16) Does this allow for bi-directional support or does it have something to do with wraparound during the convolution? What's the reason for the doubling? Thanks! — Reply to this email directly, view it on GitHub <#12>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/ABDDIIVPJSESHCBKNN2IZ7TYKXLTRAVCNFSM6AAAAABBAFROSKVHI2DSMVQWIX3LMV43ASLTON2WKOZSGA2TINBSGIYTENA> . You are receiving this because you are subscribed to this thread.Message ID: ***@***.***>