Array size and scipy.sosfiltfilt()

[ErikBsly]

Hello, today I experienced an issue while using mkl_fft on one-dimensional numpy arrays. The following code results in an error:

import numpy as np
from scipy.signal import sosfilt, sosfiltfilt, butter
from mkl_fft import fft

a = np.random.rand(100_000_000)
fft(a) # ok
print("ok 1")

a = sosfilt(butter(4, 0.1, output="sos"), a)
fft(a) # ok
print("ok 2")

a = sosfiltfilt(butter(4, 0.1, output="sos"), a)
fft(a.copy())  # ok
print("ok 3")

fft(a) # ERROR
print("ok 4")

While the first three FFTs execute normally, I get an error in der last step

File "mkl_fft_pydfti.pyx", line 155, in mkl_fft._pydfti.fft
File "mkl_fft_pydfti.pyx", line 401, in mkl_fft._pydfti._fft1d_impl
ValueError: Internal error occurred: b'Intel MKL DFTI ERROR: Inconsistent configuration parameters'

I know from other issues that mkl has problems with array lengths above 2^24 (ca. 16.7M). However, I'm surprised that the first examples execute normally while in all my tests applying fft() to the output of scipy.signal.sosfiltfilt() fails if the array is longer than 2^24 while sosfilt() works fine. I can solve the problem by using a.copy() but I smell something fishy here. Even if this might not be easily solvable I'd still be glad if someone could explain to me why this happens in the first place. Thanks!

I'm on Win11, Anaconda python 3.10, mkl 1.3.1, scipy 1.10.0

[ErikBsly]

Just to add a small remark in case this is still under investigation or someone's interested: The result of sosfiltfilt() is neither C_contiguous nor F_contiguous, probably causing the error. Surprisingly, the code always runs fine when a.size is a power of 2.

import numpy as np
from scipy.signal import sosfilt, sosfiltfilt, butter
from mkl_fft import fft

a = np.random.rand(2**24+1)

print(a.flags)
fft(a) # ok
print("ok 1")

a = sosfiltfilt(butter(4, 0.1, output="sos"), a)
print(a.flags)
fft(a) # ERROR
print("ok 4")

Output:

C_CONTIGUOUS : True
F_CONTIGUOUS : True
OWNDATA : True
WRITEABLE : True
ALIGNED : True
WRITEBACKIFCOPY : False

ok 1
C_CONTIGUOUS : False
F_CONTIGUOUS : False
OWNDATA : False
WRITEABLE : True
ALIGNED : True
WRITEBACKIFCOPY : False

Traceback (most recent call last):
File "xxx", line 14, in <module>
fft(a) # ERROR
File "mkl_fft\_pydfti.pyx", line 156, in mkl_fft._pydfti.fft
File "mkl_fft\_pydfti.pyx", line 402, in mkl_fft._pydfti._fft1d_impl
ValueError: Internal error occurred: b'Intel MKL DFTI ERROR: Inconsistent configuration parameters'