Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

BUG: pandas.Series.rolling fails on some versions of SciPy #59379

Closed
2 of 3 tasks
letrec opened this issue Aug 1, 2024 · 7 comments
Closed
2 of 3 tasks

BUG: pandas.Series.rolling fails on some versions of SciPy #59379

letrec opened this issue Aug 1, 2024 · 7 comments
Labels
Bug Window rolling, ewma, expanding

Comments

@letrec
Copy link

letrec commented Aug 1, 2024

Pandas version checks

  • I have checked that this issue has not already been reported.

  • I have confirmed this bug exists on the latest version of pandas.

  • I have confirmed this bug exists on the main branch of pandas.

Reproducible Example

import pandas as pd
pd.Series().rolling(17, win_type='triang', min_periods=1, center=True).mean()

Issue Description

Before version 2.0 pandas.Series.rolling used to import windowing functions from scipy.signal, starting from 2.0 it uses scipy.signal.windows.
I suppose the reason is that SciPy has deprecated import of windowing function from this module since 1.13.0.
The issue is that versions of SciPy prior 1.13.0 seem to be supported by pandas and work fine.
I propose making the import to take into account the installed version of SciPy.
Something like this:

from packaging import version

scipy = import_optional_dependency("scipy", extra="Scipy is required to generate window weight.")
scipy_ver = version.parse(scipy.__version__)
signal_module = 'scipy.signal' if scipy_ver < version.Version('1.13.0') else 'scipy.signal.windows'
signal = import_optional_dependency(signal_module, extra="Scipy is required to generate window weight.")

Expected Behavior

Rolling functions work with different versions of SciPy in different versions of Pandas.

Installed Versions

INSTALLED VERSIONS ------------------ commit : 2e218d1 python : 3.10.12.final.0 python-bits : 64 OS : Linux OS-release : 6.5.0-1024-aws Version : #24~22.04.1-Ubuntu SMP Thu Jul 18 10:43:12 UTC 2024 machine : x86_64 processor : x86_64 byteorder : little LC_ALL : None LANG : C.UTF-8 LOCALE : en_US.UTF-8

pandas : 1.5.3
numpy : 1.26.4
pytz : 2024.1
dateutil : 2.9.0.post0
setuptools : 59.6.0
pip : 22.0.2
Cython : None
pytest : None
hypothesis : None
sphinx : None
blosc : None
feather : None
xlsxwriter : None
lxml.etree : None
html5lib : None
pymysql : None
psycopg2 : None
jinja2 : None
IPython : None
pandas_datareader: None
bs4 : None
bottleneck : None
brotli : None
fastparquet : None
fsspec : None
gcsfs : None
matplotlib : None
numba : None
numexpr : None
odfpy : None
openpyxl : None
pandas_gbq : None
pyarrow : None
pyreadstat : None
pyxlsb : None
s3fs : None
scipy : 1.13.0
snappy : None
sqlalchemy : None
tables : None
tabulate : None
xarray : None
xlrd : None
xlwt : None
zstandard : None
tzdata : 2024.1

@letrec letrec added Bug Needs Triage Issue that has not been reviewed by a pandas team member labels Aug 1, 2024
@asishm
Copy link
Contributor

asishm commented Aug 1, 2024

Thanks for the report. The example you've shown works fine with pandas 2.2.2, scipy 1.10.0 and pandas 2.2.2, scipy 1.13.0

Your installed versions show that you are using pandas 1.5.3. Are you talking about adding a fix to the 1.5.x series? If so, I think pandas does't support that series anymore (and it's also not possible to add an upper bound on the scipy version to the existing 1.5.3 release)

@Aloqeely Aloqeely added Window rolling, ewma, expanding and removed Needs Triage Issue that has not been reviewed by a pandas team member labels Aug 1, 2024
@letrec
Copy link
Author

letrec commented Aug 1, 2024

I was hoping that it could be fixed in 1.5.x.

@letrec
Copy link
Author

letrec commented Aug 1, 2024

I'm willing to contribute a fix if that's still allowed for 1.5.x.

@asishm
Copy link
Contributor

asishm commented Aug 1, 2024

My understanding is that the 1.5.x series won't get another release. I'll let someone from the pandas team comment (maybe @lithomas1 ?)

I think the way forward (if you want to keep using 1.5.3) is to pin scipy to <1.13.0, or monkeypatch pandas/scipy. Is there any reason you are unable to update to pandas 2.0+ ?

@letrec
Copy link
Author

letrec commented Aug 1, 2024

I can't move to 2.x because of other packages I depend on.
I currently monkeypatch SciPy as a workaround, but this is obviously far from ideal.

@letrec
Copy link
Author

letrec commented Aug 1, 2024

It's unfortunate that such a relatively recent release as 1.5.x is not supported anymore.

@mroeschke
Copy link
Member

Thanks for the report, but as mentioned since this works on main so I believe there is no more action needed.

pandas' policy is that only the most recent minor version (2.2) receives backport fixes. The project doesn't have the resources or logistics to support more older versions.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Bug Window rolling, ewma, expanding
Projects
None yet
Development

No branches or pull requests

4 participants