Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Take care of deprecation warnings from dependencies #482

Merged
merged 21 commits into from
Mar 6, 2024

Conversation

timmens
Copy link
Member

@timmens timmens commented Mar 3, 2024

In this PR, we take care of deprecation warnings from estimagic's dependencies.

In particular, we

  • Call df.map() instead of df.applymap() if pandas version 2.1.0 or greater is installed
  • Add new module compat.py, in which we implement functions with compatibility issues
  • Ignore deprecation warning about MemoizeJac, which is raised in cyipopt. It seems that there are working on this as well (see here)
  • Use preferred np.prod instead of np.product, which is deprecated with numpy version 1.25.0 (see release notes here. In earlier versions of numpy np.prod was already implemented and np.product would simply call np.prod; therefore we can just swap np.product for np.prod. (There were some subtle differences, but they do not apply to our use cases; see here)
  • Use pd.Series(list).unique() instead of pd.unique(list), as the latter ist deprecated. Also use pd.Series(list).value_counts() instead of pd.value_counts(list).
  • Remove maxiter option from scipy_truncated_newton algorithm, since it is not a valid option.
  • Use DataFrame.replace() properly for categorical columns. The old implementation raised the following errors:
    estimagic/tests/inference/test_bootstrap.py:70: FutureWarning: Downcasting behavior in `replace` is deprecated and 
    will be removed in a future version. To retain the old behavior, explicitly call `result.infer_objects(copy=False)`. To opt-in 
    to the future behavior, set `pd.set_option('future.no_silent_downcasting', True)`
    df = raw.replace({"time": replacements})
    
    estimagic/tests/inference/test_bootstrap.py:70: FutureWarning: The behavior of Series.replace (and 
    DataFrame.replace) with CategoricalDtype is deprecated. In a future version, replace will only be used for cases that 
    preserve the categories. To change the categories, use ser.cat.rename_categories instead.
    df = raw.replace({"time": replacements})
  • Remove usage of DataFrameGroupBy.grouper, due to following warning:
     estimagic/src/estimagic/parameters/consolidate_constraints.py:595: FutureWarning: DataFrameGroupBy.grouper is deprecated and will be removed in a future version of pandas.
      weights["dupl_group"] = weights.groupby(list(weights.columns)).grouper.group_info[0]
    Current solution:
    weights["dupl_group"] = weights.groupby(list(weights.columns)).ngroup()
  • Set dtype to float before overwriting certain values with np.inf, due to the following warning:
    estimagic/src/estimagic/visualization/profile_plot.py:169: Setting an item of incompatible dtype is deprecated and will raise an error in a future version of pandas. Value 'inf' has dtype incompatible with int64,  please explicitly cast to a compatible dtype first.
    solution_times[~converged_info] = np.inf
    Current solution:
    • Cast data frame to float for all columns, then assign np.inf.
    • Previously all columns where we assigned np.inf where casted to float anyways, resulting in a data frame with some float and some int columns. Now they are all float.
  • Conversion of an array with ndim > 0 to a scalar is deprecated. Specifically, this occurs in
    • tranquilo (see issue #15 in tranquilo)
    • simopt strong optimize (todo: what to do about simopt)
      Current solution: Ignore the warning.
  • For CI run certain tests with pandas version < 2.0.0
    • Select tests we want to run
    • Create new CI task
    • Add new environment to .envs
  • Add Python 3.12 to linux test matrix

Closes #470.

Copy link

codecov bot commented Mar 3, 2024

Codecov Report

Attention: Patch coverage is 96.55172% with 1 lines in your changes are missing coverage. Please review.

Project coverage is 92.99%. Comparing base (3fba00e) to head (50c2c24).

Files Patch % Lines
src/estimagic/compat.py 83.33% 1 Missing ⚠️
Additional details and impacted files
@@            Coverage Diff             @@
##             main     #482      +/-   ##
==========================================
+ Coverage   92.98%   92.99%   +0.01%     
==========================================
  Files         193      194       +1     
  Lines       14638    14649      +11     
==========================================
+ Hits        13611    13623      +12     
+ Misses       1027     1026       -1     

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

@timmens timmens requested a review from hmgaudecker March 5, 2024 16:38
Copy link
Member

@hmgaudecker hmgaudecker left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Looks great to me -- just a couple of small remarks!

src/estimagic/parameters/parameter_groups.py Outdated Show resolved Hide resolved
src/estimagic/compat.py Outdated Show resolved Hide resolved
src/estimagic/visualization/profile_plot.py Outdated Show resolved Hide resolved
@timmens timmens merged commit 656319d into main Mar 6, 2024
17 checks passed
@timmens timmens deleted the deprecation-warnings branch March 6, 2024 14:55
@timmens timmens mentioned this pull request Mar 8, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

Replace DateFrame.applymap with DataFrame.map (in estimation_table)
2 participants