Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

fix: skip infinite values during serialization #99

Merged
merged 10 commits into from
Jul 8, 2024

Conversation

alespour
Copy link
Contributor

@alespour alespour commented Jul 3, 2024

Fixes https://github.com/influxdata/EAR/issues/5299

Proposed Changes

np.inf and -np.inf values in a Pandas data frame are skipped during during serialization into line protocol.

Other fixes:

  • replace polars.DataFrame.apply (deprecated, and does not exists in latest version) with polars.DataFrame.map_rows
  • replace pandas data frame replace() with apply(lambda: ) to avoid future problems
dataframe_serializer.py:250: FutureWarning: A value is trying to be set on a copy of a DataFrame or Series through chained assignment using an inplace method.

The behavior will change in pandas 3.0. This inplace method will never work because the intermediate object on which we are setting values always behaves as a copy.

For example, when doing 'df[col].method(value, inplace=True)', try using 'df.method({col: value}, inplace=True)' or df[col] = df[col].method(value) instead, to perform the operation inplace on the original object.

Checklist

  • CHANGELOG.md updated
  • Rebased/mergeable
  • A test has been added if appropriate
  • Tests pass
  • Commit messages are conventional

Copy link

codecov bot commented Jul 3, 2024

Codecov Report

All modified and coverable lines are covered by tests ✅

Project coverage is 53.26%. Comparing base (1fe1460) to head (146b987).

Additional details and impacted files
@@            Coverage Diff             @@
##             main      #99      +/-   ##
==========================================
+ Coverage   52.84%   53.26%   +0.41%     
==========================================
  Files          33       33              
  Lines        2036     2037       +1     
==========================================
+ Hits         1076     1085       +9     
+ Misses        960      952       -8     

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

@alespour alespour marked this pull request as ready for review July 3, 2024 13:53
@alespour alespour requested a review from bednar July 4, 2024 08:36
Copy link
Member

@bednar bednar left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM 🚀

Please back port this fix also to https://github.com/Influxdata/influxdb-client-python.

@alespour alespour merged commit 66776f1 into main Jul 8, 2024
13 checks passed
@alespour alespour deleted the fix/do-not-serialize-infinite-values branch July 8, 2024 10:01
@bednar bednar added this to the 0.7.0 milestone Jul 8, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants