Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

fix regression in cast from string to float that caused signed NaN to be considered valid #4259

Merged
merged 1 commit into from
Dec 2, 2021

Conversation

andygrove
Copy link
Contributor

Signed-off-by: Andy Grove andygrove@nvidia.com

Closes #4246

Changes in this PR:

  • Remove transformations that are no longer necessary now that cuDF is more flexible when parsing NaN and Inf (case insensitive, supports + and -, supports both Inf and Infinity)
  • Fixes a bug in the VALID_FLOAT_REGEX to reject signed NaN values (+NaN, -NaN) which are now supported by cuDF but not by Spark
  • Reinstate unit test

… be valid

Signed-off-by: Andy Grove <andygrove@nvidia.com>
@andygrove andygrove added the bug Something isn't working label Dec 2, 2021
@andygrove andygrove added this to the Nov 30 - Dec 10 milestone Dec 2, 2021
@andygrove andygrove self-assigned this Dec 2, 2021
@andygrove
Copy link
Contributor Author

build

@andygrove andygrove merged commit 8731ee5 into NVIDIA:branch-22.02 Dec 2, 2021
@andygrove andygrove deleted the cast-string-to-float branch December 2, 2021 15:29
res-life pushed a commit to res-life/spark-rapids that referenced this pull request Dec 6, 2021
… be valid (NVIDIA#4259)

Signed-off-by: Andy Grove <andygrove@nvidia.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
bug Something isn't working
Projects
None yet
Development

Successfully merging this pull request may close these issues.

[BUG] Regression in CastOpSuite due to cuDF change in parsing NaN
3 participants