Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[BUG] Spark 3.2.1 arithmetic_ops_test failures #5474

Closed
tgravescs opened this issue May 12, 2022 · 3 comments · Fixed by #5483
Closed

[BUG] Spark 3.2.1 arithmetic_ops_test failures #5474

tgravescs opened this issue May 12, 2022 · 3 comments · Fixed by #5483
Assignees
Labels
bug Something isn't working P0 Must have for release

Comments

@tgravescs
Copy link
Collaborator

Describe the bug
Integration test failures on Spark 3.2.1

12:05:10 FAILED ../../src/main/python/arithmetic_ops_test.py::test_mod_pmod_by_zero[pmod(cast(-12 as {}), cast(0 as {}))-Decimal(7,7)]
12:05:10 FAILED ../../src/main/python/arithmetic_ops_test.py::test_mod_pmod_by_zero[cast(-12 as {}) % cast(0 as {})-Decimal(7,7)]

This might be the ArtithmeticException mismatch.

Possibly caused by #5354

@tgravescs tgravescs added bug Something isn't working ? - Needs Triage Need team to review and classify P0 Must have for release labels May 12, 2022
@tgravescs
Copy link
Collaborator Author

@HaoYang670 can you take a look

@HaoYang670
Copy link
Collaborator

Sure, I will work on this.

@HaoYang670 HaoYang670 self-assigned this May 12, 2022
@sameerz sameerz added this to the May 2 - May 20 milestone May 12, 2022
@sameerz sameerz removed the ? - Needs Triage Need team to review and classify label May 12, 2022
@HaoYang670
Copy link
Collaborator

The reason of this bug is that in Spark 3.2 cannotChangeDecimalPrecisionError throws java.lang.ArithmeticException but not SparkArithmeticException:

java.lang.ArithmeticException: Decimal(expanded,-12,2,0}) cannot be represented as Decimal(7, 7).
  at org.apache.spark.sql.errors.QueryExecutionErrors$.cannotChangeDecimalPrecisionError(QueryExecutionErrors.scala:97)

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
bug Something isn't working P0 Must have for release
Projects
None yet
Development

Successfully merging a pull request may close this issue.

3 participants