Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

use decimal type for arithmetic functions #1244

Merged
merged 22 commits into from
Sep 13, 2022
Merged

use decimal type for arithmetic functions #1244

merged 22 commits into from
Sep 13, 2022

Conversation

jennifersp
Copy link
Contributor

@jennifersp jennifersp commented Sep 7, 2022

fixes dolthub/dolt#4269

sum will be either float64 or decimal.Decimal type, but return type is decided on the type of the child expression of sum expression. This causes some test results to be modified into int from float.

the test results in decimal.Decimal type is hard to get the exact matching using its convert method, so result is checked in string format by using StringFixed() method of the decimal.Decimal result value.

Copy link
Contributor

@fulghum fulghum left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Looks good overall; I just had a few questions I wanted to make sure I understood.

I suspect that AVG needs the same treatment as SUM. We can handle that in a separate PR, but would be good to check that out while we're in here.

enginetest/evaluation.go Show resolved Hide resolved
enginetest/queries/script_queries.go Outdated Show resolved Hide resolved
enginetest/queries/variable_queries.go Show resolved Hide resolved
sql/expression/function/aggregation/sum_test.go Outdated Show resolved Hide resolved
sql/expression/arithmetic.go Show resolved Hide resolved
sql/expression/function/aggregation/unary_agg_buffers.go Outdated Show resolved Hide resolved
Copy link
Contributor

@fulghum fulghum left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Changes look good. This turned out to be trickier than I was expecting, so thanks for all the extra attention to detail. 😁

enginetest/queries/script_queries.go Outdated Show resolved Hide resolved
sql/expression/function/aggregation/unary_agg_buffers.go Outdated Show resolved Hide resolved
sql/expression/convert.go Outdated Show resolved Hide resolved
sql/expression/function/aggregation/sum_test.go Outdated Show resolved Hide resolved
@jennifersp jennifersp merged commit e0e927b into main Sep 13, 2022
@jennifersp jennifersp deleted the jennifer/decimal branch September 13, 2022 19:33
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

sum() of decimal types with negative numbers is incorrect
2 participants