Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Fix incorrect results in BitAnd GroupsAccumulator #6957

Merged
merged 1 commit into from
Jul 13, 2023

Conversation

alamb
Copy link
Contributor

@alamb alamb commented Jul 13, 2023

Which issue does this PR close?

Resolves #6955, a bug introduced in #6904

Rationale for this change

The answer is incorrect because the starting value is always 0 and thus 0 & anything else will also be zero which is not right

What changes are included in this PR?

  1. Allow the starting value of the accumulator t be specified
  2. set the default value of the accumulator correctly for bit_and
  3. Add new tests (subset of what is in Minor: Improve aggregate test coverage more #6952)

Are these changes tested?

Yes

Are there any user-facing changes?

@github-actions github-actions bot added physical-expr Physical Expressions core Core DataFusion crate sqllogictest SQL Logic Tests (.slt) labels Jul 13, 2023
@alamb alamb changed the title Alamb/fix gby bit and Fix incorrect results in BitAnd GroupsAccumulator Jul 13, 2023
GROUP BY tag
ORDER BY tag
----
1 8 11 13 14 11 12 6 11 A
Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

before this PR, this test fails like

External error: query result mismatch:
[SQL] SELECT
  bit_and(c1),
  bit_and(c2),
  bit_and(c3),
  bit_or(c1),
  bit_or(c2),
  bit_or(c3),
  bit_xor(c1),
  bit_xor(c2),
  bit_xor(c3),
  tag
FROM bit_aggregate_functions
GROUP BY tag
ORDER BY tag
[Diff] (-expected|+actual)
-   1 8 11 13 14 11 12 6 11 A
-   33 11 NULL 33 11 NULL 33 11 NULL B
+   0 0 0 13 14 11 12 6 11 A
+   0 0 NULL 33 11 NULL 33 11 NULL B
at tests/sqllogictests/test_files/aggregate.slt:1546

@@ -279,35 +280,31 @@ impl AggregateExpr for BitAnd {
use std::ops::BitAndAssign;
match self.data_type {
DataType::Int8 => {
instantiate_primitive_accumulator!(self, Int8Type, |x, y| x
.bitand_assign(y))
instantiate_accumulator!(self, -1, Int8Type, |x, y| x.bitand_assign(y))
Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

the fix is passing in MAX / -1 here to get a bitpattern of all 1 initially

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The fix looks good. It's better to leave a comment like above in the code.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Comment added in #6964

@@ -85,7 +95,7 @@ where
let values = values[0].as_primitive::<T>();

// update values
self.values.resize(total_num_groups, T::default_value());
Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

previously this always started at zero, now it starts at the specified value.

I think I can now reduce the code in min/max accumulator as well

@alamb alamb marked this pull request as ready for review July 13, 2023 16:06
Copy link
Contributor

@izveigor izveigor left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

@alamb
Copy link
Contributor Author

alamb commented Jul 13, 2023

🤔 seems clippy failure is unrelated. I will fix that

@alamb
Copy link
Contributor Author

alamb commented Jul 13, 2023

Clippy fixes in #6959

@alamb alamb merged commit c07c26c into apache:main Jul 13, 2023
@alamb
Copy link
Contributor Author

alamb commented Jul 13, 2023

Thank you for the quick review @viirya and @izveigor

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
core Core DataFusion crate physical-expr Physical Expressions sqllogictest SQL Logic Tests (.slt)
Projects
None yet
Development

Successfully merging this pull request may close these issues.

Regression: bit_and aggregate function is incorrect
3 participants