Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Fix ApproxPercentileCont signature #8825

Merged
merged 2 commits into from
Jan 14, 2024

Conversation

joroKr21
Copy link
Contributor

The number of centroids must be an integer in coerce_types. Reflect that in the type signature.

Which issue does this PR close?

Didn't create an issue for this one.

Rationale for this change

Currently when we try to pass a floating type as the number of centroids, we get a confusing error message, e.g.

No function matches the given name and argument types APPROX_PERCENTILE_CONT(Float64, Float64, Float64)

And then later the exact same signature is listed among the candidates.
The reason is that the type signature is not in sync with the logic in coerce_types.

What changes are included in this PR?

Changed the type signature of ApproxPercentileCont to be a combination of all numeric types and all integer types for the number of centroids.

Are these changes tested?

I added a test but it's not really feasible to list all possible signatures in the test.

Are there any user-facing changes?

Only in terms of the type signature and error message. No change in behaviour:

  • Calling the function with floating centroids fails both before and after this change
  • Calling the function with floating number and integer centroids works both before and after this change
    (although confusingly the old type signature would include only parameters with types [t, Float64, t])

@github-actions github-actions bot added logical-expr Logical plan and expressions sqllogictest SQL Logic Tests (.slt) labels Jan 10, 2024
@joroKr21 joroKr21 marked this pull request as ready for review January 11, 2024 07:01
Copy link
Contributor

@alamb alamb left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thank you @joroKr21 -- the code looks good to me. I have one small question about the tests

@@ -223,7 +223,7 @@ pub fn coerce_types(
| AggregateFunction::RegrSXX
| AggregateFunction::RegrSYY
| AggregateFunction::RegrSXY => {
let valid_types = [NUMERICS.to_vec(), vec![DataType::Null]].concat();
let valid_types = [NUMERICS.to_vec(), vec![Null]].concat();
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

You didn't do this, but it is unfortunate there are special coercion rules for AggregateFunctions that are not handled by the more general purpose Function cocercion rules.

@@ -95,6 +95,9 @@ SELECT approx_percentile_cont(c3, 0.95, c1) FROM aggregate_test_100
statement error DataFusion error: Error during planning: No function matches the given name and argument types 'APPROX_PERCENTILE_CONT\(Int16, Float64, Float64\)'\. You might need to add explicit type casts\.
SELECT approx_percentile_cont(c3, 0.95, 111.1) FROM aggregate_test_100

statement error DataFusion error: Error during planning: No function matches the given name and argument types 'APPROX_PERCENTILE_CONT\(Float64, Float64, Float64\)'\. You might need to add explicit type casts\.
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Won't this test pass on main too? Maybe the problem is that the sqllogictest framework doesn't display the hints 🤔

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yeah that's the issue. Perhaps I should write a unit test instead? 🤔

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

If you could that would be great (otherwise I worry we we may break this again during a refactor or something and not realize it)

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Added a unit test 👍

@alamb
Copy link
Contributor

alamb commented Jan 14, 2024

@joroKr21 are you willing to add a new test? Or shall we find some help to do so?

The number of centroids must be an integer in `coerce_types`.
Reflect that in the type signature.
@github-actions github-actions bot added the optimizer Optimizer rules label Jan 14, 2024
Copy link
Contributor

@alamb alamb left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thank you @joroKr21

@alamb alamb merged commit af3d190 into apache:main Jan 14, 2024
22 checks passed
@joroKr21 joroKr21 deleted the bugfix/percentile-signature branch January 15, 2024 07:13
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
logical-expr Logical plan and expressions optimizer Optimizer rules sqllogictest SQL Logic Tests (.slt)
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants