Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

feat: array_contains #6618

Merged
merged 7 commits into from
Jun 27, 2023
Merged

feat: array_contains #6618

merged 7 commits into from
Jun 27, 2023

Conversation

izveigor
Copy link
Contributor

@izveigor izveigor commented Jun 9, 2023

Which issue does this PR close?

Closes #6557

Rationale for this change

What changes are included in this PR?

Are these changes tested?

Yes

Are there any user-facing changes?

Yes

@github-actions github-actions bot added core Core DataFusion crate logical-expr Logical plan and expressions physical-expr Physical Expressions sqllogictest SQL Logic Tests (.slt) labels Jun 9, 2023
@izveigor izveigor marked this pull request as draft June 9, 2023 18:37
@izveigor izveigor marked this pull request as ready for review June 23, 2023 18:59
@@ -1070,6 +1071,70 @@ pub fn array_ndims(args: &[ColumnarValue]) -> Result<ColumnarValue> {
]))))
}

macro_rules! contains {
Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I don't quite understand this comment -- are you saying that it would be better to use the in_list kernel rather than flattening it?

@izveigor
Copy link
Contributor Author

@alamb I wonder if you review this PR if you have free time.

@alamb
Copy link
Contributor

alamb commented Jun 24, 2023

I will find time to review this PR, but maybe not until Monday

Copy link
Contributor

@alamb alamb left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This looks great @izveigor -- thank you very much for this contribution.

I agree with your comments in #6557 that it would be interesting to add operators like @> such as postgres has https://www.postgresql.org/docs/current/functions-array.html. Should we file some follow on tickets to try and crowdsource that work?

@@ -1070,6 +1071,70 @@ pub fn array_ndims(args: &[ColumnarValue]) -> Result<ColumnarValue> {
]))))
}

macro_rules! contains {
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I don't quite understand this comment -- are you saying that it would be better to use the in_list kernel rather than flattening it?

@@ -179,23 +179,24 @@ Unlike to some databases the math functions in Datafusion works the same way as

## Array Expressions
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

❤️

docs/source/user-guide/sql/scalar_functions.md Outdated Show resolved Hide resolved
@alamb
Copy link
Contributor

alamb commented Jun 26, 2023

The other thing I think would help this PR is some tests for the array version in .slt (rather than just scalar) but I will write those as a follow on PR (as I want to get some experience with these functions myself).

@izveigor
Copy link
Contributor Author

@alamb I think it would be better to create a separate array function ticket with columns (Since I considered only scalars as arguments). (See the tickets: #6693 and #6709).

@alamb
Copy link
Contributor

alamb commented Jun 26, 2023

@alamb I think it would be better to create a separate array function ticket with columns (Since I considered only scalars as arguments). (See the tickets: #6693 and #6709).

Indeed @izveigor -- in fact I found exactly the same bug when writing the tests -- #6771

@alamb
Copy link
Contributor

alamb commented Jun 27, 2023

Thanks again @izveigor

maxburke pushed a commit to urbanlogiq/arrow-datafusion that referenced this pull request Jul 14, 2023
* feat: array_contains

* feat: regen.sh

* docs: array_contains

* fix: merge

* Update docs/source/user-guide/sql/scalar_functions.md

---------

Co-authored-by: Andrew Lamb <andrew@nerdnetworks.org>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
core Core DataFusion crate logical-expr Logical plan and expressions physical-expr Physical Expressions sqllogictest SQL Logic Tests (.slt)
Projects
None yet
Development

Successfully merging this pull request may close these issues.

New array method array_contains
2 participants