Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

move Left, Lpad, Reverse, Right, Rpad functions to datafusion_functions #9841

Merged
merged 17 commits into from
Mar 29, 2024

Conversation

Omega359
Copy link
Contributor

Which issue does this PR close?

Closes #9828

Rationale for this change

As part of #9285 the unicode functions should be migrated to the new datafusion-functions crate in the new structure

What changes are included in this PR?

Code, tests.

Are these changes tested?

Yes.

Are there any user-facing changes?

No.

@github-actions github-actions bot added logical-expr Logical plan and expressions physical-expr Physical Expressions labels Mar 28, 2024
@Omega359 Omega359 marked this pull request as ready for review March 28, 2024 20:12
Copy link
Contributor

@alamb alamb left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Let's keep this train moving 🚀 -- thanks @Omega359

I have some suggestions that might improve the code, but I don't think they are needed to merge this PR.

Thanks again 🙏


#[test]
fn test_functions() -> Result<()> {
#[cfg(feature = "unicode_expressions")]
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Since the whole module is cfg'd we can probably remove these guards on individual tests

Comment on lines 124 to 125
ColumnarValue::Scalar(ScalarValue::Utf8(Some(String::from("abcde")))),
ColumnarValue::Scalar(ScalarValue::Int64(Some(2))),
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Totally not needed as this code is just moved, but I think you can write this more concisely with from if you wnat:

Suggested change
ColumnarValue::Scalar(ScalarValue::Utf8(Some(String::from("abcde")))),
ColumnarValue::Scalar(ScalarValue::Int64(Some(2))),
ColumnarValue::Scalar(ScalarValue::from("abcde")),
ColumnarValue::Scalar(ScalarValue::from(2u64)),

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

(same thing applies to the rest of the tests in this file and and the others)

use crate::utils::test::test_function;

#[test]
fn test_functions() -> Result<()> {
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

(same comments as above about cfg and ScalarValue::from apply here too)

@Omega359
Copy link
Contributor Author

Thanks for the feedback - I'd like to update this PR with the feedback here rather than in a followup commit if you don't mind.

@alamb
Copy link
Contributor

alamb commented Mar 29, 2024

👨‍🍳 👌 -- looks really nice. Thanks for cleaning up the code @Omega359

@alamb alamb merged commit 09f5a54 into apache:main Mar 29, 2024
23 checks passed
@alamb
Copy link
Contributor

alamb commented Mar 29, 2024

It is like the scouts "leave things better than how you found them"

@Omega359 Omega359 deleted the feature/9828 branch March 29, 2024 02:58
Lordworms pushed a commit to Lordworms/arrow-datafusion that referenced this pull request Apr 1, 2024
…ns (apache#9841)

* Fix to_timestamp benchmark

* Remove reference to simd and nightly build as simd is no longer an available feature in DataFusion and building with nightly may not be a good recommendation when getting started.

* Fixed missing trim() function.

* Create unicode module in datafusion/functions/src/unicode and unicode_expressions feature flag, move char_length function

* move Left, Lpad, Reverse, Right, Rpad functions to datafusion_functions

* Code cleanup from PR review.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
logical-expr Logical plan and expressions physical-expr Physical Expressions
Projects
None yet
Development

Successfully merging this pull request may close these issues.

move Left, Lpad, Reverse, Right, Rpad functions to datafusion_functions
2 participants