Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Remove define_array_slice and reuse array_slice for array_pop_front/back #8401

Merged
merged 8 commits into from
Dec 9, 2023

Conversation

jayzhan211
Copy link
Contributor

@jayzhan211 jayzhan211 commented Dec 3, 2023

Which issue does this PR close?

Ref #7988

Rationale for this change

What changes are included in this PR?

Are these changes tested?

Are there any user-facing changes?

@github-actions github-actions bot added the physical-expr Physical Expressions label Dec 3, 2023
@jayzhan211 jayzhan211 marked this pull request as draft December 3, 2023 01:52
@jayzhan211 jayzhan211 changed the title Refactor array_element Remove define_array_sliceand reusearray_slice for array_pop_front/back Dec 3, 2023
@jayzhan211 jayzhan211 changed the title Remove define_array_sliceand reusearray_slice for array_pop_front/back Remove define_array_slice and reusearray_slice for array_pop_front/back Dec 3, 2023
@jayzhan211 jayzhan211 changed the title Remove define_array_slice and reusearray_slice for array_pop_front/back Remove define_array_slice and reuse array_slice for array_pop_front/back Dec 3, 2023
@jayzhan211 jayzhan211 marked this pull request as ready for review December 3, 2023 05:08
Signed-off-by: jayzhan211 <jayzhan211@gmail.com>
Signed-off-by: jayzhan211 <jayzhan211@gmail.com>
Signed-off-by: jayzhan211 <jayzhan211@gmail.com>
Signed-off-by: jayzhan211 <jayzhan211@gmail.com>
Signed-off-by: jayzhan211 <jayzhan211@gmail.com>
Signed-off-by: jayzhan211 <jayzhan211@gmail.com>
Copy link
Contributor

@alamb alamb left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thank you @jayzhan211 -- this looks great. I think we should add some more comments explaining what the functions are doing but given there are no existing comments I think this code is better than what is on main and therefore could be merged as is

The array code is really coming a long nicely

let array = array.as_any().downcast_ref::<T>().unwrap();

let array_type = array.data_type().clone();
pub fn array_element(args: &[ArrayRef]) -> Result<ArrayRef> {
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Does this need to be pub? If it does, could you please add a documentation comment explaining what the arguments are (namely that the first is a list and the second is an array of indexes?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yes, there is array_element sql function

@@ -581,45 +502,118 @@ pub fn array_except(args: &[ArrayRef]) -> Result<ArrayRef> {

pub fn array_slice(args: &[ArrayRef]) -> Result<ArrayRef> {
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Also here I think it would help to add a comment explaining what the arguments are to this function and what the expected output is

Key details might be if the index values are 1 based or 0 based, for example

false,
)
let from_array = Int64Array::from(vec![1; list_array.len()]);
let to_array = Int64Array::from(
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

this is very clever 👍

Signed-off-by: jayzhan211 <jayzhan211@gmail.com>
@jayzhan211 jayzhan211 requested a review from alamb December 9, 2023 00:31
Signed-off-by: jayzhan211 <jayzhan211@gmail.com>
Copy link
Contributor

@alamb alamb left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thank you @jayzhan211

@@ -579,47 +507,136 @@ pub fn array_except(args: &[ArrayRef]) -> Result<ArrayRef> {
}
}

/// array_slice SQL function
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

❤️

@alamb alamb merged commit 182a37e into apache:main Dec 9, 2023
22 checks passed
appletreeisyellow pushed a commit to appletreeisyellow/datafusion that referenced this pull request Dec 15, 2023
…ont/back` (apache#8401)

* array_element done

Signed-off-by: jayzhan211 <jayzhan211@gmail.com>

* clippy

Signed-off-by: jayzhan211 <jayzhan211@gmail.com>

* replace array_slice

Signed-off-by: jayzhan211 <jayzhan211@gmail.com>

* fix get_indexed_field_empty_list

Signed-off-by: jayzhan211 <jayzhan211@gmail.com>

* replace pop front and pop back

Signed-off-by: jayzhan211 <jayzhan211@gmail.com>

* clippy

Signed-off-by: jayzhan211 <jayzhan211@gmail.com>

* add doc and comment

Signed-off-by: jayzhan211 <jayzhan211@gmail.com>

* fmt

Signed-off-by: jayzhan211 <jayzhan211@gmail.com>

---------

Signed-off-by: jayzhan211 <jayzhan211@gmail.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
physical-expr Physical Expressions
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants