Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

[FEA] support lists::concatenate_rows for structs and lists #10800

Closed
revans2 opened this issue May 5, 2022 · 1 comment
Closed

[FEA] support lists::concatenate_rows for structs and lists #10800

revans2 opened this issue May 5, 2022 · 1 comment
Assignees
@nvdbaranec
Labels
feature request New feature or request

Comments

@revans2
Copy link
Contributor

revans2 commented May 5, 2022

Is your feature request related to a problem? Please describe.
We have a customer that really would like to be able to concatenate lists row by row, but the data types gets rather complicated. It includes lists and structs multiple levels deep.

val schema_a = StructType(Seq(
	StructField("a_d", DoubleType),
	StructField("a_s", StringType)))

val schema_b = ArrayType( StructType(Seq(
	StructField("b_a_1", ArrayType(schema_a)),
	StructField("b_a_2", ArrayType(schema_a)),
	StructField("b_l", LongType),
	StructField("b_s_1", StringType),
	StructField("b_s_2", StringType),
	StructField("b_s_3", StringType),
	StructField("b_a_s", ArrayType(StringType)))))

Describe the solution you'd like
I would love it if lists::concatenate_rows would work for me instead of throwing an exception saying that this is not going to work.

Describe alternatives you've considered
If it was just structs I might be able to hack something together myself, but I don't think this is really possible for me to do without some help.

Additional context
Add any other context, code examples, or references to existing implementations about the feature request here.
@revans2 revans2 added feature request New feature or request Needs Triage Need team to review and classify labels May 5, 2022
@revans2 revans2 mentioned this issue May 12, 2022
Closed
@nvdbaranec nvdbaranec mentioned this issue May 18, 2022
Merged
rapids-bot bot pushed a commit that referenced this issue May 23, 2022
@nvdbaranec
Handle nested types in cudf::concatenate_rows() (#10890)
b26aaf7 
Addresses:  #10800

Previously, `concatenate_rows()` only handled lists one level deep.   This PR adds support for arbitrary nesting.

<s>This PR is dependent on :  #10880 so some of the files currently showing up on the diff are unrelated to the `concatenate_rows()` change.</s>

Authors:
  - https://github.com/nvdbaranec

Approvers:
  - Robert (Bobby) Evans (https://github.com/revans2)
  - Vukasin Milovanovic (https://github.com/vuule)
  - Nghia Truong (https://github.com/ttnghia)

URL: #10890
@nvdbaranec
Copy link
Contributor

Done.

@bdice bdice removed the Needs Triage Need team to review and classify label Mar 4, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees

@nvdbaranec nvdbaranec

Labels
feature request New feature or request
Projects
None yet
Milestone
No milestone
Development

No branches or pull requests
4 participants
@revans2 @hyperbolic2346 @bdice @nvdbaranec