Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[FEA] Create pylibcudf.Table from a table_view and an arbitrary owning object #17040

Open
madsbk opened this issue Oct 10, 2024 · 3 comments
Open
Assignees
Labels
feature request New feature or request pylibcudf Issues specific to the pylibcudf package

Comments

@madsbk
Copy link
Member

madsbk commented Oct 10, 2024

In #17012, we have a table_view but no owning Table thus it would be useful to be able to create a pylibcudf.Table from a table_view and an arbitrary owning object (in this case a PackedColumnsinstance).

Similarly, it should be possible to create a pylibcudf.Column from a column_view and an arbitrary owning object.

cc. @wence-

@madsbk madsbk added feature request New feature or request pylibcudf Issues specific to the pylibcudf package labels Oct 10, 2024
@vyasr
Copy link
Contributor

vyasr commented Oct 10, 2024

The main challenge here would be with children. If I call from_column_view(cv, arbitrary) and cv has children, what should their owners be? The current logic is quite strict in assuming that the owning column owns the corresponding buffer for that child's data. How should this work in general? If it shouldn't, should it be forbidden? Should we assume that if we are ingesting views containing nested types that we must be coming from a Column?

@wence-
Copy link
Contributor

wence- commented Oct 11, 2024

In the cudf-classic model, if we have an arbitrary object that we can't decompose, that becomes the owner for every child.

In the cudf::unpack case I think we just have to follow that model if we want to avoid a copy. Every column (and null mask) that comes out of cudf::unpack is backed by the same single allocation, so the owner of every child is the same object.

@vyasr
Copy link
Contributor

vyasr commented Oct 11, 2024

I agree that's probably the best that we can do. Perhaps a fused type of Column | object such that we take the smarter child path for Column and the more naive "all children are owned by owner" path for everything else.

@vyasr vyasr changed the title [FEA] Create pylibcudf.Table from a table_viewand an arbitrary owning object [FEA] Create pylibcudf.Table from a table_view and an arbitrary owning object Oct 28, 2024
@Matt711 Matt711 self-assigned this Dec 5, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
feature request New feature or request pylibcudf Issues specific to the pylibcudf package
Projects
Status: Todo
Development

No branches or pull requests

4 participants