Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

[Audit] [FEA][SPARK-32986][SQL] Add bucketed scan info in query plan of data source v1 #3952

Closed
nartal1 opened this issue Oct 28, 2021 · 2 comments · Fixed by #4461
Closed

[Audit] [FEA][SPARK-32986][SQL] Add bucketed scan info in query plan of data source v1 #3952

nartal1 opened this issue Oct 28, 2021 · 2 comments · Fixed by #4461
Assignees
@HaoYang670
Labels
audit_3.3.0 Audit related tasks for 3.3.0 feature request New feature or request P1 Nice to have for release

Comments

@nartal1
Copy link
Collaborator

nartal1 commented Oct 28, 2021

Is your feature request related to a problem? Please describe.
The PR adds info into FileSourceScanExec physical query plan output whether the table is read as bucketed table. Opening this issue to get feedback if we need to update GpuFileSourceScanExec.scala to include this change.
PR - apache/spark@79515e4b6c
@nartal1 nartal1 added feature request New feature or request ? - Needs Triage Need team to review and classify audit_3.3.0 Audit related tasks for 3.3.0 labels Oct 28, 2021
@Salonijain27 Salonijain27 added P1 Nice to have for release and removed ? - Needs Triage Need team to review and classify labels Nov 2, 2021
@jlowe
Copy link
Member

jlowe commented Nov 2, 2021

I agree we should pull this in, but I don't think it needs to be shimmed. We can just always output the bucketing format added in 3.3 on all Spark versions.

@HaoYang670 HaoYang670 self-assigned this Dec 30, 2021
@HaoYang670
Copy link
Collaborator

I'd like to try.

@HaoYang670 HaoYang670 mentioned this issue Jan 5, 2022
Merged
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees

@HaoYang670 HaoYang670

Labels
audit_3.3.0 Audit related tasks for 3.3.0 feature request New feature or request P1 Nice to have for release
Projects
None yet
Milestone
No milestone
Development

Successfully merging a pull request may close this issue.

[Audit] Add bucketed scan info in query plan of data source v1 [databricks] HaoYang670/spark-rapids
4 participants
@jlowe @Salonijain27 @nartal1 @HaoYang670