Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Sync plugin support for hash-hive and shift operators #1198

Merged
merged 1 commit into from
Jul 17, 2024

Conversation

amahussein
Copy link
Collaborator

@amahussein amahussein commented Jul 17, 2024

Signed-off-by: Ahmed Hussein (amahussein) a@ahussein.me

Fixes #1193

  • HiveHash is added as a new expression. It is unlikely that Q tool enconters that expression if it appears deep inside write command for hash bucketing.
  • updated the SQL function of shift operators. Spark-4.0 added shorthand for shift operators ">>", "<<", and ">>>"
  • update unit tests

Signed-off-by: Ahmed Hussein (amahussein) <a@ahussein.me>

Fixes NVIDIA#1193

- HiveHash is added as a new epression. It is unlikely that Q tool
  enconters that expression if it appears deep inside write command for
hash bucketing.
- updated the SQL function of shift operators. Spark-4.0 added shorthand
  for shift operators ">>", "<<", and ">>>"
- update unit tests
@amahussein amahussein added bug Something isn't working core_tools Scope the core module (scala) labels Jul 17, 2024
@amahussein amahussein self-assigned this Jul 17, 2024
Copy link
Collaborator

@nartal1 nartal1 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM. Thanks @amahussein !

@amahussein amahussein requested a review from parthosa July 17, 2024 17:10
Copy link
Collaborator

@parthosa parthosa left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks @amahussein. LGTME.

@amahussein amahussein merged commit 49e6727 into NVIDIA:dev Jul 17, 2024
15 checks passed
@amahussein amahussein deleted the spark-rapids-tools-1193 branch July 17, 2024 17:34
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
bug Something isn't working core_tools Scope the core module (scala)
Projects
None yet
Development

Successfully merging this pull request may close these issues.

[FEA] Sync tools supported ops with plugin
3 participants