Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add Apache Spark 3.1.4-SNAPSHOT Shims #4861

Merged
merged 8 commits into from
Mar 10, 2022

Conversation

nartal1
Copy link
Collaborator

@nartal1 nartal1 commented Feb 24, 2022

This fixes #4789.

This PR adds Spark-3.1.4-SNAPSHOT shims. Added test to verify that it works as expected.

Signed-off-by: Niranjan Artal <nartal@nvidia.com>
Signed-off-by: Niranjan Artal <nartal@nvidia.com>
Signed-off-by: Niranjan Artal <nartal@nvidia.com>
@nartal1 nartal1 added the feature request New feature or request label Feb 24, 2022
@nartal1 nartal1 self-assigned this Feb 24, 2022
@nartal1
Copy link
Collaborator Author

nartal1 commented Feb 24, 2022

@tgravescs Could you please let me know the priority of this over #4857. I can wait until that PR is merged and upmerge my changes to reflect the new design. If we want to enable 3.1.4-SNAPSHOT nightly builds sooner then I think this should be merged earlier.

@nartal1 nartal1 added this to the Feb 14 - Feb 25 milestone Feb 24, 2022
@tgravescs
Copy link
Collaborator

either way, I think 3.1.4 can wait if the other PR is large

@nartal1 nartal1 marked this pull request as draft February 24, 2022 21:46
@nartal1
Copy link
Collaborator Author

nartal1 commented Feb 24, 2022

either way, I think 3.1.4 can wait if the other PR is large

Thanks for the confirmation. Have marked this PR as draft. Will upmerge once other PR is merged.

@razajafri
Copy link
Collaborator

Thanks @nartal1

@tgravescs
Copy link
Collaborator

@nartal1 please update, the other pr has been merged

@nartal1 nartal1 marked this pull request as ready for review March 8, 2022 23:32
@nartal1
Copy link
Collaborator Author

nartal1 commented Mar 8, 2022

build

nartal1 added 2 commits March 9, 2022 11:00
Signed-off-by: Niranjan Artal <nartal@nvidia.com>
Signed-off-by: Niranjan Artal <nartal@nvidia.com>
@tgravescs
Copy link
Collaborator

build

Copy link
Collaborator

@tgravescs tgravescs left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

We need to get integration test pipeline going for this.

@tgravescs tgravescs merged commit 5eb5e49 into NVIDIA:branch-22.04 Mar 10, 2022
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
feature request New feature or request
Projects
None yet
Development

Successfully merging this pull request may close these issues.

[FEA] Add Spark 3.1.4 shim
4 participants