Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Update the python deps #32756

Merged
merged 9 commits into from
Nov 15, 2024
Merged

Update the python deps #32756

merged 9 commits into from
Nov 15, 2024

Conversation

liferoad
Copy link
Collaborator

@liferoad liferoad commented Oct 11, 2024

Fixes #32681

Beam does not directly depend on numpy 2.0.

Following https://cwiki.apache.org/confluence/display/BEAM/Python+Tips#PythonTips-HowtoupdatedependenciesthatareinstalledinPythoncontainerimages:

./gradlew :sdks:python:container:generatePythonRequirementsAll

and https://cwiki.apache.org/confluence/display/BEAM/Python+Tips#PythonTips-InstallingPythoninterpreters, updated the base image deps.

Pin protobuf to 4.25.5 and grpcio-status to 1.62.3 since some ML packages (TF) still depend on protobuf 4.x


Thank you for your contribution! Follow this checklist to help us incorporate your contribution quickly and easily:

  • Mention the appropriate issue in your description (for example: addresses #123), if applicable. This will automatically add a link to the pull request in the issue. If you would like the issue to automatically close on merging the pull request, comment fixes #<ISSUE NUMBER> instead.
  • Update CHANGES.md with noteworthy changes.
  • If this contribution is large, please file an Apache Individual Contributor License Agreement.

See the Contributor Guide for more tips on how to make review process smoother.

To check the build health, please visit https://github.com/apache/beam/blob/master/.test-infra/BUILD_STATUS.md

GitHub Actions Tests Status (on master branch)

Build python source distribution and wheels
Python tests
Java tests
Go tests

See CI.md for more information about GitHub Actions CI or the workflows README to see a list of phrases to trigger workflows.

@liferoad
Copy link
Collaborator Author

cc @jrmccluskey

Copy link
Contributor

Assigning reviewers. If you would like to opt out of this review, comment assign to next reviewer:

R: @damccorm for label python.

Available commands:

  • stop reviewer notifications - opt out of the automated review tooling
  • remind me after tests pass - tag the comment author after tests pass
  • waiting on author - shift the attention set back to the author (any comment or push by the author will return the attention set to the reviewers)

The PR bot will only process comments in the main thread (not review comments).

Copy link
Contributor

Reminder, please take a look at this pr: @damccorm

@liferoad
Copy link
Collaborator Author

We will pin protobuf==4.25.5 for now with Beam 2.61.0.

@liferoad liferoad added this to the 2.61.0 Release milestone Oct 23, 2024
@liferoad
Copy link
Collaborator Author

cc @Abacn later I will polish this PR to update the deps. FYI.

Copy link
Contributor

Assigning new set of reviewers because Pr has gone too long without review. If you would like to opt out of this review, comment assign to next reviewer:

R: @tvalentyn for label python.

Available commands:

  • stop reviewer notifications - opt out of the automated review tooling
  • remind me after tests pass - tag the comment author after tests pass
  • waiting on author - shift the attention set back to the author (any comment or push by the author will return the attention set to the reviewers)

@liferoad
Copy link
Collaborator Author

stop reviewer notifications

Copy link
Contributor

Stopping reviewer notifications for this pull request: requested by reviewer. If you'd like to restart, comment assign set of reviewers

@liferoad liferoad changed the title Update the python deps caused by numpy 2.0 Update the python deps Oct 30, 2024
Copy link
Contributor

@jrmccluskey jrmccluskey left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

@tvalentyn
Copy link
Contributor

Given how close we are to the release cut, I would keep the dependency changes to a minimum or do this update after the release cut.

@tvalentyn
Copy link
Contributor

For example, if there is a regression in google-api-core==2.22.0, we have only two weeks to notice that.

@liferoad
Copy link
Collaborator Author

@damccorm what do you think? I am fine to do this after 2.61.0.

@damccorm
Copy link
Contributor

Its borderline, but I probably agree waiting is safer. I don't think we need to do this for the current release

@damccorm damccorm removed this from the 2.61.0 Release milestone Oct 31, 2024
@damccorm damccorm added this to the 2.62.0 Release milestone Oct 31, 2024
@liferoad
Copy link
Collaborator Author

Rebased. Based on the discussions, we should get this in ASAP after 2.61.0.

Copy link
Contributor

@damccorm damccorm left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

@liferoad
Copy link
Collaborator Author

@liferoad liferoad merged commit 7650a6d into apache:master Nov 15, 2024
85 of 90 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging this pull request may close these issues.

[Task]: Upgrade numpy to 2.x.
4 participants