Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

feat(rules): add PyExecutableInfo #2166

Merged
merged 2 commits into from
Sep 2, 2024

Conversation

rickeylev
Copy link
Contributor

The PyExecutableInfo provider exposes executable-specific information that isn't easily accessible outside the target. The main purpose of this provider is to facilitate packaging a binary or deriving a new binary based upon the original.

Within rules_python, this will be used to pull the zip-building logic out of executables and into separate rules. Within Google, this will be used for a similar "package a binary" tool.

Along the way:

  • Add runfiles references to sphinx inventory

@rickeylev rickeylev force-pushed the feat.pybininfo branch 4 times, most recently from 744da90 to 1b37eea Compare August 31, 2024 16:51
@rickeylev rickeylev requested a review from oprypin August 31, 2024 16:58
The PyExecutableInfo provider exposes executable-specific information
that isn't easily accessible outside the target. The main purpose of
this provider is to facilitate packaging a binary or deriving a new
binary based upon the original.

Within rules_python, this will be used to pull the zip-building logic
out of executables and into separate rules.
@rickeylev rickeylev marked this pull request as ready for review August 31, 2024 17:00
@rickeylev rickeylev requested a review from aignas as a code owner August 31, 2024 17:00
@aignas
Copy link
Collaborator

aignas commented Sep 1, 2024

Just brainstorming here.

This might be also beneficial for rules_oci users who would like to create a tarball with a different layout of the files.

I think in a similar fashion we could also list sources for each third party package so that users can also do whatever they want with that. Having some sort of depset map could be useful. That way people would not need to rely on site-packages string being in the path and we could also have a venv building rule.

@rickeylev
Copy link
Contributor Author

Yes, packaging a py_binary into a tar is the sort of use case I have in mind. That's not much different that a zip file, really.

rules_oci + reorganizing site-packages gives me another use case idea: packaging multiple binaries into an image and reducing duplication. So don't include the python runtime in runfiles_without_exe (it's already available through PyRuntimeInfo), and have a way to identify which pip.parse hub things came from (to create a site-packages per hub).

@rickeylev rickeylev added this pull request to the merge queue Sep 2, 2024
Merged via the queue into bazelbuild:main with commit 36bb556 Sep 2, 2024
4 checks passed
@rickeylev rickeylev deleted the feat.pybininfo branch September 3, 2024 03:10
oprypin added a commit that referenced this pull request Sep 4, 2024
PyExecutableInfo was added in #2166 with the field `runfiles_without_exe` that intentionally excludes unrelated files. But, within Google we found that we need to pick up the file with build stamp information at that stage as well, so it is being additionally exposed.
github-merge-queue bot pushed a commit that referenced this pull request Sep 6, 2024
PyExecutableInfo was added in
#2166 with the field
`runfiles_without_exe` that intentionally excludes files that are
specific to
that target/executable, such as the build data file (which may contain
the target name,
or other target-specific information).

However, consuming tools (such as ones used within Google) may need to
derive a file from
that build data, override it completely, or be happy with its content as
is. To aid that
case, expose it via PyExecutableInfo.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants