Duplication of runfiles for py_binary when using rules_python version

[brussina]

I have a question about a difference I noticed in outputs produced by the built-in py_binary versus the rules_python py_binary when building the same target. I have a trivial project that builds a single py_binary called python_binary_1. When I build python_binary_1 with the built-in py_binary I get the following output in bazel-out:

 python_binary_1
 python_binary_1.repo_mapping
 python_binary_1.runfiles
 python_binary_1.runfiles_manifest

And, when building with the rules_python py_binary I get this:

_python_binary_1   
python_binary_1 -> /home/user/.cache/bazel/_bazel_user/219bd1e9c37478b47a9e29ee5f5a29fc/execroot/project2/bazel-out/k8-fastbuild-ST-0a0f5b1d153b/bin/python_binary_1/_python_binary_1
_python_binary_1.repo_mapping
python_binary_1.repo_mapping
_python_binary_1.runfiles
python_binary_1.runfiles
_python_binary_1.runfiles_manifest
python_binary_1.runfiles_manifest

Both versions work fine. My question is why does the rules_python version produce the second copy of all the files with the leading underscore? Both the python_binary_1.runfiles and _python_binary_1.runfiles contain the exact same contents (basically a bunch of symlinks pointing back to the pip install in the external folder). The particular issue this causes us is that we have a monorepo with a large number of python projects and this change has doubled the number of files produced by a build. Since they are symlinks it's not an issue of disk space, but it is an issue from the perspective of inode use. Just wondering what's the purpose of this duplication and if there is a way to suppress it?

Thanks for your attention.

[rickeylev]

This is probably related to using a version-aware py_binary rule. Under the hood, the way it works is a non-version aware py_binary is created, then a version-aware wrapper rule (also a binary) applies a transition and depends on the inner binary. The version-aware wrapper basically just forwards along information from the inner binary. python/config_settings/transition.bzl has the code for this.

However, this shouldn't always cause both to materialize in bazel-bin. Only the requested target (the outer wrapper) should be materialized. If both are showing up, then it would mean something is explicitly building the inner binary (which shouldn't be done, its just an helper to the outer target).

[brussina]

Really appreciate the feedback. I'm definitely building a version aware py_binary. My BUILD.bazel looks like:

load("@python_3_8//:defs.bzl", py_binary_3_8="py_binary")

py_binary_3_8(
  name = "python_binary_1",
  srcs = ["main.py"],
  main = "main.py",
)

and WORKSPACE.bazel is:

workspace(name="project2")

load("@bazel_tools//tools/build_defs/repo:http.bzl", "http_archive")

http_archive(
  name = "rules_python",
  sha256 = "be04b635c7be4604be1ef20542e9870af3c49778ce841ee2d92fcb42f9d9516a",
  strip_prefix = "rules_python-0.35.0",
  url = "https://github.com/bazelbuild/rules_python/releases/download/0.35.0/rules_python-0.35.0.tar.gz",
)

load("@rules_python//python:repositories.bzl",
  "py_repositories",
  "python_register_toolchains",
)

py_repositories()


python_register_toolchains(
  name = "python_3_8",
  python_version = "3.8",
  register_coverage_tool = False
)

load("@python_3_8//:defs.bzl", "interpreter")
load("@rules_python//python:pip.bzl", "pip_parse")

pip_parse(
  name = "pip_3_8",
  python_interpreter_target = interpreter,
  requirements_lock = "//pyenv/3.8:requirements.txt"
)

load("@pip_3_8//:requirements.bzl", "install_deps")
install_deps()

Appreciate the pointer to transition.bzl. I will see if I can trace through it and see what is building both versions.