Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

experimental_target_platforms produces whl with duplicated dependencies #1873

Closed
wingsofovnia opened this issue May 3, 2024 · 0 comments · Fixed by #1874
Closed

experimental_target_platforms produces whl with duplicated dependencies #1873

wingsofovnia opened this issue May 3, 2024 · 0 comments · Fixed by #1874

Comments

@wingsofovnia
Copy link
Contributor

wingsofovnia commented May 3, 2024

🐞 bug report

Affected Rule

The issue is caused by the rule:

pip.parse

Is this a regression?

Can't test. The reproduction doesn't build on older versions due to #1810

Description

When experimental_target_platforms is used, some whl targets generated have duplicated dependencies, e.g.:

filegroup(
    name = "whl",
    srcs = ["opencv_python-4.8.1.78-cp37-abi3-macosx_11_0_arm64.whl"],
    data = ["@pip//numpy:whl"] + select(
        {
            "@@platforms//os:osx": ["@pip//numpy:whl"],
            "//conditions:default": [],
        },
    ),
    visibility = ["//visibility:public"],
)

🔬 Minimal Reproduction

https://github.com/wingsofovnia/rules_python-issues-1873

Lock files generated via bazel run //:requirements_lock.update:

  • requirements.txt.darwin_aarch64.lock generated on Mac M1
  • requirements.txt.linux_x86_32.lock generated in docker run --platform linux/amd64 container

🔥 Exception or Error

ERROR: /private/var/tmp/.../rules_python~~pip~pip_310_opencv_python/BUILD.bazel:16:10: 
   in data attribute of filegroup rule @@rules_python~~pip~pip_310_opencv_python//:whl: 
      Label '@@rules_python~~pip~pip//numpy:whl' is duplicated

🌍 Your Environment

Operating System:

> uname -a # requirements.txt.darwin_aarch64.lock
Darwin CDDEBERMC005031 23.4.0 Darwin Kernel Version 23.4.0: Fri Mar 15 00:10:42 PDT 2024; root:xnu-10063.101.17~1/RELEASE_ARM64_T6000 arm64
> uname -a # requirements.txt.linux_x86_32.lock
Linux f74bbe09936a 6.6.22-linuxkit #1 SMP Fri Mar 29 12:21:27 UTC 2024 x86_64 x86_64 x86_64 GNU/Linux

Output of bazel version:

bazel version
Bazelisk version: v1.19.0
Build label: 7.1.1
Build target: @@//src/main/java/com/google/devtools/build/lib/bazel:BazelServer
Build time: Thu Mar 21 18:08:37 2024 (1711044517)
Build timestamp: 1711044517
Build timestamp as int: 1711044517

Rules_python version:

HEAD (d3cec48e415dd598a773335532cbc5647c985a93)
aignas added a commit to aignas/rules_python that referenced this issue May 4, 2024
Before this PR we would incorrectly add deps to the platform-specific
list if there were multiple entries in the `METADATA` file. It seems
that some projects (e.g. [opencv-python]) have multiple entries in their
METADATA file to help SAT solvers with selecting the right version when
different interpreter versions are used.

In our case, we will have only one version of a given package because we
are operating with a locked dependency list, so we should ensure that we
do not have duplicates across the lists. With this PR we are solving
this during the construction of the dependency sets so that the internal
model is always consistent.

Fixes bazelbuild#1873

[opencv-python]: https://pypi.org/project/opencv-python/
github-merge-queue bot pushed a commit that referenced this issue May 12, 2024
Before this PR we would incorrectly add deps to the platform-specific
list if there were multiple entries in the `METADATA` file. It seems
that some projects (e.g. [opencv-python]) have multiple entries in their
METADATA file to help SAT solvers with selecting the right version when
different interpreter versions are used.

In our case, we will have only one version of a given package because we
are operating with a locked dependency list, so we should ensure that we
do not have duplicates across the lists. With this PR we are solving
this during the construction of the dependency sets so that the internal
model is always consistent.

Fixes #1873

[opencv-python]: https://pypi.org/project/opencv-python/
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging a pull request may close this issue.

1 participant