Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

refactor: move PyPI related extension and repository_rule setup to its own dir #2003

Merged
merged 25 commits into from
Jun 23, 2024

Conversation

aignas
Copy link
Collaborator

@aignas aignas commented Jun 22, 2024

This is to ensure that future work is easier and its clear where to add tests.
Now all of the unit tests can be run by just bazel test //tests/pypi/....

Refactor summary:

  • chore: add aignas to CODEOWNERS
  • chore: add a new directory for storing PyPI related code
  • move pypi_index_sources.bzl to private/pypi
  • chore: move parse_requirements_txt to private/pypi
  • move parse_whl_name to private/pypi
  • move whl_target_platforms to private/pypi
  • move parse_requirements to private/pypi
  • move pip_repo_name to private/pypi
  • remove unused file
  • move pip_config_settings to private/pypi
  • move pypi_index to pypi/private and rename
  • move labels.bzl to private/pypi
  • move generate_build_bazel to private/pypi
  • move render_pkg_aliases.bzl to private/pypi
  • move patch_whl.bzl to private/pypi
  • Move envsubst and render_tests to top level of tests
  • move pip_install_srcs to private/pypi
  • split and move pip_install/pip_repository.bzl to private/pypi
  • move the bzlmod extension to private/pypi

@aignas aignas marked this pull request as ready for review June 22, 2024 09:46
@aignas aignas requested a review from rickeylev as a code owner June 22, 2024 09:46
@aignas
Copy link
Collaborator Author

aignas commented Jun 22, 2024

This is ready to be reviewed. The main idea was to move all of the code that is
mostly used in the repository_rule and module context to a single location
so that code reuse would be more ergonomic. It also makes it easier to run
tests when doing refactorings as all of the tests are also collected into a
single place.

FYI @groodt, I haven't moved any of the locking code and I am wondering if we should move it to the same place.

@groodt
Copy link
Collaborator

groodt commented Jun 22, 2024

FYI @groodt, I haven't moved any of the locking code and I am wondering if we should move it to the same place.

Yes please! The structure has been driving me dizzy for ages.

Copy link
Collaborator

@groodt groodt left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

@aignas aignas added this pull request to the merge queue Jun 23, 2024
Merged via the queue into bazelbuild:main with commit 04a803c Jun 23, 2024
4 checks passed
@aignas
Copy link
Collaborator Author

aignas commented Jun 23, 2024 via email

aignas added a commit to aignas/rules_python that referenced this pull request Jun 23, 2024
github-merge-queue bot pushed a commit that referenced this pull request Jun 23, 2024
…2006)

A continuation of #2003, this time we finish moving the `pip_compile`
stuff.

Summary:
- move multi_pip_parse to private/pypi and re-export
- remove unused files leftover from #2003
- move repositories.bzl to private/pypi/deps.bzl
- move pip_compile to private/pypi
- move the pip_install tools to private
renovate bot referenced this pull request in bazel-contrib/rules_bazel_integration_test Jul 5, 2024
[![Mend
Renovate](https://app.renovatebot.com/images/banner.svg)](https://renovatebot.com)

This PR contains the following updates:

| Package | Type | Update | Change |
|---|---|---|---|
| [rules_python](https://togithub.com/bazelbuild/rules_python) |
http_archive | minor | `0.33.2` -> `0.34.0` |

---

### Release Notes

<details>
<summary>bazelbuild/rules_python (rules_python)</summary>

###
[`v0.34.0`](https://togithub.com/bazelbuild/rules_python/releases/tag/0.34.0)

[Compare
Source](https://togithub.com/bazelbuild/rules_python/compare/0.33.2...0.34.0)

#### Using Bzlmod with Bazel 6

**NOTE: bzlmod support is still beta. APIs subject to change.**

Add to your `MODULE.bazel` file:

```starlark
bazel_dep(name = "rules_python", version = "0.34.0")

pip = use_extension("@&#8203;rules_python//python/extensions:pip.bzl", "pip")

pip.parse(
    hub_name = "pip",
    python_version = "3.11",
    requirements_lock = "//:requirements_lock.txt",
)

use_repo(pip, "pip")
```

#### Using WORKSPACE

Paste this snippet into your `WORKSPACE` file:

```starlark
load("@&#8203;bazel_tools//tools/build_defs/repo:http.bzl", "http_archive")

http_archive(
    name = "rules_python",
    sha256 = "778aaeab3e6cfd56d681c89f5c10d7ad6bf8d2f1a72de9de55b23081b2d31618",
    strip_prefix = "rules_python-0.34.0",
    url = "https://github.com/bazelbuild/rules_python/releases/download/0.34.0/rules_python-0.34.0.tar.gz",
)

load("@&#8203;rules_python//python:repositories.bzl", "py_repositories")

py_repositories()
```

##### Gazelle plugin

Paste this snippet into your `WORKSPACE` file:

```starlark
load("@&#8203;bazel_tools//tools/build_defs/repo:http.bzl", "http_archive")
http_archive(
    name = "rules_python_gazelle_plugin",
    sha256 = "778aaeab3e6cfd56d681c89f5c10d7ad6bf8d2f1a72de9de55b23081b2d31618",
    strip_prefix = "rules_python-0.34.0/gazelle",
    url = "https://github.com/bazelbuild/rules_python/releases/download/0.34.0/rules_python-0.34.0.tar.gz",
)

### To compile the rules_python gazelle extension from source,
### we must fetch some third-party go dependencies that it uses.

load("@&#8203;rules_python_gazelle_plugin//:deps.bzl", _py_gazelle_deps = "gazelle_deps")

_py_gazelle_deps()
```

#### What's Changed

- fix: Remove transitive legacy struct provider usage by
[@&#8203;fmeum](https://togithub.com/fmeum) in
[https://github.com/bazelbuild/rules_python/pull/1957](https://togithub.com/bazelbuild/rules_python/pull/1957)
- docs: fix doc for how to use new bootstrap impl by
[@&#8203;rickeylev](https://togithub.com/rickeylev) in
[https://github.com/bazelbuild/rules_python/pull/1959](https://togithub.com/bazelbuild/rules_python/pull/1959)
- chore: various cleanups to make google imports/patching easier by
[@&#8203;rickeylev](https://togithub.com/rickeylev) in
[https://github.com/bazelbuild/rules_python/pull/1958](https://togithub.com/bazelbuild/rules_python/pull/1958)
- build(deps): bump idna from 3.4 to 3.7 in /tools/publish by
[@&#8203;dependabot](https://togithub.com/dependabot) in
[https://github.com/bazelbuild/rules_python/pull/1849](https://togithub.com/bazelbuild/rules_python/pull/1849)
- build(deps): bump cryptography from 41.0.6 to 42.0.4 in /tools/publish
by [@&#8203;dependabot](https://togithub.com/dependabot) in
[https://github.com/bazelbuild/rules_python/pull/1772](https://togithub.com/bazelbuild/rules_python/pull/1772)
- ci: add a custom dependabot file by
[@&#8203;aignas](https://togithub.com/aignas) in
[https://github.com/bazelbuild/rules_python/pull/1960](https://togithub.com/bazelbuild/rules_python/pull/1960)
- build(deps): bump pygments from 2.17.2 to 2.18.0 in /docs/sphinx by
[@&#8203;dependabot](https://togithub.com/dependabot) in
[https://github.com/bazelbuild/rules_python/pull/1966](https://togithub.com/bazelbuild/rules_python/pull/1966)
- build(deps): bump actions/checkout from 2 to 4 by
[@&#8203;dependabot](https://togithub.com/dependabot) in
[https://github.com/bazelbuild/rules_python/pull/1961](https://togithub.com/bazelbuild/rules_python/pull/1961)
- build(deps): bump actions/stale from 3 to 9 by
[@&#8203;dependabot](https://togithub.com/dependabot) in
[https://github.com/bazelbuild/rules_python/pull/1962](https://togithub.com/bazelbuild/rules_python/pull/1962)
- build(deps): bump markupsafe from 2.1.3 to 2.1.5 in /docs/sphinx by
[@&#8203;dependabot](https://togithub.com/dependabot) in
[https://github.com/bazelbuild/rules_python/pull/1964](https://togithub.com/bazelbuild/rules_python/pull/1964)
- build(deps): bump softprops/action-gh-release from 1 to 2 by
[@&#8203;dependabot](https://togithub.com/dependabot) in
[https://github.com/bazelbuild/rules_python/pull/1963](https://togithub.com/bazelbuild/rules_python/pull/1963)
- build(deps): bump typing-extensions from 4.9.0 to 4.12.2 in
/docs/sphinx by [@&#8203;dependabot](https://togithub.com/dependabot) in
[https://github.com/bazelbuild/rules_python/pull/1965](https://togithub.com/bazelbuild/rules_python/pull/1965)
- fix(bzlmod): use same target in requirement('foo') and
all_requirements by [@&#8203;jvolkman](https://togithub.com/jvolkman) in
[https://github.com/bazelbuild/rules_python/pull/1973](https://togithub.com/bazelbuild/rules_python/pull/1973)
- refactor(bzlmod): generate fewer targets for pip_config_settings by
[@&#8203;aignas](https://togithub.com/aignas) in
[https://github.com/bazelbuild/rules_python/pull/1974](https://togithub.com/bazelbuild/rules_python/pull/1974)
- chore: access ctx.version_file to inform Bazel stamping info is needed
by [@&#8203;rickeylev](https://togithub.com/rickeylev) in
[https://github.com/bazelbuild/rules_python/pull/1952](https://togithub.com/bazelbuild/rules_python/pull/1952)
- build(deps): bump urllib3 from 1.26.18 to 1.26.19 in /tools/publish by
[@&#8203;dependabot](https://togithub.com/dependabot) in
[https://github.com/bazelbuild/rules_python/pull/1981](https://togithub.com/bazelbuild/rules_python/pull/1981)
- build(deps): bump babel from 2.13.1 to 2.15.0 in /docs/sphinx by
[@&#8203;dependabot](https://togithub.com/dependabot) in
[https://github.com/bazelbuild/rules_python/pull/1983](https://togithub.com/bazelbuild/rules_python/pull/1983)
- docs: Fix a few typos in docs and function names by
[@&#8203;rickeylev](https://togithub.com/rickeylev) in
[https://github.com/bazelbuild/rules_python/pull/1978](https://togithub.com/bazelbuild/rules_python/pull/1978)
- fix: enable auto exec groups using attribute by
[@&#8203;rickeylev](https://togithub.com/rickeylev) in
[https://github.com/bazelbuild/rules_python/pull/1986](https://togithub.com/bazelbuild/rules_python/pull/1986)
- docs: Remove proposals by
[@&#8203;groodt](https://togithub.com/groodt) in
[https://github.com/bazelbuild/rules_python/pull/1990](https://togithub.com/bazelbuild/rules_python/pull/1990)
- chore: use `**` globs instead of explcit BUILD.bazel reference in
distribution filegroups by
[@&#8203;rickeylev](https://togithub.com/rickeylev) in
[https://github.com/bazelbuild/rules_python/pull/1991](https://togithub.com/bazelbuild/rules_python/pull/1991)
- refactor(internal): allow setting linking mode for internal cc_details
APIs by [@&#8203;rickeylev](https://togithub.com/rickeylev) in
[https://github.com/bazelbuild/rules_python/pull/1982](https://togithub.com/bazelbuild/rules_python/pull/1982)
- fix(toolchain): disable exec toolchain by default by
[@&#8203;aignas](https://togithub.com/aignas) in
[https://github.com/bazelbuild/rules_python/pull/1968](https://togithub.com/bazelbuild/rules_python/pull/1968)
- fix: Resolve the test manifest with the runfiles lib. by
[@&#8203;lberki](https://togithub.com/lberki) in
[https://github.com/bazelbuild/rules_python/pull/1993](https://togithub.com/bazelbuild/rules_python/pull/1993)
- fix: allow creating PyRuntimeInfo without specifying
interpreter_version_info by
[@&#8203;rickeylev](https://togithub.com/rickeylev) in
[https://github.com/bazelbuild/rules_python/pull/1992](https://togithub.com/bazelbuild/rules_python/pull/1992)
- doc: Changelog updates for 0.33.2 by
[@&#8203;aignas](https://togithub.com/aignas) in
[https://github.com/bazelbuild/rules_python/pull/1995](https://togithub.com/bazelbuild/rules_python/pull/1995)
- refactor!: Remove entrypoint by
[@&#8203;groodt](https://togithub.com/groodt) in
[https://github.com/bazelbuild/rules_python/pull/1987](https://togithub.com/bazelbuild/rules_python/pull/1987)
- refactor: lookup exec interpreter using toolchain resolution by
[@&#8203;rickeylev](https://togithub.com/rickeylev) in
[https://github.com/bazelbuild/rules_python/pull/1997](https://togithub.com/bazelbuild/rules_python/pull/1997)
- fix: Add `configure=True` for internal_config_repo by
[@&#8203;meteorcloudy](https://togithub.com/meteorcloudy) in
[https://github.com/bazelbuild/rules_python/pull/1998](https://togithub.com/bazelbuild/rules_python/pull/1998)
- chore: remove dead code from whl_target_platforms by
[@&#8203;aignas](https://togithub.com/aignas) in
[https://github.com/bazelbuild/rules_python/pull/2001](https://togithub.com/bazelbuild/rules_python/pull/2001)
- fix(bzlmod): pass only non-default values to whl_library by
[@&#8203;aignas](https://togithub.com/aignas) in
[https://github.com/bazelbuild/rules_python/pull/2002](https://togithub.com/bazelbuild/rules_python/pull/2002)
- refactor: move PyPI related extension and repository_rule setup to its
own dir by [@&#8203;aignas](https://togithub.com/aignas) in
[https://github.com/bazelbuild/rules_python/pull/2003](https://togithub.com/bazelbuild/rules_python/pull/2003)
- docs: Removes dead link after docs were migrated to sphinx by
[@&#8203;groodt](https://togithub.com/groodt) in
[https://github.com/bazelbuild/rules_python/pull/2005](https://togithub.com/bazelbuild/rules_python/pull/2005)
- refactor: Remove unused template substitution by
[@&#8203;groodt](https://togithub.com/groodt) in
[https://github.com/bazelbuild/rules_python/pull/2004](https://togithub.com/bazelbuild/rules_python/pull/2004)
- refactor: move the remaining PyPI related functions to private/pypi by
[@&#8203;aignas](https://togithub.com/aignas) in
[https://github.com/bazelbuild/rules_python/pull/2006](https://togithub.com/bazelbuild/rules_python/pull/2006)
- chore: adjust CODEOWNERS by
[@&#8203;aignas](https://togithub.com/aignas) in
[https://github.com/bazelbuild/rules_python/pull/2008](https://togithub.com/bazelbuild/rules_python/pull/2008)
- fix: make first default output the executable again by
[@&#8203;rickeylev](https://togithub.com/rickeylev) in
[https://github.com/bazelbuild/rules_python/pull/2010](https://togithub.com/bazelbuild/rules_python/pull/2010)
- refactor: call a function to define internal pypi flags instead of
listcomp by [@&#8203;rickeylev](https://togithub.com/rickeylev) in
[https://github.com/bazelbuild/rules_python/pull/2011](https://togithub.com/bazelbuild/rules_python/pull/2011)
- chore: bump sphinxdocs deps by
[@&#8203;aignas](https://togithub.com/aignas) in
[https://github.com/bazelbuild/rules_python/pull/2013](https://togithub.com/bazelbuild/rules_python/pull/2013)
- fix(whl_library): correctly handle arch-specific deps in wheels by
[@&#8203;aignas](https://togithub.com/aignas) in
[https://github.com/bazelbuild/rules_python/pull/2007](https://togithub.com/bazelbuild/rules_python/pull/2007)
- fix(multiplatform): Add i386 Linux support by
[@&#8203;nicbadiu](https://togithub.com/nicbadiu) in
[https://github.com/bazelbuild/rules_python/pull/1999](https://togithub.com/bazelbuild/rules_python/pull/1999)
- fix: Fix broken logger statement in parse_requirements.bzl by
[@&#8203;mark-thm](https://togithub.com/mark-thm) in
[https://github.com/bazelbuild/rules_python/pull/2017](https://togithub.com/bazelbuild/rules_python/pull/2017)
- feat: add runtime_env toolchain suite to replace "autodetecting"
toolchain by [@&#8203;rickeylev](https://togithub.com/rickeylev) in
[https://github.com/bazelbuild/rules_python/pull/2018](https://togithub.com/bazelbuild/rules_python/pull/2018)
- fix(bzlmod): only expose common packages via the requirements
constants by [@&#8203;aignas](https://togithub.com/aignas) in
[https://github.com/bazelbuild/rules_python/pull/1955](https://togithub.com/bazelbuild/rules_python/pull/1955)
- feat(gazelle): Add directives for label format & normalisation by
[@&#8203;wingsofovnia](https://togithub.com/wingsofovnia) in
[https://github.com/bazelbuild/rules_python/pull/1976](https://togithub.com/bazelbuild/rules_python/pull/1976)
- chore: Remove unused renovate.json by
[@&#8203;groodt](https://togithub.com/groodt) in
[https://github.com/bazelbuild/rules_python/pull/2021](https://togithub.com/bazelbuild/rules_python/pull/2021)
- (chore): Remove the extra indirection and `bzlmod` folder by
[@&#8203;groodt](https://togithub.com/groodt) in
[https://github.com/bazelbuild/rules_python/pull/2022](https://togithub.com/bazelbuild/rules_python/pull/2022)

#### New Contributors

- [@&#8203;lberki](https://togithub.com/lberki) made their first
contribution in
[https://github.com/bazelbuild/rules_python/pull/1993](https://togithub.com/bazelbuild/rules_python/pull/1993)
- [@&#8203;nicbadiu](https://togithub.com/nicbadiu) made their first
contribution in
[https://github.com/bazelbuild/rules_python/pull/1999](https://togithub.com/bazelbuild/rules_python/pull/1999)
- [@&#8203;mark-thm](https://togithub.com/mark-thm) made their first
contribution in
[https://github.com/bazelbuild/rules_python/pull/2017](https://togithub.com/bazelbuild/rules_python/pull/2017)
- [@&#8203;wingsofovnia](https://togithub.com/wingsofovnia) made their
first contribution in
[https://github.com/bazelbuild/rules_python/pull/1976](https://togithub.com/bazelbuild/rules_python/pull/1976)

**Full Changelog**:
bazelbuild/rules_python@0.33.1...0.34.0

</details>

---

### Configuration

📅 **Schedule**: Branch creation - At any time (no schedule defined),
Automerge - At any time (no schedule defined).

🚦 **Automerge**: Enabled.

♻ **Rebasing**: Whenever PR is behind base branch, or you tick the
rebase/retry checkbox.

🔕 **Ignore**: Close this PR and you won't be reminded about this update
again.

---

- [ ] <!-- rebase-check -->If you want to rebase/retry this PR, check
this box

---

This PR has been generated by [Mend
Renovate](https://www.mend.io/free-developer-tools/renovate/). View
repository job log
[here](https://developer.mend.io/github/bazel-contrib/rules_bazel_integration_test).

<!--renovate-debug:eyJjcmVhdGVkSW5WZXIiOiIzNy40MjEuOSIsInVwZGF0ZWRJblZlciI6IjM3LjQyMS45IiwidGFyZ2V0QnJhbmNoIjoibWFpbiIsImxhYmVscyI6W119-->

Co-authored-by: renovate[bot] <29139614+renovate[bot]@users.noreply.github.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants