Bazel: support bzlmod

[cpsauer]

What would you like Renovate to be able to do?

Hi, @rarkins. Thanks for the wonderfully great tool that is Renovate. It's been spectacular for our Bazel repo--and we point other Bazel users to you guys every chance we get, including in Bazel tools we've open sourced.

I just stumbled upon some docs indicating that Bazel working to update their external dependency management to use a different, more unified file format, and has a working version out. They call the manager bzlmod. @alexeagle, who I know you've cc'd on Bazel issues before, wrote a great blog post on it. I'll just link, because I certainly can't do a better job describing it here. https://blog.aspect.dev/bzlmod

It'll probably take a bit before things switch over, but I thought I should give a heads, and ask if it'd be something you'd consider supporting with Renovate, especially since I didn't see any other issues mentioning it. As far as I can tell, the need for Renovate with it should be as burning as ever; I haven't seen anything about notifying users about updates on the Bazel side.

Thanks so much for your consideration,
Chris

If you have any ideas on how this should be implemented, please tell us here.

Package fetching would probably need to be from the registry, possibly delegating to Bazel. The actual format is simpler than the current WORKSPACE approach, though it's be in a new file, MODULE.bazel.

[rarkins]

Seems like we can treat this as a new Renovate "manager" with name bzlmod while keeping the existing/legacy bazel in place?

Also looks like we need a new datasource for the registry part.

[github-actions]

Hi there,

You're requesting support for a new package manager. We need to know some basic information about this package manager first. Please copy/paste the new package manager questionnaire, and fill it out in full.

Once the questionnaire is filled out we will evaluate if adding support for this manager is something we want to do.

Good luck,

The Renovate team

[cpsauer]

I think, @alexeagle could probably do a better job than I, since I think he's contributing and probably knows the folks leading it. Alex, would you be down?

[alexeagle]

Yeah I'd like to help with this. The feature is still experimental until later this year. Bazel official repos use renovate too, we can probably get help from upstream.

[alexeagle]

Hey there, just a ping since the Bazel team has re-committed to this new dependency manager in their next major release 6.0.

As a concrete example, this spot https://github.com/aspect-build/rules_js/blob/main/MODULE.bazel#L9 ought to get updated whenever https://github.com/bazelbuild/bazel-central-registry/blob/main/modules/aspect_bazel_lib/metadata.json has a new release.

[rarkins]

Hi @alexeagle. Can you or anyone else involved fill out our questionnaire? i.e. #13658 (comment)

This looks like it's essentially a brand new package manager, even if we decide to keep with the "bazel" naming and support both old and new at the same time.

[alexeagle]

New package manager questionnaire

Did you read our documentation on adding a package manager?

• I've read the adding a package manager documentation.

Basics

Name of package manager

bzlmod

What language does this support?

starlark (Bazel's configuration language)

How popular is this package manager?

Not at all yet, but presumably after Bazel 6.0, most Bazel users will migrate to it.

Does this language have other (competing?) package managers?

• Yes (give names)
• No

---

Package File Detection

What type of package files and names does it use?

MODULE.bazel

What fileMatch pattern(s) should be used?

**/MODULE.bazel

Is it likely that many users would need to extend this pattern for custom file names?

• Yes
• No

Is the fileMatch pattern likely to get many "false hits" for files that have nothing to do with package management?

Don't think so

Parsing and Extraction

Can package files have "local" links to each other that need to be resolved?

It can link to local packages, but I don't think renovate would need to be aware of those.

Is there a reason why package files need to be parsed together (in serial) instead of independently?

There's only one such file.

What format/syntax is the package file in?

• JSON
• TOML
• YAML
• Custom (explain below)

Starlark

How do you suggest parsing the file?

• Off the shelf parser
• Using regex
• Custom-parsed line by line
• Other

There are parsers in a few languages, e.g. bazelbuild/starlark-go - but renovate already supports Bazel's existing WORKSPACE files which are already in starlark so maybe this is robust enough.

Does the package file structure distinguish between different "types" of dependencies? e.g. production dependencies, development dependencies, etc?

• Yes, production and development dependencies
• No, all dependencies are treated the same

List all the sources/syntaxes of dependencies that can be extracted

Describe which types of dependencies above are supported and which will be implemented in future

---

Versioning

What versioning scheme does the package file(s) use?

Semver

Does this versioning scheme support range constraints, e.g. ^1.0.0 or 1.x?

• Supports range constraints (e.g ^1.0.0 or 1.x)
• No

HOWEVER there's a https://docs.bazel.build/versions/main/bzlmod.html#compatibility-level concept which is currently poorly understood.

Is this package manager used for applications, libraries, or both? If both, is there a way to tell which is which?

• Applications
• Libraries
• Both (explain how to tell which is which)

I don't think you can tell if a Bazel module is terminal, in the sense that it doesn't allow others to depend on it.

If ranges are supported, are there any cases when Renovate should pin ranges to exact versions if rangeStrategy=auto?

---

Lookup

Is a new datasource required? Provide details

• Yes, provide details.
• No.

https://docs.bazel.build/versions/main/bzlmod.html#bazel-central-registry
This is an HTTP endpoint that is backed by a github repo.

Will users need the capability to specify a custom host/registry to look up? Can it be found within the package files, or within other files inside the repository, or would it require Renovate configuration?

The .bazelrc file can have a custom --registry value, e.g. https://github.com/aspect-build/rules_js/blob/main/e2e/bzlmod/.bazelrc#L2

Do the package files have any "constraints" on the parent language (e.g. supports only v3.x of Python) or platform (Linux, Windows, etc) that should be used in the lookup procedure?

Will users need the ability to configure language or other constraints using Renovate config?

---

Artifacts

Are lock files or checksum files used? Are they mandatory?

No, it's assumed that the registry is immutable. The shasum of the artifact is encoded as part of the registry entry, e.g. https://github.com/bazelbuild/bazel-central-registry/blob/main/modules/aspect_bazel_lib/0.4.2/source.json#L2 but this doesn't appear in the users MODULE.bazel file.

If so, what tool and exact commands should be used if updating one or more package versions in a dependency file?

If applicable, describe how the tool maintains a cache and if it can be controlled via CLI or environment variables? Do you recommend the cache be kept or disabled/ignored?

If applicable, what command should be used to generate a lock file from scratch if you already have a package file? This will be used for "lock file maintenance"

Other

Is there anything else to know about this package manager?

[alexeagle]

FYI the launch for this new package manager is coming up in the next 4-6 weeks, as the primary new feature of Bazel 6.0

[rarkins]

Does bzlmod replace WORKSPACE, or is its purpose to move bazel "modules" out of WORKSPACE and e.g. references to docker images or maven packages will still remain in WORKSPACE files?

https://registry.bazel.build/ is a front for https://github.com/bazelbuild/bazel-central-registry. Does it have a documented API?

On the other hand the syntax for a custom registry is this:

common --registry=https://raw.githubusercontent.com/aspect-build/bazel-central-registry/main/

This implies we might be best to use github to query the default registry so that we have a consistent approach for both the default registry as well as custom registries. FWIW I wonder why there was the need to standardize raw.githubusercontent.com URLs instead of allowing any valid git URL? Seems custom registries can only be hosted on GitHub?

I note the absence of release timestamps for bazel modules. Intentional, or you hope they will be reverse engineered from the commit they reference?

[alexeagle]

Sorry I've been on vacation- @fmeum can probably help to answer.

[fmeum]

Does bzlmod replace WORKSPACE, or is its purpose to move bazel "modules" out of WORKSPACE and e.g. references to docker images or maven packages will still remain in WORKSPACE files?

Not every Bazel repository will also be a Bazel module.

After a migration period where most users will use both WORKSPACE and MODULE.bazel, all Bazel-aware dependencies will be managed directly via a bazel_dep line in MODULE.bazel.

All non-Bazel-aware dependencies will continue loaded via the same http_archive/git_repository rules used today in WORKSPACE. However, since these calls can't be made directly move MODULE.bazel, but have to be moved into a so-called module extension in a separate .bzl file referenced from MODULE.bazel, these calls could be spread out. This would either require a user-configurable list of files to apply the existing update mechanism to or more sophisticated logic that can follow use_extension and load calls.

https://registry.bazel.build/ is a front for https://github.com/bazelbuild/bazel-central-registry. Does it have a documented API?

Yes, the docs are here.

On the other hand the syntax for a custom registry is this:

common --registry=https://raw.githubusercontent.com/aspect-build/bazel-central-registry/main/

This implies we might be best to use github to query the default registry so that we have a consistent approach for both the default registry as well as custom registries. FWIW I wonder why there was the need to standardize raw.githubusercontent.com URLs instead of allowing any valid git URL? Seems custom registries can only be hosted on GitHub?

No, as explained in the docs I linked above, a registry can be used under any base URL. GitHub isn't special-cased in any way, it's just that it happens to be a common source and users should know that they have to use https://raw.githubusercontent.com/... instead of https://github.com/..., as the latter doesn't just serve the raw static content.

I note the absence of release timestamps for bazel modules. Intentional, or you hope they will be reverse engineered from the commit they reference?

The registry doesn't contain such metadata explicitly, but it could be set to the author time of the commit that added it. What would such a timestamp be used for? It is not too late to influence the design of the registry if that would help tools such as Renovate interact with it better.

[fmeum]

In addition to the above, it should also be noted that bzlmod supports a relaxed version of semver with an arbitrary number of components (rather than just major.minor.patch). Version information is only used to compare versions, but not to determine whether two given versions of a module are compatible - that information is contained in a dedicated compatibility_level property on the module directive in MODULE.bazel.

[fmeum]

CC @meteorcloudy @Wyverald

[rarkins]

The registry doesn't contain such metadata explicitly, but it could be set to the author time of the commit that added it. What would such a timestamp be used for? It is not too late to influence the design of the registry if that would help tools such as Renovate interact with it better.

Users find it helpful to know how old a release which Renovate is proposing is. 1 hour? 1 day? 1 week? etc.

Additionally some users like to use this programmatically, via the Renovate setting stabilityDays, to suppress new releases for a configurable time period.

[rarkins]

I think the question still remains about whether Renovate should use the default registry API, or parse its GitHub content directly.

If Renovate needs to support alternative registries, and those won't have APIs, it makes sense to have a single, consistent way to parse all registries - i.e. the raw approach instead of API-based.

[fmeum]

I think the question still remains about whether Renovate should use the default registry API, or parse its GitHub content directly.

If Renovate needs to support alternative registries, and those won't have APIs, it makes sense to have a single, consistent way to parse all registries - i.e. the raw approach instead of API-based.

All registries, including the default one, have the same API that's fully satisfied by serving static files in the right places. It just so happens that https://raw.githubusercontent.com/... turns any suitably structured GitHub repo into an endpoint of that type. This is something that Bazel users using custom registries would be expected to know, but Renovate could just treat the values passed in via --registry flags as registry API endpoints.

[fmeum]

Users find it helpful to know how old a release which Renovate is proposing is. 1 hour? 1 day? 1 week? etc.

Additionally some users like to use this programmatically, via the Renovate setting stabilityDays, to suppress new releases for a configurable time period.

Agreed, that would be very useful to have as information. There is an open issue to include the source repo in the metadata for each Bazel module, so individual versions could potentially be linked to releases in those repos as well.

[alexeagle]

FYI now that Bazel 6.0.0 final has shipped, and this new package manager is no longer experimental, some users have started to adopt it.
The issue mentioned in the last message on this thread is now fixed: entries in the registry have pointers to the source repository.
What are next steps here? @rarkins are you hoping to find time to make the changes or expecting a PR from someone?
🙏🏻

[rarkins]

Nothing is currently planned. Personally I don't have time right now unfortunately. I suggest the best approach would be to call it a new package manager bzlmod instead of combining it with the existing manager bazel, which would mean the code can be cleaner and hopefully done faster.

[aaronmondal]

Regarding the versioning scheme, bzlmod doesn't actually require SemVer https://bazel.build/external/module#version_format. Versions like YYYYMMDD.n-bcr.x are also valid. In general the flexibility here is so large that it's probably not viable to depend on any kind of set-in-stone versioning schemes.

AFAIK lockfiles are planned as well bazelbuild/bazel#14554. Since a proposal hasn't been draftet yet I'd guess is that those will be part of bzlmod with bazel 7, 8 or 9.

[meteorcloudy]

The Bzlmod lockfile proposal is here: https://docs.google.com/document/d/1HPeH_L-lRK54g8A27gv0q7cbk18nwJ-jOq_14XEiZdc/edit#heading=h.5mcn15i0e1ch

[cgrindel]

FYI. I am starting work on implementing bzlmod support.

@rarkins Is it possible to assign the issue to me?

[renovate-release]

🎉 This issue has been resolved in version 35.123.0 🎉

The release is available on:

• GitHub release
• 35.123.0

Your semantic-release bot 📦🚀