Unexpected interaction between toolchain resolution and bzlmod #20354

lberki · 2023-11-29T09:47:07Z

Description of the bug:

When bzlmod is enabled (as it is the case in Bazel 7.0.0), toolchain registered by default can unexpectedly shadow toolchains declared in the WORKSPACE file. This makes the oucome of toolchain resolution dependent on whether bzlmod is enabled, even if the MODULE.bazel file of the main repository is empty (as it is by default)

The root cause seems to be different from #17289 : there, AFAIU that one is caused by a subtle interaction between label mapping and toolchain resolution, here it's much more straightforward.

Some debugging indicates that what happens is that RegisteredToolchainsFunction creates the list of registered toolchains by concatenating two lists: those from bzlmod, then those from the WORKSPACE file. rules_python is by default and immutably (except if bzlmod is disabled) a bzlmod module and it registers @bazel_tools//tools/python:_autodetecting_py_runtime_pair as a Python toolchain. Since that toolchain doesn't have any exec or target constraints, it seems to win toolchain selection by default, regardless of what other toolchains come after.

Therefore, no matter what is written in the WORKSPACE file, one gets the autodetecting Python toolchain.

I'm not sure if this is WAI; it could be construed as such, but it sure is surprising and a hindrance to migration to bzlmod.

Which category does this issue belong to?

No response

What's the simplest, easiest way to reproduce this bug? Please provide a minimal example if possible.

cat > WORKSPACE <<'EOF'
register_toolchains("//tc:py_toolchain")
EOF

cat > BUILD <<'EOF'
py_binary(
    name = "py",
    srcs = ["py.py"],
)
EOF

cat > py.py <<'EOF'
print("Hello, Python!")
EOF

mkdir tc

cat > tc/BUILD <<'EOF'
load("@bazel_tools//tools/python:toolchain.bzl", "py_runtime_pair")

py_runtime(
    name = "python2",
    files = ["ok.py2"],
    interpreter = "ok.py2",
    python_version = "PY2",
)

py_runtime(
    name = "python3",
    files = ["ok.py3"],
    interpreter = "ok.py3",
    python_version = "PY3",
)

py_runtime_pair(
    name = "pair",
    py2_runtime = ":python2",
    py3_runtime = ":python3",
)

toolchain(
    name = "py_toolchain",
    toolchain = ":pair",
    toolchain_type = "@bazel_tools//tools/python:toolchain_type",
)
EOF

bazel run --enable_bzlmod :py  # BUG: Prints "Hello, Python!", ignoring the toolchain
bazel run --noenable_bzlmod :py  # WORKS: Fails to find the dummy interpreter defined above

Which operating system are you running Bazel on?

Linux

What is the output of `bazel info release`?

development version

If `bazel info release` returns `development version` or `(@non-git)`, tell us how you built Bazel.

From sources

What's the output of `git remote get-url origin; git rev-parse master; git rev-parse HEAD` ?

4a29f0851d1cde0240793cdc7a2e2cab926d31b7

Is this a regression? If yes, please try to identify the Bazel commit where the bug was introduced.

Pretty obviously when --enable_bzlmod was flipped to true.

Have you found anything relevant by searching the web?

There are a number of pertinent bugs:

--enable_bzlmod breaks toolchain resolution for toolchains registered in WORKSPACE #17289 (but that has a different root cause, if I understand correctly)
Using bzlmod (at all) breaks Python interpreter version specifications rules_python#1313 (seems to be very similar, but I haven't seen evidence that the bzlmod folks looked at this and has less information as to why this happens)

Any other information, logs, or outputs that you want to share?

No response

The text was updated successfully, but these errors were encountered:

meisterT · 2023-11-29T10:03:49Z

cc @meteorcloudy

fmeum · 2023-11-29T10:23:26Z

One possible solution I see is to change the order of registration to the following in order of decreasing precedence:

Toolchains registered in the root module's MODULE.bazel file
Toolchains registered in the WORKSPACE file
Toolchains registered in dependencies' MODULE.bazel files in BFS order

Independent of that particular issue, the auto-detecting Python toolchain should have constraints to match on the auto-detected host platform, otherwise it probably breaks remote execution of Python tools.

meteorcloudy · 2023-11-29T10:38:08Z

RegisteredToolchainsFunction creates the list of registered toolchains by concatenating two lists: those from bzlmod, then those from the WORKSPACE file.

Yes, this is WAI. But we probably should document the priorities of toolchain registration more clearly.

Toolchains registered in the root module's MODULE.bazel file
Toolchains registered in the WORKSPACE file
Toolchains registered in dependencies' MODULE.bazel files in BFS order

This does look like a better order since it gives more control to users. But still could be surprising and should be documented.

Independent of that particular issue, the auto-detecting Python toolchain should have constraints to match on the auto-detected host platform, otherwise it probably breaks remote execution of Python tools.

/cc @rickeylev

lberki · 2023-11-29T10:49:07Z

Yep, I find the order proposed by @fmeum more intuitive than what's at HEAD.

fmeum · 2023-11-29T10:58:24Z

There is a potential problem with this new order though: We can't distinguish top-level toolchain registrations from those coming from WORKSPACE macros or even the suffix. This could result in unexpected results of a different kind.

katre · 2023-11-29T12:41:12Z

Ideally toolchains registered in the workplace suffix would have the lowest priority (that's what effectively happened before bzlmod was a thing).

We've discussed (and I can't currently find the issue) adding explicit priority levels to register_toolchains: that could be used with some defaults to set the order of toolchains as described, but I'd want to take some time and flesh out the problem.

Fixing the python toolchains should be very easy, if @rickeylev wants help I can provide pointers.

What's the priority on fixing bzlmod/workspace toolchain ordering?

lberki · 2023-11-29T12:50:08Z

If you told me the behavior at HEAD is by design, I'd grudgingly accept that and ask how we could help people like the Kleaf folks with their migration. They do have a point that this (and other behavior similarly surprising, if it exists) doesn't make migration to bzlmod easy. I have a number of ideas which could make this more intuitive, but given that I you and @Wyverald follow toolchains and bzlmod much closer than I do, I'd rather not voice them before being asked.

Practically speaking, I don't think there is a rush, at least from this project, given that they also need bzlmod vendoring to be able to turn bzlmod on. But let's not fall into the trap where both issues get deferred because they expect the other one to progress slowly :)

with the host platform. This should prevent them from being used inappropriately in remote execution. Part of changes for bazelbuild#20354.

katre · 2023-11-29T13:19:39Z

Went ahead and wrote up #20356 for the python part: @lberki, while tests are running, can you see if this addresses the original issue?

lberki · 2023-11-29T13:26:20Z

It doesn't fix the repro in the description of this bug, so I guess it does not?

katre · 2023-11-29T13:29:28Z

That's unfortunate. Can you share the repro with me?

lberki · 2023-11-29T13:34:35Z

It's in the description of #20354 , complete with a shell snippet to produce a repository from scratch and example Bazel invocations.

katre · 2023-11-29T13:37:27Z

Whoops, sorry

Wyverald · 2023-11-29T17:03:29Z

Is it feasible for Kleaf to move/copy their toolchain registrations from WORKSPACE to MODULE.bazel? That would be the immediate fix.

katre · 2023-11-29T17:05:43Z

Using --extra_toolchains in a bazelrc would also be a workaround.

rickeylev · 2023-11-29T17:36:43Z

rules_python is by default and immutably (except if bzlmod is disabled) a bzlmod module and it registers @bazel_tools//tools/python:_autodetecting_py_runtime_pair as a Python toolchain.

Well, yes and no. There's three different mechanisms that can cause it to get registered when you don't expect it:

As part of Bazel itself. I don't know where this puts it in the toolchain priorities, but it happens very early, when Bazel is initializing the rule types it knows about. See

bazel/src/main/java/com/google/devtools/build/lib/bazel/rules/BazelRuleClassProvider.java

Line 479 in 38e8da1

ResourceFileLoader.loadResource(BazelPyBinaryRule.class, "python.WORKSPACE"));

and

bazel/src/main/java/com/google/devtools/build/lib/bazel/rules/python/python.WORKSPACE

Line 4 in 38e8da1

# line or WORKSPACE file. (And even if we did register it here, we still
As part of older rules_python "releases" (0.4 to 0.12). I put releases in quotes because, from what I can tell, those releases are of rules_python versions that didn't actually have bzlmod support. I'm guessing they were manually created to help bootstrap things. After that, the autodetecting toolchain is no longer registered. Starting in 0.23.0, a hermetic toolchain is automatically registered. Also note that Bazel itself depends on rules_python 0.4.0 right now (I have a pending change to upgrade that to 0.22.0, which doesn't register the autodetecting toolchain). See https://github.com/search?q=repo%3Abazelbuild%2Fbazel-central-registry+path%3Amodules%2Frules_python+autodetecting_toolchain&type=code
The rules_bazel_integration_test module registers the autodetecting toolchain. So if that's in the module graph, that'll happen. See https://github.com/search?q=repo%3Abazelbuild%2Fbazel-central-registry+-path%3Amodules%2Frules_python+autodetecting_toolchain&type=code

bazelbuild/rules_python#1313

That actually sounds exactly like this issue: the user has an empty module, a workspace registering a toolchain, but then when they enable bzlmod, their toolchain is ignored.

As an aside, don't use the stuff in @bazel_tools//tools/python. Use @rules_python instead.

Wyverald · 2023-11-29T21:56:21Z

There is a potential problem with this new order though: We can't distinguish top-level toolchain registrations from those coming from WORKSPACE macros or even the suffix. This could result in unexpected results of a different kind.

This is a fixable problem, actually. We could annotate toolchains registered in WORKSPACE suffixes with a "low priority" bit (messy, but technically possible), then order the toolchains per 1) root MODULE.bazel 2) WORKSPACE sans suffix 3) non-root Bzlmod stuff 4) WORKSPACE suffix. This would address the Kleaf use case.

fmeum · 2023-12-01T07:26:52Z

I believe that this is catching more users by surprise: https://bazelbuild.slack.com/archives/CA306CEV6/p1701394561232659

It's pretty hard to come up with why this is failing if users are not aware of MODULE.bazel at all, as in this case.

meteorcloudy · 2023-12-01T11:43:32Z

@bazel-io fork 7.0.0

meteorcloudy · 2023-12-01T11:45:05Z

Since Bzlmod is enabled by default in Bazel 7 and that could trigger this issue for many users, we should probably address this in 7.0.0.

…dules RELNOTES[INC]: Toolchains and execution platforms are now registered in the following order with `--enable_bzlmod`: 1. root module's module file 2. `WORKSPACE` or `WORKSPACE.bzlmod` 3. non-root modules' module files 4. default toolchains registered by Bazel (does not apply with `WORKSPACE.bzlmod` or execution platforms) Fixes bazelbuild#20354 Closes bazelbuild#20407. Co-authored-by: Xudong Yang <wyv@google.com> PiperOrigin-RevId: 587826082 Change-Id: Ia98da6ef07b2fbf589ef369d986af2323af6f72a

…-root mo… (#20430) …dules RELNOTES[INC]: Toolchains and execution platforms are now registered in the following order with `--enable_bzlmod`: 1. root module's module file 2. `WORKSPACE` or `WORKSPACE.bzlmod` 3. non-root modules' module files 4. default toolchains registered by Bazel (does not apply with `WORKSPACE.bzlmod` or execution platforms) Fixes #20354 Closes #20407. Commit 96b3612#diff-a8d3aed419e661d4dbecb2dc6668444212d7b1707ff61330b7d8aae61e75d4df PiperOrigin-RevId: 587826082 Change-Id: Ia98da6ef07b2fbf589ef369d986af2323af6f72a Co-authored-by: Fabian Meumertzheim <fabian@meumertzhe.im> Co-authored-by: Xudong Yang <wyv@google.com>

iancha1992 · 2023-12-05T01:29:23Z

A fix for this issue has been included in Bazel 7.0.0 RC6. Please test out the release candidate and report any issues as soon as possible. Thanks!

Baseline: d60ce2c Release Notes: + **[Incompatible]** This has the side effect of changing the message on unsuccessful builds from ``` FAILED: Build did NOT complete successfully (0 packages loaded) ``` to ``` ERROR: Build did NOT complete successfully ``` + The new path variable `$(rlocationpath ...)` and its plural form `$(rlocationpaths ...)` can be used to expand labels to the paths accepted by the `Rlocation` function of runfiles libraries. This is the preferred way to access data dependencies at runtime and works on all platforms, even when runfiles are not enabled (e.g., on Windows by default). + The `aquery` and `cquery` commands now respect the `--query_file` flag just like the `query` command. + **[Incompatible]** the --experimental_async_execution flag is now a no-op. + **[Incompatible]** --experimental_replay_action_out_err is not a no-op. + **[Incompatible]** `cquery --output=files` also outputs source files. + removed outdated ctx.host_fragments + **[Incompatible]** When multiple --deleted_packages options are passed on the command line, they will be concatenated instead of the latest one taking effect. + removed outdated ctx.host_configuration + **[Incompatible]** JSON profile: Use doubles instead of strings for counter series. + Now that the host configuration is finished, `genrule` should prefer the use of `tools` and stop using `exec_tools`. + **[Incompatible]** --experimental_build_transitive_python_runfiles is flipped to false. See #16303 for details + **[Incompatible]** --incompatible_python_disable_py2 is flipped to true. See #17293 for details. + Update Android manifest merger to v30.1.3, and also drop support for legacy (pre-D8) desugaring. + Correctly encode double value positive infinity as "inf" instead of "+inf" for textprotos. + **[Incompatible]** When remote cache evicts blobs, Bazel will exit with code 39. + Add --use_target_platform_for_tests which uses the target platform for executing tests instead of the execution platform. + **[Incompatible]** `--features` only applies to targets built in the target configuration, and `--host_features` is used for the host / exec configuration (gated behind `--incompatible_use_host_features`) + Custom C++ rules on Windows calling cc_common.create_linking_context_from_compilation_outputs should review whether each target of the rule type should produce a dynamic library since a condition which blocked their creation has been moved to the rules from behind the API. + Add flag `--experimental_remote_cache_ttl` and set the default value to 3 hours. + making --incompatible_use_platforms_repo_for_constraints do nothing. Using constraints from @bazel_tools//platforms with or without the flag will throw error with message "Constraints from @bazel_tools//platforms have been removed. Please use constraints from @platforms repository embedded in Bazel, or preferably declare dependency on https://github.com/bazelbuild/platforms" + **[Incompatible]** Bazel's local CPU resource on Linux is now container aware. Use `--local_cpu_resources`, `--loading_phase_threads` or `--jobs` to override. + Subsequent settings of --extra_execution_platforms now override previous settings, instead of adding them to a list. If you currently set --extra_execution_platforms more than once, please migrate by passing a list of values to --extra_execution_platforms instead so that earlier values aren't overwritten. + @bazel_tools//config:common_settings.bzl has been removed. Use @bazel_skylib//rules:common_settings.bzl instead. + **[Incompatible]** `copy_from_rule` is exec_groups is deprecated (#17668). + cc_shared_library is no longer experimental, see #16709 for details + **[Incompatible]** --legacy_bazel_java_test is now a no-op + The flag `--distinct_host_configuration` is removed. It has been a no-op since Bazel 6.0.0. + Add `--skip_incompatible_explicit_targets` option + **[Incompatible]** `--experimental_execution_graph_log` no longer exists. Current users that want local logs need to pass `--experimental_enable_execution_graph_log --experimental_execution_graph_log_path=/some/local/path`. Current users that want logs uploaded to BEP need to pass `--experimental_enable_execution_graph_log --experimental_stream_log_file_uploads`. + Remove 'darwin' as a CPU value, use 'darwin_x86_64' instead + **[Incompatible]** Remove 'darwin' as a CPU value, use 'darwin_x86_64' instead + cc_test can now be configured by using a native.toolchain(). + The location of rules that explicitly specify `generator_name` and/or `generator_function` attributes (typically because they are incidentally copied from `native.existing_rule()`) is now the top-level call in the `BUILD` file, which is consistent with rules that do not explicitly specify these attributes. + Warnings (most notably those associated with the `deprecation` rule attribute) are no longer replayed on subsequent invocations unless the target in question is re-analyzed. Warnings are purely informational, so this change has no bearing on the correctness of the build. Downstream tests that break due to this change should update their expectations. + `--experimental_remote_build_event_upload` has been renamed to `--remote_build_event_upload` + **[Incompatible]** Remove high priority workers functionality from blaze. + [Breaking change] platform, constraint_setting, and constraint_value can no longer take an applicable_licenses value. Remediation is to remove the attribute and rely on the package level default. + `--experimental_action_cache_store_output_metadata` has been renamed to `--action_cache_store_output_metadata` + Changed the default value for `--remote_build_event_upload` to `minimal`. + `--experimental_remote_cache_compression` has been renamed to `--remote_cache_compression` + Added a new `max_compatibility_level` attribute to the `bazel_dep` directive, which allows version selection to upgrade a dependency up to the specified compatibility level. + `--experimental_remote_grpc_log` has been renamed to `--remote_grpc_log` + `--incompatible_remote_build_event_upload_respect_no_cache` is now a no-op. + json.decode now takes an optional `default` positional parameter; if this parameter is specified and decoding fails, json.decode will return the `default` value instead of failing Starlark evaluation. + --experimental_repository_disable_download is now promoted to stable and is also available under the name --repository_disable_download . + `applicable_licenses` is no longer allowed on the `platform` rule. Additionally, `default_package_metadata` from any `package` rule will not be applied. + **[Incompatible]** Duplicates in <rule-inputs> when using `query --output=xml --aspect_deps=precise` are removed. Added flag `--proto:include_attribute_source_aspects` that will include a new field `source_aspect_name` to the `Attribute` message that indicates the source aspect that the attribute comes from. + Options specified on the pseudo-command `common` in `.rc` files are now ignored by commands that do not support them as long as they are valid options for *any* Bazel command. Previously, commands that did not support all options given for `common` would fail to run. These previous semantics of `common` are now available via the new `always` pseudo-command. + **[Incompatible]** `--incompatible_check_sharding_support` is enabled by default. Sharded tests with test runners that do not properly advertise support for test sharding will fail. Refer to #18339 for migration advice. + the 'default' param of json.decode can now be used as a keyword parameter. + As a transitional step in a larger refactoring, rule transitions are applied twice. Once during dependency resolution and once right before analysis of those rules. After the refactoring is complete, rule transitions will be applied only once. + Add aquery --output=streamed_proto which writes a stream of length delimited ActionGraphContainer containing a single Artifact, Action, Target, DepSetOfFiles, Configuration, AspectDescriptor, RuleClass, PathFragment proto. This breaks up the ActionGraphContainer into multiple which will prevent large protos from crashing blaze. + (BEP) TargetConfigured events will be marked aborted instead of published when there is an analysis error. This is motivated by a low level Blaze change aimed at improving scalability. + **[Incompatible]** cc_binary targets with dynamic_deps attributes no longer link indirect dynamic_deps on Unix. This might be an incompatible change if you are using RUNPATHs (instead of RPATHs) in your cc_shared_libraries. Enable the feature "exclude_bazel_rpaths_in_transitive_libs" or "use_rpath_instead_of_runpath" for those cc_shared_libraries. + Enable starlark_doc_extract - a native rule for Starlark documentation extraction. This rule is intended mainly for internal use by Stardoc. + **[Incompatible]** Remove deprecated and unused `--experimental_show_artifacts` flag. + Added the target path of an UnresolvedSymlink action to the aquery results + Additional source inputs can now be specified for compilation in cc_library targets using the additional_compiler_inputs attribute, and these inputs can be used in the $(location) function. + More descriptive error messages for unexpected cc absolute includes; unexpected absolute includes for cc compiles shouldn't be reported as "undeclared inclusion(s)" + Enable C++ branch coverage if gcov version is 8 or newer. + Deduplicates the result of split transitions uniformly, regardless of whether they are Starlark or native transitions. This used to apply only if either the attribute or rule transition contained a Starlark transition. + Support for optimizers rewriting baseline profiles. + This is a breaking change. `cc_host_toolchain_alias` rule is obsolete and can be interchangeably used with `cc_toolchain_alias` rule. + Fixes non-ascii arguments written to parameter files as mojibake. + NA + The `genrule` attribute `exec_tools` will be removed in a future Bazel release. Please follow directions at #19132 to migrate away from it. + **[Incompatible]** Removed multi_arch_split, use transition_support.apple_platform_split_transition from rules_apple instead. + Supports for android_binary.startup_profiles attribute. + **[Incompatible]** Removed apple_crosstool_transition, use transition_support.apple_rule_transition from rules_apple instead. + **[Incompatible]** Drop deprecated 'transitive_deps', 'transitive_runtime_deps' from JavaInfo + **[Incompatible]** The --android_include_proguard_location_references flag is not supported anymore. + **[Incompatible]** The --apple_compiler command line option is not available anymore. + `rule()` and `attr.*` can no longer be (pointlessly) called during WORKSPACE evaluation and repository rule evaluation. + Expands baseline profile wildcards before optimizer tools see them. + **[Incompatible]** --no_proguard_location_reference is now added unconditionally to the command line of aapt2. + **[Incompatible]** The command line flag --apple_enable_auto_dsym_dbg is not supported anymore. + **[Incompatible]** The --apple_compiler command line option is not available anymore. + **[Incompatible]** py_transitions top-level was removed. + Android resources will no longer propagate through neverlinked libraries by default. + Set Android Databinding to v2 and Databinging AndroidX to true and remove support for Databinding V1. + Added whether or not a FileWrite action's output is executable to the aquery results + --use_single_jar_apk_builder is removed. It's been a no-op for years. + **[Incompatible]** --incompatible_merge_fixed_and_default_shell_env is flipped to true. See #19317 for details. + JVM options in environment variables JAVA_TOOL_OPTIONS and JDK_JAVA_OPTIONS now do not get to the server; use --host_jvm_args instead. + --remote_download_minimal no longer implies --nobuild_runfile_links. + attr objects in Starlark now use value equality rather than reference equality. + **[Incompatible]** `cmd_helper` module was removed + **[Incompatible]** `ctx.new_file` was removed, `ctx.actions.declare_file` is to be used instead. + **[Incompatible]** Fails on unknown attributes (even when set to None). See #19403 + Change output paths to consistently start with [cpu]-[compilation_mode] along with other cleanups to output path generation logic. + Compilation actions using the auto-configured MSVC toolchain are forced to emit error messages in English if the English language pack for Visual Studio is installed. + **[Incompatible]** Flip incompatible_enable_cc_toolchain_resolution (#7260) + The new `--consistent_labels` option on `query`, `cquery`, and `aquery` can be used to force consistent label formatting across all output modes that is also compatible with `str(Label(...))` in Starlark. + **[Incompatible]** proto_compiler attribute removed from proto_lang_toolchain (it was recently introduced, and there is no evidence of use) + **[Incompatible]** --experimental_genquery_use_graphless_query is made to be a NO-OP. + **[Incompatible]** The --call_count_output_path command line option is not supported anymore. + **[Incompatible]** The no-op --dynamic_worker_strategy command line option is no available anymore. + **[Incompatible]** the --experimental_allow_top_level_aspects_parameters, --experimental_async_execution, --experimental_availability_info_exempt, --experimental_dynamic_execution_cpu_limited, --experimental_dynamic_skip_first_build, --experimental_keep_config_nodes_on_analysis_discard, --experimental_local_memory_estimate, --experimental_multi_cpu, --experimental_multi_threaded_digest, --experimental_replay_action_out_err and --experimental_shortened_obj_file_path command line options are not available anymore. + **[Incompatible]** The --experimental_cc_shared_library_debug command line flag is not available anymore. + **[Incompatible]** testing.analysis_test is not experimental anymore + **[Incompatible]** The (no-op) --android_include_proguard_location_references flag is not available anymore. + **[Incompatible]** The --experimental_android_local_test_binary_resources command line option is not available anymore. + **[Incompatible]** The --experimental_includes_attribute_subpackage_traversal command line option is not available anymore. + **[Incompatible]** The --debug_print_action_contexts command line option is not available anymore. + **[Incompatible]** The --experimental_skyframe_include_scanning command line option is not available anymore. + **[Incompatible]** The --experimental_supports_info_crosstool_configuration command line option is not available anymore. + **[Incompatible]** THe --incompatible_avoid_conflict_dlls command line option is not available anymore. + **[Incompatible]** The --experimental_prioritize_local_actions command line option is not available anymore. + **[Incompatible]** The following command line options are not supported anymore: --incompatible_disable_cc_configuration_make_variables, --incompatible_disable_cc_toolchain_label_from_crosstool_proto, --incompatible_disable_crosstool_file, --incompatible_disable_depset_in_cc_user_flags, --incompatible_disable_legacy_cpp_toolchain_skylark_api, --incompatible_disable_legacy_cpp_toolchain_skylark_api, incompatible_disable_legacy_flags_cc_toolchain_api, --incompatible_disable_managed_directories, --incompatible_disable_runtimes_filegroups, --incompatible_disable_sysroot_from_configuration, --incompatible_disable_tools_defaults_package, --incompatible_disallow_legacy_javainfo, --incompatible_dont_emit_static_libgcc, --incompatible_enable_legacy_cpp_toolchain_skylark_api, --incompatible_linkopts_in_user_link_flags, --incompatible_provide_cc_toolchain_info_from_cc_toolchain_suite, --incompatible_require_feature_configuration_for_pic . + **[Incompatible]** The --experimental_execution_graph_log_cached and --experimental_execution_graph_log_missed command line options are not available anymore. + **[Incompatible]** The --experimental_use_sh_binary_stub_script command line option is not available anymore. + **[Incompatible]** The --incompatible_disable_expand_if_all_available_in_flag_set command line option is not available anymore. + default_applicable_licenses in package() now applies to the entire BUILD file, regardless of where in the BUILD that package() is called. + **[Incompatible]** The --incompatible_load_cc_rules_from_bzl command line option is not available anymore. + **[Incompatible]** The --incompatible_display_source_file_location command line option is not available anymore. + **[Incompatible]** The --experimental_repository_hash_file and --experimental_verify_repository_rules command line options are not available anymore. + **[Incompatible]** The following previously no-op command line options are not available anymore: --legacy_spawn_scheduler, --use_workers_with_dexbuilder, --print_workspace_in_output_paths_if_needed, --watchos_simulator_version, --watchos_simulator_device, --tvos_simulator_version, --tvos_simulator_device. + **[Incompatible]** The --show_warnings command line option of the command "canonicalize-flags" is not available anymore. + **[Incompatible]** The command line option --remove_all_convenience_symlinks is not available anymore. + **[Incompatible]** java_test doesn't produce deploy jars anymore + **[Incompatible]** The lazy template expansion is not experimental anymore (--experimental_lazy_template_expansion command line option is not available anymore). + **[Incompatible]** The --incompatible_disallow_legacy_javainfo command line option is not available anymore. + **[Incompatible]** The --incompatible_remove_cpu_and_compiler_attributes_from_cc_toolchain command line option is not available anymore. Using the "cpu" and "compiler" options of the cc_toolchain rule now results in a slightly different error that is signaled earlier. + **[Incompatible]** THe --incompatible_force_strict_header_check_from_starlark command line option is not available anymore. + **[Incompatible]** --incompatible_visibility_private_attributes_at_definition is flipped to true. See #19330 for details. + **[Incompatible]** The --experimental_java_proto_library_default_has_services command line option and the Starlark symbol java_common.experimental_java_proto_library_default_has_services function are not available anymore. + **[Incompatible]** The no-op --experimental_cpp_compile_argv_ignore_param_file command line option is not available anymore. + Make variable expansion of `objc_library` produces better error messages. + **[Incompatible]** rule function takes only implementation function positionally, other parameters have to be named + **[Incompatible]** The --incompatible_genquery_use_graphless_query command line options is not available anymore. + Added support for a new directive `use_repo_rule` in MODULE.bazel files, which allows you to conveniently declare repos that are only visible within your module. + Bazel now throws an error if the root module specifies overrides on nonexistent modules. + Fix includes attribute of `objc_library` when sibling repository layout is used. + Directories used by sandboxed non-worker execution may be reused to avoid unnecessary setup costs. Disable behavior with --noreuse_sandbox_directories. + **[Incompatible]** The sandboxfs sandboxing strategy is removed. It hadn't been maintained for a long time, it didn't work for most users and it was not consistently faster while being complex to set up. sandboxfs performance is heavily dependent on the specific setup (setup costs are lower, but you have to pay a penalty for the use of each input) and there are scenarios where it is faster and scenarios where it is slower. Overall it is not worth its weight. + Symlink trees created by sandboxing will be deleted asynchronously. Disable behavior with --experimental_sandbox_async_tree_delete_idle_threads=0. + **[Incompatible]** Bzlmod is enabled by default, please consider migrating your external dependencies from WORKSPACE to MODULE.bazel. Find more details at #18958 + Enable Platforms and Toolchains for Android. Android projects will need to stop passing the legacy flag `--fat_apk_cpu`, and instead use `--android_platforms` using platforms defined with the `@platforms//os:android` constraint. The https://github.com/bazelbuild/rules_android repository defines four standard Android platforms for projects that use those rules, `@rules_android//:armeabi-v7a`, `@rules_android//:arm64-v8a`, `@rules_android//:x86`, `@rules_android//:x86_64`. + **[Incompatible]** Java runtime toolchains created via `local_java_repository` from `@bazel_tools//tools/jdk:local_java_repository.bzl`, which includes `local_jdk`, now have `target_compatible_with` set to the auto-detected host constraints. This can result in errors about toolchain resolution failures for `@bazel_tools//tools/jdk:runtime_toolchain_type`, especially when cross-compiling. These failures can be fixed in the following ways (listed in decreasing order of preference): * Replace `java_binary` targets that aren't meant to be run with `bazel run` or as tools during the build with `java_single_jar` (available in `@rules_java//java:java_single_jar.bzl`). Such targets do not require a Java runtime for the target configuration. * Set `--java_runtime_version=remotejdk_N` for some Java version `N` to let Bazel choose and download an appropriate remote JDK for the current target platform. This setting defaults to `local_jdk`, which means that Bazel can only use the local JDK, which isn't compatible with any other platform. * Manually define and register a `local_java_runtime` with no value set for `exec_compatible_with` (defaults to `[]`) and select it by setting `--java_runtime_version` to its `name`. This fully restores the previous behavior, but can result in incorrect results when cross-compiling (see #18265). Closes #18262. Commit f79ca02 + none Commit 7d87996 + None Commit 0a1dce2 + **[Incompatible]** The attribute `new_local_repository.build_file` no longer accepts a path; a label must be passed instead. Closes #19992. Commit 76d71d9 + **[Incompatible]** Delete the --experimental_execution_log_file flag. Use --execution_log_binary_file in conjunction with --noexecution_log_sort instead. + **[Incompatible]** transition is removed from objc_library (#19688) (cherry picked from commit b4292d2) Co-authored-by: Googler <ilist@google.com> + **[Incompatible]** All labels in Bazel error messages, log output, Build Event Protocol, etc. are now prefixed with double-at (`@@`) instead of single-at (`@`) where applicable, to properly denote that they contain canonical repo names. + None. Commit 491284b + **[Incompatible]** `--incompatible_sandbox_hermetic_tmp` is enabled by default. See #19915 for migration advice. Closes #19943. Commit e2c0276 + **[Incompatible]** The `--experimental_repository_cache_urls_as_default_canonical_id` flag is no longer available. Instead, the `http_archive`, `http_file`, `http_jar`, `jvm_maven_import_external`, and `jvm_import_external` repository rules now use the URLs as the canonical ID if none is provided explicitly. If this behavior is not desired, it can be disabled via `--repo_env=BAZEL_HTTP_RULES_URLS_AS_DEFAULT_CANONICAL_ID=0`. Fixes #19749 Closes #20047. + None Commit 65f847a + None Commit 9bbc2cb + none Fixes #19920 Commit 886b13f + **[Incompatible]** The `BAZEL_CURRENT_REPOSITORY` preprocessor variable, which holds the canonical name of the Bazel repository containing a `cc_*` target, is now only set during compilation if the target depends on the C/C++ runfiles library `@bazel_tools//tools/cpp/runfiles` via `deps` or `implementation_deps`. Fixes #20371 Closes #20388. Commit 454b550 + `--zip_undeclared_test_outputs` now preserves symlinks when zipping `$TEST_UNDECLARED_OUTPUTS_DIR`. Closes #19948. Change-Id: Ia4a8a9699e4e2f40498342af55babc5554a9ac93 Commit 8e639df + **[Incompatible]** Toolchains and execution platforms are now registered in the following order with `--enable_bzlmod`: 1. root module's module file 2. `WORKSPACE` or `WORKSPACE.bzlmod` 3. non-root modules' module files 4. default toolchains registered by Bazel (does not apply with `WORKSPACE.bzlmod` or execution platforms) Fixes #20354 Closes #20407. Commit 96b3612#diff-a8d3aed419e661d4dbecb2dc6668444212d7b1707ff61330b7d8aae61e75d4df Acknowledgements: This release contains contributions from many people at Google, as well as Adam Lavin, Adrian Imboden, Alan Falloon, Alessandro Patti, Alex Eagle, Alexander Grund, AlexTereshenkov, Amanda L Martin, Amet Umerov, Andreas Herrmann, Andy Hamon, andyrinne12, Ankit Agarwal, Ankush Goyal, Anshuman Mishra, Anthony Ter-Saakov, Antoine Musso, Artem Zinnatullin, arun.sampathkumar, aryeh, Austin Schuh, bazel.build machine account, Ben Lee, Benjamin Lee, Benjamin Peterson, Benjamin Sigonneau, Boleyn Su, Brentley Jones, Cameron Martin, Chirag Ramani, Chris Sauer, Christopher Peterson Sauer, Christopher Rydell, Christopher Sauer, Conall O'Brien, Daniel Grunwald, Daniel KT, Daniel Wagner-Hall, David Ostrovsky, dhmemi, Dimi Shahbaz, eareilg, Ed Schouten, Emil Kattainen, Ezekiel Warren, Fabian Meumertzheim, Fahrzin Hemmati, Gaspare Vitta, George Gensure, Greg, Greg Magolan, Grzegorz Lukasik, Guillaume Maudoux, Gunnar Wagenknecht, Halil Sener, Hao Yuan, hvadehra, hvd, Ilan Keshet, Ivan Golub, Jack Dai, James Ma, Jason Mobarak, Jasper, Jay Conrod, Jeff Hodges, Jeremy Volkman, Jimm chja20, Joel Jeske, John Hinnegan, Jon Landis, Jon Parise, jonrose-dev, Joseph Lisee, Joseph Tilahun, Julio Merino, Kai Zhang, keertk, Keith Smiley, Konstantin Erman, Krzysztof Naglik, kshyanashree, Kun-Lu, Laurenz Altenmller, Lee Mracek, Letu Ren, lripoche, Maksim, Malte Poll, Marc Redemske, Marc Zych, Matt Mackay, Matt Vollmer, Mauricio G, Mauricio Galindo, Maxim Matyunin, Maxwell Elliott, mohamadk, Nathan (Blaise) Bruer, nathyong, NelsonLi0701, nglevin, Nicholas Junge, Nick Biryulin, Nick Korostelev, Oliver Lee, Orion Hodson, Oscar Garzon, Patrice Duroux, Patrick Balestra, Peter Lobsinger, Philipp Schrader, Rasrack, Red Daly, redwrasse, Robin Tweedie, robincaloudis, Roger Hu, Romain Chossart, Roman Salvador, Sagar Pathare, Saleem Abdulrasool, Sam Shadwell, Sara Adams, Sascha Moecker, Scott Minor, Severin Strobl, Shaygan Hooshyari, Siddhesh Bhupendra Kuakde, Simon Mavi Stewart, something_vague, Son Luong Ngoc, Steve Barrau, Stiopa Koltsov, stoozy, Sushain Cherivirala, Takeo Sawada, Taylor Barrella, tbaing, Ted Kaplan, Ted Logan, Thi Don, Thulio Ferraz Assis, Timothy Gu, Tom Cnops, Tyler Williams, Ulf Adams, Ulrik Falklof, Uri Baghin, Vertexwahn, Xavier Bonaventura, Xdng Yng, Xiangquan Xiao, Xùdōng Yáng, Yannic, Yannic Bonenberger, Yi Cheng, Yuval Kaplan, yuzhy8701, Zheng Wei Tan.

Baseline: d60ce2c Release Notes: + **[Incompatible]** This has the side effect of changing the message on unsuccessful builds from ``` FAILED: Build did NOT complete successfully (0 packages loaded) ``` to ``` ERROR: Build did NOT complete successfully ``` + The new path variable `$(rlocationpath ...)` and its plural form `$(rlocationpaths ...)` can be used to expand labels to the paths accepted by the `Rlocation` function of runfiles libraries. This is the preferred way to access data dependencies at runtime and works on all platforms, even when runfiles are not enabled (e.g., on Windows by default). + The `aquery` and `cquery` commands now respect the `--query_file` flag just like the `query` command. + **[Incompatible]** the --experimental_async_execution flag is now a no-op. + **[Incompatible]** --experimental_replay_action_out_err is not a no-op. + **[Incompatible]** `cquery --output=files` also outputs source files. + removed outdated ctx.host_fragments + **[Incompatible]** When multiple --deleted_packages options are passed on the command line, they will be concatenated instead of the latest one taking effect. + removed outdated ctx.host_configuration + **[Incompatible]** JSON profile: Use doubles instead of strings for counter series. + Now that the host configuration is finished, `genrule` should prefer the use of `tools` and stop using `exec_tools`. + **[Incompatible]** --experimental_build_transitive_python_runfiles is flipped to false. See bazelbuild#16303 for details + **[Incompatible]** --incompatible_python_disable_py2 is flipped to true. See bazelbuild#17293 for details. + Update Android manifest merger to v30.1.3, and also drop support for legacy (pre-D8) desugaring. + Correctly encode double value positive infinity as "inf" instead of "+inf" for textprotos. + **[Incompatible]** When remote cache evicts blobs, Bazel will exit with code 39. + Add --use_target_platform_for_tests which uses the target platform for executing tests instead of the execution platform. + **[Incompatible]** `--features` only applies to targets built in the target configuration, and `--host_features` is used for the host / exec configuration (gated behind `--incompatible_use_host_features`) + Custom C++ rules on Windows calling cc_common.create_linking_context_from_compilation_outputs should review whether each target of the rule type should produce a dynamic library since a condition which blocked their creation has been moved to the rules from behind the API. + Add flag `--experimental_remote_cache_ttl` and set the default value to 3 hours. + making --incompatible_use_platforms_repo_for_constraints do nothing. Using constraints from @bazel_tools//platforms with or without the flag will throw error with message "Constraints from @bazel_tools//platforms have been removed. Please use constraints from @platforms repository embedded in Bazel, or preferably declare dependency on https://github.com/bazelbuild/platforms" + **[Incompatible]** Bazel's local CPU resource on Linux is now container aware. Use `--local_cpu_resources`, `--loading_phase_threads` or `--jobs` to override. + Subsequent settings of --extra_execution_platforms now override previous settings, instead of adding them to a list. If you currently set --extra_execution_platforms more than once, please migrate by passing a list of values to --extra_execution_platforms instead so that earlier values aren't overwritten. + @bazel_tools//config:common_settings.bzl has been removed. Use @bazel_skylib//rules:common_settings.bzl instead. + **[Incompatible]** `copy_from_rule` is exec_groups is deprecated (bazelbuild#17668). + cc_shared_library is no longer experimental, see bazelbuild#16709 for details + **[Incompatible]** --legacy_bazel_java_test is now a no-op + The flag `--distinct_host_configuration` is removed. It has been a no-op since Bazel 6.0.0. + Add `--skip_incompatible_explicit_targets` option + **[Incompatible]** `--experimental_execution_graph_log` no longer exists. Current users that want local logs need to pass `--experimental_enable_execution_graph_log --experimental_execution_graph_log_path=/some/local/path`. Current users that want logs uploaded to BEP need to pass `--experimental_enable_execution_graph_log --experimental_stream_log_file_uploads`. + Remove 'darwin' as a CPU value, use 'darwin_x86_64' instead + **[Incompatible]** Remove 'darwin' as a CPU value, use 'darwin_x86_64' instead + cc_test can now be configured by using a native.toolchain(). + The location of rules that explicitly specify `generator_name` and/or `generator_function` attributes (typically because they are incidentally copied from `native.existing_rule()`) is now the top-level call in the `BUILD` file, which is consistent with rules that do not explicitly specify these attributes. + Warnings (most notably those associated with the `deprecation` rule attribute) are no longer replayed on subsequent invocations unless the target in question is re-analyzed. Warnings are purely informational, so this change has no bearing on the correctness of the build. Downstream tests that break due to this change should update their expectations. + `--experimental_remote_build_event_upload` has been renamed to `--remote_build_event_upload` + **[Incompatible]** Remove high priority workers functionality from blaze. + [Breaking change] platform, constraint_setting, and constraint_value can no longer take an applicable_licenses value. Remediation is to remove the attribute and rely on the package level default. + `--experimental_action_cache_store_output_metadata` has been renamed to `--action_cache_store_output_metadata` + Changed the default value for `--remote_build_event_upload` to `minimal`. + `--experimental_remote_cache_compression` has been renamed to `--remote_cache_compression` + Added a new `max_compatibility_level` attribute to the `bazel_dep` directive, which allows version selection to upgrade a dependency up to the specified compatibility level. + `--experimental_remote_grpc_log` has been renamed to `--remote_grpc_log` + `--incompatible_remote_build_event_upload_respect_no_cache` is now a no-op. + json.decode now takes an optional `default` positional parameter; if this parameter is specified and decoding fails, json.decode will return the `default` value instead of failing Starlark evaluation. + --experimental_repository_disable_download is now promoted to stable and is also available under the name --repository_disable_download . + `applicable_licenses` is no longer allowed on the `platform` rule. Additionally, `default_package_metadata` from any `package` rule will not be applied. + **[Incompatible]** Duplicates in <rule-inputs> when using `query --output=xml --aspect_deps=precise` are removed. Added flag `--proto:include_attribute_source_aspects` that will include a new field `source_aspect_name` to the `Attribute` message that indicates the source aspect that the attribute comes from. + Options specified on the pseudo-command `common` in `.rc` files are now ignored by commands that do not support them as long as they are valid options for *any* Bazel command. Previously, commands that did not support all options given for `common` would fail to run. These previous semantics of `common` are now available via the new `always` pseudo-command. + **[Incompatible]** `--incompatible_check_sharding_support` is enabled by default. Sharded tests with test runners that do not properly advertise support for test sharding will fail. Refer to bazelbuild#18339 for migration advice. + the 'default' param of json.decode can now be used as a keyword parameter. + As a transitional step in a larger refactoring, rule transitions are applied twice. Once during dependency resolution and once right before analysis of those rules. After the refactoring is complete, rule transitions will be applied only once. + Add aquery --output=streamed_proto which writes a stream of length delimited ActionGraphContainer containing a single Artifact, Action, Target, DepSetOfFiles, Configuration, AspectDescriptor, RuleClass, PathFragment proto. This breaks up the ActionGraphContainer into multiple which will prevent large protos from crashing blaze. + (BEP) TargetConfigured events will be marked aborted instead of published when there is an analysis error. This is motivated by a low level Blaze change aimed at improving scalability. + **[Incompatible]** cc_binary targets with dynamic_deps attributes no longer link indirect dynamic_deps on Unix. This might be an incompatible change if you are using RUNPATHs (instead of RPATHs) in your cc_shared_libraries. Enable the feature "exclude_bazel_rpaths_in_transitive_libs" or "use_rpath_instead_of_runpath" for those cc_shared_libraries. + Enable starlark_doc_extract - a native rule for Starlark documentation extraction. This rule is intended mainly for internal use by Stardoc. + **[Incompatible]** Remove deprecated and unused `--experimental_show_artifacts` flag. + Added the target path of an UnresolvedSymlink action to the aquery results + Additional source inputs can now be specified for compilation in cc_library targets using the additional_compiler_inputs attribute, and these inputs can be used in the $(location) function. + More descriptive error messages for unexpected cc absolute includes; unexpected absolute includes for cc compiles shouldn't be reported as "undeclared inclusion(s)" + Enable C++ branch coverage if gcov version is 8 or newer. + Deduplicates the result of split transitions uniformly, regardless of whether they are Starlark or native transitions. This used to apply only if either the attribute or rule transition contained a Starlark transition. + Support for optimizers rewriting baseline profiles. + This is a breaking change. `cc_host_toolchain_alias` rule is obsolete and can be interchangeably used with `cc_toolchain_alias` rule. + Fixes non-ascii arguments written to parameter files as mojibake. + NA + The `genrule` attribute `exec_tools` will be removed in a future Bazel release. Please follow directions at bazelbuild#19132 to migrate away from it. + **[Incompatible]** Removed multi_arch_split, use transition_support.apple_platform_split_transition from rules_apple instead. + Supports for android_binary.startup_profiles attribute. + **[Incompatible]** Removed apple_crosstool_transition, use transition_support.apple_rule_transition from rules_apple instead. + **[Incompatible]** Drop deprecated 'transitive_deps', 'transitive_runtime_deps' from JavaInfo + **[Incompatible]** The --android_include_proguard_location_references flag is not supported anymore. + **[Incompatible]** The --apple_compiler command line option is not available anymore. + `rule()` and `attr.*` can no longer be (pointlessly) called during WORKSPACE evaluation and repository rule evaluation. + Expands baseline profile wildcards before optimizer tools see them. + **[Incompatible]** --no_proguard_location_reference is now added unconditionally to the command line of aapt2. + **[Incompatible]** The command line flag --apple_enable_auto_dsym_dbg is not supported anymore. + **[Incompatible]** The --apple_compiler command line option is not available anymore. + **[Incompatible]** py_transitions top-level was removed. + Android resources will no longer propagate through neverlinked libraries by default. + Set Android Databinding to v2 and Databinging AndroidX to true and remove support for Databinding V1. + Added whether or not a FileWrite action's output is executable to the aquery results + --use_single_jar_apk_builder is removed. It's been a no-op for years. + **[Incompatible]** --incompatible_merge_fixed_and_default_shell_env is flipped to true. See bazelbuild#19317 for details. + JVM options in environment variables JAVA_TOOL_OPTIONS and JDK_JAVA_OPTIONS now do not get to the server; use --host_jvm_args instead. + --remote_download_minimal no longer implies --nobuild_runfile_links. + attr objects in Starlark now use value equality rather than reference equality. + **[Incompatible]** `cmd_helper` module was removed + **[Incompatible]** `ctx.new_file` was removed, `ctx.actions.declare_file` is to be used instead. + **[Incompatible]** Fails on unknown attributes (even when set to None). See bazelbuild#19403 + Change output paths to consistently start with [cpu]-[compilation_mode] along with other cleanups to output path generation logic. + Compilation actions using the auto-configured MSVC toolchain are forced to emit error messages in English if the English language pack for Visual Studio is installed. + **[Incompatible]** Flip incompatible_enable_cc_toolchain_resolution (bazelbuild#7260) + The new `--consistent_labels` option on `query`, `cquery`, and `aquery` can be used to force consistent label formatting across all output modes that is also compatible with `str(Label(...))` in Starlark. + **[Incompatible]** proto_compiler attribute removed from proto_lang_toolchain (it was recently introduced, and there is no evidence of use) + **[Incompatible]** --experimental_genquery_use_graphless_query is made to be a NO-OP. + **[Incompatible]** The --call_count_output_path command line option is not supported anymore. + **[Incompatible]** The no-op --dynamic_worker_strategy command line option is no available anymore. + **[Incompatible]** the --experimental_allow_top_level_aspects_parameters, --experimental_async_execution, --experimental_availability_info_exempt, --experimental_dynamic_execution_cpu_limited, --experimental_dynamic_skip_first_build, --experimental_keep_config_nodes_on_analysis_discard, --experimental_local_memory_estimate, --experimental_multi_cpu, --experimental_multi_threaded_digest, --experimental_replay_action_out_err and --experimental_shortened_obj_file_path command line options are not available anymore. + **[Incompatible]** The --experimental_cc_shared_library_debug command line flag is not available anymore. + **[Incompatible]** testing.analysis_test is not experimental anymore + **[Incompatible]** The (no-op) --android_include_proguard_location_references flag is not available anymore. + **[Incompatible]** The --experimental_android_local_test_binary_resources command line option is not available anymore. + **[Incompatible]** The --experimental_includes_attribute_subpackage_traversal command line option is not available anymore. + **[Incompatible]** The --debug_print_action_contexts command line option is not available anymore. + **[Incompatible]** The --experimental_skyframe_include_scanning command line option is not available anymore. + **[Incompatible]** The --experimental_supports_info_crosstool_configuration command line option is not available anymore. + **[Incompatible]** THe --incompatible_avoid_conflict_dlls command line option is not available anymore. + **[Incompatible]** The --experimental_prioritize_local_actions command line option is not available anymore. + **[Incompatible]** The following command line options are not supported anymore: --incompatible_disable_cc_configuration_make_variables, --incompatible_disable_cc_toolchain_label_from_crosstool_proto, --incompatible_disable_crosstool_file, --incompatible_disable_depset_in_cc_user_flags, --incompatible_disable_legacy_cpp_toolchain_skylark_api, --incompatible_disable_legacy_cpp_toolchain_skylark_api, incompatible_disable_legacy_flags_cc_toolchain_api, --incompatible_disable_managed_directories, --incompatible_disable_runtimes_filegroups, --incompatible_disable_sysroot_from_configuration, --incompatible_disable_tools_defaults_package, --incompatible_disallow_legacy_javainfo, --incompatible_dont_emit_static_libgcc, --incompatible_enable_legacy_cpp_toolchain_skylark_api, --incompatible_linkopts_in_user_link_flags, --incompatible_provide_cc_toolchain_info_from_cc_toolchain_suite, --incompatible_require_feature_configuration_for_pic . + **[Incompatible]** The --experimental_execution_graph_log_cached and --experimental_execution_graph_log_missed command line options are not available anymore. + **[Incompatible]** The --experimental_use_sh_binary_stub_script command line option is not available anymore. + **[Incompatible]** The --incompatible_disable_expand_if_all_available_in_flag_set command line option is not available anymore. + default_applicable_licenses in package() now applies to the entire BUILD file, regardless of where in the BUILD that package() is called. + **[Incompatible]** The --incompatible_load_cc_rules_from_bzl command line option is not available anymore. + **[Incompatible]** The --incompatible_display_source_file_location command line option is not available anymore. + **[Incompatible]** The --experimental_repository_hash_file and --experimental_verify_repository_rules command line options are not available anymore. + **[Incompatible]** The following previously no-op command line options are not available anymore: --legacy_spawn_scheduler, --use_workers_with_dexbuilder, --print_workspace_in_output_paths_if_needed, --watchos_simulator_version, --watchos_simulator_device, --tvos_simulator_version, --tvos_simulator_device. + **[Incompatible]** The --show_warnings command line option of the command "canonicalize-flags" is not available anymore. + **[Incompatible]** The command line option --remove_all_convenience_symlinks is not available anymore. + **[Incompatible]** java_test doesn't produce deploy jars anymore + **[Incompatible]** The lazy template expansion is not experimental anymore (--experimental_lazy_template_expansion command line option is not available anymore). + **[Incompatible]** The --incompatible_disallow_legacy_javainfo command line option is not available anymore. + **[Incompatible]** The --incompatible_remove_cpu_and_compiler_attributes_from_cc_toolchain command line option is not available anymore. Using the "cpu" and "compiler" options of the cc_toolchain rule now results in a slightly different error that is signaled earlier. + **[Incompatible]** THe --incompatible_force_strict_header_check_from_starlark command line option is not available anymore. + **[Incompatible]** --incompatible_visibility_private_attributes_at_definition is flipped to true. See bazelbuild#19330 for details. + **[Incompatible]** The --experimental_java_proto_library_default_has_services command line option and the Starlark symbol java_common.experimental_java_proto_library_default_has_services function are not available anymore. + **[Incompatible]** The no-op --experimental_cpp_compile_argv_ignore_param_file command line option is not available anymore. + Make variable expansion of `objc_library` produces better error messages. + **[Incompatible]** rule function takes only implementation function positionally, other parameters have to be named + **[Incompatible]** The --incompatible_genquery_use_graphless_query command line options is not available anymore. + Added support for a new directive `use_repo_rule` in MODULE.bazel files, which allows you to conveniently declare repos that are only visible within your module. + Bazel now throws an error if the root module specifies overrides on nonexistent modules. + Fix includes attribute of `objc_library` when sibling repository layout is used. + Directories used by sandboxed non-worker execution may be reused to avoid unnecessary setup costs. Disable behavior with --noreuse_sandbox_directories. + **[Incompatible]** The sandboxfs sandboxing strategy is removed. It hadn't been maintained for a long time, it didn't work for most users and it was not consistently faster while being complex to set up. sandboxfs performance is heavily dependent on the specific setup (setup costs are lower, but you have to pay a penalty for the use of each input) and there are scenarios where it is faster and scenarios where it is slower. Overall it is not worth its weight. + Symlink trees created by sandboxing will be deleted asynchronously. Disable behavior with --experimental_sandbox_async_tree_delete_idle_threads=0. + **[Incompatible]** Bzlmod is enabled by default, please consider migrating your external dependencies from WORKSPACE to MODULE.bazel. Find more details at bazelbuild#18958 + Enable Platforms and Toolchains for Android. Android projects will need to stop passing the legacy flag `--fat_apk_cpu`, and instead use `--android_platforms` using platforms defined with the `@platforms//os:android` constraint. The https://github.com/bazelbuild/rules_android repository defines four standard Android platforms for projects that use those rules, `@rules_android//:armeabi-v7a`, `@rules_android//:arm64-v8a`, `@rules_android//:x86`, `@rules_android//:x86_64`. + **[Incompatible]** Java runtime toolchains created via `local_java_repository` from `@bazel_tools//tools/jdk:local_java_repository.bzl`, which includes `local_jdk`, now have `target_compatible_with` set to the auto-detected host constraints. This can result in errors about toolchain resolution failures for `@bazel_tools//tools/jdk:runtime_toolchain_type`, especially when cross-compiling. These failures can be fixed in the following ways (listed in decreasing order of preference): * Replace `java_binary` targets that aren't meant to be run with `bazel run` or as tools during the build with `java_single_jar` (available in `@rules_java//java:java_single_jar.bzl`). Such targets do not require a Java runtime for the target configuration. * Set `--java_runtime_version=remotejdk_N` for some Java version `N` to let Bazel choose and download an appropriate remote JDK for the current target platform. This setting defaults to `local_jdk`, which means that Bazel can only use the local JDK, which isn't compatible with any other platform. * Manually define and register a `local_java_runtime` with no value set for `exec_compatible_with` (defaults to `[]`) and select it by setting `--java_runtime_version` to its `name`. This fully restores the previous behavior, but can result in incorrect results when cross-compiling (see bazelbuild#18265). Closes bazelbuild#18262. Commit bazelbuild@f79ca02 + none Commit bazelbuild@7d87996 + None Commit bazelbuild@0a1dce2 + **[Incompatible]** The attribute `new_local_repository.build_file` no longer accepts a path; a label must be passed instead. Closes bazelbuild#19992. Commit bazelbuild@76d71d9 + **[Incompatible]** Delete the --experimental_execution_log_file flag. Use --execution_log_binary_file in conjunction with --noexecution_log_sort instead. + **[Incompatible]** transition is removed from objc_library (bazelbuild#19688) (cherry picked from commit b4292d2) Co-authored-by: Googler <ilist@google.com> + **[Incompatible]** All labels in Bazel error messages, log output, Build Event Protocol, etc. are now prefixed with double-at (`@@`) instead of single-at (`@`) where applicable, to properly denote that they contain canonical repo names. + None. Commit bazelbuild@491284b + **[Incompatible]** `--incompatible_sandbox_hermetic_tmp` is enabled by default. See bazelbuild#19915 for migration advice. Closes bazelbuild#19943. Commit bazelbuild@e2c0276 + **[Incompatible]** The `--experimental_repository_cache_urls_as_default_canonical_id` flag is no longer available. Instead, the `http_archive`, `http_file`, `http_jar`, `jvm_maven_import_external`, and `jvm_import_external` repository rules now use the URLs as the canonical ID if none is provided explicitly. If this behavior is not desired, it can be disabled via `--repo_env=BAZEL_HTTP_RULES_URLS_AS_DEFAULT_CANONICAL_ID=0`. Fixes bazelbuild#19749 Closes bazelbuild#20047. + None Commit bazelbuild@65f847a + None Commit bazelbuild@9bbc2cb + none Fixes bazelbuild#19920 Commit bazelbuild@886b13f + **[Incompatible]** The `BAZEL_CURRENT_REPOSITORY` preprocessor variable, which holds the canonical name of the Bazel repository containing a `cc_*` target, is now only set during compilation if the target depends on the C/C++ runfiles library `@bazel_tools//tools/cpp/runfiles` via `deps` or `implementation_deps`. Fixes bazelbuild#20371 Closes bazelbuild#20388. Commit bazelbuild@454b550 + `--zip_undeclared_test_outputs` now preserves symlinks when zipping `$TEST_UNDECLARED_OUTPUTS_DIR`. Closes bazelbuild#19948. Change-Id: Ia4a8a9699e4e2f40498342af55babc5554a9ac93 Commit bazelbuild@8e639df + **[Incompatible]** Toolchains and execution platforms are now registered in the following order with `--enable_bzlmod`: 1. root module's module file 2. `WORKSPACE` or `WORKSPACE.bzlmod` 3. non-root modules' module files 4. default toolchains registered by Bazel (does not apply with `WORKSPACE.bzlmod` or execution platforms) Fixes bazelbuild#20354 Closes bazelbuild#20407. Commit bazelbuild@96b3612#diff-a8d3aed419e661d4dbecb2dc6668444212d7b1707ff61330b7d8aae61e75d4df Acknowledgements: This release contains contributions from many people at Google, as well as Adam Lavin, Adrian Imboden, Alan Falloon, Alessandro Patti, Alex Eagle, Alexander Grund, AlexTereshenkov, Amanda L Martin, Amet Umerov, Andreas Herrmann, Andy Hamon, andyrinne12, Ankit Agarwal, Ankush Goyal, Anshuman Mishra, Anthony Ter-Saakov, Antoine Musso, Artem Zinnatullin, arun.sampathkumar, aryeh, Austin Schuh, bazel.build machine account, Ben Lee, Benjamin Lee, Benjamin Peterson, Benjamin Sigonneau, Boleyn Su, Brentley Jones, Cameron Martin, Chirag Ramani, Chris Sauer, Christopher Peterson Sauer, Christopher Rydell, Christopher Sauer, Conall O'Brien, Daniel Grunwald, Daniel KT, Daniel Wagner-Hall, David Ostrovsky, dhmemi, Dimi Shahbaz, eareilg, Ed Schouten, Emil Kattainen, Ezekiel Warren, Fabian Meumertzheim, Fahrzin Hemmati, Gaspare Vitta, George Gensure, Greg, Greg Magolan, Grzegorz Lukasik, Guillaume Maudoux, Gunnar Wagenknecht, Halil Sener, Hao Yuan, hvadehra, hvd, Ilan Keshet, Ivan Golub, Jack Dai, James Ma, Jason Mobarak, Jasper, Jay Conrod, Jeff Hodges, Jeremy Volkman, Jimm chja20, Joel Jeske, John Hinnegan, Jon Landis, Jon Parise, jonrose-dev, Joseph Lisee, Joseph Tilahun, Julio Merino, Kai Zhang, keertk, Keith Smiley, Konstantin Erman, Krzysztof Naglik, kshyanashree, Kun-Lu, Laurenz Altenmller, Lee Mracek, Letu Ren, lripoche, Maksim, Malte Poll, Marc Redemske, Marc Zych, Matt Mackay, Matt Vollmer, Mauricio G, Mauricio Galindo, Maxim Matyunin, Maxwell Elliott, mohamadk, Nathan (Blaise) Bruer, nathyong, NelsonLi0701, nglevin, Nicholas Junge, Nick Biryulin, Nick Korostelev, Oliver Lee, Orion Hodson, Oscar Garzon, Patrice Duroux, Patrick Balestra, Peter Lobsinger, Philipp Schrader, Rasrack, Red Daly, redwrasse, Robin Tweedie, robincaloudis, Roger Hu, Romain Chossart, Roman Salvador, Sagar Pathare, Saleem Abdulrasool, Sam Shadwell, Sara Adams, Sascha Moecker, Scott Minor, Severin Strobl, Shaygan Hooshyari, Siddhesh Bhupendra Kuakde, Simon Mavi Stewart, something_vague, Son Luong Ngoc, Steve Barrau, Stiopa Koltsov, stoozy, Sushain Cherivirala, Takeo Sawada, Taylor Barrella, tbaing, Ted Kaplan, Ted Logan, Thi Don, Thulio Ferraz Assis, Timothy Gu, Tom Cnops, Tyler Williams, Ulf Adams, Ulrik Falklof, Uri Baghin, Vertexwahn, Xavier Bonaventura, Xdng Yng, Xiangquan Xiao, Xùdōng Yáng, Yannic, Yannic Bonenberger, Yi Cheng, Yuval Kaplan, yuzhy8701, Zheng Wei Tan.

lberki added type: bug untriaged labels Nov 29, 2023

lberki assigned iancha1992 and rakshitasingh05 Nov 29, 2023

lberki added the area-Bzlmod Bzlmod-specific PRs, issues, and feature requests label Nov 29, 2023

lberki assigned Wyverald and katre and unassigned iancha1992 and rakshitasingh05 Nov 29, 2023

lberki changed the title ~~Unexpected interaction between toolchain resolution and~~ Unexpected interaction between toolchain resolution and bzlmod Nov 29, 2023

meteorcloudy added P2 We'll consider working on this in future. (Assignee optional) and removed untriaged labels Nov 29, 2023

katre added a commit to katre/bazel that referenced this issue Nov 29, 2023

Ensure that the autodetected python runtimes are marked as compatible

64cfc99

with the host platform. This should prevent them from being used inappropriately in remote execution. Part of changes for bazelbuild#20354.

katre mentioned this issue Nov 29, 2023

Ensure that the autodetected python runtimes are marked as compatible #20356

Closed

Wyverald added the team-ExternalDeps External dependency handling, remote repositiories, WORKSPACE file. label Nov 30, 2023

Wyverald added this to Bzlmod Nov 30, 2023

github-project-automation bot moved this to Todo in Bzlmod Nov 30, 2023

Wyverald added this to the 7.1.0 release blockers milestone Nov 30, 2023

bazel-io mentioned this issue Dec 1, 2023

[7.0.0] Unexpected interaction between toolchain resolution and bzlmod #20406

Closed

fmeum mentioned this issue Dec 1, 2023

Give WORKSPACE toolchains and platforms precedence over non-root modules #20407

Closed

copybara-service bot closed this as completed in 96b3612 Dec 4, 2023

github-project-automation bot moved this from Todo to Done in Bzlmod Dec 4, 2023

iancha1992 mentioned this issue Dec 4, 2023

[7.0.0] Give WORKSPACE toolchains and platforms precedence over non-root mo… #20430

Merged

Wyverald modified the milestones: 7.1.0 release blockers, Mainline issues targeted for 7.1.0 Jan 11, 2024

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Unexpected interaction between toolchain resolution and bzlmod #20354

Unexpected interaction between toolchain resolution and bzlmod #20354

lberki commented Nov 29, 2023 •

edited

Loading

meisterT commented Nov 29, 2023

fmeum commented Nov 29, 2023

meteorcloudy commented Nov 29, 2023

lberki commented Nov 29, 2023

fmeum commented Nov 29, 2023

katre commented Nov 29, 2023

lberki commented Nov 29, 2023

katre commented Nov 29, 2023

lberki commented Nov 29, 2023

katre commented Nov 29, 2023

lberki commented Nov 29, 2023

katre commented Nov 29, 2023

Wyverald commented Nov 29, 2023

katre commented Nov 29, 2023

rickeylev commented Nov 29, 2023 •

edited

Loading

Wyverald commented Nov 29, 2023

fmeum commented Dec 1, 2023

meteorcloudy commented Dec 1, 2023

meteorcloudy commented Dec 1, 2023

iancha1992 commented Dec 5, 2023

Unexpected interaction between toolchain resolution and bzlmod #20354

Unexpected interaction between toolchain resolution and bzlmod #20354

Comments

lberki commented Nov 29, 2023 • edited Loading

Description of the bug:

Which category does this issue belong to?

What's the simplest, easiest way to reproduce this bug? Please provide a minimal example if possible.

Which operating system are you running Bazel on?

What is the output of bazel info release?

If bazel info release returns development version or (@non-git), tell us how you built Bazel.

What's the output of git remote get-url origin; git rev-parse master; git rev-parse HEAD ?

Is this a regression? If yes, please try to identify the Bazel commit where the bug was introduced.

Have you found anything relevant by searching the web?

Any other information, logs, or outputs that you want to share?

meisterT commented Nov 29, 2023

fmeum commented Nov 29, 2023

meteorcloudy commented Nov 29, 2023

lberki commented Nov 29, 2023

fmeum commented Nov 29, 2023

katre commented Nov 29, 2023

lberki commented Nov 29, 2023

katre commented Nov 29, 2023

lberki commented Nov 29, 2023

katre commented Nov 29, 2023

lberki commented Nov 29, 2023

katre commented Nov 29, 2023

Wyverald commented Nov 29, 2023

katre commented Nov 29, 2023

rickeylev commented Nov 29, 2023 • edited Loading

Wyverald commented Nov 29, 2023

fmeum commented Dec 1, 2023

meteorcloudy commented Dec 1, 2023

meteorcloudy commented Dec 1, 2023

iancha1992 commented Dec 5, 2023

lberki commented Nov 29, 2023 •

edited

Loading

What is the output of `bazel info release`?

If `bazel info release` returns `development version` or `(@non-git)`, tell us how you built Bazel.

What's the output of `git remote get-url origin; git rev-parse master; git rev-parse HEAD` ?

rickeylev commented Nov 29, 2023 •

edited

Loading