x test error_index_generator triggers 'nested groups are not supported by GHA' check

[oli-obk]

Command

x test error_index_generator

Output

Testing stage1 error-index (x86_64-unknown-linux-gnu)
Building tool rustdoc (stage0 -> stage1, x86_64-unknown-linux-gnu)
thread 'main' panicked at 'nested groups are not supported by GHA!', tool.rs:486:30
note: run with `RUST_BACKTRACE=1` environment variable to display a backtrace

This is a regression from #113514.

[jyn514]

in general the fix for this is usually to add let rustdoc = builder.rustdoc_cmd before the builder.msg line so that they don't overlap

[jyn514]

we might want to consider changing DryRun::SelfCheck to bypass the type-based cache so it catches ordering bugs like this

cc @Mark-Simulacrum

[Mark-Simulacrum]

I don't understand how dry run is relevant here, isn't this a problem in all cases? Dry run and real should have roughly the same ensure graph to the extent possible and our tests rely on that...

It also seems like based on the linked PR here this assertion is always guaranteed to show up at runtime pretty much constantly anytime someone accidentally adds additional nesting with a group? I could see that the right fix is to track the group and end + restart it so that there's no nesting and instead a linear non-nested list of groups...

[jyn514]

dry-run and real have the same problems, yes. my idea was to do extra checks in dry-run so that x test without arguments would have caught this - the reason it only gets triggered with the error_index_generator filter is because normally, an earlier Step will happen to call ensure(tool::Rustdoc), so the dynamic GHA check never gets run.

this assertion is always guaranteed to show up at runtime pretty much constantly anytime someone accidentally adds additional nesting with a group

correct. i was hoping that the check was eager enough they'd notice immediately and fix it in order to make CI pass, but this issue shows that it's non-trivial to do so because of the way it depends on the cache. this is how dry-run gets involved - when it's enabled, bypassing the cache is cheap and we can just do it unconditionally, which would have caught this bug before it got merged.

I could see that the right fix is to track the group and end + restart it so that there's no nesting and instead a linear non-nested list of groups...

hmm, that might handle our current cases ok, but it's not correct in general - consider something like

let _guard = builder.msg("test rustdoc");
run_cargo_test(compiler, "-p librustdoc"); // takes e.g. 60 seconds
builder.ensure(RustdocUi); // builds rustdoc internally

that will split up the single message before and after, which doesn't seem quite right, although i suppose it might be better than panicking?

::group::test rustdoc
// 60 seconds
::endgroup
::group::build rustdoc
::endgroup
::group::test rustdoc
// takes non-trivial time to run tests/rustdoc-ui suite
::endgroup

[Mark-Simulacrum]

Ah, I see. I'm not convinced that we'd not stop catching other problems and/or introduce them by changing dry run.

The split example you suggest feels right to me - in some sense more right - it means we are putting things in finest granularity groups. It also matches what we do for metrics emission, I believe...

I will say that I don't really understand the value in the grouping, so perhaps that would change my thinking there....

At minimum, we might consider not panicking outside of GHA workloads, since the grouping doesn't matter then?