add test tools team

[calebcartwright]

Setting up the new team as per rust-lang/rfcs#3455

cc @rust-lang/devtools as the primary parent team
cc @epage @Muscraft @thomcc @weihanglo

Also want to call out that in this proposal I opted to adjust the name of the team to be Test tools team. I don't feel too strongly, and am unsure if this is even permissible since we had a different name in the RFC. However, the more I looked at words in text, file names, etc. that had "Test" and "Testing" the more it felt like a "testing 1-2-3" or even a potential typo.

Lmk if this needs to be changed back to T-testing though, and/or if we'd like to do a quick bikeshed

[rylev]

r? @Manishearth

[epage]

Also want to call out that in this proposal I opted to adjust the name of the team to be Test tools team. I don't feel too strongly, and am unsure if this is even permissible since we had a different name in the RFC. However, the more I looked at words in text, file names, etc. that had "Test" and "Testing" the more it felt like a "testing 1-2-3" or even a potential typo.

My only nit with the name is it makes it sound like we are focusing on cargo test, cargo bench, etc and not on libtest.

[calebcartwright]

Naming things is hard. Would you prefer sticking with T-testing? Or is there an alternative name you'd suggest?

What would you think about something like... Testing experience team?

[epage]

To add to naming being hard is I saw some confusion over people thinking we were focusing on testing of rustc...

While not ideal, I'm fine with "test" and "testing". I'm also fine with emphasizing the user experience in the name

[thomcc]

T-testing feels like it captures the scope better, but I don't really care. I also don't know if I think that the possible confusion is a huge problem.

[epage]

For myself, I'm still not sold on "tools". I understand that for some, it more is being used like "resources" but naming matters and can help shape thought and behavior, both of the team and those who approach it. We'd be working against the psychological aspects of that by biasing the name (this is especially true when the membership is also biased towards "tools").

[epage]

This also focuses only on the tooling and not on the library side of things

From the RFC, we put our mission down as

This team would be primarily focused on iterating on the test writing and analysis experience, cargo test, and enabling integration points and features for external tools like CI or IDEs.

[calebcartwright]

To make sure I fully understand, are you saying you'd rather reuse that sentence from the RFC verbatim, or just that you feel this isn't a complete description and needs to be changed?

FWIW I don't think the one liner blurbs from the website trump the official scope articulated in places like the RFC and associated team repos. I've always looked at the governance pages of the website as extending to a much broader audience, including those who've potentially never seen (much less written) a line of Rust code, and I think it's helpful to try to generalize things as much as possible in that context.

I personally view "tooling" as such a generalized word that would extend to frameworks, runners, libraries, etc. and not be exclusively scoped to command line executables/subcommands.

However, I'm fully open to any alternative phrasing though if there's any suggestions

[epage]

I was using that line as a point of comparison or idea generation.

For me, tooling is programs / automation and not the whole experience and I suspect a lot of people would be thrown off by the intent of the group if the name and/or description emphasizes tooling.

[calebcartwright]

That's fair. I'd like to avoid mentioning cargo test, cargo bench, etc. in this description though as I think that would also be a line of potential confusion given this would show on the same page with the description of the cargo team.

I'll think on this some more and try to come up with some proposals, and would encourage others to share any ideas as well

[epage]

As @thomcc says, this can always be changed later so it just has to be "good enough"

[Manishearth]

r+ though I think it should be testing-tools maybe? idk, not a huge fan of the name but it's not terrible

(the name is unimportant and to some extent changeable, so it's not a huge deal it doesn't match the one in the rfc)

[thomcc]

So... there's been no discussion for a week despite this being a blocker for the team setting up and getting any planning or initial work¹ started. This is unfortunate, we should try to at least get this settled before rustconf, ideally sooner.

The outstanding issues are around which color to paint a few of the bikeshed-shaped organizational attributes, where nobody seems to feel that strongly about the colors beyond hypothetical concerns that it could lead to confusion, especially about the the sorts of bikes which may be stored in these sheds.

There seem to be two outstanding issues: description and name, both of which are bikeshed-shaped organizational attributes which pretty much everybody² seems to be in agreement that they don't really matter that much.

---

So, description: I'm pretty sure that changing the description later would just be a matter of making a PR to this repo, and nobody would object to us updating it. That is to say, very very easy, even if we need to do it more than once.

If so: This doesn't need to be solved now, we can just land what's already committed now or something else, which is good enough as a starting point.

That said, I actually do think getting the team description right is pretty worthwhile, and think it's not entirely clear about the scope at the moment (and IMO description is the best place to resolve the potential confusion around the team's scope and so on, especially for teams like this one which has slightly too many tendrils for it to easily be captured in their name). So probably after this PR lands (and a t-testing zulip is set up), we should start a thread in it to continue that particular bikeshed a bit more.

---

For the name, unlike description, IMO we just need to pick now and live with it -- even if technically we could change it, I don't think we should without an extremely good reason (even if it's just to avoid causing someone on t-infra or w/e to have to do extra work), and IDK if I think the issues around mentioned above³ with the name would be sufficient to justify this.

So, I was not a fan of the name t-test-tools initially, and honestly I was a little sour on the fact that this was a last-minute change made to the team name between the RFC and here, done without reaching out to other members in advance. That said, it's pretty clear from the PR comment that nothing was meant by this, and the intention was that there would be a quick bikeshed color discussion rather than the medium-sized bikeshed-painting quagmire which occurred. (And not for nothing, it's also hard for me to be actually annoyed about not checking with a team that does not technically exist yet)

Anyway I've mostly come around on t-test-tools (although I still probably prefer the name t-testing). My concrete complaint above was that t-test-tools did not seem to capture the whole scope as well, but I don't think matters as much -- test-tools gets enough of the scope to be sufficient for description to pick up the slack.

That is: if someone hears t-test-tools or "test tools team" or whatever and thinks "ah, cargo test, cargo bench, etc", even tho that's not the full/actual scope, the name still got them a lot of the way there (especially because I suspect many Rust users don't know the difference between cargo test and libtest). Like, the rest of the scope is fairly easy to fill in for them if it matters, and if not, they still could tell what the painting was of, even if they couldn't make out any detail. Which is IMO, plenty for the name alone.

---

TLDR: I vote that we just pick something (concretely, I'd be fine with just merging what's PRed as-is):

• Nobody's cared enough to say anything for a week, and this is basically the last blocker for the team existing.
• The description is something we can workshop after the PR lands, so it doesn't need to be perfect, just "good enough for now", and IMO the existing one is.
• And the names are both fine, and neither seems likely to lead to that much confusion (I think I still prefer the old name, but more weakly before).
• (A bunch of rambling thoughts are also present above, especially around why i think that test-tools is fine, and why we don't need a name which perfectly captures the scope)

Hopefully this is enough to kick this issue into gear again.

Footnotes

1. Deciding meeting schedule and when to hold the first meeting (and other sorts of setup), issue filing and/or tagging, planning and initial discussions on zulip. ↩

2. Sorry if this misrepresents anybody's opinion, it's not intentional if so. ↩

3. Keep in mind that teems like t-crate-maintainers or t-*-contributors in my experience, usually require further explanation to those outside the project, and it doesn't actually cause many problems in practice. So the bar is pretty low for confusing team names. ↩

[epage]

I've kept my replies to the relevant threads to avoid further fracturing the conversation

[calebcartwright]

Appreciate the bump @thomcc and I agree with striving to get this settled before RustConf next week.

I've posted a pseudo-poll in Zulip under https://rust-lang.zulipchat.com/#narrow/stream/301329-t-devtools/topic/testing.20subteam.3F/near/389537299 to hopefully settle the team name

I share the desire and enthusiasm to get the team up and running, and I similarly wouldn't want to lose momentum debating something like a team name ad nauseam. However, it is something that I'm willing to spend a bit of time on up front as I agree we should treat it as (relatively) immutable once we get going; as we all know, the focus area of this team is something that's arguably needed some attention for years, so I'm comfortable spending a week or two on this.

[Manishearth]

I don't think the name is a blocker. I think the council merging this approved-by-team-lead PR is the blocker. I mentioned the name in case people had better ideas, not to block the PR. I said as much in my comment bringing it up.

In general you often need to poke the people with permission to get team PRs merged, that's all that's happening here, I don't think we need to have a whole discussion now.

[rylev]

Sorry I did not realize this was blocked on me - I believed the discussion was blocking. I'll merge this, and we can adjust if we need to. Sorry again for the delay!

[calebcartwright]

My apologies all, especially @Manishearth and @rylev. This was blocked, but not on you nor anything you'd said. It was blocked on me, or at least blocked on a question I'd posed to the members of the new team about a topic we hadn't really yet discussed collectively.

I should've made this PR a draft or otherwise made it more explicit that we were blocked/pending a decision, sorry about that.

We did resolve that topic yesterday, so I'll open a new PR later tonight to update the team def. to reflect that consensus.