Document browser/engine adoption and removal requirements

[ddbeck]

We should probably have data guidelines which describe what criteria must be satisfied to introduce a new browser into BCD and what criteria may permit a browser to be removed.

I'm writing these up together because I think one data guideline obligates us to have the other. If we don't have a plan for removing browsers, it becomes very risky to add a browser—it's a high bar to say we're going to take on a browser permanently. If we don't have a plan for adding a browser, then it makes removal seem like censure, rather than an ongoing stewardship decision.

My actual ideas for the guidelines themselves are going into separate comments below, to make them easier to reference individually.

This issue inspired by #7232 (review). Related to:

• Document JSDOM Web API and HTML support #3180 (closed)
• Data on things that aren't browsers? #3280
• new engine/platform Deno #5476
• include JavaScriptCore in compatibility table #6000
• Show Firefox Reality as a separate browser for XR APIs #6512

[ddbeck]

For new browsers and engines, we probably need (in decreasing order of importance) these things to consider adding one to BCD:

• a compelling consumer story (e.g., a BCD consumer, such as MDN or caniuse, expresses an interest, or someone is planning to do something with the data that might plausibly grow BCD's reach)
• reviewers (e.g., two or more people with interest and ability to test data relating to new and existing releases, or at least one reviewer acting on behalf of the vendor in BCD)
• published release information (e.g., release notes with version numbers and dates etc.)
• documentation (e.g., how to get and test a feature in that browser, links to resources that might help with it, etc.)

The first is probably strictly required, while the others need to make a strong showing for acceptance.

[ddbeck]

For browsers and engines already in BCD, a browser probably needs to show habitual (perhaps six months or more) evidence of (in decreasing order of importance):

• negative/neutral consumer interest in the browser's data (e.g., MDN or caniuse does not object to removal)
• poor data coverage with negative trends (for example, few features with limited or flat growth, few features with real values, etc.)
• infrequent community or vendor involvement in issues or PRs relating to the browser
• infrequent new PRs relating to the browser (e.g., weeks or months go by without PRs touching the browser's data)

These guidelines probably imply we should drop the UC and QQ browsers.

[sideshowbarker]

Agreed on all counts

[sideshowbarker]

In the interest of having a concrete patch that we can all iterate on together, I took the liberty of raising #7244.

The initial content of the patch is just the content from #7238 (comment) and #7238 (comment), lightly edited.

[ddbeck]

@chrisdavidmills Could I have your take on this proposal (and its effects)? I think this represents an editorial decision, so I thought you might like to be aware of it and have the opportunity to comment first. Here's a bit of a recap, since there's a few threads coming together here:

@sideshowbarker got the core of the matter more succinctly than I would have in #7232 (comment) :

Regardless of how much developer interest there is in [UC and QQ data], if we don’t have anybody committed to adding data for them and keeping it up to date, then it can’t really be serving the needs of developers.

UC and QQ data in BCD is in poor shape. It represents a bit of a barrier to eliminating non-real data and #7232 is an example. In that PR, we recognized and had apparent consensus for dropping those browsers. This issue (and @sideshowbarker's follow-up PR #7244) documents more generally what criteria we might follow to consider adopting a new browser or removing an old one.

The real consequence is in #7240: removing UC and QQ from the data and schema. The docs in #7244 leave the door open to that data some returning, provided there's real interest and effort behind it.

Is it OK to proceed with it? Any questions or concerns?

[chrisdavidmills]

@ddbeck I think this all makes sense. We need to have guidelines for this, to inform future decisions. A few thoughts:

1. I think "reviewers" should also be strictly required in the criteria for adding a new browser or engine — without dedicated reviewers the PR flow for an engine will grind to a standstill, regardless of how freely the data is flowing in and what quality it is.
2. I also think a strict requirement should be "evidence of active personnel adding data", and perhaps also "suport form the vendor behind that vendor or browser". This is the other side of the coin to 1. — we need an active, enthusiastic community adding data regularly so that data for the browser or engine does not begin to stagnate. Ideally there should be interest from the vendor themselves to help drive this. If they're not interested, then why bother?
3. We should probably also have a page that lists the folks present to work on data additions and reviews for ech vendor / engine. Rather than just say we have this, show it. This also provides evidence for what we'll say for the above criteria.

[ddbeck]

@chrisdavidmills This is really helpful feedback. Thank you! Some thoughts on your thoughts:

I also think a strict requirement should be "evidence of active personnel adding data", and perhaps also "suport form the vendor behind that vendor or browser"

Do you want to make that explicit in the data guideline? Right now, #7244 doesn't describe anything as strict.

For what it's worth, I agree that, as an ideal, we ought to have vendor participation to include something. But it feels hard to turn into a formal rule. For example, imagine we were considering accepting Node.js today. Who's the vendor, exactly? Committers on Node.js? The OpenJS foundation? Or is it Google, because of V8? It's a bit fuzzy.

Plus it would seem that the editorial question—how can we best support web developers?—is overriding. For example, we don't get sustained help from Apple for Safari data, but it's not as if we're going to drop that data under these guidelines.

We should probably also have a page that lists the folks present to work on data additions and reviews for ech vendor / engine. Rather than just say we have this, show it. This also provides evidence for what we'll say for the above criteria.

The governance doc does annotate some peers and owners with their employer or specialty. Those lists need updating anyway, so perhaps we could make that a formal part of it while we're at it.

[chrisdavidmills]

@chrisdavidmills This is really helpful feedback. Thank you! Some thoughts on your thoughts:

I also think a strict requirement should be "evidence of active personnel adding data", and perhaps also "suport form the vendor behind that vendor or browser"

Do you want to make that explicit in the data guideline? Right now, #7244 doesn't describe anything as strict.

For what it's worth, I agree that, as an ideal, we ought to have vendor participation to include something. But it feels hard to turn into a formal rule. For example, imagine we were considering accepting Node.js today. Who's the vendor, exactly? Committers on Node.js? The OpenJS foundation? Or is it Google, because of V8? It's a bit fuzzy.

Plus it would seem that the editorial question—how can we best support web developers?—is overriding. For example, we don't get sustained help from Apple for Safari data, but it's not as if we're going to drop that data under these guidelines.

Yeah, you've got some good points here. OK, let's leave it as ideal but not a strict requirement.

We should probably also have a page that lists the folks present to work on data additions and reviews for ech vendor / engine. Rather than just say we have this, show it. This also provides evidence for what we'll say for the above criteria.

The governance doc does annotate some peers and owners with their employer or specialty. Those lists need updating anyway, so perhaps we could make that a formal part of it while we're at it.

Awesome, sounds good.

[developit]

@ddbeck - thanks for connecting me to this thread. I'm curious to know whether dropping browsers like UC and QQ will force users of the compat data to acknowledge the removal, or if it's possible we'll end up with caniuse showing misleading data for those browsers in order to avoid lumping them into an "unknown" grouping. Thoughts?

My hope is that dropping these engines won't be interpreted as them being assumed frozen in time. They're often the browsers that cause over-transpilation by tools that operate based on compat data.

[ddbeck]

@developit Thanks for your input on this. This is an important question I forgot to consider. I checked in with caniuse and got some good news: it will not retain BCD's removed data. It'll probably reflect a more truthful state of affairs, showing that the UC and QQ data is incomplete (at least, in the places where caniuse doesn't have its own data).

[ddbeck]

@Elchi3 This is the issue about adding and removing browsers that we talked about. You can get a preview here, but I think the next step is for me to open a PR with the guidelines from #7238 (comment) and #7238 (comment). If I could get your review on that eventual PR, that would be ideal.

[developit]

@ddbeck that's absolutely fantastic news then!

[ddbeck]

Oh, whoops, I completely forgot: @sideshowbarker already opened a PR with new guidelines in #7244. I'll review this myself, but I've also tagged @Elchi3 for review too.

[Elchi3]

I think this can be closed given we've merged #7244

[chrisdavidmills]

Agreed.