Clarify usage patterns for feature gates

[vados-cosmonic]

Hey all, after some discussion in the following PRs:

bytecodealliance/wasm-tools#1689
bytecodealliance/jco#459

It seems like now would be a good time to discuss and really pin down the expected usage pattern of feature gates.

Intuitions seem to clash on how the gates should be used, so the decision should likely be discussed and made (hopefully quickly, if possible!) with a record for people to see the whys.

Context

At present, the WIT design feature gates section describes a reality that requires/implies a few things:

1. Package versions may not be present
There is no mention that a package containing @since() that is not itself versioned is a bug.

There are currently some tests in wasm-tools that use @since in packages that have no version specified (i.e. this is expected to be "right" or at least legal).

2. @since(version = v, feature = f) works as an OR

Then, once the feature is stable (and, in a WASI context, voted upon), the @unstable gate is switched to a @SInCE gate. To enable a smooth transition (during which producer toolchains are targeting a version earlier than the @since-specified version), the @SInCE gate contains an optional feature field that, when present, says to enable the feature when either the target version is greator-or-equal or the feature name is explicitly enabled by the developer.

This necessarily means that version is used or the feature is used.

**3. @since() may specify versions in the future

Thus, c is enabled if the version is 0.2.2 or newer or the fancy-foo feature is explicitly enabled by the developer.

The first part of this "if" can only work if you were able to specify a future (unreleased) version. It doesn't make sense to describe it like this unless a situation like this is possible:

// there's obviously quite a gap in these versions but hopefully it's useful for the sake of discussion.
package some:pkg@0.1.0;

@since(version = 0.2.2, feature = access-early-a)
interface a { ... }

Options for moving forward

There seem to be at least these three ways of handling it:

1. OR (current)
In this situation, we keep the functionality as it exists now. This requires the caveats mentioned above.

One of the questions associated with this approach is whether it makes sense for a package at version 0.1.0 to be able to have a @since(version = 0.2.0) in it at all.

2. AND (new)

In this situation version and feature are always both considered for enabling an item.

This has a few benefits I can think of:

• Always knowing an import requires all the @since related options makes it easy to do things like infer requirements from an import (versus not knowing if that means you need a version or a feature)
• It's possible to write features that actually change across versions, and versions that act differently w/ certain features enabled

3. Strict @unstable -> @since with ignored features after stabilization

Alex has laid out his intuition for this, which makes also makes sense.

Strictly moving from @unstable to @since and then ignoring features once enabled, along with making sure that future unreleased versions cannot be used in @since removes a lot of ambiguity.

What needs to be clarified

It would be nice if we could lay out the actual intended evolution of a feature through gates -- this is kind of already spelled out already in the WIT design doc, but there seems to be two levels of progression here:

• unstable -> stable (experimental features)
• not enabled version -> enabled version (future feature prep)

I think we're done at the point when we have:

• A flow (probably best written out by steps) that people agree with that most importantly seems intuitive for users to follow (@unstable -> @since, @since(version = v) -> @since(version = v + 1) ?)
• Which levels of progression the feature gates feature should be handling

[alexcrichton]

cc @yoshuawuyts as well

[lukewagner]

Great points and thanks for the clear writeup! Agreed this does seem like an important issue to clarify in WIT.md.

My initial impression is that we need to define some validation criteria on top of our current WIT grammar for gates to reject the following cases:

• @since gates inside an unversioned package
• multiple @since or @deprecated gates

(Multiple @unstable feature names seem potentially useful, but maybe we could adopt a simpler "no repeated gates" rule if instead we allowed @unstable to have a comma-delimited list of names? In any case...)

With these restrictions, the OR option (1) you listed above I think gets simpler to think about: a feature is enabled if any of its @unstable feature names are enabled OR the developer's chosen target version is >= the @since gate.

In terms of the expected evolution of a feature, I agree that, for a WIT package author, the first step is @unstable and then a feature may progress to @since. Once a feature reaches @since, I don't imagine the @since-gate's declared version ever changing. What does change, independently, is the "target version" that a developer builds with (which I think is necessarily <= the WIT package version). If the developer keeps the "target version" locked to the current WIT package version then (with the above restrictions) @unstable should have no effect (everything is enabled), but if a developer makes the "target version" trail the current WIT package version, then @unstable would be useful for allowing the developer to turn on a newer feature they want explicitly.

At least that's my current working understanding. It might not be right though; happy to hear what others think.

[vados-cosmonic]

Waiting to hear what others think but wanted to comment on this:

What does change, independently, is the "target version" that a developer builds with (which I think is necessarily <= the WIT package version). If the developer keeps the "target version" locked to the current WIT package version then (with the above restrictions) @unstable should have no effect (everything is enabled), but if a developer makes the "target version" trail the current WIT package version, then @unstable would be useful for allowing the developer to turn on a newer feature they want explicitly.

This is quite a new use-case for me but in talking about version stuff yesterday (and during the package SIG meeting) it did occur to me -- right now it seems to me that there's one way for developers to "choose" a version: pulling in the WIT file(s) with the right package version at the top.

Just to make sure we're on the same page -- are we talking here about about a future where some build tool allows a user to choose a different package version despite what is written in the local WIT, explicitly? Is this the unevenly-distributed present somewhere already?

[alexcrichton]

Ah yeah this is something different from what I was assuming as well. I've not considered the ability of a developer to, for example, locally have WASI 0.2.3 and then say "I'd like to target 0.2.1 instead". Rather the developer would be required to download WASI 0.2.1 WITs instead. Personally I think it's a big can of worms to try to view newer versions of files as-if they were older versions of files and it's more robust to use the exact copies of the older files as they were.

[lukewagner]

Ah thanks, that's very helpful. I had forgotten that we (now) have a versioned repository of WIT packages (and was imagining the toolchain just had a single most-recent-version and could "backdate" to older versions by subtracting @since-gated fields), but saying that we just fetch the published package for the version we want and possibly multiple of them (which we'd need to do anyways to support multiple major versions) of course makes sense.

Ok, so in that case, if @since can't exceed the package's version, then if the toolchain/developer targets a WIT package version X, everything in X is presented (to bindgen), and so is there any value to having a residual @unstable gate or, as an additional validation requirement (to those I listed above), could we say you can't have both @since and @unstable?

[alexcrichton]

I think it's reasonable to say @since xor @unstable yeah

[lukewagner]

Ok, if that sounds right to others too, then maybe a summary of the fix is: in #377 we revert this commit and go back to a single-gate and then also add the "packages containing @since must declare a version" rule and everything is clear?

[lukewagner]

Oops, sorry, I forgot about @deprecated making sense in conjunction with @since. So then revising that, is the summary: @since xor @unstable (which I think we could capture in the grammar) + explicit package version?

[vados-cosmonic]

A bit late here but looks like we hit consensus here, thanks ya'll -- I'll make the change to the wasm-tools PR and I'll make a follow up PR to this repository to add an explicit textual explanation to the section in WIT.md.