Configurable multistream-select protocol. V1Lazy variant.

[romanb]

This is a proposal for making the multistream-select protocol (version) configurable both on transport upgrades as well as for individual substreams. It adds a new "lazy" version of multistream-select 1.0 that is identical on the wire but delays sending of negotiation protocol frames as much as possible, which is only safe to use under additional assumptions that go beyond what is required by the multistream-select v1 specification (see the inline doc-comment on Version::V1Lazy and the discussion in #1212).

There are a couple of motivations behind this:

1. It will arguably be necessary eventually to be able to specify the multistream-select protocol version to use, at least as a dialer, to ever support 0-RTT negotiation in the presence of multiple protocol versions to choose from.

2. By defaulting to the standard V1 version, the optimisations provided by lazy negotiation are still provided as an opt-in per transport and per substream protocol, without losing full compatibility by default (i.e. addressing multistream-select interoperability issue with JS #1241 in a different way than Disable multistream optimization to be compatible #1244). It may thus offer an opt-in fast-path for specific projects or protocols, especially ones that want to make heavy use of the substream-per-request pattern, such as in network: Use "one shot" protocol handler. paritytech/substrate#3520.

3. The differences between the versions is constrained to the multistream-select library, not requiring outside workarounds for completely safe V1 compatibility like there is currently in libp2p-core with the additional AwaitNegotiated step for dialer upgrades. I.e. the distinction between fully conformant / interoperable V1 and the V1Lazy variant are fully inside multistream-select, with the choice exposed through the choice of Version for the dialer (the listener can always safely support all protocol versions, which also provides a general upgrade path from older to newer versions).

An example for configuring transport upgrades to use lazy negotiation:

            let transport = libp2p_tcp::TcpConfig::new()
                .upgrade(upgrade::Version::V1Lazy)
                .authenticate(libp2p_secio::SecioConfig::new(local_key))
                .multiplex(libp2p_mplex::MplexConfig::new())

An example for configuring outbound substreams on a handler to use lazy negotiation:

let proto = SubstreamProtocol::new(p).with_upgrade_protocol(upgrade::Version::V1Lazy);
OneShotHandler::new(proto, self.config.inactivity_timeout)

The only thing that would still be nice to have is to default the multistream protocol version used for substreams to the one used for the transport upgrades, but I didn't see an easy way to do that kind of propagation at the moment.

[tomaka]

I'm not so much of a fan of this. I think there should be a default version that the user can override.
However I see that the builder would have to be rewritten to account for this, so it's fine.

[romanb]

There is a default version - Version even implements Default. Would you prefer having two different variants of Transport::upgrade, one without a Version parameter and one with such a parameter, whereby the former delegates to Builder::new(...) with the default version? If so, any name suggestions for the upgrade variant taking a version? upgrade_with? upgrade_using? upgrade_with_protocol? As you see my reason for not doing so was that I struggled with one of the hardest problems in computer science: naming things.

[tomaka]

Hmmm, I thought the idea was to keep /multistream/1.0.0, but I guess this works as well.

[romanb]

Without a different protocol name, there can be no different Versions based on which both dialer and listener implement different behaviour (e.g. when to flush). I don't see how that could be done otherwise.

[tomaka]

From the client's point of view, dialing V1 and V1Lazy is the same (except for the protocol name), no?

[romanb]

From the client's point of view, dialing V1 and V1Lazy is the same (except for the protocol name), no?

No, for V1 the dialer now never prematurely goes into the Expecting state, it flushes every protocol proposal and reads the reply. This was done previously in libp2p_core::upgrade::apply_outbound anyway by waiting for negotiation to complete between upgrades, as I mentioned in the PR description, to avoid the edge-cases discussed in the other PRs. Now this behaviour is localised to multistream-select and subject to the chosen version.

Also, the listener needs to know whether the dialer wants the lazy variant or not, based on which it decides whether it needs to immediately flush its protocol response (addressing #1241 as mentioned in the PR description). Without a different protocol name on the wire, there can be no parsing into different Versions.

Apart from that, even though V1 and V1Lazy are identical as far as the protocol messages on the wire are concerned, important expectations on the granularity with which requests and responses are sent and received or buffered that can cause incompatibilities if not observed by both peers are part of the protocol and thus such differences deserve a different protocol name from that perspective as well, I think.

[tomaka]

I may be mistaken, but isn't it possible to also do the optimization where, in the case of a single protocol, we assume that the remote supports it, and close the connection if it doesn't?

[romanb]

I may be mistaken, but isn't it possible to also do the optimization where, in the case of a single protocol, we assume that the remote supports it, and close the connection if it doesn't?

For safety of nested protocol negotiations, only V1Lazy can optimistically assume a protocol proposal to succeed and it does that if the dialer only supports a single protocol (or he gets to his last protocol proposal, i.e. runs out of options). The dialer then expects to get confirmation for that protocol and otherwise fails negotiation. Whether failed negotiation results in closing the connection is of course not up to multistream-select itself, since it operates on an opaque I/O resource which may or may not be a connection, but e.g. reading from a Negotiated<C> (or a graceful shutdown after writing) will fail with an error if negotiation fails.