WebTransport over HTTP/3 support

[yutakahirano]

Hi,

We are developing WebTransport over HTTP/3 which replaces WebTransport over QUIC (a.k.a. QUICTransport). I would like to use aioquic to implement a test server for web-platform tests (similar to what I did, writing a QuicTransport server on top of aioquic for testing).

https://tools.ietf.org/html/draft-ietf-webtrans-http3-00 is the spec draft.

To write a test server, I would like to implement some features specific to WebTransport over HTTP/3. Specifically,

1. We would like to handle SETTINGS frames.
2. WebTransport over HTTP/3 defines some custom frames (0x54 and 0x41).

To support WebTransport over HTTP/3, we need to...

1. support WebTransport over HTTP/3 directly in H3Connection,
2. make H3Connection a bit more flexisible, or
3. re-implement H3Connection out of aioquic.

I would like to avoid the last solution if possible.

Do you have any suggestions? I'm happy to make PRs if needed. Thank you!

cc: @vasilvv

[jlaine]

Hi @yutakahirano that's exciting news! I'll need a bit of time to familiarise myself with the draft spec before I can give you a meaningful answer.

[jlaine]

I had a first look at the specs, and I'm pretty sure this is something we want support for directly in H3Connection. My main concerns at this stage are that the APIs in the specs are asynchronous, where as I have strived to make H3Connection event-based, so as to allow it to be wrapper in whatever framework users see fit. Any thoughts here?

@pgjones you may want to keep track of this issue :)

[yutakahirano]

Thank you very much!

My main concerns at this stage are that the APIs in the specs are asynchronous

Can you give some examples?

[guest271314]

Has any progress been made? I was using aioquic before deprecation/removal of QuicTransport/quic-transport then opened this discussion is aiohttp aio-libs/aiohttp#5581.

[jlaine]

I have started work on this in PR #204

[jlaine]

@yutakahirano I've got some basic support for unidirectional and bidirectional streams, and am now looking at datagram support. One question to which I have not found the answer in the specs is : what is the expected behaviour if a DATAGRAM cannot be parsed due to a truncated Session ID?

[yutakahirano]

@DavidSchinazi @vasilvv can you answer the above question?

[DavidSchinazi]

If a DATAGRAM is too short to parse the ID at the start, then the peer is violating the protocol. I'd suggest closing the connection with a H3_GENERAL_PROTOCOL_ERROR in that case. That's what draft-ietf-masque-h3-datagram mandates.

@vasilvv we should tweak draft-ietf-webtrans-http3 to have it reference draft-ietf-masque-h3-datagram for things like this.

[guest271314]

@jlaine If this is the appropriate venue and issue, can you provide guidance on what needs to be updated in https://github.com/GoogleChrome/samples/blob/gh-pages/webtransport/quic_transport_server.py for the ability to test the changes in #204 at the front-end?

[jlaine]

It's not going to be "tweak" of that example, it will look very different as now everything is built on top of HTTP/3 and not directly on top of QUIC.

My plan is to update the http3_server.py example, providing some basic abstraction to allow writing an example app (outside of the server code). For a more thorough integration into the Python web app ecosystem under the ASGI umbrella, I've started a conversation here:

django/asgiref#280

[jlaine]

PR #204 has been merged, I'm going to close this issue once we have a minimal working demo in the http3_server.py example.

[jlaine]

PR #209 has been merged. It extends the HTTP/3 demo server with a minimal WebTransport demo. As usual the demo is accessible on https://quic.aiortc.org/

[yutakahirano]

Awesome! Thank you very much!

[guest271314]

As usual the demo is accessible on https://quic.aiortc.org/

How to test the demo from HTML? What does

You must set the :protocol pseudo-header to "webtransport".

mean and what does that look like in code?

[jlaine]

@guest271314 you are right there is currently no JavaScript on the demo page exercising WebTransport as it is still behind a flag.

You can do the following:

• launch Chrome with --enable-experimental-web-platform-features
• open the Developer Tools
• run the following code:

let transport = new WebTransport('https://quic.aiortc.org/wt');
await transport.ready;

let stream = await transport.createBidirectionalStream();
let reader = stream.readable.getReader();
let writer = stream.writable.getWriter()

await writer.write(new Uint8Array([65, 66, 67]));
let received = await reader.read();
await transport.close();

console.log('received', received);

If all is well you should see:

[guest271314]

@jlaine I am running Chromium dev channel with the flag enabled

Running the code at https://quic.aiortc.org/ does produce the expected result.

Installing dependencies

sudo apt install libssl-dev
git clone https://github.com/aiortc/aioquic.git
cd aioquic
pip install -e .
pip install asgiref dnslib httpbin starlette wsproto

followed by

python3 examples/http3_server.py --certificate tests/ssl_cert.pem --private-key tests/ssl_key.pem

example code at console

try {
  let transport = new WebTransport('https://localhost:4433');
  await transport.ready;

  let stream = await transport.createBidirectionalStream();
  let reader = stream.readable.getReader();
  let writer = stream.writable.getWriter()

  await writer.write(new Uint8Array([65, 66, 67]));
  let received = await reader.read();
  await transport.close();

  console.log('received', received);
} catch(err) {
  console.error(err.message);
}

results in

VM165:15 Connection lost.

at console

and

2021-07-23 17:32:30,399 INFO quic [42fa797bc32090aa] Connection close received (code 0x12E, reason 199:TLS handshake failure (ENCRYPTION_HANDSHAKE) 46: certificate unknown)

at terminal.

If this needs to be moved to a different issue, kind say so.

I had QuicTransport and WebTransport with quic-transport working using the instructions at
https://github.com/GoogleChrome/samples/blob/1f9ed024a19b9fbb0244585e60e8ef325a93b856/webtransport/quic_transport_server.py#L52-L71.

I tried running the code with tests/ssl_cert.pem though there appears to be an error.

Can you kindly create a minimal, verifiable, complete example, including all of the steps necessary to run WebTransport locally using this repository?

cat ssl_cert.pem | openssl rsa -pubin -outform der | openssl dgst -sha256 -binary | base64
unable to load Public Key
139793936639296:error:0909006C:PEM routines:get_name:no start line:../crypto/pem/pem_lib.c:745:Expecting: PUBLIC KEY
47DEQpj8HBSa+/TImW+5JCeuQeRkm5NMpJWZG3hSuFU=

[jlaine]

Come on please use some common sense, this is not specific to WebTransport. You are seeing a warning about a certificate not being valid, as well you should. Maybe the certificate is the issue?

Try looking for how the --ignore-certificate-errors-spki-list flag works.

[guest271314]

Come on please use some common sense

The universe is based on questions and answers; logic. Thus I asked the experts the question.

I found #127.

Got the server working. The URL passed to WebTransport needs to be 'https://localhost:4433/wt'.

It would be helpful to place the steps necessary to run the example on Chromium/Chrome in https://github.com/aiortc/aioquic/tree/main/examples.

Something very simple like

"To run the examples on Chromium or Chrome

openssl x509 -in tests/ssl_cert.pem -pubkey -noout | openssl pkey -pubin -outform der | openssl dgst -sha256 -binary | openssl enc -base64

the launch with

google-chrome --ignore-certificate-errors-spki-list=BSQJ0jkQ7wwhR7KvPZ+DSNk2XTZ/MS6xCbo9qu++VdQ= --origin-to-force-quic-on=localhost:4433

make the request to 'https://localhost:4433/wt'".

will suffice to help users who might not be as well-versed in Python and certificates as others.

[totaam]

@jlaine I must be missing some common sense, somewhere.

To ensure that I was not the victim of typos, I scripted all the steps so that the certificate is generated and the browsers are launched with the correct hash, both on the command line (for Chrome - note: you may need to exit your browser session for the command line flags to take effect) and in the Javascript serverCertificateHashes attribute of WebTransport:
https://github.com/Xpra-org/xpra/blob/master/tests/xpra/net/webtransport.sh

I also tried various openssl command lines for using different kinds of certificates - since I saw some references saying that only EC certs could be used with serverCertificateHashes.

Then I also tried to use mkcert: the certificate is accepted by both Chrome and Firefox for https connnections to my local web server (so the certificate is OK?), but still, not for WebTransport connections to the examples/http3_server.py.
The same browsers quite happily make connections to https://quic.aiortc.org/wt. (so the browsers are fine too?)
Chrome says that this cert uses PKCS #1 SHA-256 With RSA Encryption.

What am I missing?
What magic incantation do I need to be able to use my cert with aioquic and WebTransport?