Dynamic selection of the transport to use based on ALPN on the client side

[nhenneaux]

I want to implement a JAX-RS client that support HTTP/1.1 and HTTP/2 for a JAX-RS client.

The idea is to use HTTP/2 if the server supports ALPN with HTTP/2 and uses HTTP/1.1 if the server does not provide any information.

I have read some articles about ALPN and it seems to be possible but I don't find anything supporting these protocol at the same time. I did not manage to plug these together.

I am currently using the Jetty HTTP/1.1 transport connector for Jersey and a custom implementation for HTTP/2 transport connector.

Could you implement this dynamic client selection as part of the Jetty project?

This issue is based on this StackOverflow question.

[nhenneaux]

@sbordet is there any plan to implement this issue?

[sbordet]

@nhenneaux we would like to have this implemented, yes.
It is hard-ish to do while covering all existing cases - so it's non-trivial work that has not received enough time yet.

[nhenneaux]

@sbordet ok fine, thanks for your feedback! If you need help please tell me

[sbordet]

@elyograg posted the same will on a different issue, see #1308 (comment).

[elyograg]

@sbordet Thanks for your assistance today.

I had an impression that if I used HttpClient with HttpClientTransportOverHTTP2 and called setUseALPN(true) on the transport, that it might work with both protocols. But either that doesn't work or I've done something wrong. The chance of the latter is probably high -- I'm working with code that I didn't write and barely understand.

The fact that this issue exists (and #1308 before it) has me thinking that there might not be any way to do it currently.

[sbordet]

@elyograg that is correct, there is no way to do it currently. An instance of HttpClient is created with a specific transport and currently cannot switch to another transport.

You can partially workaround this by having two HttpClient instances, one with the HTTP/1.1 transport and one with the HTTP/2 transport, but you have to try one, if it fails try the other, it's clunky.

[barrotsteindev]

Hey,
sorry to drop on by unannounced,
but I was wondering what is the status for this issue?

[sbordet]

I have been working on this a bit, but had to put it aside for other stuff.
It's long overdue, however, and needs to be done, so I'll resume work on it.

[sbordet]

Thoughts dump for future reference.

The issue is currently complicated by the fact that we have (for historical reasons) HttpClient with its own SelectorManager and HTTP2Client also with its own SelectorManager.

When opening a connection, one SelectorManager is used and may have to transfer the connection to the other - not the right way to do things.
The right way is to implement #132.

The difference between a ServerConnector and a ClientConnector is that the server has a fixed "root" ConnectionFactory, while for the client the "root" ClientConnectionFactorys depends on the Origin (where Origin is the tuple (scheme, host, port)).

For example, a for a client to connect to originA it may need proxy_protocol->http1, while to connect to originB it may need tls->alpn->http2.

This is further complicated by existence of HTTP proxies (and support for HTTP CONNECT) and by ALPN, where the protocol selection after alpn is dynamic.

The server-side ConnectionFactory.newConnection() takes the Connector and the EndPoint as parameters. The first is used as a configuration object to extract components such as the Executor, the Scheduler, the ByteBufferPool and - importantly - ConnectionFactorys. The second is used to extract connection specific components such as the SSLEngine.

The client-side has the same needs, plus the need of notifying the application that the connection completed (successfully or not). This requires a Promise also to be passed around, but in general also other parameters that are unknown and that depend on the protocol.

For example, creating a HTTP2ClientConnection needs a HTTP2ClientSession that needs a Session.Listener that can only be provided by the application every time it calls a connect() primitive.
So I am afraid that - differently from the server - a Map<String, Object> will be required in the signature of the generic method that creates client connections (like the current ClientConnectionFactory.newConnection() method).

We would need to support the following "protocols":

• HTTP/1.1
• HTTP/2
• FCGI
• TLS (with ALPN)
• PROXY
• SOCKS 4 (as HTTP proxy)
• HTTP CONNECT (as HTTP proxy)
• UNIX Sockets

Likely the concept of a HttpClientTransport goes away, replaced by a ClientConnector with a number of ClientConnectionFactorys that are retrieved and combined based on the Origin, the HTTP proxy configuration.
These can also be dynamically recombined at a later time, like happens after ALPN negotiation or after HTTP CONNECT.

@gregw comments?

[joakime]

is WebSocket a protocol here? or just built into HTTP/1.1?
If so, what happens when we implement the draft WS-over-H2 spec?

[sbordet]

WebSocket is a protocol for ClientConnector, yes. It needs a different API, so can't be done with the higher level HttpClient.
I imagine WebSocketClient configuring HttpClient's ClientConnector with an additional WebSocket ClientConnectionFactory and after receiving the 101 upgrading the existing HTTP connection.

Have not looked at WS-over-H2 yet. I imagine ClientConnector be configured for HTTP/2 and then WebSocket is a just a payload on DATA frames?

Both good points though.

[CaoManhDat]

Hi, right now https://github.com/apache/lucene-solr/tree/jira/http2 only care about H2C. From https://tools.ietf.org/html/rfc7540#section-3 It seems that h2c and old HTTP/1.1 is compatible with others. Does Jetty client support that kind of upgrade?

[sbordet]

It seems that h2c and old HTTP/1.1 is compatible with others.

What do you exactly mean by that?

Currently HttpClient does not support upgrading from HTTP/1.1 to HTTP/2 as specified here.

[Nutropy]

Hi everyone!

We would very much appreciate it if we could use one HttpClient for HTTP/1.1 and HTTP/2 connections.
Is this issue on the release plan in the near future?

Thanks for the great client and keep up the good work!

[sbordet]

Major work for this issue has been merged in.
Before closing this issue we need to check if we support http/1.1 to h2c upgrade on the client.

[sbordet]

HTTP/1 upgrade to HTTP/2 is tracked in #4382.