HTTP/2 CONNECT

[gsusanto]

Jetty version: 10.0.10

Java version: JavaSE-11 (JRE [17.0.3])

Question
HTTP/2 CONNECT method

This issue has actually been resolved here. I have been trying to use HTTP/2 CONNECT from the client side to connect to a proxy, then try to do a GET request through the proxy to google.com:433.

What I am trying to acomplish can be done by doing a curl command:
curl -x --http2 <proxy-ip>:<proxy-port> https://www.google.com. This will return me a page from google.com/

What I think this command does (by doing -v) is:

1. Client opens HTTP/2 connection to proxy.com, establishes tunnel
2. In a stream in the tunnel, client issues CONNECT request to www.google.com
3. Once CONNECT succeeds, client issues GET in stream that is connected to www.google.com

I might not get that detailed sequence of events right, but what I am trying to accomplish is build a tunnel using HTTP/2 CONNECT to communicate with a server through a proxy. The proxy I am using is h2o server.

I am trying to do the same thing using Jetty. The current code that I have written is:

HTTP2Client client = new HTTP2Client();
client.start();
   
   // Connect to host
FuturePromise<Session> sessionPromise = new FuturePromise<>();
client.connect(new InetSocketAddress(host, port), new ServerSessionListener.Adapter(), sessionPromise);

//Obtain Client Session
Session session = sessionPromise.get();

HttpFields requestFields = HttpFields.build()
	.put(HttpHeader.USER_AGENT, "Jetty HTTP2Client 10.0.10");
MetaData.Request metaDataRequest = new MetaData.Request(HttpMethod.CONNECT.asString(), 
	null, new HostPortHttpField("google.com:443"), null, 
	HttpVersion.HTTP_2, requestFields, -1);
   HeadersFrame headersFrame = new HeadersFrame(metaDataRequest, null, false);

//Listen to response frames.
Stream.Listener responseListener = new Stream.Listener.Adapter() {
	@Override
	public void onHeaders(Stream stream, HeadersFrame frame) {
		MetaData.Response response = (MetaData.Response)frame.getMetaData();
		System.out.println("on Headers frame. Status code: " + response.getStatus());
	}
	
	@Override
	public void onData(Stream stream, DataFrame frame, Callback callback) {
		System.out.println("on data frame end " + StandardCharsets.UTF_8.decode(frame.getData()).toString());
	}
}

FuturePromise<Stream> streamPromise = new FuturePromise<>();
session.newStream(headersFrame, streamPromise, responseListener);
HTTP2Stream stream = (HTTP2Stream)streamPromise.get(5, TimeUnit.SECONDS);

Until this point, I have successfully sent a HTTP/2 CONNECT request to the proxy, get a response with 200 status code and no body, and openned a stream between my client and the proxy, and subsequently to google.com:443 server.

How do I actually proceed from here? Should I:

• open up a new stream to the proxy and send a GET / request (in the hope that I get a response from Google through the proxy), specifying the stream id of the stream I have already openned in the header? --> I tried this and the proxy sends back a 404 error probably because it does not service GET at its own / endpoint
• use the stream and send a new GET request header? --> If so, which function should I use? stream.data()? stream.headers()? stream.push()? What frame to send (is it supposed to be ByteBuffer.wrap(headersFrame.toString()))? And since stream is a bidirectional flow of frames, how can the client, say, print out the response from the proxy (and subsequently from the server)?

I might not get the current code right. Even my understanding about HTTP/2 Connect might not been entirely accurate. But thank you in advance for the help!

[joakime]

Is there a specific reason you are using the low level HTTP/2 Client? and not the higher level HttpClient with HTTP/2 support?

[gsusanto]

Because I am following the ConnectTunnelTest.java example. Is it possible to use the higher level one?

Also, I might want to do CONNECT UDP, if possible?

[sbordet]

@gsusanto the way it typically works is that the client issues a CONNECT to the proxy and the proxy opens a connection to the server.
In the specific case of HTTP/2, the stream used to send the CONNECT is kept open so that the client can send and receive DATA frames as long as the tunnel is open. Only when the tunnel is closed you will get a DATA frame with endStream=true.

Typically the client wants to send a request to the server (through the tunnel) and the request is typically encrypted with TLS, so you would have to do that -- but that's a lot of work when using the low level HTTP/2 APIs.

Fortunately we have done the work already and it's available out of the box in the high level HttpClient.
We have written a test with all the combinations, see ProxyWithDynamicTransportTest.testProxy().

Assuming that the proxy is non-secure and you want to send a request to google.com:443 (therefore encrypted), this is what should happen:

// client opens h2c stream e.g. 7 to the proxy, which opens a TCP connection to the server.
client -> HEADERS(7, CONNECT google.com:443) -> proxy -> TCP connection -> google.com:443

// client initiates TLS handshake and server completes the TLS handshake.
client -> DATA(7, body=(TLS handshake)) -> proxy -> TLS handshake -> google.com:443
client <- DATA(7, body=(TLS handshake reply)) <- proxy <- TLS handshake reply <- google.com:443

// client encrypts the GET request for the server using the crypto material obtained during the TLS handshake.
client -> DATA(7, body=TLS(body=HEADERS(9, GET /))) -> proxy -> TLS(body=HEADERS(9, GET /)) -> google.com:443

// server replies with 200.
client <- DATA(7, body=TLS(body=HEADERS(9, 200))) <- proxy <- TLS(body=HEADERS(9, 200)) <- google.com:443

Note how the client uses the stream established with the proxy and uses DATA frames to wrap anything that wants to send to the server. The proxy unwraps the DATA frame and sends the body to the server.
For the server, it looks like a normal HTTP/2 over TLS connection from a direct client.

At the time we implemented this, not many proxies were able to support HTTP/2; now things may have changed.
However, I would not be surprised if a client still uses HTTP/1.1 to establish the tunnel, and only after uses HTTP/2 over TLS.

I don't know what do you mean by "CONNECT UDP", can you clarify?

[github-actions]

This issue has been automatically marked as stale because it has been a
full year without activity. It will be closed if no further activity occurs.
Thank you for your contributions.

[sbordet]

Closing as answered.