x/net/http2: Unhandled Setting: [MAX_HEADER_LIST_SIZE = 1048896]

[bradfitz]

The Go http2 server sends MAX_HEADER_LIST_SIZE in its initial SETTINGS frame, but the Transport doesn't use it.

Fix.

/cc @bmizerany

[bradfitz]

This isn't worth it for Go 1.6.

Similar to #14048 (validate transmitted header field names & values), this would involve making the Transport fail earlier before transmitting anything, telling the user that the server is likely to reject it. (except in #14048's case, the rejection is due to a spec violation, not an "advisory" soft limit, as this is).

[bradfitz]

This is tricky because it requires us to back up the hpack encoder state if we find out we wrote too much in encodeHeaders. I guess encodeHeaders needs to ask hpack.Encoder return a snapshot its state, then we write everything into the bytes.Buffer and count the size as we go. Or we need to do first pass of everything, just counting size[1], then another pass doing the encoding.

[1] Remember that this isn't about the hpack-encoded bytes size, but the Sum(EachField(len(key)+len(value)+32)-ish thing:

This advisory setting informs a peer of the maximum size of header list that the sender is prepared to accept, in octets. The value is based on the uncompressed size of header fields, including the length of the name and value in octets plus an overhead of 32 octets for each header field.

I'm demoting this to a Go1.7Maybe.

[bradfitz]

(I'm going away for a month. If anybody else wants to fix this before Jun 23rd, feel free.)

[appleby]

Is this issue still up for grabs?

[bradfitz]

Yes.

[appleby]

Ok, I'll have a look. My idea is to update (*ClientConn) encodeHeaders to implement your second suggestion from above, i.e., do a first pass counting bytes, then a second pass to write the headers. Does that sound reasonable?

[appleby]

Above comment refers to client-side only. Looking into the server side, I see there is a corresponding encodeHeaders function, but it seems there are also a few calls to encKV sprinkled throughout write.go. These additional calls to encKV are responsible for only a few tens of bytes I would guess, and at first glance it looks like they could all be shoehorned into encodeHeaders anyway.

[gopherbot]

CL https://golang.org/cl/29242 mentions this issue.

[gopherbot]

CL https://golang.org/cl/29243 mentions this issue.

[appleby]

I've mailed a changelist covering only the client portion in order to get feedback before I start on the server side (assuming we even want to handle this on the server).

[bradfitz]

/cc @tombergan

[appleby]

The client-side fix has been committed. Thanks @tombergan for reviewing.

It will be at least a month before I have a chance to look into the server-side (laptop dead, parts on the slow boat from China), so If anyone else wants to tackle it before then, feel free.

[gopherbot]

Change https://golang.org/cl/71611 mentions this issue: net/http: update bundled http2