compress/gzip: cannot decompress files containing large gzip header comments

[tim-george]

Go version

go version go1.21.9 linux/amd64

Output of go env in your module/workspace:

...

What did you do?

Trying to decompress a gzipped file that contains a large gzip header comment.

The gzip Reader structure uses a fixed 512 byte buffer to process the header, but comments can be arbitrary length.

src/compress/gzip/gunzip.go

type Reader struct {
	Header       // valid after NewReader or Reader.Reset
	r            flate.Reader
	decompressor io.ReadCloser
	digest       uint32 // CRC-32, IEEE polynomial (section 8)
	size         uint32 // Uncompressed size (section 2.3.1)
	buf          [512]byte
	err          error
	multistream  bool
}

// readString reads a NUL-terminated string from z.r.
// It treats the bytes read as being encoded as ISO 8859-1 (Latin-1) and
// will output a string encoded using UTF-8.
// This method always updates z.digest with the data read.
func (z *Reader) readString() (string, error) {
	var err error
	needConv := false
	for i := 0; ; i++ {
		if i >= len(z.buf) {
			return "", ErrHeader
		}
		z.buf[i], err = z.r.ReadByte()
		if err != nil {
			return "", err
		}
		if z.buf[i] > 0x7f {
			needConv = true
		}
		if z.buf[i] == 0 {
			// Digest covers the NUL terminator.
			z.digest = crc32.Update(z.digest, crc32.IEEETable, z.buf[:i+1])

			// Strings are ISO 8859-1, Latin-1 ([RFC 1952](https://www.rfc-editor.org/rfc/rfc1952), section 2.3.1).
			if needConv {
				s := make([]rune, 0, i)
				for _, v := range z.buf[:i] {
					s = append(s, rune(v))
				}
				return string(s), nil
			}
			return string(z.buf[:i]), nil
		}
	}
}

For example openbsd packages (e.g. https://cdn.openbsd.org/pub/OpenBSD/7.5/packages/amd64/go-1.22.1.tgz ) stick a signature in the comment, which exceeds 512 bytes.

What did you see happen?

gzip.NewReader() fails with gzip: invalid header

What did you expect to see?

If it is not practical to handle arbitrary length comments it would be helpful to either have a more descriptive error or have a note about the limitation in the docs.

[seankhliao]

Duplicate of #14639