net/http: Client following redirects won't preserve Host header

[timonwong]

Please answer these questions before submitting your issue. Thanks!

What version of Go are you using (go version)?

go version go1.9.1 darwin/amd64

Does this issue reproduce with the latest release?

Yes

What operating system and processor architecture are you using (go env)?

GOARCH="amd64"
GOBIN=""
GOEXE=""
GOHOSTARCH="amd64"
GOHOSTOS="darwin"
GOOS="darwin"
GOPATH="/Users/timonwong/go"
GORACE=""
GOROOT="/usr/local/Cellar/go/1.9.1/libexec"
GOTOOLDIR="/usr/local/Cellar/go/1.9.1/libexec/pkg/tool/darwin_amd64"
GCCGO="gccgo"
CC="clang"
GOGCCFLAGS="-fPIC -m64 -pthread -fno-caret-diagnostics -Qunused-arguments -fmessage-length=0 -fdebug-prefix-map=/var/folders/kf/f_mg9bgj05v7dwjr_gm95g_w0000gn/T/go-build556164700=/tmp/go-build -gno-record-gcc-switches -fno-common"
CXX="clang++"
CGO_ENABLED="1"
CGO_CFLAGS="-g -O2"
CGO_CPPFLAGS=""
CGO_CXXFLAGS="-g -O2"
CGO_FFLAGS="-g -O2"
CGO_LDFLAGS="-g -O2"
PKG_CONFIG="pkg-config"

What did you do?

It seems that #8027 was not resolved in #4800. In #4800, custom headers are copied, however, Host field in http.Request is not preserved even if the Location header is just a relative path, see following Go playground link for example:

https://play.golang.org/p/CG45o1eBz8

What did you expect to see?

>> SUCCESS
>> SUCCESS

What did you see instead?

>> ERROR: Got unexpected response code: 403
>> SUCCESS

[ianlancetaylor]

CC @tombergan

[tombergan]

Definitely a bug. Thanks for the repro.

[tombergan]

Implementation notes:

As mentioned in Brad's comment here, the HTTP RFCs do not specify exactly which headers should be copied. The HTML Fetch Spec does specify which headers are carried through the fetch, but Fetch does not allow setting a custom Host header, so this issue cannot apply. This also implies that we cannot look to browsers for guidance because they cannot run into this issue.

One approach is to copy the original Host header through all redirect requests. This is apparently what Python does, however, there are good arguments that Python's behavior is wrong. What @timonwong proposes is what curl does: copy the Host header only if the location is relative. This seems reasonable to me.

It's also worth mentioning that setting req.Host different from req.URL.Host is a niche use case that should basically never be done unless the caller really knows what they are doing.

[gopherbot]

Change https://golang.org/cl/70792 mentions this issue: net/http: preserve Host header following a relative redirect