net/http: Get request hangs until timeout, but responds immediately over curl

[nanoandrew4]

What version of Go are you using (go version)?

$ go version go1.19.2 linux/amd64

Does this issue reproduce with the latest release?

Yes, I reproduced it on 1.19.2, but also on 1.20.7 and 1.20

What operating system and processor architecture are you using (go env)?

go env Output

$ go env
GO111MODULE=""
GOARCH="amd64"
GOBIN=""
GOCACHE="/home/andres/.cache/go-build"
GOENV="/home/andres/.config/go/env"
GOEXE=""
GOEXPERIMENT=""
GOFLAGS=""
GOHOSTARCH="amd64"
GOHOSTOS="linux"
GOINSECURE=""
GOMODCACHE="/home/andres/go/pkg/mod"
GONOPROXY="git.pro.sisdig.dgrp.io"
GONOSUMDB="git.pro.sisdig.dgrp.io"
GOOS="linux"
GOPATH="/home/andres/go"
GOPRIVATE="git.pro.sisdig.dgrp.io"
GOPROXY="https://proxy.golang.org,direct"
GOROOT="/opt/go/go-latest"
GOSUMDB="sum.golang.org"
GOTMPDIR=""
GOTOOLDIR="/opt/go/go-latest/pkg/tool/linux_amd64"
GOVCS=""
GOVERSION="go1.19.2"
GCCGO="gccgo"
GOAMD64="v1"
AR="ar"
CC="gcc"
CXX="g++"
CGO_ENABLED="1"
GOMOD="/dev/null"
GOWORK=""
CGO_CFLAGS="-g -O2"
CGO_CPPFLAGS=""
CGO_CXXFLAGS="-g -O2"
CGO_FFLAGS="-g -O2"
CGO_LDFLAGS="-g -O2"
PKG_CONFIG="pkg-config"
GOGCCFLAGS="-fPIC -m64 -pthread -Wl,--no-gc-sections -fmessage-length=0 -fdebug-prefix-map=/tmp/go-build3877364693=/tmp/go-build -gno-record-gcc-switches"

What did you do?

package main

import (
	"fmt"
	"io"
	"net/http"
	"time"
)

func main() {
	client := &http.Client{
		Timeout: 5 * time.Second,
	}
	resp, err := client.Get("https://www2.agenciatributaria.gob.es/static_files/common/internet/dep/aplicaciones/es/aeat/burt/jdit/ws/VNifV2.wsdl")
	if err != nil {
		panic(err)
	}

	body, err := io.ReadAll(resp.Body)
	if err != nil {
		panic(err)
	}
	fmt.Println(string(body))
}

I have not included the go.dev link because when I run the program from there I get a no route to host error, which is not what I am seeing on my machine. I am able to fetch the resource through my browser fine, and curl also returns it correctly.

What did you expect to see?

The response body containing the XML that the link returns via curl or through the browser. I also tested making a GET request from a test java program and it returned the XML correctly.

What did you see instead?

2023/08/07 21:33:32 http2: Transport failed to get client conn for www2.agenciatributaria.gob.es:443: http2: no cached connection was available
panic: Get "https://www2.agenciatributaria.gob.es/static_files/common/internet/dep/aplicaciones/es/aeat/burt/jdit/ws/VNifV2.wsdl": context deadline exceeded (Client.Timeout exceeded while awaiting headers)

We see this error both when opening the connection from our own machine, as well as from our k8s cluster hosted on GCP. I have searched around and seen that six years ago, a similar issue to mine was reported, #21301, which was said to be a duplicate of #20979. However, none of the listed solutions seem to resolve my issue.

When requesting the resource through curl, I get the following response:

> curl -v https://www2.agenciatributaria.gob.es/static_files/common/internet/dep/aplicaciones/es/aeat/burt/jdit/ws/VNifV2.wsdl
*   Trying 195.77.198.18:443...
* Connected to www2.agenciatributaria.gob.es (195.77.198.18) port 443 (#0)
* ALPN, offering h2
* ALPN, offering http/1.1
*  CAfile: /etc/ssl/certs/ca-certificates.crt
*  CApath: /etc/ssl/certs
* TLSv1.0 (OUT), TLS header, Certificate Status (22):
* TLSv1.3 (OUT), TLS handshake, Client hello (1):
* TLSv1.2 (IN), TLS header, Certificate Status (22):
* TLSv1.3 (IN), TLS handshake, Server hello (2):
* TLSv1.2 (IN), TLS header, Finished (20):
* TLSv1.2 (IN), TLS header, Supplemental data (23):
* TLSv1.3 (IN), TLS handshake, Encrypted Extensions (8):
* TLSv1.2 (IN), TLS header, Supplemental data (23):
* TLSv1.3 (IN), TLS handshake, Certificate (11):
* TLSv1.2 (IN), TLS header, Supplemental data (23):
* TLSv1.3 (IN), TLS handshake, CERT verify (15):
* TLSv1.2 (IN), TLS header, Supplemental data (23):
* TLSv1.3 (IN), TLS handshake, Finished (20):
* TLSv1.2 (OUT), TLS header, Finished (20):
* TLSv1.3 (OUT), TLS change cipher, Change cipher spec (1):
* TLSv1.2 (OUT), TLS header, Supplemental data (23):
* TLSv1.3 (OUT), TLS handshake, Finished (20):
* SSL connection using TLSv1.3 / TLS_AES_128_GCM_SHA256
* ALPN, server did not agree to a protocol
* Server certificate:
*  subject: C=ES; L=Madrid; jurisdictionC=ES; O=Agencia Estatal de Administracion Tributaria; businessCategory=Government Entity; serialNumber=1990-12-27; CN=sede.agenciatributaria.gob.es
*  start date: Nov  3 09:20:58 2022 GMT
*  expire date: Nov 22 09:20:58 2023 GMT
*  subjectAltName: host "www2.agenciatributaria.gob.es" matched cert's "www2.agenciatributaria.gob.es"
*  issuer: C=ES; O=Entrust EU, S.L.; organizationIdentifier=VATES-B81188047; CN=Entrust Certification Authority - ES QWAC2
*  SSL certificate verify ok.
* TLSv1.2 (OUT), TLS header, Supplemental data (23):
> GET /static_files/common/internet/dep/aplicaciones/es/aeat/burt/jdit/ws/VNifV2.wsdl HTTP/1.1
> Host: www2.agenciatributaria.gob.es
> User-Agent: curl/7.81.0
> Accept: */*
> 
* TLSv1.2 (IN), TLS header, Supplemental data (23):
* Mark bundle as not supporting multiuse
< HTTP/1.1 200 OK
< date: Mon, 07 Aug 2023 18:58:39 GMT
< server: Apache
< x-frame-options: SAMEORIGIN
< last-modified: Thu, 22 Jun 2017 12:30:16 GMT
< accept-ranges: bytes
< content-length: 3042
< x-xss-protection: 1; mode=block
< vary: origin
< x-site: Sede
< referrer-policy: strict-origin-when-cross-origin
< 
<?xml version="1.0" encoding="UTF-8"?>

<wsdl:definitions 
xmlns:VNifV2="http://www2.agenciatributaria.gob.es/static_files/common/internet/dep/aplicaciones/es/aeat/burt/jdit/ws/VNifV2.wsdl" 
xmlns:VNifV2Ent="http://www2.agenciatributaria.gob.es/static_files/common/internet/dep/aplicaciones/es/aeat/burt/jdit/ws/VNifV2Ent.xsd" 
xmlns:VNifV2Sal="http://www2.agenciatributaria.gob.es/static_files/common/internet/dep/aplicaciones/es/aeat/burt/jdit/ws/VNifV2Sal.xsd" 
xmlns:wsdl="http://schemas.xmlsoap.org/wsdl/" 
xmlns:wsdlsoap="http://schemas.xmlsoap.org/wsdl/soap/" 
xmlns:xsd="http://www.w3.org/2001/XMLSchema" 
targetNamespace="http://www2.agenciatributaria.gob.es/static_files/common/internet/dep/aplicaciones/es/aeat/burt/jdit/ws/VNifV2.wsdl">

	<wsdl:types>
		<schema elementFormDefault="qualified"  
		targetNamespace="http://www2.agenciatributaria.gob.es/static_files/common/internet/dep/aplicaciones/es/aeat/burt/jdit/ws/VNifV2.wsdl"
		xmlns:VNifV2Ent="http://www2.agenciatributaria.gob.es/static_files/common/internet/dep/aplicaciones/es/aeat/burt/jdit/ws/VNifV2Ent.xsd" 
		xmlns:VNifV2Sal="http://www2.agenciatributaria.gob.es/static_files/common/internet/dep/aplicaciones/es/aeat/burt/jdit/ws/VNifV2Sal.xsd" 
		xmlns="http://www.w3.org/2001/XMLSchema">
			<xsd:import namespace="http://www2.agenciatributaria.gob.es/static_files/common/internet/dep/aplicaciones/es/aeat/burt/jdit/ws/VNifV2Ent.xsd" schemaLocation="VNifV2Ent.xsd"/>
			<xsd:import namespace="http://www2.agenciatributaria.gob.es/static_files/common/internet/dep/aplicaciones/es/aeat/burt/jdit/ws/VNifV2Sal.xsd" schemaLocation="VNifV2Sal.xsd"/>
		</schema>
	</wsdl:types>
	
	<wsdl:message name="Entrada">
		<wsdl:part name="VNifV2Ent" element="VNifV2Ent:VNifV2Ent"/>
	</wsdl:message>
	<wsdl:message name="Salida">
		<wsdl:part name="VNifV2Sal" element="VNifV2Sal:VNifV2Sal"/>
	</wsdl:message>

	<wsdl:portType name="VNifV2">
		<wsdl:operation name="VNifV2">
			<wsdl:input name="Entrada" message="VNifV2:Entrada"/>
			<wsdl:output name="Salida" message="VNifV2:Salida"/>
		</wsdl:operation>
	</wsdl:portType>
	
	<wsdl:binding name="VNifV2SoapBinding" type="VNifV2:VNifV2">
		<wsdlsoap:binding style="document" transport="http://schemas.xmlsoap.org/soap/http"/>
		<wsdl:operation name="VNifV2">
			<wsdlsoap:operation soapAction=""/>
			<wsdl:input name="Entrada">
				<wsdlsoap:body use="literal"/>
			</wsdl:input>
			<wsdl:output name="Salida">
				<wsdlsoap:body use="literal"/>
			</wsdl:output>
		</wsdl:operation>
* TLSv1.2 (IN), TLS header, Supplemental data (23):
	</wsdl:binding>

	<wsdl:service name="VNifV2Service">
		<!--Puerto para conexiones desde internet en PRO -->
		<wsdl:port name="VNifPort1" binding="VNifV2:VNifV2SoapBinding">
			<wsdlsoap:address location="https://www1.agenciatributaria.gob.es/wlpl/BURT-JDIT/ws/VNifV2SOAP"/>
			<wsdlsoap:address location="https://www10.agenciatributaria.gob.es/wlpl/BURT-JDIT/ws/VNifV2SOAP"/>
		</wsdl:port>
	</wsdl:service>
	
* Connection #0 to host www2.agenciatributaria.gob.es left intact
</wsdl:definitions>

[ianlancetaylor]

CC @neild @bradfitz

[ianlancetaylor]

Setting 1.22 milestone as it may be a bug that should be fixed.

[seankhliao]

This is the remote end not liking the user agent.