Node 18.18.2 : unable to override the Host header in fetch

[fieteboerner]

Version

v18.18.2

Platform

Linux hostname 5.15.131-1-MANJARO #1 SMP PREEMPT Thu Sep 7 11:05:18 UTC 2023 x86_64 GNU/Linux

Subsystem

No response

What steps will reproduce the bug?

fetch("https://postman-echo.com/get", {
    headers: {
        Test: 1234,
        Host: "foo.postman-echo.com",
    },
})
    .then((r) => {
        return r.text();
    })
    .then((text) => console.log(text))
    .catch((e) => {
        console.log(e);
    });

run this code

The test works successfullty with v18.18.0 & v18.18.0

How often does it reproduce? Is there a required condition?

ive run the tests numerous times
the only condition I know is to run with v18.18.2

What is the expected behavior? Why is that the expected behavior?

• should send a request topostman-echo.com but should override the Host to foo.postman-echo.com Header
• the reason: this is how a http client should behave
• see vor example: curl -H 'Host: foobar.com' https://postman-echo.com/get

What do you see instead?

• the host header has not been overidden by the headers config
• the initial set host in the request url is taken

Additional information

• this was also tested in several docker images
  • node:18-alpine (contains bug)
  • node:18.18.2-alpine (contains bug)
  • node:18.18.2-alpine (no bug)

[panva]

Host is a forbidden header in the Fetch Standard. The Friday October 13 2023 Security Releases disallowed its (and other forbidden headers') use. This also aligned the Host header behaviour with other Fetch implementations such as Deno, Bun, Workerd, and others.

• https://github.com/nodejs/node/releases/tag/v18.18.2
• deps: update undici to v5.26.3 #50153
• https://cve.mitre.org/cgi-bin/cvename.cgi?name=CVE-2023-45143
• disallow setting host header in fetch undici#2322

[paulrutter]

@panva @mcollina as i also mentioned in nodejs/undici#2318 (comment), there are there are valid usecases for setting the host header on a request in Node.js environment.
For example, if you have a multi-tenant application with specific hostnames, you want to able to supply the IP address + host header, not just the hostname in the url.

We are running into this issue when upgrading from Node 18.17.1 to Node 20.8.1, as the host header doesn't seem to be passed along to the application anymore (thus, breaking the integration).

Is this really the desired behavior?
It's not a backwards compatible change, and i think the RFC should not be followed when not in a browser environment.
I also don't read in the RFC that the host should not be overridable, just that the agent should generate one (if not available via the custom headers).

Is there a workaround to still allow setting the host header via fetch? Or are we back to using third-party HTTP clients for this?
Java has a workaround by supplying a property: -Djdk.httpclient.allowRestrictedHeaders=host.

[mcollina]

Or are we back to using third-party HTTP clients for this?

TL;DR, you'll have a much better production experience anyway. fetch() is significantly slower than undici.request() and almost on par with http.request(). Your use case seem specific enough that a custom client for Node.js is recommended.

[paulrutter]

Thanks for your response @mcollina, but this is not really what i would have expected.
Isn't fetch supposed to be the HTTP client to use? I was under the impression that fetch would be the preferred HTTP client to use (after years of using request, got, axios and node-fetch). It would be a shame to now go back to a userland library again.

I also think that using a host header is not that specific that it warrants a custom HTTP client; lots of applications have custom hostnames. Is something like the java solution a path forward? Java has a workaround by supplying a property: -Djdk.httpclient.allowRestrictedHeaders=host.
Or Deno's solution to the same issue:
denoland/deno#17968 (comment).

Another question; isn't undici what drives the fetch API under the hood? How can it then be slower than using undici directly?

[paulrutter]

Decided to use undici.request instead of fetch. Then we can still leverage the same HTTP client as fetch uses under the hood, but get the freedom we need.

See nodejs/undici#2369

[pano9000]

have run into the same issue here, where I need to work with an API, that required the host header to be set for the Authentication to work, e.g.:
https://kiwihr.com/api/docs/authentication.doc.html

this used to work fine with previous versions, but now since I cannot overide the host header anymore, this fails with the built-in fetch, which is a bit of a bummer.

would be nice to have a way to do this with the built-in fetch in the future again

[paulrutter]

Following nodejs/undici#2369 (comment), using fetch is not advocated for backend development.

Undici.request is the advised approach.

[matthias-hertel-db]

using fetch is not advocated for backend development.

fetch API and behavior was defined as a API in end-user browser level JavaScript applications. For this use-case the host header must fit to the domain name used for DNS based IP retrieval. That closes some security concerns in combination with Script injection etc.

When you have to implement backend components like content inspection & security scanning that you face the situation that you like to pass the host header "as is" to a backend server(having an internal domain name).

We don't need 2 libraries just 2 different behaviors.

[gongfan99]

I have the similar issue. My backend server#1 needs to call the backend server#2 by IP/hostname since the server#2 is a multi-tenant. Not allowing Host header makes fetch() not suitable for the task.

Personally I am Ok to use undici.request() instead since it obviously allows the Host header.

However, server#1 is for Nextjs which patches fetch() to facilitate deduplication and request-side caching. If I use undici.request() in server#1, then I will lose deduplication and caching.

For de-dup, it is Ok since I do not use it that much.

For caching, it is also probably Ok because Nextjs' handling of caching does not make sense anyway: whether or not caching should not be decided by the request-side; the request-side should simply respect the Cache-Control header sent by the response-side.

So the question is, by default, does undici.request() respect the Cache-Control header just like a real browser? Or do I need to config it to make it behave that way? I tried to dive into the undici code but it is too deep for me :-)

[Netail]

Facing the same issue for our local reverse proxy, since we switch from node-fetch to native fetch :(

E.g. in /etc/hosts you direct 2 custom domains to 127.0.0.1

127.0.0.1 local.google.nl local.google.be local.google.uk

All go through the same reverse proxy client, but based on the Host header we can determine on which domain the user is based.

@paulrutter it's weird to force backend people to use a different fetch package, just because of the Host override.
NodeJS is also largely used for backend... Would it be possible to remove this override @mcollina ?

[paulrutter]

See nodejs/undici#2369 (comment) for more details.

[Netail]

See nodejs/undici#2369 (comment) for more details.

Aye I've read it, but it feels weird to add a library just because of the different Host header library. What about a check if it's in the browser or not? Only override the header if it's browser-side. And it is pretty confusing to see the Host header not working...

[paulrutter]

I agree with you on that, but got the same response several times, so decided to move to undici (which works fine, and is still the library behind fetch).