HTTP Parse Error: Invalid header value char

[gc]

• Node.js Version: 12.18.1
• OS: Debian 10
• Scope (install, code, runtime, meta, other?): Runtime
• Module (and version) (if relevant): node-fetch 2.6.0

Code:

const fetch = require('node-fetch');

fetch('https://secure.runescape.com/m=hiscore_oldschool/index_lite.ws?player=magnaboy');

Problem:

Fails with this error:

 FetchError: request to https://secure.runescape.com/m=hiscore_oldschool/index_lite.ws?player=magnaboy failed, reason: Parse Error: Invalid header value char
     at ClientRequest.<anonymous> (C:\project\node_modules\node-fetch\lib\index.js:1455:11)
     at ClientRequest.emit (events.js:315:20)
     at TLSSocket.socketOnData (_http_client.js:514:9)
     at TLSSocket.emit (events.js:315:20)
     at addChunk (_stream_readable.js:302:12)
     at readableAddChunk (_stream_readable.js:278:9)
     at TLSSocket.Readable.push (_stream_readable.js:217:10)
     at TLSWrap.onStreamRead (internal/stream_base_commons.js:186:23)

I tried to call this API with the exact same headers in other programming languages, in curl, wget and in all browsers - and it works in everything except nodejs, that I can find. Is it possible anyone can shed some light on this and if its possibly a problem in nodejs?

I've exhausted the depth of my ability to debug this error, and don't know how I should proceed. I reported it to the website owners but it doesnt seem practical or any priority at all for them to fix the root cause of whichever http header causes this.

[sapics]

When I use https module, it looks work. Could you try this?

const https = require('https');
const req = https.request('https://secure.runescape.com/m=hiscore_oldschool/index_lite.ws?player=magnaboy', (res) => {
  res.on('data', (chunk) => {
    console.log(`BODY: ${chunk}`);
  });
  res.on('end', () => {
    console.log('FINISH');
  });
});
req.on('error', (e) => {
  console.error(`ERROR： ${e.message}`);
});
req.end();

[sapics]

This issue relates to nodejs/node#27711.
Therefore, your code will run with node --insecure-http-parser.

[toxxicjtag]

Also having this issue this week from Jagex's API. Thanks for the temporary fix @sapics.

[gc]

@toxxicjtag if you want to help this get fixed, drop a reply to jagexallstar here so he knows its affecting many people. https://twitter.com/JagexAllstar/status/1281615931211427844

I'm closing this now because I found the reason in another issue: nodejs/http-parser#520 (comment)

[toxxicjtag]

I just don't have a twitter, hope he does look into it, i have looked into alternative options like ge-tracker for their icon files or downloading them locally at this point.

[addaleax]

Maybe node-fetch could be modified to pass through Node.js HTTP options like the insecureHTTPParser one?

[gc]

They fixed this problem now btw, the cause was a bad header - more info here: nodejs/http-parser#520 (comment)

[ggorlen]

This occurred for me with native fetch in Node v18.4.0. Adding the --insecure-http-parser flag wasn't sufficient, nor was using http or axios alone. But combining --insecure-http-parser and either axios or http worked.

This is on a different site than OP. OP's site works for me at the current time with or without flags and with any request library. In case it's relevant, the response has non-ASCII characters in it.