http: reject control characters in http.request() #8923

bnoordhuis · 2016-10-04T10:28:19Z

First commit:

Unsanitized paths containing line feed characters can be used for header injection and request splitting so reject them with an exception.

Second commit:

The first commit is the result of nodejs-security@ discussion but I had a change of heart. I can't see any reasonable use case for allowing control characters (characters <= 31) but I can think of several scenarios where they can be used to exploit software bugs so let's ban them altogether.

There is a a potential compatibility issue in that tabs in paths have been observed in the wild, but, to the best of my knowledge, only in requests from buggy HTTP clients. Here too I don't see a reason to allow them in requests that node.js initiates.

@nodejs/http @nodejs/security

indutny

LGTM

evanlucas

LGTM

jbergstroem · 2016-10-04T13:51:01Z

CI: https://ci.nodejs.org/job/node-test-commit/5435/

tarqd · 2016-10-05T16:35:14Z

lib/_http_client.js

@@ -43,13 +43,14 @@ function ClientRequest(options, cb) {
  if (self.agent && self.agent.protocol)
    expectedProtocol = self.agent.protocol;

-  if (options.path && / /.test(options.path)) {
+  if (options.path && /[\n\r ]/.test(options.path)) {


Does it make sense to reject tabs here as well? The logic being that when you send an unescaped tab to servers like Apache the server will interpret the request as the tab splitting the path and the HTTP version. Instead of failing silently and allowing malformed requests through, the developer will get an error thrown that clearly says "You have to escape this"

@ilsken second commit includes the tabs.

@lpinca oh sorry I missed that

silverwind

LGTM with suggestion

silverwind · 2016-10-05T17:00:07Z

lib/_http_client.js

-    // why it only scans for spaces because those are guaranteed to create
-    // an invalid request.
+    // well, and b) possibly too restrictive for real-world usage.
+    // Restrict the filter to control characters and spaces.
    throw new TypeError('Request path contains unescaped characters');


RangeError?

Ack, but for a different PR because people may want to be careful and consider that semver-major?

Hmm, I think this would be a unecessary breaking change if we consider error types as API. So, LGTM as-is.

addaleax

LGTM

lpinca

LGTM

lpinca · 2016-10-06T08:12:46Z

test/parallel/test-http-client-unescaped-path.js

-}, /contains unescaped characters/);
+function* bad() {
+  for (let i = 0; i <= 32; i += 1)
+    yield 'bad' + String.fromCharCode(i) + 'path';


Nit: it doesn't add any real value but what about using a template literal?

I initially had that but I felt it was less readable than concatenation. Consider:

yield 'bad' + String.fromCharCode(i) + 'path';

Vs.

yield `bad${String.fromCharCode(i)}path`;

Yeah no big deal.

jasnell

LGTM

jasnell · 2016-10-06T18:32:23Z

Marking semver-major

jasnell · 2016-10-06T18:33:01Z

@nodejs/ctc ... I'd like to include this in v7 assuming it lands in master. Need ctc approval tho as a semver-major.

addaleax · 2016-10-06T18:37:33Z

Mhhh, I would like to hear other opinions, but this seems like more of a security bug fix than an actual semver-major change, and I Can’t Think Of Any Situation™ in which somebody would rely on the current behaviour?

jasnell · 2016-10-06T18:40:13Z

It can be viewed in both ways. There aren't any good reasons why someone would rely on the current behavior but there are examples of software using tab and space where they do not belong. The other control characters are much more rare and generally indicate buggy or unsafe implementations. I'm happy with this landing as less than a semver-major as long as the change is documented clearly in the release notes of whatever release it lands in.

addaleax · 2016-10-06T18:45:19Z

@jasnell In that case, how about considering the first commit here as a semver-patch one and the second as semver-major?

and space where they do not belong

(That’s forbidden already.)

jasnell · 2016-10-06T19:01:34Z

I'd say if we're going to go that route, let's just treat them both as semver-patch and document the change in the release notes.

Trott · 2016-10-06T20:03:32Z

Maybe we can revisit this whole "semver majors need CTC approval to land in v7" thing. Did that even come out of the CTC or was that an LTS WG decision?

jasnell · 2016-10-06T20:10:10Z

@Trott ... let's talk about that in the v7.0.0-proposal PR: #8503

bnoordhuis · 2016-10-07T10:35:42Z

Mhhh, I would like to hear other opinions, but this seems like more of a security bug fix than an actual semver-major change

Correct. I consider this a security/correctness fix and as such exempt from our regular semver policies. I plan on back-porting it to the release branches once it lands in master. I'll remove the semver-major label.

bnoordhuis · 2016-10-11T18:45:04Z

New CI: https://ci.nodejs.org/job/node-test-pull-request/4465/

Unsanitized paths containing line feed characters can be used for header injection and request splitting so reject them with an exception. There seems to be no reasonable use case for allowing control characters (characters <= 31) while there are several scenarios where they can be used to exploit software bugs so reject control characters altogether. PR-URL: nodejs#8923 Reviewed-By: Anna Henningsen <anna@addaleax.net> Reviewed-By: Evan Lucas <evanlucas@me.com> Reviewed-By: Fedor Indutny <fedor@indutny.com> Reviewed-By: James M Snell <jasnell@gmail.com> Reviewed-By: Luigi Pinca <luigipinca@gmail.com> Reviewed-By: not-an-aardvark <not-an-aardvark@users.noreply.github.com>

Unsanitized paths containing line feed characters can be used for header injection and request splitting so reject them with an exception. There seems to be no reasonable use case for allowing control characters (characters <= 31) while there are several scenarios where they can be used to exploit software bugs so reject control characters altogether. PR-URL: #8923 Reviewed-By: Anna Henningsen <anna@addaleax.net> Reviewed-By: Evan Lucas <evanlucas@me.com> Reviewed-By: Fedor Indutny <fedor@indutny.com> Reviewed-By: James M Snell <jasnell@gmail.com> Reviewed-By: Luigi Pinca <luigipinca@gmail.com> Reviewed-By: not-an-aardvark <not-an-aardvark@users.noreply.github.com>

MylesBorins · 2016-11-11T19:28:50Z

@bnoordhuis @jasnell I've added the v6.x watch label to this. Is this safe to land now or is it something we should let live on v7 a bit longer? Should this come in a patch release or a minor release?

MylesBorins · 2016-12-21T18:13:05Z

ping @bnoordhuis

MylesBorins · 2016-12-21T19:30:10Z

this should land with dbf4bf2

bnoordhuis · 2016-12-22T07:23:28Z

@thealphanerd See #8923 (comment), this can land in a patch release.

Unsanitized paths containing line feed characters can be used for header injection and request splitting so reject them with an exception. There seems to be no reasonable use case for allowing control characters (characters <= 31) while there are several scenarios where they can be used to exploit software bugs so reject control characters altogether. PR-URL: #8923 Reviewed-By: Anna Henningsen <anna@addaleax.net> Reviewed-By: Evan Lucas <evanlucas@me.com> Reviewed-By: Fedor Indutny <fedor@indutny.com> Reviewed-By: James M Snell <jasnell@gmail.com> Reviewed-By: Luigi Pinca <luigipinca@gmail.com> Reviewed-By: not-an-aardvark <not-an-aardvark@users.noreply.github.com>

Notable changes * performance: The performance of several APIs has been improved. - `Buffer.compare()` is up to 35% faster on average. (Brian White) #10927 - `buffer.toJSON()` is up to 2859% faster on average. (Brian White) #10895 - `fs.*statSync()` functions are now up to 9.3% faster on average. (Brian White) #11522 - `os.loadavg` is up to 151% faster. (Brian White) #11516 - `process.memoryUsage()` is up to 34% faster. (Brian White) #11497 - `querystring.unescape()` for `Buffer`s is 15% faster on average. (Brian White) #10837 - `querystring.stringify()` is up to 7.8% faster on average. (Brian White) #10852 - `querystring.parse()` is up to 21% faster on average. (Brian White) #10874 * IPC: - Batched writes have been enabled for process IPC on platforms that support Unix Domain Sockets. (Alexey Orlenko) #10677 - Performance gains may be up to 40% for some workloads. * child_process: - `spawnSync` now returns a null `status` when child is terminated by a signal. (cjihrig) #11288 - This fixes the behavior to act like `spawn()` does. * http: - Control characters are now always rejected when using `http.request()`. (Ben Noordhuis) #8923 - Debug messages have been added for cases when headers contain invalid values. (Evan Lucas) #9195 * node: - Heap statistics now support values larger than 4GB. (Ben Noordhuis) #10186 * timers: - Timer callbacks now always maintain order when interacting with domain error handling. (John Barboza) #10522 PR-URL: #11759

Notable Changes: * buffer: - The performance of `.toJSON()` is now up to 2859% faster on average (Brian White) #10895 * IPC: - Batched writes have been enabled for process IPC on platforms that support Unix Domain Sockets. (Alexey Orlenko) #10677 - Performance gains may be up to 40% for some workloads. * http: - Control characters are now always rejected when using `http.request()`. (Ben Noordhuis) #8923 * node: - Heap statistics now support values larger than 4GB. (Ben Noordhuis) #10186

Notable Changes: * buffer: - The performance of `.toJSON()` is now up to 2859% faster on average (Brian White) #10895 * IPC: - Batched writes have been enabled for process IPC on platforms that support Unix Domain Sockets. (Alexey Orlenko) #10677 - Performance gains may be up to 40% for some workloads. * http: - Control characters are now always rejected when using `http.request()`. (Ben Noordhuis) #8923 * node: - Heap statistics now support values larger than 4GB. (Ben Noordhuis) #10186 PR-URL: #11760

Notable changes * performance: The performance of several APIs has been improved. - `Buffer.compare()` is up to 35% faster on average. (Brian White) #10927 - `buffer.toJSON()` is up to 2859% faster on average. (Brian White) #10895 - `fs.*statSync()` functions are now up to 9.3% faster on average. (Brian White) #11522 - `os.loadavg` is up to 151% faster. (Brian White) #11516 - `process.memoryUsage()` is up to 34% faster. (Brian White) #11497 - `querystring.unescape()` for `Buffer`s is 15% faster on average. (Brian White) #10837 - `querystring.stringify()` is up to 7.8% faster on average. (Brian White) #10852 - `querystring.parse()` is up to 21% faster on average. (Brian White) #10874 * IPC: - Batched writes have been enabled for process IPC on platforms that support Unix Domain Sockets. (Alexey Orlenko) #10677 - Performance gains may be up to 40% for some workloads. * child_process: - `spawnSync` now returns a null `status` when child is terminated by a signal. (cjihrig) #11288 - This fixes the behavior to act like `spawn()` does. * http: - Control characters are now always rejected when using `http.request()`. (Ben Noordhuis) #8923 - Debug messages have been added for cases when headers contain invalid values. (Evan Lucas) #9195 * node: - Heap statistics now support values larger than 4GB. (Ben Noordhuis) #10186 * timers: - Timer callbacks now always maintain order when interacting with domain error handling. (John Barboza) #10522 PR-URL: #11759

Notable Changes: * buffer: - The performance of `.toJSON()` is now up to 2859% faster on average (Brian White) nodejs/node#10895 * IPC: - Batched writes have been enabled for process IPC on platforms that support Unix Domain Sockets. (Alexey Orlenko) nodejs/node#10677 - Performance gains may be up to 40% for some workloads. * http: - Control characters are now always rejected when using `http.request()`. (Ben Noordhuis) nodejs/node#8923 * node: - Heap statistics now support values larger than 4GB. (Ben Noordhuis) nodejs/node#10186 Signed-off-by: Ilkka Myller <ilkka.myller@nodefield.com>

Notable changes * performance: The performance of several APIs has been improved. - `Buffer.compare()` is up to 35% faster on average. (Brian White) nodejs/node#10927 - `buffer.toJSON()` is up to 2859% faster on average. (Brian White) nodejs/node#10895 - `fs.*statSync()` functions are now up to 9.3% faster on average. (Brian White) nodejs/node#11522 - `os.loadavg` is up to 151% faster. (Brian White) nodejs/node#11516 - `process.memoryUsage()` is up to 34% faster. (Brian White) nodejs/node#11497 - `querystring.unescape()` for `Buffer`s is 15% faster on average. (Brian White) nodejs/node#10837 - `querystring.stringify()` is up to 7.8% faster on average. (Brian White) nodejs/node#10852 - `querystring.parse()` is up to 21% faster on average. (Brian White) nodejs/node#10874 * IPC: - Batched writes have been enabled for process IPC on platforms that support Unix Domain Sockets. (Alexey Orlenko) nodejs/node#10677 - Performance gains may be up to 40% for some workloads. * child_process: - `spawnSync` now returns a null `status` when child is terminated by a signal. (cjihrig) nodejs/node#11288 - This fixes the behavior to act like `spawn()` does. * http: - Control characters are now always rejected when using `http.request()`. (Ben Noordhuis) nodejs/node#8923 - Debug messages have been added for cases when headers contain invalid values. (Evan Lucas) nodejs/node#9195 * node: - Heap statistics now support values larger than 4GB. (Ben Noordhuis) nodejs/node#10186 * timers: - Timer callbacks now always maintain order when interacting with domain error handling. (John Barboza) nodejs/node#10522 PR-URL: nodejs/node#11759 Signed-off-by: Ilkka Myller <ilkka.myller@nodefield.com>

Notable Changes: * buffer: - The performance of `.toJSON()` is now up to 2859% faster on average (Brian White) nodejs/node#10895 * IPC: - Batched writes have been enabled for process IPC on platforms that support Unix Domain Sockets. (Alexey Orlenko) nodejs/node#10677 - Performance gains may be up to 40% for some workloads. * http: - Control characters are now always rejected when using `http.request()`. (Ben Noordhuis) nodejs/node#8923 * node: - Heap statistics now support values larger than 4GB. (Ben Noordhuis) nodejs/node#10186 Signed-off-by: Ilkka Myller <ilkka.myller@nodefield.com>

Notable changes * performance: The performance of several APIs has been improved. - `Buffer.compare()` is up to 35% faster on average. (Brian White) nodejs/node#10927 - `buffer.toJSON()` is up to 2859% faster on average. (Brian White) nodejs/node#10895 - `fs.*statSync()` functions are now up to 9.3% faster on average. (Brian White) nodejs/node#11522 - `os.loadavg` is up to 151% faster. (Brian White) nodejs/node#11516 - `process.memoryUsage()` is up to 34% faster. (Brian White) nodejs/node#11497 - `querystring.unescape()` for `Buffer`s is 15% faster on average. (Brian White) nodejs/node#10837 - `querystring.stringify()` is up to 7.8% faster on average. (Brian White) nodejs/node#10852 - `querystring.parse()` is up to 21% faster on average. (Brian White) nodejs/node#10874 * IPC: - Batched writes have been enabled for process IPC on platforms that support Unix Domain Sockets. (Alexey Orlenko) nodejs/node#10677 - Performance gains may be up to 40% for some workloads. * child_process: - `spawnSync` now returns a null `status` when child is terminated by a signal. (cjihrig) nodejs/node#11288 - This fixes the behavior to act like `spawn()` does. * http: - Control characters are now always rejected when using `http.request()`. (Ben Noordhuis) nodejs/node#8923 - Debug messages have been added for cases when headers contain invalid values. (Evan Lucas) nodejs/node#9195 * node: - Heap statistics now support values larger than 4GB. (Ben Noordhuis) nodejs/node#10186 * timers: - Timer callbacks now always maintain order when interacting with domain error handling. (John Barboza) nodejs/node#10522 PR-URL: nodejs/node#11759 Signed-off-by: Ilkka Myller <ilkka.myller@nodefield.com>

bnoordhuis added the http Issues or PRs related to the http subsystem. label Oct 4, 2016

nodejs-github-bot added the http Issues or PRs related to the http subsystem. label Oct 4, 2016

indutny approved these changes Oct 4, 2016

View reviewed changes

evanlucas approved these changes Oct 4, 2016

View reviewed changes

tarqd reviewed Oct 5, 2016

View reviewed changes

silverwind approved these changes Oct 5, 2016

View reviewed changes

addaleax approved these changes Oct 5, 2016

View reviewed changes

lpinca approved these changes Oct 6, 2016

View reviewed changes

lpinca reviewed Oct 6, 2016

View reviewed changes

jasnell approved these changes Oct 6, 2016

View reviewed changes

jasnell added semver-major PRs that contain breaking changes and should be released in the next major version. error-handling labels Oct 6, 2016

bnoordhuis added lts-watch-v4.x and removed semver-major PRs that contain breaking changes and should be released in the next major version. labels Oct 7, 2016

not-an-aardvark approved these changes Oct 12, 2016

View reviewed changes

bnoordhuis force-pushed the reject-bad-http-request-chars branch from a2a9168 to 4f62acd Compare October 13, 2016 11:40

bnoordhuis closed this Oct 13, 2016

bnoordhuis deleted the reject-bad-http-request-chars branch October 13, 2016 11:40

MylesBorins added the lts-watch-v6.x label Nov 11, 2016

evanlucas mentioned this pull request Dec 8, 2016

http: Reject paths containing non-ASCII characters #3062

Closed

MylesBorins mentioned this pull request Dec 21, 2016

test: simplify test-http-client-unescaped-path #9649

Closed

MylesBorins added land-on-v6.x and removed lts-watch-v6.x labels Mar 7, 2017

MylesBorins mentioned this pull request Mar 9, 2017

v6.10.1 proposal #11759

Merged

MylesBorins mentioned this pull request Mar 9, 2017

v4.8.1 proposal #11760

Merged

addaleax mentioned this pull request Oct 19, 2017

http: disallow two-byte characters in URL path #16237

Closed

3 tasks

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

http: reject control characters in http.request() #8923

http: reject control characters in http.request() #8923

bnoordhuis commented Oct 4, 2016

indutny left a comment

evanlucas left a comment

jbergstroem commented Oct 4, 2016

tarqd Oct 5, 2016

lpinca Oct 6, 2016

tarqd Oct 6, 2016

silverwind left a comment

silverwind Oct 5, 2016

addaleax Oct 5, 2016

silverwind Oct 6, 2016

addaleax left a comment

lpinca left a comment

lpinca Oct 6, 2016

bnoordhuis Oct 6, 2016

lpinca Oct 6, 2016

jasnell left a comment

jasnell commented Oct 6, 2016

jasnell commented Oct 6, 2016

addaleax commented Oct 6, 2016

jasnell commented Oct 6, 2016

addaleax commented Oct 6, 2016

jasnell commented Oct 6, 2016

Trott commented Oct 6, 2016

jasnell commented Oct 6, 2016

bnoordhuis commented Oct 7, 2016

bnoordhuis commented Oct 11, 2016

MylesBorins commented Nov 11, 2016

MylesBorins commented Dec 21, 2016

MylesBorins commented Dec 21, 2016

bnoordhuis commented Dec 22, 2016

http: reject control characters in http.request() #8923

http: reject control characters in http.request() #8923

Conversation

bnoordhuis commented Oct 4, 2016

indutny left a comment

Choose a reason for hiding this comment

evanlucas left a comment

Choose a reason for hiding this comment

jbergstroem commented Oct 4, 2016

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

silverwind left a comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

addaleax left a comment

Choose a reason for hiding this comment

lpinca left a comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

jasnell left a comment

Choose a reason for hiding this comment

jasnell commented Oct 6, 2016

jasnell commented Oct 6, 2016

addaleax commented Oct 6, 2016

jasnell commented Oct 6, 2016

addaleax commented Oct 6, 2016

jasnell commented Oct 6, 2016

Trott commented Oct 6, 2016

jasnell commented Oct 6, 2016

bnoordhuis commented Oct 7, 2016

bnoordhuis commented Oct 11, 2016

MylesBorins commented Nov 11, 2016

MylesBorins commented Dec 21, 2016

MylesBorins commented Dec 21, 2016

bnoordhuis commented Dec 22, 2016