util: allow wildcards in debuglog() #17609

TylerYang · 2017-12-11T15:19:17Z

Checklist

make -j4 test (UNIX), or vcbuild test (Windows) passes
tests and/or benchmarks are included
documentation is changed or added
commit message follows commit guidelines

Affected core subsystem(s)
util

fixes #17605

Fixes: nodejs#17605

Fishrock123 · 2017-12-11T17:47:41Z

I think this is a great idea. Do you think you'd be able to rebase out the merge commit?

TylerYang · 2017-12-11T21:29:32Z

@Fishrock123 Yes, just rebased. Feel free to add more comments : )

targos · 2017-12-12T09:49:11Z

lib/util.js

+      .replace(/\*/g, '.*');
+  });
+
+  return new RegExp(strArr.join('|'), 'i');


The RegExp must enforce start and end: new RegExp('^(' + strArr.join('|') + ')$'), otherwise any string acts as one with wildcards at the beginning and end (and since the default is an empty string, not setting NODE_DEBUG results in all debug messages being printed).

thx, updated.

BridgeAR

A few small comments that should be addressed, but otherwise LGTM.

BridgeAR · 2017-12-12T20:24:45Z

lib/util.js

+      .replace(/\*/g, '.*');
+  });
+
+  return new RegExp(`^${strArr.join('|')}$`, 'i');


Instead of calling map and join you could use reduce

const regExpStr = strArr.reduce((str, entry, i) => { let part = entry.replace(regExpSpecialChars, '\\$&') .replace(/\*/g, '.*'); if (i -1 !== strArr.length) part += '|'; return `${str}${part}`; }, ''); return new RegExp(`^${regExpStr}$`, 'i');

But that might just be my personal preference.

I think the current code is a bit more readable (not having to deal with the indices is nice, at least :))

BridgeAR · 2017-12-12T20:27:33Z

lib/util.js

-    debugEnviron = new Set(
-      (process.env.NODE_DEBUG || '').split(',').map((s) => s.toUpperCase()));
+    debugEnviron = Array.from(new Set(
+      (process.env.NODE_DEBUG || '').split(',').map((s) => s.toUpperCase())));


This part is weird. First an Array is created, then the Set, then an Array from the Set.

@BridgeAR This happens for deduplication, do you have other suggestions? Converting to a Set and back is usualy what I do too

The only thing I can think of is improving readability by adding one more statement like,

debugEnviron = (process.env.NODE_DEBUG || '').split(',').map((s) => s.toUpperCase()) debugEnviron = Array.from(new Set(debugEnviron));

But i wonder if it worth.

wait, we can convert NODE_DEBUG to uppercase before splitting so that we could remove the map operator.

debugEnviron = (process.env.NODE_DEBUG || '').toUpperCase().split(','); debugEnviron = Array.from(new Set(debugEnviron));

or just

debugEnviron = Array.from(new Set((process.env.NODE_DEBUG || '').toUpperCase().split(',')));

what do you think? @BridgeAR @addaleax

I think it's just that we use [...spread] syntax more than Array.from(), which has the added advantage of being safer in case Array or Array.from is overridden. In any case I'd prefer the first variant for clarity.

Thx @TimothyGu Nice solution! Updated.

@addaleax the current implementation was not about deduplication, see #13841. I guess it was chosen to make sure you do not have to lookup the entry in a array as that is slower. Depulication for a environment variable that is passed in by the user is definitely surprising to me. I would expect people not to include the same entry multiple times ;-)
As this is a one time thing and debugging is always slower, I will not block it from being included if you really think that is necessary (even though I do not see the point in it. Nothing bad can happen if a entry is a duplicate).

What I would like is to move the replacement to before the splitting. Running the RegExp once is a lot better than running it for each entry (even though there are probably very few).

I would also argue that the part being lazy does not make all that much sense here and I would rather move it out instead (it became lazy in a independent commit 22c68fd).

In case the replacement is done right away, the splitting and joining could also be replaced by a replace().

Hi @BridgeAR . Thx for details, appreciate that!

What I would like is to move the replacement to before the splitting. Running the RegExp once is a lot better than running it for each entry (even though there are probably very few).

make sense, thx

In case the replacement is done right away, the splitting and joining could also be replaced by a replace().

~~Yes, it could. Would this reduce the readability?~~
Just try, it's more intuitive. thx.

BridgeAR · 2017-12-12T20:29:38Z

doc/api/util.md

 it will output something like:

 ```txt
-FOO 3245: hello from foo [123]
+FOO-BAR 3245: hello from foo [123]


It would be nice to have another example specifically for the wildcard. Including counter examples. Especially people who are new to coding might want to have more explicit explanations.

thx, added.

BridgeAR · 2017-12-12T20:31:21Z

doc/api/util.md

@@ -75,7 +75,7 @@ is wrapped in an `Error` with the original value stored in a field named
 added: v0.11.3
 -->

-* `section` {string} A string identifying the portion of the application for
+* `section` {string} A wildcard string identifying the portion of the application for


The string itself is not a wildcard string as long as there is no asterisk included.

thx, updated.

BridgeAR

Almost LGTM

BridgeAR · 2017-12-14T04:46:04Z

lib/util.js

  }
  set = set.toUpperCase();
  if (!debugs[set]) {
-    if (debugEnviron.has(set)) {
+    const regex = stringToRegExp(debugEnv);


The RegExp should only be calculated once. It has to be done in the if (debugEnv === undefined) { ... } check. I personally would actually move that out of the debuglog to be not lazy (meaning the NODE_DEBUG env should be checked when util is required and the RegExp should also be created at that point).

E.g.

let debugRegExp if (process.env.NODE_DEBUG) { debugEnv = process.env.NODE_DEBUG.replace(...).replace(...).replace(...).toUpperCase() debugRegExp = new RegExp(...) } function debuglog(set) { ... }

Good point, updated.

apapirovski · 2017-12-14T16:36:32Z

@TylerYang It doesn't appear that any of the commits here associated with your GitHub account. See the Contributing guide for info on how to fix that, as otherwise you won't get the GitHub "Contributor" badge next to your name.

TylerYang · 2017-12-14T18:02:13Z

@apapirovski Thx for pointed out! Updated my account to the github one.

BridgeAR · 2017-12-14T18:24:19Z

lib/util.js

+
+let debugEnvRegex;
+const regExpSpecialChars = /[|\\{}()[\]^$+?.]/g;
+if (debugEnvRegex === undefined) {


This if statement will always evaluates to true. It should actually test for process.env.NODE_DEBUG and only if that is truthy it should be evaluated.

This is how I would write it

const debugs = {}; let debugEnvRegex = /^$/; if (process.env.NODE_DEBUG) { const debugEnv = process.env.NODE_DEBUG // Escape RegExp special characters .replace(/[|\\{}()[\]^$+?.]/g, '\\$&') .replace(/\*/g, '.*') .replace(/,/g, '$|^') .toUpperCase(); debugEnvRegex = new RegExp(`^${debugEnv}$`, 'i'); }

You are right! I should consider that corner case at the first time. Updated. Thx : )

BridgeAR · 2017-12-14T18:25:13Z

@TylerYang one minor issue left. Thanks a lot for being so patient!

BridgeAR · 2017-12-14T18:45:23Z

lib/util.js

+const debugs = {};
+
+let debugEnvRegex = /^$/;
+const regExpSpecialChars = /[|\\{}()[\]^$+?.]/g;


Nit - please move this into the if. There is no point in keeping the variable around after the RegExp got created.

targos · 2017-12-15T09:37:42Z

CI: https://ci.nodejs.org/job/node-test-pull-request/12117/

targos · 2017-12-15T09:39:17Z

Whoever lands this, feel free to squash all commits and remove my authorship

PR-URL: nodejs#17609 Fixes: nodejs#17605 Reviewed-By: Anna Henningsen <anna@addaleax.net> Reviewed-By: Ruben Bridgewater <ruben@bridgewater.de> Reviewed-By: James M Snell <jasnell@gmail.com> Reviewed-By: Colin Ihrig <cjihrig@gmail.com> Reviewed-By: Michaël Zasso <targos@protonmail.com>

BridgeAR · 2017-12-15T21:34:48Z

Landed in fafd9fb

@TylerYang congratulations on your first commit to Node.js! 🎉

PR-URL: #17609 Fixes: #17605 Reviewed-By: Anna Henningsen <anna@addaleax.net> Reviewed-By: Ruben Bridgewater <ruben@bridgewater.de> Reviewed-By: James M Snell <jasnell@gmail.com> Reviewed-By: Colin Ihrig <cjihrig@gmail.com> Reviewed-By: Michaël Zasso <targos@protonmail.com>

Notable change: * async_hooks: - deprecate AsyncHooks Sensitive API and runInAsyncIdScope. Neither API were documented. (Andreas Madsen) #16972 * deps: - update nghttp2 to 1.29.0 (James M Snell) #17908 - upgrade npm to 5.6.0 (Kat Marchán) #17535 - cherry-pick 50f7455 from upstream V8 (Michaël Zasso) #16591 * events: - remove reaches into _events internals (Anatoli Papirovski) #17440 * http: - add rawPacket in err of `clientError` event (XadillaX) #17672 * http2: - implement maxSessionMemory (James M Snell) #17967 - add initial support for originSet (James M Snell) #17935 - add altsvc support (James M Snell) #17917 - perf_hooks integration (James M Snell) #17906 * net: - remove Socket.prototype.write (Anna Henningsen) #17644 - remove Socket.prototype.listen (Ruben Bridgewater) #13735 * repl: - show lexically scoped vars in tab completion (Michaël Zasso) #16591 * stream: - rm {writeable/readable}State.length (Calvin Metcalf) #12857 - add flow and buffer properties to streams (Calvin Metcalf) #12855 * util: - allow wildcards in NODE_DEBUG variable (Tyler) #17609 * zlib: - add ArrayBuffer support (Jem Bezooyen) #16042 * Addedew collaborator** - [starkwang](https://github.com/starkwang) Weijia Wang * Addedew TSC member** - [danbev](https://github.com/danbev) Daniel Bevenius PR-URL: #18069

Notable change: * async_hooks: - deprecate AsyncHooks Sensitive API and runInAsyncIdScope. Neither API were documented. (Andreas Madsen) #16972 * deps: - update nghttp2 to 1.29.0 (James M Snell) #17908 - upgrade npm to 5.6.0 (Kat Marchán) #17535 - cherry-pick 50f7455 from upstream V8 (Michaël Zasso) #16591 * events: - remove reaches into _events internals (Anatoli Papirovski) #17440 * http: - add rawPacket in err of `clientError` event (XadillaX) #17672 * http2: - implement maxSessionMemory (James M Snell) #17967 - add initial support for originSet (James M Snell) #17935 - add altsvc support (James M Snell) #17917 - perf_hooks integration (James M Snell) #17906 - Refactoring and cleanup of Http2Session and Http2Stream destroy (James M Snell) #17406 * net: - remove Socket.prototype.write (Anna Henningsen) #17644 - remove Socket.prototype.listen (Ruben Bridgewater) #13735 * repl: - show lexically scoped vars in tab completion (Michaël Zasso) #16591 * stream: - rm {writeable/readable}State.length (Calvin Metcalf) #12857 - add flow and buffer properties to streams (Calvin Metcalf) #12855 * util: - allow wildcards in NODE_DEBUG variable (Tyler) #17609 * zlib: - add ArrayBuffer support (Jem Bezooyen) #16042 * Addedew collaborator** - [starkwang](https://github.com/starkwang) Weijia Wang * Addedew TSC member** - [danbev](https://github.com/danbev) Daniel Bevenius PR-URL: #18069

MylesBorins · 2018-01-23T06:52:20Z

Seems like this should have perhaps been labelled semver-minor... is that mistaken?

BridgeAR · 2018-01-23T10:04:28Z

@MylesBorins I totally agree.

MylesBorins · 2018-08-02T14:46:54Z

This would likely be useful for 8.x, but it may rely on a Semver-Major commit. Can someone confirm this is OK to land and desirable for 8.x

util: allow wildcards in NODE_DEBUG variable

bf5f3ac

Fixes: nodejs#17605

nodejs-github-bot added the util Issues and PRs related to the built-in util module. label Dec 11, 2017

TylerYang mentioned this pull request Dec 11, 2017

Tyler refactor debuglog targos/node#3

Closed

4 tasks

TylerYang changed the title ~~Tyler refactor debuglog~~ util: allow wildcards in debuglog() Dec 11, 2017

TylerYang force-pushed the tyler-refactor-debuglog branch from e8722a5 to 668fa2b Compare December 11, 2017 21:28

addaleax approved these changes Dec 11, 2017

View reviewed changes

targos mentioned this pull request Dec 12, 2017

util: allow wildcards in NODE_DEBUG variable #17607

Closed

4 tasks

targos reviewed Dec 12, 2017

View reviewed changes

TylerYang force-pushed the tyler-refactor-debuglog branch from ec25c33 to fd258c8 Compare December 12, 2017 11:00

BridgeAR approved these changes Dec 12, 2017

View reviewed changes

jasnell approved these changes Dec 12, 2017

View reviewed changes

TylerYang force-pushed the tyler-refactor-debuglog branch 2 times, most recently from 560a840 to 7a998e1 Compare December 13, 2017 08:33

cjihrig approved these changes Dec 13, 2017

View reviewed changes

TylerYang force-pushed the tyler-refactor-debuglog branch 2 times, most recently from b7b0c51 to 9d8ce29 Compare December 14, 2017 03:52

BridgeAR reviewed Dec 14, 2017

View reviewed changes

TylerYang force-pushed the tyler-refactor-debuglog branch from 9d8ce29 to 88e5777 Compare December 14, 2017 05:14

TylerYang force-pushed the tyler-refactor-debuglog branch from 88e5777 to 13f032e Compare December 14, 2017 17:58

BridgeAR reviewed Dec 14, 2017

View reviewed changes

TylerYang force-pushed the tyler-refactor-debuglog branch from 13f032e to b8953b2 Compare December 14, 2017 18:33

BridgeAR reviewed Dec 14, 2017

View reviewed changes

TylerYang added 2 commits December 15, 2017 10:39

util: refactor debuglog() from array regex to regex string

a92714d

doc: updated wildcard debuglog example on util

1496f3f

TylerYang force-pushed the tyler-refactor-debuglog branch from b8953b2 to 1496f3f Compare December 15, 2017 02:39

targos approved these changes Dec 15, 2017

View reviewed changes

targos added the author ready PRs that have at least one approval, no pending requests for changes, and a CI started. label Dec 15, 2017

BridgeAR closed this Dec 15, 2017

MylesBorins mentioned this pull request Jan 10, 2018

v9.4.0 proposal #18069

Merged

TimothyGu removed the author ready PRs that have at least one approval, no pending requests for changes, and a CI started. label Jan 13, 2018

MylesBorins added the semver-minor PRs that contain new features and should be released in the next minor version. label Jan 23, 2018

MylesBorins added the backport-requested-v8.x label Aug 2, 2018

util: allow wildcards in debuglog() #17609

util: allow wildcards in debuglog() #17609

Conversation

TylerYang commented Dec 11, 2017 • edited Loading

Checklist

Fishrock123 commented Dec 11, 2017

TylerYang commented Dec 11, 2017

targos Dec 12, 2017 • edited Loading

Choose a reason for hiding this comment

Choose a reason for hiding this comment

BridgeAR left a comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

TylerYang Dec 13, 2017 • edited Loading

Choose a reason for hiding this comment

TylerYang Dec 13, 2017 • edited Loading

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

BridgeAR Dec 13, 2017 • edited Loading

Choose a reason for hiding this comment

Choose a reason for hiding this comment

TylerYang Dec 14, 2017 • edited Loading

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

BridgeAR left a comment

Choose a reason for hiding this comment

BridgeAR Dec 14, 2017 • edited Loading

Choose a reason for hiding this comment

Choose a reason for hiding this comment

apapirovski commented Dec 14, 2017

TylerYang commented Dec 14, 2017

Choose a reason for hiding this comment

TylerYang Dec 14, 2017 • edited Loading

Choose a reason for hiding this comment

BridgeAR commented Dec 14, 2017

Choose a reason for hiding this comment

Choose a reason for hiding this comment

targos commented Dec 15, 2017

targos commented Dec 15, 2017

BridgeAR commented Dec 15, 2017

MylesBorins commented Jan 23, 2018

BridgeAR commented Jan 23, 2018

MylesBorins commented Aug 2, 2018

TylerYang commented Dec 11, 2017 •

edited

Loading

targos Dec 12, 2017 •

edited

Loading

TylerYang Dec 13, 2017 •

edited

Loading

TylerYang Dec 13, 2017 •

edited

Loading

BridgeAR Dec 13, 2017 •

edited

Loading

TylerYang Dec 14, 2017 •

edited

Loading

BridgeAR Dec 14, 2017 •

edited

Loading

TylerYang Dec 14, 2017 •

edited

Loading