-
Notifications
You must be signed in to change notification settings - Fork 29.6k
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
crypto: allow adding extra certs to well-known CAs #9139
Conversation
const server = tls.createServer(options, function(s) { | ||
s.end('bye'); | ||
}).listen(0) | ||
.on('listening', function() { |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
This function can be passed as a second argument to listen()
to make things shorter.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Also, you might wrap it in a common.mustCall()
to be extra safe.
.on('listening', function() { | ||
process.env.CHILD = 'yes'; | ||
process.env.PORT = this.address().port; | ||
process.env.NODE_EXTRA_CA_CERTS = common.fixturesDir + '/keys/ca1-cert.pem'; |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
It would be better to instead set these variables in the config object passed to fork()
:
const env = {
CHILD: 'yes',
PORT: this.address().port,
NODE_EXTRA_CA_CERTS: common.fixturesDir + '/keys/ca1-cert.pem'
};
fork(__filename, { env: env }).on('exit', common.mustCall(function(status) {
ERR_clear_error(); | ||
} else if (err) { | ||
fprintf(stderr, "Ignoring extra certs from `%s`, load failed: %s\n", | ||
extra_root_certs_file, ERR_error_string(err, NULL)); |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
This should be indented more to align with the other function arguments.
unsigned long err = ERR_peek_error(); // NOLINT(runtime/int) | ||
// Clear error if its EOF/no start line found. | ||
if (ERR_GET_LIB(err) == ERR_LIB_PEM | ||
&& ERR_GET_REASON(err) == PEM_R_NO_START_LINE) { |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
&&
should be at end of the previous line.
if (!bio) { | ||
unsigned long err = ERR_get_error(); // NOLINT(runtime/int) | ||
fprintf(stderr, "Ignoring extra certs from `%s`, load failed: %s\n", | ||
extra_root_certs_file, ERR_error_string(err, NULL)); |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
This should be indented more to align with the other function arguments.
@@ -298,6 +298,15 @@ asynchronous when outputting to a TTY on platforms which support async stdio. | |||
Setting this will void any guarantee that stdio will not be interleaved or | |||
dropped at program exit. **Use of this mode is not recommended.** | |||
|
|||
### `NODE_EXTRA_CA_CERTS=file` | |||
|
|||
When set, the well known "root" CAs (like VeriSign) will be extended with the |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
There's an extra space between 'the' and 'well'.
|
||
When set, the well known "root" CAs (like VeriSign) will be extended with the | ||
exta certificates in `file`. The file should consist of one or more trusted | ||
certificates in PEM format. A message will be printed (once) if the file is |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
There's an extra space before 'A message'.
I think the |
### `NODE_EXTRA_CA_CERTS=file` | ||
|
||
When set, the well known "root" CAs (like VeriSign) will be extended with the | ||
exta certificates in `file`. The file should consist of one or more trusted |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Typo: extra
a691889
to
87e2514
Compare
I'll look around to see if I can call Note that there is a pre-existing Line 5788 in 7014566
emitWarning , too?
|
@sam-github No, because that's a more fatal error and the process will abort after it prints that message. |
c133999
to
83c7a88
Compare
@mscdex @bnoordhuis Now calling Also, a design detail: currently, the value of the
It would be possible to call But, I think how it is now is consistent with general node practice, and that anyone who can edit the program source could also just provide the extra CA certs themselves via the options. Agreed? |
If the call to |
assert.equal(status, 0, 'client did not succeed in connecting'); | ||
})); | ||
|
||
server.unref(); |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Is this necessary? What about just calling server.close()
inside the server's connection handler?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I don't really mind, though I don't see how that is preferable. unref allows node to close gracefully when it has nothing else to do, closing would also have the same effect, why is it better?
node::OneByteString(env->isolate(), warning, strlen(warning)) | ||
}; | ||
|
||
f.As<v8::Function>()->Call(process, arraysize(args), args); |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
It might be better to use MakeCallback()
here instead of calling it directly.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
@mscdex, for my benefit when would you suggest MakeCallback instead of Call or vice versa. I see that existing code has a mix of both.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I assume MakeCallback()
should almost always be used because of domains/async_hooks and such?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I did find this more detailed discussion on the topic from a while back: nodejs/nan#284. I think the net is that unless you have a specific reason that you don't want process._tickCallback() to be called, then MakeCallback() is preferred over the direct Call.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
@mhdawson 's link above says
The latter (MakeCallback) case calls the JS callback function, but then also ends the current tick, calling process._tickCallback() and any other callbacks registered via nextTick(). This form is intended only for use by C++ code that was dispatched from the libuv event loop, and not for C++ code that was invoked from JS code.
It sounds like MakeCallback()
has lots of side effects, are you sure its right? My usage seems to fit the "not C++ invoked from the event loop" case.
It occured to me that process.emitWarning()
could actually emit... which would call user code, but I checked, and it emits in the next tick, so isn't going to hit any user-supplied js.
I don't know enough about the distinction to choose, I'll need some guidance here.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
@bnoordhuis do you have an opinion on Call/MakeCallback in this instance ?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
@trevnorris I think that MakeCallback can cause running microtasks as a side-effect (
Line 1268 in 09d41e2
env->isolate()->RunMicrotasks(); |
Call()
, or node::MakeCallback()
?
My failed attempt at making a "microtask" :-( at https://gist.github.com/927a451526de55847d53694b53b01a20
LGTM subject to fixing up @mscdex remaining comments. |
conversation in #9139 (comment) looks outdated because I added |
648a62d
to
6c50346
Compare
@@ -120,6 +120,8 @@ const char* const root_certs[] = { | |||
#include "node_root_certs.h" // NOLINT(build/include_order) | |||
}; | |||
|
|||
const char* extra_root_certs_file; |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Should be static
.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
done, note that root_certs
a few lines above is not static, but appears that it could be (I was mirroring its definition). root_cert_store
below is explicitly declared extern, it shouldn't be static.
extra_root_certs_file = fname; | ||
} | ||
|
||
static void emitWarning(Environment* env, const char* fmt, ...) { |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
EmitWarning
vsnprintf(warning, sizeof(warning), fmt, ap); | ||
va_end(ap); | ||
|
||
#if 0 |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Don't leave dead code in.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Its not dead code. I am asking for confirmation that MakeCallbacks()
is the right thing to do, I believe the #if 0
branch is the correct one, and that MakeCallbacks()
is designed for callbacks from uv, and other async usage, not for this kind of direct js-to-c++-to-js usage. That it is leaking handles and running microtasks make it seem the wrong tool. @trevnorris @bnoordhuis Shouldn't this be a simple Call()
?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Call()
interacts poorly with domains and other mechanisms that hang off async_hooks so it's really only safe for internal JS code. W.r.t. handle leaks, Call
does that as well (the return value.)
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
@bnoordhuis what is "internal JS code"? In this case I am calling
node/lib/internal/process/warning.js
Lines 28 to 49 in 6845d6e
process.emitWarning = function(warning, name, ctor) { | |
if (typeof name === 'function') { | |
ctor = name; | |
name = 'Warning'; | |
} | |
if (warning === undefined || typeof warning === 'string') { | |
warning = new Error(warning); | |
warning.name = name || 'Warning'; | |
Error.captureStackTrace(warning, ctor || process.emitWarning); | |
} | |
if (!(warning instanceof Error)) { | |
throw new TypeError('\'warning\' must be an Error object or string.'); | |
} | |
if (warning.name === 'DeprecationWarning') { | |
if (process.noDeprecation) | |
return; | |
if (process.throwDeprecation) | |
throw warning; | |
} | |
process.nextTick(() => process.emit('warning', warning)); | |
}; | |
} |
process.emit()
, which can call user-code. Is that the difference here? That user code (process.on('warning', user....
) was triggered from within createServer()
, so we need to use MakeCallback()
to keep the accounting correct for domains, etc.?
Otherwise, this doesn't seem so different from url.js, which calls into c++ to do some work, and synchronously calls back with ->Call()
to internal js code. Or node_http_parser, which uses Call()
to call pushValueToArray()
, a js function from lib/internal/bootstrap_node.js
I (and @mhdawson , perhaps others), would like to understand the basic principle here, so I correctly distinguish between the two in the future.
Also, with MakeCallback()
:
require("tls").createServer()
// -> MakeCallback
// -> process.emitWarning()
// -> env->isolate()->RunMicrotasks()
// -> env->tick_callback_function()->Call()
// return from createServer()
How would I demonstrate that createServer()
, when it uses MakeCallback()
, is causing micro tasks and ticks to get called? Or do they not? I'm trying to see what the side-effects are, but couldn't prove there were any with https://gist.github.com/sam-github/927a451526de55847d53694b53b01a20 (I was looking to cause output between the BEGIN
and END
, which should not happen).
I addressed your other comments, thanks, and I will fix the handle leak, looking into that now.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Oh, that's a good point; I didn't realize emitWarning() defers to the next tick. In that case Call()
should be fine.
#else | ||
Local<Object> process = env->process_object(); | ||
Local<Value> args[] = { | ||
node::OneByteString(env->isolate(), warning, strlen(warning)) |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
You can leave out the strlen()
.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Oh, and you could write this as:
Local<Value> arg = node::OneByteString(env->isolate(), warning);
MakeCallback(env, process, "emitWarning", 1, &arg);
It's a little more succinct but whatever you prefer.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I did used the array form because I had the impression it was more idiomatic, but I simplified the code as you suggest.
}; | ||
MakeCallback(env, process, "emitWarning", arraysize(args), args); | ||
#endif | ||
} |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
This function needs a HandleScope
, it's leaking handles into the scope of the caller.
if (!bio) { | ||
unsigned long err = ERR_get_error(); // NOLINT(runtime/int) | ||
emitWarning(env, "Ignoring extra certs from `%s`, load failed: %s\n", | ||
extra_root_certs_file, ERR_error_string(err, NULL)); |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Arguments should line up here and below.
if (!extra_root_certs_file) | ||
return; | ||
|
||
BIO *bio = BIO_new_file(extra_root_certs_file, "r"); |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Style: BIO* bio = ...
emitWarning(env, "Ignoring extra certs from `%s`, load failed: %s\n", | ||
extra_root_certs_file, ERR_error_string(err, NULL)); | ||
} | ||
} |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
This function could share code with SSL_CTX_use_certificate_chain. Also, you could add a MarkPopErrorOnReturn mark_pop_error_on_return
at the top so you don't have to manually juggle the error stack.
@@ -770,6 +830,8 @@ void SecureContext::AddRootCerts(const FunctionCallbackInfo<Value>& args) { | |||
BIO_free_all(bp); | |||
X509_free(x509); | |||
} | |||
|
|||
LoadExtraCaCerts(Environment::GetCurrent(args)); |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
You could use sc->env()
here instead.
@@ -76,6 +76,8 @@ class SecureContext : public BaseObject { | |||
|
|||
static void Initialize(Environment* env, v8::Local<v8::Object> target); | |||
|
|||
static void UseExtraCaCerts(const char* file); |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
It's kind of odd that this is a (static) method but LoadExtraCaCerts()
is not. Neither manipulate SecureContext state so they probably shouldn't be methods, static or not.
I have a feeling we can move much of the logic to JS land if we simply exposed the root CA list somewhere. |
PR-URL: #9139 Reviewed-By: Ben Noordhuis <info@bnoordhuis.nl> Reviewed-By: Fedor Indutny <fedor.indutny@gmail.com>
In closed environments, self-signed or privately signed certificates are commonly used, and rejected by Node.js since their root CAs are not well-known. Allow extending the set of well-known compiled-in CAs via environment, so they can be set as a matter of policy. PR-URL: #9139 Reviewed-By: Ben Noordhuis <info@bnoordhuis.nl> Reviewed-By: Fedor Indutny <fedor.indutny@gmail.com>
PR-URL: #9139 Reviewed-By: Ben Noordhuis <info@bnoordhuis.nl> Reviewed-By: Fedor Indutny <fedor.indutny@gmail.com>
In closed environments, self-signed or privately signed certificates are commonly used, and rejected by Node.js since their root CAs are not well-known. Allow extending the set of well-known compiled-in CAs via environment, so they can be set as a matter of policy. PR-URL: #9139 Reviewed-By: Ben Noordhuis <info@bnoordhuis.nl> Reviewed-By: Fedor Indutny <fedor.indutny@gmail.com>
In closed environments, self-signed or privately signed certificates are commonly used, and rejected by Node.js since their root CAs are not well-known. Allow extending the set of well-known compiled-in CAs via environment, so they can be set as a matter of policy. PR-URL: nodejs#9139 Reviewed-By: Ben Noordhuis <info@bnoordhuis.nl> Reviewed-By: Fedor Indutny <fedor.indutny@gmail.com>
PR-URL: #9139 Reviewed-By: Ben Noordhuis <info@bnoordhuis.nl> Reviewed-By: Fedor Indutny <fedor.indutny@gmail.com>
In closed environments, self-signed or privately signed certificates are commonly used, and rejected by Node.js since their root CAs are not well-known. Allow extending the set of well-known compiled-in CAs via environment, so they can be set as a matter of policy. PR-URL: #9139 Reviewed-By: Ben Noordhuis <info@bnoordhuis.nl> Reviewed-By: Fedor Indutny <fedor.indutny@gmail.com>
In closed environments, self-signed or privately signed certificates are commonly used, and rejected by Node.js since their root CAs are not well-known. Allow extending the set of well-known compiled-in CAs via environment, so they can be set as a matter of policy. PR-URL: #9139 Reviewed-By: Ben Noordhuis <info@bnoordhuis.nl> Reviewed-By: Fedor Indutny <fedor.indutny@gmail.com>
Notable Changes: * child_process: add shell option to spawn() (cjihrig) #4598 * crypto: * add ALPN Support (Shigeki Ohtsu) #2564 * allow adding extra certs to well-known CAs (Sam Roberts) #9139 * deps: * v8: expose statistics about heap spaces (Ben Ripkens) #4463 * fs: add the fs.mkdtemp() function. (Florian MARGAINE) #5333 * process: * add `externalMemory` to `process` (Fedor Indutny) #9587 * add process.cpuUsage() (Patrick Mueller) #10796
Notable Changes: The SEMVER-MINOR changes include: * crypto: allow adding extra certs to well-known CAs (Sam Roberts) #9139 * deps: Upgrade INTL ICU to version 58 (Steven R. Loomis) #9234 * process: add `process.memoryUsage.external` (Fedor Indutny) #9587 * src: add wrapper for process.emitWarning() (Sam Roberts) #9139 Notable SEMVER-PATCH changes include: * fs: cache non-symlinks in realpathSync. (Jeremy Yallop) #10253 * repl: allow autocompletion for scoped packages (Evan Lucas) #10296
Notable Changes: * child_process: add shell option to spawn() (cjihrig) #4598 * crypto: * add ALPN Support (Shigeki Ohtsu) #2564 * allow adding extra certs to well-known CAs (Sam Roberts) #9139 * deps: * v8: expose statistics about heap spaces (Ben Ripkens) #4463 * fs: add the fs.mkdtemp() function. (Florian MARGAINE) #5333 * process: * add `externalMemory` to `process` (Fedor Indutny) #9587 * add process.cpuUsage() (Patrick Mueller) #10796 PR-URL: #10973
Notable Changes: The SEMVER-MINOR changes include: * crypto: allow adding extra certs to well-known CAs (Sam Roberts) #9139 * deps: Upgrade INTL ICU to version 58 (Steven R. Loomis) #9234 * process: add `process.memoryUsage.external` (Fedor Indutny) #9587 * src: add wrapper for process.emitWarning() (Sam Roberts) #9139 Notable SEMVER-PATCH changes include: * fs: cache non-symlinks in realpathSync. (Jeremy Yallop) #10253 * repl: allow autocompletion for scoped packages (Evan Lucas) #10296 PR-URL: #10974
Notable Changes: * child_process: add shell option to spawn() (cjihrig) nodejs/node#4598 * crypto: * add ALPN Support (Shigeki Ohtsu) nodejs/node#2564 * allow adding extra certs to well-known CAs (Sam Roberts) nodejs/node#9139 * deps: * v8: expose statistics about heap spaces (Ben Ripkens) nodejs/node#4463 * fs: add the fs.mkdtemp() function. (Florian MARGAINE) nodejs/node#5333 * process: * add `externalMemory` to `process` (Fedor Indutny) nodejs/node#9587 * add process.cpuUsage() (Patrick Mueller) nodejs/node#10796 Signed-off-by: Ilkka Myller <ilkka.myller@nodefield.com>
Notable Changes: The SEMVER-MINOR changes include: * crypto: allow adding extra certs to well-known CAs (Sam Roberts) nodejs/node#9139 * deps: Upgrade INTL ICU to version 58 (Steven R. Loomis) nodejs/node#9234 * process: add `process.memoryUsage.external` (Fedor Indutny) nodejs/node#9587 * src: add wrapper for process.emitWarning() (Sam Roberts) nodejs/node#9139 Notable SEMVER-PATCH changes include: * fs: cache non-symlinks in realpathSync. (Jeremy Yallop) nodejs/node#10253 * repl: allow autocompletion for scoped packages (Evan Lucas) nodejs/node#10296 Signed-off-by: Ilkka Myller <ilkka.myller@nodefield.com>
Notable Changes: * child_process: add shell option to spawn() (cjihrig) nodejs/node#4598 * crypto: * add ALPN Support (Shigeki Ohtsu) nodejs/node#2564 * allow adding extra certs to well-known CAs (Sam Roberts) nodejs/node#9139 * deps: * v8: expose statistics about heap spaces (Ben Ripkens) nodejs/node#4463 * fs: add the fs.mkdtemp() function. (Florian MARGAINE) nodejs/node#5333 * process: * add `externalMemory` to `process` (Fedor Indutny) nodejs/node#9587 * add process.cpuUsage() (Patrick Mueller) nodejs/node#10796 Signed-off-by: Ilkka Myller <ilkka.myller@nodefield.com>
Notable Changes: The SEMVER-MINOR changes include: * crypto: allow adding extra certs to well-known CAs (Sam Roberts) nodejs/node#9139 * deps: Upgrade INTL ICU to version 58 (Steven R. Loomis) nodejs/node#9234 * process: add `process.memoryUsage.external` (Fedor Indutny) nodejs/node#9587 * src: add wrapper for process.emitWarning() (Sam Roberts) nodejs/node#9139 Notable SEMVER-PATCH changes include: * fs: cache non-symlinks in realpathSync. (Jeremy Yallop) nodejs/node#10253 * repl: allow autocompletion for scoped packages (Evan Lucas) nodejs/node#10296 Signed-off-by: Ilkka Myller <ilkka.myller@nodefield.com>
Checklist
make -j8 test
(UNIX), orvcbuild test nosign
(Windows) passesAffected core subsystem(s)
crypto,tls,https
Description of change
In closed environments, self-signed or privately signed certificates are
commonly used, and rejected by Node.js since their root CAs are not
well-known. Allow extending the set of well-known compiled-in CAs via
environment, so they can be set as a matter of policy.
Note that #8334 addresses a similar but not identical use-case, and works better for those (like linux distributions) that are willing to recompile Node.js to use OpenSSL's default certificate store. #8334
doesn't address those who cannot recompile, do not have access to the system certificate store, or
who are are on a system where the default certificate store is not exposed as an OpenSSL compatible
store (for example, OS X and Windows).
See #3159, #8334
Partially fixes #4175 (particularly #4175 (comment)), and may also address microsoft/tfs-cli#118 and apigee/microgateway-core#9 without forcing them to recompile node.