yarn kbn bootstrap failed, behind a corporate proxy!

[azulay7]

Hi all :)
when I run kibana dev mode from source code.
"yarn kbn bootstrap".

I get an errors:

OS: windows 10
node: v8.11.3
yarn: 1.7.0

Thank you all in Advance!

[liza-mae]

Hi @azulay7 can you try with yarn 1.6.0 and see if the problem goes away.

[liza-mae]

Here are some reference links I found: yarnpkg/yarn#5876 and yarnpkg/yarn#6016 on this problem with yarn 1.7.0.

[azulay7]

Hey, I've install yarn 1.6.0 but got another error.. :(
error C:\dev\kibana\node_modules\geckodriver: Command failed.

RequestError: self signed certificate in certificate chain

[liza-mae]

@azulay7 I have not seen that error before :)

cc: @spalger do you think you can help ?

[liza-mae]

If you did a yarn kbn clean and then tried yarn kbn bootstrap again, does that work ?

[spalger]

"self signed certificate in certificate chain" seems a little bonkers... never seen that before, have you ever seen that before when trying to install node modules @azulay7?

[azulay7]

@liza-mae yarn kbn clean didn't help

[ytzlax]

Hi @azulay7 ,good to see you here.
I had the same issue,and I resolved the problem.
This is what you need to do:

1. Install yarn 1.6.0
2. Remove node_modules directory
3. Go to kibana directory and type "yarn kbn bootstrap".

[azulay7]

@ytzlax didn't work :(

[timroes]

@azulay7 Are you running behind a proxy? That error usually means, someone (like a corporate proxy) is intercepting your SSL traffic and resigning it with their own certificate.

[azulay7]

@timroes You are right! Yes I'm behind a corporate proxy
my npm configure with:
npm config set strict-ssl false

but even when I get it back to true, I get same failure :(

i have tried on other machine on AZURE, without the proxy, and it worked.

[timroes]

@azulay7 That's nothing that we could solve on the Kibana side, but since I've been in the same situation in the past, lemme shortly explain what happens here:

tl;dr: Modern web development is NOT possible behind corporate proxies.

The main problem here, is that there is no global "node" proxy settings or something like strict-ssl on a node level. So you can configure something like strict-ssl or a proxy in your npm config, and it will effect download of packages via npm. (Unfortunately) every package can have a postinstall script in its package.json which will be executed after installing the package.

Quiet some packages (in that case it seems to be chromedriver) use that mechanism to execute a script, that downloads a platform dependent binary for that specific OS and architecture. The script executed is often just another Node JavaScript script, that will be executed and uses e.g. the node HTTP API to download the binary. And there we'll now hit the problem, that there is no such thing as a global Node proxy. This request won't even use the proxy, if that postinstall script isn't intentionally looking up some proxy settings (most reasonable from the npm config, but maybe also http_proxy environment variables). If the script doesn't do it, it just won't use any proxy, and there is nearly nothing you can do against it. The same of course applies to something like strict-ssl setting. If the postinstall script doesn't manually look up that setting and ignores SSL errors in case it's enabled, you cannot do anything against it.

Is there any possible solution? There are potential solutions around the problem that a script completely ignores a proxy. For different platforms there are tools like Proxifier (OS X and Windows) or redsocks (Linux), which basically collect all the traffic outgoing from the system and forward it to a proxy configured in that tool. That way you could potentially forward all traffic from a system (no matter a postinstall script is using npm proxy setting or not) onto the proxy.

But that won't help you at all for the proxy intercepting SSL. So the only thing you can do there is: open a PR against every package that fails, so that they will read out the strict-ssl setting in their postinstall script and take it into consideration in case they hit SSL errors. And then of course wait for that fix to be released and all the downstream project to be updated to that new version.

The only other way you can truly workaround that: You need to make clear, that modern web development doesn't work behind a corporate proxy breaking SSL, since even if npm supports such settings to ignore SSL errors (and let's not even talk about that danger of setting that flag), you never know if you won't hit a postinstall script that doesn't manually read out those settings and use them (in lack of a central "node proxy and SSL setting"). In companies I've seen in the past, the only actual workaround was: don't break SSL at your proxy or get a separated developer network, that doesn't try to intercept requests. In case you are jailed behind that corporate proxy breaking SSL, I fear nothing you can do will truly solve the problem, just make it potentially working for now ... until any library you/we are using adds a new postinstall script ignoring that settings again.

[azulay7]

@timroes WOW!! thank you for the detailed answer!

[Ardhen98]

Hi @azulay7 ,good to see you here.
I had the same issue,and I resolved the problem.
This is what you need to do:

1. Install yarn 1.6.0
2. Remove node_modules directory
3. Go to kibana directory and type "yarn kbn bootstrap".

Hello, where is node_modules directory located?

[nimaansary]

@azulay7 That's nothing that we could solve on the Kibana side, but since I've been in the same situation in the past, lemme shortly explain what happens here:

tl;dr: Modern web development is NOT possible behind corporate proxies.

The main problem here, is that there is no global "node" proxy settings or something like strict-ssl on a node level. So you can configure something like strict-ssl or a proxy in your npm config, and it will effect download of packages via npm. (Unfortunately) every package can have a postinstall script in its package.json which will be executed after installing the package.

Quiet some packages (in that case it seems to be chromedriver) use that mechanism to execute a script, that downloads a platform dependent binary for that specific OS and architecture. The script executed is often just another Node JavaScript script, that will be executed and uses e.g. the node HTTP API to download the binary. And there we'll now hit the problem, that there is no such thing as a global Node proxy. This request won't even use the proxy, if that postinstall script isn't intentionally looking up some proxy settings (most reasonable from the npm config, but maybe also http_proxy environment variables). If the script doesn't do it, it just won't use any proxy, and there is nearly nothing you can do against it. The same of course applies to something like strict-ssl setting. If the postinstall script doesn't manually look up that setting and ignores SSL errors in case it's enabled, you cannot do anything against it.

Is there any possible solution? There are potential solutions around the problem that a script completely ignores a proxy. For different platforms there are tools like Proxifier (OS X and Windows) or redsocks (Linux), which basically collect all the traffic outgoing from the system and forward it to a proxy configured in that tool. That way you could potentially forward all traffic from a system (no matter a postinstall script is using npm proxy setting or not) onto the proxy.

But that won't help you at all for the proxy intercepting SSL. So the only thing you can do there is: open a PR against every package that fails, so that they will read out the strict-ssl setting in their postinstall script and take it into consideration in case they hit SSL errors. And then of course wait for that fix to be released and all the downstream project to be updated to that new version.

The only other way you can truly workaround that: You need to make clear, that modern web development doesn't work behind a corporate proxy breaking SSL, since even if npm supports such settings to ignore SSL errors (and let's not even talk about that danger of setting that flag), you never know if you won't hit a postinstall script that doesn't manually read out those settings and use them (in lack of a central "node proxy and SSL setting"). In companies I've seen in the past, the only actual workaround was: don't break SSL at your proxy or get a separated developer network, that doesn't try to intercept requests. In case you are jailed behind that corporate proxy breaking SSL, I fear nothing you can do will truly solve the problem, just make it potentially working for now ... until any library you/we are using adds a new postinstall script ignoring that settings again.

dear friend, You are absolutely right, but I do not understand why a proxy can not replace free internet