deps: upgrade npm to 3.5.0

[othiym23]

Contains the changes in these releases:

• https://github.com/npm/npm/releases/tag/v3.4.0
• https://github.com/npm/npm/releases/tag/v3.4.1
• https://github.com/npm/npm/releases/tag/v3.5.0

Most notably, it includes the clarified split between Terms of Service and the CLI's own license (see the discussion in https://github.com/npm/npm/releases/tag/v3.5.1 and #3959 for more context). It also includes several important changes to how edge cases involving bundleDependencies are handled.

Unfortunately, a few tests are failing when run against the Node build produced by current master, so some mixture of my, @iarna's, and probably @Fishrock123's time will be involved to sort out where those errors are coming from. Because there was some urgency expressed on the part of some folks on the TSC around the new licensing terms, I'm sending this PR anyway. We can pick up the discussion of the technical details next week, after the long US weekend.

r: @Fishrock123
r: @jasnell
r: @mikeal

[Fishrock123]

Note:

warning: squelched 771 whitespace errors
warning: 591 lines applied after fixing whitespace errors

[othiym23]

I'll rebase with the whitespace fix either tonight or early next week. Sorry! I was in a bit of a hurry to get this up before the weekend.

[Fishrock123]

Getting these errors:

test/tap/circular-dep.js ............................. 9/10
  installing a package that depends on the current package
  not ok no error output
    at:
      file: test/tap/circular-dep.js
      line: 48
      column: 9
    stack: |
      test/tap/circular-dep.js:48:9
      f (node_modules/once/once.js:17:25)
      ChildProcess.<anonymous> (test/common-tap.js:56:5)
      maybeClose (internal/child_process.js:819:16)
      Socket.<anonymous> (internal/child_process.js:320:11)
      Pipe._onclose (net.js:470:12)

test/tap/gently-rm-cmdshims.js ........................ 4/6
  remove-cmd-shims
  not ok cmd-shim was removed
    +++ found                                                          
    --- wanted                                                         
    -"ENOENT"                                                          
    +[null]                                                            
    compare: ===
    at:
      file: test/tap/gently-rm-cmdshims.js
      line: 133
      column: 18
    source: |
      fs.stat(doremove_example_cmd, function (er) {
    stack: |
      test/tap/gently-rm-cmdshims.js:133:18
      FSReqWrap.oncomplete (fs.js:83:15)

  remove-cmd-shims
  not ok cmd-shim cygwin script was removed
    +++ found                                                          
    --- wanted                                                         
    -"ENOENT"                                                          
    +[null]                                                            
    compare: ===
    at:
      file: test/tap/gently-rm-cmdshims.js
      line: 136
      column: 18
    source: |
      fs.stat(doremove_example_cygwin, function (er) {
    stack: |
      test/tap/gently-rm-cmdshims.js:136:18
      FSReqWrap.oncomplete (fs.js:83:15)

This last error is different though. @othiym23 was this one added recently?

I experienced this same issue with Kat and the 2.x PRs. This error is because node is built from source and is versioned as 6.0.0-pre. Really, this is not reliable in a test environment and I don't think it should be causing the tests to fail in any way.

test/tap/prune.js ................................... 16/17
  npm install
  not ok Should not get data on stderr: WARN engine underscore@1.3.1: wanted: {"node":"*"} (current: {"node":"6.0.0-pre","npm":"3.5.0"})
    at:
      file: test/tap/prune.js
      line: 57
      column: 7
    stack: |
      test/tap/prune.js:57:7
      f (node_modules/once/once.js:17:25)
      ChildProcess.<anonymous> (test/common-tap.js:56:5)
      maybeClose (internal/child_process.js:819:16)
      Process.ChildProcess._handle.onexit (internal/child_process.js:212:5)

[othiym23]

@Fishrock123 those are the same test failures I was seeing, which aren't happening with 5.0.0 or Node 4 LTS. Like I said in the PR, we'll need to dig into this, probably next week, because I have no idea what's causing any of those failures and didn't have time to investigate.

You, @zkat, and I should get on a hangout soon to talk about how to make the tests work better with -pre builds. Kat and I talked through, what we'd need to do write the tests in such a way that they can handle the warning output coming from engine validation failures. It's a fair amount of work, but we're redoing npm's test suite anyway right now, so now's the time.

[othiym23]

@Fishrock123 Rebased, fixed, and generally cleaned up (the API documentation managed to sneak in yet again). We'll deal with the tests next week!

[Fishrock123]

Commits in master vs v5.x:

• [93739f48ff] - buffer: default to UTF8 in byteLength() (Tom Gallacher) #4010
• [22478d3669] - buffer: move checkFloat from lib into src (Matt Loring) #3763
• [a881b53954] - (SEMVER-MINOR) build,src: add Intel Vtune profiling support (Chunyang Dai) #3785
• [91ccbf0201] - configure: v8_use_snapshot should be true (Fedor Indutny) #3962
• [a5cce79ec3] - (SEMVER-MAJOR) console: delete timers that have ended (Vladimir Varankin) #3562
• [c83d9b7065] - crypto: update root certificates (Ben Noordhuis) #3951
• [56a2b9a246] - crypto: disable crypto.createCipher in FIPS mode (Stefan Budeanu) #3754
• [b33e9da8f9] - debugger: introduce exec method for debugger (Jackson Tian)
• [ab25589f59] - deps: backport 819b40a from V8 upstream (Michaël Zasso) #3937
• [6f87aa9a12] - doc: message.header duplication correction (Bryan English) #3997
• [8d37bbe9e9] - doc: fix typo in README (Rich Trott) #4000
• [3becac2e60] - doc: replace sane with reasonable (Lewis Cowper) #3980
• [d16def5326] - doc: Adding best practises for crypto.pbkdf2 (Tom Gallacher) #3290
• [615fba3bcf] - doc: numeric flags to fs.open (Carl Lei) #3641
• [3f4409e571] - doc: clarify that fs streams expect blocking fd (Carl Lei) #3641
• [94c3507f5c] - doc: fix broken references (Alexander Gromnitsky) #3944
• [55b1eccae6] - doc: move fs.existsSync() deprecation message (Martin Forsberg) #3942
• [5900d14966] - doc: clarify module loading behavior (cjihrig) #3920
• [5f3ed61094] - doc: add reference for buffer.inspect() (cjihrig) #3921
• [2e38079ea4] - docs: fs - remove encoding list and link to buffer (fansworld-claudio)
• [8b97249893] - (SEMVER-MAJOR) fs: fix the error report on fs.link(Sync) (yorkie) #3917
• [b2e7a4d479] - installer: install the tick processor (Matt Loring) #3032
• [d01eb6882f] - lib: add 'pid' prefix in internal/util (Minwoo Jung) #3878
• [20285ad177] - (SEMVER-MAJOR) lib: Consistent error messages in all modules (micnic) #3374
• [b70dc67828] - (SEMVER-MAJOR) lib,test: remove publicly exposed freelist (cjihrig) #3738
• [dfee4e3712] - module: fix column offsets in errors (Tristian Flanagan) #2867
• [d7b199d9e3] - net: add local address/port for better errors (Jan Schär) #3946
• [9472a0cfad] - net: small code cleanup (Jan Schär) #3943
• [ca2e8b292f] - (SEMVER-MAJOR) readline: deprecate undocumented exports (cjihrig) #3862
• [459b106d6c] - repl: attach location info to syntax errors (cjihrig) #4013
• [2ccde01980] - src: add BE support to StringBytes::Encode() (Bryon Leung) #3410
• [16db4c4d5e] - test: mark cluster-net-send test flaky on windows (Rich Trott) #4006
• [fc47e0f8cb] - test: mark fork regression test flaky on windows (Rich Trott) #4005
• [b061e3af55] - test: skip test if in FreeBSD jail (Rich Trott) #3995
• [658842280d] - test: remove flaky status for cluster test (Rich Trott) #3975
• [41519fd1a4] - test: add TAP diagnostic message for retried tests (Rich Trott) #3960
• [8bc8038687] - test: retry on smartos if ECONNREFUSED (Rich Trott) #3941
• [487de19684] - test: address flaky test-http-client-timeout-event (Rich Trott) #3968
• [804cc7670d] - test: numeric flags to fs.open (Carl Lei) #3641
• [174d4e484e] - test: http complete list of non-concat headers (Bryan English) #3930
• [6de82c69a0] - test: fix race condition in unrefd interval test (Michael Cornacchia) #3550
• [e499ea849c] - test: skip/replace weak crypto tests in FIPS mode (Stefan Budeanu) #3757
• [e4e5b13efd] - test: avoid test timeouts on rpi (Stefan Budeanu) #3902
• [ac319c3547] - test: fix flaky test-child-process-spawnsync-input (Rich Trott) #3889
• [339d3840c8] - test: add OS X to module loading error test (Evan Lucas) #3901
• [9a628e2dac] - test: module loading error fix solaris node5: test-module-loading-error.js fails #3798 (fansworld-claudio) #3855
• [2c33819370] - (SEMVER-MAJOR) test: fix tests that check error messages (cjihrig) #3727
• [df268f97bc] - (SEMVER-MAJOR) tls: use SHA1 for sessionIdContext (Stefan Budeanu) #3866
• [8c4deff55d] - tools: update certdata.txt (Ben Noordhuis) #3951
• [a2c0aa84e0] - (SEMVER-MAJOR) tty: Remove deprecated setRawMode wrapper (Wyatt Preul) #2528
• [8ca412b7a3] - util: add decorateErrorStack() (cjihrig) #4013
• [007cfea308] - (SEMVER-MAJOR) util: remove pump (Wyatt Preul) #2531
• [4cf19ad1bb] - (SEMVER-MAJOR) util: Remove exec, has been deprecated for years (Wyatt Preul) #2530
• [34a35919e1] - (SEMVER-MAJOR) util: improve typed array formatting (Ben Noordhuis) #3793
• [5169311bf9] - util,src: allow lookup of hidden values (cjihrig) #3988

[Fishrock123]

@othiym23 @iarna please let me know when there is time to work on this.

[Martii]

@Fishrock123
This is a much needed leveraging of Copyright...

[Fishrock123]

Um, pardon? You mean All rights reserved. needs to remain?

[Martii]

@Fishrock123
I'll need to examine the full document but GH won't allow me to comment on lines that are too far out from the diff.

Basically npm should be able to define Terms of Service for npmjs.com and Terms of Use for the Software... spelling out exactly which is which and no "npm" (only) references. I know this directly from contractual obligations with closed-source agreements.

I have no difficultly with accepting a TOS for npmjs.com as it is their right to prevent bugged modules from being distributed being a pass through provider as long as it's clear in their TOS. With the FSF though as @scriptjs has pointed out the actual Code License hopefully continues to be FOSS. (Their TOU)

[Martii]

You mean All rights reserved. needs to remain?

No... that would not be leveraging the Copyright... which means node couldn't distribute it period as a module without a written agreement on paper registered with a legal authority... that costs $$ to get it registered and pretty much defeats the concept of FOSS.

[Martii]

@Fishrock123
Going exclusively off this PR's topic.. specific to the 3.5.0 part I recommend against merging that version... e.g. 3.5.1 (@next as of yesterday) appears less confusing as I dealt with @othiym23 very recently... and I'd rather not see node and npm involved with a slew of issue reports from others regarding some confusion bugs that were fixed in the next version (at least there is a reference point if someone asks that question to do some "light reading")... somewhere along the roller coaster it should be updated though to reflect the corrected licensing.

[othiym23]

@Fishrock I wanted to get to this today, but ran out of time first. I'll be digging into this first thing tomorrow.

[Fishrock123]

@othiym23 Sure thing. Just as a note I'll probably be busy from 12PM your time onwards though.

[Fishrock123]

@othiym23 If you have some instructions for running individual test files I can try to do a bisect.

[othiym23]

@Fishrock123 You can just run the individual tests with Node, and they all produce tap output.

[Fishrock123]

Looks like the tests fail on v5.x, I had previously thought it was an issue on master but it does not appear to be.

[mhdawson]

I'm waiting on npm 3.5.1 landing in master to re-test for AIX. From the earlier comments sounds like the plan may be merge that instead. @Fishrock123 is that correct ?

[mhdawson]

@Fishrock123 is the process for merging/testing a new npm level documented somewhere ? I did a straight copy of 3.5.3 into a master checkout and a CI run looks clean (https://ci.nodejs.org/job/node-test-commit/1751/), but from the discussion I can tell you are also running the npm tests which I don't see a CI job for, and a straight copy might not be the way we merge.

[medns]

I'm waiting on npm > 3.5 landing in v5.x version

[Fishrock123]

I'm not really sure.

@nodejs/npm could you open a new, updated PR we can look at for test failures?

[othiym23]

@Fishrock123 Sorry for the slow re-emergence from end of the year doldrums and lack of communication, but @iarna has been working on fixing npm's tests to work properly with prerelease versions of Node (and making npm just generally less noisy in that case). Expect to see a new PR containing those changes, but probably not until we get that version of npm released, in 10 days or so.

[Fishrock123]

@othiym23 Awesome, thanks for the update.

[rvagg]

@iarna has been working on fixing npm's tests to work properly with prerelease versions of Node (and making npm just generally less noisy in that case)

👯 yay! thanks for this, will go well with our new canary-ish strategy coming soon

[iarna]

This is being superseded by a PR for 3.6.0, which I'll be creating soon, as such I'm going to close this now.

[rvagg]

Thanks @iarna, if you can remember to run tools/license-builder.sh on top of 3.6.0 when it comes in to ensure we have the latest license text in LICENSE, then, combined with the change Kat just merged, we should be able to square away the licensing stuff on Node's end.

[iarna]

Sure will! =) Just waiting on a final test run to PR.