Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Error: write after end when installing packages with npm 5.8.0 or higher #180

Closed
jbcpollak opened this issue May 16, 2018 · 5 comments
Closed

Error: write after end when installing packages with npm 5.8.0 or higher #180

jbcpollak opened this issue May 16, 2018 · 5 comments

Comments

@jbcpollak
Copy link

Hi, I've been working with @zkat on an npm issue which appears to have a root cause in node-tar. The stack trace I am seeing is the following. We have been able to reproduce this with any version of npm > 5.8.0. (we have not tested with npm 5.7.x)

❯ rm -rf /tmp/empty-cache && rm -rf node_modules && npx npmc install --cache /tmp/empty-cache
npx: installed 763 in 28.723s
npm notice CANARY npmc is experimental software. If you find an issue, please file it in the main npm repository, and call out that you were using npmc.
/Users/jpollak/.npm/_npx/99478/lib/node_modules/npmc/node_modules/minipass/index.js:81
      throw new Error('write after end')
      ^

Error: write after end
    at MiniPass.write (/Users/jpollak/.npm/_npx/99478/lib/node_modules/npmc/node_modules/minipass/index.js:81:13)
    at ReadEntry.entry.on (/Users/jpollak/.npm/_npx/99478/lib/node_modules/npmc/node_modules/pacote/lib/extract-stream.js:30:21)
    at ReadEntry.emit (events.js:185:15)
    at ReadEntry.emit (/Users/jpollak/.npm/_npx/99478/lib/node_modules/npmc/node_modules/tar/node_modules/minipass/index.js:296:25)
    at ReadEntry.[maybeEmitEnd] (/Users/jpollak/.npm/_npx/99478/lib/node_modules/npmc/node_modules/tar/node_modules/minipass/index.js:249:12)
    at ReadEntry.end (/Users/jpollak/.npm/_npx/99478/lib/node_modules/npmc/node_modules/tar/node_modules/minipass/index.js:162:27)
    at Unpack.[consumeBody] (/Users/jpollak/.npm/_npx/99478/lib/node_modules/npmc/node_modules/tar/lib/parse.js:210:13)
    at Unpack.[consumeChunkSub] (/Users/jpollak/.npm/_npx/99478/lib/node_modules/npmc/node_modules/tar/lib/parse.js:391:40)
    at Unpack.[consumeChunk] (/Users/jpollak/.npm/_npx/99478/lib/node_modules/npmc/node_modules/tar/lib/parse.js:360:30)
    at Unzip.(anonymous function).on.chunk (/Users/jpollak/.npm/_npx/99478/lib/node_modules/npmc/node_modules/tar/lib/parse.js:291:59)
    at Unzip.emit (events.js:180:13)
    at Unzip.emit (/Users/jpollak/.npm/_npx/99478/lib/node_modules/npmc/node_modules/tar/node_modules/minipass/index.js:296:25)
    at Unzip.write (/Users/jpollak/.npm/_npx/99478/lib/node_modules/npmc/node_modules/tar/node_modules/minipass/index.js:99:17)
    at Unzip.write (/Users/jpollak/.npm/_npx/99478/lib/node_modules/npmc/node_modules/tar/node_modules/minizlib/index.js:284:29)
    at Unpack.write (/Users/jpollak/.npm/_npx/99478/lib/node_modules/npmc/node_modules/tar/lib/parse.js:307:19)
    at PassThrough.ondata (_stream_readable.js:651:20)
/Users/jpollak/.npm/_npx/99478/lib/node_modules/npmc/node_modules/minipass/index.js:81auth@0.9.7
      throw new Error('write after end')
      ^
@jbcpollak jbcpollak mentioned this issue May 16, 2018
Closed
8 tasks
@zkat
Copy link
Contributor

zkat commented May 16, 2018

@isaacs for context, this started happening for some people, sometimes, when working off network-sourced tarballs (rather than cached ones), and it's happening in the transform callback for pacote's extract-stream. The error was first reported when it was using PassThrough, but I changed it to use Minipass and stopped using the .end(....data...) format for last writes just in case, and shipped a canary for @jbcpollak to try, and it still reproed.

@zkat zkat added the bug label May 16, 2018
@jbcpollak
Copy link
Author

One small clarification - for us this happens every time we do fresh install of our projects. We have a bootstrap script for new developers that checks out all our repos and runs npm i for them - it fails every time, for every developer with this error. I'm not sure if its always the same package - so I suppose it could be random, but we must have enough dependencies in our projects that it always fails.

@jakeNiemiec jakeNiemiec mentioned this issue May 17, 2018
Open
@DevDwarf DevDwarf mentioned this issue May 17, 2018
Closed
This was referenced May 18, 2018
Merged
Closed
isaacs added a commit to isaacs/minipass that referenced this issue May 18, 2018
@isaacs
Should not allow 'end' events to happen more than once
55ed167 
Likely root cause of:
zkat/pacote#151
isaacs/node-tar#180
laravel/homestead#866
@isaacs
Copy link
Owner

isaacs commented May 24, 2018

Fixed on latest pacote, in npm 6.1.

@isaacs isaacs closed this as completed May 24, 2018
@ghost
Copy link

ghost commented Jun 12, 2018 •
edited by ghost
Loading

This issue is reproduced on macos

$ npm doctor

Check                               Value                        Recommendation
npm ping                            OK
npm -v                              v6.1.0
node -v                             v10.3.0
npm config get registry             https://registry.npmjs.org/
which git                           /usr/bin/git
Perms check on cached files         ok
Perms check on global node_modules  ok
Perms check on local node_modules   ok
Verify cache contents               verified 2 tarballs

installation log

Error [ERR_STREAM_WRITE_AFTER_END]: write after end
    at writeAfterEnd (_stream_writable.js:243:12)
    at PassThrough.Writable.write (_stream_writable.js:291:5)
    at PassThrough.Writable.end (_stream_writable.js:579:10)
    at ReadEntry.entry.on (/usr/local/lib/node_modules/npm/node_modules/libcipm/node_modules/pacote/lib/extract-stream.js:19:41)
    at ReadEntry.emit (events.js:187:15)
    at ReadEntry.emit (/usr/local/lib/node_modules/npm/node_modules/tar/node_modules/minipass/index.js:301:25)
    at ReadEntry.[maybeEmitEnd] (/usr/local/lib/node_modules/npm/node_modules/tar/node_modules/minipass/index.js:252:12)
    at ReadEntry.end (/usr/local/lib/node_modules/npm/node_modules/tar/node_modules/minipass/index.js:165:27)
    at Unpack.[consumeBody] (/usr/local/lib/node_modules/npm/node_modules/tar/lib/parse.js:210:13)
    at Unpack.[consumeChunkSub] (/usr/local/lib/node_modules/npm/node_modules/tar/lib/parse.js:391:40)
    at Unpack.[consumeChunk] (/usr/local/lib/node_modules/npm/node_modules/tar/lib/parse.js:362:30)
    at Unzip.(anonymous function).on.chunk (/usr/local/lib/node_modules/npm/node_modules/tar/lib/parse.js:291:59)
    at Unzip.emit (events.js:182:13)
    at Unzip.emit (/usr/local/lib/node_modules/npm/node_modules/tar/node_modules/minipass/index.js:301:25)
    at Unzip.write (/usr/local/lib/node_modules/npm/node_modules/tar/node_modules/minipass/index.js:102:17)
    at Unzip.write (/usr/local/lib/node_modules/npm/node_modules/tar/node_modules/minizlib/index.js:284:29)
Emitted 'error' event at:
    at writeAfterEnd (_stream_writable.js:245:10)
    at PassThrough.Writable.write (_stream_writable.js:291:5)
    [... lines matching original stack trace ...]
    at Unzip.write (/usr/local/lib/node_modules/npm/node_modules/tar/node_modules/minizlib/index.js:284:29)
events.js:167
      throw er; // Unhandled 'error' event
      ^

Error [ERR_STREAM_WRITE_AFTER_END]: write after end
    at writeAfterEnd (_stream_writable.js:243:12)
    at PassThrough.Writable.write (_stream_writable.js:291:5)
    at PassThrough.Writable.end (_stream_writable.js:579:10)
    at ReadEntry.entry.on (/usr/local/lib/node_modules/npm/node_modules/libcipm/node_modules/pacote/lib/extract-stream.js:19:41)
    at ReadEntry.emit (events.js:187:15)
    at ReadEntry.emit (/usr/local/lib/node_modules/npm/node_modules/tar/node_modules/minipass/index.js:301:25)
    at ReadEntry.[maybeEmitEnd] (/usr/local/lib/node_modules/npm/node_modules/tar/node_modules/minipass/index.js:252:12)
    at ReadEntry.end (/usr/local/lib/node_modules/npm/node_modules/tar/node_modules/minipass/index.js:165:27)
    at Unpack.[consumeBody] (/usr/local/lib/node_modules/npm/node_modules/tar/lib/parse.js:210:13)
    at Unpack.[consumeChunkSub] (/usr/local/lib/node_modules/npm/node_modules/tar/lib/parse.js:391:40)
    at Unpack.[consumeChunk] (/usr/local/lib/node_modules/npm/node_modules/tar/lib/parse.js:362:30)
    at Unzip.(anonymous function).on.chunk (/usr/local/lib/node_modules/npm/node_modules/tar/lib/parse.js:291:59)
    at Unzip.emit (events.js:182:13)
    at Unzip.emit (/usr/local/lib/node_modules/npm/node_modules/tar/node_modules/minipass/index.js:301:25)
    at Unzip.write (/usr/local/lib/node_modules/npm/node_modules/tar/node_modules/minipass/index.js:102:17)
    at Unzip.write (/usr/local/lib/node_modules/npm/node_modules/tar/node_modules/minizlib/index.js:284:29)
Emitted 'error' event at:
    at writeAfterEnd (_stream_writable.js:245:10)
    at PassThrough.Writable.write (_stream_writable.js:291:5)
    [... lines matching original stack trace ...]
    at Unzip.write (/usr/local/lib/node_modules/npm/node_modules/tar/node_modules/minizlib/index.js:284:29)
events.js:167
      throw er; // Unhandled 'error' event
      ^

Error [ERR_STREAM_WRITE_AFTER_END]: write after end
    at writeAfterEnd (_stream_writable.js:243:12)
    at PassThrough.Writable.write (_stream_writable.js:291:5)
    at PassThrough.Writable.end (_stream_writable.js:579:10)
    at ReadEntry.entry.on (/usr/local/lib/node_modules/npm/node_modules/libcipm/node_modules/pacote/lib/extract-stream.js:19:41)
    at ReadEntry.emit (events.js:187:15)
    at ReadEntry.emit (/usr/local/lib/node_modules/npm/node_modules/tar/node_modules/minipass/index.js:301:25)
    at ReadEntry.[maybeEmitEnd] (/usr/local/lib/node_modules/npm/node_modules/tar/node_modules/minipass/index.js:252:12)
    at ReadEntry.end (/usr/local/lib/node_modules/npm/node_modules/tar/node_modules/minipass/index.js:165:27)
    at Unpack.[consumeBody] (/usr/local/lib/node_modules/npm/node_modules/tar/lib/parse.js:210:13)
    at Unpack.[consumeChunkSub] (/usr/local/lib/node_modules/npm/node_modules/tar/lib/parse.js:391:40)
    at Unpack.[consumeChunk] (/usr/local/lib/node_modules/npm/node_modules/tar/lib/parse.js:362:30)
    at Unzip.(anonymous function).on.chunk (/usr/local/lib/node_modules/npm/node_modules/tar/lib/parse.js:291:59)
    at Unzip.emit (events.js:182:13)
    at Unzip.emit (/usr/local/lib/node_modules/npm/node_modules/tar/node_modules/minipass/index.js:301:25)
    at Unzip.write (/usr/local/lib/node_modules/npm/node_modules/tar/node_modules/minipass/index.js:102:17)
    at Unzip.write (/usr/local/lib/node_modules/npm/node_modules/tar/node_modules/minizlib/index.js:284:29)
Emitted 'error' event at:
    at writeAfterEnd (_stream_writable.js:245:10)
    at PassThrough.Writable.write (_stream_writable.js:291:5)
    [... lines matching original stack trace ...]
    at Unzip.write (/usr/local/lib/node_modules/npm/node_modules/tar/node_modules/minizlib/index.js:284:29)
events.js:167
      throw er; // Unhandled 'error' event
      ^

Error [ERR_STREAM_WRITE_AFTER_END]: write after end
    at writeAfterEnd (_stream_writable.js:243:12)
    at PassThrough.Writable.write (_stream_writable.js:291:5)
    at PassThrough.Writable.end (_stream_writable.js:579:10)
    at ReadEntry.entry.on (/usr/local/lib/node_modules/npm/node_modules/libcipm/node_modules/pacote/lib/extract-stream.js:19:41)
    at ReadEntry.emit (events.js:187:15)
    at ReadEntry.emit (/usr/local/lib/node_modules/npm/node_modules/tar/node_modules/minipass/index.js:301:25)
    at ReadEntry.[maybeEmitEnd] (/usr/local/lib/node_modules/npm/node_modules/tar/node_modules/minipass/index.js:252:12)
    at ReadEntry.end (/usr/local/lib/node_modules/npm/node_modules/tar/node_modules/minipass/index.js:165:27)
    at Unpack.[consumeBody] (/usr/local/lib/node_modules/npm/node_modules/tar/lib/parse.js:210:13)
    at Unpack.[consumeChunkSub] (/usr/local/lib/node_modules/npm/node_modules/tar/lib/parse.js:391:40)
    at Unpack.[consumeChunk] (/usr/local/lib/node_modules/npm/node_modules/tar/lib/parse.js:362:30)
    at Unzip.(anonymous function).on.chunk (/usr/local/lib/node_modules/npm/node_modules/tar/lib/parse.js:291:59)
    at Unzip.emit (events.js:182:13)
    at Unzip.emit (/usr/local/lib/node_modules/npm/node_modules/tar/node_modules/minipass/index.js:301:25)
    at Unzip.write (/usr/local/lib/node_modules/npm/node_modules/tar/node_modules/minipass/index.js:102:17)
    at Unzip.write (/usr/local/lib/node_modules/npm/node_modules/tar/node_modules/minizlib/index.js:284:29)
Emitted 'error' event at:
    at writeAfterEnd (_stream_writable.js:245:10)
    at PassThrough.Writable.write (_stream_writable.js:291:5)
    [... lines matching original stack trace ...]
    at Unzip.write (/usr/local/lib/node_modules/npm/node_modules/tar/node_modules/minizlib/index.js:284:29)
events.js:167
      throw er; // Unhandled 'error' event
      ^

Error [ERR_STREAM_WRITE_AFTER_END]: write after end
    at writeAfterEnd (_stream_writable.js:243:12)
    at PassThrough.Writable.write (_stream_writable.js:291:5)
    at PassThrough.Writable.end (_stream_writable.js:579:10)
    at ReadEntry.entry.on (/usr/local/lib/node_modules/npm/node_modules/libcipm/node_modules/pacote/lib/extract-stream.js:19:41)
    at ReadEntry.emit (events.js:187:15)
    at ReadEntry.emit (/usr/local/lib/node_modules/npm/node_modules/tar/node_modules/minipass/index.js:301:25)
    at ReadEntry.[maybeEmitEnd] (/usr/local/lib/node_modules/npm/node_modules/tar/node_modules/minipass/index.js:252:12)
    at ReadEntry.end (/usr/local/lib/node_modules/npm/node_modules/tar/node_modules/minipass/index.js:165:27)
    at Unpack.[consumeBody] (/usr/local/lib/node_modules/npm/node_modules/tar/lib/parse.js:210:13)
    at Unpack.[consumeChunkSub] (/usr/local/lib/node_modules/npm/node_modules/tar/lib/parse.js:391:40)
    at Unpack.[consumeChunk] (/usr/local/lib/node_modules/npm/node_modules/tar/lib/parse.js:362:30)
    at Unzip.(anonymous function).on.chunk (/usr/local/lib/node_modules/npm/node_modules/tar/lib/parse.js:291:59)
    at Unzip.emit (events.js:182:13)
    at Unzip.emit (/usr/local/lib/node_modules/npm/node_modules/tar/node_modules/minipass/index.js:301:25)
    at Unzip.write (/usr/local/lib/node_modules/npm/node_modules/tar/node_modules/minipass/index.js:102:17)
    at Unzip.write (/usr/local/lib/node_modules/npm/node_modules/tar/node_modules/minizlib/index.js:284:29)
Emitted 'error' event at:
    at writeAfterEnd (_stream_writable.js:245:10)
    at PassThrough.Writable.write (_stream_writable.js:291:5)
    [... lines matching original stack trace ...]
    at Unzip.write (/usr/local/lib/node_modules/npm/node_modules/tar/node_modules/minizlib/index.js:284:29)
events.js:167
      throw er; // Unhandled 'error' event
      ^

Error [ERR_STREAM_WRITE_AFTER_END]: write after end
    at writeAfterEnd (_stream_writable.js:243:12)
    at PassThrough.Writable.write (_stream_writable.js:291:5)
    at PassThrough.Writable.end (_stream_writable.js:579:10)
    at ReadEntry.entry.on (/usr/local/lib/node_modules/npm/node_modules/libcipm/node_modules/pacote/lib/extract-stream.js:19:41)
    at ReadEntry.emit (events.js:187:15)
    at ReadEntry.emit (/usr/local/lib/node_modules/npm/node_modules/tar/node_modules/minipass/index.js:301:25)
    at ReadEntry.[maybeEmitEnd] (/usr/local/lib/node_modules/npm/node_modules/tar/node_modules/minipass/index.js:252:12)
    at ReadEntry.end (/usr/local/lib/node_modules/npm/node_modules/tar/node_modules/minipass/index.js:165:27)
    at Unpack.[consumeBody] (/usr/local/lib/node_modules/npm/node_modules/tar/lib/parse.js:210:13)
    at Unpack.[consumeChunkSub] (/usr/local/lib/node_modules/npm/node_modules/tar/lib/parse.js:391:40)
    at Unpack.[consumeChunk] (/usr/local/lib/node_modules/npm/node_modules/tar/lib/parse.js:362:30)
    at Unzip.(anonymous function).on.chunk (/usr/local/lib/node_modules/npm/node_modules/tar/lib/parse.js:291:59)
    at Unzip.emit (events.js:182:13)
    at Unzip.emit (/usr/local/lib/node_modules/npm/node_modules/tar/node_modules/minipass/index.js:301:25)
    at Unzip.write (/usr/local/lib/node_modules/npm/node_modules/tar/node_modules/minipass/index.js:102:17)
    at Unzip.write (/usr/local/lib/node_modules/npm/node_modules/tar/node_modules/minizlib/index.js:284:29)
Emitted 'error' event at:
    at writeAfterEnd (_stream_writable.js:245:10)
    at PassThrough.Writable.write (_stream_writable.js:291:5)
    [... lines matching original stack trace ...]
    at Unzip.write (/usr/local/lib/node_modules/npm/node_modules/tar/node_modules/minizlib/index.js:284:29)
events.js:167
      throw er; // Unhandled 'error' event
      ^

Error [ERR_STREAM_WRITE_AFTER_END]: write after end
    at writeAfterEnd (_stream_writable.js:243:12)
    at PassThrough.Writable.write (_stream_writable.js:291:5)
    at PassThrough.Writable.end (_stream_writable.js:579:10)
    at ReadEntry.entry.on (/usr/local/lib/node_modules/npm/node_modules/libcipm/node_modules/pacote/lib/extract-stream.js:19:41)
    at ReadEntry.emit (events.js:187:15)
    at ReadEntry.emit (/usr/local/lib/node_modules/npm/node_modules/tar/node_modules/minipass/index.js:301:25)
    at ReadEntry.[maybeEmitEnd] (/usr/local/lib/node_modules/npm/node_modules/tar/node_modules/minipass/index.js:252:12)
    at ReadEntry.end (/usr/local/lib/node_modules/npm/node_modules/tar/node_modules/minipass/index.js:165:27)
    at Unpack.[consumeBody] (/usr/local/lib/node_modules/npm/node_modules/tar/lib/parse.js:210:13)
    at Unpack.[consumeChunkSub] (/usr/local/lib/node_modules/npm/node_modules/tar/lib/parse.js:391:40)
    at Unpack.[consumeChunk] (/usr/local/lib/node_modules/npm/node_modules/tar/lib/parse.js:362:30)
    at Unzip.(anonymous function).on.chunk (/usr/local/lib/node_modules/npm/node_modules/tar/lib/parse.js:291:59)
    at Unzip.emit (events.js:182:13)
    at Unzip.emit (/usr/local/lib/node_modules/npm/node_modules/tar/node_modules/minipass/index.js:301:25)
    at Unzip.write (/usr/local/lib/node_modules/npm/node_modules/tar/node_modules/minipass/index.js:102:17)
    at Unzip.write (/usr/local/lib/node_modules/npm/node_modules/tar/node_modules/minizlib/index.js:284:29)
Emitted 'error' event at:
    at writeAfterEnd (_stream_writable.js:245:10)
    at PassThrough.Writable.write (_stream_writable.js:291:5)
    [... lines matching original stack trace ...]
    at Unzip.write (/usr/local/lib/node_modules/npm/node_modules/tar/node_modules/minizlib/index.js:284:29)

npm ERR! A complete log of this run can be found in:
npm ERR!     /root/.npm/_logs/2018-06-12T10_11_27_506Z-debug.log
lerna ERR! npm ci exited 1 in 'root'
lerna ERR! npm ci exited 1 in 'root'

@VvorkCloudTechnologies
Copy link

no such file or directory, open '/home/g3006761319ishan/vvork-website/node_modules/.staging/core-js-aaa88aee/client/library.min.js.map'

i'm face the same issue

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
None yet
Projects
None yet
Milestone
No milestone
Development

No branches or pull requests
4 participants
@isaacs @zkat @jbcpollak @VvorkCloudTechnologies