[BUG] npm publish ignores symbolic links

[matheusmoreira]

Is there an existing issue for this?

• I have searched the existing issues

This issue exists in the latest npm version

• I am using the latest npm

Current Behavior

I have a monorepo with several packages. One of the packages is the main package. I have symlinked its README.md file to the file in the root repository.

README.md -> ../../README.md
LICENSE.MIT
cli.js
index.js
package.json

However, when I tried to publish the package, npm did not include the README file:

=== Tarball Contents ===
LICENSE.MIT
cli.js
index.js
package.json

Expected Behavior

=== Tarball Contents ===
README.md
LICENSE.MIT
cli.js
index.js
package.json

Steps To Reproduce

npm init
touch test
ln -s test README.md
npm publish --dry-run

Environment

• npm: 9.8.1
• Node.js: v20.2.0
• OS Name: Arch Linux
• npm config:

foreground-scripts = true

; node bin location = /usr/bin/node
; node version = v20.2.0
; npm local prefix = /home/matheus/dev/npm-test
; npm version = 9.8.1
; cwd = /home/matheus/dev/npm-test
; HOME = /home/matheus

[ljharb]

I believe this has always been the case; symlinks must be reified as the actual files pre publish.

[Iisus]

Same issue as #6744?

[matheusmoreira]

@Iisus Almost. That person expected the symbolic link itself to be packaged in the tarball. I expected npm to transparently read the file pointed to by the symbolic link and package the actual file into the tarball.

[shellscape]

There are issues around this going back to 2013. Pretty wild a fix was never implemented. And every company from Shopify to Hashicorp has had to around around it. See: npm/npm#3310 (comment). One instance that isaacs was strongly opinionated about but very clearly incorrect.

[tomjakubowski]

There are workarounds available for the lack of support for symlinks in npm packages. Two examples you can implement today in your package:

1. resolve symlinks at publish pack time, making copies of referent files, using a prepublish prepack step
2. create symlinks your package needs at extraction time, using a postinstall script

What to do about creating or extracting packages for platforms which do not support symlinks is ambiguous, as you can see from the disagreement in this very thread (#6746 (comment)), so it makes sense for npm to leave it up to the package to decide how to handle it.

I'd add it would be a nice feature if npm warned when packing ignores a symlink; currently npm pack skips over symlinks without logging them.

[ljharb]

(i'd suggest using a prepack step instead of prepublish, altho both will work in the normal case)

[matheusmoreira]

@shellscape

I agree with isaacs.

The referenced issue starts with:

When it's packed, the linked files are included in the archive, but the links themselves are being omitted.

That's exactly what I expected to happen and what isaacs refers to as "working as intended". The fact it's not happening anymore is the exact subject of this bug report.

The referenced issue proves that npm did in fact work correctly in the past, that the creator of npm deliberately made it work that way and that npm's behavior was changed at some point after that issue.

@tomjakubowski

There are workarounds available for the lack of support for symlinks in npm packages.

This issue isn't about putting symbolic links in packages though. It's about the fact npm silently refuses to follow symbolic links and omits files as a result. It was supposed to follow the link and put the real file in the package. This is how all other programs work.

What to do about creating or extracting packages for platforms which do not support symlinks is ambiguous

I don't feel that way. I think it's clear what's supposed to happen in this case. Symbolic links work transparently. Programs are generally not even supposed to be aware of them. They are supposed to just open, read, write and close while remaining none the wiser.

The evidence for that is the fact no additional work is needed to do the right thing. Programs actually need additional platform-specific code in order to handle symbolic links in a special way.

There could be a configuration option that makes npm go out of its way to avoid following the link. Not essential. Certainly not the default.

so it makes sense for npm to leave it up to the package to decide how to handle it.

What npm did is the exact opposite of "leaving it up to the package to decide how to handle it" though.

I decided how to handle it when I created the symbolic link. The problem is npm went out of its way to circumvent my handling of the matter. Instead of transparently following the link like it was supposed to, it actually detected the link and then just unilaterally decided to silently ignore the link itself and the file it refers to.

That decision directly led to a cascade of embarrassing and frustrating mistakes: a package was published without the expected files, got immediately unpublished as a result and then a non-sensical twenty four hour count down was imposed where no new correct versions could be published.

So not only did npm not handle the situation, it went out of its way to undo my handling of the situation. Then it proceeded to publish the resulting nonsensical package to the npm registry. And then I was the one who had to go and undo all that stuff. Which it also made as painful as possible.

I'd add it would be a nice feature if npm warned when packing ignores a symlink; currently npm pack skips over symlinks without logging them.

It should straight up fail with an error message in that case.