Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

rfc: new npm link command #3

Merged
merged 5 commits into from
Feb 7, 2019
Merged

rfc: new npm link command #3

merged 5 commits into from
Feb 7, 2019

Conversation

iarna
Copy link
Contributor

@iarna iarna commented Apr 26, 2018

This was originally discussed here: npm/npm#19421

Broken out by command:

### `npm link`
### `npm unlink
Copy link

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Missing backtick here (and is this supposed to be npm link & npm unlink together (like the heading below)? or is there something missing for npm link?)

@gilly3
Copy link

gilly3 commented May 3, 2018

What you are calling "ephemeral" may be a misnomer. The distinction in that use case seems to be that you are linking to a package in a separate repo, and that it may not be linked for all (or any other) developers. Whether or not it is a temporary state for "trying things out" doesn't matter. I have use cases where links will differ between developers, but the links may exist long-term, even permanently.

I work on several different apps that share libraries. Each app or library is in its own repo. Not all developers on my team work on all apps or libraries, so it would be inappropriate to commit a package-lock dictating links to repos that may not even be cloned on other developers' systems.

Further complicating matters, we prefix all our internal packages with our company initials to avoid naming collisions. (These are published in a private registry.) However, when changing directories in the command line, I find it tedious typing that prefix, so I clone the repos into directories named without the prefix. Other developers do not do this. So, where I might want to link to ../fancylib, my co-worker would link to ../abc-fancylib.

The point is that for many developer workflows, linking to local packages is going to be specific to the developer, and not specific to the package. Thus, committing a package-lock with links will cause headaches.

@JDvorak
Copy link

JDvorak commented May 9, 2018

I consider npm link to be a local configuration override on the default behavior, not a specification to be shared between developers.

For instance, if I am working on an ecosystem of modules, then I will be (tirelessly) maintaining links between my local copies in a directory, or between my version of a microcomponent and the surrounding ecosystem depending on it. But, when I incrementally publish my improvements, I would not expect the links to be carried. Nor, would I want them to-- especially in cases where there is a 1-component:1-developer workflow.

@evocateur
Copy link

@JDvorak npm publish never includes package-lock.json, nor is it ever respected by a consuming npm install <pkg>, why would it matter what local links exist in it?

@zkat zkat self-requested a review May 9, 2018 21:49
@iarna iarna added the cli label May 9, 2018
@JDvorak
Copy link

JDvorak commented May 9, 2018 via email

@iarna iarna requested a review from a team May 9, 2018 22:27
@JDvorak
Copy link

JDvorak commented May 9, 2018

On further reflection, I realize that there are two different understandings of what a dependency is:

  1. A dependency is a parameter of a module, and therefore a contingent attribute of its runtime. Like a component in an assembly. "A module is itself only logic overlaying a configuration of other modules, and may over time or in different conditions depend on different modules. Compatible versions, and entirely different modules can and should be swappable." This metaphor arises in the module developer's use of npm link as an explicit alternate binding, and the praise of semver as ideal. You can also see this metaphor in action when someone uses proxyquire for testing, or writes a shim to approximate the separation from the declaration that something is required and the initialization of a compatible instance. I think you might also see this reasoning in most discussions of npm aliases.

  2. A dependency is a part of a module, and therefore an essential attribute of its functioning. Like an organ in a body. "A module is an immutable block of code composed of smaller immutable blocks of code. Every call to it should be exactly the same, no matter the conditions or time." It is not a variable element but a specific value. The metaphor of choice for pragmatists who recognize irreducible coupling between modules in business logic. Here I feel is where the idea of package-lock appears obvious, while in the other world it appears equivalent to caching layer.

But, if you notice, the community seems to keep reinventing ways to express a separation between a variable's declaration and its initialization. The npm link I would want is one that allows me to set a different runtime initialization without changing the formal declaration of dependency.

@Dwood15
Copy link

Dwood15 commented May 9, 2018

@JDvorak

The npm link I would want is one that allows me to set a different runtime initialization without changing the formal declaration of dependency.

I'm having trouble seeing the difference between your version of npm link and the aliasing that was just ratified.

@JDvorak
Copy link

JDvorak commented May 9, 2018

If I could just do what I used to do with npm link (and it stops breaking when I run npm install) but under the hood it merely aliases to local directories, well then that sounds great. And, on reading, it does seem that is the case. But, now I'm not sure why we would need a new npm link if we are getting a proper aliasing feature.

@iarna
Copy link
Contributor Author

iarna commented Jun 1, 2018

@gilly3

What you are calling "ephemeral" may be a misnomer. The distinction in that use case seems to be that you are linking to a package in a separate repo, and that it may not be linked for all (or any other) developers.

By "you" I think you mean @giltayar, who I was quoting there. (Tagging them so they can clarify their intent.) What I understood them to be asking for was very-short-term links, something closer to what npm's current behavior results in (where they're kept until you run npm i again).

The point is that for many developer workflows, linking to local packages is going to be specific to the developer, and not specific to the package. Thus, committing a package-lock with links will cause headaches.

If I said that your concern is that there will be package lock churn between your installation and your co-workers because you have atypical way of cloning your repos and so when they make links they'd stomp on yours and vice versa?

Would your concerns be alayed by storing this in a different file? (I'm not super keen on introducing Yet Another File, but in so far as I can tell, it's what's here, adding something new or remove the npm link command entirely.)

@JDvorak

Because it is respected by anyone running npm install from within a cloned repository.

It would not. npm link create would be needed to replicate your local linking.

If I could just do what I used to do with npm link (and it stops breaking when I run npm install) but under the hood it merely aliases to local directories, well then that sounds great. And, on reading, it does seem that is the case. But, now I'm not sure why we would need a new npm link if we are getting a proper aliasing feature.

I'm confused as to how you think aliasing would actually solve this problem for you? Can you elaborate? Aliases are published as a part of the module metadata. They change the source of the module for everyone that uses them. They are not a local development tool, but a way to ease the use of forks (so you can switch without changing your source code).

@giltayar
Copy link

giltayar commented Jun 2, 2018

@iarna

What I understood them to be asking for was very-short-term links, something closer to what npm's current behavior results in (where they're kept until you run npm i again).

Yes! With the caveat that removing the links on each npm install/ci/update is a bit too trigger happy: I npm install/update all the time, and don't want my links to be removed when I do. I believe that @gilly3 has a point: the word "ephemeral" is not perfect. Maybe "private": I want "private/ephemeral" links to stay until I manually decide to remove them, but I don't want them officially introduced to my team members, as they are private to my workspace.

Would your concerns be alayed by storing this in a different file? (I'm not super keen on introducing Yet Another File)

I'm really not keen on introducing another file either! You could put the information in the npm home folder (usuually ~/.npm?), keyed on the file path of the package (the package that is linking to another package).

@wearhere
Copy link

+1 for @giltayar's "ephemeral" use case as described here. My coworkers and I use links all the time but I'm pretty sure we wouldn't be interested in sharing them with each other. All our packages are in different repos; and so even if multiple developers were working on the same branch of repo A, which branch required that each developer instantiate a link, committing that link to the branch would not be sufficient to "enable" the link for all developers and would probably be otherwise confusing.

I am very interested in not having npm install blow away links though (which I imagine was part of the motivation for saving to the lockfile).

Separately, very interested in having npm unlink undo npm link and restore a non-linked copy of the dependency, which I had filed as https://npm.community/t/npm-unlink-foo-doesnt-reinstall-the-real-foo/713 before being directed here.

@Venryx
Copy link

Venryx commented Jul 20, 2018

I just want to add a quick note, for others using npm 5 and wanting to restore the old behavior, where "npm install" does not delete symlinked packages: try updating to the latest version of npm 5. (npm 5.10.0, atm)

I had been using the npm 5.6.0 that came with node 8.11, but then saw this comment. I tried updating to the latest 5.X version of npm, and this seems to have restored the old npm install behavior where it just ignores symlinked packages. (or at least doesn't delete them)

This is good enough for me, until we get a revamped npm link system in npm 7. (the npm 5.6.0 behavior of silently deleting all symlinked packages, on any npm install command, was horrendous -- not only did it remove the symlinks, but once it even deleted all the source library folders, which would have meant days of work lost if they weren't also stored on Github)
==========
Oh also, another +1 for @giltayar's "ephemeral" use case. That's how I use linked packages as well, as a way of locally "overriding" a package's contents with a customized version, without having to change anything in the actual project repo. (eg. creating a new web-accessible variant and changing project.json to point to it, or committing the change directly into a now-partially-git-managed node_modules folder)

@thw0rted
Copy link

Does anybody else have concerns about the revised link having no support for pack / publish? I currently develop an application and its dependency modules in tandem using link and then distribute the application to our test machines using pack. Today's (6.x) behavior kind of works, but the whole dependency directory is shoved in which is obviously sub-optimal.

I'd love to see pack recurse into linked deps and pack them, then use the resulting output in lieu of a straight directory copy (6.X) or just giving up (this RFC). Does that make sense? Is it too difficult to implement?

@stacktracer
Copy link

stacktracer commented Oct 18, 2018

Anybody know the answer to the following question (copied from the previous thread)?

  1. What happens when the version changes in a linked package? When is it noticed? What is the expected behavior if the new version causes it to no longer match the dependency spec of the consuming package?

@giltayar
Copy link

giltayar commented Feb 1, 2019

The latest commit (npm-link: Revise for non-lock-file storage) looks perfect. Just what the doctor ordered. My monorepo build tool just got simpler! :-)

Copy link
Contributor

@zkat zkat left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yay Ratification!!one

@zkat zkat merged commit 6b62490 into latest Feb 7, 2019
@zkat zkat deleted the npm-link-rfc branch February 7, 2019 21:53
@ties-s
Copy link

ties-s commented Mar 28, 2021

So what's the status on this? npm 7 still deletes linked modules

@Venryx
Copy link

Venryx commented Mar 29, 2021

@ties-s What version of npm did you try it with?

I ask because, in this issue, an npm dev (@nlf) suggests the bug may have been fixed by some changes to npm link between version 7.2.0 and 7.5.2, but the OP did not confirm whether those changes fixed it or not. (the lack of response suggested it was resolved, though)

It will be good for someone to confirm if the bug is still present in the latest versions. (7.5.2+) [I'll eventually test it myself if no one else gets to it, but am too busy/lazy atm.]

@ties-s
Copy link

ties-s commented Mar 29, 2021

@Venryx I have npm@7.6.1

@ljharb
Copy link
Contributor

ljharb commented Mar 29, 2021

Try v7.7.5 just in case, and if there’s still problems, please file an issue.

@cyberbiont
Copy link

Just quickly tried it with npm 7.8.0 and the bug is still there.

@joe-gre
Copy link

joe-gre commented Apr 19, 2021

I tried with npm 7.8.0 and the bug is still there. Running npm install removes linked packages. Also running npm link <package_name> removes all other links, which is a newer issue that didn't occur on more recent versions of npm whereas the other issue has occurred since npm 5 I believe. The only way to setup multiple links is to run npm link <package_1> <package_1> and run this command again after running npm install.

This is a major issue for my team. We have to waste a lot of time relinking after installing new dependency versions.

@thw0rted
Copy link

@joe-gre it sounds like you need to comment on npm/cli#2372 which is marked as closed, and apparently shouldn't be.

@IronGeek
Copy link

Still happens with npm 7.11.2... 😞

@baelter
Copy link

baelter commented Jun 30, 2021

And 7.18.1 ...

@8enSmith
Copy link

And 7.24.0...

@and0p
Copy link

and0p commented Nov 22, 2021

I tried with npm 7.8.0 and the bug is still there. Running npm install removes linked packages. Also running npm link <package_name> removes all other links, which is a newer issue that didn't occur on more recent versions of npm whereas the other issue has occurred since npm 5 I believe. The only way to setup multiple links is to run npm link <package_1> <package_1> and run this command again after running npm install.

This is a major issue for my team. We have to waste a lot of time relinking after installing new dependency versions.

This threw me off as well, since an NPM upgrade on one machine made only the last-linked stick around. I now have a link.sh in the repo for all of my projects that need it, which runs npm install followed by npm link local1 local2 .... It's not ideal but saves on a lot of typing.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.