Add package.json "sustainability" property

[kemitchell]

This small PR begins what will likely be a larger conversation on making package.json a more effective communication channel from developers to users about project sustainability needs.

Implementation

Standardize a package.json property for sustainability data that points to a JSON endpoint developers can update out-of-band.

Allow developers to put sustainability data, on a simple schema, on master in their publicly hosted Git repos. Then to set sustainability in package.json to the URL where the Git host will serve the JSON file. For many npm devs, that will be some endpoint on raw.githubusercontent.com.

Per the schema, the JSON data may include references to other endpoints for specific contributors. That way, individual developers, businesses, and nonprofits can define sustainability.json files or similar on their own domains, and edit sustainability metadata for specific projects only to include reference to permalinks for sustainability endpoints specific to them.

Operating Environment

The approach I propose here reflects two key observations:

1. package.json is a package's passport across developer, user, and contributor lines. If developers need to send messages to users, package.json is their best bet.

   In that sense, messages developers send via package.json are reliable. In most situations, they will be received if the code is.

2. At the same time, there is no guarantee that users will have or receive the latest versions of packages or their package.json files at any given time. Folks download old versions of packages all the time. As a result, package.json files can be worse for real-time communication than postal mail. When the messages developers want to send have to be current, relying on package.json will only teach users not to rely on what they see there.

   In that sense, messages developers send via package.json are high latency. They will be received, but God only knows how soon.

Sustainability information, such as the availability of individuals for hire, of paid products and services to buy, and even of lists of who has contributed or currently drives a project, all needs to be current to feel reliable, and therefore to be actionable, and therefore to be effective.

This PR proposes a solution that plays to package.json's messaging strengths by using it to send permalinks that can in turn provide more time-sensitive data to those interested.

Prior Art

Too much to list here. But foremost in my mind have been:

• @feross' thanks, a CLI app that recurses node_modules, compares results to packaged maps from author, organization, and package to URIs. The approach I propose here evolves that of thanks by decentralizing publication of actionable sustainability data. @feross shouldn't have to merge a PR for every dev and project asking for donations, and every dev changing their approach or service provider.

• My own License Zero, which uses non-standard package.json properties to encode signed metadata about the availability of licenses for use in closed software development.

• OpenCollective's collective property, which functions similarly, as a pointer and trigger for HTTP requests for up-to-date data.

Motivation

There are myriad opinions about how to address open source "sustainability". I hold no less than a few myself, including a profound regret that "sustainability" has won out as the going term for this issue.

No matter which approach folks prefer, the common thread is lack of rich information about who is behind what software, and what users can do for them. What we see now is a Cambrian Explosion of attempts to tunnel sustainability information through various existing communication channels---documentation, licensing, and so on. I don't think we're anywhere close to being able to call the winner. And I don't think npm should put its thumb on the scales, in favor of any particular "sustainability as a service" vendor or approach.

But I seen an opportunity to give structure to that diversity, so users can consume it en masse. And I see an opportunity for npm to facilitate in a self-consciously neutral way. Every sustained effort I've seen to facilitate support for open software developers has URLs with calls to action. Turning package.json into a way to systematically inventory and follow up on those calls is what this PR is about.

Process

Folks working at npm, Inc. have never been shy telling the world that I represent npm, Inc. as legal counsel. Contributions to npm CLI probably had more than a little to do with my receiving that professional opportunity. But then, as now, this proposal to npm CLI is well outside my legal responsibilities to the company. npm does not pay me to code, and the CLI team did not ask for this PR.

Per the contributing guide, I fully embrace that I bring this PR as a "community member", and not an "npm, Inc. employee". I'll do my best to confine back-and-forth on this proposal to public fora, like PR comments.

I wouldn't send this if I didn't think it worth landing as proposed. But I know better than to think I might have got it right, out of the gate, by anything but blind luck. Very much looking forward to thoughts and views of others.

[wesleytodd]

Hey, saw this on twitter so thought I would come drop this link here:

nodejs/package-maintenance#220

We have been working on something similar with the Package Maintenance WG, anyone is welcome to join the conversation there and then once we finalize it we were planning on proposing it back here. I think we even have some participation from npm team members over there already.

[voxpelli]

Linking to a Twitter thread here for future reference: https://twitter.com/feross/status/1158813996620926976?s=21

Eg. some feedback on the naming, that another keyword than “sustainability” could be a better fit 🙂

[kemitchell]

@voxpelli I'm not a big fan of the word "sustainability", but I think it has won the race to be name of the issue. GitHub used "funding". The shorter "fund" might work. Others have floated "support", though that term is overloaded.

[ljharb]

@kemitchell perhaps you’d like to join ahmad on the node package maintenance calls, where we discuss the proposal linked above?

[kemitchell]

@wesleytodd thanks for linking the Node PR. It's great to see so many folks I know chiming in there.

Do I correctly understand that the upshot of anything landed there would essentially be a PR back to npm's package.json spec?

[ljharb]

Yep! Without npm’s cooperation, the effort isn’t likely to gain adoption; without node’s, it’s harder to gain agreement among alternative package managers, also hindering adoption - so it’s great that both npm and node are participating in the working group :-)

[kemitchell]

@ljharb Thanks for inviting me to the calls. I'm excited to see more folks working on this kind of thing, but I'm also very leery of taking the process out of the open development workflow.

Who is npm's rep in the WG?

[ljharb]

@kemitchell at the moment, the CTO, Ahmad.

[wesleytodd]

but I'm also very leery of taking the process out of the open development workflow

The process in the WG is completely open and welcoming to anyone who wants to participate. The more the merrier 🎉

[kemitchell]

@ljharb thanks for letting me know. As I mentioned above, I don't represent npm on this proposal. But I should be in touch with Ahmad.

@wesleytodd I want to respect everyone's personal definition of "open". But I wouldn't consider phone calls part of my typical open development workflow. I'd rather keep things in writing, async, in public forums like public repo issues.

[wesleytodd]

@kemitchell I understand your concern. In this case all meetings are streamed and available on the node youtube. Also we publish meeting notes on the repo. For example, here is the meeting form last week:

nodejs/package-maintenance#232

And from the meetings notes PR you also can watch the whole meeting:

https://www.youtube.com/watch?v=xY4kvp9jtlo

All of which can be done async, and all of which is "open" in any definition I can think of.

EDIT: Also, all of the topics discussed in the meeting are originally in issues on GH. So the reason to talk about it in the meeting is to resolve issues which are more efficiently done synchronously "face to face". But none of it is "original", it is all just based on the conversations happening in issues you can read and participate in asynchronously.

[kemitchell]

I've updated this PR, as well as sustainability-schema and get-sustainability packages. The schema itself is now more flexible. The PR now includes code to report sustainability-related links for contributors to newly installed packages on npm install.

Printing a sustainability report entails a bunch of network calls to fetch the JSON data for packages that point to it. That has called into question the idea of putting URL pointers in package.json, rather than JSON data.

I still think that's worth the cost. Unlike licensing, authorship, and other point-in-time information about a package, the point of sustainability information is how to initiate a present-tense or continuing relationship. That information will change independently of package content, and potentially many times. It is also highly likely to be duplicated across packages and projects.

Looking back at the funding-approach histories of a few friends and colleagues who've published prolifically to npm, I think it's clear that baking sustainability data into package tarballs would have produced lots of makework. For example, when a developer changes their preferred payment portal, they would have had to update every one of their package's metadata files. Even so, most tarballs on the registry at any given time would have included outdated information.

The natural response to that would be to set up a permalink, perhaps in a purpose-built GitHub repo, GitHub Pages site, or personal domain, like https://kemitchell.com/sustainability, and linking to specifics from there. But that would eliminate the ability of machines to read and parse the relevant information.

[kemitchell]

• Implement timeouts on GET requests for sustainability data. We don't want 404s or slow web servers holding up npm install invocations.
• Add a separate subcommand to output just the sustainability report. Support the usual --json flag for version-stable output.

[kanongil]

Awesome, this will make it trivial to track all installations of my packages, even with --no-script.

[kemitchell]

I am in the process of replacing this PR with another one that works somewhat differently. npm install will mention a new npm funding subcommand when there is relevant metadata present in newly installed packages. The new subcommand will fetch JSON data for installed packages, but not on every install.

[ljharb]

@kemitchell are you planning on using the format the package maintenance group is currently recommending, or something different? It would be ideal to align our efforts.

[kemitchell]

@ljharb, I am currently planning to use a much, much simpler format, but also to include a schema property, so the client can identify different formats going forward.

[wesleytodd]

@kemitchell, do you have concerns about the format we have proposed? If so, could you share them with us? It would be best if we could align on this before more people put in time to solutions without consensus. It is your time and effort, so you do what you want, but I think working together is best :)

[kemitchell]

@wesleytodd: I am all for the working group and glad to see more people contributing! But right now I think npm needs an MVP that’s as simple as possible, if not simpler. A subcommand that lists projects, people, and links, human readable and JSON.

Next time I get an hour to work on this, I can spend that implementing the subcommand and finishing the PR or wading through all the Markdown and comments in the WG repo. I think npm and especially the folks impacted by the recent policy change on ads need me to do the former.

That being said, I want to make sure that the group can get its work into the CLI. So we namespace, version, start small, and ship.

[wesleytodd]

But right now I think npm needs an MVP that’s as simple as possible, if not simpler.

I completely agree that we need something, and that something should be as simple as possible. I am a bit worried because if it rolls out and gains popularity it will be difficult to undo/redo when/if the WG proposal is adopted.

wading through all the Markdown and comments in the WG repo

Maybe this will help: https://github.com/pkgjs/support/

This is a basic json schema built against the state of the proposal as of landing the PR. It includes some examples, both simple and complex. More to come on this pacakge.

[kemitchell]

@wesleytodd this PR is likely to be superseded very soon, but since I have you here: Any chance the WG's format is stable enough that we could go ahead and sketch a function taking the metadata as input, and outputting a list of URLs or similar calls to action for project and individual support? In other words, could we code a short function to recognize WG metadata on sight, and then to filter it down to the M for minimum subset of data for the MVP?

Even something very preliminary would be much appreciated. I want to make sure the connection between my current push and the broader work of the WG becomes clear in a concrete way, as soon as I can have code to propose.

[wesleytodd]

Hey @kemitchell, the format should be stable enough to build tooling around. We have some additions planned, but those should just be minor releases for any tooling.

I have a branch in the package I linked above where I started putting together the cli portion, which would include a command to this effect. I will push that soon, this weekend at the latest, is that good for you?

[kemitchell]

@wesleytodd please don't promise me your weekend!

I am nearly ready with a starting point for a new PR. From there, we can get the subcommand processing WG's data format.

I still find myself a little bit lost in the shear volume of issues, PRs, comments, and Markdown. But I understand there's some discussion ongoing about metadata in package.json versus metadata referenced by URL from package.json. Might I ask you to link me to the proper place to participate in that discussion? I'd like to add my experience with several initiatives tunneling support-related information through package.json to the pot.

[ljharb]

The gist is that unless it’s inline in package.json, it’s not both immutable (only the registry is immutable) and fetchable without downloading the entire tarball.

[kemitchell]

@ljharb hey there! Thanks for the summary.

Has that discussion mainly happened by phone or in person? Is there someplace on GitHub I can go to share my two cents?

[wesleytodd]

There are a bunch of related comments because this was one issue we went back and forth on many times. The first comment is by @ljharb bringing up just this issue :)

nodejs/package-maintenance#220 (comment)

I believe all of the comments are in this thread, and the only thing that happened in the meeting was that we agreed it lowest common thing we could agree on was it in package.json, so that is what we started with on the spec.

I think the best place to continue this conversation would probably be this issue: nodejs/package-maintenance#241

[kemitchell]

Closing in favor of #246, currently a draft.