Using ESM for js-ipfs and js-libp2p codebases

[daviddias]

This is a thread for a open discussion as I would like to hear everyone's opinion. We won't change things overnight and definitely not before we are 100% confident this will be a good thing.

I read yesterday John Dalton's announcement on std/esm -- https://medium.com/web-on-the-edge/es-modules-in-node-today-32cff914e4b --. tl;dr; it enables the usage of ESM without having to add transpilation steps and with little to no overhead.

I'm exciting for this announcement as it speeds up the adoption of ESM which otherwise we would have for April 2018 to have a LTS version with unflagged support or to 2020 until every LTS version of Node.js has full support.

ESM offer better treeshaking for browser bundles -- https://webpack.js.org/guides/tree-shaking/ -- as it is capable of eliminating dead code, making our bundles way smaller which is one of the main feature requests by our users.

What are other thoughts?

[dignifiedquire]

🎉 🎉 🎉 🎉

[mitra42]

Its great to see this - as its crazy that in JS you end up adding a MB or so to a file each time you use one function from another package as the tree expands.

I'm trying to get my head around std/esm ! Currently I have modules that can be loaded in either node directly, or browserify can include them into a single file that can be loaded by a browser, the main difference is a "browser.js" file and a couple of places where the code has to check whether its running in the browser or node.

I'm unclear how that change would work with std/esm, will browserify still gather all the requirements, or should some other tool do it ?

[mitra42]

Any thoughts on this . would using ESM make it hard to have javascript that can be used in both node & browserify (or equivalent)

[daviddias]

Not at all, @mitra42. John Dalton does a great job of explaining it on his post, make sure to check it out.

[mitra42]

Yes - I'd read that post before asking. It refers to switching format to .mjs and changing the import format to:
import a from "./a"
Its not at all clear (at least to me) from that post whether browserify then breaks because of not reading these files correctly and finding extensions.
I dont' mind switching to a different bundling tool, but we obviously need people to be able to write code that can be loaded in either node or browserify or code maintenance (two parallel set of files, with different import/require lines) becomes a nightmare.

[dignifiedquire]

The @std/esm loader is a loader specifically designed for node.js not for browser bundlers.

If we would change our code base to use ES Modules with an interop index.js as described in the article the following changes would happen.

Consequences

for Node.js users

If we use @std/esm, none except that they would have the choice of using

• src/index.js which is commonjs compatible
• src/index.mjs which is ES module compatible

for Browserify users

• Requires use of either rollupify or babelify transforms to transpile and resolve ES modules if the source is used.
• If the dist file is used nothing changes

for Webpack@3 users

• No changes as webpack supports tree shaking and ES modules out of the box without changes

[dignifiedquire]

@diasdavid what do you want to see/do before moving forward with this?

[thisconnect]

If we use @std/esm, none except that they would have the choice of using

• src/index.js which is commonjs compatible
• src/index.mjs which is ES module compatible

Assuming src/index.js is generated of of src/index.mjs it is a bit uncommon to distribute that in src/, most libraries that can be consumed in Node.js and the browser generate UMD and ship dist/index.js in this case (afaik).

package.json's module field should point to src/index.mjs, while the main field would point to dist/index.js, additionally some use browser field too.

[thisconnect]

Also worth having a close look at rollup, as it does NOT wrap each module in a function (maybe webpack3 stopped doing that too).

those articles where written before Node.js decided on .mjs

• https://medium.com/webpack/webpack-and-rollup-the-same-but-different-a41ad427058c
• https://nolanlawson.com/2016/08/15/the-cost-of-small-modules/

lodash, three.js, react, d3, angular (afaik) use rollup for bundling their libraries.

[dignifiedquire]

src/index.js would not be generated, it would be a wrapper for node.js using std/esm while generated files would still go into dist

[dignifiedquire]

and re package.json fields we would probably do

• main: src/index.js
• module: src/index.mjs
• browser: dist/index.js

[dignifiedquire]

re scope hoisting, webpack3 can do that but only if you use es modules all the way, same as rollup

[lastmjs]

Hey, it would be great to have an ES modules version that can be imported directly into the browser with minimal tooling. I would like to use scripts of type module from unpkg or just import the files myself using import map-like tools to resolve bare imports.

[snickell]

FYI: there's a little discussion on the skypack.dev github issues about the problems supporting ipfs on skypack:
skypackjs/skypack-cdn#34

[mikeal]

All the newest multiformats and ipld JS libraries are native ESM now. We had to write some build tooling to make them work nice but we’re pretty happy with it all now.

However, migrating to this will change the required versions of Node.js to those that are aware of package.json exports so I’m no sure this is a change js-ipfs is ready to make.

[teohhanhui]

@mikeal nodejs v10 LTS end-of-life is coming up on 2021-04-30, so I think it's time to revisit this decision.

https://nodejs.org/en/about/releases/

https://nodejs.org/api/packages.html#packages_exports (see the "History")

[jcc10]

I would also like to add that by moving to ESM it would make it far easier for someone to make a fork that is compatible with Deno / Browser without having to preform significant modifications.

[achingbrain]

js-ipfs is being deprecated in favor of Helia. You can #4336 and read the migration guide.

js-IPFS switched to ESM in version 0.63.x so this is complete!