@arcgis/core: require() of ES modules is not supported. (Cannot import in Node.js app)

[sandromartis]

TL:DR:
Looks like there are no CommonJS modules yet so that @arcgis/core can be used in Node.js
Are there any plans to add it?

Use case
Using the geometryEngine and other geometry features of the JS API in a Node.js application.

Describe the bug
I'm trying to import @arcgis/core into a simple Typescript example and run it in Node.js.

The typescript file compiles fine to Javascript but when running it either directly with Node or through ts-node I get an error:

Error [ERR_REQUIRE_ESM]: Must use import to load ES Module: /Users/Sandro/Development/arcgis-core-testing/node_modules/@arcgis/core/WebMap.js
require() of ES modules is not supported.
require() of /Users/Sandro/Development/arcgis-core-testing/node_modules/@arcgis/core/WebMap.js from /Users/Sandro/Development/arcgis-core-testing/test.js is an ES module file as it is a .js file whose nearest parent package.json contains "type": "module" which defines all .js files in that package scope as ES modules.
Instead rename WebMap.js to end in .cjs, change the requiring code to use import(), or remove "type": "module" from /Users/Sandro/Development/arcgis-core-testing/node_modules/@arcgis/core/package.json.

Framework and build tools

• Typescript
• ts-node
• Node.js

To Reproduce
Run npx ts-node main.ts with the following code snipped and package.json/tsconfig.json setup.

Running first npx tsc main.ts and then node main.js produces the same result.

main.ts

import WebMap from '@arcgis/core/WebMap';

const webmap = new WebMap({
  portalItem: { id: "bla" }
});

package.json

{
  "name": "arcgis-core-testing",
  "version": "1.0.0",
  "dependencies": {
    "@arcgis/core": "^4.18.0-next.20201104"
  },
  "devDependencies": {
    "@types/node": "^14.14.6",
    "ts-node": "^9.0.0",
    "typescript": "^4.0.5"
  }
}

tsconfig.json

{
  "compilerOptions": {
    "module": "commonjs",
    "strict": true,
    "esModuleInterop": true,
    "skipLibCheck": true,         
  }
}

Additional context
It looks Typescript finds the imports and compiles just fine. But then the execution throws the error.
It does not matter if compiled and executed directly with ts-node or first compiled and then manually executed with node.

I also tried importing @arcgis/core into another bigger Typescript/Node project encountering the same error.
This project works otherwise fine with various kinds of ES6 libraries.

[andygup]

Hi Sandro, the JS API is ESM, not CJS. You have to use Node's native ESM support, or convert to CJS. Also note, we have a PR pending on this repo for a node sample.

[sandromartis]

@andygup Thanks for the quick reply. I will have a look at the node examples once they are in.
Are there any plans to distribute a CJS build through npm in the future?

[andygup]

No plans for now since you can transpile the ES modules into the module type of your choice. Here's the link to the new node sample: https://github.com/Esri/feedback-js-api-next/tree/master/esm-samples/jsapi-node

[sandromartis]

@andygup I wanted to give some more feedback on this one.

I was able to get @arcgis/core working in our project but it was not an easy task.

Native modules don't work for us because we still have some older node versions on our build systems.
So I tried transpiling with rollup.

It took me a while but I got it working in the end. However, there are a couple of issues that make the whole process not ideal.

Bundler overhead

Adding rollup and a transpile step is a big overhead. We don't use a bundler, it's a pure Node.js/Typescript app, so adding a bundler just for the JS API is not ideal.

Typescript

After transpiling @arcgis/core we lose all the typescript definitions. I couldn't figure out how to preserve them with Rollup, so I created a separate .d.ts that imports from the original @arcgis/core module and exports again.
This caused some other issues with Jest which I was able to "hack around".
Maybe there's a better way?
Do you have any suggestions on how to solve the Typescript part?

I do understand that Node.js is not a priority atm for the JS API, but it would be really nice if @arcgis/core would be shipped with CommonJS modules. Offloading the transpile step to users is not ideal in my opinion.

standard-things/esm

Btw. I also tried this package https://github.com/standard-things/esm but no success there. Ran into a couple of issues as well, mainly this one: standard-things/esm#868

[odoe]

I'm a little unclear on where the issue may be. Since core has a "type": "module", it can cause some issues when trying to use ts-node with non-modules. You don't need to transpile the whole library, You can just compile your app.

Here is a sample building the node samples with TypeScript.
https://github.com/odoe/jsapi-node-ts

[andygup]

@sandromartis any luck?

[sandromartis]

@andygup Thanks for getting back to me.

Rollup

You don't need to transpile the whole library, You can just compile your app.

We are not using a bundler for our app and would like to avoid having to run the whole app through a bundler just to get a single dependency (@arcgis/core) working.
However, I was able to transpile just the JS API modules that we need (geometryEngine) and that seems to work.
I was also able to get the type definitions working. (Importing from node_modules and re-exporting.)

It is a bit cumbersome but it works now. 👍

standard-things/esm

I also tried standard-things/esm again and was actually able to dynamically load in the @arcgis/core esm modules.
This requires only a couple of lines of code and doesn't need a transpile step.
It also works with typescript using the same approach as above.

The only problem is that Jest doesn't play nicely with it, but there is an open issue to get that solved: standard-things/esm#706

standard-things/esm would be a nicer solution than rollup as it doesn't need a transpile step.
But the rollup way works for us now, so that's nice. :)

But still, importing CommonJS modules directly from @arcgis/core would be the perfect solution.

Have you guys looked at how arcgis-rest-js is doing it?
They export both CommonJS and ESM modules.
Wouldn't such an approach work as well for @arcgis/core?

Anyway, thanks for the help! 👍

[odoe]

The API is too large. It would double the on-disk size of the npm install and it's not something we plan to do.

[sandromartis]

The API is too large. It would double the on-disk size of the npm install and it's not something we plan to do.

Okay, that explanation makes sense 👍 Better to keep the npm install fast.

I guess this ticket can be closed then, thanks for the help 👍