Cannot import sqlite3 via npm and use it

[kazuma0129]

I tried to import sqlite3 via npm. As shown below,

import sqlite from "npm:sqlite3";
console.log(sqlite);

 deno --unstable run -A tmp.ts

Then I got the following error.

error: Uncaught Error: Cannot find module '/Users/x/Library/Caches/deno/npm/registry.npmjs.org/sqlite3/5.0.11/lib/binding/napi-v6-darwin-unknown-arm64/node_sqlite3.node'
Require stack:
- /Users/x/Library/Caches/deno/npm/registry.npmjs.org/sqlite3/5.0.11/lib/sqlite3-binding.js
- /Users/x/Library/Caches/deno/npm/registry.npmjs.org/sqlite3/5.0.11/lib/sqlite3.js
- /Users/x/Library/Caches/deno/npm/registry.npmjs.org/sqlite3/5.0.11/lib/sqlite3.js
    at Function.Module._resolveFilename (deno:ext/node/02_require.js:609:17)
    at Function.Module._load (deno:ext/node/02_require.js:441:29)
    at Module.require (deno:ext/node/02_require.js:652:21)
    at require (deno:ext/node/02_require.js:777:18)
    at Object.<anonymous> (file:///Users/x/Library/Caches/deno/npm/registry.npmjs.org/sqlite3/5.0.11/lib/sqlite3-binding.js:4:17)
    at Object.<anonymous> (file:///Users/x/Library/Caches/deno/npm/registry.npmjs.org/sqlite3/5.0.11/lib/sqlite3-binding.js:7:4)
    at Module._compile (deno:ext/node/02_require.js:713:36)
    at Object.Module._extensions..js (deno:ext/node/02_require.js:740:12)
    at Module.load (deno:ext/node/02_require.js:630:34)
    at Function.Module._load (deno:ext/node/02_require.js:487:14)

I am not sure if this is an error caused by the deno side or not. If anyone else is facing this problem, I would like to know the solution. Thanks!

[dsherret]

We don’t have node API support yet. Once this PR lands it will be possible: #13633

[bartlomieju]

I just tried this issue with branch for #13633 and the error is the same. I think this might be related to postinstall script or Node GYP

[trgwii]

sqlite3 and other npm modules are implied to be FULLY working inside Deno without much effort on the part of the user on the 1.27 blog post.
I just checked ALL of the modules listed there in Deno 1.28.1 with deno run --unstable -A --allow-ffi, and they ALL crash except for usb, which loads, but prints a bunch of debug messages about napi_add_finalizer not being supported.

[bartlomieju]

Thanks for reporting @trgwii, it seems to be a regression that we missed. We'll investigate and fix it.

Just to be sure, which OS did you check it on?

[trgwii]

Just to be sure, which OS did you check it on?

Debian 11 x86_64

[kt3k]

sqlite3 has install life cycle script (node-pre-gyp install --fallback-to-build) and it sets up the native bindings binary at sqlite3/lib/binding/napi-v6-[os-arch]/node_sqlite3.node.

If I manuall copied that file to appropriate path, then the example in the first comment worked.

[kt3k]

BTW if I hit the commant deno run npm:node-pre-gyp install --fallback-to-build manually in node_modules/sqlite3, the native addon (in my case lib/binding/napi-v6-darwin-unknown-x64/node_sqlite3.node) was correctly set up, and I was able to run deno run -A --node-modules-dir --unstable example.js in the root

[bartlomieju]

So it seems a matter of running postinstall script to make it work?

[kt3k]

Looks so to me (technically it's install instead of postinstall in the case of sqlite3. There seem 7 life cycle scripts involved in npm install:

ref: https://docs.npmjs.com/cli/v9/using-npm/scripts

[kt3k]

FYI an alternative module better-sqlite3 also has the similar step https://github.com/WiseLibs/better-sqlite3/blob/b992d041cb8b1084238a1ef5b3adf6238548a3bb/package.json#L39

[kt3k]

Replacing install script with optionalDependencies with 'os-arch' suffix (like esbuild does https://github.com/evanw/esbuild/blob/50ae05ba9433c86ac227d917f3b92cec484ccfc5/npm/esbuild/package.json) might be a solution, but in that case it loses the support of minor platforms where prebuilt binaries are unavailable.

[kazuma0129]

eplacing install script with optionalDependencies with 'os-arch' suffix

@kt3k Good! I'll try it.

[buckett]

I've just hit what looks to be the same issue with duckdb where I imported it with an npm specifier. When attempting to use the duckdb library I got:

❯ deno run -A --unstable main.ts 
error: Uncaught Error: Cannot find module '/Users/buckett/Library/Caches/deno/npm/registry.npmjs.org/duckdb/0.6.1/lib/binding/duckdb.node'
Require stack:
- /Users/buckett/Library/Caches/deno/npm/registry.npmjs.org/duckdb/0.6.1/lib/duckdb-binding.js
- /Users/buckett/Library/Caches/deno/npm/registry.npmjs.org/duckdb/0.6.1/lib/duckdb.js
- /Users/buckett/Library/Caches/deno/npm/registry.npmjs.org/duckdb-async/0.6.1/dist/duckdb-async.js
- /Users/buckett/Library/Caches/deno/npm/registry.npmjs.org/duckdb-async/0.6.1/dist/duckdb-async.js
    at Function.Module._resolveFilename (deno:ext/node/02_require.js:624:17)
    at Function.Module._load (deno:ext/node/02_require.js:456:29)
    at Module.require (deno:ext/node/02_require.js:667:21)
    at require (deno:ext/node/02_require.js:807:18)
    at Object.<anonymous> (file:///Users/buckett/Library/Caches/deno/npm/registry.npmjs.org/duckdb/0.6.1/lib/duckdb-binding.js:4:15)
    at Object.<anonymous> (file:///Users/buckett/Library/Caches/deno/npm/registry.npmjs.org/duckdb/0.6.1/lib/duckdb-binding.js:7:4)
    at Module._compile (deno:ext/node/02_require.js:734:36)
    at Object.Module._extensions..js (deno:ext/node/02_require.js:767:12)
    at Module.load (deno:ext/node/02_require.js:645:34)
    at Function.Module._load (deno:ext/node/02_require.js:502:14)

looking in package.json for the library it has a similar scripts section (node-pre-gyp):

  "scripts": {
    "install": "node-pre-gyp install --fallback-to-build",
    "pretest": "node test/support/createdb.js",
    "test": "mocha -R spec --timeout 480000 --expose-gc",
    "pack": "node-pre-gyp package"
  },

as suggested by @kt3k (👍 thanks) I switched to using --node-modules-dir and then cd to node_modules/.deno/duckdb\@0.6.1/node_modules/duckdb/, ran deno run --allow-all npm:node-pre-gyp install --fallback-to-build which installed the missing platform dependant file. Then cd back to the project and added --node-modules-dir to my run command (deno run --allow-all --unstable --node-modules-dir main.ts) and it resolved my issue.

So it seems that if a module uses an install script to download required binaries for the current platform it doesn't work with deno. I'm currently running:

❯ deno --version
deno 1.29.1 (release, aarch64-apple-darwin)
v8 10.9.194.5
typescript 4.9.4

[bartlomieju]

So it seems that if a module uses an install script to download required binaries for the current platform it doesn't work with deno. I'm currently running:

That's correct, it's tracked in #16164

[littledivy]

Let's continue to track this in #16164 since this is now a postinstall issue and not Node-API. Closing