Deno support

[ghost]

Hi!
It would be very nice to make an ES import for Deno and place the documentation and examples in the main repository at https://deno.land

For example:

import { sharp } from 'https://deno.land/x/sharp@v0.27.0/mod.ts'

Thanks!

[lovell]

Hi, sharp is a "native" module that interfaces with Node.js through N-API. This API isn't currently available in Deno, although there appears to be (or at least have been) a desire to support it - see denoland/deno#237 (comment) and denoland/deno#241

[shadowtime2000]

I know very little about native/systems programming and stuff - but couldn't you create Rust bindings to the C++ API and then have it hook into Deno's unstable native plugin API or have it compile to WASM/WASI? Again, I know very little about this stuff.

[ghost]

Perhaps this link will be useful to you https://github.com/emscripten-core/emscripten
Deno has very good wasm support!
You can connect the module in several lines: https://deno.land/manual@v1.8.0/getting_started/webassembly

[shadowtime2000]

Sorry if this sounds rude, but it doesn't seem like there is an interest from the sharp maintainers so I guess someone from the community could just use emscripten with a simple c++ project importing sharp and just publish that.

[ghost]

At the moment, there are no photo processing modules for Deno.
I was forced to make an interface for the most popular library for photo processing in Python - Pillow.
If you're interested, I can post my code on Github in the coming days.

[tomByrer]

Here is the lib mentioned above:
https://github.com/michailVestnik/python_pillow

@michailVestnik can you reopen please? I & I'm sure others would love to use this lib on Deno someday....

[jgod]

Would be killer

[lovell]

For those who haven't yet seen, Deno is in the process of adding support for Node-API (N-API) via denoland/deno#13633

Once this is made available we can assess how much additional work, if any, would be required by sharp to integrate.

[MarkBennett]

Looks like denoland/deno#13633 was merged yesterday @lovell! 😄

[deeprobin]

Why is there no deno-native implementation?

[lino-levan]

Node API (formerly NAPI), was implemented into Deno a while ago. Could this issue be opened again, I think it deserves some discussion

[lovell]

Node API (formerly NAPI), was implemented into Deno a while ago

The merged PR was part of Deno v1.26.1 published on the 6th October, 35 days ago. I haven't yet seen any documentation or examples relating to this feature.

Has anyone tried using sharp with Deno since then? Happy to reopen if there are specific things that need to be done in sharp.

I note Deno has its own dependency management, and features of the Node.js/npm/yarn ecosystem such as install scripts are not (yet?) available, so it's unclear how to get platform and architecture specific binaries installed. Perhaps manually-managed import maps might be required to handle this?

[lovell]

A quick check reveals that the latest Deno allows me to use sharp, albeit after running npm install within the Deno cache and with napi_add_finalizer is not yet supported warnings (ignorable?).

$ deno --version
deno 1.27.2 (release, x86_64-unknown-linux-gnu)
v8 10.8.168.4
typescript 4.8.3

Here's my test.ts file:

import sharp from 'npm:sharp@0.31.2'
console.log(sharp.format)

Running it first time populates Deno's cache from npm, but doesn't install binaries and fails.

$ cat test.ts | deno run --allow-all --unstable -
error: Uncaught Error: 
Something went wrong installing the "sharp" module
...

Manually install the binaries by running npm install within Deno's cache for now (perhaps Deno will do this bit for you at some point).

$ cd ~/.cache/deno/npm/registry.npmjs.org/sharp/0.31.2
$ npm install

Running the test script again appears to work:

$ cat test.ts | deno run --allow-all --unstable -
napi_add_finalizer is not yet supported.
...
{
  jpeg: {
    id: "jpeg",
    input: { file: true, buffer: true, stream: true, fileSuffix: [ ".jpg", ".jpeg", ".jpe" ] },
    output: { file: true, buffer: true, stream: true, alias: [ "jpe", "jpg" ] }
  },
...

Is anyone aware of a better/easier way to get Deno to run install scripts, rather than manually running npm install?

In summary, we should add something to the documentation about this, but it looks like there are no code changes required. 🚀

[MarkBennett]

Deno is talking about postinstall script support, but with a prompt (and assume a flag to force?)

denoland/deno#16164

[deeprobin]

Why was this closed?

/cc @lovell

[lovell]

@deeprobin The person who opened the issue also closed it (on the 7th April 2021). Had there been an obvious change required to sharp then I would happily have reopened it.

At the time it appeared that Deno was planning to add support for Node-API, and now they have and it works, which is great news, and I'm sure you'll join me in thanking the Deno maintainers for this.

[jgod]

Thanks Lovell.

I can maybe port some of my old node stuff over to Deno now, which I find more maintainable (less dev tooling/packaging overhead) for little scripts/CLIs.

[UrielCh]

I have tried on a windows host

nodeJS version: v18.12.0

deno version: 1.29.4

with import sharp from 'npm:sharp@0.31.3';

and import sharp from 'npm:sharp@0.31.2';

I start the npm install from the np, cached folded, and I still get the error:

deno run -A --unstable .\test-sharp.ts
error: Uncaught Error: 
Something went wrong installing the "sharp" module

Error in native callback

Possible solutions:
- Install with verbose logging and look for errors: "npm install --ignore-scripts=false --foreground-scripts --verbose sharp"
- Install for the current win32-x64 runtime: "npm install --platform=win32 --arch=x64 sharp"
- Consult the installation documentation: https://sharp.pixelplumbing.com/install
    at Object.<anonymous> (file:///C:/Users/uriel/AppData/Local/deno/npm/registry.npmjs.org/sharp/0.31.2/lib/sharp.js:34:9)
    at Object.<anonymous> (file:///C:/Users/uriel/AppData/Local/deno/npm/registry.npmjs.org/sharp/0.31.2/lib/sharp.js:37:4)
    at Module._compile (deno:ext/node/02_require.js:747:36)
    at Object.Module._extensions..js (deno:ext/node/02_require.js:780:12)
    at Module.load (deno:ext/node/02_require.js:658:34)
    at Function.Module._load (deno:ext/node/02_require.js:515:14)
    at Module.require (deno:ext/node/02_require.js:680:21)
    at require (deno:ext/node/02_require.js:820:18)
    at Object.<anonymous> (file:///C:/Users/uriel/AppData/Local/deno/npm/registry.npmjs.org/sharp/0.31.2/lib/constructor.js:8:1)
    at Object.<anonymous> (file:///C:/Users/uriel/AppData/Local/deno/npm/registry.npmjs.org/sharp/0.31.2/lib/constructor.js:425:4)

I all so tried to run npm install --ignore-scripts=false --foreground-scripts --verbose sharp in deno cache npm package

and with npm install --platform=win32 --arch=x64 sharp

Same error.

[xenova]

Are there any updates on this? Some users have expressed interest to use Transformers.js in Deno, but haven't been able to due to this issue.

[lovell]

The somewhat manual steps outlined in #2583 (comment) should already allow the use of sharp with Deno.

Installation will hopefully be much simpler when the feature being tracked at denoland/deno#16164 is made available, at which point we can do things like add Deno to CI environments and update documentation.

[xenova]

The somewhat manual steps outlined in #2583 (comment) should already allow the use of sharp with Deno.

Installation will hopefully be much simpler when the feature being tracked at denoland/deno#16164 is made available, at which point we can do things like add Deno to CI environments and update documentation.

Great, thanks for the quick response.

[VelinGeorgiev]

I manually installed Sharp as suggested above, but I am getting the following error on Mac M1. Looks like I will park Deno - Sharp experiments for now.

error: Uncaught Error: 
Something went wrong installing the "sharp" module

Unable to find napi_register_module_v1 symbol in /Users/xx/Library/Caches/deno/npm/registry.npmjs.org/sharp/0.32.4/build/Release/sharp-darwin-arm64v8.node

Possible solutions:
- Install with verbose logging and look for errors: "npm install --ignore-scripts=false --foreground-scripts --verbose sharp"
- Install for the current darwin-arm64v8 runtime: "npm install --platform=darwin --arch=arm64v8 sharp"
- Consult the installation documentation: https://sharp.pixelplumbing.com/install
    at Object.<anonymous> (file:///Users/xx/Library/Caches/deno/npm/registry.npmjs.org/sharp/0.32.4/lib/sharp.js:37:9)
    at Object.<anonymous> (file:///Users/xx/Library/Caches/deno/npm/registry.npmjs.org/sharp/0.32.4/lib/sharp.js:40:4)
    at Module._compile (node:module:718:36)
    at Object.Module._extensions..js (node:module:737:12)
    at Module.load (node:module:652:34)
    at Function.Module._load (node:module:534:16)
    at Module.require (node:module:671:23)
    at require (node:module:771:20)
    at Object.<anonymous> (file:///Users/xx/Library/Caches/deno/npm/registry.npmjs.org/sharp/0.32.4/lib/constructor.js:11:1)
    at Object.<anonymous> (file:///Users/xx/Library/Caches/deno/npm/registry.npmjs.org/sharp/0.32.4/lib/constructor.js:441:4)

[lovell]

@VelinGeorgiev Best guess would be that denoland/deno@512d533 altered Deno to expect a native module to register itself using NODE_API_ADDON / napi_register_module_v1 and therefore potentially remove support for modules that use NODE_API_MODULE / napi_module_register. If intentional, this sounds like it might be a semver major change. Perhaps ask at the Deno repo about this - nodejs/node#45657 is relevant.

[VelinGeorgiev]

Thanks @lovell! I will ask at the Deno repo.

[oscarotero]

Is anyone aware of a better/easier way to get Deno to run install scripts, rather than manually running npm install?

In summary, we should add something to the documentation about this, but it looks like there are no code changes required. 🚀

@lovell I propose to change the way the binary is installed in sharp. Instead of using the install script, use dependencies. This allows to use the package in Deno without requiring any manual step.

For example, Esbuild publishes a binary for each platform/architecture supported as optionalDependencies: https://github.com/evanw/esbuild/blob/main/npm/esbuild/package.json

Pagefind does something similar: https://github.com/CloudCannon/pagefind/blob/main/wrappers/node/package.json

The package manager (NPM or Deno) pick automatically the dependency based on this suffix.

Edit: Sorry I just realized this is on the roadmap (great!): #3750

[some-user123]

I'm using sharp with deno:

import sharp from "npm:sharp@0.33.0";
...

That works great if I just run it with deno run ....
It doesn't work if I compile it deno compile ... and run the executable.

error: Uncaught (in promise) Error: Could not load the "sharp" module using the win32-x64 runtime                                                                                                             
undefined: LoadLibraryExW failed                                                                                                                                                                              
Possible solutions:
- Add platform-specific dependencies:
    npm install --os=win32 --cpu=x64 sharp
  or
    npm install --force @img/sharp-win32-x64
- Consult the installation documentation: https://sharp.pixelplumbing.com/install

Is that a know limitation? Or is there a solution?

[lovell]

@some-user123 For deno compile please see #3912