Electron on Windows: The specified module could not be found

[ivancuric]

Reproduction repo: https://github.com/ivancuric/native-repro-sharp

In this repo I am using 3 different native modules: an included one built with cmake (video-module), a dummy native module (native-hello-world), and sharp. I am running this on Windows 10.

Sharp is the only module that throws an error after the initial installation and build on yarn start, even though the module is present at the location:

D:\dev\native-repro-sharp\node_modules\sharp\lib\constructor.js:32 Uncaught Error: 
Something went wrong installing the "sharp" module

The specified module could not be found.
\\?\D:\dev\native-repro-sharp\node_modules\sharp\build\Release\sharp.node

- Remove the "node_modules/sharp" directory then run
  "npm install --ignore-scripts=false --verbose sharp" and look for errors
- Consult the installation documentation at https://sharp.pixelplumbing.com/install
- Search for this error at https://github.com/lovell/sharp/issues

    at Object.<anonymous> (D:\dev\native-repro-sharp\node_modules\sharp\lib\constructor.js:32)
    at Object.<anonymous> (D:\dev\native-repro-sharp\node_modules\sharp\lib\constructor.js:394)
    at Module._compile (internal/modules/cjs/loader.js:1078)
    at Object.Module._extensions..js (internal/modules/cjs/loader.js:1108)
    at Module.load (internal/modules/cjs/loader.js:935)
    at Module._load (internal/modules/cjs/loader.js:776)
    at Function.f._load (electron/js2c/asar_bundle.js:5)
    at Function.o._load (electron/js2c/renderer_init.js:33)
    at Module.require (internal/modules/cjs/loader.js:959)
    at require (internal/modules/cjs/helpers.js:88)

Removing node_modules/sharp doesn't help. What does help is running yarn install --force an additional time.

You can see the difference in the node_modules/sharp directory between the initial and second installation here:
https://editor.mergely.com/3aNte8dC/

What could be causing this? Seems that the install script in sharp's package.json isn't being run properly because running it manually afterwards from the node_modules directory works fine.

On the branch forge-hook-hack, I am running node node_modules/sharp/install/dll-copy after every rebuild:

$ node node_modules/sharp/install/dll-copy
sharp: Creating D:\dev\native-repro-sharp\node_modules\sharp\build\Release
sharp: Copying DLLs from D:\dev\native-repro-sharp\node_modules\sharp\vendor\8.10.6\lib to D:\dev\native-repro-sharp\node_modules\sharp\build\Release

[lovell]

Hi, does this still happen if you remove the native-addons/video-module dependency? If so, please can you remove that from native-repro-sharp to make it more minimal and therefore easier for someone else to try to reproduce.

[ivancuric]

It does, and I removed the module. The issue is (somewhat) resolved in the https://github.com/ivancuric/native-repro-sharp/tree/forge-hook-hack branch.

What I needed to do is run node node_modules/sharp/install/dll-copy after the initial rebuild that happens on the first run.

This is done before each app run in the forge config:

    generateAssets: async () => {
      return new Promise((resolve, reject) => {
        const npmInstall = spawn('yarn', ['fix-sharp'], {
          stdio: 'inherit',
          shell: true,
        });
        npmInstall.on('close', (code) => {
          if (code === 0) {
            resolve();
          } else {
            reject(new Error('process finished with error code ' + code));
          }
        });
        npmInstall.on('error', (error) => {
          reject(error);
        });
      });
    },

[lovell]

Thanks, it looks like you're affected by electron/rebuild#520

sharp provides prebuilt binaries based on Node-API (N-API) that are designed to "just work" with all supported versions of Node.js and Electron, but electron-rebuild is erroneously ignoring these.

[ivancuric]

Thanks for the input, gonna see if I can forward this issue!

[lovell]

I've heard from other sharp+Electron users (there are many) that electron-builder provides better support for Node-API based modules than electron-forge, so that might be worth investigating too.

[ivancuric]

Thanks, I'll check it out.

…

On Tue, Jun 29, 2021, 10:36 Lovell Fuller ***@***.***> wrote: I've heard from other sharp+Electron users (there are many) that electron-builder provides better support for Node-API based modules than electron-forge, so that might be worth investigating too. — You are receiving this because you authored the thread. Reply to this email directly, view it on GitHub <#2764 (comment)>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/AAVSGJ56LLPFYAERWD6HSRDTVGAXDANCNFSM47FSYDUA> .

[hiukky]

@lovell i'm having the same problem with electron-builder with asar enabled.

[timfish]

I've been looking into this more and I'm working on a fix for the above mentioned issue with electron-rebuild where it fails to pass the correct parameters to prebuild-install.

However, if loading a prebuilt binary fails, electron-rebuild falls back to building from source. In my testing, it does in fact do this and it completes successfully. It looks like there is something wrong with the binary output which stops it from being loaded at runtime.

@ivancuric reports this which appears to suggest he has a similar result:

even though the module is present at the location

Is sharp expected to work when building from source, or does it only work from prebuilt binaries?

I've tried ref-napi and some other Node-API modules and electron-rebuild successfully builds them from source and they load correctly.

[lovell]

@timfish Thank you for working on a fix for this. sharp can be built from source and should work, but for Windows note the DLL-copying command that runs as part of npm install:

sharp/package.json

Line 83 in 7467fa8

"install": "(node install/libvips && node install/dll-copy && prebuild-install) || (node install/can-compile && node-gyp rebuild && node install/dll-copy)",

I took a quick look at your PR. Perhaps running electron-rebuild should be a "no-op" when using Node-API as the existing file(s) already in place will work?

[timfish]

electron-rebuild is often used to compile binaries for different architectures to the current one so you can't rely on the existing binaries being good. For example most devs will be using x64 node but many Electron apps are packaged for ia32 on Windows so they run on both platforms.

There is also the issue that there's no easy way to detect if a module uses Node-API. There's binary.napi_versions in package.json but that is only there for modules that use prebuild tools. ref-napi for example doesn't have anything obvious to denote that it's Node-API.

Whereas the node ecosystem relies on install scripts to copy binaries with the platform and arch swapped via environment variables, electron-rebuild instead relies on directly calling node-gyp, prebuild-install, etc. This means that it's currently not possible to release a native module built with anything other than gyp. It's a bit of a mess 🤷‍♂️

[lovell]

The use of prebuild-install will be going away with #3750 - please subscribe to it for updates. (I'm unsure how this will impact the various Electron packagers specifically but the proposed approach is designed to be more package manager friendly overall.)

[LZQCN]

The specified module could not be found.

The reason for this error may be the .node file cannot find its required .dll files.

[lovell]

v0.33.0 is now available and no longer depends on prebuild-install - everything is controlled via your choice of package manager.