Changed command fails when hotloaded with pnpm dlx in version 7 #3807

SF97 · 2023-08-15T22:53:05Z

Current Behavior

I use "changed" command in a pnpm repo in CI/CD pipelines. It was working fine until version 6. But the command fails after upgrading to version 7 of Lerna

pnpm dlx lerna@7.0.0 changed --include-merged-tags --all --json

Output:

.../share/pnpm/store/v3/tmp/dlx-123579   | +523 ++++++++++++++++++++++++++++++++++++++++++++++++++++
.../share/pnpm/store/v3/tmp/dlx-123579   | Progress: resolved 531, reused 520, downloaded 3, added 523, done
lerna notice cli v7.0.0
lerna ERR! TypeError [ERR_INVALID_ARG_TYPE]: Failed to process the project graph with "nx-js-graph-plugin".
lerna ERR! The "data" argument must be of type string or an instance of Buffer, TypedArray, or DataView. Received undefined
lerna ERR!     at writeFileSync (node:fs:2314:5)
lerna ERR!     at writeLastProcessedLockfileHash (/home/<redacted>/.local/share/pnpm/store/v3/tmp/dlx-123579/node_modules/.pnpm/nx@16.7.0/node_modules/nx/src/plugins/js/index.js:53:28)
lerna ERR!     at /home/<redacted>/.local/share/pnpm/store/v3/tmp/dlx-123579/node_modules/.pnpm/nx@16.7.0/node_modules/nx/src/plugins/js/index.js:26:17
lerna ERR!     at Generator.next (<anonymous>)
lerna ERR!     at /home/<redacted>/.local/share/pnpm/store/v3/tmp/dlx-123579/node_modules/.pnpm/tslib@2.6.1/node_modules/tslib/tslib.js:169:75
lerna ERR!     at new Promise (<anonymous>)
lerna ERR!     at Object.__awaiter (/home/<redacted>/.local/share/pnpm/store/v3/tmp/dlx-123579/node_modules/.pnpm/tslib@2.6.1/node_modules/tslib/tslib.js:165:16)
lerna ERR!     at Object.processProjectGraph (/home/<redacted>/.local/share/pnpm/store/v3/tmp/dlx-123579/node_modules/.pnpm/nx@16.7.0/node_modules/nx/src/plugins/js/index.js:16:57)
lerna ERR!     at /home/<redacted>/.local/share/pnpm/store/v3/tmp/dlx-123579/node_modules/.pnpm/nx@16.7.0/node_modules/nx/src/project-graph/build-project-graph.js:135:42
lerna ERR! lerna Failed to process the project graph with "nx-js-graph-plugin".
lerna ERR! lerna The "data" argument must be of type string or an instance of Buffer, TypedArray, or DataView. Received undefined
lerna ERR! lerna 
lerna ERR! lerna Because of the error the Nx daemon process has exited. The next Nx command is going to restart the daemon process.
lerna ERR! lerna If the error persists, please run "nx reset".

When using latest version, the error is exactly the same. Looks like it was introduced in version 7

When using the latest patch of version 6, everything works correctly:

pnpm dlx lerna@6.6.2 changed --include-merged-tags --all --json

Output:

.../share/pnpm/store/v3/tmp/dlx-124820   | +603 ++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++
.../share/pnpm/store/v3/tmp/dlx-124820   | Progress: resolved 610, reused 603, downloaded 0, added 603, done
lerna notice cli v6.6.2
lerna info Looking for changed packages since 1.0.0
lerna info No changed packages found

Expected Behavior

Lerna 7 "changed" command should give the same exact output as version 6

Steps to Reproduce

Clone this repository: https://github.com/SF97/lerna-bug-demonstration
Run pnpm dlx lerna@7.1.5 changed --include-merged-tags --all --json at the root of the repository

Failure Logs / Configuration

lerna.json

{
  "packages": [
    "packages/lib/shared",
    "packages/lib/backend/*",
    "packages/services/**"
  ],
  "npmClient": "pnpm",
  "version": "1.0.0",
  "$schema": "node_modules/lerna/schemas/lerna-schema.json"
}

lerna-debug.log

0 silly argv {
0 silly argv   _: [ 'changed' ],
0 silly argv   'include-merged-tags': true,
0 silly argv   includeMergedTags: true,
0 silly argv   all: true,
0 silly argv   a: true,
0 silly argv   json: true,
0 silly argv   lernaVersion: '7.1.5',
0 silly argv   '$0': '../../.local/share/pnpm/store/v3/tmp/dlx-126483/node_modules/lerna/dist/cli.js'
0 silly argv }
1 notice cli v7.1.5
2 verbose packageConfigs Explicit "packages" configuration found in lerna.json. Resolving packages using the configured glob(s): ["<redacted>"]
3 verbose rootPath /home/<redacted>/Desktop/<redacted>
4 error TypeError [ERR_INVALID_ARG_TYPE]: Failed to process the project graph with "nx-js-graph-plugin".
4 error The "data" argument must be of type string or an instance of Buffer, TypedArray, or DataView. Received undefined
4 error     at writeFileSync (node:fs:2314:5)
4 error     at writeLastProcessedLockfileHash (/home/<redacted>/.local/share/pnpm/store/v3/tmp/dlx-126483/node_modules/.pnpm/nx@16.7.0/node_modules/nx/src/plugins/js/index.js:53:28)
4 error     at /home/<redacted>/.local/share/pnpm/store/v3/tmp/dlx-126483/node_modules/.pnpm/nx@16.7.0/node_modules/nx/src/plugins/js/index.js:26:17
4 error     at Generator.next (<anonymous>)
4 error     at /home/<redacted>/.local/share/pnpm/store/v3/tmp/dlx-126483/node_modules/.pnpm/tslib@2.6.1/node_modules/tslib/tslib.js:169:75
4 error     at new Promise (<anonymous>)
4 error     at Object.__awaiter (/home/<redacted>/.local/share/pnpm/store/v3/tmp/dlx-126483/node_modules/.pnpm/tslib@2.6.1/node_modules/tslib/tslib.js:165:16)
4 error     at Object.processProjectGraph (/home/<redacted>/.local/share/pnpm/store/v3/tmp/dlx-126483/node_modules/.pnpm/nx@16.7.0/node_modules/nx/src/plugins/js/index.js:16:57)
4 error     at /home/<redacted>/.local/share/pnpm/store/v3/tmp/dlx-126483/node_modules/.pnpm/nx@16.7.0/node_modules/nx/src/project-graph/build-project-graph.js:135:42

daemon.log

 >  NX   Failed to parse pnpm lockfile

   Please open an issue at `https://github.com/nrwl/nx/issues/new?template=1-bug.yml` and provide a reproduction.
   
Original error: Could not find ".modules.yaml" at "/home/<redacted>/Desktop/<redacted>/node_modules/.modules.yaml"


   Error: Could not find ".modules.yaml" at "/home/<redacted>/Desktop/<redacted>/node_modules/.modules.yaml"
    at loadPnpmHoistedDepsDefinition (/home/<redacted>/.local/share/pnpm/store/v3/tmp/dlx-136550/node_modules/.pnpm/nx@16.7.0/node_modules/nx/src/plugins/js/lock-file/utils/pnpm-normalizer.js:25:15)
    at addNodes (/home/<redacted>/.local/share/pnpm/store/v3/tmp/dlx-136550/node_modules/.pnpm/nx@16.7.0/node_modules/nx/src/plugins/js/lock-file/pnpm-parser.js:51:77)
    at parsePnpmLockfile (/home/<redacted>/.local/share/pnpm/store/v3/tmp/dlx-136550/node_modules/.pnpm/nx@16.7.0/node_modules/nx/src/plugins/js/lock-file/pnpm-parser.js:13:5)
    at parseLockFile (/home/<redacted>/.local/share/pnpm/store/v3/tmp/dlx-136550/node_modules/.pnpm/nx@16.7.0/node_modules/nx/src/plugins/js/lock-file/lock-file.js:79:49)
    at /home/<redacted>/.local/share/pnpm/store/v3/tmp/dlx-136550/node_modules/.pnpm/nx@16.7.0/node_modules/nx/src/plugins/js/index.js:25:64
    at Generator.next (<anonymous>)
    at /home/<redacted>/.local/share/pnpm/store/v3/tmp/dlx-136550/node_modules/.pnpm/tslib@2.6.1/node_modules/tslib/tslib.js:169:75
    at new Promise (<anonymous>)
    at Object.__awaiter (/home/<redacted>/.local/share/pnpm/store/v3/tmp/dlx-136550/node_modules/.pnpm/tslib@2.6.1/node_modules/tslib/tslib.js:165:16)
    at Object.processProjectGraph (/home/<redacted>/.local/share/pnpm/store/v3/tmp/dlx-136550/node_modules/.pnpm/nx@16.7.0/node_modules/nx/src/plugins/js/index.js:16:57)

[NX Daemon Server] - 2023-08-15T22:32:44.789Z - Error detected when creating a project graph: Failed to process the project graph with "nx-js-graph-plugin".
The "data" argument must be of type string or an instance of Buffer, TypedArray, or DataView. Received undefined
[NX Daemon Server] - 2023-08-15T22:32:44.789Z - [REQUEST]: Responding to the client with an error. Error when preparing serialized project graph. Failed to process the project graph with "nx-js-graph-plugin".
The "data" argument must be of type string or an instance of Buffer, TypedArray, or DataView. Received undefined
TypeError [ERR_INVALID_ARG_TYPE]: Failed to process the project graph with "nx-js-graph-plugin".
The "data" argument must be of type string or an instance of Buffer, TypedArray, or DataView. Received undefined
    at writeFileSync (node:fs:2314:5)
    at writeLastProcessedLockfileHash (/home/sergio/.local/share/pnpm/store/v3/tmp/dlx-136550/node_modules/.pnpm/nx@16.7.0/node_modules/nx/src/plugins/js/index.js:53:28)
    at /home/sergio/.local/share/pnpm/store/v3/tmp/dlx-136550/node_modules/.pnpm/nx@16.7.0/node_modules/nx/src/plugins/js/index.js:26:17
    at Generator.next (<anonymous>)
    at /home/sergio/.local/share/pnpm/store/v3/tmp/dlx-136550/node_modules/.pnpm/tslib@2.6.1/node_modules/tslib/tslib.js:169:75
    at new Promise (<anonymous>)
    at Object.__awaiter (/home/sergio/.local/share/pnpm/store/v3/tmp/dlx-136550/node_modules/.pnpm/tslib@2.6.1/node_modules/tslib/tslib.js:165:16)
    at Object.processProjectGraph (/home/sergio/.local/share/pnpm/store/v3/tmp/dlx-136550/node_modules/.pnpm/nx@16.7.0/node_modules/nx/src/plugins/js/index.js:16:57)
    at /home/sergio/.local/share/pnpm/store/v3/tmp/dlx-136550/node_modules/.pnpm/nx@16.7.0/node_modules/nx/src/project-graph/build-project-graph.js:135:42
    at Generator.next (<anonymous>) {
  code: 'ERR_INVALID_ARG_TYPE'
}
[NX Daemon Server] - 2023-08-15T22:32:44.791Z - Done responding to the client null

Environment

It started happening in CI/CD, but I was able to reproduce it locally

Additional information

This looks like an issue with NX, not with Lerna. I've reported here since Lerna is the one who triggered the issue for me, but I'm happy to move it to NX's repo if needed

When Lerna is a dependency of the project, and I the same command without dlx, it works correctly. Looks like its something related to hotloading the module

The text was updated successfully, but these errors were encountered:

fahslaj · 2023-08-21T18:38:46Z

Hi @SF97 , thank you for reporting this issue. This appears to happen because pnpm install has not been run in the source directory. This has been fixed in the latest version (v16.7.2) of Nx. In this case, an error exit code will no longer be returned.

However, it is still recommended to perform an install command before running any Lerna or Nx operations. Metadata generated from the install command is used to create a project graph with accurate dependencies. Because of this, just "hotloading" Nx or Lerna is not recommended.

SF97 · 2023-08-22T20:38:25Z

Hi @fahslaj

Thank you very much for the reply and for the quick fix :)

Thanks for the recommendation. I'll install the dependencies before running the Lerna changed command. But I guess it will be supported whenever Lerna upgrades NX to v16.7.2, even if not recommended. Would it be fine if I made a PR that upgrades to that NX version, but also adds a warning in the docs to mention the limitation?

fahslaj · 2023-08-22T21:28:55Z

@SF97 Even though the Lerna repo is currently only on Nx 16.5.1, the Lerna package itself depends on a range of Nx: >=16.5.1 < 17, so the fix should be pulled in automatically as pnpm dlx defaults to the latest version of nx that will satisfy the range.

A PR to add a warning to the docs mentioning this limitation would be greatly appreciated!

SF97 · 2023-08-24T21:56:07Z

Hello @fahslaj

I created a PR to address this :) #3817

SF97 · 2023-08-26T17:32:11Z

Closing this task since #3817 has been merged

SF97 added the type: bug label Aug 15, 2023

SF97 mentioned this issue Aug 17, 2023

16.4: Failed to process the project graph with "nx-js-graph-plugin" nrwl/nx#17987

Closed

4 tasks

fahslaj mentioned this issue Aug 17, 2023

fix(js): fail silently on missing .modules.yaml file nrwl/nx#18680

Closed

meeroslav mentioned this issue Aug 17, 2023

fix(core): nx should not break if packages were not installed nrwl/nx#18687

Merged

SF97 mentioned this issue Aug 24, 2023

docs(website): adds hotloading limitation section to commands #3817

Merged

10 tasks

SF97 closed this as completed Aug 26, 2023

jaysoo mentioned this issue Aug 29, 2023

fix(core): check for lerna before parsing lockfiles to prevent errors nrwl/nx#18884

Merged

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Changed command fails when hotloaded with pnpm dlx in version 7 #3807

Changed command fails when hotloaded with pnpm dlx in version 7 #3807

SF97 commented Aug 15, 2023 •

edited

Loading

fahslaj commented Aug 21, 2023 •

edited

Loading

SF97 commented Aug 22, 2023

fahslaj commented Aug 22, 2023

SF97 commented Aug 24, 2023

SF97 commented Aug 26, 2023

Changed command fails when hotloaded with pnpm dlx in version 7 #3807

Changed command fails when hotloaded with pnpm dlx in version 7 #3807

Comments

SF97 commented Aug 15, 2023 • edited Loading

Current Behavior

Expected Behavior

Steps to Reproduce

Failure Logs / Configuration

Environment

Additional information

fahslaj commented Aug 21, 2023 • edited Loading

SF97 commented Aug 22, 2023

fahslaj commented Aug 22, 2023

SF97 commented Aug 24, 2023

SF97 commented Aug 26, 2023

SF97 commented Aug 15, 2023 •

edited

Loading

fahslaj commented Aug 21, 2023 •

edited

Loading