End-user yarn dependency resolution fails due to remote archive doesn't match the expected checksum

[dustinbyrne]

E.g.

➤ YN0018: │ d3-path@npm:1.0.9: The remote archive doesn't match the expected checksum
➤ YN0018: │ d3-selection@npm:1.4.2: The remote archive doesn't match the expected checksum
➤ YN0018: │ crypto-js@npm:4.1.1: The remote archive doesn't match the expected checksum
➤ YN0018: │ d3-drag@npm:1.2.5: The remote archive doesn't match the expected checksum
➤ YN0018: │ d3-shape@npm:1.3.7: The remote archive doesn't match the expected checksum
➤ YN0018: │ d3-timer@npm:1.0.10: The remote archive doesn't match the expected checksum
➤ YN0018: │ d3-transition@npm:1.3.2: The remote archive doesn't match the expected checksum
➤ YN0018: │ d3-zoom@npm:1.8.3: The remote archive doesn't match the expected checksum
➤ YN0018: │ dashdash@npm:1.14.1: The remote archive doesn't match the expected checksum
➤ YN0018: │ data-urls@npm:2.0.0: The remote archive doesn't match the expected checksum
➤ YN0018: │ debounce-fn@npm:4.0.0: The remote archive doesn't match the expected checksum
➤ YN0018: │ debug@npm:4.3.4: The remote archive doesn't match the expected checksum
➤ YN0018: │ decompress-response@npm:4.2.1: The remote archive doesn't match the expected checksum
➤ YN0018: │ dagre@npm:0.8.5: The remote archive doesn't match the expected checksum
➤ YN0000: └ Completed in 0s 583ms
➤ YN0000: Failed with errors in 3s 118ms

[symwell]

Log.

dustinbyrne asks "Do we need checksumBehavior: reset here?"

Theres' a suggestion to use

YARN_CHECKSUM_BEHAVIOR=update yarn

It'll update the lockfile with the new cache checksums.

[symwell]

Reproduced in appmap-js by editing yarn.lock and changing one of the checksums to be incorrect.

test@work[2022-12-16_13:51:09]:~/src/appmap-js$ yarn
➤ YN0000: ┌ Resolution step
...
➤ YN0000: ┌ Fetch step
➤ YN0013: │ @ampproject/remapping@npm:2.1.1 can't be found in the cache and will be fetched from the remote registry
➤ YN0018: │ @ampproject/remapping@npm:2.1.1: The remote archive doesn't match the expected checksum
➤ YN0000: └ Completed in 1s 84ms
➤ YN0000: Failed with errors in 1s 574ms

Fixed by running

test@work[2022-12-16_13:51:15]:~/src/appmap-js$ YARN_CHECKSUM_BEHAVIOR=update yarn
➤ YN0000: ┌ Resolution step
...
➤ YN0000: ┌ Fetch step
➤ YN0013: │ @ampproject/remapping@npm:2.1.1 can't be found in the cache and will be fetched from the remote registry
➤ YN0000: └ Completed in 0s 776ms
➤ YN0000: ┌ Link step
➤ YN0000: └ Completed in 1s 501ms
➤ YN0000: Done with warnings in 3s 263ms

Running with reset didn't fix it.

test@work[2022-12-16_14:13:14]:~/src/appmap-js$ YARN_CHECKSUM_BEHAVIOR=reset yarn
➤ YN0000: ┌ Resolution step
...
➤ YN0000: ┌ Fetch step
➤ YN0018: │ @ampproject/remapping@npm:2.1.1: The remote archive doesn't match the expected checksum
➤ YN0000: └ Completed in 0s 633ms
➤ YN0000: Failed with errors in 1s 125ms

[dividedmind]

While I'm not against this, I still don't understand why we would need this or what it would fix exactly. If so many checksums don't match as in the original report, obviously there is something weird going on with the network and the archives are coming back corrupted (perhaps a captive proxy is returning an error page instead of them).

Did we see any instance of checksum corruption that doesn't fit this pattern?

[symwell]

It could fix yarn calculating an incorrect mtime to produce/verify .zip archives. Maybe users running a version of yarn without this bug fix see this error.

yarn pack seems to produce different output per OS.

It could also be something weird with the network.

[symwell]

Did we see any instance of checksum corruption that doesn't fit this pattern?

I didn't. I spot checked 10+ instances of this error. There is some variance in the list of packages but the checksum corruption pattern is the same.

[symwell]

I don't see this error in the logs anymore. Closing.