Preview build setup sometimes results in installation failures

[Gudahtt]

It seems that the GitHub registry we have setup for the preview builds isn't mirroring packages from npm.

As a consequence, you may see installation failures non-preview @metamask packages when using an .npmrc file setup to point at the GitHub registry. I've found that I can get past the error by ensuring the non-preview packages are installed locally before putting the .npmrc file in-place. This is a viable workaround but it requires frequently adding and removing this file, which is rather inconvenient.

[Gudahtt]

I was unable to find any way to configure Yarn/npm or GitHub to support looking at this custom registry first, then falling back to npm.

We may want to investigate alternatives to GitHub Packages. Or perhaps we could write a script to sync all of our npm packages to the GitHub registry.

[BelfordZ]

can we publish previews to npm instead?

solves:

1. does not require a token to view previews
2. doesnt require any yarnrc configuration to use them

tradeoffs:

1. gh action will require an npm token that can publish releases. We can resolve security risk by pushing them to a different org (@metamask-previews for example)
2. ??

[mcmire]

@BelfordZ This does seem like the easiest plan. @Gudahtt What say you to using a different NPM scope?

[Gudahtt]

As long as we can resolve to the new package name, that works. I'm not sure if that's possible with Yarn v3 for transitive dependencies.

e.g. if we have a new @metamask/eth-sig-util@5.0.1 release, how do we point all copies of @metamask/eth-sig-util@5.0.0 in our dependency tree at @metamask-previews/eth-sig-util@something?

Maybe Yarn resolutions can do this? Not sure.

[mcmire]

I think that Yarn resolutions can accomplish this as well, but we would definitely need to test this out somehow. I can try experimenting with this.

[xinnanyemm]

Hey team! Please add your planning poker estimate with Zenhub @cryptodev-2s @Gudahtt @mcmire @mikesposito

[mcmire]

I tried to test the above idea by using Verdaccio to publish a @metamask-previews package locally. However, I could not get Verdaccio to work with Yarn v3. After pointing Yarn to using http://localhost:4873 instead of the Yarn registry, I kept getting an ECONNREFUSED error. I'll have to try something else, perhaps using the GitHub registry instead.

[mcmire]

I've tested this using the GitHub registry instead and confirmed that the resolutions field works to get Yarn to "see" one package under another name in the dependency tree.

I first added the following to .yarnrc.yml in this repo:

npmRegistries:
  "https://npm.pkg.github.com":
    npmAlwaysAuth: true
    npmAuthToken: "${GITHUB_NPM_TOKEN-}"

npmScopes:
  mcmire:
    npmRegistryServer: "https://npm.pkg.github.com"

I then opened the package.json for controller-utils and:

• changed its name from @metamask/controller-utils to @mcmire/controller-utils in package.json
• changed the version by appending -preview.aaa to the end
• updated publishConfig.registry to "https://npm.pkg.github.com"

I then found every instance of @metamask/controller-utils in package.json across the whole monorepo and replaced it with @mcmire/controller-utils, then ran:

yarn install

I then ran:

GITHUB_NPM_TOKEN="<token>" yarn workspace @mcmire/controller-utils npm publish --tag preview

Over in the extension I then added the same section to .yarnrc.yml as above. When I added this to the resolutions field:

"@metamask/controller-utils": "npm:@mcmire/controller-utils@preview"

and ran:

GITHUB_NPM_TOKEN="<token>" yarn install

and then

yarn why @metamask/controller-utils

I saw that every instance of controller-utils in the dependency tree was replaced:

├─ @metamask/address-book-controller@npm:3.1.0
│  └─ @mcmire/controller-utils@npm:4.3.1-preview.aaa::__archiveUrl=https%3A%2F%2Fnpm.pkg.github.com%2Fdownload%2F%40mcmire%2Fcontroller-utils%2F4.3.1-preview.aaa%2F5d1425f907a37d76fc29c60e898e4e859fdc0f5c (via npm:@mcmire/controller-utils@latest)
│
├─ @metamask/assets-controllers@npm:9.2.0
│  └─ @mcmire/controller-utils@npm:4.3.1-preview.aaa::__archiveUrl=https%3A%2F%2Fnpm.pkg.github.com%2Fdownload%2F%40mcmire%2Fcontroller-utils%2F4.3.1-preview.aaa%2F5d1425f907a37d76fc29c60e898e4e859fdc0f5c (via npm:@mcmire/controller-utils@latest)
│
├─ @metamask/assets-controllers@npm:9.2.0 [15918]
│  └─ @mcmire/controller-utils@npm:4.3.1-preview.aaa::__archiveUrl=https%3A%2F%2Fnpm.pkg.github.com%2Fdownload%2F%40mcmire%2Fcontroller-utils%2F4.3.1-preview.aaa%2F5d1425f907a37d76fc29c60e898e4e859fdc0f5c (via npm:@mcmire/controller-utils@latest)
│
├─ @metamask/base-controller@npm:2.0.0
│  └─ @mcmire/controller-utils@npm:4.3.1-preview.aaa::__archiveUrl=https%3A%2F%2Fnpm.pkg.github.com%2Fdownload%2F%40mcmire%2Fcontroller-utils%2F4.3.1-preview.aaa%2F5d1425f907a37d76fc29c60e898e4e859fdc0f5c (via npm:@mcmire/controller-utils@latest
...