`eth-json-rpc-provider` migration - Integration into `packages/` #1738

MajorLift · 2023-09-27T18:53:05Z

Explanation

This PR implements the following incremental steps in the process for migrating eth-json-rpc-provider into the core monorepo:

Phase B: Staging from `merged-packages/`

5. Port tags

See: Add script for tag porting when migrating libraries into the core monorepo #1800

Push ported tags to core repo

Verify that the tag diff links in CHANGELOG are working

Phase C: Integration into `packages/`

1. The big leap

Move migration target from merged-packages/ to packages/.
Run yarn install in the root directory.
Check that all tests are passing in migration target by running yarn workspace @metamask/<package-name> test.

2. Update downstream repos

Add tsconfig reference paths for migration target in downstream packages and root.
Bump migration target version in downstream packages and root.

3. Linter fixes

Apply yarn constraints fixes to migration target package.json file: yarn constraints --fix (run twice).
Identify validator fixes for CHANGELOG using yarn workspace @metamask/<package-name> changelog:validate and apply the diffs.

4. Resolve downstream errors

deps: Bump @metamask/{eth-json-rpc-provider,rpc-errors} #1653
- If introducing the migration target breaks any downstream repos:
  - Resolve simple errors
  - Mark and ignore complex errors using @ts-expect-error TODO: annotations.
- Create a separate issue for resolving the marked errors as soon as the migration is completed.

5. Finalize merge

Check that all tests are passing in all subpackages of core and CI.
Merge packages/<package-name> directory into core main branch.

See #1551 (comment) for an outline of the entire process.

Next Steps

The next PR(s) will implement the final steps of the migration process (D-1 in the migration checklist).

Blocked by

Dependencies:
- typescript bump: Update typescript v4.8 #1718
- @metamask/utils bump: deps: @metamask/utils@^6.2.0->^8.1.0 #1639
Downstream type errors:
Tag porting:
- Add script for tag porting when migrating libraries into the core monorepo #1802
- "Unreleased" tag diff link shows entire history of core: https://github.com/MetaMask/core/compare/@metamask/eth-json-rpc-provider@2.2.0...HEAD

References

Contributes to Migrate @metamask/eth-json-rpc-provider to core monorepo #1685
Contributes to Fully migrate eth-json-rpc-provider into core monorepo #1551

Changelog

`@metamask/eth-json-rpc-provider`

CHANGED: Export SafeEventEmitterProvider as a value (class), not just a type.

`@metamask/network-controller`

CHANGED: Bump dependency on @metamask/eth-json-rpc-provider to ^2.2.0

Checklist

I've updated the test suite for new or updated code as appropriate
I've updated documentation (JSDoc, Markdown, etc.) for new or updated code as appropriate
I've highlighted breaking changes using the "BREAKING" category above as appropriate

socket-security · 2023-09-27T18:54:14Z

New dependencies detected. Learn more about Socket for GitHub ↗︎

Packages	Version	New capabilities	Transitives	Size	Publisher
jest-it-up	2.0.2	None	`+2`	206 kB	rbardini

socket-security · 2023-09-27T18:54:17Z

👍 Dependency issues cleared. Learn more about Socket for GitHub ↗︎

This PR previously contained dependency changes with security issues that have been resolved, removed, or ignored.

Ignoring: json5@2.2.1, convert-source-map@2.0.0

Next steps

Take a deeper look at the dependency

Take a moment to review the security alert above. Review the linked package source code to understand the potential risk. Ensure the package is not malicious before proceeding. If you're unsure how to proceed, reach out to your security team or ask the Socket team for help at support [AT] socket [DOT] dev.

Remove the package

If you happen to install a dependency that Socket reports as Known Malware you should immediately remove it and select a different dependency. For other alert types, you may may wish to investigate alternative packages or consider if there are other ways to mitigate the specific risk posed by the dependency.

Mark a package as acceptable risk

To ignore an alert, reply with a comment starting with @SocketSecurity ignore followed by a space separated list of package-name@version specifiers. e.g. @SocketSecurity ignore foo@1.0.0 bar@* or ignore all packages with @SocketSecurity ignore-all

MajorLift · 2023-09-27T20:18:05Z

@SocketSecurity ignore-all

All packages versions are current with main branch.

mcmire

Marking this as "request changes" because of the change to network-controller (see comment there).

mcmire · 2023-10-12T23:23:42Z

packages/eth-json-rpc-provider/src/index.test.ts

@@ -4,6 +4,7 @@ describe('Package exports', () => {
  it('has expected exports', () => {
    expect(Object.keys(allExports)).toMatchInlineSnapshot(`
      Array [
+        "SafeEventEmitterProvider",


What's the reason for this?

This one follows from the export statement change for SafeEventEmitterProvider https://github.com/MetaMask/core/pull/1738/files#r1357548245. The test is checking what non-types are being exported from index.ts.

mcmire · 2023-10-12T23:24:01Z

packages/eth-json-rpc-provider/src/index.ts

@@ -1,3 +1,3 @@
 export * from './provider-from-engine';
 export * from './provider-from-middleware';
-export type { SafeEventEmitterProvider } from './safe-event-emitter-provider';


This changes what is being exported (the whole class instead of just the type). What's the reason for this change?

See: first line of network-controller/tests/fake-provider.

Okay, no problem. I guess this wouldn't be a breaking change, so it's okay.

mcmire · 2023-10-12T23:25:10Z

packages/network-controller/package.json

@@ -33,7 +33,7 @@
    "@metamask/controller-utils": "^5.0.2",
    "@metamask/eth-json-rpc-infura": "^9.0.0",
    "@metamask/eth-json-rpc-middleware": "^12.0.0",
-    "@metamask/eth-json-rpc-provider": "^2.1.0",
+    "@metamask/eth-json-rpc-provider": "^2.2.0",


Hmm. Given that we are affecting other packages in this PR, should we bump everything to ^2.2.0 first? I think it'd be better to keep this PR just focused to eth-json-rpc-provider if we can.

This is necessary because of the reference path shadowing thing we discussed. Once eth-json-rpc-provider is moved into packages/, downstream packages can no longer use previous versions by pointing to node_modules/, because all imports from @metamask/eth-json-rpc-provider will always point to the path in packages/, and therefore to the latest version. This seems to happen even if the migrated package isn't added to the reference paths in the downstream pkgs' typescript config files.

That makes sense, but isn't this happening because network-controller is currently using version 2.1.0? If we bumped that first then the latest version of the package being used and current version of the package as it exists after being migrated would be the same.

We could bump eth-json-rpc-provider to 2.2.0 everywhere first in a separate PR, but that PR wouldn't include the changes being made to network-controller/ here (the last two commits).

Because fixing the ../dist/.. import statements requires fixing that export statement in eth-json-rpc-provider, we'd still end up with changes being made to both packages simultaneously before the migration can be merged.

I guess we could change the SafeEventEmitterProvider export statement in the original repo and release 2.2.1 first, but that would mean redoing the migration process from the start because of the new git history.

MajorLift · 2023-10-12T23:41:05Z

packages/network-controller/tests/fake-provider.ts

@@ -1,4 +1,4 @@
-import { SafeEventEmitterProvider } from '@metamask/eth-json-rpc-provider/dist/safe-event-emitter-provider';
+import { SafeEventEmitterProvider } from '@metamask/eth-json-rpc-provider';


See line 85:

export class FakeProvider extends SafeEventEmitterProvider {

Importing SafeEventEmitterProvider as a type breaks that line, because it's being used as a class.

This is why the previous import statement pointed directly to node_modules/.../dist/... instead of using the SafeEventEmitterProvider type exported by the package.

That import statement can no longer be used (it won't point to node_modules with eth-json-rpc-provider in packages/) which is why the export from eth-json-rpc-provider needs to be fixed from type to class.

Ah, I see. Thank you for explaining, somehow that was not connecting.

MajorLift · 2023-10-13T15:48:51Z

Marking this as "request changes" because of the change to network-controller (see comment there).

@mcmire Are there still any changes to the PR you'd like to see, or maybe a different approach we could try for the network-controller fixes?

…downstream packages

…ages

Removing this as I've determined that the network controller changes are non-breaking.

mcmire

I'm good with this PR!

cryptodev-2s

LGTM!

…changes from recent releases (#1807) ## Explanation - Updating root README to include @metamask/eth-json-rpc-provider. - Updating dependency graph to reflect changes made between releases v78.0.0 through v81.0.0. ## References - See #1738 ## Changelog ### `core` - **CHANGED**: Updated README dependency graph. ## Checklist - [ ] I've updated the test suite for new or updated code as appropriate - [ ] I've updated documentation (JSDoc, Markdown, etc.) for new or updated code as appropriate - [ ] I've highlighted breaking changes using the "BREAKING" category above as appropriate

MajorLift force-pushed the 230927-integrate-eth-json-rpc-provider branch from 8175a45 to 4984cb6 Compare September 27, 2023 18:56

MajorLift self-assigned this Sep 27, 2023

MajorLift added enhancement New feature or request dependencies Pull requests that update a dependency file team-wallet-framework Epic and removed Epic labels Sep 27, 2023

MajorLift mentioned this pull request Oct 2, 2023

Bump @metamask/eth-json-rpc-provider from 1.0.0 to 2.2.0 #1750

Closed

MajorLift force-pushed the 230927-integrate-eth-json-rpc-provider branch 3 times, most recently from 046290c to fcd3b95 Compare October 2, 2023 18:34

This was referenced Oct 2, 2023

Bump semver from 6.3.0 to 6.3.1 in /merged-packages/eth-json-rpc-provider #1751

Closed

deps: Bump @metamask/{eth-json-rpc-provider,rpc-errors} #1653

Merged

MajorLift changed the title ~~Migrate eth-json-rpc-provider into core monorepo (2/2)~~ eth-json-rpc-provider migration - C: Integration into packages/ Oct 3, 2023

MajorLift force-pushed the 230927-integrate-eth-json-rpc-provider branch from b6c95cc to 43fae9b Compare October 3, 2023 20:52

MajorLift mentioned this pull request Oct 3, 2023

Fully migrate eth-json-rpc-provider into core monorepo #1551

Closed

MajorLift changed the base branch from main to 231003-staging-eth-json-rpc-provider October 4, 2023 01:07

MajorLift changed the base branch from 231003-staging-eth-json-rpc-provider to main October 5, 2023 22:28

MajorLift changed the base branch from main to 231003-staging-eth-json-rpc-provider October 5, 2023 22:28

MajorLift changed the base branch from 231003-staging-eth-json-rpc-provider to main October 5, 2023 22:30

MajorLift changed the base branch from main to 231003-staging-eth-json-rpc-provider October 5, 2023 22:39

MajorLift changed the base branch from 231003-staging-eth-json-rpc-provider to main October 8, 2023 04:09

MajorLift force-pushed the 230927-integrate-eth-json-rpc-provider branch from a014d40 to ebcfc39 Compare October 11, 2023 17:01

MajorLift mentioned this pull request Oct 11, 2023

eth-json-rpc-provider migration - D: Repair breaking packages and finalize merge #1770

Closed

9 tasks

MajorLift force-pushed the 230927-integrate-eth-json-rpc-provider branch from 615fb8e to 4412ff7 Compare October 11, 2023 17:22

MajorLift marked this pull request as ready for review October 11, 2023 17:30

MajorLift requested a review from a team as a code owner October 11, 2023 17:30

MajorLift force-pushed the 230927-integrate-eth-json-rpc-provider branch from 03c8d0d to 69a276d Compare October 12, 2023 18:00

MajorLift requested a review from cryptodev-2s October 12, 2023 18:58

MajorLift force-pushed the 230927-integrate-eth-json-rpc-provider branch from 69a276d to e42c619 Compare October 12, 2023 21:59

mcmire previously requested changes Oct 12, 2023

View reviewed changes

MajorLift commented Oct 12, 2023

View reviewed changes

MajorLift requested a review from legobeat October 13, 2023 00:27

MajorLift added 9 commits October 13, 2023 08:50

Move eth-json-rpc-provider into packages/ directory.

a0f04f8

Install eth-json-rpc-provider as a dependency of the monorepo.

58b7e5e

Add tsconfig reference paths for eth-json-rpc-provider in root and …

fe985a2

…downstream packages

Bump eth-json-rpc-provider dep versions in root and downstream pack…

26dcd51

…ages

Apply validator fixes for eth-json-rpc-provider CHANGELOG.md

7befb4e

Apply constraints fixes to eth-json-rpc-provider package.json file

b3338ce

Fix lockfile

f02b824

Export SafeEventEmitterProvider as class instead of type

4c8fea7

Fix import statements for SafeEventEmitterProvider

a32e68a

MajorLift force-pushed the 230927-integrate-eth-json-rpc-provider branch from 0b71faa to a32e68a Compare October 13, 2023 15:51

mcmire approved these changes Oct 13, 2023

View reviewed changes

MajorLift changed the title ~~eth-json-rpc-provider migration - C: Integration into packages/~~ eth-json-rpc-provider migration - Integration into packages/ Oct 13, 2023

cryptodev-2s approved these changes Oct 13, 2023

View reviewed changes

MajorLift merged commit b171486 into main Oct 13, 2023
111 checks passed

MajorLift deleted the 230927-integrate-eth-json-rpc-provider branch October 13, 2023 16:57

This was referenced Oct 13, 2023

Post-migration clean-up for @metamask/eth-json-rpc-provider #1838

Closed

Post-migration clean-up for @metamask/eth-json-rpc-provider #1842

Closed

This was referenced Oct 25, 2023

Moving json-rpc-engine from merged-packages into the packages #1919

Closed

Moving json-rpc-middleware-stream from merged-packages into packages #1920

Closed

MajorLift mentioned this pull request Nov 3, 2023

Expose SafeEventEmitterProvider as a value, not just a type #1974

Closed

mcmire mentioned this pull request Nov 7, 2023

Record CHANGELOG entries from @metamask/json-rpc-{engine,middleware-stream} migrations #2003

Merged

6 tasks

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

`eth-json-rpc-provider` migration - Integration into `packages/` #1738

`eth-json-rpc-provider` migration - Integration into `packages/` #1738

MajorLift commented Sep 27, 2023 •

edited by mcmire

Loading

socket-security bot commented Sep 27, 2023 •

edited

Loading

socket-security bot commented Sep 27, 2023 •

edited

Loading

MajorLift commented Sep 27, 2023

mcmire left a comment

mcmire Oct 12, 2023

MajorLift Oct 12, 2023

mcmire Oct 12, 2023

MajorLift Oct 12, 2023

mcmire Oct 13, 2023

mcmire Oct 12, 2023

MajorLift Oct 12, 2023

mcmire Oct 12, 2023

MajorLift Oct 12, 2023 •

edited

Loading

MajorLift Oct 12, 2023 •

edited

Loading

MajorLift Oct 12, 2023

mcmire Oct 13, 2023

MajorLift commented Oct 13, 2023 •

edited

Loading

mcmire left a comment

cryptodev-2s left a comment

		@@ -1,4 +1,4 @@
		import { SafeEventEmitterProvider } from '@metamask/eth-json-rpc-provider/dist/safe-event-emitter-provider';
		import { SafeEventEmitterProvider } from '@metamask/eth-json-rpc-provider';

eth-json-rpc-provider migration - Integration into packages/ #1738

eth-json-rpc-provider migration - Integration into packages/ #1738

Conversation

MajorLift commented Sep 27, 2023 • edited by mcmire Loading

Explanation

Phase B: Staging from merged-packages/

5. Port tags

Phase C: Integration into packages/

1. The big leap

2. Update downstream repos

3. Linter fixes

4. Resolve downstream errors

5. Finalize merge

Next Steps

Blocked by

References

Changelog

@metamask/eth-json-rpc-provider

@metamask/network-controller

Checklist

socket-security bot commented Sep 27, 2023 • edited Loading

socket-security bot commented Sep 27, 2023 • edited Loading

Next steps

MajorLift commented Sep 27, 2023

mcmire left a comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

MajorLift Oct 12, 2023 • edited Loading

Choose a reason for hiding this comment

MajorLift Oct 12, 2023 • edited Loading

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

MajorLift commented Oct 13, 2023 • edited Loading

mcmire left a comment

Choose a reason for hiding this comment

cryptodev-2s left a comment

Choose a reason for hiding this comment

`eth-json-rpc-provider` migration - Integration into `packages/` #1738

`eth-json-rpc-provider` migration - Integration into `packages/` #1738

MajorLift commented Sep 27, 2023 •

edited by mcmire

Loading

Phase B: Staging from `merged-packages/`

Phase C: Integration into `packages/`

`@metamask/eth-json-rpc-provider`

`@metamask/network-controller`

socket-security bot commented Sep 27, 2023 •

edited

Loading

socket-security bot commented Sep 27, 2023 •

edited

Loading

MajorLift Oct 12, 2023 •

edited

Loading

MajorLift Oct 12, 2023 •

edited

Loading

MajorLift commented Oct 13, 2023 •

edited

Loading