deployment: Changes for keystone #14837

archseer · 2024-10-18T04:59:00Z

No description provided.

github-actions · 2024-10-21T09:13:20Z

AER Report: CI Core ran successfully ✅

aer_workflow , commit

AER Report: Operator UI CI ran successfully ✅

aer_workflow , commit

archseer · 2024-10-22T08:01:33Z

integration-tests/deployment/keystone/deploy_test.go

+	require.NoError(t, err)
+	// sepoliaArbitrumChainSel, err := chainsel.SelectorFromChainId(sepoliaArbitrumChainId)
+	// require.NoError(t, err)
+	// aptosChainSel := uint64(999) // TODO:


This is currently disabled because non-EVMs aren't listed in chain-selectors yet

archseer · 2024-10-22T08:02:04Z

integration-tests/deployment/memory/job_client.go

+		switch family {
+		case chainsel.FamilyEVM:
+			ocrtype = chaintype.EVM
+		case chainsel.FamilySolana:
+			ocrtype = chaintype.Solana
+		case chainsel.FamilyStarknet:
+			ocrtype = chaintype.StarkNet
+		case chainsel.FamilyCosmos:
+			ocrtype = chaintype.Cosmos
+		case chainsel.FamilyAptos:
+			ocrtype = chaintype.Aptos
+		default:
+			panic(fmt.Sprintf("Unsupported chain family %v", family))
+		}


there's a bunch of these type conversions but I'm not sure where to put the helper functions

krehermann · 2024-10-23T12:48:52Z

integration-tests/web/sdk/internal/genqlient.graphql

@@ -45,6 +45,20 @@ query FetchAccounts {
    }
 }

+query FetchKeys {


why is this in this change? all of the data should be coming from the JD right?

this is used by node-operations create-ccip-chain (which can be used for keystone too) when it's generating ChainConfig for the nodes (which is then exposed by JD) -- take a look at CreateCCIPOCRSupportedChains

krehermann · 2024-10-23T13:03:57Z

integration-tests/deployment/keystone/types.go

@@ -164,41 +163,40 @@ func makeNodeKeysSlice(nodes []*ocr2Node) []NodeKeys {
 // in is in a convenient form to handle the CLO representation of the nop data
 type DonCapabilities struct {
 	Name         string
-	Nops         []*models.NodeOperator               // each nop is a node operator and may have multiple nodes
+	Nodes        []string


this needs to be p2pkey. it may be more awkward wrt to the JD api, but it makes the definition independent of jd instance b/c it doesn't leak jd-instance-specific state

are we able to query by p2pkey on the ListNodes API?

i don't know, we could request it a feature.

at worst we list all the nodes in jd and filter them locally until they give us a better api

integration-tests/deployment/clo/offchain_client_impl.go

cl-sonarqube-production · 2024-10-31T09:39:31Z

Quality Gate passed

Issues
13 New issues
2 Fixed issues
0 Accepted issues

Measures
0 Security Hotspots
0.0% Coverage on New Code
0.2% Duplication on New Code

See analysis details on SonarQube

github-actions · 2024-11-05T00:56:40Z

Flaky Test Detector for `deployment/go.mod` project has failed ❌

Ran new or updated tests between develop and 0d9c915 (aptos-deployments).

View Flaky Detector Details | Compare Changes

Failed Tests

Ran 7 tests in total for all affected test packages. Below are the tests identified as flaky, with a pass ratio lower than the 100% threshold:

TestPackage                                                TestName       PassRatio  RunCount   Skipped
---------                                                  ---------      ---------  ---------  ---------
github.com/smartcontractkit/chainlink/deployment/keystone  TestDeployCLO  0%         1          false

github-actions · 2024-11-05T07:08:31Z

Flaky Test Detector for `deployment/go.mod` project has failed ❌

Ran new or updated tests between develop and a2d4640 (aptos-deployments).

View Flaky Detector Details | Compare Changes

Failed Tests

Ran 7 tests in total for all affected test packages. Below are the tests identified as flaky, with a pass ratio lower than the 100% threshold:

TestPackage                                                TestName       PassRatio  RunCount   Skipped
---------                                                  ---------      ---------  ---------  ---------
github.com/smartcontractkit/chainlink/deployment/keystone  TestDeployCLO  0%         1          false

github-actions · 2024-11-05T07:17:04Z

I see you updated files related to core. Please run pnpm changeset in the root directory to add a changeset as well as in the text include at least one of the following tags:

#added For any new functionality added.
#breaking_change For any functionality that requires manual action for the node to boot.
#bugfix For bug fixes.
#changed For any change to the existing functionality.
#db_update For any feature that introduces updates to database schema.
#deprecation_notice For any upcoming deprecation functionality.
#internal For changesets that need to be excluded from the final changelog.
#nops For any feature that is NOP facing and needs to be in the official Release Notes for the release.
#removed For any functionality/config that is removed.
#updated For any functionality that is updated.
#wip For any change that is not ready yet and external communication about it should be held off till it is feature complete.

🎖️ No JIRA issue number found in: PR title, commit message, or branch name. Please include the issue ID in one of these.

github-actions · 2024-11-05T07:22:16Z

Flaky Test Detector for `deployment/go.mod` project has failed ❌

Ran new or updated tests between develop and 8f58772 (aptos-deployments).

View Flaky Detector Details | Compare Changes

Failed Tests

Ran 7 tests in total for all affected test packages. Below are the tests identified as flaky, with a pass ratio lower than the 100% threshold:

TestPackage                                                TestName       PassRatio  RunCount   Skipped
---------                                                  ---------      ---------  ---------  ---------
github.com/smartcontractkit/chainlink/deployment/keystone  TestDeployCLO  0%         1          false

connorwstein · 2024-11-05T16:25:36Z

deployment/environment/memory/environment.go

@@ -29,6 +29,18 @@ type MemoryEnvironmentConfig struct {
 	RegistryConfig deployment.CapabilityRegistryConfig
 }

+// For placeholders like aptos
+func NewMemoryChain(t *testing.T, selector uint64) deployment.Chain {


Note TODO here regarding the thinking for non-evms

chainlink/deployment/environment.go

Line 63 in c6c8fbd

// TODO: Add SolChains, AptosChain etc.

. Fine if we intend to come back to it, but intention is we'd add some aptos go sdk based Chain there (will require bumping geth to 1.14, which I'm working on with @jmank88)

I ended up not even using this in the tests since chain-selectors doesn't support non-EVMs, but most other chains don't have a simulated backend. We'd need to start a local node and test against that. For Keystone tests at the moment we don't actually need a fully configured node, just an instance of the node with the Aptos keys available so we can configure the contracts.

chain-selectors fully was intended to support non-evms, should be fixed soon.

but most other chains don't have a simulated backend. We'd need to start a local node and test against that.

The onchain interface isn't tied to a simulated backend. The intention there is to use the docker based environment for families where that doesn't exist. The AptosOnchainClient or w/e interface would just have an implementation only in the devenv not memory

connorwstein · 2024-11-05T16:28:18Z

deployment/keystone/deploy.go

+	Capabilities []kcr.CapabilitiesRegistryCapability // every capability is hosted on each node
+}
+
+type Node struct {


this looks product agnostic - should be able to combine/merge with

chainlink/deployment/environment.go

Line 217 in c6c8fbd

type Node struct {

?

connorwstein · 2024-11-05T16:29:01Z

deployment/keystone/deploy.go

+	ChainConfigs []*nodev1.ChainConfig
+}
+
+func NodesFromJD(name string, nodeIDs []string, jd deployment.OffchainClient) ([]Node, error) {


ditto here should be a way to either combine with or have another helper alongside

chainlink/deployment/environment.go

Line 247 in c6c8fbd

func NodeInfo(nodeIDs []string, oc NodeChainConfigsLister) (Nodes, error) {

The code in there was very CCIP specific, only supporting EVM which needs a larger cleanup. Keystone also uses peer IDs for node IDs so our lookup differs. I'd prefer to leave a TODO here and fix this in a follow up

I can see it only support EVM at the moment, but what part is CCIP specific?

archseer force-pushed the aptos-deployments branch from 465fd3c to d34c3c4 Compare October 21, 2024 08:55

archseer force-pushed the aptos-deployments branch 5 times, most recently from 9d56a91 to 898fd46 Compare October 22, 2024 07:45

archseer requested a review from krehermann October 22, 2024 08:01

archseer commented Oct 22, 2024

View reviewed changes

archseer force-pushed the aptos-deployments branch from 361ed0f to b60faa0 Compare October 23, 2024 06:44

krehermann reviewed Oct 23, 2024

View reviewed changes

krehermann requested changes Oct 23, 2024

View reviewed changes

integration-tests/deployment/clo/offchain_client_impl.go Outdated Show resolved Hide resolved

archseer force-pushed the aptos-deployments branch 5 times, most recently from f65cf81 to 786390a Compare October 30, 2024 02:33

archseer added 6 commits October 30, 2024 11:51

deployment: Support other chain types in CreateCCIPOCRSupportedChains

58956ba

nix: Upgrade to postgres 15

cbfbdc1

keystone: Migrate from CLO to JD

ac23edc

CLO compat

a54149f

Allow setting labels on nodes

4044d5e

Rename function

0fc498b

archseer force-pushed the aptos-deployments branch from 786390a to 0fc498b Compare October 30, 2024 02:55

krehermann previously approved these changes Oct 30, 2024

View reviewed changes

Tag nodes with p2p_id for easy lookup

87f50ad

archseer dismissed krehermann’s stale review via 87f50ad October 31, 2024 09:20

archseer added 2 commits October 31, 2024 18:53

Lookup nodes according to p2p_id

02f2575

Implement label & id filtering in the memory job client

e99f24d

krehermann marked this pull request as ready for review November 5, 2024 00:37

krehermann requested review from a team as code owners November 5, 2024 00:37

krehermann requested a review from EasterTheBunny November 5, 2024 00:37

Merge branch 'develop' into aptos-deployment

0d9c915

Update the CLO job client as well

a2d4640

go mod tidy

8f58772

archseer requested a review from a team as a code owner November 5, 2024 07:14

Fix DeployCLO

5c981eb

Fix CLO job client test

b029545

archseer requested review from krehermann, bolekk and connorwstein November 5, 2024 09:50

krehermann previously approved these changes Nov 5, 2024

View reviewed changes

archseer enabled auto-merge November 5, 2024 13:59

connorwstein reviewed Nov 5, 2024

View reviewed changes

Merge remote-tracking branch 'origin/develop' into aptos-deployments

bb34269

archseer dismissed krehermann’s stale review via bb34269 November 7, 2024 04:24

archseer added 3 commits November 7, 2024 13:36

add TODOs

dfea6a1

Merge remote-tracking branch 'origin/develop' into aptos-deployments

1c53546

fix up tests again

8e7a624

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

deployment: Changes for keystone #14837

deployment: Changes for keystone #14837

archseer commented Oct 18, 2024

github-actions bot commented Oct 21, 2024 •

edited

Loading

archseer Oct 22, 2024

archseer Oct 22, 2024

krehermann Oct 23, 2024

archseer Oct 23, 2024

krehermann Oct 23, 2024 •

edited

Loading

archseer Oct 23, 2024

krehermann Oct 23, 2024

cl-sonarqube-production bot commented Oct 31, 2024

github-actions bot commented Nov 5, 2024

github-actions bot commented Nov 5, 2024

github-actions bot commented Nov 5, 2024

github-actions bot commented Nov 5, 2024

connorwstein Nov 5, 2024

archseer Nov 6, 2024

connorwstein Nov 6, 2024 •

edited

Loading

connorwstein Nov 5, 2024

connorwstein Nov 5, 2024

archseer Nov 6, 2024

connorwstein Nov 6, 2024

deployment: Changes for keystone #14837

Are you sure you want to change the base?

deployment: Changes for keystone #14837

Conversation

archseer commented Oct 18, 2024

github-actions bot commented Oct 21, 2024 • edited Loading

AER Report: CI Core ran successfully ✅

AER Report: Operator UI CI ran successfully ✅

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

krehermann Oct 23, 2024 • edited Loading

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

cl-sonarqube-production bot commented Oct 31, 2024

Quality Gate passed

github-actions bot commented Nov 5, 2024

Flaky Test Detector for deployment/go.mod project has failed ❌

Failed Tests

github-actions bot commented Nov 5, 2024

Flaky Test Detector for deployment/go.mod project has failed ❌

Failed Tests

github-actions bot commented Nov 5, 2024

github-actions bot commented Nov 5, 2024

Flaky Test Detector for deployment/go.mod project has failed ❌

Failed Tests

Choose a reason for hiding this comment

Choose a reason for hiding this comment

connorwstein Nov 6, 2024 • edited Loading

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

github-actions bot commented Oct 21, 2024 •

edited

Loading

krehermann Oct 23, 2024 •

edited

Loading

Flaky Test Detector for `deployment/go.mod` project has failed ❌

Flaky Test Detector for `deployment/go.mod` project has failed ❌

Flaky Test Detector for `deployment/go.mod` project has failed ❌

connorwstein Nov 6, 2024 •

edited

Loading