Skip to content

Commit

Permalink
Initial move of integration-tests/deployment to /deployment as a sepa…
Browse files Browse the repository at this point in the history
…rate module. (#14893)

This updates the dependencies, and makes the integration-tests depend on both chainlink and the deployments package.

A few things to fix.  The go.mod entry for deployments is off, because there is no version currently checked-in. It's overridden in a replace statement, but I'm not sure if there are any contexts in which that goes wrong. It just seems weird to keep an incorrect version number there.

Also:
  * Rename the node client to nodeclient, to disambiguate.
  * Move the various non-product elements into 'environment'
  • Loading branch information
cgruber authored Oct 29, 2024
1 parent 33dba89 commit e116f74
Show file tree
Hide file tree
Showing 185 changed files with 3,640 additions and 823 deletions.
5 changes: 5 additions & 0 deletions .changeset/hot-mice-yell.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,5 @@
---
"chainlink": patch
---

Relocation of the deployment directory into a direct subdir of the root. #internal
1 change: 1 addition & 0 deletions .dockerignore
Original file line number Diff line number Diff line change
Expand Up @@ -10,6 +10,7 @@ tmp/
contracts/node_modules
examples/

deployment/
integration/
integration-scripts/
integration-tests/
Expand Down
8 changes: 4 additions & 4 deletions .github/CODEOWNERS
Validating CODEOWNERS rules …
Original file line number Diff line number Diff line change
Expand Up @@ -36,7 +36,7 @@
/core/services/ocr2/plugins/ccip @smartcontractkit/ccip

# VRF-related services
/core/services/vrf @smartcontractkit/dev-services @smartcontractkit/core
/core/services/vrf @smartcontractkit/dev-services @smartcontractkit/core
/core/services/blockhashstore @smartcontractkit/dev-services @smartcontractkit/core
/core/services/blockheaderfeeder @smartcontractkit/dev-services @smartcontractkit/core
/core/services/pipeline/task.vrf.go @smartcontractkit/dev-services
Expand Down Expand Up @@ -134,9 +134,9 @@ core/scripts/gateway @smartcontractkit/dev-services

# Deployment tooling
# Initially the common structures owned by CCIP
/integration-tests/deployment @smartcontractkit/ccip @smartcontractkit/keystone @smartcontractkit/core
/integration-tests/deployment/ccip @smartcontractkit/ccip @smartcontractkit/core
/integration-tests/deployment/keystone @smartcontractkit/keystone @smartcontractkit/core
/deployment @smartcontractkit/ccip @smartcontractkit/keystone @smartcontractkit/core
/deployment/ccip @smartcontractkit/ccip @smartcontractkit/core
/deployment/keystone @smartcontractkit/keystone @smartcontractkit/core
# TODO: As more products add their deployment logic here, add the team as an owner

# CI/CD
Expand Down
3 changes: 1 addition & 2 deletions .github/workflows/ci-core.yml
Original file line number Diff line number Diff line change
Expand Up @@ -45,9 +45,8 @@ jobs:
with:
filters: |
changes:
- 'integration-tests/deployment/**'
- 'deployment/**'
- '!integration-tests/**'
- 'integration-tests/deployment/**'
- name: Ignore Filter On Workflow Dispatch
if: ${{ github.event_name == 'workflow_dispatch' }}
id: ignore-filter
Expand Down
4 changes: 2 additions & 2 deletions core/scripts/go.mod
Original file line number Diff line number Diff line change
Expand Up @@ -5,7 +5,7 @@ go 1.22.8
// Make sure we're working with the latest chainlink libs
replace github.com/smartcontractkit/chainlink/v2 => ../../

replace github.com/smartcontractkit/chainlink/integration-tests => ../../integration-tests
replace github.com/smartcontractkit/chainlink/deployment => ../../deployment

require (
github.com/docker/docker v27.3.1+incompatible
Expand All @@ -25,7 +25,7 @@ require (
github.com/shopspring/decimal v1.4.0
github.com/smartcontractkit/chainlink-automation v0.8.1
github.com/smartcontractkit/chainlink-common v0.3.1-0.20241025132045-cfad02139595
github.com/smartcontractkit/chainlink/integration-tests v0.0.0-00010101000000-000000000000
github.com/smartcontractkit/chainlink/deployment v0.0.0-00010101000000-000000000000
github.com/smartcontractkit/chainlink/v2 v2.0.0-00010101000000-000000000000
github.com/smartcontractkit/libocr v0.0.0-20241007185508-adbe57025f12
github.com/spf13/cobra v1.8.1
Expand Down
4 changes: 2 additions & 2 deletions core/scripts/go.sum
Original file line number Diff line number Diff line change
Expand Up @@ -946,8 +946,8 @@ github.com/onsi/gomega v1.7.1/go.mod h1:XdKZgCCFLUoM/7CFJVPcG8C1xQ1AJ0vpAezJrB7J
github.com/onsi/gomega v1.10.1/go.mod h1:iN09h71vgCQne3DLsj+A5owkum+a2tYe+TOCB1ybHNo=
github.com/onsi/gomega v1.17.0/go.mod h1:HnhC7FXeEQY45zxNK3PPoIUhzk/80Xly9PcubAlGdZY=
github.com/onsi/gomega v1.19.0/go.mod h1:LY+I3pBVzYsTBU1AnDwOSxaYi9WoWiqgwooUqq9yPro=
github.com/onsi/gomega v1.34.2 h1:pNCwDkzrsv7MS9kpaQvVb1aVLahQXyJ/Tv5oAZMI3i8=
github.com/onsi/gomega v1.34.2/go.mod h1:v1xfxRgk0KIsG+QOdm7p8UosrOzPYRo60fd3B/1Dukc=
github.com/onsi/gomega v1.33.1 h1:dsYjIxxSR755MDmKVsaFQTE22ChNBcuuTWgkUDSubOk=
github.com/onsi/gomega v1.33.1/go.mod h1:U4R44UsT+9eLIaYRB2a5qajjtQYn0hauxvRm16AVYg0=
github.com/opencontainers/go-digest v1.0.0 h1:apOUWs51W5PlhuyGyz9FCeeBIOUDA/6nW8Oi/yOhh5U=
github.com/opencontainers/go-digest v1.0.0/go.mod h1:0JzlMkj0TRzQZfJkVvzbP0HBR3IKzErnv2BNG4W4MAM=
github.com/opencontainers/image-spec v1.1.0 h1:8SG7/vwALn54lVB/0yZ/MMwhFrPYtpEHQb2IpWsCzug=
Expand Down
2 changes: 1 addition & 1 deletion core/scripts/keystone/src/01_deploy_contracts_cmd.go
Original file line number Diff line number Diff line change
Expand Up @@ -12,7 +12,7 @@ import (
"github.com/ethereum/go-ethereum/common"

helpers "github.com/smartcontractkit/chainlink/core/scripts/common"
ksdeploy "github.com/smartcontractkit/chainlink/integration-tests/deployment/keystone"
ksdeploy "github.com/smartcontractkit/chainlink/deployment/keystone"
"github.com/smartcontractkit/chainlink/v2/core/gethwrappers/keystone/generated/forwarder"
"github.com/smartcontractkit/chainlink/v2/core/gethwrappers/keystone/generated/ocr3_capability"
)
Expand Down
2 changes: 1 addition & 1 deletion core/scripts/keystone/src/88_gen_jobspecs.go
Original file line number Diff line number Diff line change
Expand Up @@ -7,7 +7,7 @@ import (
"strings"

helpers "github.com/smartcontractkit/chainlink/core/scripts/common"
ksdeploy "github.com/smartcontractkit/chainlink/integration-tests/deployment/keystone"
ksdeploy "github.com/smartcontractkit/chainlink/deployment/keystone"
)

type spec []string
Expand Down
2 changes: 1 addition & 1 deletion core/scripts/keystone/src/88_gen_ocr3_config.go
Original file line number Diff line number Diff line change
Expand Up @@ -2,7 +2,7 @@ package src

import (
helpers "github.com/smartcontractkit/chainlink/core/scripts/common"
ksdeploy "github.com/smartcontractkit/chainlink/integration-tests/deployment/keystone"
ksdeploy "github.com/smartcontractkit/chainlink/deployment/keystone"
)

func mustReadConfig(fileName string) (output ksdeploy.TopLevelConfigSource) {
Expand Down
2 changes: 1 addition & 1 deletion core/scripts/keystone/src/99_fetch_keys.go
Original file line number Diff line number Diff line change
Expand Up @@ -13,7 +13,7 @@ import (
"github.com/urfave/cli"

helpers "github.com/smartcontractkit/chainlink/core/scripts/common"
ksdeploy "github.com/smartcontractkit/chainlink/integration-tests/deployment/keystone"
ksdeploy "github.com/smartcontractkit/chainlink/deployment/keystone"
ubig "github.com/smartcontractkit/chainlink/v2/core/chains/evm/utils/big"
"github.com/smartcontractkit/chainlink/v2/core/cmd"
"github.com/smartcontractkit/chainlink/v2/core/web/presenters"
Expand Down
81 changes: 81 additions & 0 deletions deployment/.golangci.yml
Original file line number Diff line number Diff line change
@@ -0,0 +1,81 @@
run:
timeout: 15m
linters:
enable:
- exhaustive
- exportloopref
- revive
- goimports
- gosec
- misspell
- rowserrcheck
- errorlint
linters-settings:
exhaustive:
default-signifies-exhaustive: true
goimports:
local-prefixes: github.com/smartcontractkit/chainlink
golint:
min-confidence: 0.999
gosec:
excludes:
- G101
govet:
enable:
- shadow
revive:
confidence: 0.8
rules:
- name: blank-imports
- name: context-as-argument
- name: context-keys-type
- name: dot-imports
- name: error-return
- name: error-strings
- name: error-naming
- name: if-return
- name: increment-decrement
# - name: var-naming // doesn't work with some generated names
- name: var-declaration
- name: package-comments
- name: range
- name: receiver-naming
- name: time-naming
- name: unexported-return
- name: indent-error-flow
- name: errorf
- name: empty-block
- name: superfluous-else
#- name: unused-parameter
- name: unreachable-code
- name: redefines-builtin-id
- name: waitgroup-by-value
- name: unconditional-recursion
- name: struct-tag
- name: string-format
- name: string-of-int
- name: range-val-address
- name: range-val-in-closure
- name: modifies-value-receiver
- name: modifies-parameter
- name: identical-branches
- name: get-return
# - name: flag-parameter // probably one we should work on doing better at in the future
# - name: early-return // probably one we should work on doing better at in the future
- name: defer
- name: constant-logical-expr
- name: confusing-naming
- name: confusing-results
- name: bool-literal-in-expr
- name: atomic
issues:
exclude-rules:
- path: memory/(.+)\.go
linters:
- revive
- text: "^G404: Use of weak random number generator"
linters:
- gosec
- linters:
- govet
text: "declaration of \"err\" shadows"
61 changes: 29 additions & 32 deletions integration-tests/deployment/README.md → deployment/README.md
Original file line number Diff line number Diff line change
@@ -1,13 +1,11 @@
### Overview
The deployment package in the integration-tests Go module serves
as a product agnostic set of environment abstractions used
to deploy and configure products including both on/offchain
dependencies. The environment abstractions allow for
complex and critical deployment/configuration logic to be tested
against ephemeral environments and then exposed for use in persistent
environments like testnet/mainnet.
## Overview
The deployment Go module serves as a product agnostic set of environment
abstractions used to deploy and configure products including both on/offchain
dependencies. The environment abstractions allow for complex and critical
deployment/configuration logic to be tested against ephemeral environments
and then exposed for use in persistent environments like testnet/mainnet.

### Directory structure
## Directory structure

/deployment
- package name `deployment`
Expand All @@ -19,11 +17,6 @@ environments like testnet/mainnet.
- In-memory environment for fast integration testing
- EVM only

/deployment/devenv
- package name `devenv`
- Docker environment for higher fidelity testing
- Support non-EVMs (yet to be implemented)

/deployment/ccip
- package name `ccipdeployment`
- Files and tests per product deployment/configuration workflows
Expand All @@ -38,24 +31,28 @@ environments like testnet/mainnet.
- They should be go functions where the first argument is an
environment and the second argument is a config struct which can be unique to the
changeset. The return value should be a `deployment.ChangesetOutput` and an error.
```Go
do_something.go
func DoSomethingChangeSet(env deployment.Environment, c ccipdeployment.Config) (deployment.ChangesetOutput, error)
{
// Deploy contracts, generate MCMS proposals, generate
// job specs according to contracts etc.
return deployment.ChangesetOutput{}, nil
}
do_something_test.go
func TestDoSomething(t *testing.T)
{
// Set up memory env
// DoSomethingChangeSet function
// Take the artifacts from ChangeSet output
// Apply them to the memory env
// Send traffic, run assertions etc.
}
```

- `do_something.go`
```Go
func DoSomethingChangeSet(env deployment.Environment, c ccipdeployment.Config) (deployment.ChangesetOutput, error)
{
// Deploy contracts, generate MCMS proposals, generate
// job specs according to contracts etc.
return deployment.ChangesetOutput{}, nil
}
```

- `do_something_test.go`
```Go
func TestDoSomething(t *testing.T)
{
// Set up memory env
// DoSomethingChangeSet function
// Take the artifacts from ChangeSet output
// Apply them to the memory env
// Send traffic, run assertions etc.
}
```
- Changesets are exposed and applied via a different repo.

/deployment/llo
Expand Down
File renamed without changes.
File renamed without changes.
Original file line number Diff line number Diff line change
Expand Up @@ -6,7 +6,7 @@ import (

"github.com/ethereum/go-ethereum/common"

"github.com/smartcontractkit/chainlink/integration-tests/deployment"
"github.com/smartcontractkit/chainlink/deployment"
"github.com/smartcontractkit/chainlink/v2/core/gethwrappers/ccip/generated/offramp"

"github.com/smartcontractkit/chainlink/v2/core/gethwrappers/ccip/generated/fee_quoter"
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -9,7 +9,7 @@ import (

"github.com/smartcontractkit/chainlink-testing-framework/lib/utils/testcontext"

"github.com/smartcontractkit/chainlink/integration-tests/deployment"
"github.com/smartcontractkit/chainlink/deployment"
"github.com/smartcontractkit/chainlink/v2/core/gethwrappers/ccip/generated/offramp"
"github.com/smartcontractkit/chainlink/v2/core/logger"
)
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -7,8 +7,8 @@ import (
"github.com/smartcontractkit/ccip-owner-contracts/pkg/proposal/mcms"
"github.com/smartcontractkit/ccip-owner-contracts/pkg/proposal/timelock"

"github.com/smartcontractkit/chainlink/integration-tests/deployment"
ccdeploy "github.com/smartcontractkit/chainlink/integration-tests/deployment/ccip"
"github.com/smartcontractkit/chainlink/deployment"
ccdeploy "github.com/smartcontractkit/chainlink/deployment/ccip"
cctypes "github.com/smartcontractkit/chainlink/v2/core/capabilities/ccip/types"
"github.com/smartcontractkit/chainlink/v2/core/gethwrappers/ccip/generated/ccip_home"
"github.com/smartcontractkit/chainlink/v2/core/gethwrappers/keystone/generated/capabilities_registry"
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -13,13 +13,13 @@ import (
cciptypes "github.com/smartcontractkit/chainlink-ccip/pkg/types/ccipocr3"
"github.com/smartcontractkit/chainlink-ccip/pluginconfig"

"github.com/smartcontractkit/chainlink/integration-tests/deployment"
"github.com/smartcontractkit/chainlink/deployment"

"github.com/stretchr/testify/require"

jobv1 "github.com/smartcontractkit/chainlink-protos/job-distributor/v1/job"

ccdeploy "github.com/smartcontractkit/chainlink/integration-tests/deployment/ccip"
ccdeploy "github.com/smartcontractkit/chainlink/deployment/ccip"

"github.com/smartcontractkit/chainlink/v2/core/logger"
)
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -4,14 +4,14 @@ import (
"fmt"
"math/big"

ccipdeployment "github.com/smartcontractkit/chainlink/integration-tests/deployment/ccip"
ccipdeployment "github.com/smartcontractkit/chainlink/deployment/ccip"

"github.com/smartcontractkit/chainlink/v2/core/capabilities/ccip/types"

"github.com/smartcontractkit/ccip-owner-contracts/pkg/proposal/mcms"
"github.com/smartcontractkit/ccip-owner-contracts/pkg/proposal/timelock"

"github.com/smartcontractkit/chainlink/integration-tests/deployment"
"github.com/smartcontractkit/chainlink/deployment"
"github.com/smartcontractkit/chainlink/v2/core/gethwrappers/ccip/generated/fee_quoter"
"github.com/smartcontractkit/chainlink/v2/core/gethwrappers/ccip/generated/onramp"
)
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -4,7 +4,7 @@ import (
"testing"
"time"

ccipdeployment "github.com/smartcontractkit/chainlink/integration-tests/deployment/ccip"
ccipdeployment "github.com/smartcontractkit/chainlink/deployment/ccip"

"github.com/smartcontractkit/chainlink/v2/core/capabilities/ccip/types"

Expand All @@ -19,7 +19,7 @@ import (

"github.com/smartcontractkit/chainlink-testing-framework/lib/utils/testcontext"

"github.com/smartcontractkit/chainlink/integration-tests/deployment"
"github.com/smartcontractkit/chainlink/deployment"

"github.com/smartcontractkit/chainlink/v2/core/gethwrappers/ccip/generated/offramp"
"github.com/smartcontractkit/chainlink/v2/core/gethwrappers/ccip/generated/router"
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -3,8 +3,8 @@ package changeset
import (
"github.com/smartcontractkit/ccip-owner-contracts/pkg/proposal/timelock"

"github.com/smartcontractkit/chainlink/integration-tests/deployment"
ccipdeployment "github.com/smartcontractkit/chainlink/integration-tests/deployment/ccip"
"github.com/smartcontractkit/chainlink/deployment"
ccipdeployment "github.com/smartcontractkit/chainlink/deployment/ccip"
)

// Separated changset because cap reg is an env var for CL nodes.
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -3,9 +3,9 @@ package changeset
import (
"github.com/smartcontractkit/ccip-owner-contracts/pkg/proposal/timelock"

"github.com/smartcontractkit/chainlink/integration-tests/deployment"
"github.com/smartcontractkit/chainlink/deployment"

ccipdeployment "github.com/smartcontractkit/chainlink/integration-tests/deployment/ccip"
ccipdeployment "github.com/smartcontractkit/chainlink/deployment/ccip"
)

// We expect the change set input to be unique per change set.
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -3,8 +3,10 @@ package changeset
import (
"testing"

"github.com/smartcontractkit/chainlink/integration-tests/deployment"
ccdeploy "github.com/smartcontractkit/chainlink/integration-tests/deployment/ccip"

"github.com/smartcontractkit/chainlink/deployment"

ccdeploy "github.com/smartcontractkit/chainlink/deployment/ccip"

jobv1 "github.com/smartcontractkit/chainlink-protos/job-distributor/v1/job"

Expand Down
File renamed without changes.
Original file line number Diff line number Diff line change
Expand Up @@ -18,7 +18,7 @@ import (
"github.com/smartcontractkit/chainlink/v2/core/gethwrappers/ccip/generated/rmn_home"
"github.com/smartcontractkit/chainlink/v2/core/gethwrappers/generated/aggregator_v3_interface"

"github.com/smartcontractkit/chainlink/integration-tests/deployment"
"github.com/smartcontractkit/chainlink/deployment"

"github.com/smartcontractkit/chainlink/v2/core/gethwrappers/ccip/generated/ccip_home"
"github.com/smartcontractkit/chainlink/v2/core/gethwrappers/ccip/generated/maybe_revert_message_receiver"
Expand Down
Loading

0 comments on commit e116f74

Please sign in to comment.