Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

fix: add mock cluster creation to integration test #21

Merged
merged 106 commits into from
Mar 1, 2023
Merged
Show file tree
Hide file tree
Changes from 100 commits
Commits
Show all changes
106 commits
Select commit Hold shift + click to select a range
f1e31a2
fix: add mock cluster creation to integration test
dnitsch Feb 15, 2023
f9e5064
fix: unit tests againgst running cluster
dnitsch Feb 16, 2023
adc5217
fix: add kubebuilder controller
dnitsch Feb 16, 2023
5e33b58
fix: interim before cmd rework on kubebuilder
dnitsch Feb 17, 2023
d5f0f8f
Merge branch 'main' of github.com:dnitsch/strategyrestseeder into fea…
dnitsch Feb 17, 2023
df3757a
fix: more test set up
dnitsch Feb 21, 2023
80ed294
fix: update deps
dnitsch Feb 21, 2023
b132f30
Merge branch 'main' of github.com:dnitsch/strategyrestseeder into fea…
dnitsch Feb 22, 2023
a0be099
Merge branch 'main' of github.com:dnitsch/strategyrestseeder into fea…
dnitsch Feb 22, 2023
e8e91f3
fix: add kind destroy
dnitsch Feb 23, 2023
d4e24ab
fix: correct false positive test
dnitsch Feb 23, 2023
0a55c38
fix: add more tests
dnitsch Feb 23, 2023
1a10296
fix: make test fixed
dnitsch Feb 23, 2023
a0693ef
fix: add kubebuilder to sonar analysis
dnitsch Feb 23, 2023
72e152e
fix: run tests with echo to see if Docker is there
dnitsch Feb 23, 2023
d26549d
fix: run tests on host
dnitsch Feb 23, 2023
db8bab8
fix: run back in container but use unsafe mount :D
dnitsch Feb 23, 2023
090c359
fix: add docker client
dnitsch Feb 23, 2023
cebf3c9
fix: remove global docker hosty
dnitsch Feb 23, 2023
43c0b53
fix: remove sudo
dnitsch Feb 23, 2023
0183131
fix: update repo after adding
dnitsch Feb 23, 2023
938b341
fix: revert to host running
dnitsch Feb 23, 2023
7b2e18e
fix: add go set up
dnitsch Feb 23, 2023
096b1c0
fix: run go test directly
dnitsch Feb 23, 2023
ec950ca
fix: test in docker again
dnitsch Feb 23, 2023
17db402
fix: echo stuff out
dnitsch Feb 23, 2023
f20a7bb
fix: export KUBECONFIG
dnitsch Feb 23, 2023
c2243b5
fix: add kube config path
dnitsch Feb 23, 2023
a916e27
fix: mkdir .kube dir
dnitsch Feb 23, 2023
d9ed296
fix: log out errors
dnitsch Feb 23, 2023
5a2cd5b
fix: try with kind action
dnitsch Feb 23, 2023
b278932
fix: try with priviliged
dnitsch Feb 23, 2023
7995e67
fix: try on host
dnitsch Feb 23, 2023
cc74ada
fix: bump version
dnitsch Feb 23, 2023
a338ae9
fix: try in docker in docker container
dnitsch Feb 27, 2023
465c832
fix: set CGO_ENABLED=0
dnitsch Feb 27, 2023
391d758
fix: remove -race
dnitsch Feb 27, 2023
f9a3d1c
fix: try creating and tearing down kind
dnitsch Feb 27, 2023
5930911
fix: remove install as run on host
dnitsch Feb 27, 2023
df5eb80
fix: use local kind
dnitsch Feb 27, 2023
8420c77
fix: add DCOKER_HOST
dnitsch Feb 27, 2023
70a464a
fix: enable global CGO
dnitsch Feb 27, 2023
70c8882
fix: temp remove version job
dnitsch Feb 27, 2023
8da9058
fix: add gcc since go setup doesnt seem to do it
dnitsch Feb 27, 2023
2543077
fix: try this
dnitsch Feb 27, 2023
a6aa95a
fix: udpate to tests
dnitsch Feb 27, 2023
a317e29
fix: add this
dnitsch Feb 27, 2023
f2d6d38
fix: remove kubectl
dnitsch Feb 27, 2023
88e383b
fix: remove engineerd
dnitsch Feb 27, 2023
401ae89
fix: temporarily disable CI conditional in test env start up
dnitsch Feb 27, 2023
c66968b
fix: add KUBECONFIG back to the job
dnitsch Feb 27, 2023
cc96e5f
fix: _?
dnitsch Feb 27, 2023
f07a181
fix: add more logging
dnitsch Feb 27, 2023
cfb3cd9
fix: __?
dnitsch Feb 27, 2023
84ad282
fix: remove flag
dnitsch Feb 27, 2023
8e9f72a
fix: add timeout to commandline
dnitsch Feb 27, 2023
a89725c
fix: test this
dnitsch Feb 28, 2023
407f5b2
fix: add echo to kind test
dnitsch Feb 28, 2023
01e793b
fix: add kubectl tool
dnitsch Feb 28, 2023
a0593c3
fix: see config
dnitsch Feb 28, 2023
d6b619b
fix: try grabbing the IP inside tghe container network
dnitsch Feb 28, 2023
fdc9301
fix: see if serverUrl override will work
dnitsch Feb 28, 2023
7ffc245
fix: try with port
dnitsch Feb 28, 2023
f9f7fdc
fix(test): change the server to internal IP{
dnitsch Feb 28, 2023
6874392
fix: try this fron GHA
dnitsch Feb 28, 2023
9388089
fix: remove KUBCONFIG export
dnitsch Feb 28, 2023
f1b7a84
fix: try with IP
dnitsch Feb 28, 2023
69a0b62
fix: try with config supplied
dnitsch Feb 28, 2023
1b91c13
cat out
dnitsch Feb 28, 2023
04b4c8d
fix: formatting
dnitsch Feb 28, 2023
f8d17b5
fix: add ootb url
dnitsch Feb 28, 2023
75cb8e2
try this
dnitsch Feb 28, 2023
8069b0c
fix: try setting KIND_EXPERIMENTAL_DOCKER_NETWORK before create cluster
dnitsch Feb 28, 2023
a64d61d
fix: remove kind job
dnitsch Feb 28, 2023
3fef55a
fix: remove env var global
dnitsch Feb 28, 2023
992c99d
fix: install kind
dnitsch Feb 28, 2023
7d42689
fix: create command
dnitsch Feb 28, 2023
5456321
fix: ...
dnitsch Feb 28, 2023
c0c38f7
fix: use full path
dnitsch Feb 28, 2023
5ae06cf
fix: use = for flags
dnitsch Feb 28, 2023
5f613c6
fix: confi
dnitsch Feb 28, 2023
41011e1
fix: remove ports
dnitsch Feb 28, 2023
c2a2894
fix: enable tests on kubebuilder
dnitsch Feb 28, 2023
9230109
fix: add DOCKER_HOST
dnitsch Feb 28, 2023
5415fcd
fix: add eventually from BDD
dnitsch Feb 28, 2023
ae0b3ce
fix: apply in test kind
dnitsch Feb 28, 2023
9e96a65
fix: run with logger in getting pods
dnitsch Feb 28, 2023
8472aa6
fix: try with fmt write to stdout
dnitsch Feb 28, 2023
3edc776
fix: add separate job container
dnitsch Feb 28, 2023
11ba97b
fix: what a hack
dnitsch Feb 28, 2023
7285581
fix: try not setting the masterUrl
dnitsch Feb 28, 2023
cb17b56
fix: try without start cluster
dnitsch Mar 1, 2023
23a36f6
fix: remove skip
dnitsch Mar 1, 2023
45a8b57
fix: remove condition
dnitsch Mar 1, 2023
4c15e24
fix: try this
dnitsch Mar 1, 2023
4109cbb
fix: try this
dnitsch Mar 1, 2023
2104206
fix: terminate quote
dnitsch Mar 1, 2023
fd97f55
fix: correct kube config
dnitsch Mar 1, 2023
b503da1
fix: try on the host directly
dnitsch Mar 1, 2023
71c0e41
fix: add all tests back
dnitsch Mar 1, 2023
a1ed36e
fix: add test helper package
dnitsch Mar 1, 2023
2fdd6e1
fix: remove set version from CI workflow
dnitsch Mar 1, 2023
dff9f33
fix: remove needs
dnitsch Mar 1, 2023
a6a4b1d
fix: ignore scratch pad
dnitsch Mar 1, 2023
45bb3d8
fix: remove unused methods
dnitsch Mar 1, 2023
823e8fa
fix: address comments
dnitsch Mar 1, 2023
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
21 changes: 14 additions & 7 deletions .github/workflows/build.yml
Original file line number Diff line number Diff line change
Expand Up @@ -32,25 +32,32 @@ jobs:
echo "VERSION -> $GITVERSION_SEMVER"
test:
runs-on: ubuntu-latest
container:
image: golang:1.19-bullseye
needs: set-version
# container:
# image: dnitsch/gha-dind:0.0.6
# volumes:
# - /var/run/docker.sock:/var/run/docker.sock
# options: --privileged=true --group-add=docker
# needs: set-version
env:
SEMVER: ${{ needs.set-version.outputs.semVer }}
GIT_TAG: ${{ needs.set-version.outputs.semVer }}
GOVCS: false
# CGO_ENABLED: "0"
steps:
- uses: actions/checkout@v3
with:
fetch-depth: 1
- name: install deps
- name: git prep
run: |
apt update && apt install -y jq git
git config --global --add safe.directory "$GITHUB_WORKSPACE"
git config user.email ${{ github.actor }}-ci@gha.org
git config user.name ${{ github.actor }}
- uses: actions/setup-go@v3
with:
go-version: '1.19.x'
- name: make test
run: |
export DOCKER_HOST=unix:///var/run/docker.sock
make REVISION=$GITHUB_SHA test
- name: Publish Junit style Test Report
uses: mikepenz/action-junit-report@v3
Expand All @@ -66,5 +73,5 @@ jobs:
with:
args:
-Dsonar.projectVersion=${{ needs.set-version.outputs.semVer }}
-Dsonar.go.tests.reportPaths=/github/workspace/controller/.coverage/report-junit.xml,/github/workspace/seeder/.coverage/report-junit.xml
-Dsonar.go.coverage.reportPaths=/github/workspace/controller/.coverage/out,/github/workspace/seeder/.coverage/out
-Dsonar.go.tests.reportPaths=/github/workspace/controller/.coverage/report-junit.xml,/github/workspace/seeder/.coverage/report-junit.xml,/github/workspace/kubebuilder-controller/.coverage/report-junit.xml
-Dsonar.go.coverage.reportPaths=/github/workspace/controller/.coverage/out,/github/workspace/seeder/.coverage/out,/github/workspace/kubebuilder-controller/.coverage/out
6 changes: 2 additions & 4 deletions .github/workflows/release.yml
Original file line number Diff line number Diff line change
Expand Up @@ -64,8 +64,8 @@ jobs:
with:
args:
-Dsonar.projectVersion=${{ needs.set-version.outputs.semVer }}
-Dsonar.go.tests.reportPaths=/github/workspace/controller/.coverage/report-junit.xml,/github/workspace/seeder/.coverage/report-junit.xml
-Dsonar.go.coverage.reportPaths=/github/workspace/controller/.coverage/out,/github/workspace/seeder/.coverage/out
-Dsonar.go.tests.reportPaths=/github/workspace/controller/.coverage/report-junit.xml,/github/workspace/seeder/.coverage/report-junit.xml,/github/workspace/kubebuilder-controller/.coverage/report-junit.xml
-Dsonar.go.coverage.reportPaths=/github/workspace/controller/.coverage/out,/github/workspace/seeder/.coverage/out,/github/workspace/kubebuilder-controller/.coverage/out

publish-binary:
name: tag repo and publish binary
Expand All @@ -89,8 +89,6 @@ jobs:
make GIT_TAG=$SEMVER REVISION=${{ github.sha }} PAT=${{ secrets.GITHUB_TOKEN }} build_ci
- name: release library
run: |
git config user.email ${{ github.actor }}-ci@gha.org
git config user.name ${{ github.actor }}
make GIT_TAG=$SEMVER REVISION=${{ github.sha }} tag
- name: release cli
run: |
Expand Down
53 changes: 53 additions & 0 deletions .github/workflows/test-ignore.yml
Original file line number Diff line number Diff line change
@@ -0,0 +1,53 @@
name: CI-ScratchPad-PR
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Is this still needed or is it just for reference?

Copy link
Owner Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

that's just for reference but can be removed actually...


on:
workflow_run: null
# pull_request:
# branches: [ master, main ]

jobs:
kind-test-scratch-pad:
runs-on: ubuntu-latest
container:
image: dnitsch/gha-dind:0.0.6
volumes:
- /var/run/docker.sock:/var/run/docker.sock
options: --privileged=true --group-add=docker
env:
GOVCS: false
steps:
- uses: actions/checkout@v3
with:
fetch-depth: 1
- name: DL tools
run: |
curl -LO https://dl.k8s.io/release/v1.26.1/bin/linux/amd64/kubectl
install -o root -g root -m 0755 kubectl /usr/local/bin/kubectl
curl -Lo ./kind https://kind.sigs.k8s.io/dl/v0.17.0/kind-linux-amd64
install -o root -g root -m 0755 kind /usr/local/bin/kind
- name: Get KubeConfig
run: |
export KUBECONFIG=$HOME/.kube/config
export KIND_EXPERIMENTAL_DOCKER_NETWORK=${{ job.container.network }}
echo 'kind: Cluster
apiVersion: kind.x-k8s.io/v1alpha4
networking:
apiServerAddress: "127.0.0.1"
apiServerPort: 6443' > $PWD/kind-config.yml

cat $PWD/kind-config.yml

kind create cluster --config=$PWD/kind-config.yml

echo "get internal"
kind get kubeconfig --internal
echo "get normal"
kind get kubeconfig

kubectl config set-cluster kind-kind --server=https://kind-control-plane:6443

kubectl get pods -n kube-system
kubectl cluster-info
kubectl apply -f ./kubebuilder-controller/config/crd/bases/seeder.dnitsch.net_reststrategies.yaml

kubectl get pods -n kube-system
3 changes: 3 additions & 0 deletions .gitignore
Original file line number Diff line number Diff line change
Expand Up @@ -13,6 +13,9 @@

# build dir
dist/
bin
testbin/*
Dockerfile.cross

# Output of the go coverage tool, specifically when used with LiteIDE
*.out
Expand Down
27 changes: 20 additions & 7 deletions Makefile
Original file line number Diff line number Diff line change
Expand Up @@ -33,11 +33,16 @@ tag:
release:
OWNER=$(OWNER) NAME=$(NAME) PAT=$(PAT) VERSION=$(VERSION) . hack/release.sh

docker_release:
docker_build_raw_controller:
cd controller && \
docker build --build-arg REVISION=$(REVISION) --build-arg VERSION=$(VERSION) -t ghcr.io/dnitsch/reststrategy:$(VERSION) . && \
docker push ghcr.io/dnitsch/reststrategy:$(VERSION)
docker build --build-arg REVISION=$(REVISION) --build-arg VERSION=$(VERSION) -t ghcr.io/dnitsch/reststrategy:raw-$(VERSION) .

docker_build_kubebuilder_controller:
cd kubebuilder-controller && \
docker build --build-arg REVISION=$(REVISION) --build-arg VERSION=$(VERSION) -t ghcr.io/dnitsch/reststrategy:kubebuilder-$(VERSION) .

docker_release: docker_build_kubebuilder_controller docker_build_raw_controller
docker push ghcr.io/dnitsch/reststrategy:$(VERSION)

# for local development install all dependencies
# in workspace
Expand All @@ -50,13 +55,18 @@ test_seeder:
gocov convert seeder/.coverage/out | gocov-xml > seeder/.coverage/report-cobertura.xml

test_controller:
go test ./controller/... -v -mod=readonly -race -coverprofile=controller/.coverage/out | go-junit-report > controller/.coverage/report-junit.xml && \
go test ./controller/... -v -mod=readonly -race -timeout 10m0s -coverprofile=controller/.coverage/out | go-junit-report > controller/.coverage/report-junit.xml && \
gocov convert controller/.coverage/out | gocov-xml > controller/.coverage/report-cobertura.xml

test: test_prereq test_seeder test_controller
test_kubebuilder_controller:
go test ./kubebuilder-controller/... -v -mod=readonly -timeout 10m0s -race -coverprofile=kubebuilder-controller/.coverage/out | go-junit-report > kubebuilder-controller/.coverage/report-junit.xml && \
gocov convert kubebuilder-controller/.coverage/out | gocov-xml > kubebuilder-controller/.coverage/report-cobertura.xml

# running in CI on an alpine container without gcc so only running -race on local set up
test: test_prereq test_seeder test_controller test_kubebuilder_controller

test_prereq:
mkdir -p seeder/.coverage controller/.coverage
mkdir -p seeder/.coverage controller/.coverage kubebuilder-controller/.coverage
go install github.com/jstemmer/go-junit-report/v2@latest && \
go install github.com/axw/gocov/gocov@latest && \
go install github.com/AlekSi/gocov-xml@latest
Expand All @@ -69,4 +79,7 @@ coverage_seeder: test_seeder
go tool cover -html=seeder/.coverage/out

coverage_controller: test_controller
go tool cover -html=controller/.coverage/out
go tool cover -html=controller/.coverage/out

coverage_kubebuilder_controller: test_kubebuilder_controller
go tool cover -html=kubebuilder-controller/.coverage/out
10 changes: 10 additions & 0 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -80,3 +80,13 @@ The templating function is provided already with a replaced value: `P4s$w0rd123!
This is a bit of experiment with controller structures and workspaces to see if some re-usable patterns can be gleamed and used in some code generation scaffolding.

At the very least a copy and paste into a new workspace of existing structure and Make tasks -> deleting following `seeder`, `controller/pkg/rstservice`, removing k8sutils should leave you with a fairly re-useable controller pkg which can be exchanged with other top level types coming from custom/new `apis` module.

## Controller generation

Using [kubebuilder](https://book.kubebuilder.io/quick-start.html) can be found in this dir [kubebuilder-controller](./kubebuilder-controller/README.md).

`kubebuilder init --domain dnitsch.net --repo github.com/dnitsch/reststrategy/controller`

`kubebuilder create api --plugins=base.go.kubebuilder.io/v3 --group seeder --version v1alpha1 --kind RestStrategy --controller --namespaced --plural reststrategies --resource --force`

`make manifests`
29 changes: 20 additions & 9 deletions apis/go.mod
Original file line number Diff line number Diff line change
Expand Up @@ -9,10 +9,23 @@ require (
k8s.io/code-generator v0.26.1
)

require (
github.com/fatih/color v1.14.1 // indirect
github.com/go-logr/zerologr v1.2.3 // indirect
github.com/gobuffalo/flect v0.3.0 // indirect
github.com/google/uuid v1.3.0 // indirect
github.com/inconshreveable/mousetrap v1.1.0 // indirect
github.com/kr/pretty v0.3.0 // indirect
github.com/onsi/ginkgo/v2 v2.6.1 // indirect
github.com/rogpeppe/go-internal v1.9.0 // indirect
github.com/spf13/cobra v1.6.1 // indirect
k8s.io/apiextensions-apiserver v0.26.1 // indirect
)

require (
github.com/a8m/envsubst v1.3.1-0.20221106143949-dffd7176fbc4 // indirect
github.com/davecgh/go-spew v1.1.1 // indirect
github.com/dnitsch/simplelog v1.5.1 // indirect
github.com/dnitsch/simplelog v1.7.0 // indirect
github.com/emicklei/go-restful/v3 v3.10.1 // indirect
github.com/evanphx/json-patch v5.6.0+incompatible // indirect
github.com/go-logr/logr v1.2.3 // indirect
Expand All @@ -26,7 +39,6 @@ require (
github.com/google/gofuzz v1.2.0 // indirect
github.com/josharian/intern v1.0.0 // indirect
github.com/json-iterator/go v1.1.12 // indirect
github.com/kr/pretty v0.3.0 // indirect
github.com/mailru/easyjson v0.7.7 // indirect
github.com/mattn/go-colorable v0.1.13 // indirect
github.com/mattn/go-isatty v0.0.17 // indirect
Expand All @@ -38,11 +50,11 @@ require (
github.com/spf13/pflag v1.0.5 // indirect
github.com/spyzhov/ajson v0.7.2 // indirect
golang.org/x/mod v0.7.0 // indirect
golang.org/x/net v0.5.0 // indirect
golang.org/x/oauth2 v0.4.0 // indirect
golang.org/x/sys v0.4.0 // indirect
golang.org/x/term v0.4.0 // indirect
golang.org/x/text v0.6.0 // indirect
golang.org/x/net v0.7.0 // indirect
golang.org/x/oauth2 v0.5.0 // indirect
golang.org/x/sys v0.5.0 // indirect
golang.org/x/term v0.5.0 // indirect
golang.org/x/text v0.7.0 // indirect
golang.org/x/time v0.3.0 // indirect
golang.org/x/tools v0.5.0 // indirect
google.golang.org/appengine v1.6.7 // indirect
Expand All @@ -55,9 +67,8 @@ require (
k8s.io/klog/v2 v2.90.0 // indirect
k8s.io/kube-openapi v0.0.0-20230123231816-1cb3ae25d79a // indirect
k8s.io/utils v0.0.0-20230115233650-391b47cb4029 // indirect
sigs.k8s.io/controller-tools v0.11.3
sigs.k8s.io/json v0.0.0-20221116044647-bc3834ca7abd // indirect
sigs.k8s.io/structured-merge-diff/v4 v4.2.3 // indirect
sigs.k8s.io/yaml v1.3.0 // indirect
)

replace github.com/dnitsch/reststrategy/seeder => ../seeder
Loading