Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

fix(kuma-cp): memory store keeps children after owner update #5372

Merged

Conversation

jakubdyszkiewicz
Copy link
Contributor

Memory store was not keeping children after owner update.
Production is not affected, because memory store is not used on prod, only in tests and local development.
This Postgres and K8S is NOT affected.

That was the real cause of flaky update policy on KDS.
Default mesh resources created by tr-test mesh (TrafficRoute suite) were there after the test was done.
Then Global CP was syncing them down to Zone CP and Zone CP was rejecting all TrafficRoute resources because mesh was absent.

2022-11-24T13:30:04.268Z	INFO	kds-zone	creating a new resource from upstream	{"type": "CircuitBreaker", "name": "circuit-breaker-all-tr-test", "mesh": "tr-test"}
2022-11-24T13:30:04.270Z	INFO	kds-zone	error during callback received, sending NACK	{"peer-id": "global", "err": "failed to create k8s resource: admission webhook \"owner-reference.kuma-admission.kuma.io\" denied the request: Mesh.kuma.io \"tr-test\" not found", "errVerbose": "admission webhook \"owner-reference.kuma-admission.kuma.io\" denied the request: Mesh.kuma.io \"tr-test\" not found\nfailed to create k8s resource\ngh.neting.cc/kumahq/kuma/pkg/plugins/resources/k8s.(*KubernetesStore).Create\n\t/home/circleci/project/pkg/plugins/resources/k8s/store.go:75\ngh.neting.cc/kumahq/kuma/pkg/core/resources/store.(*paginationStore).Create\n\t/home/circleci/project/pkg/core/resources/store/pagination_store.go:30\ngh.neting.cc/kumahq/kuma/pkg/metrics/store.(*MeteredStore).Create\n\t/home/circleci/project/pkg/metrics/store/store.go:38\ngh.neting.cc/kumahq/kuma/pkg/core/resources/store.(*customizableResourceStore).Create\n\t/home/circleci/project/pkg/core/resources/store/customizable_store.go:30\ngh.neting.cc/kumahq/kuma/pkg/kds/store.(*syncResourceStore).Sync\n\t/home/circleci/project/pkg/kds/store/sync.go:152\ngh.neting.cc/kumahq/kuma/pkg/kds/zone.Callbacks.func1\n\t/home/circleci/project/pkg/kds/zone/components.go:145\ngh.neting.cc/kumahq/kuma/pkg/kds/client.(*kdsSink).Receive\n\t/home/circleci/project/pkg/kds/client/sink.go:64\ngh.neting.cc/kumahq/kuma/pkg/kds/zone.Setup.func1.2\n\t/home/circleci/project/pkg/kds/zone/components.go:76\nruntime.goexit\n\t/home/circleci/go/src/runtime/asm_amd64.s:1571"}

Checklist prior to review

  • Link to docs PR or issue --
  • Link to UI issue or PR --
  • Is the issue worked on linked? --
  • The PR does not hardcode values that might break projects that depend on kuma (e.g. "kumahq" as a image registry) --
  • The PR will work for both Linux and Windows, system specific functions like syscall.Mkfifo have equivalent implementation on the other OS --
  • Unit Tests --
  • E2E Tests --
  • Manual Universal Tests --
  • Manual Kubernetes Tests --
  • Do you need to update UPGRADE.md? --
  • Does it need to be backported according to the backporting policy? --
  • Do you need to explicitly set a > Changelog: entry here or add a ci/ label to run fewer/more tests?

Signed-off-by: Jakub Dyszkiewicz <jakub.dyszkiewicz@gmail.com>
@jakubdyszkiewicz jakubdyszkiewicz added the ci/run-full-matrix PR: Runs all possible e2e test combination (expensive use carefully) label Nov 24, 2022
@jakubdyszkiewicz jakubdyszkiewicz requested a review from a team as a code owner November 24, 2022 15:21
@jakubdyszkiewicz jakubdyszkiewicz enabled auto-merge (squash) November 24, 2022 16:11
@jakubdyszkiewicz jakubdyszkiewicz merged commit 676a62b into kumahq:master Nov 24, 2022
@jakubdyszkiewicz jakubdyszkiewicz deleted the fix/memory-store-children branch November 24, 2022 16:35
@lahabana lahabana mentioned this pull request Dec 13, 2022
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
ci/run-full-matrix PR: Runs all possible e2e test combination (expensive use carefully)
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants