Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

fix(kuma-cp) stop components on leader election lost #2318

Merged
merged 1 commit into from
Jul 6, 2021

Conversation

jakubdyszkiewicz
Copy link
Contributor

Summary

We were not stopping leader components on the lost leader event.
This only affects Universal deployments, Kubernetes CP is killed when the leader is lost.

In a case when there are 2 instances (CP1, CP2)

  1. CP1 is a leader, CP2 is not
  2. Postgres is down
  3. CP1 was still running leader components although there should be no leader in this case
  4. CP1 has troubles connecting to Postgres but CP2 can connect
  5. CP1 and CP2 is now a leader.

It's not that CP1 leader without Postgres connection can do much, but that should not be the case.

Documentation

  • - no docs

Testing

  • Unit tests - tested with E2E test
  • E2E tests - added
  • Manual testing on Universal
  • Manual testing on Kubernetes - Kubernetes not affected

Signed-off-by: Jakub Dyszkiewicz <jakub.dyszkiewicz@gmail.com>
Copy link
Contributor

@bartsmykla bartsmykla left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

lgtm

@codecov-commenter
Copy link

Codecov Report

Merging #2318 (cf9ea33) into master (d6d37cc) will increase coverage by 0.25%.
The diff coverage is 11.00%.

Impacted file tree graph

@@            Coverage Diff             @@
##           master    #2318      +/-   ##
==========================================
+ Coverage   50.78%   51.03%   +0.25%     
==========================================
  Files         870      875       +5     
  Lines       40541    40664     +123     
==========================================
+ Hits        20587    20754     +167     
+ Misses      17959    17887      -72     
- Partials     1995     2023      +28     
Impacted Files Coverage Δ
pkg/plugins/runtime/universal/outbound/outbound.go 67.85% <ø> (+5.35%) ⬆️
test/e2e/resilience/leader_election_postgres.go 0.00% <0.00%> (ø)
test/framework/interface.go 0.00% <ø> (ø)
test/framework/k8s_cluster.go 0.00% <0.00%> (ø)
test/framework/k8s_clusters.go 0.00% <0.00%> (ø)
test/framework/k8s_controlplane.go 0.00% <0.00%> (ø)
test/framework/universal_cluster.go 0.00% <0.00%> (ø)
test/framework/universal_clusters.go 0.00% <0.00%> (ø)
test/framework/universal_controlplane.go 0.00% <0.00%> (ø)
pkg/core/runtime/component/component.go 76.19% <66.66%> (+4.76%) ⬆️
... and 27 more

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update d6d37cc...cf9ea33. Read the comment docs.

@jakubdyszkiewicz jakubdyszkiewicz merged commit 1c6a7c2 into master Jul 6, 2021
@jakubdyszkiewicz jakubdyszkiewicz deleted the fix/leader-election-lost branch July 6, 2021 14:47
mergify bot pushed a commit that referenced this pull request Jul 6, 2021
Signed-off-by: Jakub Dyszkiewicz <jakub.dyszkiewicz@gmail.com>
(cherry picked from commit 1c6a7c2)
jakubdyszkiewicz pushed a commit that referenced this pull request Jul 9, 2021
Signed-off-by: Jakub Dyszkiewicz <jakub.dyszkiewicz@gmail.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

4 participants