Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add reconciler sharding capability based on label selector #365

Merged
merged 3 commits into from
Mar 30, 2023
Merged

Conversation

makkes
Copy link
Member

@makkes makkes commented Mar 29, 2023

The new --watch-label-selector lets operators provide a label to the controller manager which in turn uses it to reconcile only those resources (ImageRepositories and ImagePolicies) that match the given label.

This way operators can deploy multiple instances of IRC, each reconciling a distinct set of resources based on their labels and effectively scale IRC horizontally.

⚠️ WARNING

If sharding is enabled, all ImagePolicy resources can only refer to those ImageRepository resources that are captured by the exact same label selector as the ImagePolicies, otherwise the ImagePolicies will not become ready.

Example

Given two Deployments with different shard keys:

  1. Deployment image-reflector-controller-1 with --watch-label-selector=sharding.fluxcd.io/shard=shard1
  2. Deployment image-reflector-controller-2 with --watch-label-selector=sharding.fluxcd.io/shard=shard2

An ImageRepository like this:

apiVersion: image.toolkit.fluxcd.io/v1beta2
kind: ImageRepository
metadata:
  labels:
    sharding.fluxcd.io/shard: shard1
[...]

An ImagePolicy like this:

apiVersion: image.toolkit.fluxcd.io/v1beta2
kind: ImagePolicy
metadata:
  labels:
    sharding.fluxcd.io/shard: shard1
[...]
spec:
  imageRepositoryRef:
    name: podinfo
[...]

The ImagePolicy will successfully be reconciled.

If the ImagePolicy would have the label sharding.fluxcd.io/shard: shard2 reconciliation would fail because it would be referring to an ImageRepository from a different shard.

closes #364

The new `--watch-label-selector` lets operators provide a label to the
controller manager which in turn uses it to reconcile only those
resources (ImageRepositories and ImagePolicies) that match the given
label.

This way operators can deploy multiple instances of IRC, each
reconciling a distinct set of resources based on their labels and
effectively scale IRC horizontally.

closes #364

Signed-off-by: Max Jonas Werner <mail@makk.es>
@stefanprodan stefanprodan changed the title add label sharding capability to controller Add reconciler sharding capability based on label selector Mar 29, 2023
@stefanprodan
Copy link
Member

stefanprodan commented Mar 29, 2023

For IRC things are a more complicated, can you please explain in the PR description that the same shard hey must be used for all policies referring to a repo including that repo. Also writing down an example like I did in fluxcd/pkg#525 would be great.

@makkes makkes marked this pull request as ready for review March 29, 2023 15:30
go.mod Show resolved Hide resolved
@stefanprodan stefanprodan added the enhancement New feature or request label Mar 29, 2023
Signed-off-by: Max Jonas Werner <mail@makk.es>
@stefanprodan
Copy link
Member

The github.com/fluxcd/pkg/apis/meta v0.19.1 should be v1.0.0

Signed-off-by: Max Jonas Werner <mail@makk.es>
Copy link
Member

@stefanprodan stefanprodan left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

Thanks @makkes

@makkes makkes merged commit be240df into main Mar 30, 2023
@makkes makkes deleted the sharding branch March 30, 2023 09:05
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
enhancement New feature or request
Projects
Status: Done
Development

Successfully merging this pull request may close these issues.

Add reconciler sharding capability based on label selector
2 participants