Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

misc: snapshotter: Add k3s and rke2 overlays to the snapshotter daemonset #577

Conversation

fidencio
Copy link
Contributor

This is a small series that brings support to using the nydus-snapshotter daemonset with different flavours of kubernetes.

The background of why we're doing this, is mainly because we rely on those in the Kata Containers CI.

Please, take a look at each commit message for more details.

@fidencio fidencio marked this pull request as draft January 31, 2024 15:17
@fidencio fidencio force-pushed the topic/misc-snapshotter-ds-add-overlays-for-other-k8s-flavours branch from e259957 to 94e76e3 Compare January 31, 2024 15:32
@fidencio fidencio marked this pull request as ready for review January 31, 2024 15:32
Let's have it under `base`, as we want to add a few overlays in order to
make sure this will work as expected with kubernetes flavours such as
`k3s`, or `rke2`.

Signed-off-by: Fabiano Fidêncio <fabiano.fidencio@intel.com>
In order to be able to override this accordingly to the kubernetes
flavour being used, we want to make it slightly more flexible on our end
and pass the directory instead of the file itself.

Signed-off-by: Fabiano Fidêncio <fabiano.fidencio@intel.com>
We're doing this in order to, later on this series, use kubectl to
determine which kubernetes flavour we're running on the node.

Signed-off-by: Fabiano Fidêncio <fabiano.fidencio@intel.com>
@fidencio fidencio force-pushed the topic/misc-snapshotter-ds-add-overlays-for-other-k8s-flavours branch from 94e76e3 to 9517008 Compare January 31, 2024 15:48
@fidencio
Copy link
Contributor Author

cc @ChengyuZhu6

Right now this is very basic and simply will let us know whether we're
running on a node that's using containerd or not.

This ended up with the move of parts of the logic on detecting
containerd and containerd configuration files being removed from a few
specific functions and being put directly into the main one.

In the coming patches of this series we'll expand it to also detect
other kubernetes flavours, such as `k3s` or `rke2`.

Signed-off-by: Fabiano Fidêncio <fabiano.fidencio@intel.com>
With these changes we're now able to also run the nydus-snapshotter
daemonset on `k3s` clusters.

Signed-off-by: Fabiano Fidêncio <fabiano.fidencio@intel.com>
With these changes we're now able to also run the nydus-snapshotter
daemonset on `rke2` clusters.

Signed-off-by: Fabiano Fidêncio <fabiano.fidencio@intel.com>
Differently than `k3s` or `rke2`, getting `k0s` to work as expected is
quite tricky, as it relies on a drop-in configuration file, and I'm not
so confident we can easily drop-in a file with all the needed bits.

With this in mind, at least for now, let's just error out in case k0s is
used, as at least this will make sure that the users will at least have
a clue on what's been gone wrong.

Signed-off-by: Fabiano Fidêncio <fabiano.fidencio@intel.com>
@fidencio fidencio force-pushed the topic/misc-snapshotter-ds-add-overlays-for-other-k8s-flavours branch from 9517008 to de19966 Compare February 1, 2024 07:02
@fidencio
Copy link
Contributor Author

fidencio commented Feb 1, 2024

@imeoer, I'm trying to understand whether the E2E failures are related to this PR or not.

@ChengyuZhu6
Copy link
Contributor

@imeoer, I'm trying to understand whether the E2E failures are related to this PR or not.

@fidencio You can display the pod's log for e2e test.

Let's make sure the ClusterRole, ClusterRoleBinding, and the correct
environment variables (such as NODE_NAME) are properly set as part of
the E2E tests.

Signed-off-by: ChengyuZhu6 <chengyu.zhu@intel.com>
Signed-off-by: Fabiano Fidêncio <fabiano.fidencio@intel.com>
@fidencio fidencio force-pushed the topic/misc-snapshotter-ds-add-overlays-for-other-k8s-flavours branch from a54f732 to 3c70256 Compare February 1, 2024 13:49
@ChengyuZhu6
Copy link
Contributor

LGTM @fidencio !

@fidencio
Copy link
Contributor Author

fidencio commented Feb 1, 2024

@imeoer, I was able to get the full Ci green, please, let me know if anything else is needed from my side.

Copy link
Collaborator

@imeoer imeoer left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM! Thanks all!

@imeoer imeoer merged commit 3fcac86 into containerd:main Feb 2, 2024
16 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants