Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Change upgrade logic for worker nodes #1772

Closed
mrajashree opened this issue Nov 13, 2019 · 1 comment
Closed

Change upgrade logic for worker nodes #1772

mrajashree opened this issue Nov 13, 2019 · 1 comment
Assignees

Comments

@mrajashree
Copy link
Contributor

mrajashree commented Nov 13, 2019

Worker nodes should be upgraded in user configurable batches. Users can specify maximum number of nodes that can be unavailable during the upgrade. The upgrade will follow a sliding window approach based on maxUnavailable value users specify.
The nodes will be cordoned before upgrade. Users can also optionally drain the node.
Nodes will be uncordoned after upgrade.
Users can specify maxUnavailable and drain input through cluster.yml

upgrade_strategy:
  max_unavailable: 1
  drain: true
  node_drain_input:
    ignore_daemonsets: true
    force:
    delete_local_data:
    grace_period:
    timeout:
@soumyalj
Copy link

soumyalj commented Feb 20, 2020

Tested with rke cli - v1.1.0-rc5
Some of the tests executed :

  1. Verified that worker nodes are upgraded in batches . maxUnavailable can be configured which defines the number of nodes that get upgraded at a time.
  2. Verified worker node upgrade by specifying the drain=true
 upgrade_strategy:
  max_unavailable: 3
  drain: true
  node_drain_input:
    force: False
    delete_local_data: false
    grace_period: 30
    timeout:120
  1. Update cluster and specify drain=true option and change kubelet flag or change any parameter other than K8s version.
  2. Drain failure during cluster update Drain failure during cluster update
  3. Add a worker node in rke file and then upgrade specifying maxUnavailable field
  4. Remove a worker node in rke file and then upgrade specifying maxUnavailable field

Worker node upgrade succeeds in all the cases as expected. The nodes have updated images for worker node components after upgrade (when there is a change in image)

@zube zube bot removed the [zube]: Done label Oct 13, 2020
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

4 participants