diff --git a/docs/BOOTING.md b/docs/BOOTING.md index 3ae8ab0128..66f4455f0c 100644 --- a/docs/BOOTING.md +++ b/docs/BOOTING.md @@ -99,15 +99,15 @@ in grub.cfg with graphical GRUB menu to get the device to boot again. 1. Cgroup related options. Please follow the [example](README.md#eve-cgroups) to write these values in the correct GRUB variables: 1. `dom0_mem` option of `hv_dom0_mem_settings` variable - memory limit for eve cgroup - (default is `set_global hv_dom0_mem_settings "dom0_mem=800M,max:800M"`) + (default is `set_global hv_dom0_mem_settings "dom0_mem=640M,max:800M"`) 2. `dom0_max_vcpus` option of `hv_dom0_cpu_settings` variable - cpu limit for eve cgroup (default is `set_global hv_dom0_cpu_settings "dom0_max_vcpus=1"`) 3. `eve_mem` - memory limit for cgroups with services of EVE - (default is `set_global hv_eve_mem_settings "eve_mem=650M,max:650M"`) + (default is `set_global hv_eve_mem_settings "eve_mem=520M,max:650M"`) 4. `eve_max_vcpus` option of `hv_eve_mem_settings` variable - cpu limit for cgroups with services of EVE (default is `set_global hv_eve_cpu_settings "eve_max_vcpus=1""`) 5. `ctrd_mem` option of `hv_ctrd_mem_settings` variable - memory limit for cgroups with containerd-shims of EVE - (default is `set_global hv_ctrd_mem_settings "ctrd_mem=400M,max:400M"`) + (default is `set_global hv_ctrd_mem_settings "ctrd_mem=320M,max:400M"`) 6. `ctrd_max_vcpus` option of `hv_ctrd_cpu_settings` variable - cpu limit for cgroups with containerd-shims of EVE (default is `set_global hv_ctrd_cpu_settings "ctrd_max_vcpus=1"`) 2. Installer options. These options are used during [the installation](DEPLOYMENT.md) process and may be set by adding diff --git a/docs/MEMORY-SETTINGS.md b/docs/MEMORY-SETTINGS.md index 0001ef30a0..8afc832800 100644 --- a/docs/MEMORY-SETTINGS.md +++ b/docs/MEMORY-SETTINGS.md @@ -99,6 +99,12 @@ We use explicit memory settings for several of the cgroups. Namely, we set memory limits for the `eve`, `eve/containerd`, `pillar`, and all the sub-cgroups of the `eve-user-apps` cgroup. +By default, we set the soft memory limits to 80% of the hard memory limits. It's +done to give the Kernel target value for the memory usage of the cgroup. +When memory rebalancing is needed, the kernel will try to keep the memory usage +of the cgroup at the soft limit. The rebalancing is triggered in different +situations, for example, when the system is under memory pressure. + For the EVE-related cgroups, we set the memory limits in the kernel command line arguments. For the user applications, we set the memory in runtime using the values of RAM. @@ -142,12 +148,12 @@ If the hard limit is not specified, the soft limit is used as the hard limit. The mapping of the `` to the cgroup is not straightforward and is misleading. The following table shows the mapping: -| Component | cgroup | default value | -|-------------|-------------------|---------------| -| `dom0` | `eve` | 800M | -| `ctrd` | `eve/containerd/` | 400M | -| `eve` | `eve/services/` | 650M | -| `eve` | `eve/services/*` | 650M | +| Component | cgroup | default soft limit | default hard limit | +|-------------|-------------------|--------------------|--------------------| +| `dom0` | `eve` | 640M | 800M | +| `ctrd` | `eve/containerd/` | 320M | 400M | +| `eve` | `eve/services/` | 520M | 650M | +| `eve` | `eve/services/*` | 520M | 650M | That way, the `dom0_mem` setting sets the memory limits for the `eve` cgroup, the `ctrd_mem` setting sets the memory limits for the `eve/containerd` cgroup, @@ -308,14 +314,6 @@ components in EVE OS. For each setting, we provide guidelines on when it makes sense to increase or decrease the values, as well as the potential impact of setting them too high or too low. -### Note on soft and hard limits - -Currently, the soft and hard limits are set to the same value. Theoretically, -the soft limit should be set to a value that is lower than the hard limit, and -it can be used to handle the memory pressure in the system more effectively. -However, in practice, we have not tested this configuration yet and do not -have precise recommendations on how to set the soft limit. - ### dom0_mem The `dom0_mem` setting controls the memory allocated to the EVE OS's system diff --git a/docs/README.md b/docs/README.md index 3c1001506e..05ce2ddd08 100644 --- a/docs/README.md +++ b/docs/README.md @@ -366,10 +366,10 @@ Memory and CPU limits of `eve`, `eve/services` and `eve/containerd` cgroups can Example: ```text -set_global hv_dom0_mem_settings "dom0_mem=800M,max:800M" +set_global hv_dom0_mem_settings "dom0_mem=640M,max:800M" set_global hv_dom0_cpu_settings "dom0_max_vcpus=1 dom0_vcpus_pin" -set_global hv_eve_mem_settings "eve_mem=650M,max:650M" +set_global hv_eve_mem_settings "eve_mem=520M,max:650M" set_global hv_eve_cpu_settings "eve_max_vcpus=1" -set_global hv_ctrd_mem_settings "ctrd_mem=400M,max:400M" +set_global hv_ctrd_mem_settings "ctrd_mem=320M,max:400M" set_global hv_ctrd_cpu_settings "ctrd_max_vcpus=1" ``` diff --git a/pkg/grub/rootfs.cfg b/pkg/grub/rootfs.cfg index b7584dc581..ef274a00bf 100644 --- a/pkg/grub/rootfs.cfg +++ b/pkg/grub/rootfs.cfg @@ -154,13 +154,13 @@ function set_generic { set_global hv_eve_cpu_settings "eve_max_vcpus=1" #temporarily increase memory settings for kubevirt if [ "$eve_flavor" = "kubevirt" ]; then - set_global hv_dom0_mem_settings "dom0_mem=8000M,max:8000M" - set_global hv_eve_mem_settings "eve_mem=6500M,max:6500M" - set_global hv_ctrd_mem_settings "ctrd_mem=4000M,max:4000M" + set_global hv_dom0_mem_settings "dom0_mem=6400M,max:8000M" + set_global hv_eve_mem_settings "eve_mem=5200M,max:6500M" + set_global hv_ctrd_mem_settings "ctrd_mem=3200M,max:4000M" else - set_global hv_dom0_mem_settings "dom0_mem=800M,max:800M" - set_global hv_eve_mem_settings "eve_mem=650M,max:650M" - set_global hv_ctrd_mem_settings "ctrd_mem=400M,max:400M" + set_global hv_dom0_mem_settings "dom0_mem=640M,max:800M" + set_global hv_eve_mem_settings "eve_mem=520M,max:650M" + set_global hv_ctrd_mem_settings "ctrd_mem=320M,max:400M" fi set_global hv_ctrd_cpu_settings "ctrd_max_vcpus=1" set_global hv_platform_tweaks "smt=false"