-
Notifications
You must be signed in to change notification settings - Fork 75
Node CPU Frequency Job Details
Turbonomic represents CPUs in terms of their actual CPU frequencies to keep its calculations uniform across various platforms. To do this it needs the cpu frequency of each node from the kubernetes cluster. Historically this has been available from the machineInfo
API at the /spec
API endpoint from kubelet. This method however is not available from kubernetes 1.18 version onwards.
As an alternative, we run a very lightweight CPU frequency getter job on each node each time the kubeturbo pod is start/restarted and when a new node is started/detected, which simply queries the cpu frequency and exits. In most environments this job only takes about 30 seconds or less to run on each node. The job has very minimally defined CPU and Memory limit and request set by default:
resources:
limits:
cpu: 10m
memory: 50Mi
requests:
cpu: 10m
memory: 50Mi
As a user you have options on controlling aspects of how this job runs.
- Image:To keep it super lightweight, as of 8.7.6+ we use the cpufreqgetter image (icr.io/cpopen/turbonomic/cpufreqgetter:latest) based on alpine. For 8.7.5 and older, we use a busybox image (index.docker.io/turbonomic/busybox:latest). For those wishing to use images from Red Hat, you can use the ubi-minimal image (registry.access.redhat.com/ubi8/ubi-minimal).
To configure the image used modify the following arguments: Yaml method add to the kubeturbo deployment under args:
--busybox-image=yourRepo
or for the kubeturbo-release Custom Resource in OpenShift use--busybox-image=registry.access.redhat.com/ubi8/ubi-minimal
. Note any image which can provide a bash shell is sufficient. See examples below.
-
Node Selection: You can chose to exclude certain nodes from running this job. Note that Turbo will then use a default CPU speed value of 2800 MHz per core. If you are running a heterogenous CPU speed environment, it is recommended that you do not disable this job from getting an accurate value.
-
Yaml method add to the kubeturbo deployment under args:
--cpufreq-job-exclude-node-labels=kubernetes.io/key=value
. - For the operator method, update the kubeturbo-release Custom Resource in OpenShift to use
busyboxExcludeNodeLabels: "kubernetes.io/key=value"
. - For the Helm method, add
busyboxExcludeNodeLabels: "kubernetes.io/key=value"
to the values.yaml file and upgrade the chart.
-
Yaml method add to the kubeturbo deployment under args:
Examples of modifying the CPU Frequency Jobs below with YAML Method on the kubeturbo DEPLOYMENT resource:
spec:
template:
spec:
containers:
args:
# Uncomment to override default of icr.io/cpopen/turbonomic/cpufreqgetter, and specify your own location
#- --busybox-image=docker.io/busybox
# or uncomment below to pull from RHCC
#- --busybox-image=registry.access.redhat.com/ubi8/ubi-minimal
# Uncomment to specify the secret name which holds the credentials to busybox image
#- --busybox-image-pull-secret=<secret-name>
# Specify nodes to exclude from cpu frequency getter job.
# Note kubernetes.io/os=windows and/or beta.kubernetes.io/os=windows labels will be automatically excluded by default.
# If specified all the labels will be used to select the node ignoring the default.
#- --cpufreq-job-exclude-node-labels=kubernetes.io/key=value
and with Operator Custom Resource Method:
# Uncomment next lines to specify a repository and image tag for kubeturbo
#image:
# repository: registry.connect.redhat.com
# tag: 8.4.0
# Uncomment to use an image from RHCC for cpu-frequency getter job - predefined in OCP Operator Hub version. Must have `image` uncommented
# busyboxRepository: registry.access.redhat.com/ubi8/ubi-minimal
# Note busy box will use the same pull secret as set for the kubeturbo container image
# imagePullSecret: secretName
# Uncomment out `args` and the parameters you want to control the cpu frequency job (string input)
#args:
# busyboxExcludeNodeLabels: 'kubernetes.io/key=value'
Note you control which namespace this job will run in by using the downward api as an environment variable. See the next section for an example
Note if you are working with an older deployment of Kubeturbo (version 8.4.x and older), you may find that the jobs started in the
default
namespace. You can either redeploy Kubeturbo with more current YAMLS, or you can update your deployment with the environment values in the this section.
Kubeturbo sample yamls and other installation methods use the downward api to detect the namespace kubeturbo is running in. The snippet of the section that appears in the kubeturbo deployment spec is listed below:
env:
- name: KUBETURBO_NAMESPACE
valueFrom:
fieldRef:
fieldPath: metadata.namespace
We inject the env variable named
KUBETURBO_NAMESPACE
for kubeturbo to know its own namespace and ensure that any additional resources, for example the CPU freq job that runs for each node is deployed in the same namespace. If this section is omitted, kubeturbo will use namespacedefault
to create and manage these resources. This env variable can obviously be configured to any other namespace as below:
env:
- name: KUBETURBO_NAMESPACE
value: <namespace-name>
Introduction
Kubeturbo Use Cases
Kubeturbo Deployment
Kubeturbo Config Details and Custom Configurations
Actions and how to leverage them
- Overview
-
Resizing or Vertical Scaling of Containerized Workloads
a. DeploymentConfigs with manual triggers in OpenShift Environments - Node Provision and Suspend (Cluster Scaling)
- SLO Horizontal Scaling
- Turbonomic Pod Moves (continuous rescheduling)
-
Pod move action technical details
a. Red Hat Openshift Environments
b. Pods with PVs
IBM Cloud Pak for Data & Kubeturbo:Evaluation Edition
Troubleshooting
- Startup and Connectivity Issues
- KubeTurbo Health Notification
- Logging: kubeturbo log collection and configuration options
- Startup or Validation Issues
- Stitching Issues
- Data Collection Issues
- Collect data for investigating Kubernetes deployment issue
- Changes to Cluster Role Names and Cluster Role Binding Names