Bsi app 4.4 a20to21 #11997

ermeratos · 2024-05-15T13:51:27Z

Description:

Notes / Rules for BSI APP4.4.A21 added.

Rationale:

As we have multiple customers asking for a BSI profile to be included in the compliance-operator, we are contributing a profile. To provide a better review process, the individual controle are implemented as separate PRs.

openshift-ci · 2024-05-15T13:51:38Z

Hi @ermeratos. Thanks for your PR.

I'm waiting for a ComplianceAsCode member to verify that this patch is reasonable to test. If it is, they should reply with /ok-to-test on its own line. Until that is done, I will not automatically test new commits in this PR, but the usual testing commands by org members will still work. Regular contributors should join the org to skip this step.

Once the patch is verified, the new status will be reflected by the ok-to-test label.

I understand the commands that are listed here.

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes-sigs/prow repository.

github-actions · 2024-05-15T13:52:47Z

Start a new ephemeral environment with changes proposed in this pull request:

ocp4 (from CTF) Environment (using Fedora as testing environment)

Fedora Testing Environment

Oracle Linux 8 Environment

github-actions · 2024-05-15T13:56:35Z

🤖 A k8s content image for this PR is available at:
ghcr.io/complianceascode/k8scontent:11997
This image was built from commit: f7fc5a2

Click here to see how to deploy it

If you alread have Compliance Operator deployed:
utils/build_ds_container.py -i ghcr.io/complianceascode/k8scontent:11997

Otherwise deploy the content and operator together by checking out ComplianceAsCode/compliance-operator and:
CONTENT_IMAGE=ghcr.io/complianceascode/k8scontent:11997 make deploy-local

applications/openshift/general/kube_descheduler_interval.var

applications/openshift/general/kube_descheduler_operator_exists/rule.yml

controls/bsi_app_4_4.yml

applications/openshift/general/kube_descheduler_lifecycle_policy/rule.yml

yuumasato · 2024-05-22T10:09:15Z

/test e2e-aws-ocp4-bsi
/test e2e-aws-ocp4-bsi-node
/test e2e-aws-rhcos4-bsi

yuumasato · 2024-05-22T14:48:57Z

applications/openshift/general/kube_descheduler_operator_exists/rule.yml

Not required right now, but it would be nice to have e2e tests.
Something similar to container_security_operator/tests .

yuumasato · 2024-05-22T14:52:57Z

applications/openshift/general/kube_descheduler_operator_exists/rule.yml

+  name: yamlfile_value
+  vars:
+    ocp_data: "true"
+    filepath: {{{ openshift_filtered_path('/apis/apiextensions.k8s.io/v1/customresourcedefinitions?limit=500', jqfilter) }}}


By the way, would it be more reliable and simpler to check for the operator's subscription instead of looking for its CRDs?
Check what container_security_operator_exists rule does.

content/applications/openshift/risk-assessment/container_security_operator_exists/rule.yml

Line 55 in cd0c045

filepath: '/apis/operators.coreos.com/v1alpha1/namespaces/openshift-operators/subscriptions/container-security-operator'

BhargaviGudi · 2024-07-19T09:42:43Z

/hold for test

BhargaviGudi · 2024-07-19T09:44:05Z

@yuumasato Could you please let us know if the PR still needs modification or is ready to merge? Thanks

BhargaviGudi · 2024-07-19T15:13:37Z

Rule upstream-ocp4-bsi-kube-descheduler-operator-exists failed with below error.
Note: Same was not observed yesterday with the same steps while testing PR # 519

$ oc describe ccr upstream-ocp4-bsi-kube-descheduler-operator-exists 
Name:         upstream-ocp4-bsi-kube-descheduler-operator-exists
Namespace:    openshift-compliance
Labels:       compliance.openshift.io/check-severity=medium
              compliance.openshift.io/check-status=FAIL
              compliance.openshift.io/scan-name=upstream-ocp4-bsi
              compliance.openshift.io/suite=test
Annotations:  compliance.openshift.io/rule: kube-descheduler-operator-exists
API Version:  compliance.openshift.io/v1alpha1
Description:  Ensure that the Kube Descheduler operator is deployed
If there is an increased risk of external influences and a very high need for protection, pods should be stopped and restarted regularly. No pod should run for more than 24 hours. The availability of the applications in the pod should be ensured.
Id:            xccdf_org.ssgproject.content_rule_kube_descheduler_operator_exists
Instructions:  To check if the Kube Descheduler Operator is installed, run the following command:
oc get sub -n descheduler-operator cluster-kube-descheduler-operator -ojsonpath='{.status.installedCSV}'
the output should return the version of the CSV that represents the installed
operator.
Kind:  ComplianceCheckResult
Metadata:
  Creation Timestamp:  2024-07-19T12:30:36Z
  Generation:          1
  Managed Fields:
    API Version:  compliance.openshift.io/v1alpha1
    Fields Type:  FieldsV1
    fieldsV1:
      f:description:
      f:id:
      f:instructions:
      f:metadata:
        f:annotations:
          .:
          f:compliance.openshift.io/rule:
        f:labels:
          .:
          f:compliance.openshift.io/check-severity:
          f:compliance.openshift.io/check-status:
          f:compliance.openshift.io/scan-name:
          f:compliance.openshift.io/suite:
        f:ownerReferences:
          .:
          k:{"uid":"fa564e52-97c6-4eeb-a699-09b8d07a5d59"}:
      f:rationale:
      f:severity:
      f:status:
    Manager:    compliance-operator
    Operation:  Update
    Time:       2024-07-19T12:30:36Z
  Owner References:
    API Version:           compliance.openshift.io/v1alpha1
    Block Owner Deletion:  true
    Controller:            true
    Kind:                  ComplianceScan
    Name:                  upstream-ocp4-bsi
    UID:                   fa564e52-97c6-4eeb-a699-09b8d07a5d59
  Resource Version:        177171
  UID:                     0de7a893-4dc1-48c2-a6f2-1b64bf37ee8c
Rationale:                 If there is an increased risk of external influences and a very high need for protection, pods should be stopped and restarted regularly.
Severity:                  medium
Status:                    FAIL
Events:                    <none>

$ oc get kubedeschedulers.operator.openshift.io cluster -n openshift-kube-descheduler-operator -o=jsonpath='{.spec.deschedulingIntervalSeconds}' 
3600

@ermeratos Could you please help me with the issue? Thanks

BhargaviGudi · 2024-07-26T10:28:37Z

Verification failed with 4.17.0-0.nightly-2024-07-20-191204 + #519 code + #11997 code

Install CO

NAME              CONTENTIMAGE                                 CONTENTFILE         STATUS
ocp4              ghcr.io/complianceascode/k8scontent:latest   ssg-ocp4-ds.xml     VALID
rhcos4            ghcr.io/complianceascode/k8scontent:latest   ssg-rhcos4-ds.xml   VALID
upstream-ocp4     ghcr.io/complianceascode/k8scontent:11997    ssg-ocp4-ds.xml     VALID
upstream-rhcos4   ghcr.io/complianceascode/k8scontent:11997    ssg-rhcos4-ds.xml   VALID

Create kubedescheduler operator from GUI and create instance

$ oc get csv -n openshift-kube-descheduler-operator
NAME                                    DISPLAY                     VERSION   REPLACES                                             PHASE
clusterkubedescheduleroperator.v5.0.1   Kube Descheduler Operator   5.0.1     clusterkubedescheduleroperator.4.14.0-202311021650   Succeeded
$ oc get kubeschedulers.operator.openshift.io -n openshift-kube-descheduler-operator
NAME      AGE
cluster   6h37m

Create ssb

$ oc compliance bind -N test -S default-auto-apply profile/upstream-ocp4-bsi profile/upstream-ocp4-bsi-node profile/upstream-rhcos4-bsi
Creating ScanSettingBinding test
$ oc get scan
NAME                            PHASE   RESULT
upstream-ocp4-bsi               DONE    NON-COMPLIANT
upstream-ocp4-bsi-node-master   DONE    COMPLIANT
upstream-ocp4-bsi-node-worker   DONE    COMPLIANT
upstream-rhcos4-bsi-master      DONE    COMPLIANT
upstream-rhcos4-bsi-worker      DONE    COMPLIANT

Check for rule kube-descheduler-operator-exists and kube-descheduler-lifecycle-policy

$ oc get ccr | grep kube-descheduler-operator-exists
upstream-ocp4-bsi-kube-descheduler-operator-exists                          FAIL     medium
$ oc describe ccr upstream-ocp4-bsi-kube-descheduler-operator-exists | tail
    Controller:            true
    Kind:                  ComplianceScan
    Name:                  upstream-ocp4-bsi
    UID:                   5d451da3-89fd-442b-ad6c-fe48b8bd489d
  Resource Version:        79649
  UID:                     76df2124-072e-43a5-a22f-1f645c425515
Rationale:                 If there is an increased risk of external influences and a very high need for protection, pods should be stopped and restarted regularly.
Severity:                  medium
Status:                    FAIL
Events:                    <none>

$ oc get ccr | grep kube-descheduler-lifecycle-policy
upstream-ocp4-bsi-kube-descheduler-lifecycle-policy                         PASS     medium

Check the value for variable kube-descheduler-interval

$ oc describe variables.compliance.openshift.io upstream-ocp4-kube-descheduler-interval | grep Value
Value:                     86400

Make sure the LifecycleAndUtilization profile is listed under .spec.profiles

$ oc get kubedeschedulers.operator.openshift.io cluster -n openshift-kube-descheduler-operator -o=jsonpath='{.spec.profiles}' 
["LifecycleAndUtilization"]

Check descheduler runs time is set under .spec.deschedulingIntervalSeconds

$ oc get kubedeschedulers.operator.openshift.io cluster -n openshift-kube-descheduler-operator -o=jsonpath='{.spec.deschedulingIntervalSeconds}' 
3600

Instruction of the rule kube-descheduler-operator-exists works as expected

$ oc get rule upstream-ocp4-kube-descheduler-operator-exists -ojsonpath={.instructions}
To check if the Kube Descheduler Operator is installed, run the following command:
oc get sub -n descheduler-operator cluster-kube-descheduler-operator -ojsonpath='{.status.installedCSV}'
the output should return the version of the CSV that represents the installed
operator.
$ oc get sub -n openshift-kube-descheduler-operator cluster-kube-descheduler-operator -ojsonpath='{.status.installedCSV}'
clusterkubedescheduleroperator.v5.0.1

Rule ocp4-bsi-kube-descheduler-operator-exists failed even after multiple rescan

$ oc compliance rerun-now scansettingbinding test
Rerunning scans from 'test': upstream-ocp4-bsi, upstream-ocp4-bsi-node-master, upstream-ocp4-bsi-node-worker, upstream-rhcos4-bsi-master, upstream-rhcos4-bsi-worker
Re-running scan 'openshift-compliance/upstream-ocp4-bsi'
Re-running scan 'openshift-compliance/upstream-ocp4-bsi-node-master'
Re-running scan 'openshift-compliance/upstream-ocp4-bsi-node-worker'
Re-running scan 'openshift-compliance/upstream-rhcos4-bsi-master'
Re-running scan 'openshift-compliance/upstream-rhcos4-bsi-worker'
$ oc get ccr | grep kube-descheduler-operator-exists
upstream-ocp4-bsi-kube-descheduler-operator-exists                          FAIL     medium

sluetze · 2024-07-31T13:25:42Z

@BhargaviGudi can you post the output of
oc get --raw /apis/operators.coreos.com/v1alpha1/namespaces/descheduler-operator/subscriptions/cluster-kube-descheduler-operator ? I cannot recreate the issue on 4.16.4

I suspect something changed in the structure of the subscription resource.

Also: is container-security-operator-exists rule checked and if yes, does it fail? (this rule is not part of the bsi profile iirc, but should be in stig or others).

BhargaviGudi · 2024-08-01T06:34:07Z

Verification failed.
Raised a bug OCPBUGS-37790 to track the issue.

yuumasato · 2024-08-08T17:25:19Z

@sluetze Hi, rule kube-descheduler-operator-exists works for me.
But I didn't get the rule kube-descheduler-lifecycle-policy to pass.

I have the following descheduler:

apiVersion: operator.openshift.io/v1
kind: KubeDescheduler
metadata:
 creationTimestamp: "2024-08-08T17:14:57Z"
 generation: 1
 name: cluster-lc
 namespace: descheduler-operator
 resourceVersion: "97382"
 uid: eb0d6708-6233-4c3c-bc31-743383a338f8
spec:
 deschedulingIntervalSeconds: 3600
 logLevel: Normal
 managementState: Managed
 mode: Predictive
 operatorLogLevel: Normal
 profileCustomizations:
   devLowNodeUtilizationThresholds: Medium
 profiles:
 - LifecycleAndUtilization

yuumasato · 2024-08-08T17:31:42Z

It seems like the path doesn't point to the right place as CO says it doesn't exist:
<object id="oval:ssg-object_kube_descheduler_lifecycle_policy:obj:1" version="1" flag="does not exist">

sluetze · 2024-08-08T17:39:27Z

Do you have the permission patch for the compliance operator active?

with ComplianceAsCode/content#11997 the compliance-operator needs read access to kubedescheduler resources for the validation. We do this on clusterlevel, as the namespaces where the kubedescheduler can be deployed is configurable

applications/openshift/general/kube_descheduler_lifecycle_policy/rule.yml

yuumasato · 2024-08-12T09:57:40Z

I also played a bit with the jqfilters here as well:
https://jqplay.org/s/WJhh2iPfMoI2r8t

Co-authored-by: Watson Yuuma Sato <wsato@redhat.com>

sluetze · 2024-08-20T10:38:23Z

There are at least two more issues.

deschedulingIntervalSeconds doesnt refer to the age of a pod, but it defines the time between runs of the descheduler. Checking this does nothing for the requirement. (see https://docs.redhat.com/en/documentation/openshift_container_platform/4.16/html/nodes/controlling-pod-placement-onto-nodes-scheduling#nodes-descheduler-configuring-interval_nodes-descheduler-configuring )

the descheduler doesn't act if the mode is not set to Automatic (see https://docs.redhat.com/en/documentation/openshift_container_platform/4.16/html/nodes/controlling-pod-placement-onto-nodes-scheduling#nodes-descheduler-installing_nodes-descheduler-configuring)

I ll review the complete rule again to ensure it does what we intent to. I am sorry for this.

codeclimate · 2024-08-20T10:40:10Z

Code Climate has analyzed commit ab8fd59 and detected 0 issues on this pull request.

The test coverage on the diff in this pull request is 100.0% (50% is the threshold).

This pull request will bring the total coverage in the repository to 59.4% (0.0% change).

View more on Code Climate.

yuumasato · 2024-08-20T11:41:56Z

There are at least two more issues.

Nice finds @sluetze

deschedulingIntervalSeconds doesnt refer to the age of a pod, but it defines the time between runs of the descheduler. Checking this does nothing for the requirement. (see https://docs.redhat.com/en/documentation/openshift_container_platform/4.16/html/nodes/controlling-pod-placement-onto-nodes-scheduling#nodes-descheduler-configuring-interval_nodes-descheduler-configuring )

Still, I think the deschedulingIntervalSeconds is important, it should be set to a value of at least 24h.
Otherwise, it could happen that pods are not descheduled in time because the runs are too wide apart from each other, even if podLifetime is set to 24h or less.

yuumasato

Looks great to me.
I just have a small nit over what I think is a typo.

applications/openshift/general/kube_descheduler_lifecycle_policy/rule.yml

Co-authored-by: Watson Yuuma Sato <wsato@redhat.com>

openshift-ci bot added the needs-ok-to-test Used by openshift-ci bot. label May 15, 2024

ermeratos mentioned this pull request May 15, 2024

APP.4.4.A21 sig-bsi-grundschutz/content#47

Closed

yuumasato self-assigned this May 21, 2024

yuumasato requested changes May 21, 2024

View reviewed changes

yuumasato reviewed May 22, 2024

View reviewed changes

BhargaviGudi mentioned this pull request Jul 17, 2024

Update api_resource_collector_cluster_role.yaml ComplianceAsCode/compliance-operator#519

Closed

openshift-ci bot added the do-not-merge/hold Used by openshift-ci-robot bot. label Jul 19, 2024

ermeratos and others added 4 commits July 22, 2024 10:13

bsi_app_4_4_A21

c94a4dc

bsi_app_4_4_a21 update

dd7b22f

bsi_app_4_4_a21 final edit

8f7b314

bsi_app_4_4_a21 update

12f28c1

ermeratos force-pushed the bsi-app-4.4-a20to21 branch from 8b49082 to 12f28c1 Compare July 22, 2024 08:14

bsi_app_4.4 update references and rebase

4f2d09f

marcusburghardt added OpenShift OpenShift product related. BSI PRs or issues for the BSI profile. labels Jul 31, 2024

lichtblaugue and others added 3 commits August 1, 2024 15:17

Change namespace of cluster-kube-descheduler-operator to default value

59556e4

updated rule to select all namespaces and look for at least one

5ee5580

update rule to select all namespaces

af1c4f8

sluetze mentioned this pull request Aug 12, 2024

Add kubedeschedulers to api_resource_collector_clusterrole ComplianceAsCode/compliance-operator#587

Merged

yuumasato requested changes Aug 12, 2024

View reviewed changes

applications/openshift/general/kube_descheduler_lifecycle_policy/rule.yml Show resolved Hide resolved

applications/openshift/general/kube_descheduler_lifecycle_policy/rule.yml Outdated Show resolved Hide resolved

fix filter and entity_check as suggested

ab8fd59

Co-authored-by: Watson Yuuma Sato <wsato@redhat.com>

sluetze added 2 commits August 20, 2024 18:14

add check for the mode

a86411c

add manual policy to check for podLifetime

1310443

yuumasato approved these changes Aug 30, 2024

View reviewed changes

applications/openshift/general/kube_descheduler_lifecycle_policy/rule.yml Outdated Show resolved Hide resolved

fix typo

f7fc5a2

Co-authored-by: Watson Yuuma Sato <wsato@redhat.com>

yuumasato added this to the 0.1.75 milestone Sep 3, 2024

yuumasato merged commit c31b956 into ComplianceAsCode:master Sep 3, 2024
88 of 93 checks passed

sluetze mentioned this pull request Oct 1, 2024

APP.4.4.A20 sig-bsi-grundschutz/content#46

Closed

sluetze deleted the bsi-app-4.4-a20to21 branch October 28, 2024 21:08

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Bsi app 4.4 a20to21 #11997

Bsi app 4.4 a20to21 #11997

ermeratos commented May 15, 2024

openshift-ci bot commented May 15, 2024

github-actions bot commented May 15, 2024

github-actions bot commented May 15, 2024 •

edited

Loading

yuumasato commented May 22, 2024

yuumasato May 22, 2024

yuumasato May 22, 2024

BhargaviGudi commented Jul 19, 2024

BhargaviGudi commented Jul 19, 2024 •

edited

Loading

BhargaviGudi commented Jul 19, 2024 •

edited

Loading

BhargaviGudi commented Jul 26, 2024

sluetze commented Jul 31, 2024 •

edited

Loading

BhargaviGudi commented Aug 1, 2024

yuumasato commented Aug 8, 2024

yuumasato commented Aug 8, 2024

sluetze commented Aug 8, 2024

yuumasato commented Aug 12, 2024

sluetze commented Aug 20, 2024

codeclimate bot commented Aug 20, 2024

yuumasato commented Aug 20, 2024

yuumasato left a comment

Bsi app 4.4 a20to21 #11997

Bsi app 4.4 a20to21 #11997

Conversation

ermeratos commented May 15, 2024

Description:

Rationale:

openshift-ci bot commented May 15, 2024

github-actions bot commented May 15, 2024

github-actions bot commented May 15, 2024 • edited Loading

yuumasato commented May 22, 2024

yuumasato May 22, 2024

Choose a reason for hiding this comment

yuumasato May 22, 2024

Choose a reason for hiding this comment

BhargaviGudi commented Jul 19, 2024

BhargaviGudi commented Jul 19, 2024 • edited Loading

BhargaviGudi commented Jul 19, 2024 • edited Loading

BhargaviGudi commented Jul 26, 2024

sluetze commented Jul 31, 2024 • edited Loading

BhargaviGudi commented Aug 1, 2024

yuumasato commented Aug 8, 2024

yuumasato commented Aug 8, 2024

sluetze commented Aug 8, 2024

yuumasato commented Aug 12, 2024

sluetze commented Aug 20, 2024

codeclimate bot commented Aug 20, 2024

yuumasato commented Aug 20, 2024

yuumasato left a comment

Choose a reason for hiding this comment

github-actions bot commented May 15, 2024 •

edited

Loading

BhargaviGudi commented Jul 19, 2024 •

edited

Loading

BhargaviGudi commented Jul 19, 2024 •

edited

Loading

sluetze commented Jul 31, 2024 •

edited

Loading