Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

[PERFSCALE-3438] Introduce UDN Density L3 workload #57161

Merged
merged 25 commits into from
Oct 11, 2024
Merged

[PERFSCALE-3438] Introduce UDN Density L3 workload #57161

merged 25 commits into from
Oct 11, 2024

Conversation

krishvoor
Copy link
Member

This PR adds the newest workload to our kube-burner-ocp repo
as udn-density-l3-pods

kube-burner/kube-burner-ocp#106

@krishvoor
[PERFSCALE-3438] Introduce UDN Density L3 workload
37bd674 
This PR adds newest introduction to our kube-burner-ocp repo
as udn-density-l3-pods

kube-burner/kube-burner-ocp#106

Signed-off-by: Krishna Harsha Voora <krvoora@redhat.com>
@krishvoor
Copy link
Member Author

We need a new version/branch to be released on kube-burner-ocp and updates to e2e-benchmarking.

@openshift-ci-robot
Copy link
Contributor

@krishvoor, pj-rehearse: unable to determine affected jobs. This could be due to a branch that needs to be rebased. ERROR:

could not determine changed registry steps: could not load step registry: failed to load registry file /var/tmp/gitrepo357335877/ci-operator/step-registry/openshift-qe/udn-density-l3-pods/openshift-qe-udn-density-l3-pods-ref.yaml: reference openshift-qe-udn-density-l3-pods has invalid command file path; command should be set to openshift-qe-udn-density-l3-pods-commands (with an optional extension like .sh)
Interacting with pj-rehearse

Comment: /pj-rehearse to run up to 5 rehearsals
Comment: /pj-rehearse skip to opt-out of rehearsals
Comment: /pj-rehearse {test-name}, with each test separated by a space, to run one or more specific rehearsals
Comment: /pj-rehearse more to run up to 10 rehearsals
Comment: /pj-rehearse max to run up to 25 rehearsals
Comment: /pj-rehearse auto-ack to run up to 5 rehearsals, and add the rehearsals-ack label on success
Comment: /pj-rehearse abort to abort all active rehearsals

Once you are satisfied with the results of the rehearsals, comment: /pj-rehearse ack to unblock merge. When the rehearsals-ack label is present on your PR, merge will no longer be blocked by rehearsals.
If you would like the rehearsals-ack label removed, comment: /pj-rehearse reject to re-block merging.

@josecastillolema
Copy link
Contributor

We need a new version/branch to be released on kube-burner-ocp and updates to e2e-benchmarking.

We also need to wait for the TP enablement step that @jtaleric has been working on @krishvoor in order to be able to test this.

@krishvoor
Copy link
Member Author

We need a new version/branch to be released on kube-burner-ocp and updates to e2e-benchmarking.

We also need to wait for the TP enablement step that @jtaleric has been working on @krishvoor in order to be able to test this.

Currently, I'm focused on adding/introducing the workload through the PR.

For self-managed clusters, we can enable FEATURE_GATES which is an environment variable part of this workflow
https://steps.ci.openshift.org/workflow/openshift-qe-installer-aws#environment
And tackle it in a different PR.

@krishvoor
Minor nit
903389c 
Signed-off-by: Krishna Harsha Voora <krvoora@redhat.com>
@openshift-ci-robot openshift-ci-robot added the rehearsals-ack Signifies that rehearsal jobs have been acknowledged label Sep 26, 2024
@krishvoor
Copy link
Member Author

/pj-rehearse ack

@openshift-ci-robot
Copy link
Contributor

@krishvoor: now processing your pj-rehearse request. Please allow up to 10 minutes for jobs to trigger or cancel.

@jtaleric
Copy link
Contributor

Digging through the current ipi-conf-aws it appears introducing FEATURE_GATE at install would be good to get in before this, to ensure we can test this PR prior to merge, wdyt?

@krishvoor
Copy link
Member Author

krishvoor commented Sep 26, 2024 •
edited
Loading

Digging through the current ipi-conf-aws it appears introducing FEATURE_GATE at install would be good to get in before this, to ensure we can test this PR prior to merge, wdyt?

Yes, your observations are in line with my findings:
#57161 (comment)

we can make use of FEATURE_GATE along with FEATURE_SET allowing us to enable TechPreviewNoUpgrade, in the process of adding new 4.18 with 24 nodes job to get things rolling.

@krishvoor
Copy link
Member Author

We need a new version/branch to be released on kube-burner-ocp and updates to e2e-benchmarking.

We also need to wait for the TP enablement step that @jtaleric has been working on @krishvoor in order to be able to test this.

Currently, I'm focused on adding/introducing the workload through the PR.

For self-managed clusters, we can enable FEATURE_GATES which is an environment variable part of this workflow https://steps.ci.openshift.org/workflow/openshift-qe-installer-aws#environment And tackle it in a different PR.

@krishvoor krishvoor closed this Sep 26, 2024
@krishvoor krishvoor reopened this Sep 26, 2024
@krishvoor
Add 4.18 24 node job
ddf942c 
Signed-off-by: Krishna Harsha Voora <krvoora@redhat.com>
@krishvoor
Adds make ci-operator-prowgen
3308ec2 
Signed-off-by: Krishna Harsha Voora <krvoora@redhat.com>
@openshift-ci-robot openshift-ci-robot removed the rehearsals-ack Signifies that rehearsal jobs have been acknowledged label Sep 26, 2024
@krishvoor
Copy link
Member Author

/pj-rehearse pull-ci-openshift-qe-ocp-qe-perfscale-ci-main-aws-4.18-nightly-x86-udn-density-l3-pods-24nodes

@openshift-ci-robot
Copy link
Contributor

@krishvoor: now processing your pj-rehearse request. Please allow up to 10 minutes for jobs to trigger or cancel.

@openshift-ci-robot
Copy link
Contributor

@krishvoor, pj-rehearse: failed to create rehearsal jobs ERROR:

no config for buildcluster default provided

If the problem persists, please contact Test Platform.

@krishvoor
Copy link
Member Author

/pj-rehearse pull-ci-openshift-qe-ocp-qe-perfscale-ci-main-aws-4.18-nightly-x86-udn-density-l3-pods-24nodes

@openshift-ci-robot
Copy link
Contributor

@krishvoor: now processing your pj-rehearse request. Please allow up to 10 minutes for jobs to trigger or cancel.

@krishvoor
Copy link
Member Author

/pj-rehearse pull-ci-openshift-qe-ocp-qe-perfscale-ci-main-aws-4.18-nightly-x86-udn-density-l3-pods-24nodes

@openshift-ci-robot
Copy link
Contributor

@krishvoor: now processing your pj-rehearse request. Please allow up to 10 minutes for jobs to trigger or cancel.

@krishvoor
Copy link
Member Author

krishvoor commented Sep 26, 2024 •
edited
Loading

From one of the worker nodes, the TechPreviewNoUpgrade FeatureGate is enabled.

cat /etc/kubernetes/kubelet.conf | grep TechPreviewNoUpgrade
  TechPreviewNoUpgrade: true
sh-5.1# exit
exit
sh-5.1# exit
exit

Removing debug pod ...
$ oc version
Client Version: 4.12.0-rc.6
Kustomize Version: v4.5.7
Kubernetes Version: v1.30.3
$

@krishvoor
Copy link
Member Author

For some weird reason, the infra nodes are not coming online - needs investigation.

@krishvoor
Copy link
Member Author

/pull-ci-openshift-qe-ocp-qe-perfscale-ci-main-aws-4.18-nightly-x86-udn-density-l3-pods-24nodes

@krishvoor
Copy link
Member Author

/pj-rehearse pull-ci-openshift-qe-ocp-qe-perfscale-ci-main-aws-4.18-nightly-x86-udn-density-l3-pods-24nodes

@openshift-ci-robot
Copy link
Contributor

@krishvoor: now processing your pj-rehearse request. Please allow up to 10 minutes for jobs to trigger or cancel.

@krishvoor
Copy link
Member Author

The changes here worked as expected: cloud-bulldozer/e2e-benchmarking#724
CI Job: https://prow.ci.openshift.org/view/gs/test-platform-results/pr-logs/pull/openshift_release/57161/rehearse-57161-pull-ci-openshift-qe-ocp-qe-perfscale-ci-main-aws-4.18-nightly-x86-udn-density-l3-pods-24nodes/1841666847351508992

@krishvoor
Copy link
Member Author

/pj-rehearse ack

@openshift-ci-robot
Copy link
Contributor

@krishvoor: now processing your pj-rehearse request. Please allow up to 10 minutes for jobs to trigger or cancel.

@openshift-ci-robot openshift-ci-robot added rehearsals-ack Signifies that rehearsal jobs have been acknowledged and removed rehearsals-ack Signifies that rehearsal jobs have been acknowledged labels Oct 3, 2024
@krishvoor
Copy link
Member Author

/pj-rehearse ack

@openshift-ci-robot
Copy link
Contributor

@krishvoor: now processing your pj-rehearse request. Please allow up to 10 minutes for jobs to trigger or cancel.

@openshift-ci-robot openshift-ci-robot added the rehearsals-ack Signifies that rehearsal jobs have been acknowledged label Oct 3, 2024
@openshift-ci-robot openshift-ci-robot removed the rehearsals-ack Signifies that rehearsal jobs have been acknowledged label Oct 8, 2024
@openshift-ci-robot openshift-ci-robot added the rehearsals-ack Signifies that rehearsal jobs have been acknowledged label Oct 11, 2024
@openshift-ci-robot
Copy link
Contributor

[REHEARSALNOTIFIER]
@krishvoor: no rehearsable tests are affected by this change

Interacting with pj-rehearse

Comment: /pj-rehearse to run up to 5 rehearsals
Comment: /pj-rehearse skip to opt-out of rehearsals
Comment: /pj-rehearse {test-name}, with each test separated by a space, to run one or more specific rehearsals
Comment: /pj-rehearse more to run up to 10 rehearsals
Comment: /pj-rehearse max to run up to 25 rehearsals
Comment: /pj-rehearse auto-ack to run up to 5 rehearsals, and add the rehearsals-ack label on success
Comment: /pj-rehearse abort to abort all active rehearsals
Comment: /pj-rehearse network-access-allowed to allow rehearsals of tests that have the restrict_network_access field set to false. This must be executed by an openshift org member who is not the PR author

Once you are satisfied with the results of the rehearsals, comment: /pj-rehearse ack to unblock merge. When the rehearsals-ack label is present on your PR, merge will no longer be blocked by rehearsals.
If you would like the rehearsals-ack label removed, comment: /pj-rehearse reject to re-block merging.

@krishvoor
Copy link
Member Author

/pj-rehearse ack

@openshift-ci-robot
Copy link
Contributor

@krishvoor: now processing your pj-rehearse request. Please allow up to 10 minutes for jobs to trigger or cancel.

@jtaleric
Copy link
Contributor

/lgtm

@openshift-ci openshift-ci bot added the lgtm Indicates that a PR is ready to be merged. label Oct 11, 2024
@openshift-ci OpenShift CI
Copy link
Contributor

openshift-ci bot commented Oct 11, 2024

[APPROVALNOTIFIER] This PR is APPROVED

This pull-request has been approved by: jtaleric, krishvoor, venkataanil

The full list of commands accepted by this bot can be found here.

The pull request process is described here

Needs approval from an approver in each of these files:

Approvers can indicate their approval by writing /approve in a comment
Approvers can cancel approval by writing /approve cancel in a comment

@openshift-ci openshift-ci bot added the approved Indicates a PR has been approved by an approver from all required OWNERS files. label Oct 11, 2024
@openshift-ci OpenShift CI
Copy link
Contributor

openshift-ci bot commented Oct 11, 2024

@krishvoor: The following tests failed, say /retest to rerun all failed tests or /retest-required to rerun all mandatory failed tests:

Test name Commit Details Required Rerun command
ci/prow/generated-config a0a3c23 link true /test generated-config
ci/prow/ordered-prow-config a0a3c23 link true /test ordered-prow-config

Full PR test history. Your PR dashboard.

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes-sigs/prow repository. I understand the commands that are listed here.

@openshift-merge-bot openshift-merge-bot bot merged commit 31cf4b1 into openshift:master Oct 11, 2024
10 checks passed
@jtaleric jtaleric deleted the cp-udn-workload branch October 11, 2024 18:17
josecastillolema pushed a commit to josecastillolema/release that referenced this pull request Oct 15, 2024
@krishvoor @josecastillolema
[PERFSCALE-3438] Introduce UDN Density L3 workload (openshift#57161)
19625b9 
* [PERFSCALE-3438] Introduce UDN Density L3 workload

This PR adds newest introduction to our kube-burner-ocp repo
as udn-density-l3-pods

kube-burner/kube-burner-ocp#106

Signed-off-by: Krishna Harsha Voora <krvoora@redhat.com>

* Minor nit

Signed-off-by: Krishna Harsha Voora <krvoora@redhat.com>

* minor nit

* Add 4.18 24 node job

Signed-off-by: Krishna Harsha Voora <krvoora@redhat.com>

* Adds make ci-operator-prowgen

Signed-off-by: Krishna Harsha Voora <krvoora@redhat.com>

* Adds the buildcluster info

* Updates the worker instance type and disable infra node

* Reverts the OPENSHIFT_INFRA_NODE_INSTANCE_TYPE for data-path jobs

* make updates

* no need to move any components

* Bump benchmarking version

* Testing

Signed-off-by: Krishna Harsha Voora <krvoora@redhat.com>

* Workaround Fix

* Adds TechPreviewNoUpgrade

* Remove FeatureGate

* Set POD_READY_THRESHOLD:1400s

* Adds documentation reference

* Minor nits

1) Updates to ITERATIONS logic
2) Add ITERATION_MULTIPLIER_ENV
3) House keeping of undesired code/comments

Signed-off-by: Krishna Harsha Voora <krvoora@redhat.com>

* fixes makes build

* Remove the Pull Job

* Minor Nits

---------

Signed-off-by: Krishna Harsha Voora <krvoora@redhat.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@jtaleric jtaleric jtaleric left review comments

@venkataanil venkataanil venkataanil approved these changes

@josecastillolema josecastillolema Awaiting requested review from josecastillolema

@vishnuchalla vishnuchalla Awaiting requested review from vishnuchalla

Assignees

@jtaleric jtaleric

@venkataanil venkataanil

@mohit-sheth mohit-sheth

Labels
approved Indicates a PR has been approved by an approver from all required OWNERS files. lgtm Indicates that a PR is ready to be merged. rehearsals-ack Signifies that rehearsal jobs have been acknowledged
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.
6 participants
@krishvoor @openshift-ci-robot @josecastillolema @jtaleric @mohit-sheth @venkataanil