OCPEDGE-1165: fix: Ensure no racy CSI plugin registration, better startup behavior for vgmanager, failing test summary artifact

[jakobmoellerdev]

This stops vgmanager from going healthy too early as we need to signal healthiness only when the driver has been started. Fixes the racy registration server that now properly starts and shuts down and is part of the healthiness check we use in vgmanager. That will ensure that no vgmanager gets ready that isnt registered in the kubelet. Ensures that vgmanager pods are gone before LVMCluster is freed.

Adds an artifact into the e2e step on failure that can be used to see the immediate LVMS cluster state when a failure occurs in case of a cleanup in the post failure steps.

Validates in tests that all CSINodes have proper registrations for vgmanager so we dont have weird flakes in the future.

[jakobmoellerdev]

/test all

[openshift-ci]

Skipping CI for Draft Pull Request.
If you want CI signal for your change, please convert it to an actual PR.
You can still manually trigger a test run with /test all

[openshift-ci]

[APPROVALNOTIFIER] This PR is APPROVED

This pull-request has been approved by: jakobmoellerdev

The full list of commands accepted by this bot can be found here.

The pull request process is described here

Needs approval from an approver in each of these files:

• OWNERS [jakobmoellerdev]

Approvers can indicate their approval by writing /approve in a comment
Approvers can cancel approval by writing /approve cancel in a comment

[codecov-commenter]

Codecov Report

Attention: Patch coverage is 24.48980% with 74 lines in your changes missing coverage. Please review.

Project coverage is 58.50%. Comparing base (b0c38da) to head (65137a4).
Report is 6 commits behind head on main.

Additional details and impacted files

@@            Coverage Diff             @@
##             main     #642      +/-   ##
==========================================
+ Coverage   58.00%   58.50%   +0.49%     
==========================================
  Files          54       54              
  Lines        4117     4258     +141     
==========================================
+ Hits         2388     2491     +103     
- Misses       1492     1521      +29     
- Partials      237      246       +9     

Files	Coverage Δ
...rollers/lvmcluster/resource/vgmanager_daemonset.go	97.20% <100.00%> (+0.16%)	⬆️
internal/csi/grpc_runner.go	0.00% <0.00%> (ø)
...trollers/lvmcluster/resource/topolvm_csi_driver.go	76.00% <30.00%> (-10.05%)	⬇️
...ernal/controllers/lvmcluster/resource/vgmanager.go	68.23% <48.14%> (-11.77%)	⬇️
internal/csi/registrar.go	0.00% <0.00%> (ø)
cmd/vgmanager/vgmanager.go	0.00% <0.00%> (ø)

... and 6 files with indirect coverage changes

[jakobmoellerdev]

/test all

[jakobmoellerdev]

/test all

[jakobmoellerdev]

/test all

[jakobmoellerdev]

/test all

[jakobmoellerdev]

/test all

[openshift-ci-robot]

@jakobmoellerdev: This pull request references OCPEDGE-1165 which is a valid jira issue.

In response to this:

This stops vgmanager from going healthy too early as we need to signal healthiness only when the driver has been started.

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the openshift-eng/jira-lifecycle-plugin repository.

[jakobmoellerdev]

/test e2e-aws
/test e2e-aws-single-node

[jakobmoellerdev]

/test e2e-aws
/test e2e-aws-single-node

[jakobmoellerdev]

/hold
putting on hold until Ive confirmed the flakiness is gone and I added repeat tests that we dont want to have on main

[openshift-ci-robot]

@jakobmoellerdev: This pull request references OCPEDGE-1165 which is a valid jira issue.

In response to this:

This stops vgmanager from going healthy too early as we need to signal healthiness only when the driver has been started. Fixes the racy registration server that now properly starts and shuts down and is part of the healthiness check we use in vgmanager. That will ensure that no vgmanager gets ready that isnt registered in the kubelet. Ensures that vgmanager pods are gone before LVMCluster is freed.

Adds an artifact into the e2e step on failure that can be used to see the immediate LVMS cluster state when a failure occurs in case of a cleanup in the post failure steps.

Validates in tests that all CSINodes have proper registrations for vgmanager so we dont have weird flakes in the future.

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the openshift-eng/jira-lifecycle-plugin repository.

[jakobmoellerdev]

/hold

[jakobmoellerdev]

/unhold

[suleymanakbas91]

LGTM, just small nits

[suleymanakbas91]

/lgtm

[openshift-ci]

@jakobmoellerdev: The following tests failed, say /retest to rerun all failed tests or /retest-required to rerun all mandatory failed tests:

Test name	Commit	Details	Required	Rerun command
ci/prow/lvm-operator-e2e-aws-sno	f956eba	link	true	/test lvm-operator-e2e-aws-sno
ci/prow/lvm-operator-e2e-aws	f956eba	link	true	/test lvm-operator-e2e-aws

Full PR test history. Your PR dashboard.

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes-sigs/prow repository. I understand the commands that are listed here.