DataVolume Controller uses VolumeCloneSource Populator #2750

mhenriks · 2023-06-13T03:08:12Z

What this PR does / why we need it:

The DataVolume controller should use our populators internally for CSI storage provisioners. This PR builds off of #2722 to include VolumeCloneSource populators.

Which issue(s) this PR fixes (optional, in fixes #<issue number>(, fixes #<issue_number>, ...) format, will close the issue(s) when PR gets merged):
Fixes #

Special notes for your reviewer:

I think everything is here but I bet there will be more whackamole on func tests

Release note:

DataVolume Controller uses VolumeCloneSource Populator

mhenriks · 2023-06-13T03:08:21Z

/hold

mhenriks · 2023-06-13T03:18:52Z

pkg/controller/datavolume/pvc-clone-controller.go

@@ -468,20 +468,32 @@ func (r *PvcCloneReconciler) isSourceReadyToClone(datavolume *cdiv1.DataVolume)

 // detectCloneSize obtains and assigns the original PVC's size when cloning using an empty storage value
 func (r *PvcCloneReconciler) detectCloneSize(syncState *dvSyncState) (bool, error) {
-	var targetSize int64


special note on all the changes done in this function...was all done to get this crazy test to work:

containerized-data-importer/tests/cloner_test.go

Line 576 in 35f1c79

It("bz:2079781 Should clone data from filesystem to block, when using storage API ", func() {

mhenriks · 2023-06-13T11:54:11Z

/test pull-containerized-data-importer-e2e-ceph

alromeros

Very good start! Just some comments after an initial review.

alromeros · 2023-06-13T16:29:21Z

pkg/controller/datavolume/clone-controller-base.go

@@ -84,7 +85,7 @@ const (
 	// MessageSmartCloneInProgress provides a const to form snapshot for smart-clone is in progress message
 	MessageSmartCloneInProgress = "Creating snapshot for smart-clone is in progress (for pvc %s/%s)"
 	// MessageCloneFromSnapshotSourceInProgress provides a const to form clone from snapshot source is in progress message
-	MessageCloneFromSnapshotSourceInProgress = "Creating PVC from snapshot source is in progress (for snapshot %s/%s)"
+	MessageCloneFromSnapshotSourceInProgress = "Creating PVC from snapshot source is in progress (for %s %s/%s)"


Since you are now requiring the source type as argument, maybe change the message so it can be used for both snapshots and pvcs?

Yeah I think that error string can be improved

Actually I think I'm not sure what you're asking...the message is used whenever we create a pvc from a snapshot. That happens with pvc-smart clone and snapshot-clone. Are you suggesting the message be use when a pvc is snapshotted as well? That only happens with pvc-smart clone. Examples would be great.

the name of the variable should change maybe drop the "snapshot"

Hmmm...I think the source part may be the most confusing since someone may be thinking of the DataVolume source and not the pvc dataSource.

alromeros · 2023-06-13T16:29:42Z

pkg/controller/common/util.go

+}
+
+// MergePatch patches a resource
+func MergePatch(ctx context.Context, args *PatchArgs) error {


alromeros · 2023-06-13T16:43:03Z

pkg/controller/datavolume/clone-controller-base.go

@@ -178,6 +187,67 @@ func (r *CloneReconcilerBase) ensureExtendedToken(pvc *corev1.PersistentVolumeCl
 	return nil
 }

+func (r *CloneReconcilerBase) reconcileVolumeCloneSourceCR(syncState *dvSyncState, kind string) error {
+	dv := syncState.dvMutated
+	cloneSource := &cdiv1.VolumeCloneSource{}


I think it'd improve readability to use volumeCloneSource and volumeCloneSourceName to differentiate from the actual clone source.

alromeros · 2023-06-13T17:15:26Z

pkg/controller/datavolume/pvc-clone-controller.go

 	sourcePvc, err := r.findSourcePvc(syncState.dvMutated)
 	if err != nil {
 		return false, err
 	}
+
+	// because of filesystem overhead calculations when cloning
+	// even if storage size is requested we have to calucuale source size


Typo: calucuale

alromeros · 2023-06-13T17:51:06Z

pkg/controller/datavolume/pvc-clone-controller.go

+		// TODO: Fix this in next PR that uses actual size also in validation
+		isPermissiveClone = sourceCapacity.CmpInt64(targetSize) == 1
+	} else {
+		isPermissiveClone = requestedSize.CmpInt64(targetSize) >= 0


I guess this has to do with the test you commented above, right? But why would we need to set the permissiveClone annotation if the comparison is equal or larger than 0?

requestedSize may still be smaller than source capacity (as in that contrived test case) but populator layer doesn't know that and will understandably not let you clone your 4G source to a 2G target

alromeros · 2023-06-13T18:15:41Z

pkg/controller/upload-controller.go

@@ -203,10 +203,15 @@ func (r *UploadReconciler) reconcilePVC(log logr.Logger, pvc *corev1.PersistentV
 		}

 		if len(podsUsingPVC) > 0 {
+			es, err := cc.GetAnnotatedEventSource(context.TODO(), r.client, pvc)


Sorry to bother this much about styling but I feel like these variable names work for more common objects like dvs or pvcs. For most cases I think it's better to use the full version, wdyt?

Not sure what you are suggesting here...you don't like 'es'?

Yeah, I've seen some of these in the PR and I always think it'd be better to have the full name, like cloneType instead of ct. Just a small complaint though, not a must.

I don't necessarily disagree with you. I typically use descriptive names (I think). But I also like to use short names to signify "this is something you shouldn't really care about". Similar to 'i' in loops. Variables that are very short in scope.

mhenriks · 2023-06-16T02:47:47Z

/retest-required

ShellyKa13

Looks good! have some comments, as usual going over commit by commit some were already addressed..

ShellyKa13 · 2023-06-13T13:04:51Z

pkg/controller/datavolume/pvc-clone-controller.go

@@ -20,7 +20,6 @@ import (
 	"context"


I do love the bye bye commit message, but maybe share why is it being removed?

Pretty sure you know why, but since populators are handling all CSI storage the DataVolume controller will only be doing host assisted clones

I meant share in the commit message

ShellyKa13 · 2023-06-13T13:11:15Z

pkg/controller/datavolume/snapshot-clone-controller.go

@@ -582,3 +590,77 @@ func (r *SnapshotCloneReconciler) isSnapshotValidForClone(snapshot *snapshotv1.V
 	}
 	return true, nil
 }
+
+func newPvcFromSnapshot(obj metav1.Object, name string, snapshot *snapshotv1.VolumeSnapshot, targetPvcSpec *corev1.PersistentVolumeClaimSpec) (*corev1.PersistentVolumeClaim, error) {


is this function relevant to this this commit? I dont see it being used anywhere

it was in smart clone controller file which was deleted but this function and a couple other defs are needed

ShellyKa13 · 2023-06-13T13:12:20Z

pkg/controller/datavolume/snapshot-clone-controller.go

-const snapshotCloneControllerName = "datavolume-snapshot-clone-controller"
+const (
+	//AnnSmartCloneRequest sets our expected annotation for a CloneRequest
+	AnnSmartCloneRequest = "k8s.io/SmartCloneRequest"


if newPvcFromSnapshot is not relevant to this commit (as asked in another comment) then I guess this const are neither?

remnants from a deleted file

ShellyKa13 · 2023-06-19T08:52:49Z

pkg/controller/common/util.go

-// GetCloneSourceNameAndNamespace returns the name and namespace of the cloning source
-func GetCloneSourceNameAndNamespace(dv *cdiv1.DataVolume) (name, namespace string) {
-	var sourceName, sourceNamespace string
+// GetCloneSourceInfo returns the name and namespace of the cloning source


can also update the comment with returns sourceType

ShellyKa13 · 2023-06-19T09:09:34Z

pkg/controller/datavolume/clone-controller-base.go

@@ -84,7 +85,7 @@ const (
 	// MessageSmartCloneInProgress provides a const to form snapshot for smart-clone is in progress message
 	MessageSmartCloneInProgress = "Creating snapshot for smart-clone is in progress (for pvc %s/%s)"
 	// MessageCloneFromSnapshotSourceInProgress provides a const to form clone from snapshot source is in progress message
-	MessageCloneFromSnapshotSourceInProgress = "Creating PVC from snapshot source is in progress (for snapshot %s/%s)"
+	MessageCloneFromSnapshotSourceInProgress = "Creating PVC from snapshot source is in progress (for %s %s/%s)"


the name of the variable should change maybe drop the "snapshot"

ShellyKa13 · 2023-06-19T11:23:50Z

pkg/controller/populators/clone-populator.go

 	cc.AddAnnotation(claimCpy, AnnCloneError, lastError.Error())

 	if !apiequality.Semantic.DeepEqual(pvc, claimCpy) {
-		if err := r.client.Update(ctx, claimCpy); err != nil {
+		if err := r.patchClaim(ctx, log, pvc, claimCpy); err != nil {
 			r.log.V(1).Info("error setting error annotations")


use here log instead of r.log?

ShellyKa13 · 2023-06-19T11:31:58Z

pkg/controller/datavolume/clone-controller-base.go

+		},
+		Spec: cdiv1.VolumeCloneSourceSpec{
+			Source: corev1.TypedLocalObjectReference{
+				Kind: "PersistentVolumeClaim",


I guess you wanted to put Kind: kind

ShellyKa13 · 2023-06-20T10:05:12Z

pkg/controller/datavolume/pvc-clone-controller.go

+		return nil
+	}
+
+	return r.reconcileVolumeCloneSourceCR(syncState, "PersistentVolumeClaim")


calling to this function even when not using populators will work but not sure if we should..

ShellyKa13 · 2023-06-20T10:16:40Z

pkg/controller/populators/clone-populator.go

+var desiredCloneAnnotations []string
+
+func init() {
+	desiredCloneAnnotations = append(desiredCloneAnnotations, desiredAnnotations...)


I think you can just add the AnnCloneOf to the desiredannotations list.. it wont hurt anything, maybe the name is not the best and should change maybe you have better name

ShellyKa13 · 2023-06-20T10:36:07Z

pkg/controller/datavolume/snapshot-clone-controller.go

+	}
+
+	if dv.DeletionTimestamp != nil {
+		if err := r.reconcileVolumeCloneSourceCR(syncState); err != nil {


sure we dont want to call this also when dv succeeded?

It will get called here:

containerized-data-importer/pkg/controller/datavolume/snapshot-clone-controller.go

Line 233 in 9da9c01

if err := r.reconcileVolumeCloneSourceCR(&syncRes); err != nil {

if for some reason we will exit here:

containerized-data-importer/pkg/controller/datavolume/snapshot-clone-controller.go

Line 181 in 9da9c01

return syncRes, syncErr

or here:

containerized-data-importer/pkg/controller/datavolume/snapshot-clone-controller.go

Line 193 in 9da9c01

return syncRes, nil

then we will never delete the volumeclonesource. dont see why shouldn't just call it few lines after https://github.com/kubevirt/containerized-data-importer/pull/2750/files/ac7a342dcd831bb0746e634be774942d05d20086..9da9c01e032c1c6d26e242f3b3fe5789c456eb6e#diff-57c4756c3e19744255585cd838f48d98a4410e30b31813b74b23874c998b3268R441
after the check of deleted and succeeded..

Unless there is an unexpected error (which is retried), it should get called. But I suppose the behavior of the base controller could change and mess things up as well. This is such spaghetti code. IMO controllers should always execute the same code block at each reconcile. I don't like that the derived controller sync is not called if the DataVolume is deleted.

Anyway, my intention was to reduce special cases and have the cleanup function do less. Because I don't think it should exist in the first place.

mhenriks · 2023-06-22T00:15:24Z

@arnongilboa - please check general integration with DataVolume controllers. A lot of code is gone!
@akalenyu - please check out clone from snapshot path

akalenyu · 2023-06-22T10:13:37Z

Not sure what that consistent after suite failure is about, seems some DV has a finalizer remaining

Some content in the namespace has finalizers remaining: cdi.kubevirt.io/dataVolumeFinalizer in 1 resource instances}]}}]

We might want to dump out some info about dangling resources after func test runs

akalenyu

@akalenyu - please check out clone from snapshot path

Looks good, we should also have most permutations covered by snapclone func tests
so that makes me feel better

My only concern was that we are losing coverage in that host assisted path

akalenyu · 2023-06-22T10:08:19Z

pkg/controller/datavolume/snapshot-clone-controller.go

-			return syncRes, err
-		}
-		targetHostAssistedPvc, err := r.createPvcForDatavolume(datavolume, pvcSpec, r.updateAnnotations)
+		targetPvc, err := r.createPvcForDatavolume(datavolume, pvcSpec, pvcModifier)


I went ahead and debugged the sizeless clone failure from volumesnapshot clones:
Since we now create the target PVC through createPvcForDatavolume, nothing takes
care of filling out the missing size

spec.resources[storage]: Invalid value: \"0\": must be greater than zero

I think we can simply have something similar to this func

containerized-data-importer/pkg/controller/datavolume/pvc-clone-controller.go

Line 501 in 75e1297

syncState.pvcSpec.Resources.Requests[corev1.ResourceStorage] = targetCapacity

(but way simpler, just grabs the size over from the snapshot)

akalenyu · 2023-06-22T10:35:38Z

tests/cloner_test.go

@@ -772,11 +723,6 @@ var _ = Describe("all clone tests", func() {
 						targetDiskImagePath = testBaseDir
 					}

-					if cloneType == "snapshot" && sourceRef {


akalenyu · 2023-06-22T10:46:12Z

tests/cloner_test.go

+			}
+
+			f.ForceBindIfWaitForFirstConsumer(targetPvc)
+
 			cloneType := utils.GetCloneType(f.CdiClient, dataVolume)
 			if cloneType != "copy" {
 				Skip("only valid for copy clone")


wasn't this skip enough?

With populators don't know clone type until populator starts working (first consumer triggered)

akalenyu · 2023-06-22T10:52:39Z

tests/cloner_test.go

-					Expect(pvc.Annotations[controller.AnnCloneRequest]).To(HaveSuffix(suffix))
-					Expect(pvc.Spec.DataSource).To(BeNil())
-					Expect(pvc.Spec.DataSourceRef).To(BeNil())
+					if pvc.Spec.DataSourceRef == nil {


Maybe we should go for a more correct way of ensuring dumbclone took place?
IIUC when this runs on a CSI lane the check is not exercised

This is specifically for checking non-csi case. With populators, the pvc that was used for dumb clone is deleted after rebinding to target. But can check clonetype annotation on the dv

akalenyu · 2023-06-22T16:55:08Z

/test pull-containerized-data-importer-e2e-destructive
/test pull-cdi-linter

destructive lane fails because of #2744
apidocs fails because of [kubevirt-dev] k/k: pull-kubevirt-api failing due to missing gradle dependency

mhenriks · 2023-06-24T11:44:06Z

/retest-required

akalenyu · 2023-06-25T10:44:17Z

One failure seems to be exclusive to this PR:
[test_id:1360]: DV conditions have Ready=False but "phase": "Succeeded":
https://search.ci.kubevirt.io/?search=1360&maxAge=336h&context=1&type=bug%2Bissue%2Bjunit&name=containerized-data-importer&excludeName=&maxMatches=5&maxBytes=20971520&groupBy=job

Another one isn't:
[test_id:8569]: the DV Update call fails (confusing from gomega output but this is what happens):
https://search.ci.kubevirt.io/?search=8569&maxAge=336h&context=1&type=bug%2Bissue%2Bjunit&name=containerized-data-importer&excludeName=&maxMatches=5&maxBytes=20971520&groupBy=job

containerized-data-importer/tests/cloner_test.go

Line 3245 in d173e08

    
           Expect(f.CdiClient.CdiV1beta1().DataVolumes(dvNamespace).Update(context.TODO(), dv, metav1.UpdateOptions{})).Error().ToNot(HaveOccurred())

akalenyu · 2023-06-26T13:28:37Z

/test pull-containerized-data-importer-e2e-nfs
/test pull-containerized-data-importer-e2e-destructive

https://search.ci.kubevirt.io/?search=+preallocate+data+on+target+PVC&maxAge=336h&context=1&type=bug%2Bissue%2Bjunit&name=containerized-data-importer&excludeName=&maxMatches=5&maxBytes=20971520&groupBy=job
This seems to fail exclusively on this PR - suggests preallocation not applied sometimes?

Let me know how you want to proceed... for the other flakes could use a review at
#2769
#2744

mhenriks · 2023-06-26T19:33:06Z

/test pull-containerized-data-importer-e2e-nfs /test pull-containerized-data-importer-e2e-destructive

https://search.ci.kubevirt.io/?search=+preallocate+data+on+target+PVC&maxAge=336h&context=1&type=bug%2Bissue%2Bjunit&name=containerized-data-importer&excludeName=&maxMatches=5&maxBytes=20971520&groupBy=job This seems to fail exclusively on this PR - suggests preallocation not applied sometimes?

Let me know how you want to proceed... for the other flakes could use a review at #2769 #2744

Thanks @akalenyu I think I have a lead on the failure in this pr. I'll take a look at the fixes you reference

Signed-off-by: Michael Henriksen <mhenriks@redhat.com>

also add more validation to some snapshot clone tests Signed-off-by: Michael Henriksen <mhenriks@redhat.com>

Using the controller runtime Patch API with controller runtime cached client seems to be a pretty bad fit At least given the way the CR API is designed where an old object is compared to new. I like patch in theory though and will revisit Signed-off-by: Michael Henriksen <mhenriks@redhat.com>

It was possible to miss "preallocation applied" annotation otherwise Signed-off-by: Michael Henriksen <mhenriks@redhat.com>

akalenyu · 2023-06-27T08:18:05Z

/test pull-containerized-data-importer-e2e-nfs

Signed-off-by: Michael Henriksen <mhenriks@redhat.com>

Should have been done back when annotations were addded to "progress" Also, if pvc is bound do not call phase Reconcile functions only Status Signed-off-by: Michael Henriksen <mhenriks@redhat.com>

mhenriks · 2023-06-27T20:23:00Z

/unhold

kubevirt-bot · 2023-06-28T01:18:43Z

@mhenriks: The following test failed, say /retest to rerun all failed tests or /retest-required to rerun all mandatory failed tests:

Test name	Commit	Details	Required	Rerun command
pull-cdi-apidocs	`99bf0dd`	link	false	`/test pull-cdi-apidocs`

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes/test-infra repository. I understand the commands that are listed here.

mhenriks · 2023-06-28T11:55:40Z

/retest-required

akalenyu

/cc @ShellyKa13

akalenyu · 2023-06-28T15:24:29Z

whoops
/lgtm cancel
/approve

kubevirt-bot · 2023-06-28T15:24:39Z

[APPROVALNOTIFIER] This PR is APPROVED

This pull-request has been approved by: akalenyu

The full list of commands accepted by this bot can be found here.

The pull request process is described here

Needs approval from an approver in each of these files:

~~OWNERS~~ [akalenyu]

Approvers can indicate their approval by writing /approve in a comment
Approvers can cancel approval by writing /approve cancel in a comment

ShellyKa13

/lgtm

mhenriks · 2023-06-30T13:17:40Z

/cherrypick release-v1.57

kubevirt-bot · 2023-06-30T13:20:32Z

@mhenriks: new pull request created: #2783

In response to this:

/cherrypick release-v1.57

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes/test-infra repository.

kubevirt-bot requested review from aglitke and awels June 13, 2023 03:08

kubevirt-bot added do-not-merge/hold Indicates that a PR should not merge because someone has issued a /hold command. size/XXL labels Jun 13, 2023

mhenriks commented Jun 13, 2023

View reviewed changes

alromeros reviewed Jun 13, 2023

View reviewed changes

kubevirt-bot added the needs-rebase Indicates a PR cannot be merged because it has merge conflicts with HEAD. label Jun 14, 2023

mhenriks force-pushed the clone-pop-int branch from 22964cf to f0da0c0 Compare June 15, 2023 00:46

kubevirt-bot removed the needs-rebase Indicates a PR cannot be merged because it has merge conflicts with HEAD. label Jun 15, 2023

mhenriks force-pushed the clone-pop-int branch from 53c1b2e to 9da9c01 Compare June 16, 2023 18:25

ShellyKa13 reviewed Jun 20, 2023

View reviewed changes

ShellyKa13 mentioned this pull request Jun 21, 2023

Allow ImmediateBinding annotation when using populators #2765

Merged

mhenriks force-pushed the clone-pop-int branch from 9da9c01 to 75e1297 Compare June 22, 2023 00:10

mhenriks changed the title ~~[WIP] DataVolume Controller uses VolumeCloneSource Populator~~ DataVolume Controller uses VolumeCloneSource Populator Jun 22, 2023

kubevirt-bot removed the do-not-merge/work-in-progress Indicates that a PR should not merge because it is a work in progress. label Jun 22, 2023

akalenyu reviewed Jun 22, 2023

View reviewed changes

mhenriks force-pushed the clone-pop-int branch from 99bf0dd to d173e08 Compare June 23, 2023 20:58

mhenriks added 5 commits June 26, 2023 21:05

remove skipped test added in kubevirt#2759

6e49650

Signed-off-by: Michael Henriksen <mhenriks@redhat.com>

attempt address AfterSuite and generate-verify failures

7ca275c

Signed-off-by: Michael Henriksen <mhenriks@redhat.com>

handle snapshot clone with no target size specified

c76d2d6

also add more validation to some snapshot clone tests Signed-off-by: Michael Henriksen <mhenriks@redhat.com>

Clone populator should plan and execute even if PVC is bound

691028e

It was possible to miss "preallocation applied" annotation otherwise Signed-off-by: Michael Henriksen <mhenriks@redhat.com>

mhenriks force-pushed the clone-pop-int branch from 2e75b5a to 75819aa Compare June 27, 2023 01:05

ShellyKa13 mentioned this pull request Jun 27, 2023

Add documentation for cdi populators #2776

Merged

mhenriks added 2 commits June 27, 2023 16:21

add long term token to datavolume

f605718

Signed-off-by: Michael Henriksen <mhenriks@redhat.com>

Rename ProgressReporter to StatusReporter

7edeaee

Should have been done back when annotations were addded to "progress" Also, if pvc is bound do not call phase Reconcile functions only Status Signed-off-by: Michael Henriksen <mhenriks@redhat.com>

mhenriks force-pushed the clone-pop-int branch from 75819aa to 7edeaee Compare June 27, 2023 20:22

kubevirt-bot removed the do-not-merge/hold Indicates that a PR should not merge because someone has issued a /hold command. label Jun 27, 2023

akalenyu approved these changes Jun 28, 2023

View reviewed changes

kubevirt-bot requested a review from ShellyKa13 June 28, 2023 15:24

kubevirt-bot assigned akalenyu Jun 28, 2023

kubevirt-bot added the lgtm Indicates that a PR is ready to be merged. label Jun 28, 2023

kubevirt-bot removed the lgtm Indicates that a PR is ready to be merged. label Jun 28, 2023

kubevirt-bot added the approved Indicates a PR has been approved by an approver from all required OWNERS files. label Jun 28, 2023

ShellyKa13 reviewed Jun 29, 2023

View reviewed changes

kubevirt-bot assigned ShellyKa13 Jun 29, 2023

kubevirt-bot added the lgtm Indicates that a PR is ready to be merged. label Jun 29, 2023

kubevirt-bot merged commit 4ce9272 into kubevirt:main Jun 29, 2023

kubevirt-bot mentioned this pull request Jun 30, 2023

[release-v1.57] DataVolume Controller uses VolumeCloneSource Populator #2783

Merged

DataVolume Controller uses VolumeCloneSource Populator #2750

DataVolume Controller uses VolumeCloneSource Populator #2750

Conversation

mhenriks commented Jun 13, 2023 • edited Loading

mhenriks commented Jun 13, 2023

Choose a reason for hiding this comment

mhenriks commented Jun 13, 2023

alromeros left a comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

mhenriks commented Jun 16, 2023

ShellyKa13 left a comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

ShellyKa13 Jun 21, 2023 • edited Loading

Choose a reason for hiding this comment

Choose a reason for hiding this comment

mhenriks commented Jun 22, 2023

akalenyu commented Jun 22, 2023 • edited Loading

akalenyu left a comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

akalenyu commented Jun 22, 2023

mhenriks commented Jun 24, 2023

akalenyu commented Jun 25, 2023 • edited Loading

akalenyu commented Jun 26, 2023

mhenriks commented Jun 26, 2023

akalenyu commented Jun 27, 2023

mhenriks commented Jun 27, 2023

kubevirt-bot commented Jun 28, 2023 • edited Loading

mhenriks commented Jun 28, 2023

akalenyu left a comment

Choose a reason for hiding this comment

akalenyu commented Jun 28, 2023

kubevirt-bot commented Jun 28, 2023

ShellyKa13 left a comment

Choose a reason for hiding this comment

mhenriks commented Jun 30, 2023

kubevirt-bot commented Jun 30, 2023

mhenriks commented Jun 13, 2023 •

edited

Loading

ShellyKa13 Jun 21, 2023 •

edited

Loading

akalenyu commented Jun 22, 2023 •

edited

Loading

akalenyu commented Jun 25, 2023 •

edited

Loading

kubevirt-bot commented Jun 28, 2023 •

edited

Loading