-
Notifications
You must be signed in to change notification settings - Fork 3.2k
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
feat: Search by name for WorkflowTemplates in UI #11684
feat: Search by name for WorkflowTemplates in UI #11684
Conversation
…plate Signed-off-by: shmruin <meme_hm@naver.com>
Signed-off-by: sunyeongchoi <sn0716@naver.com>
Signed-off-by: leeeuijoo <euijoo3233@gmail.com>
Signed-off-by: shmruin <meme_hm@naver.com>
Signed-off-by: sunyeongchoi <sn0716@naver.com>
unused namePattern props delete, change inputFilter name to camelcase Signed-off-by: sunyeongchoi <sn0716@naver.com>
Signed-off-by: sunyeongchoi <sn0716@naver.com>
Signed-off-by: sunyeongchoi <sn0716@naver.com>
4f11b7e
to
ddd63fd
Compare
Signed-off-by: sunyeongchoi <sn0716@naver.com>
ddd63fd
to
f387879
Compare
// Sort by resourceVersion desc | ||
sort.Slice(items, func(i, j int) bool { | ||
itemIRV, _ := strconv.Atoi(items[i].ResourceVersion) | ||
itemJRV, _ := strconv.Atoi(items[j].ResourceVersion) | ||
return itemIRV > itemJRV | ||
}) | ||
|
||
// Do resourceVersion filtering if continue exist | ||
if resourceVersion != "" { | ||
newItems := []v1alpha1.WorkflowTemplate{} | ||
for _, item := range items { | ||
targetRV, _ := strconv.Atoi(item.ResourceVersion) | ||
receivedRV, _ := strconv.Atoi(resourceVersion) | ||
if targetRV < receivedRV { | ||
newItems = append(newItems, item) | ||
} | ||
items = newItems | ||
} | ||
} | ||
|
||
// Indexing list by limit count | ||
if limit != 0 { | ||
endIndex := int(limit) | ||
if endIndex > len(items) || limit == 0 { | ||
endIndex = len(items) | ||
} | ||
wfList.Items = items[0:endIndex] | ||
} else { | ||
wfList.Items = items | ||
} | ||
|
||
// Calculate new offset for next batch | ||
if limit != 0 && len(wfList.Items) == int(limit) { | ||
wfList.ListMeta.Continue = wfList.Items[len(wfList.Items)-1].ResourceVersion | ||
} |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Can you add some explanation of this implementation in the comments?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
@terrytangyuan
Of course!
We have implemented namePattern filtering.
Since Kubernetes doesn't provide filtering like namePattern, we implemented logic to retrieve results from Kubernetes, then separately filter the results.
In the previous logic, we were utilizing the "continue" option for Kubernetes' built-in pagination.
However, when customizing Kubernetes' results, features like namePattern disrupt the proper usage of "continue" pagination.
Hence, we implemented our own pagination by utilizing Kubernetes' resourceVersion for cursor pagination.
- Sort in descending order based on resourceVersion.
- Filter only smaller values compared to the resourceVersion held from the previous page.
- Index the filtered results based on the limit.
- Store the resourceVersion for use in the next page.
The pagination logic is implemented in the sequence described above. Thank you.
Also I added a comment about the logic and committed it. e939bb8
Signed-off-by: sunyeongchoi <sn0716@naver.com>
// Search whole with Limit 0. | ||
// Reset the Continue "" to prevent Kubernetes native pagination. | ||
// Kubernetes api will search for all results without limit and pagination. | ||
k8sOptions.Continue = "" | ||
k8sOptions.Limit = 0 |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Would it be possible to stick with the native pagination when search functionality is not used?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I think maybe it is possible. I will try to implement it like that.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I uploaded commit about sticking with native pagination when search functionality is not used. 392222b
If this PR is reflected, we plan to implement the same namePattern function on other pages as well. I will consider having cursorPaginatonByResourceVersion
common in one place and one function.
…ement-filter-by-name Signed-off-by: sunyeongchoi <sn0716@naver.com>
…ot used Signed-off-by: sunyeongchoi <sn0716@naver.com>
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Thanks! Great work 👍
if req.NamePattern != "" { | ||
// Search whole with Limit 0. | ||
// Reset the Continue "" to prevent Kubernetes native pagination. | ||
// Kubernetes api will search for all results without limit and pagination. | ||
k8sOptions.Continue = "" | ||
k8sOptions.Limit = 0 |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
correct me if I'm wrong, but won't this have the same problem as #11761 / i.e. #12025 when there are enough WorkflowTemplates
? It looks like the exact problem was mentioned in #11004 (comment)
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
The workaround here may be to only retrieve the limit
at a time from the k8s API. If there aren't enough after the filter, then retrieve more.
That might make the request take longer (potentially significantly, but maybe less than retrieving all at once, which also takes time, and, crucially, memory), but would make sure we never retrieve too many (i.e. it throttles it, effectively). There are some other side-effects to that too (the k8s API rate limit, for instance).
The other way may be to do this fully client-side or partially client-side.
Fully would be the UI (and CLI? a flag wasn't implemented here though) fetches and does a filter, and then if it doesn't have enough, requests more.
Partially would be the API does a filter but can underfetch (it only filters on the limit
), in which case the UI has to request more by moving up the cursor.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I'm also thinking we may want to explicitly point out in the UI which search fields are not k8s native and can therefore be non-performant when used; basically a "hey we implemented this, but you might want to be careful using this".
and at that point, potentially let admins disable them with flags too, if they don't want users to overload the Argo Server or k8s API accidentally.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
For future reference, the current pattern to mitigate this would be to follow the SQLite pattern here a la #12736, see also #13633 (comment) et al
k8s.io.apimachinery.pkg.apis.meta.v1.ListOptions listOptions = 2; | ||
string namePattern = 2; | ||
k8s.io.apimachinery.pkg.apis.meta.v1.ListOptions listOptions = 3; |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
changing the tag numbers is apparently a breaking change in protobufs, see #13160 (comment) for more details.
/> | ||
</div> | ||
<div className='columns small-2 xlarge-12'> | ||
<p className='wf-filters-container__title'>Name Pattern</p> |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
"Pattern" is a bit ambiguous as it can mean several things. "Contains" is a more explicit naming. See #13160 (comment) for more details
Fixes #11004
Motivation
It was implemented because the function to filter the result of the workflow template by name was considered necessary.
If the corresponding PR is merged, the same will be applied to other resource pages.
Modifications
Verification