[Concurrent Segment Search] Change ConcurrentSearchTasksIT to use custom slice count

[jed326]

Now that #9107 and #8884 have been merged, we should fix ConcurrentSearchTasksIT to not hard code the segment count and instead validate against the customer slice count.

Ref: https://github.com/opensearch-project/OpenSearch/blob/main/server/src/internalClusterTest/java/org/opensearch/action/admin/cluster/node/tasks/ConcurrentSearchTasksIT.java#L111

[dipeshsingh253]

Hi @jed326 , I would like to work on this issue. can you provide me with some context here? I visited this file but it looked like a simple test only.

[jed326]

Hey @dipeshsingh253 , thanks for the interest! You can refer to https://opensearch.org/docs/latest/search-plugins/concurrent-segment-search/#slicing-mechanisms for an explanation of how "slicing" works for concurrent search.

In this issue specifically we would like to convert the test from using the hard coded slice calculation here (the left side of the assertion statement):

OpenSearch/server/src/internalClusterTest/java/org/opensearch/action/admin/cluster/node/tasks/ConcurrentSearchTasksIT.java

Lines 111 to 113 in fe98aad

	// Concurrent search forks each slice of 5 segments to different thread (see please
	// https://github.com/apache/lucene/issues/12498)
	assertEquals((int) Math.ceil(getSegmentCount(INDEX_NAME) / 5.0) + 1, threadStats.size());

And instead set the slice count via the cluster settings and then use that set value in the test assertion.

[dipeshsingh253]

Hi @jed326 , thanks for your assistance and this opportunity. Can you provide some resource on how exactly we are going to get slice count via the cluster settings ?

[jed326]

Hi @dipeshsingh253, the cluster settings API can be used to get and set cluster settings: https://opensearch.org/docs/latest/api-reference/cluster-api/cluster-settings/

The other link I provided above has details on the specific cluster setting to be used.

[dipeshsingh253]

Hi @jed326 , this was helpful but it still did not demonstrate how to use it in this project, for example :

assertEquals((int) Math.ceil(getSegmentCount(INDEX_NAME) / 5.0) + 1, threadStats.size()); 
//in above line we have to remove the static part and get something like below,
assertEquals((int) Math.ceil(getSliceCount(), threadStats.size()); 

so my doubt is how do we use Java code to set or get values in/from cluster setting? If you have any reference to such an example or resource, please point them here. Also if there is any existing file that uses the same sort of logic, please mention that as well.

Thanks in advance !!!

[rampreeth]

Hi @jed326 , can I pick this up?

[rampreeth]

Hi @jed326 , please let me know if this can be picked up. Thanks!

[jed326]

Hey @rampreeth please go ahead!