Allow backgrounding complex kibana search tasks RFC

[ppisljar]

With search sessions its possible to send a set of searches to background and restore them at a later time (when they are complete). This only works for simple searches against elasticsearch and doesn't work for the cases when at the time of storing the session not all searches have started.

This can happen in many places, some of them include:

• expressions can be complex, there could be multiple functions doing searches.
• other bucket in search source as well as histogram (for retrieving min/max)
• maps does multiple searchsource requests
• Discover field statics sends a lot of follow up requests: [Discover][Field statistics] Number of searches in SearchSession Management #123027

In order to support that we would need to allow backgrounding complex tasks:

• the task needs to be able to run on the server. expressions and searchsource are both supported on the server
• session would need to hold complex task configs (for example the expression and its input)
• we would need to run the complex tasks on the server at a later point in time on behalf of the user
• once the complex tasks completes we know we collected all the search ids and we can notify the user about completion

Related:

• implement searchsource as a search strategy #111417
• implement expressions search strategy #111423

[elasticmachine]

Pinging @elastic/kibana-app-services (Team:AppServices)

[elastic-jb]

@ppisljar can you help me understand how this manifests itself to users right now? Can you describe a scenario to explain the limitation this would address?

[ppisljar]

searchsource can do multiple subrequests (for histogram, figuring out min and max and for terms other bucket), maps does multiple searchsource requests to get the data, inside an expression you can do any number of requests. none of this can be correctly backgrounded right now and this will allow for backgrounding any of those. (in context of backgrounding search sessions)

[ppisljar]

possibly blocked by: #111418

[Dosant]

Added new use-case:

• [Discover][Field statistics] Number of searches in SearchSession Management #123027

cc @kertal

[Dosant]

I think this would also block making SQL search strategy (x-pack/plugins/canvas/server/lib/essql_strategy.ts) to use async search because internally it scrolls through the results:

kibana/x-pack/plugins/canvas/server/lib/essql_strategy.ts

Lines 62 to 63 in 28eb46d

	while (rows.length < count && body.cursor !== undefined) {
	response = await esClient.asCurrentUser.sql.query({

Alternatively, maybe elasticsearch could remove a limitation of index.max_result_window on SQL search results when an async search is used?

[elasticmachine]

Pinging @elastic/kibana-data-discovery (Team:DataDiscovery)

[kertal]

Closing this because it's not planned to be resolved in the foreseeable future. It will be tracked in our Icebox and will be re-opened if our priorities change. Feel free to re-open if you think it should be melted sooner.