[APM] Investigate support for async search

[sorenlouv]

Async search is a new capability in elasticsearch that improves long/slow running queries.

It is not uncommon for APM users to have very large datasets that causes the ui to be unresponsive for several seconds or even timeout completely. This is a really bad user experience and async search might be able to remedy this.

Requirements for adding async search:

• Elasticsearch request should call /_async_search instead of /search
• Requests made by the client should add polling support
• useFetcher needs to have a new property isPartial to indicate in the UI whether the currently displayed content is fully loaded or partially loaded.

Investigate: We might be able to get the first two items on the list for free if we migrate to Kibana Search Service.

[elasticmachine]

Pinging @elastic/apm-ui (Team:apm)

[dgieselaar]

Some qs:

• what happens with ui indices being fetched from saved objects?
• some, maybe most of our API calls consist of multiple requests to ES. How do we make that work?
• can you clarify the purpose of polling?

[sorenlouv]

what happens with ui indices being fetched from saved objects?

This wouldn't be fetched with async search, if that's what you mean?

some, maybe most of our API calls consist of multiple requests to ES. How do we make that work?

Really good point. That could make things a more tricky - we would have to keep track of multiple long running queries per API endpoint.

can you clarify the purpose of polling?

If a query takes 20s to return the entire response, the user currently has to wait for the full 20s before seeing anything. With async search we can poll for the state of the query and get partial results as they become available, eg. every second.

LMK if that makes sense

[dgieselaar]

This wouldn't be fetched with async search, if that's what you mean?

Right, I see async search only requires an ID so that should be fine.

Really good point. That could make things a more tricky - we would have to keep track of multiple long running queries per API endpoint.

Yeah - I'm having a hard time seeing how we can reasonably do this (without having something like materialized views).

If a query takes 20s to return the entire response, the user currently has to wait for the full 20s before seeing anything. With async search we can poll for the state of the query and get partial results as they become available, eg. every second.

👍🏻 I guess this could work if we compose our queries w/ observables and stream data into it, and use something like server-sent events or websockets to push new data to the browser. I don't really see an easy way to do this though, do you?

[sorenlouv]

I don't really see an easy way to do this though, do you?

Good old long polling also works, doesn't it?
I'm not against WS or SSE but I'm not 100% it'll work in the diverse environments where Kibana is used (behind proxies that might terminate/not support anything but simple http requests).

[dgieselaar]

SSE will work AFAIK. Long polling too. But the bigger change would be observables. We could make a client.search$ function that returns an observable that updates its value as more results come in. Partial results might be worth it, I'm not so sure about the "come back later when your dashboard has loaded" thing. That would require us to store request state somewhere.

[sorenlouv]

But the bigger change would be observables.

Observables make a lot of sense if we use SSE or WS. Not so much for long polling since this is initiated from the client.

Partial results might be worth it, I'm not so sure about the "come back later when your dashboard has loaded" thing. That would require us to store request state somewhere.

yes, I think for APM partial data is the main value proposition of async search.

[dgieselaar]

Observables make a lot of sense if we use SSE or WS. Not so much for long polling since this is initiated from the client.

How do you see this working without something like Observables? I'm not sure what the difference would be server-side whether we use SSE/WS or long polling (other than the protocal), can you clarify?

[sorenlouv]

With long polling the server will be stateless and each request from the client will contain the es request key.

With web sockets or SSE the server would be stateful and periodically read the progress of the ES request and send the events to the client as they are available.

In the latter case I think observables make a lot of sense. In the former not so much.

[dgieselaar]

@sqren ah yeah, I mistook long polling for streaming responses. I have a hard time seeing if the request id approach will work out, but I'll gladly be proven wrong 😃

[sorenlouv]

I have a hard time seeing if the request id approach will work out, but I'll gladly be proven wrong

Why wouldn't it work? afaict from the docs that is the recommended way to retrieve a running search instance.