Make GetIndexAction cancellable

[DaveCTurner]

The get-indices API does some nontrivial work on the master and at high
index counts the response may be very large and could take a long time
to compute. Some clients will time out and retry if it takes too long.
Today this API is not properly cancellable which leads to a good deal of
wasted work in this situation, and the potentially-enormous response is
serialized on a transport worker thread.

With this commit we make the API cancellable and move the serialization
to a MANAGEMENT thread.

Relates #77466

[elasticmachine]

Pinging @elastic/es-data-management (Team:Data Management)

[elasticsearchmachine]

Hi @DaveCTurner, I've created a changelog YAML for you.

[DaveCTurner]

Backporting this to 7.17 is debatable, although I think it's reasonable to do so.

[martijnvg]

LGTM, thanks David.
I wonder what other APIs we may need to make this change for?
(currently: cluster state, get snapshot, get mapping, segment stats APIs?)

[elasticsearchmachine]

💔 Backport failed

Status	Branch	Result
❌	7.17	Commit could not be cherrypicked due to conflicts
❌	8.3	Commit could not be cherrypicked due to conflicts

You can use sqren/backport to manually backport by running backport --upstream elastic/elasticsearch --pr 87681

[DaveCTurner]

Thanks Martijn.

I wonder what other APIs we may need to make this change for?

IMO all of them, at least it'd be great for cancellation to be an opt-out thing rather than opt-in. Unfortunately it's a bit hard to determine where to focus our efforts without evidence from users showing that cancellation support is needed, which is what happened here. I count 28 APIs that already support cancellation, which is a good start.

[martijnvg]

Backporting this to 7.17 is debatable, although I think it's reasonable to do so.

I forgot to mention, but yes, I think this change also qualifies for backporting to 7.17.

IMO all of them, at least it'd be great for cancellation to be an opt-out thing rather than opt-in.

+1

Unfortunately it's a bit hard to determine where to focus our efforts without evidence from users showing that cancellation support is needed, which is what happened here. I count 28 APIs that already support cancellation, which is a good start.

YMaybe we should have a meta issue for this. I will ask around whether this is a good idea.