Force execution of delete index requests

[s1monw]

The management threadpool is now bouned and can reject executions.
We should make sure that delete index operations are forced on the
threadpool.

[s1monw]

@martijnvg / @kimchy can you take a look

[martijnvg]

LGTM. I wonder if we should also do this in MetaDataDeleteIndexService#createIndex() ?

[s1monw]

I wonder if we should also do this in MetaDataDeleteIndexService#createIndex()

so I didn't do this since my rational behind this change is that we should clear resources even if we are under pressure since it might be the source of the pressure. Yet this is not true for creating resources so I think we shouldn't do that for the create case?

[kimchy]

@s1monw I am concerned that just queuing will not help much? if there is some reason that the server is under load and can't process metadata requests, and indices are not the reason for it (network, something else), then we get back to the original reason of why the management TP because bounded in the first place. With force queue of the delete request, then we will just keep on increasing the queue?

[s1monw]

like the reason for an overloaded management queue is too many mapping updates. I think we should force the execution of the delete calls here since they free resources. if you find a better way to make the delete index reliable and not rejected just because somebody goes wild with dynamic mappings is fine with me too :)

[s1monw]

we moved back to a scaling threadpool here so I am closing this since it's invalid