Cluster manager task throttling [DOC]

[dhwanilpatel]

What do you want to do?

• Request a change to existing documentation
• Add new documentation
• Report a technical problem with the documentation
• Other

Tell us about your request. Provide a summary of the request and all versions that are affected.

Give a brief description of your request.
We have introduced new settings for cluster manager task throttling. User can define throttling limit for task types in dynamic settings. Cluster manager node will throttle new task submission from data node if total task of given type in pending task queue breaches the threshold limit. Data node will internally perform retries on it.

Setting format:

{
    "persistent": {
        "cluster_manager.throttling.thresholds" : {
            "<task-type-name>" : {
                "value" : <threshold limit>
            }
        }
    }
}

Sample e.g

{
    "persistent": {
        "cluster_manager.throttling.thresholds" : {
            "put-mapping" : {
                "value" : "100"
            }
        }
    }
}

Set threshold limit to -1 to disable the throttling on given task type.

Default value of these new settings are -1 i.e. disabled throttling for task types.

Task types supported for throttling framework:

create-index, update-settings, cluster-update-settings, auto-create, delete-index, delete-dangling-index, create-data-stream, remove-data-stream, rollover-index, index-aliases, put-mapping, create-index-template, remove-index-template, create-component-template, remove-component-template, create-index-template-v2, remove-index-template-v2, put-pipeline, delete-pipeline, create-persistent-task, finish-persistent-task, remove-persistent-task, update-task-state, put-script, delete-script, put-repository, delete-repository, create-snapshot, delete-snapshot, update-snapshot-state, restore-snapshot, cluster-reroute-api

Which versions are affected?
2.4.0

What other resources are available? Provide links to related issues, POCs, steps for testing, etc.

Github Issue
#479

PRs

• Basic Throttler Framework / Exponential Basic back off policy. ( Add basic thorttler/exponential backoff policy for retry/Defination o… OpenSearch#3527 )
• Changes required in Master node to perform throttling.( Master node changes for master task throttling OpenSearch#3882)
• Changes required in Data node to perform retry on throttling.( Data node changes for master task throttling OpenSearch#4204 )
• Provide support for all task type in throttling framework.( Onboarding of few task types to throttling OpenSearch#4542 )
• Integration Tests (Fix timeout exception and Add Integ test for Master task throttling OpenSearch#4588 )

Point of Contact for technical question
dhwanip / ruhelag