[Segment Replication] Cluster level setting to restrict replication type in index level settings

[dreamer-89]

Summary

Today, we have cluster.indices.replication.strategy cluster setting that sets the default replication type for any index in the cluster which can be overridden by index level settings i.e. index.replication.type value. As part of this change, we need to ensure the cluster level settings are not overridden by index level settings. This will be useful for admins to provide a more strict control on index replication type where they set cluster level replication type using cluster.indices.replication.strategy settings and allows users/consumers to create their indices. We attempted this previously but it has a bug where snapshot restore functionality was broken. Proposed new setting name is cluster.index.restrict.replication.type.

Describe the solution you'd like

Introduce a new cluster setting would help enforce a replication strategy and prevent override by the index level settings. The new settings will be opt-in which means it will false by default.

Related

1. Introduce a cluster setting that would help restrict replication type in a cluster #10868
2. [BUG] Need to revert cluster.restrict.index.replication-type setting #11078

[dreamer-89]

We should allow index.replication.type setting when its value matches with cluster level replication type i.e. cluster.indices.replication.strategy and throw exception ONLY when the two settings are different. Also, there are operations which results in new index creation and we should block all those operations (listed below).

New indices passing in explicit index.replication.type

New indices creation using replication.type which mis-matches with cluster level replication type should not be allowed.

New indices using Index templates and composbale templates

Similar to above, new index creations where index attains replication type setting from templates should not be allowed when replication.type setting from template does not match with cluster level setting.

New indices via data streams (backup index creation and rollover)

Data streams create automatic backup writeable index pulling in settings from index templates. The new index behavior creation should be similar to above.

Snapshot restore

SImilar to above, the snapshot restore should not be allowed when restored index has mis-matching setting from cluster level replication type setting.

Shrink, Split, Clone

Similar to above, these operations should not be allowed when target index has explicit index replication type setting which does not matches with cluster level.

Tagging @dagneyb @elfisher for feedback on above intended behavior.

CC @anasalkouz @mch2

[dreamer-89]

Ping: @dagneyb @elfisher to get feedback on above expected behavior.

[rohin]

I agree with the suggestion.

I only have a follow up question whether we considered a cluster level setting that can be used to check if override is allowed or not, this way we can ignore index level changes always and it might not be restricted to just replication but used wider for better cluster control.

like - cluster.indices.replication.allowoverride or something of the likes.

[dreamer-89]

I agree with the suggestion.

I only have a follow up question whether we considered a cluster level setting that can be used to check if override is allowed or not, this way we can ignore index level changes always and it might not be restricted to just replication but used wider for better cluster control.

like - cluster.indices.replication.allowoverride or something of the likes.

Thanks @rohin for the feedback. I really like the idea of having a unique cluster level setting that governs index level overrides are allowed or not. On thinking more on this proposal, I see below challenges.

1. Single setting favours simplicity though at the expense of granular control. Users may like to have different levers for different features/restrictions.
2. Capping all (or multiple) index level override settings for features which are not related, under a single cluster level setting may be confusing for the users ? I am taking into consideration future changes where we add more restrictions for different aspects of an index.

What are your thoughts on this ?

[dreamer-89]

Closing this issue as change has been implemented.