Feature Request: Check the first region's replication state before starting tidb-server

[disksing]

Feature Request

Is your feature request related to a problem? Please describe:

When initializing the cluster, PD/TiKV needs to spend some time to add replicas to the first region. If tidb-server is started before finish adding replica, the data may be written to single replica. The issue was initially reported from jepsen test.

Describe the feature you'd like:

There was a new flag in PD to indicate whether the first region is fully replicated. The API was introduced by tikv/pd#1555 (scheduled to release in 3.0.0-rc.3) and there is a PR in tidb-ansible as a usage example pingcap/tidb-ansible#774

Describe alternatives you've considered:

It can also be done by calling the /regions API, which I think is less straight forward.

Teachability, Documentation, Adoption, Migration Strategy:

To make it compatible with old version, when the response has no is_initialized field, the check step can be skipped. (tidb-ansible strictly checks it because it has version control scheme to avoid run with old version)

[tennix]

@disksing What if deploying only one TiKV pod which is now possible with tidb-operator?

[disksing]

@tennix You are right about the compatibility problem. As suggested in pingcap/tidb-ansible#774, if we want start a cluster with only 1 tikv-server, we should either set max-replicas to 1, or disable the check initialized feature.

[github-actions]

This issue is stale because it has been open 60 days with no activity. Remove stale label or comment or this will be closed in 15 days

[disksing]

any update?