separate tikv and tiflash tombstone stores (#2472)

[sre-bot]

cherry-pick #2472 to release-1.1

---

What problem does this PR solve?

fix #2458

What is changed and how does it work?

Separate the tikv and tiflash tombstone stores

Check List

Tests

• Manual test (add detailed scripts or steps below)
  • Delete some tikv and tiflash stores, check the tidbcluster status that status.tikv.tombstoneStores only includes tikv tombstone stores and status.tiflash.tombstoneStores only includes tiflash tombstone stores

  tiflash:
    image: pingcap/tiflash:v3.1.0-rc
    phase: Normal
    statefulSet:
      collisionCount: 0
      currentReplicas: 5
      currentRevision: test310-tiflash-64684cfbdb
      observedGeneration: 9
      readyReplicas: 5
      replicas: 5
      updateRevision: test310-tiflash-64684cfbdb
      updatedReplicas: 5
    stores:
      "157":
        id: "157"
        ip: test310-tiflash-1.test310-tiflash-peer.test310.svc
        lastHeartbeatTime: "2020-05-14T13:04:07Z"
        lastTransitionTime: "2020-05-14T11:11:40Z"
        leaderCount: 0
        podName: test310-tiflash-1
        state: Up
      "1026":
        id: "1026"
        ip: test310-tiflash-2.test310-tiflash-peer.test310.svc
        lastHeartbeatTime: "2020-05-14T13:04:11Z"
        lastTransitionTime: "2020-05-14T09:24:42Z"
        leaderCount: 0
        podName: test310-tiflash-2
        state: Up
      "1029":
        id: "1029"
        ip: test310-tiflash-3.test310-tiflash-peer.test310.svc
        lastHeartbeatTime: "2020-05-14T13:04:11Z"
        lastTransitionTime: "2020-05-14T06:44:04Z"
        leaderCount: 0
        podName: test310-tiflash-3
        state: Up
      "1030":
        id: "1030"
        ip: test310-tiflash-4.test310-tiflash-peer.test310.svc
        lastHeartbeatTime: "2020-05-14T13:04:07Z"
        lastTransitionTime: "2020-05-14T07:00:47Z"
        leaderCount: 0
        podName: test310-tiflash-4
        state: Up
    synced: true
    tombstoneStores:
      "152":
        id: "152"
        ip: test310-tiflash-0.test310-tiflash-peer.test310.svc
        lastHeartbeatTime: null
        lastTransitionTime: null
        leaderCount: 0
        podName: test310-tiflash-0
        state: Tombstone
      "1034":
        id: "1034"
        ip: test310-tiflash-5.test310-tiflash-peer.test310.svc
        lastHeartbeatTime: null
        lastTransitionTime: null
        leaderCount: 0
        podName: test310-tiflash-5
        state: Tombstone
  tikv:
    image: pingcap/tikv:v3.1.0-rc
    phase: Upgrade
    statefulSet:
      collisionCount: 0
      currentReplicas: 1
      currentRevision: test310-tikv-bcbb454bc
      observedGeneration: 2
      readyReplicas: 4
      replicas: 4
      updateRevision: test310-tikv-bcbb454bc
      updatedReplicas: 1
    stores:
      "1":
        id: "1"
        ip: test310-tikv-1.test310-tikv-peer.test310.svc
        lastHeartbeatTime: "2020-05-14T13:04:05Z"
        lastTransitionTime: "2020-05-14T09:27:20Z"
        leaderCount: 20
        podName: test310-tikv-1
        state: Up
      "5":
        id: "5"
        ip: test310-tikv-2.test310-tikv-peer.test310.svc
        lastHeartbeatTime: "2020-05-14T13:04:05Z"
        lastTransitionTime: "2020-05-14T09:27:20Z"
        leaderCount: 15
        podName: test310-tikv-2
        state: Up
      "1045":
        id: "1045"
        ip: test310-tikv-3.test310-tikv-peer.test310.svc
        lastHeartbeatTime: "2020-05-14T13:04:03Z"
        lastTransitionTime: "2020-05-14T09:49:00Z"
        leaderCount: 11
        podName: test310-tikv-3
        state: Up
    synced: true
    tombstoneStores:
      "4":
        id: "4"
        ip: test310-tikv-0.test310-tikv-peer.test310.svc
        lastHeartbeatTime: "2020-05-14T11:47:58Z"
        lastTransitionTime: null
        leaderCount: 0
        podName: test310-tikv-0
        state: Tombstone

Code changes

• Has Go code change

Related changes

• Need to cherry-pick to the release branch

Does this PR introduce a user-facing change?:

NONE

[Yisaer]

/merge

[sre-bot]

/run-all-tests

[DanielZhangQD]

LGTM

[DanielZhangQD]

/merge