Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

docs: add the design of Enhance Split Table Stability and Usability #29036

Closed
Changes from 1 commit
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
114 changes: 114 additions & 0 deletions docs/design/2021-10-22-Enhance-Split-Table-Stability.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,114 @@
# Proposal: Enhance Split Table Stability and Usability

- Author(s): [hzh](https://github.com/hzh0425)
- Discussion PR: https://github.com/tikv/pd/issues/4095
- Tracking Issue: https://github.com/pingcap/tidb/issues/29034

## Table of Contents

* [Introduction](#introduction)
* [Motivation or Background](#motivation-or-background)
* [Detailed Design](#detailed-design)
* [Test Design](#test-design)
* [Follow-up work](#Follow-up work)

## Introduction

Currently, the split table(regions) in tidb is constructed by following 2 steps:

1. Notify tikv to split regions
2. Notify pd to scatter regions

But this will let the time cost for the whole splitting table become unstable and rather time-consuming.
Yisaer marked this conversation as resolved.
Show resolved Hide resolved

This proposal aims to split and scatter regions in pd directly
Yisaer marked this conversation as resolved.
Show resolved Hide resolved

## Motivation or Background

Split table region : split a region into multi regions by split-keys
Yisaer marked this conversation as resolved.
Show resolved Hide resolved

Scatter table regions: regions are still in the original stores after split. Scatter is to load balance regions and evenly distribute regions to other stores
Yisaer marked this conversation as resolved.
Show resolved Hide resolved

However, before this proposal, TIDB and some tools were inconsistent when calling PD API -- split and scatter regions.

Our purpose is to add a new PD API -- SplitAndScatterRegions(), so that TIDB and other tools can easily call this API uniformly
Yisaer marked this conversation as resolved.
Show resolved Hide resolved

## Detailed Design

### kvproto: pdpb.proto

add a new interface SplitAndScatterRegions in pdpb.proto

```
rpc SplitAndScatterRegions(SplitAndScatterRegionsRequest) returns (SplitAndScatterRegionsResponse) {}
```

### PD: Grpc_Service, client.go

Implement SplitAndScatterRegions in grpc_service and client

grpc_service can directly call the original implementation splitRegions and scatterregions to achieve simultaneous split and scatter

client can

### TIDB

1.add new interface SplitAndScatterRegions in kv/kv.go/SplittableStore

```
SplitAndScatterRegions(ctx context.Context, splitKeys [][]byte, tableID *int64) (regionID []uint64, err error)
```

this interface will be implement by client-go

2.use new api SplitAndScatterRegions in executor/split.go

### client-go

Implement SplitAndScatterRegions in tikv/split_regions.go




Yisaer marked this conversation as resolved.
Show resolved Hide resolved

Yisaer marked this conversation as resolved.
Show resolved Hide resolved
## Test Design

I will test the stability and performance by controlling variables under different parameter conditions

For example:

- Split a empty table into 32 regions / 400 regions ....
- Split a table that contains data into 32 regions / 400 regions ....
- Split a table which dose not contain primary key into 32 regions / 400 regions ....
- ...............

I will record the time spent in these cases



Yisaer marked this conversation as resolved.
Show resolved Hide resolved
## Follow-up work

Enhance split region statement























Yisaer marked this conversation as resolved.
Show resolved Hide resolved