From 4a4f5d8c962d923d7332bc807d2e03dca9fadc8f Mon Sep 17 00:00:00 2001 From: hehechen Date: Thu, 23 Feb 2023 16:55:06 +0800 Subject: [PATCH] doc: Add stale read design doc (#6867) ref pingcap/tiflash#4483 --- .../2023-02-21-tiflash-supports-stale-read.md | 31 +++++++++++++++++++ 1 file changed, 31 insertions(+) create mode 100644 docs/design/2023-02-21-tiflash-supports-stale-read.md diff --git a/docs/design/2023-02-21-tiflash-supports-stale-read.md b/docs/design/2023-02-21-tiflash-supports-stale-read.md new file mode 100644 index 00000000000..8bbcba98128 --- /dev/null +++ b/docs/design/2023-02-21-tiflash-supports-stale-read.md @@ -0,0 +1,31 @@ +# TiFlash supports stale read + +- Author: [hehechen](https://github.com/hehechen) + +## Introduction + +This proposal aims to make TiFlash support Stale Read. + +## Motivation or Background + +Currently TiKV has already supported [stale read](https://docs.pingcap.com/tidb/dev/stale-read), but TiFlash does not. +There are two benefits of TiFlash to support stale read: +- Reduces the read/wait index duration during reading. Small queries can run faster. +- Avoid calling read_index API of TiKV leaders, to reduce the impact of OLTP workloads. + +## Goals + +1. TiFlash supports stale read, including [AS OF TIMESTAMP](https://docs.pingcap.com/tidb/dev/as-of-timestamp), [TIDB_BOUNDED_STALENES](https://docs.pingcap.com/tidb/dev/as-of-timestamp#syntax), [tidb_read_staleness](https://docs.pingcap.com/tidb/dev/tidb-read-staleness), [tidb_external_ts](https://docs.pingcap.com/tidb/dev/tidb-external-ts). +2. Since TiFlash MPP may involve the computation among all TiFlash nodes, supporting [using stale read to read TiFlash data nearby](https://docs.pingcap.com/tidb/dev/three-dc-local-read) is not the goal of this RFC. + +## Design +1. Check the [safe ts](https://github.com/pingcap/tiflash/blob/e732eaba68e309a0aec0e443c7f1a0e9368731b3/dbms/src/Storages/Transaction/RegionTable.cpp#L508) before read_index, if the stale read requirements are met (safe ts > read tso), read_index can be skipped. +2. The assumption of the current MPP framework is that start_ts is globally unique and increasing in most cases. But in stale read scenarios, users can specify the timestamp, so the start_ts of different MPP queries issued by TiDB may be repeated. So the MPP framework needs to be modified. + Specifically, we need to use a globally unique query_id to replace start_ts as the identifier of MPP queries. The query_id consists of three parts of ids: + - The local time of TiDB when the query is initiated: query_ts. + - An id incremented in TiDB local memory: local_query_id. + - ETCD guarantees a globally unique serverID for each TiDB server: server_id. + + And The MinTSOScheduler will schedule MPP queries [in the order of query_ts](https://github.com/pingcap/tiflash/blob/e732eaba68e309a0aec0e443c7f1a0e9368731b3/dbms/src/Flash/Mpp/MPPTaskId.cpp#L27) in most cases. +1. TiDB client-go supports to get and cache the min_safe_ts from TiFlash stores. + client-go will query and cache min_safe_ts from all TiKV and TiFlash stores for all of their local regions periodically. When users use [TIDB_BOUNDED_STALENES](https://docs.pingcap.com/tidb/dev/as-of-timestamp#syntax) or [tidb_read_staleness](https://docs.pingcap.com/tidb/dev/tidb-read-staleness) to specify the stale read time range, TiDB will get min_safe_ts from the client-go cache, and generate the start_ts of the query based on min_safe_ts. \ No newline at end of file