Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

system-variables: Merge with tidb-specific-sysvars (#3152) #3261

Merged
merged 4 commits into from
Jul 14, 2020
Merged
Show file tree
Hide file tree
Changes from 3 commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
4 changes: 1 addition & 3 deletions TOC.md
Original file line number Diff line number Diff line change
Expand Up @@ -412,9 +412,7 @@
+ [tikv-server](/tikv-configuration-file.md)
+ [tiflash-server](/tiflash/tiflash-configuration.md)
+ [pd-server](/pd-configuration-file.md)
+ System Variables
+ [MySQL System Variables](/system-variables.md)
+ [TiDB Specific System Variables](/tidb-specific-system-variables.md)
+ [System Variables](/system-variables.md)
+ Storage Engines
+ TiKV
+ [TiKV Overview](/tikv-overview.md)
Expand Down
2 changes: 1 addition & 1 deletion best-practices/high-concurrency-best-practices.md
Original file line number Diff line number Diff line change
Expand Up @@ -181,7 +181,7 @@ In this case, the table is simple. In other cases, you might also need to consid

**Problem one:**

If a table does not have a primary key, or the primary key is not the `Int` type and you do not want to generate a randomly distributed primary key ID, TiDB provides an implicit `_tidb_rowid` column as the row ID. Generally, when you do not use the `SHARD_ROW_ID_BITS` parameter, the values of the `_tidb_rowid` column are also monotonically increasing, which might causes hotspots too. Refer to [`SHARD_ROW_ID_BITS` description](/tidb-specific-system-variables.md#shard_row_id_bits) for more details.
If a table does not have a primary key, or the primary key is not the `Int` type and you do not want to generate a randomly distributed primary key ID, TiDB provides an implicit `_tidb_rowid` column as the row ID. Generally, when you do not use the `SHARD_ROW_ID_BITS` parameter, the values of the `_tidb_rowid` column are also monotonically increasing, which might causes hotspots too. Refer to [`SHARD_ROW_ID_BITS`](/sql-statements/sql-statement-create-table.md#shard_row_id_bits) for more details.

To avoid the hotspot problem in this situation, you can use `SHARD_ROW_ID_BITS` and `PRE_SPLIT_REGIONS` when creating a table. For more details about `PRE_SPLIT_REGIONS`, refer to [Pre-split Regions](/sql-statements/sql-statement-split-region.md#pre_split_regions).

Expand Down
2 changes: 1 addition & 1 deletion dashboard/dashboard-overview.md
Original file line number Diff line number Diff line change
Expand Up @@ -60,7 +60,7 @@ By default, this area shows the latest 10 slow queries in the entire cluster ove

![Recent slow queries](/media/dashboard/dashboard-overview-slow-query.png)

By default, the SQL query that is executed longer than 300 milliseconds is counted as a slow query and displayed on the table. You can change this threshold by modifying the [tidb_slow_log_threshold](/tidb-specific-system-variables.md#tidb_slow_log_threshold) variable or the [slow-threshold](/tidb-configuration-file.md#slow-threshold) TiDB parameter.
By default, the SQL query that is executed longer than 300 milliseconds is counted as a slow query and displayed on the table. You can change this threshold by modifying the [tidb_slow_log_threshold](/system-variables.md#tidb_slow_log_threshold) variable or the [slow-threshold](/tidb-configuration-file.md#slow-threshold) TiDB parameter.

The content displayed in this area is consistent with the more detailed [Slow Queries Page](/dashboard/dashboard-slow-query.md). You can click the **Recent Slow Queries** title to view the complete list. For details of the columns in this table, see this [Slow Queries Page](/dashboard/dashboard-slow-query.md).

Expand Down
2 changes: 1 addition & 1 deletion dashboard/dashboard-slow-query.md
Original file line number Diff line number Diff line change
Expand Up @@ -9,7 +9,7 @@ aliases: ['/docs/stable/dashboard/dashboard-slow-query/','/docs/v4.0/dashboard/d

On the Slow Queries page of TiDB Dashboard, you can search and view all slow queries in the cluster.

By default, SQL queries with an execution time of more than 300 milliseconds are considered as slow queries. These queries are recorded in the [slow query logs](/identify-slow-queries.md) and can be searched via TiDB Dashboard. You can adjust the threshold of slow queries through the [`tidb_slow_log_threshold`](/tidb-specific-system-variables.md#tidb_slow_log_threshold) session variable or the [`slow-threshold`](/tidb-configuration-file.md#slow-threshold) TiDB parameter.
By default, SQL queries with an execution time of more than 300 milliseconds are considered as slow queries. These queries are recorded in the [slow query logs](/identify-slow-queries.md) and can be searched via TiDB Dashboard. You can adjust the threshold of slow queries through the [`tidb_slow_log_threshold`](/system-variables.md#tidb_slow_log_threshold) session variable or the [`slow-threshold`](/tidb-configuration-file.md#slow-threshold) TiDB parameter.

> **Note:**
>
Expand Down
6 changes: 3 additions & 3 deletions faq/tidb-faq.md
Original file line number Diff line number Diff line change
Expand Up @@ -561,7 +561,7 @@ When TiDB is executing a SQL statement, the query will be `EXPENSIVE_QUERY` if e

#### How to control or change the execution priority of SQL commits?

TiDB supports changing the priority on a [per-session](/tidb-specific-system-variables.md#tidb_force_priority), [global](/tidb-configuration-file.md#force-priority) or individual statement basis. Priority has the following meaning:
TiDB supports changing the priority on a [per-session](/system-variables.md#tidb_force_priority), [global](/tidb-configuration-file.md#force-priority) or individual statement basis. Priority has the following meaning:

- `HIGH_PRIORITY`: this statement has a high priority, that is, TiDB gives priority to this statement and executes it first.

Expand Down Expand Up @@ -1024,7 +1024,7 @@ See [Introduction to Statistics](/statistics.md).

#### How to optimize `select count(1)`?

The `count(1)` statement counts the total number of rows in a table. Improving the degree of concurrency can significantly improve the speed. To modify the concurrency, refer to the [document](/tidb-specific-system-variables.md#tidb_distsql_scan_concurrency). But it also depends on the CPU and I/O resources. TiDB accesses TiKV in every query. When the amount of data is small, all MySQL is in memory, and TiDB needs to conduct a network access.
The `count(1)` statement counts the total number of rows in a table. Improving the degree of concurrency can significantly improve the speed. To modify the concurrency, refer to the [document](/system-variables.md#tidb_distsql_scan_concurrency). But it also depends on the CPU and I/O resources. TiDB accesses TiKV in every query. When the amount of data is small, all MySQL is in memory, and TiDB needs to conduct a network access.

Recommendations:

Expand Down Expand Up @@ -1089,7 +1089,7 @@ See [The TiDB Command Options](/command-line-flags-for-tidb-configuration.md).

#### How to scatter the hotspots?

In TiDB, data is divided into Regions for management. Generally, the TiDB hotspot means the Read/Write hotspot in a Region. In TiDB, for the table whose primary key (PK) is not an integer or which has no PK, you can properly break Regions by configuring `SHARD_ROW_ID_BITS` to scatter the Region hotspots. For details, see the introduction of `SHARD_ROW_ID_BITS` in [TiDB Specific System Variables and Syntax](/tidb-specific-system-variables.md#shard_row_id_bits).
In TiDB, data is divided into Regions for management. Generally, the TiDB hotspot means the Read/Write hotspot in a Region. In TiDB, for the table whose primary key (PK) is not an integer or which has no PK, you can properly break Regions by configuring `SHARD_ROW_ID_BITS` to scatter the Region hotspots. For details, see the introduction of `SHARD_ROW_ID_BITS` in [`CREATE TABLE`](/sql-statements/sql-statement-create-table.md#shard_row_id_bits).

### TiKV

Expand Down
2 changes: 1 addition & 1 deletion identify-expensive-queries.md
Original file line number Diff line number Diff line change
Expand Up @@ -6,7 +6,7 @@ aliases: ['/docs/stable/identify-expensive-queries/','/docs/v4.0/identify-expens

# Identify Expensive Queries

TiDB allows you to identify expensive queries during SQL execution, so you can diagnose and improve the performance of SQL execution. Specifically, TiDB prints the information about statements whose execution time exceeds [`tidb_expensive_query_time_threshold`](/tidb-specific-system-variables.md#tidb_expensive_query_time_threshold) (60 seconds by default) or memory usage exceeds [`mem-quota-query`](/tidb-configuration-file.md#mem-quota-query) (1 GB by default) to the [tidb-server log file](/tidb-configuration-file.md#logfile) ("tidb.log" by default).
TiDB allows you to identify expensive queries during SQL execution, so you can diagnose and improve the performance of SQL execution. Specifically, TiDB prints the information about statements whose execution time exceeds [`tidb_expensive_query_time_threshold`](/system-variables.md#tidb_expensive_query_time_threshold) (60 seconds by default) or memory usage exceeds [`mem-quota-query`](/tidb-configuration-file.md#mem-quota-query) (1 GB by default) to the [tidb-server log file](/tidb-configuration-file.md#logfile) ("tidb.log" by default).

> **Note:**
>
Expand Down
2 changes: 1 addition & 1 deletion partitioned-table.md
Original file line number Diff line number Diff line change
Expand Up @@ -1052,4 +1052,4 @@ select * from t;

The `tidb_enable_table_partition` environment variable controls whether to enable the partitioned table feature. If this variable is set to `off`, the partition information will be ignored when a table is created, and this table will be created as a normal table.

This variable is only used in table creation. After the table is created, modify this variable value takes no effect. For details, see [TiDB specific system variables](/tidb-specific-system-variables.md#tidb_enable_table_partition).
This variable is only used in table creation. After the table is created, modify this variable value takes no effect. For details, see [system variables](/system-variables.md#tidb_enable_table_partition).
2 changes: 1 addition & 1 deletion pessimistic-transaction.md
Original file line number Diff line number Diff line change
Expand Up @@ -15,7 +15,7 @@ To make the usage of TiDB closer to traditional databases and reduce the cost of

## Switch transaction mode

You can set the transaction mode by configuring the [`tidb_txn_mode`](/tidb-specific-system-variables.md#tidb_txn_mode) system variable. The following command sets all explicit transactions (that is, non-autocommit transactions) executed by newly created sessions in the cluster to the pessimistic transaction mode:
You can set the transaction mode by configuring the [`tidb_txn_mode`](/system-variables.md#tidb_txn_mode) system variable. The following command sets all explicit transactions (that is, non-autocommit transactions) executed by newly created sessions in the cluster to the pessimistic transaction mode:

{{< copyable "sql" >}}

Expand Down
8 changes: 4 additions & 4 deletions releases/release-2.1-ga.md
Original file line number Diff line number Diff line change
Expand Up @@ -92,15 +92,15 @@ On November 30, 2018, TiDB 2.1 GA is released. See the following updates in this

- [Add the `auto_analyze_ratio` system variables to contorl the ratio of Analyze](/faq/tidb-faq.md#whats-the-trigger-strategy-for-auto-analyze-in-tidb)

- [Add the `tidb_retry_limit` system variable to control the automatic retry times of transactions](/tidb-specific-system-variables.md#tidb_retry_limit)
- [Add the `tidb_retry_limit` system variable to control the automatic retry times of transactions](/system-variables.md#tidb_retry_limit)

- [Add the `tidb_disable_txn_auto_retry` system variable to control whether the transaction retries automatically](/tidb-specific-system-variables.md#tidb_disable_txn_auto_retry)
- [Add the `tidb_disable_txn_auto_retry` system variable to control whether the transaction retries automatically](/system-variables.md#tidb_disable_txn_auto_retry)

- [Support using`admin show slow` statement to obtain the slow queries](/identify-slow-queries.md#admin-show-slow-command)

- [Add the `tidb_slow_log_threshold` environment variable to set the threshold of slow log automatically](/tidb-specific-system-variables.md#tidb_slow_log_threshold)
- [Add the `tidb_slow_log_threshold` environment variable to set the threshold of slow log automatically](/system-variables.md#tidb_slow_log_threshold)

- [Add the `tidb_query_log_max_len` environment variable to set the length of the SQL statement to be truncated in the log dynamically](/tidb-specific-system-variables.md#tidb_query_log_max_len)
- [Add the `tidb_query_log_max_len` environment variable to set the length of the SQL statement to be truncated in the log dynamically](/system-variables.md#tidb_query_log_max_len)

+ DDL

Expand Down
2 changes: 1 addition & 1 deletion releases/release-2.1.18.md
Original file line number Diff line number Diff line change
Expand Up @@ -49,7 +49,7 @@ TiDB Ansible version: 2.1.18
- Fix the issue that the `COM_STMT_FETCH` time record in slow query logs is inconsistent with that in MySQL [#12953](https://github.com/pingcap/tidb/pull/12953)
- Add an error code in the error message for write conflicts to quickly locate the cause [#12878](https://github.com/pingcap/tidb/pull/12878)
+ DDL
- Disallow dropping the `AUTO INCREMENT` attribute of a column by default. Modify the value of the `tidb_allow_remove_auto_inc` variable if you do need to drop this attribute. See [TiDB Specific System Variables](/tidb-specific-system-variables.md#tidb_allow_remove_auto_inc-new-in-v2118-and-v304) for more details. [#12146](https://github.com/pingcap/tidb/pull/12146)
- Disallow dropping the `AUTO INCREMENT` attribute of a column by default. Modify the value of the `tidb_allow_remove_auto_inc` variable if you do need to drop this attribute. See [System Variables](/system-variables.md#tidb_allow_remove_auto_inc-new-in-v2118-and-v304) for more details. [#12146](https://github.com/pingcap/tidb/pull/12146)
- Support multiple `unique`s when creating a unique index in the `Create Table` statement [#12469](https://github.com/pingcap/tidb/pull/12469)
- Fix a compatibility issue that if the foreign key constraint in `CREATE TABLE` statement has no schema, schema of the created table should be used instead of returning a `No Database selected` error [#12678](https://github.com/pingcap/tidb/pull/12678)
- Fix the issue that the `invalid list index` error is reported when executing `ADMIN CANCEL DDL JOBS` [#12681](https://github.com/pingcap/tidb/pull/12681)
Expand Down
15 changes: 15 additions & 0 deletions sql-statements/sql-statement-alter-table.md
Original file line number Diff line number Diff line change
Expand Up @@ -59,6 +59,21 @@ mysql> EXPLAIN SELECT * FROM t1 WHERE c1 = 3;
2 rows in set (0.00 sec)
```

## SHARD_ROW_ID_BITS

For tables with a non-integer `PRIMARY KEY` or without a `PRIMARY KEY`, TiDB uses an implicit auto-increment ROW ID. Because regions are automatically sharded using a range-based scheme on the `PRIMARY KEY`, hotspots can occur when there are a large number of `INSERT` operations.

To mitigate the hot spot issue, you can configure `SHARD_ROW_ID_BITS`. The ROW ID is scattered and the data is written into multiple different Regions. But setting an overlarge value might lead to an excessively large number of RPC requests, which increases the CPU and network overhead.

- `SHARD_ROW_ID_BITS = 4` indicates 16 shards
- `SHARD_ROW_ID_BITS = 6` indicates 64 shards
- `SHARD_ROW_ID_BITS = 0` indicates the default 1 shard

Usage:

- `CREATE TABLE`: `CREATE TABLE t (c int) SHARD_ROW_ID_BITS = 4;`
- `ALTER TABLE`: `ALTER TABLE t SHARD_ROW_ID_BITS = 4;`

## MySQL compatibility

* All of the data types except spatial types are supported. For other unsupported cases, refer to: [compatibility of DDL statements with MySQL](/mysql-compatibility.md#ddl).
Expand Down
2 changes: 1 addition & 1 deletion sql-statements/sql-statement-create-index.md
Original file line number Diff line number Diff line change
Expand Up @@ -144,7 +144,7 @@ Currently, the optimizer can use the indexed expressions when the expressions ar

## Associated session variables

The global variables associated with the `CREATE INDEX` statement are `tidb_ddl_reorg_worker_cnt`, `tidb_ddl_reorg_batch_size` and `tidb_ddl_reorg_priority`. Refer to [TiDB-specific system variables](/tidb-specific-system-variables.md#tidb_ddl_reorg_worker_cnt) for details.
The global variables associated with the `CREATE INDEX` statement are `tidb_ddl_reorg_worker_cnt`, `tidb_ddl_reorg_batch_size` and `tidb_ddl_reorg_priority`. Refer to [system variables](/system-variables.md#tidb_ddl_reorg_worker_cnt) for details.

## MySQL compatibility

Expand Down
17 changes: 16 additions & 1 deletion sql-statements/sql-statement-create-table.md
Original file line number Diff line number Diff line change
Expand Up @@ -224,7 +224,7 @@ table_option:
| STATS_PERSISTENT [=] {DEFAULT|0|1}
```

The `table_option` currently only supports `AUTO_INCREMENT`, `SHARD_ROW_ID_BITS` (see [TiDB Specific System Variables](/tidb-specific-system-variables.md#shard_row_id_bits) for details), `PRE_SPLIT_REGIONS`, `CHARACTER SET`, `COLLATE`, and `COMMENT`, while the others are only supported in syntax. The clauses are separated by a comma `,`. See the following table for details:
The `table_option` currently only supports `AUTO_INCREMENT`, `SHARD_ROW_ID_BITS`, `PRE_SPLIT_REGIONS`, `CHARACTER SET`, `COLLATE`, and `COMMENT`, while the others are only supported in syntax. The clauses are separated by a comma `,`. See the following table for details:

| Parameters | Description | Example |
| ---------- | ---------- | ------- |
Expand Down Expand Up @@ -269,6 +269,21 @@ mysql> SELECT * FROM t1;
1 row in set (0.00 sec)
```

## SHARD_ROW_ID_BITS

For tables with a non-integer `PRIMARY KEY` or without a `PRIMARY KEY`, TiDB uses an implicit auto-increment ROW ID. Because regions are automatically sharded using a range-based scheme on the `PRIMARY KEY`, hotspots can occur when there are a large number of `INSERT` operations.

To mitigate the hot spot issue, you can configure `SHARD_ROW_ID_BITS`. The ROW ID is scattered and the data is written into multiple different Regions. But setting an overlarge value might lead to an excessively large number of RPC requests, which increases the CPU and network overhead.

- `SHARD_ROW_ID_BITS = 4` indicates 16 shards
- `SHARD_ROW_ID_BITS = 6` indicates 64 shards
- `SHARD_ROW_ID_BITS = 0` indicates the default 1 shard

Usage:

- `CREATE TABLE`: `CREATE TABLE t (c int) SHARD_ROW_ID_BITS = 4;`
- `ALTER TABLE`: `ALTER TABLE t SHARD_ROW_ID_BITS = 4;`

## MySQL compatibility

* TiDB does not support temporary tables, but it ignores the `CREATE TEMPORARY TABLE` syntax.
Expand Down
6 changes: 5 additions & 1 deletion sql-statements/sql-statement-show-variables.md
Original file line number Diff line number Diff line change
Expand Up @@ -25,7 +25,7 @@ This statement shows a list of variables for the scope of either `GLOBAL` or `SE

## Examples

List all TiDB specific variables. For detailed description, refer to [TiDB Specific System Variables](/tidb-specific-system-variables.md).
List all TiDB specific variables. For detailed description, refer to [System Variables](/system-variables.md).

```sql
mysql> SHOW GLOBAL VARIABLES LIKE 'tidb%';
Expand Down Expand Up @@ -155,3 +155,7 @@ mysql> SHOW GLOBAL VARIABLES LIKE 'time_zone%';
## MySQL compatibility

This statement is understood to be fully compatible with MySQL. Any compatibility differences should be [reported via an issue](/report-issue.md) on GitHub.

## See also

* [`SET [GLOBAL|SESSION]`](/sql-statements/sql-statement-set-variable.md)
9 changes: 3 additions & 6 deletions sql-statements/sql-statement-split-region.md
Original file line number Diff line number Diff line change
Expand Up @@ -391,10 +391,7 @@ region3: [ 2<<61 , 3<<61 )
region4: [ 3<<61 , +inf )
```

## Related session variable
## See also

There are two `SPLIT REGION` related session variables: `tidb_scatter_region`, `tidb_wait_split_region_finish` and `tidb_wait_split_region_timeout`. For details, see [TiDB specific system variables and syntax](/tidb-specific-system-variables.md).

## Reference

[SHOW TABLE REGIONS](/sql-statements/sql-statement-show-table-regions.md)
* [SHOW TABLE REGIONS](/sql-statements/sql-statement-show-table-regions.md)
* Session variables: [`tidb_scatter_region`](/system-variables.md#tidb_scatter_region), [`tidb_wait_split_region_finish`](/system-variables.md#tidb_wait_split_region_finish) and [`tidb_wait_split_region_timeout`](/system-variables.md#tidb_wait_split_region_timeout).
Loading