Docs: Online DDL, @@migration_context, --skip-preflight (#1558)

* Docs: Online DDL, @@migration_context, --skip-preflight

Signed-off-by: Shlomi Noach <2607934+shlomi-noach@users.noreply.github.com>

* vtctlclient -> vtctldclient

Signed-off-by: Shlomi Noach <2607934+shlomi-noach@users.noreply.github.com>

---------

Signed-off-by: Shlomi Noach <2607934+shlomi-noach@users.noreply.github.com>

content/en/docs/18.0/concepts/query-rewriting.md

@@ -95,6 +95,7 @@ Here is a list of all the system variables that are handled by Vitess and how th
 | charset                                 | VitessAware    |
 | names                                   | VitessAware    |
 | session_uuid                            | VitessAware    |
+| migration_context                       | VitessAware    |
 | enable_system_settings                  | VitessAware    |
 | read_after_write_gtid                   | VitessAware    |
 | read_after_write_timeout                | VitessAware    |

content/en/docs/18.0/reference/programs/vtctl/schema-version-permissions.md

@@ -149,11 +149,11 @@ Validates that the schema on the primary tablet for shard 0 matches the schema o
 
 ### ApplySchema
 
-Applies the schema change to the specified keyspace on all shards. The recommended method of applying schema changes is via [Online DDL](../../../../user-guides/schema-changes/managed-online-schema-changes/), which ensures migrations are non-blocking and keeps replication in sync throughout the operation. --ddl_strategy is used to instruct migrations via vreplication, `gh-ost` or `pt-osc` with optional parameters. -request_context allows the user to specify a custom request context for online DDL migrations. If --skip_preflight, SQL goes directly to shards without going through sanity checks.
+Applies the schema change to the specified keyspace on all shards. The recommended method of applying schema changes is via [Online DDL](../../../../user-guides/schema-changes/managed-online-schema-changes/), which ensures migrations are non-blocking and keeps replication in sync throughout the operation. --ddl_strategy is used to instruct migrations via vreplication, `gh-ost` or `pt-osc` with optional parameters. -request_context allows the user to specify a custom request context for online DDL migrations.
 
 #### Example
 
-<pre class="command-example">ApplySchema -- [--wait_replicas_timeout=10s] [--ddl_strategy=<ddl_strategy>] [--request_context=<unique-request-context>] [--skip_preflight] {--sql=&lt;sql&gt; || --sql-file=&lt;filename&gt;} &lt;keyspace&gt;</pre>
+<pre class="command-example">ApplySchema -- [--wait_replicas_timeout=10s] [--ddl_strategy=<ddl_strategy>] [--request_context=<unique-request-context>] {--sql=&lt;sql&gt; || --sql-file=&lt;filename&gt;} &lt;keyspace&gt;</pre>
 
 #### Flags
 
@@ -162,7 +162,6 @@ Applies the schema change to the specified keyspace on all shards. The recommend
 | caller_id | string | Effective caller ID used for the operation and should map to an ACL name which grants this identity the necessary permissions to perform the operation (this is only necessary when strict table ACLs are used). |
 | ddl_strategy | string | Online DDL strategy, compatible with @@ddl_strategy session variable (examples: 'gh-ost', 'pt-osc', 'gh-ost --max-load=Threads_running=100' (default "direct"). |
 | request_context | string | For Only DDL, optionally supply a custom unique string used as context for the migration(s) in this command. By default a unique context is auto-generated by Vitess. |
-| skip_preflight | Boolean | Skip pre-apply schema checks, and directly forward schema change query to shards. Ignored when using Online DDL |
 | sql | string | A list of semicolon-delimited SQL commands. |
 | sql-file | string | Identifies the file that contains the SQL commands. This file needs to exist on the server, rather than on the client. |
 | wait_replicas_timeout | Duration | The amount of time to wait for replicas to receive the schema change via replication (default 10s). |

content/en/docs/18.0/user-guides/migration/troubleshooting.md

@@ -137,9 +137,9 @@ This can be caused by a DDL executed on the source table as by default — contr
 If you want the same or similar DDL to be applied on the target then you can apply that DDL on the target keyspace
 and then restart the workflow. For example, using the example above:
 ```bash
-$ vtctlclient ApplySchema -- --ddl_strategy=direct --sql="alter table customer add notes varchar(100) not null" customer
+$ vtctldclient ApplySchema --ddl-strategy=direct --sql="alter table customer add notes varchar(100) not null" customer
 
-$ vtctlclient Workflow -- customer.commerce2customer start
+$ vtctldclient Workflow customer.commerce2customer start
 ``` 
 
 If the tables are not very large or the workflow has not made much progress, you can alternatively `Cancel` the current

content/en/docs/18.0/user-guides/schema-changes/advanced-usage.md

@@ -33,9 +33,9 @@ Thus, the new migration does not get to execute if an identical previous migrati
 Usage:
 
 ```sh
-$ vtctldclient ApplySchema -- --migration_context="1111-2222" --skip_preflight --ddl_strategy='vitess' --sql "alter table customer add column status int unsigned not null" commerce
+$ vtctldclient ApplySchema -- --migration_context="1111-2222" --ddl_strategy='vitess' --sql "alter table customer add column status int unsigned not null" commerce
 
-$ vtctldclient ApplySchema -- --migration_context="1111-2222" --skip_preflight --ddl_strategy='vitess' --sql "alter table customer add column status int unsigned not null" commerce
+$ vtctldclient ApplySchema -- --migration_context="1111-2222" --ddl_strategy='vitess' --sql "alter table customer add column status int unsigned not null" commerce
 ```
 
 In the above, the two calls are identical. Specifically, they share the exact same `--migration_context` value of `1111-2222`, and the exact same `--sql`.
@@ -49,16 +49,26 @@ Notes:
 - Continuing the above example: if the new migration is successful, the previous migration remains in `failed` state.
 - The decision whether to run the migration or to mark it as implicitly `complete` only takes place when the migration is scheduled to run.
 
+The context may also be set via VTGate:
+
+```sql
+mysql> set @@ddl_strategy='vitess --';
+mysql> set @@migration_context='1111-2222';
+mysql> alter table customer add column status int unsigned not null;
+```
+
+By default, the migration context for an Online DDL issued via VTGate is the value of `@@session_uuid`. If `@@migration_context` is non-empty, then its value is used.
+
 ## Duplicate migration indication
 
 You may go one step beyond [duplicate migration detection](#duplicate-migration-detection) by explicitly supplying migration UUIDs such that duplicate migrations are never submitted in the first place.
 
 Consider the following example, note `--uuid_list` flag:
 
 ```sh
-$ vtctldclient ApplySchema -- --uuid_list "73380089_7764_11ec_a656_0a43f95f28a3" --skip_preflight --ddl_strategy='vitess' --sql "alter table customer add column status int unsigned not null" commerce
+$ vtctldclient ApplySchema -- --uuid_list "73380089_7764_11ec_a656_0a43f95f28a3" --ddl_strategy='vitess' --sql "alter table customer add column status int unsigned not null" commerce
 
-$ vtctldclient ApplySchema -- --uuid_list "73380089_7764_11ec_a656_0a43f95f28a3" --skip_preflight --ddl_strategy='vitess' --sql "alter table customer add column status int unsigned not null" commerce
+$ vtctldclient ApplySchema -- --uuid_list "73380089_7764_11ec_a656_0a43f95f28a3" --ddl_strategy='vitess' --sql "alter table customer add column status int unsigned not null" commerce
 ```
 
 Normally Vitess generates a `UUID` for each migration, thus having a new, unique ID per migration. With `-uuid_list`, you can force Vitess into using your own supplied UUID. There cannot be two migrations with the same `UUID`. Therefore, any subsequent submission of a migration with an already existing `UUID` is implicitly discarded. The 2nd call does return the migration `UUID`, but is internally discarded.
@@ -86,7 +96,7 @@ The user may submit multiple migrations such that non auto-completes. The user c
 Consider the following:
 
 ```sh
-$ vtctldclient ApplySchema --skip_preflight --ddl_strategy='vitess --postpone-completion --allow-concurrent' --sql "alter table customer add column country int not null default 0; alter table order add column shipping_country int not null default 0" commerce
+$ vtctldclient ApplySchema --ddl_strategy='vitess --postpone-completion --allow-concurrent' --sql "alter table customer add column country int not null default 0; alter table order add column shipping_country int not null default 0" commerce
 29231906_776f_11ec_a656_0a43f95f28a3
 3cc4ae0e_776f_11ec_a656_0a43f95f28a3
 ```
@@ -98,14 +108,14 @@ A `show vitess_migrations like '29231906_776f_11ec_a656_0a43f95f28a3'` presents
 When all migrations for the relevant UUIDs show `1` for `ready_to_complete`, the user can then either:
 
 ```sh
-$ vtctldclient ApplySchema --skip_preflight --sql "alter vitess_migration complete all" commerce
+$ vtctldclient ApplySchema --sql "alter vitess_migration complete all" commerce
 ```
 
 Assuming these are the only migrations awaiting completion, or, explicitly issue a complete for each of the migrations:
 
 ```sh
-$ vtctldclient ApplySchema --skip_preflight --sql "alter vitess_migration '29231906_776f_11ec_a656_0a43f95f28a3' complete all" commerce
-$ vtctldclient ApplySchema --skip_preflight --sql "alter vitess_migration '3cc4ae0e_776f_11ec_a656_0a43f95f28a3' complete all" commerce
+$ vtctldclient ApplySchema --sql "alter vitess_migration '29231906_776f_11ec_a656_0a43f95f28a3' complete all" commerce
+$ vtctldclient ApplySchema --sql "alter vitess_migration '3cc4ae0e_776f_11ec_a656_0a43f95f28a3' complete all" commerce
 ```
 
 ## Near instant REVERTs
@@ -125,14 +135,14 @@ The use case and workflow is as follows:
 Consider the following example. We run a 5 hour long migration to drop an index:
 
 ```sh
-$ vtctldclient ApplySchema --skip_preflight --ddl_strategy='vitess' --sql "alter table customer drop index joined_timestamp_idx" commerce
+$ vtctldclient ApplySchema --ddl_strategy='vitess' --sql "alter table customer drop index joined_timestamp_idx" commerce
 29231906_776f_11ec_a656_0a43f95f28a3
 ```
 
 As soon as the migration completes, we run:
 
 ```sh
-$ vtctldclient ApplySchema --skip_preflight --ddl_strategy='vitess --postpone-completion --allow-concurrent' --sql "revert vitess_migration '29231906_776f_11ec_a656_0a43f95f28a3'" commerce
+$ vtctldclient ApplySchema --ddl_strategy='vitess --postpone-completion --allow-concurrent' --sql "revert vitess_migration '29231906_776f_11ec_a656_0a43f95f28a3'" commerce
 3cc4ae0e_776f_11ec_a656_0a43f95f28a3
 ```
 
@@ -141,15 +151,15 @@ The above begins a `REVERT` migration that is open-ended (does not complete), vi
 Finally, if we are satisfied that the `drop index` migration went well, we issue:
 
 ```sh
-$ vtctldclient ApplySchema --skip_preflight --sql "alter vitess_migration '3cc4ae0e_776f_11ec_a656_0a43f95f28a3' cancel" commerce
+$ vtctldclient ApplySchema --sql "alter vitess_migration '3cc4ae0e_776f_11ec_a656_0a43f95f28a3' cancel" commerce
 ```
 
 That is, we cancel the `REVERT` operation.
 
 Or, should we have not dropped the index? If our migration seems to have been wrong, we run:
 
 ```sh
-$ vtctldclient ApplySchema --skip_preflight --sql "alter vitess_migration '3cc4ae0e_776f_11ec_a656_0a43f95f28a3' complete" commerce
+$ vtctldclient ApplySchema --sql "alter vitess_migration '3cc4ae0e_776f_11ec_a656_0a43f95f28a3' complete" commerce
 ```
 
 Which means we want to apply the revert. Since the revert is already running in the background, it is likely that binary log processing is up to date, and cut-over is near instantaneous.
@@ -171,7 +181,7 @@ Note that there can be scenarios with impossible ordering. Those hardly make sen
 An example for in-order submission:
 
 ```sh
-$ vtctldclient ApplySchema --skip_preflight --ddl_strategy='vitess --allow-concurrent --in-order-completion' --sql "create table t1 (id int primary key); create view v1 as select id from t1;" commerce
+$ vtctldclient ApplySchema --ddl_strategy='vitess --allow-concurrent --in-order-completion' --sql "create table t1 (id int primary key); create view v1 as select id from t1;" commerce
 ```
 
 {{< info >}}

content/en/docs/18.0/user-guides/schema-changes/audit-and-control.md

@@ -60,13 +60,13 @@ mysql> drop table customer;
 You may use `vtctl` or `vtctlclient` (the two are interchangeable for the purpose of this document) to apply schema changes. The `ApplySchema` command supports both synchronous and online schema migrations. To run an online schema migration you will supply the `--ddl_strategy` command line flag:
 
 ```shell
-$ vtctlclient -- ApplySchema --ddl_strategy "vitess" --sql "ALTER TABLE demo MODIFY id bigint UNSIGNED" commerce
+$ vtctldclient ApplySchema --ddl-strategy "vitess" --sql "ALTER TABLE demo MODIFY id bigint UNSIGNED" commerce
 a2994c92_f1d4_11ea_afa3_f875a4d24e90
 ```
 
  You my run multiple migrations withing the same `ApplySchema` command:
 ```shell
-$ vtctlclient -- ApplySchema --skip_preflight --ddl_strategy "vitess" --sql "ALTER TABLE demo MODIFY id bigint UNSIGNED; CREATE TABLE sample (id int PRIMARY KEY); DROP TABLE another;" commerce
+$ vtctldclient ApplySchema --ddl-strategy "vitess" --sql "ALTER TABLE demo MODIFY id bigint UNSIGNED; CREATE TABLE sample (id int PRIMARY KEY); DROP TABLE another;" commerce
 3091ef2a_4b87_11ec_a827_0a43f95f28a3
 ```
 
@@ -75,7 +75,6 @@ $ vtctlclient -- ApplySchema --skip_preflight --ddl_strategy "vitess" --sql "ALT
 - `--ddl_strategy`: by default migrations run directly via MySQL standard DDL. This flag must be aupplied to indicate an online strategy. See also [DDL strategies](../ddl-strategies) and [ddl_strategy flags](../ddl-strategy-flags).
 - `--migration_context <unique-value>`: all migrations in a `ApplySchema` command are logically grouped via a unique _context_. A unique value will be supplied automatically. The user may choose to supply their own value, and it's their responsibility to provide with a unique value. Any string format is accepted.
   The context can then be used to search for migrations, via `SHOW VITESS_MIGRATIONS LIKE 'the-context'`. It is visible in `SHOW VITESS_MIGRATIONS ...` output as the `migration_context` column.
-- `--skip_preflight`: skip an internal Vitess schema validation. When running an online DDL it's recommended to add `--skip_preflight`. In future Vitess versions this flag may be removed or default to `true`.
 
 ## Tracking migrations
 
@@ -328,19 +327,19 @@ Query OK, 1 row affected (0.01 sec)
 Launch a specific migration:
 
 ```shell
-$ vtctlclient -- ApplySchema --sql "alter vitess_migration '9e8a9249_3976_11ed_9442_0a43f95f28a3' launch" commerce
+$ vtctldclient ApplySchema --sql "alter vitess_migration '9e8a9249_3976_11ed_9442_0a43f95f28a3' launch" commerce
 ```
 
 Or launch a specific migration on a specific shard:
 
 ```shell
-$ vtctlclient -- ApplySchema --sql "alter vitess_migration '9e8a9249_3976_11ed_9442_0a43f95f28a3' launch vitess_shards '-40,40-80'" commerce
+$ vtctldclient ApplySchema --sql "alter vitess_migration '9e8a9249_3976_11ed_9442_0a43f95f28a3' launch vitess_shards '-40,40-80'" commerce
 ```
 
 Or launch all:
 
 ```shell
-$ vtctlclient -- ApplySchema --sql "alter vitess_migration launch all" commerce
+$ vtctldclient ApplySchema --sql "alter vitess_migration launch all" commerce
 ```
 
 ## Completing a migration
@@ -366,13 +365,13 @@ Query OK, 1 row affected (0.01 sec)
 Complete a specific migration:
 
 ```shell
-$ vtctlclient -- ApplySchema --sql "alter vitess_migration '9e8a9249_3976_11ed_9442_0a43f95f28a3' complete" commerce
+$ vtctldclient ApplySchema --sql "alter vitess_migration '9e8a9249_3976_11ed_9442_0a43f95f28a3' complete" commerce
 ```
 
 Or complete all:
 
 ```shell
-$ vtctlclient -- ApplySchema --sql "alter vitess_migration complete all" commerce
+$ vtctldclient ApplySchema --sql "alter vitess_migration complete all" commerce
 ```
 
 ## Cancelling a migration
@@ -490,7 +489,7 @@ Examples for a 4-shard cluster:
 
 
 ```shell
-$ vtctlclient -- ApplySchema --sql "alter vitess_migration cancel all" commerce
+$ vtctldclient ApplySchema --sql "alter vitess_migration cancel all" commerce
 ```
 
 Also available via `vtctlclient OnlineDDL` command:
@@ -589,7 +588,7 @@ completed_timestamp: NULL
 
 
 ```shell
-$ vtctlclient -- ApplySchema --sql "alter vitess_migration '2201058f_f266_11ea_bab4_0242c0a8b007' retry" commerce
+$ vtctldclient ApplySchema --sql "alter vitess_migration '2201058f_f266_11ea_bab4_0242c0a8b007' retry" commerce
 ```
 
 Also available via `vtctlclient OnlineDDL` command:
@@ -658,7 +657,7 @@ Query OK, 1 row affected (0.00 sec)
 
 #### Via vtctlclient/ApplySchema
 
-Execute via `vtctlclient -- ApplySchema --sql "..." <keyspace>` like previous commands, or use `OnlineDDL` command:
+Execute via `vtctldclient ApplySchema --sql "..." <keyspace>` like previous commands, or use `OnlineDDL` command:
 
 
 ```shell
@@ -728,8 +727,8 @@ Create Table: CREATE TABLE `corder` (
 
 ### Via vtctlclient/ApplySchema
 
-```
-$ vtctlclient -- ApplySchema --ddl_strategy "vitess" --sql "revert vitess_migration '1a689113_8d77_11eb_815f_f875a4d24e90'" commerce
+```sh
+$ vtctldclient ApplySchema --ddl-strategy "vitess" --sql "revert vitess_migration '1a689113_8d77_11eb_815f_f875a4d24e90'" commerce
 ```
 
 ## Controlling throttling
@@ -786,7 +785,7 @@ The command `show vitess_throttled_apps;` is a general purpose throttler command
 
 ### Via vtctlclient/ApplySchema
 
-Execute via `vtctlclient -- ApplySchema --sql "..." <keyspace>` like previous commands, or use `OnlineDDL` commands:
+Execute via `vtctldclient ApplySchema --sql "..." <keyspace>` like previous commands, or use `OnlineDDL` commands:
 
 
 ```shell

content/en/docs/18.0/user-guides/schema-changes/concurrent-migrations.md

@@ -46,7 +46,7 @@ mysql> create table sample_table(id int primary key);
 or, via `vtctl`:
 
 ```shell
-vtctl ApplySchema -- --skip_preflight --ddl_strategy "vitess --allow-concurrent" -sql "REVERT VITESS_MIGRATION '3091ef2a_4b87_11ec_a827_0a43f95f28a3'"
+vtctldclient ApplySchema --ddl-strategy "vitess --allow-concurrent" -sql "REVERT VITESS_MIGRATION '3091ef2a_4b87_11ec_a827_0a43f95f28a3'"
 ```
 
 ## Restrictions and eligibility

content/en/docs/18.0/user-guides/schema-changes/ddl-strategies.md

@@ -23,12 +23,12 @@ You will set either `@@ddl_strategy` session variable, or `--ddl_strategy` comma
 #### Via vtctlclient
 
 ```shell
-$ vtctlclient -- ApplySchema --ddl_strategy "vitess" --sql "ALTER TABLE demo MODIFY id bigint UNSIGNED" commerce
+$ vtctldclient ApplySchema --ddl-strategy "vitess" --sql "ALTER TABLE demo MODIFY id bigint UNSIGNED" commerce
 a2994c92_f1d4_11ea_afa3_f875a4d24e90
 ```
 
 ```shell
-$ vtctlclient -- ApplySchema --ddl_strategy "gh-ost --max-load Threads_running=200" --sql "ALTER TABLE demo add column status int" commerce
+$ vtctldclient ApplySchema --ddl-strategy "gh-ost --max-load Threads_running=200" --sql "ALTER TABLE demo add column status int" commerce
 ```
 
 #### Via VTGate
@@ -86,7 +86,7 @@ Vitess takes care of setting up the necessary command line flags. It automatical
 
 - `set @@ddl_strategy='gh-ost --max-load Threads_running=200';`
 - `set @@ddl_strategy='gh-ost --max-load Threads_running=200 --critical-load Threads_running=500 --critical-load-hibernate-seconds=60 --default-retries=512';`
-- `vtctlclient -- ApplySchema --ddl_strategy "gh-ost --allow-nullable-unique-key --chunk-size 200" ...`
+- `vtctldclient --ddl-strategy "gh-ost --allow-nullable-unique-key --chunk-size 200" ...`
 
 **Note:** Do not override the following flags: `alter, database, table, execute, max-lag, force-table-names, serve-socket-file, hooks-path, hooks-hint-token, panic-flag-file`. Overriding any of these may cause Vitess to lose control and track of the migration, or even to migrate the wrong table.
 
@@ -108,7 +108,7 @@ Vitess takes care of supplying the command line flags, the DSN, the username & p
 
 - `set @@ddl_strategy='pt-osc --null-to-not-null';`
 - `set @@ddl_strategy='pt-osc --max-load Threads_running=200';`
-- `vtctlclient -- ApplySchema --ddl_strategy "pt-osc --alter-foreign-keys-method auto --chunk-size 200" ...`
+- `vtctldclient ApplySchema --ddl-strategy "pt-osc --alter-foreign-keys-method auto --chunk-size 200" ...`
 
 Vitess tracks the state of the `pt-osc` migration. If it fails, Vitess makes sure to drop the migration triggers. Vitess keeps track of the migration even if the tablet itself restarts for any reason. Normally that would terminate the migration; Vitess will cleanup the triggers if so, or will happily let the migration run to completion if not.