From 9d895377cd59ae9f8237deabc67a835278908f10 Mon Sep 17 00:00:00 2001 From: Matt Lord Date: Thu, 25 Apr 2024 12:38:29 -0400 Subject: [PATCH 1/2] Add some docs on VTGate buffering for MoveTables and Reshard traffic switching Signed-off-by: Matt Lord --- content/en/docs/20.0/concepts/move-tables.md | 2 +- .../20.0/reference/features/vtgate-buffering.md | 13 +++++++++++++ .../docs/20.0/reference/vreplication/movetables.md | 8 ++++++++ .../en/docs/20.0/reference/vreplication/reshard.md | 8 ++++++++ 4 files changed, 30 insertions(+), 1 deletion(-) diff --git a/content/en/docs/20.0/concepts/move-tables.md b/content/en/docs/20.0/concepts/move-tables.md index ced9abf99..cf72e817d 100644 --- a/content/en/docs/20.0/concepts/move-tables.md +++ b/content/en/docs/20.0/concepts/move-tables.md @@ -2,7 +2,7 @@ title: MoveTables --- -MoveTables is a new workflow based on VReplication. It enables you to relocate tables between keyspaces, and therefore physical MySQL instances, without downtime. +MoveTables is a workflow based on VReplication. It enables you to relocate tables between keyspaces, and therefore physical MySQL instances, without downtime. ## Identifying Candidate Tables diff --git a/content/en/docs/20.0/reference/features/vtgate-buffering.md b/content/en/docs/20.0/reference/features/vtgate-buffering.md index 6e831a2e8..1c317388e 100644 --- a/content/en/docs/20.0/reference/features/vtgate-buffering.md +++ b/content/en/docs/20.0/reference/features/vtgate-buffering.md @@ -82,6 +82,19 @@ endeavor to handle errors appropriately if/when they occur (e.g. unplanned outages/fail overs, etc.) {{< /warning >}} +## What happens during a MoveTables or Reshard SwitchTraffic or ReverseTraffic with Buffering + +Fundamentally Vitess will: + + * Hold up and buffer any queries sent to the tables (MoveTables) or shards (Reshard) for which traffic is being switched. + * Perform the traffic switching work so that application traffic against the tables (MoveTables) or shards (Reshard) are transparently switched to the new keyspace (MoveTables) or shards (Reshard). + * Drain the buffered queries to the new keyspace or shards — or if the switch failed then back to the original keyspace or shards. + +{{< warning >}} +This process is not guaranteed to eliminate errors to the application, but +rather reduce them or make them less frequent. The application should still +endeavor to handle errors appropriately if/when they occur. +{{< /warning >}} ## How does it work? diff --git a/content/en/docs/20.0/reference/vreplication/movetables.md b/content/en/docs/20.0/reference/vreplication/movetables.md index 10939712f..ef89a1417 100644 --- a/content/en/docs/20.0/reference/vreplication/movetables.md +++ b/content/en/docs/20.0/reference/vreplication/movetables.md @@ -80,6 +80,10 @@ It is too expensive to get real-time row counts of tables, using _count(*)_, say [`switchtraffic`](../../programs/vtctldclient/vtctldclient_movetables/vtctldclient_movetables_switchtraffic/) switches traffic forward for the `tablet-types` specified. You can switch all traffic with just one command, and this is the default behavior. Note that you can now switch replica, rdonly, and primary traffic in any order. +{{< info >}} +Note that VTGate can [buffer queries](../../features/vtgate-buffering/) when switching traffic which can virtually eliminate any visible impact on application users. +{{}} + #### ReverseTraffic @@ -87,6 +91,10 @@ It is too expensive to get real-time row counts of tables, using _count(*)_, say [`reversetraffic`](../../programs/vtctldclient/vtctldclient_movetables/vtctldclient_movetables_reversetraffic/) switches traffic in the reverse direction for the `tablet-types` specified. The traffic should have been previously switched forward using `SwitchTraffic` for the `cells` and `tablet-types` specified. +{{< info >}} +Note that VTGate can [buffer queries](../../features/vtgate-buffering/) when switching traffic which can virtually eliminate any visible impact on application users. +{{}} + #### Cancel diff --git a/content/en/docs/20.0/reference/vreplication/reshard.md b/content/en/docs/20.0/reference/vreplication/reshard.md index aa6b011d2..55763e74a 100644 --- a/content/en/docs/20.0/reference/vreplication/reshard.md +++ b/content/en/docs/20.0/reference/vreplication/reshard.md @@ -64,6 +64,10 @@ It is too expensive to get real-time row counts of tables, using _count(*)_, say [`SwitchTraffic`](../../programs/vtctldclient/vtctldclient_movetables/vtctldclient_movetables_switchtraffic) switches traffic forward for the `tablet-types` specified. This replaces the previous `SwitchReads` and `SwitchWrites` commands with a single one. It is now possible to switch all traffic with just one command, and this is the default behavior. Also, you can now switch replica, rdonly and primary traffic in any order: earlier you needed to first `SwitchReads` (for replicas and rdonly tablets) first before `SwitchWrites`. +{{< info >}} +Note that VTGate can [buffer queries](../../features/vtgate-buffering/) when switching traffic which can virtually eliminate any visible impact on application users. +{{}} + #### ReverseTraffic @@ -71,6 +75,10 @@ It is too expensive to get real-time row counts of tables, using _count(*)_, say [`ReverseTraffic`](../../programs/vtctldclient/vtctldclient_movetables/vtctldclient_movetables_reversetraffic/) switches traffic in the reverse direction for the `tablet-types` specified. The traffic should have been previously switched forward using `SwitchTraffic` for the `cells` and `tablet_types` specified. +{{< info >}} +Note that VTGate can [buffer queries](../../features/vtgate-buffering/) when switching traffic which can virtually eliminate any visible impact on application users. +{{}} + #### Cancel From acfd019bc88349c9985dfc9b1bf2c858a876faf4 Mon Sep 17 00:00:00 2001 From: Matt Lord Date: Tue, 30 Apr 2024 16:54:25 -0400 Subject: [PATCH 2/2] Address review comments Signed-off-by: Matt Lord --- .../reference/features/vtgate-buffering.md | 20 +++++++------------ .../20.0/reference/vreplication/movetables.md | 2 +- .../20.0/reference/vreplication/reshard.md | 2 +- 3 files changed, 9 insertions(+), 15 deletions(-) diff --git a/content/en/docs/20.0/reference/features/vtgate-buffering.md b/content/en/docs/20.0/reference/features/vtgate-buffering.md index 1c317388e..aad5a228e 100644 --- a/content/en/docs/20.0/reference/features/vtgate-buffering.md +++ b/content/en/docs/20.0/reference/features/vtgate-buffering.md @@ -22,6 +22,13 @@ with the simplest case: that of buffering during a PRS (PlannedReparentShard) operation. Examples of various edge cases can be found in [Buffering Scenarios](../../../user-guides/configuration-advanced/buffering-scenarios/). +{{< warning >}} +The buffering feature is not guaranteed to eliminate errors sent to the application, but +rather reduce them or make them less frequent. The application should still endeavor to +handle errors appropriately if/when they occur (e.g. unplanned outages, planned failovers, +VReplication migrations, etc.) +{{< /warning >}} + ## VTGate flags to enable buffering First, let us cover the flags that need to be set in VTGate to enable @@ -75,13 +82,6 @@ Fundamentally Vitess will: * Drain the buffered queries to the new `PRIMARY` tablet. * Begin the countdown timer for `buffer_max_failover_duration`. - {{< warning >}} -This process is not guaranteed to eliminate errors to the application, but -rather reduce them or make them less frequent. The application should still -endeavor to handle errors appropriately if/when they occur (e.g. unplanned -outages/fail overs, etc.) -{{< /warning >}} - ## What happens during a MoveTables or Reshard SwitchTraffic or ReverseTraffic with Buffering Fundamentally Vitess will: @@ -90,12 +90,6 @@ Fundamentally Vitess will: * Perform the traffic switching work so that application traffic against the tables (MoveTables) or shards (Reshard) are transparently switched to the new keyspace (MoveTables) or shards (Reshard). * Drain the buffered queries to the new keyspace or shards — or if the switch failed then back to the original keyspace or shards. -{{< warning >}} -This process is not guaranteed to eliminate errors to the application, but -rather reduce them or make them less frequent. The application should still -endeavor to handle errors appropriately if/when they occur. -{{< /warning >}} - ## How does it work? The following steps are considerably simplified, but to give a high level diff --git a/content/en/docs/20.0/reference/vreplication/movetables.md b/content/en/docs/20.0/reference/vreplication/movetables.md index ef89a1417..0ca61cb88 100644 --- a/content/en/docs/20.0/reference/vreplication/movetables.md +++ b/content/en/docs/20.0/reference/vreplication/movetables.md @@ -92,7 +92,7 @@ Note that VTGate can [buffer queries](../../features/vtgate-buffering/) when swi [`reversetraffic`](../../programs/vtctldclient/vtctldclient_movetables/vtctldclient_movetables_reversetraffic/) switches traffic in the reverse direction for the `tablet-types` specified. The traffic should have been previously switched forward using `SwitchTraffic` for the `cells` and `tablet-types` specified. {{< info >}} -Note that VTGate can [buffer queries](../../features/vtgate-buffering/) when switching traffic which can virtually eliminate any visible impact on application users. +Note that VTGate can [buffer queries](../../features/vtgate-buffering/) when reversing traffic which can virtually eliminate any visible impact on application users. {{}} diff --git a/content/en/docs/20.0/reference/vreplication/reshard.md b/content/en/docs/20.0/reference/vreplication/reshard.md index 55763e74a..1bf3a55d8 100644 --- a/content/en/docs/20.0/reference/vreplication/reshard.md +++ b/content/en/docs/20.0/reference/vreplication/reshard.md @@ -76,7 +76,7 @@ Note that VTGate can [buffer queries](../../features/vtgate-buffering/) when swi [`ReverseTraffic`](../../programs/vtctldclient/vtctldclient_movetables/vtctldclient_movetables_reversetraffic/) switches traffic in the reverse direction for the `tablet-types` specified. The traffic should have been previously switched forward using `SwitchTraffic` for the `cells` and `tablet_types` specified. {{< info >}} -Note that VTGate can [buffer queries](../../features/vtgate-buffering/) when switching traffic which can virtually eliminate any visible impact on application users. +Note that VTGate can [buffer queries](../../features/vtgate-buffering/) when reversing traffic which can virtually eliminate any visible impact on application users. {{}}