Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Fix insert query of blocked_recovery table in VTOrc #12091

Merged
merged 2 commits into from
Jan 17, 2023
Merged

Fix insert query of blocked_recovery table in VTOrc #12091

merged 2 commits into from
Jan 17, 2023

Conversation

GuptaManan100
Copy link
Member

Description

This PR fixes the insertion query for blocked_recovery table for VTOrc which was broken in #12012.

Related Issue(s)

Checklist

  • "Backport to:" labels have been added if this change should be back-ported
  • Tests were added or are not required
  • Documentation was added or is not required

Deployment Notes

@GuptaManan100
feat: add failing test and fix the query of insertion
d9914ca 
Signed-off-by: Manan Gupta <manan@planetscale.com>
@GuptaManan100 GuptaManan100 added Type: Bug Component: VTorc Vitess Orchestrator integration labels Jan 13, 2023
@vitess-bot vitess-bot bot added NeedsDescriptionUpdate The description is not clear or comprehensive enough, and needs work NeedsWebsiteDocsUpdate What it says labels Jan 13, 2023
@vitess-bot
Copy link
Contributor

vitess-bot bot commented Jan 13, 2023 •
edited by mattlord
Loading

Review Checklist

Hello reviewers! 👋 Please follow this checklist when reviewing this Pull Request.

General

  • Ensure that the Pull Request has a descriptive title.
  • If this is a change that users need to know about, please apply the release notes (needs details) label so that merging is blocked unless the summary release notes document is included.
  • If a test is added or modified, there should be a documentation on top of the test to explain what the expected behavior is what the test does.

If a new flag is being introduced:

  • Is it really necessary to add this flag?
  • Flag names should be clear and intuitive (as far as possible)
  • Help text should be descriptive.
  • Flag names should use dashes (-) as word separators rather than underscores (_).

If a workflow is added or modified:

  • Each item in Jobs should be named in order to mark it as required.
  • If the workflow should be required, the maintainer team should be notified.

Bug fixes

  • There should be at least one unit or end-to-end test.
  • The Pull Request description should include a link to an issue that describes the bug.

Non-trivial changes

  • There should be some code comments as to why things are implemented the way they are.

New/Existing features

  • Should be documented, either by modifying the existing documentation or creating new documentation.
  • New features should have a link to a feature request issue or an RFC that documents the use cases, corner cases and test cases.

Backward compatibility

  • Protobuf changes should be wire-compatible.
  • Changes to _vt tables and RPCs need to be backward compatible.
  • vtctl command output order should be stable and awk-able.
  • RPC changes should be compatible with vitess-operator
  • If a flag is removed, then it should also be removed from VTop, if used there.

@GuptaManan100 GuptaManan100 removed NeedsDescriptionUpdate The description is not clear or comprehensive enough, and needs work NeedsWebsiteDocsUpdate What it says labels Jan 13, 2023
@GuptaManan100
empty-commit
cf1f2c2 
Signed-off-by: Manan Gupta <manan@planetscale.com>
@GuptaManan100 GuptaManan100 merged commit 3ba0aa3 into vitessio:main Jan 17, 2023
@GuptaManan100 GuptaManan100 deleted the blocked-recoveries-fix branch January 17, 2023 03:43
timvaillancourt pushed a commit to slackhq/vitess that referenced this pull request Sep 11, 2023
@GuptaManan100 @timvaillancourt
Fix insert query of blocked_recovery table in VTOrc (vitessio#12091)
dc1948b 
* feat: add failing test and fix the query of insertion

Signed-off-by: Manan Gupta <manan@planetscale.com>

* empty-commit

Signed-off-by: Manan Gupta <manan@planetscale.com>

Signed-off-by: Manan Gupta <manan@planetscale.com>
@timvaillancourt timvaillancourt mentioned this pull request Sep 11, 2023
Closed
4 tasks
timvaillancourt pushed a commit to slackhq/vitess that referenced this pull request Mar 15, 2024
@GuptaManan100 @timvaillancourt
Fix insert query of blocked_recovery table in VTOrc (vitessio#12091)
a1ae5b3 
* feat: add failing test and fix the query of insertion

Signed-off-by: Manan Gupta <manan@planetscale.com>

* empty-commit

Signed-off-by: Manan Gupta <manan@planetscale.com>

Signed-off-by: Manan Gupta <manan@planetscale.com>
@timvaillancourt timvaillancourt mentioned this pull request Mar 15, 2024
Merged
4 tasks
timvaillancourt added a commit to slackhq/vitess that referenced this pull request May 16, 2024
@timvaillancourt @GuptaManan100 @dbussink
slack-vitess-r15.0.5: backport v16 vtorc fixes (#258)
36d8315 
* VTOrc running PRS when database_instance empty bug fix. (vitessio#12019)

* feat: convert join with database_instance to a left join and prevent fixes from running if the information from database_instance is unavailable

Signed-off-by: Manan Gupta <manan@planetscale.com>

* test: add tests to verify the fix works

Signed-off-by: Manan Gupta <manan@planetscale.com>

Signed-off-by: Manan Gupta <manan@planetscale.com>

* Timeout Fixes and VTOrc Improvement (vitessio#11881)

* refactor: move tests out of newfeaturestest so that they run on upgrade-downgrade tests too

Signed-off-by: Manan Gupta <manan@planetscale.com>

* feat: add failing ers test for handling multiple vttablet failures with default values of flags

Signed-off-by: Manan Gupta <manan@planetscale.com>

* feat: add a new lock-timeout flag and use that instead of remote-operation-timeout

Signed-off-by: Manan Gupta <manan@planetscale.com>

* feat: augment DownPrimary test to reproduce the issue of VTOrc not handling multiple failures

Signed-off-by: Manan Gupta <manan@planetscale.com>

* feat: remove LockShardTimeout configuration from VTOrc and add parallelism to refresh of tablets

Signed-off-by: Manan Gupta <manan@planetscale.com>

* log: add more logging lines around ers in vtorc

Signed-off-by: Manan Gupta <manan@planetscale.com>

* test: get the test to work

Signed-off-by: Manan Gupta <manan@planetscale.com>

* feat: fix usage of wait for replicas timeout

Signed-off-by: Manan Gupta <manan@planetscale.com>

* test: fix flags expected output

Signed-off-by: Manan Gupta <manan@planetscale.com>

* test: fix race in test now that the function is called in parallel multiple times

Signed-off-by: Manan Gupta <manan@planetscale.com>

* feat: fix default of onCloseTimeout to 1 second

Signed-off-by: Manan Gupta <manan@planetscale.com>

* test: add failing unit test to refreshTabletsInKeyspaceShard

Signed-off-by: Manan Gupta <manan@planetscale.com>

* feat: fix vtorc to not forget a tablet which has been deleted

Signed-off-by: Manan Gupta <manan@planetscale.com>

* feat: fix backward compatibility, add tests and release notes docs

Signed-off-by: Manan Gupta <manan@planetscale.com>

* test: fix flags output

Signed-off-by: Manan Gupta <manan@planetscale.com>

* test: use disable-replication-manager instead of disable-active-reparents to allow vttablets to setup replication when restarted

Signed-off-by: Manan Gupta <manan@planetscale.com>

* test: fix flaky test by not checking for an error

Signed-off-by: Manan Gupta <manan@planetscale.com>

* feat: handle the case of empty hostname in tablet initialization

Signed-off-by: Manan Gupta <manan@planetscale.com>

* feat: update onclose timeout to 10 seconds

Signed-off-by: Manan Gupta <manan@planetscale.com>

* test: fix unit test

Signed-off-by: Manan Gupta <manan@planetscale.com>

* feat: address review comments

Signed-off-by: Manan Gupta <manan@planetscale.com>

* docs: add comments explaining the test functions

Signed-off-by: Manan Gupta <manan@planetscale.com>

* feat: add summary docs for 'lock-shard-timeout' deprecation

Signed-off-by: Manan Gupta <manan@planetscale.com>

Signed-off-by: Manan Gupta <manan@planetscale.com>

* log: also log error in DiscoverInstance when force discovery is specified (vitessio#11936)

Signed-off-by: Manan Gupta <manan@planetscale.com>

Signed-off-by: Manan Gupta <manan@planetscale.com>

* VTOrc Code Cleanup - generate_base, replace cluster_name with keyspace and shard. (vitessio#12012)

* feat: refactor generate commands of VTOrc to be in a single file

Signed-off-by: Manan Gupta <manan@planetscale.com>

* refactor: cleanup create table formatting

Signed-off-by: Manan Gupta <manan@planetscale.com>

* feat: cleanup the usage of IsSQLite and IsMySQL

Signed-off-by: Manan Gupta <manan@planetscale.com>

* feat: remove unused minimal instance

Signed-off-by: Manan Gupta <manan@planetscale.com>

* feat: remove unused table cluster_domain_name

Signed-off-by: Manan Gupta <manan@planetscale.com>

* feat: fix vtorc database to store keyspace and shard instead of cluster

Signed-off-by: Manan Gupta <manan@planetscale.com>

* feat: remove unused attributes

Signed-off-by: Manan Gupta <manan@planetscale.com>

* feat: remove unused cluster domain

Signed-off-by: Manan Gupta <manan@planetscale.com>

* feat: change GetClusterName to GetKeyspaceAndShardName

Signed-off-by: Manan Gupta <manan@planetscale.com>

* feat: fix insertion into database_instance

Signed-off-by: Manan Gupta <manan@planetscale.com>

* feat: fix SnapshotTopologies

Signed-off-by: Manan Gupta <manan@planetscale.com>

* feat: remove inject unseen primary and inject seed

Signed-off-by: Manan Gupta <manan@planetscale.com>

* feat: remove ClusterName from Instance

Signed-off-by: Manan Gupta <manan@planetscale.com>

* feat: fix Audit operations

Signed-off-by: Manan Gupta <manan@planetscale.com>

* feat: add Keyspace and Shard to cluster information to replace ClusterName

Signed-off-by: Manan Gupta <manan@planetscale.com>

* feat: fix attempt failure detection registeration

Signed-off-by: Manan Gupta <manan@planetscale.com>

* feat: fix blocked topology recoveries

Signed-off-by: Manan Gupta <manan@planetscale.com>

* feat: fix topology recovery

Signed-off-by: Manan Gupta <manan@planetscale.com>

* feat: reading recovery instances

Signed-off-by: Manan Gupta <manan@planetscale.com>

* feat: fix get replication and analysis

Signed-off-by: Manan Gupta <manan@planetscale.com>

* feat: fix bug in query

Signed-off-by: Manan Gupta <manan@planetscale.com>

* test: add tests to check that filtering by keyspace works for APIs

Signed-off-by: Manan Gupta <manan@planetscale.com>

* feat: remove remaining usages of ClusterName

Signed-off-by: Manan Gupta <manan@planetscale.com>

* refactor: fix comment explaining sleep in the test

Signed-off-by: Manan Gupta <manan@planetscale.com>

* feat: add code to prevent filtering just by shard and add tests for it

Signed-off-by: Manan Gupta <manan@planetscale.com>

Signed-off-by: Manan Gupta <manan@planetscale.com>

* Fix insert query of blocked_recovery table in VTOrc (vitessio#12091)

* feat: add failing test and fix the query of insertion

Signed-off-by: Manan Gupta <manan@planetscale.com>

* empty-commit

Signed-off-by: Manan Gupta <manan@planetscale.com>

Signed-off-by: Manan Gupta <manan@planetscale.com>

* Fix: VTOrc forgetting old instances (vitessio#12089)

* test: add a failing test for the case where the port changes for a tablet

Signed-off-by: Manan Gupta <manan@planetscale.com>

* feat: fix the issue by adding alias as a unique field

Signed-off-by: Manan Gupta <manan@planetscale.com>

* empty-commit

Signed-off-by: Manan Gupta <manan@planetscale.com>

Signed-off-by: Manan Gupta <manan@planetscale.com>

* Move vtorc from go-sqlite3 to modernc.org/sqlite (vitessio#12214)

* Move vtorc from go-sqlite3 to modernc.org/sqlite

This moves vtorc from the go-sqlite3 library that uses CGO, to use
modernc.org/sqlite which is a pure Go implementation.

vtorc is the only component we have to build with CGO but it's causing
pain for releases since we need to build it against an old Linux for
linking against glibc.

Using modernc.org/sqlite allows for using Go only again and makes all
Vitess components buildable without CGO.

In
https://datastation.multiprocess.io/blog/2022-05-12-sqlite-in-go-with-and-without-cgo.html
someone ran some basic benchmarks. It shows that the pure Go version can
be twice as slow, but the usage of vtorc is very limited and we operate
on small datasets, so I think the performance impact purely of a
somewhat slower sqlite implementation is negligable.

None of this is in a hot query serving path or anything like that, so I
have little concern performance wise.

Signed-off-by: Dirkjan Bussink <d.bussink@gmail.com>

* Fix error handling in RowToArray

Signed-off-by: Dirkjan Bussink <d.bussink@gmail.com>

---------

Signed-off-by: Dirkjan Bussink <d.bussink@gmail.com>

* see if CI passes on v14.0.5 as previous release

Signed-off-by: Tim Vaillancourt <tim@timvaillancourt.com>

* Revert "see if CI passes on v14.0.5 as previous release"

This reverts commit 53a0e0c.

---------

Signed-off-by: Manan Gupta <manan@planetscale.com>
Signed-off-by: Dirkjan Bussink <d.bussink@gmail.com>
Signed-off-by: Tim Vaillancourt <tim@timvaillancourt.com>
Co-authored-by: Manan Gupta <35839558+GuptaManan100@users.noreply.github.com>
Co-authored-by: Dirkjan Bussink <d.bussink@gmail.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@mattlord mattlord mattlord approved these changes

@deepthi deepthi deepthi approved these changes

@shlomi-noach shlomi-noach Awaiting requested review from shlomi-noach shlomi-noach is a code owner

@rsajwani rsajwani Awaiting requested review from rsajwani

Assignees
No one assigned
Labels
Component: VTorc Vitess Orchestrator integration Type: Bug
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.

Bug Report: VTOrc insertions into Blocked recoveries table is broken
3 participants
@GuptaManan100 @deepthi @mattlord