Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Ensure that binary logs for PITR are in a shared directory #541

Open
wants to merge 27 commits into
base: main
Choose a base branch
from
Open
Show file tree
Hide file tree
Changes from 1 commit
Commits
Show all changes
27 commits
Select commit Hold shift + click to select a range
d259730
Ensure that binary logs for PITR are restored to a shared location
mattlord Mar 12, 2024
e2e5e8b
Add ability to specify flags for mysqlctld
mattlord Mar 16, 2024
9992074
Merge remote-tracking branch 'origin/main' into point-in-time-recovery
mattlord Mar 26, 2024
10a9524
It's a tablet flag not a mysqlctld one
mattlord Mar 26, 2024
af6ebcb
Merge remote-tracking branch 'origin/main' into point-in-time-recovery
mattlord Mar 26, 2024
48ba49f
Move the flag handling code
mattlord Mar 27, 2024
411993c
Only set flag on 2.12+
mattlord Mar 27, 2024
79bf28d
Add PITR restore to backup test
mattlord Mar 27, 2024
8277844
Use recent vtctldclient
mattlord Mar 27, 2024
9d2b978
Use different date command
mattlord Mar 27, 2024
db311ed
Use different date format
mattlord Mar 27, 2024
63abf80
Test incremental backup and restore
mattlord Mar 27, 2024
425b725
WiP
mattlord Mar 28, 2024
75c701d
attempt to fix tests
shlomi-noach Mar 28, 2024
f09d7be
more debug output
shlomi-noach Mar 28, 2024
7716135
more debug info
shlomi-noach Mar 28, 2024
3452359
list backups as debug step
shlomi-noach Mar 28, 2024
b1a0cb3
GetBackups
shlomi-noach Mar 28, 2024
919119b
Get test working locally
mattlord Mar 28, 2024
49b9332
Ignore vtdataroot path created when running local tests
mattlord Apr 10, 2024
5c809cc
Merge remote-tracking branch 'origin/main' into point-in-time-recovery
mattlord Apr 10, 2024
2aeb36b
Fix backup test; use percona image
mattlord Apr 10, 2024
573b44f
Merge remote-tracking branch 'origin/main' into point-in-time-recovery
mattlord Apr 30, 2024
2f7768a
Use /vt/bin for mysqlbinlog and add it to PATH
mattlord May 1, 2024
34b93ad
Merge remote-tracking branch 'origin/main' into point-in-time-recovery
mattlord Jul 12, 2024
ce28c33
Add mysqlbinlog to $VTROOT/bin in backup pods
mattlord Jul 12, 2024
5e7df2b
Testing...
mattlord Jul 12, 2024
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
18 changes: 10 additions & 8 deletions pkg/operator/vttablet/pod.go
Original file line number Diff line number Diff line change
Expand Up @@ -26,7 +26,6 @@ import (
"k8s.io/apimachinery/pkg/util/intstr"
"k8s.io/utils/pointer"
"planetscale.dev/vitess-operator/pkg/operator/mysql"
"planetscale.dev/vitess-operator/version"
"sigs.k8s.io/controller-runtime/pkg/client"
topodatapb "vitess.io/vitess/go/vt/proto/topodata"
"vitess.io/vitess/go/vt/topo/topoproto"
Expand Down Expand Up @@ -93,14 +92,17 @@ func UpdatePod(obj *corev1.Pod, spec *Spec) {
if _, ok := vttabletAllFlags["builtinbackup-incremental-restore-path"]; !ok {
// This flag only exists in 2.12.0 and later as it includes vitess
// 20.0 as a dependency.
parts := strings.Split(version.Version, ".")
if len(parts) > 0 {
major, _ := strconv.Atoi(parts[0])
minor, _ := strconv.Atoi(parts[1])
if major >= 2 && minor >= 12 {
vttabletAllFlags["builtinbackup-incremental-restore-path"] = vtDataRootPath
/*
parts := strings.Split(version.Version, ".")
if len(parts) > 0 {
major, _ := strconv.Atoi(parts[0])
minor, _ := strconv.Atoi(parts[1])
if major >= 2 && minor >= 12 {
vttabletAllFlags["builtinbackup-incremental-restore-path"] = vtDataRootPath
}
}
}
*/
vttabletAllFlags["builtinbackup-incremental-restore-path"] = vtDataRootPath
}
mysql.UpdateMySQLServerVersion(vttabletAllFlags, spec.Images.Mysqld.Image())

Expand Down
7 changes: 6 additions & 1 deletion test/endtoend/utils.sh
Original file line number Diff line number Diff line change
Expand Up @@ -103,17 +103,22 @@ function takeBackup() {

sleep 10

mysql 'commerce:-' -e "flush binary logs" || echo "flushing binlogs failed"
mattlord marked this conversation as resolved.
Show resolved Hide resolved

# Now perform an incremental backup.
insertWithRetry
INCREMENTAL_RESTORE_TIMESTAMP=$(date -u "+%Y-%m-%dT%H:%M:%SZ")
sleep 1
insertWithRetry

mysql 'commerce:-' -e "flush binary logs" || echo "flushing binlogs failed"

sleep 10

vtctldclient BackupShard --incremental-from-pos=auto "${keyspaceShard}"
let finalBackupCount=${finalBackupCount}+1

mysql 'commerce:-' -e "flush binary logs" || echo "flushing binlogs failed"

for i in {1..600} ; do
out=$(kubectl get vtb --no-headers | wc -l)
if echo "${out}" | grep -c "${finalBackupCount}" >/dev/null; then
Expand Down
Loading