Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Parameterize compress/decompress commands #624

Merged
merged 1 commit into from
Nov 18, 2020

Conversation

stankevich
Copy link
Contributor

@stankevich stankevich commented Nov 3, 2020

An improved version of #517.

Adds backupCompressCommand and backupDecompressCommand options that allow using any compressor/decompressor for backups. These options support arbitrary command line args as well.

Falls back to using gzip if the options are not provided.

---
apiVersion: mysql.presslabs.org/v1alpha1
kind: MysqlCluster
metadata:
  name: example
spec:
  # [...]
  backupCompressCommand:
    - pigz # https://zlib.net/pigz/
    - --processes 8
    - --stdout
  backupDecompressCommand:
    - pigz
    - --decompress
  # [...]

  • I've made sure the Changelog.md will remain up-to-date after this PR is merged.

Copy link
Contributor

@AMecea AMecea left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

You should include COMPRESS_COMMAND env on the sidecar container because new replicas (scaling-up) are added using the same mechanism as backups.

Let's check if only one of those field is specified and warn the user if so (logging the message should be enough).

Dockerfile.sidecar Show resolved Hide resolved
pkg/apis/mysql/v1alpha1/mysqlcluster_types.go Outdated Show resolved Hide resolved
@stankevich stankevich force-pushed the parameterize-compress-commands branch 2 times, most recently from 71f13a6 to 0d59685 Compare November 5, 2020 13:21
@stankevich
Copy link
Contributor Author

stankevich commented Nov 5, 2020

You should include COMPRESS_COMMAND env on the sidecar container

I might be missing something, but it looks like the sidecar container is the server and it simply runs xtrabackup without any compression. Adding a new replica streams an uncompressed backup directly to xbstream. Compression is currently only used in pushBackupFromTo, and decompression in cloneFromBucket. I have a local branch that enables compression when streaming but haven't had a chance to test it in somewhat real conditions yet.

check if only one of those field is specified and warn the user if so

Done: 0d59685

Copy link
Contributor

@AMecea AMecea left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM 👍

Just one thing, please squash commits to keep a cleaner history. 😄
Thank you for your contribution!

@stankevich
Copy link
Contributor Author

please squash commits to keep a cleaner history

Of course! Done.

@AMecea AMecea merged commit 156afd9 into bitpoke:master Nov 18, 2020
@cndoit18 cndoit18 linked an issue Sep 17, 2021 that may be closed by this pull request
@cndoit18 cndoit18 removed a link to an issue Sep 17, 2021
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants