Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

unify all S3 compliant storage to support backup and restore #1088

Merged
merged 8 commits into from
Nov 6, 2019
Merged

unify all S3 compliant storage to support backup and restore #1088

merged 8 commits into from
Nov 6, 2019

Conversation

onlymellb
Copy link
Contributor

@onlymellb onlymellb commented Oct 31, 2019 •
edited
Loading

What problem does this PR solve?

Resolve #1038

What is changed and how does it work?

Check List

Tests

  • Unit test
  • Manual test

Code changes

  • Has Go code change

Side effects

  • Breaking backward compatibility

Related changes

  • Need to cherry-pick to the release branch
  • Need to update the documentation

Does this PR introduce a user-facing change?:

Unify all S3 compliant storage to support backup and restore

@onlymellb onlymellb added area/backup cloud/aws Amazon Web Services status/PTAL PR needs to be reviewed labels Nov 1, 2019
@onlymellb onlymellb added this to the v1.1.0 milestone Nov 1, 2019
@onlymellb
Copy link
Contributor Author

/run-e2e-in-kind

@cofyc
Copy link
Contributor

cofyc commented Nov 3, 2019 

[2019-11-03T03:33:47.892Z] + '[' -f tests/images/test-apiserver/bin/tidb-apiserver ']'
[2019-11-03T03:33:47.892Z] + docker build -t localhost:5000/pingcap/test-apiserver:08c8ca3 tests/images/test-apiserver
[2019-11-03T03:33:48.148Z] unable to prepare context: unable to evaluate symlinks in Dockerfile path: lstat /home/jenkins/workspace/operator_ghpr_e2e_test_kind/go/src/github.com/pingcap/tidb-operator/tests/images/test-apiserver/Dockerfile: no such file or directory

cc @aylei

@onlymellb
Copy link
Contributor Author

I have fixed this problem.

@tennix tennix requested a review from DanielZhangQD November 4, 2019 03:24
@aylei
Copy link
Contributor

aylei commented Nov 4, 2019

/run-e2e-in-kind

1 similar comment
@aylei
Copy link
Contributor

aylei commented Nov 4, 2019

/run-e2e-in-kind

@onlymellb
Copy link
Contributor Author

/run-e2e-in-kind

LinuxGit
LinuxGit reviewed Nov 6, 2019
View reviewed changes
pkg/apis/pingcap/v1alpha1/types.go Outdated

// +k8s:openapi-gen=true
// S3StoregeProviderType represents the specific storage provider that implements the S3 interface
type S3StoregeProviderType string
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

S3StoregeProviderType typo Storage

shuijing198799
shuijing198799 previously approved these changes Nov 6, 2019
View reviewed changes
Copy link
Contributor

@shuijing198799 shuijing198799 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

@shuijing198799
Copy link
Contributor

/run-e2e-in-kind

@shuijing198799 shuijing198799 self-requested a review November 6, 2019 02:47
LinuxGit
LinuxGit reviewed Nov 6, 2019
View reviewed changes
pkg/apis/pingcap/v1alpha1/types.go Outdated
Endpoint string `json:"endpoint"`
// StorageClass represents the storage class
StorageClass string `json:"storageClass"`
Copy link
Contributor

@LinuxGit LinuxGit Nov 6, 2019 •
edited
Loading

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Why does S3 object storage need StorageClass field?
If Provider is ceph, what values should Acl and StorageClass fields be set?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Because s3 supports many different types of storage, if you do not set this parameter, we set it to STANDARD_IA by default. If provider is ceph, these fields do not need to be set.

LinuxGit
LinuxGit approved these changes Nov 6, 2019
View reviewed changes
Copy link
Contributor

@LinuxGit LinuxGit left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

@onlymellb
Copy link
Contributor Author

/run-e2e-in-kind

aylei
aylei approved these changes Nov 6, 2019
View reviewed changes
Copy link
Contributor

@aylei aylei left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

@onlymellb onlymellb merged commit 2a9ff5d into pingcap:master Nov 6, 2019
@onlymellb onlymellb deleted the support-backup-to-aws branch November 6, 2019 04:20
shuijing198799
shuijing198799 approved these changes Nov 6, 2019
View reviewed changes
Copy link
Contributor

@shuijing198799 shuijing198799 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@shuijing198799 shuijing198799 shuijing198799 approved these changes

@aylei aylei aylei approved these changes

@LinuxGit LinuxGit LinuxGit approved these changes

@weekface weekface Awaiting requested review from weekface

@tennix tennix Awaiting requested review from tennix

@DanielZhangQD DanielZhangQD Awaiting requested review from DanielZhangQD

Assignees
No one assigned
Labels
area/backup cloud/aws Amazon Web Services status/PTAL PR needs to be reviewed
Projects
None yet
Milestone
v1.1.0
Development

Successfully merging this pull request may close these issues.

support backup to AWS S3 and restore from it
5 participants
@onlymellb @cofyc @aylei @shuijing198799 @LinuxGit