Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

add tls support between drainer and downstream database server #2993

Merged
merged 8 commits into from
Jul 22, 2020

Conversation

lichunzhu
Copy link
Contributor

@lichunzhu lichunzhu commented Jul 20, 2020

What problem does this PR solve?

resolve #2969
add tls support between drainer and downstream database server.

What is changed and how does it work?

add tls syncer part for drainer.

Check List

Tests

  • Unit test
  • E2E test
  • Stability test
  • Manual test (add detailed scripts or steps below)
    • Write to downstream database without tls.
    • Write to downstream database with only downstream tls specified.
    • Write to downstream but checkpoint to another database with/without tls enabled.

Related changes

  • Need to cherry-pick to the release branch
  • Need to update the documentation

Does this PR introduce a user-facing change?:

Add TLS support between drainer and downstream database server

@codecov-commenter
Copy link

codecov-commenter commented Jul 20, 2020

Codecov Report

Merging #2993 into master will increase coverage by 0.22%.
The diff coverage is n/a.

@@            Coverage Diff             @@
##           master    #2993      +/-   ##
==========================================
+ Coverage   41.41%   41.64%   +0.22%     
==========================================
  Files         155      156       +1     
  Lines       16744    16801      +57     
==========================================
+ Hits         6934     6996      +62     
+ Misses       9250     9241       -9     
- Partials      560      564       +4     
Flag Coverage Δ
#unittest 41.64% <ø> (?)

Flags with carried forward coverage won't be shown. Click here to find out more.

@lichunzhu lichunzhu requested a review from weekface July 21, 2020 06:18
@lichunzhu
Copy link
Contributor Author

/test pull-e2e-kind

@lichunzhu
Copy link
Contributor Author

/test pull-e2e-kind-serial

@@ -48,6 +48,22 @@ tlsCluster:
certAllowedCN: []
# - TiDB

# The tls config between drainer and the downstream database server (MySQL/TiDB)
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
# The tls config between drainer and the downstream database server (MySQL/TiDB)
# The TLS config between drainer and the downstream database server (MySQL/TiDB)

@@ -48,6 +48,22 @@ tlsCluster:
certAllowedCN: []
# - TiDB

# The tls config between drainer and the downstream database server (MySQL/TiDB)
tlsSyncer:
tlsClientSecretName:
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Please add comments to explain what's this field is for and what keys should be included in the secret or add an example command as in L41-L43.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggest commenting out the new configurations.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
tlsClientSecretName:
# tlsClientSecretName: ""

tlsSyncer:
tlsClientSecretName:

# certAllowedCN is the Common Name that allowed
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

What's the difference between this one and the one in L64, please make it clear.

certAllowedCN: []
# - TiDB

# checkpoint is the tls config for the database we save binlog checkpoint
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
# checkpoint is the tls config for the database we save binlog checkpoint
# checkpoint is the TLS config for the database we save binlog checkpoint

# - TiDB

# checkpoint is the tls config for the database we save binlog checkpoint
# Omit this part if you just want to save checkpoint in downstream database
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

What does this mean?

# Omit this part if you just want to save checkpoint in downstream database
checkpoint:
tlsClientSecretName:
# certAllowedCN is the Common Name that allowed
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Make it clear for the difference with L56.

@@ -48,6 +48,22 @@ tlsCluster:
certAllowedCN: []
# - TiDB

# The tls config between drainer and the downstream database server (MySQL/TiDB)
tlsSyncer:
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
tlsSyncer:
tlsSyncer: {}

tlsClientSecretName:

# certAllowedCN is the Common Name that allowed
certAllowedCN: []
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
certAllowedCN: []
# certAllowedCN: []

Comment on lines 61 to 62
checkpoint:
tlsClientSecretName:
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
checkpoint:
tlsClientSecretName:
# checkpoint:
# tlsClientSecretName: ""

checkpoint:
tlsClientSecretName:
# certAllowedCN is the Common Name that allowed
certAllowedCN: []
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
certAllowedCN: []
# certAllowedCN: []

weekface
weekface previously approved these changes Jul 22, 2020
Copy link
Contributor

@weekface weekface left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

Please also update the TLS document.

@DanielZhangQD
Copy link
Contributor

/test pull-e2e-kind

Copy link
Contributor

@DanielZhangQD DanielZhangQD left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

@lichunzhu
Copy link
Contributor Author

@weekface PTAL again

@lichunzhu lichunzhu merged commit 741b18e into pingcap:master Jul 22, 2020
@lichunzhu lichunzhu deleted the addTlsForDrainerSyncer branch July 22, 2020 06:26
ti-srebot pushed a commit to ti-srebot/tidb-operator that referenced this pull request Jul 22, 2020
Signed-off-by: ti-srebot <ti-srebot@pingcap.com>
@ti-srebot
Copy link
Contributor

cherry pick to release-1.1 in PR #3000

DanielZhangQD added a commit that referenced this pull request Jul 22, 2020
Signed-off-by: ti-srebot <ti-srebot@pingcap.com>

Co-authored-by: Chunzhu Li <lichunzhu@stu.xjtu.edu.cn>
Co-authored-by: DanielZhangQD <36026334+DanielZhangQD@users.noreply.github.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging this pull request may close these issues.

Support TLS to tidb-binlog drainer downstream
5 participants