*: update ks3 SDK and add some workaround for ks3 SDK #48277

lance6716 · 2023-11-03T08:08:34Z

What problem does this PR solve?

Issue Number: close #48278 close #48318

Problem Summary:

What is changed and how it works?

Check List

Tests

Unit test
Integration test
Manual test (add detailed scripts or steps below)
No need to test
- I checked and no code files have been changed.

Side effects

Performance regression: Consumes more CPU
Performance regression: Consumes more Memory
Breaking backward compatibility

Documentation

Release note

Please refer to Release Notes Language Style Guide to write a quality release note.

None

Signed-off-by: lance6716 <lance6716@gmail.com>

tiprow · 2023-11-03T08:08:50Z

Hi @lance6716. Thanks for your PR.

PRs from untrusted users cannot be marked as trusted with /ok-to-test in this repo meaning untrusted PR authors can never trigger tests themselves. Collaborators can still trigger tests on the PR using /test all.

I understand the commands that are listed here.

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes/test-infra repository.

lance6716 · 2023-11-03T08:11:24Z

/cc @D3Hunter @wjhuang2016

Signed-off-by: lance6716 <lance6716@gmail.com>

codecov · 2023-11-03T08:34:47Z

Codecov Report

Merging #48277 (b6d3929) into master (f04ed92) will increase coverage by 1.7306%.
Report is 38 commits behind head on master.
The diff coverage is 0.0000%.

Additional details and impacted files

@@               Coverage Diff                @@
##             master     #48277        +/-   ##
================================================
+ Coverage   71.4467%   73.1774%   +1.7306%     
================================================
  Files          1402       1425        +23     
  Lines        406693     416601      +9908     
================================================
+ Hits         290569     304858     +14289     
+ Misses        96219      92940      -3279     
+ Partials      19905      18803      -1102

Flag	Coverage Δ
integration	`43.9389% <0.0000%> (?)`
unit	`71.8501% <0.0000%> (+0.4033%)`	⬆️

Flags with carried forward coverage won't be shown. Click here to find out more.

Components	Coverage Δ
dumpling	`53.9874% <ø> (ø)`
parser	`∅ <ø> (∅)`
br	`48.6377% <0.0000%> (-4.2850%)`	⬇️

Signed-off-by: lance6716 <lance6716@gmail.com>

lance6716 · 2023-11-06T06:38:58Z

/retest

tiprow · 2023-11-06T06:39:19Z

@lance6716: Cannot trigger testing until a trusted user reviews the PR and leaves an /ok-to-test message.

In response to this:

/retest

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes/test-infra repository.

lance6716 · 2023-11-06T07:42:14Z

/build

D3Hunter · 2023-11-06T08:41:59Z

pkg/executor/importer/import.go

 		if err != nil {
 			return err
 		}
 		e.GlobalSortStore = s
 	}
 	return nil
 }
+
+func (*LoadDataController) initSelfManagedExternalStore(ctx context.Context, u *url.URL, target string) (storage.ExternalStorage, error) {


what you mean by self-managed?

s3 does not belong to user, it belongs to our cloud infra

Signed-off-by: lance6716 <lance6716@gmail.com>

This reverts commit 1784b5f.

Signed-off-by: lance6716 <lance6716@gmail.com>

ywqzzy · 2023-11-09T03:08:02Z

/lgtm

ti-chi-bot · 2023-11-09T03:08:12Z

[LGTM Timeline notifier]

Timeline:

2023-11-07 03:34:35.332154119 +0000 UTC m=+3528872.919264265: ☑️ agreed by D3Hunter.
2023-11-09 03:08:11.635363667 +0000 UTC m=+3700089.222473813: ☑️ agreed by ywqzzy.

ti-chi-bot · 2023-11-09T03:15:38Z

[APPROVALNOTIFIER] This PR is APPROVED

This pull-request has been approved by: 3pointer, D3Hunter, ywqzzy

The full list of commands accepted by this bot can be found here.

The pull request process is described here

Needs approval from an approver in each of these files:

~~OWNERS~~ [3pointer,D3Hunter]
~~br/OWNERS~~ [3pointer]

Approvers can indicate their approval by writing /approve in a comment
Approvers can cancel approval by writing /approve cancel in a comment

lance6716 · 2023-11-09T04:08:59Z

/retest

tiprow · 2023-11-09T04:09:22Z

@lance6716: Cannot trigger testing until a trusted user reviews the PR and leaves an /ok-to-test message.

In response to this:

/retest

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes/test-infra repository.

lance6716 · 2023-11-09T04:12:57Z

/retest

tiprow · 2023-11-09T04:13:20Z

@lance6716: Cannot trigger testing until a trusted user reviews the PR and leaves an /ok-to-test message.

In response to this:

/retest

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes/test-infra repository.

ti-chi-bot · 2023-11-09T04:35:55Z

In response to a cherrypick label: new pull request created to branch release-7.5: #48459.

close pingcap#48278, close pingcap#48318

*: disable SSL when the s3 endpoint is self managed

1784b5f

Signed-off-by: lance6716 <lance6716@gmail.com>

ti-chi-bot bot added do-not-merge/needs-linked-issue do-not-merge/needs-tests-checked release-note-none Denotes a PR that doesn't merit a release note. size/M Denotes a PR that changes 30-99 lines, ignoring generated files. labels Nov 3, 2023

ti-chi-bot bot added needs-cherry-pick-release-7.5 Should cherry pick this PR to release-7.5 branch. and removed do-not-merge/needs-linked-issue do-not-merge/needs-tests-checked labels Nov 3, 2023

ti-chi-bot bot requested review from D3Hunter and wjhuang2016 November 3, 2023 08:11

fix bazel

d8df954

Signed-off-by: lance6716 <lance6716@gmail.com>

lance6716 added 3 commits November 6, 2023 14:09

fix panic

5879d6e

Signed-off-by: lance6716 <lance6716@gmail.com>

upgrade ks3 library

4c3a551

Signed-off-by: lance6716 <lance6716@gmail.com>

fix bazel

04d7f86

Signed-off-by: lance6716 <lance6716@gmail.com>

lance6716 changed the title *: disable SSL when the s3 endpoint is self managed *: update ks3 SDK, disable SSL when the s3 endpoint is self managed Nov 6, 2023

Merge remote-tracking branch 'upstream/master' into disable-ssl

41e8615

D3Hunter reviewed Nov 6, 2023

View reviewed changes

fix SDK error

73afea0

Signed-off-by: lance6716 <lance6716@gmail.com>

D3Hunter approved these changes Nov 7, 2023

View reviewed changes

ti-chi-bot bot added the needs-1-more-lgtm Indicates a PR needs 1 more LGTM. label Nov 7, 2023

D3Hunter mentioned this pull request Nov 7, 2023

merge-sort step pre-alloc too many memory #48342

Closed

Revert "*: disable SSL when the s3 endpoint is self managed"

04c9f59

This reverts commit 1784b5f.

ti-chi-bot bot added size/S Denotes a PR that changes 10-29 lines, ignoring generated files. and removed size/M Denotes a PR that changes 30-99 lines, ignoring generated files. labels Nov 8, 2023

fix bazel

b6d3929

Signed-off-by: lance6716 <lance6716@gmail.com>

lance6716 changed the title *: update ks3 SDK, disable SSL when the s3 endpoint is self managed *: update ks3 SDK and add some workaround for ks3 SDK Nov 9, 2023

ti-chi-bot bot added lgtm and removed needs-1-more-lgtm Indicates a PR needs 1 more LGTM. labels Nov 9, 2023

3pointer approved these changes Nov 9, 2023

View reviewed changes

ti-chi-bot bot added the approved label Nov 9, 2023

ti-chi-bot bot merged commit 75e0253 into pingcap:master Nov 9, 2023
24 of 27 checks passed

ti-chi-bot mentioned this pull request Nov 9, 2023

*: update ks3 SDK and add some workaround for ks3 SDK (#48277) #48459

Closed

13 tasks

ti-chi-bot pushed a commit to ti-chi-bot/tidb that referenced this pull request Nov 9, 2023

*: update ks3 SDK and add some workaround for ks3 SDK (pingcap#48277)

b9283ed

close pingcap#48278, close pingcap#48318

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

*: update ks3 SDK and add some workaround for ks3 SDK #48277

*: update ks3 SDK and add some workaround for ks3 SDK #48277

lance6716 commented Nov 3, 2023 •

edited

Loading

tiprow bot commented Nov 3, 2023

lance6716 commented Nov 3, 2023

codecov bot commented Nov 3, 2023 •

edited

Loading

lance6716 commented Nov 6, 2023

tiprow bot commented Nov 6, 2023

lance6716 commented Nov 6, 2023

D3Hunter Nov 6, 2023

lance6716 Nov 6, 2023

ywqzzy commented Nov 9, 2023

ti-chi-bot bot commented Nov 9, 2023

ti-chi-bot bot commented Nov 9, 2023

lance6716 commented Nov 9, 2023

tiprow bot commented Nov 9, 2023

lance6716 commented Nov 9, 2023

tiprow bot commented Nov 9, 2023

ti-chi-bot commented Nov 9, 2023

*: update ks3 SDK and add some workaround for ks3 SDK #48277

*: update ks3 SDK and add some workaround for ks3 SDK #48277

Conversation

lance6716 commented Nov 3, 2023 • edited Loading

What problem does this PR solve?

What is changed and how it works?

Check List

Release note

tiprow bot commented Nov 3, 2023

lance6716 commented Nov 3, 2023

codecov bot commented Nov 3, 2023 • edited Loading

Codecov Report

lance6716 commented Nov 6, 2023

tiprow bot commented Nov 6, 2023

lance6716 commented Nov 6, 2023

D3Hunter Nov 6, 2023

Choose a reason for hiding this comment

lance6716 Nov 6, 2023

Choose a reason for hiding this comment

ywqzzy commented Nov 9, 2023

ti-chi-bot bot commented Nov 9, 2023

[LGTM Timeline notifier]

ti-chi-bot bot commented Nov 9, 2023

lance6716 commented Nov 9, 2023

tiprow bot commented Nov 9, 2023

lance6716 commented Nov 9, 2023

tiprow bot commented Nov 9, 2023

ti-chi-bot commented Nov 9, 2023

lance6716 commented Nov 3, 2023 •

edited

Loading

codecov bot commented Nov 3, 2023 •

edited

Loading