Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Sorter go routine size #3

Open
wants to merge 76 commits into
base: reduce-go-routine-usage
Choose a base branch
from
Open

Sorter go routine size #3

wants to merge 76 commits into from

Conversation

sdojjy
Copy link
Owner

@sdojjy sdojjy commented Jan 24, 2022

What problem does this PR solve?

Issue Number: close #xxx

What is changed and how it works?

Check List

Tests

  • Unit test
  • Integration test
  • Manual test (add detailed scripts or steps below)
  • No code

Code changes

  • Has exported function/method change
  • Has exported variable/fields change
  • Has interface methods change
  • Has persistent data change

Side effects

  • Possible performance regression
  • Increased code complexity
  • Breaking backward compatibility

Related changes

  • Need to cherry-pick to the release branch
  • Need to update the documentation
  • Need to update key monitor metrics in both TiCDC document and official document

Release note 

Please add a release note.
If you don't think this PR needs a release note then fill it with `None`.

@sdojjy sdojjy changed the base branch from master to reduce-go-routine-usage January 24, 2022 02:44
sdojjy and others added 23 commits January 24, 2022 11:03
@sdojjy
Merge branch 'reduce-go-routine-usage' into sorter-go-routine-size
3f730cb 
# Conflicts:
#	cdc/sorter/unified/merger.go
@dependabot
build(deps): bump nanoid from 3.1.30 to 3.2.0 in /dm/ui (pingcap#4437)
239a21f 
ref pingcap#4418
@sdojjy
wait merger
e5ee9da
@sdojjy
wait merger
9dfefbb
@sdojjy
pipeline(ticdc): reduce go routine size (pingcap#4334)
0236882 
ref pingcap#4351
@sdojjy
wait merger
7461a8f
@asddongmen
mq (ticdc): Make mq sink flush resolvedTs async (pingcap#4356)
3e43ac0 
ref pingcap#4222
@asddongmen
mq (ticdc): Make mq sink flush resolvedTs async (pingcap#4356)
39aaf99 
ref pingcap#4222
@sdojjy
use ticker instead of timer
a39d7b7
@sdojjy
use once
2b93ae4
@asddongmen
openAPI (ticdc): add some unit tests and migrate related test to test…
b196894 
…ify (pingcap#4387)

ref pingcap#2411
@overvenus
tests(ticdc): disable hibernate-regions-compatible to fix tiflash tes…
7a227b4 
…ts (pingcap#4460)

ref pingcap#4461
@Ehco1996
grafana(dm): fix grafana expr (pingcap#4405)
9d0ec39 
close pingcap#4404
@3AceShowHand
consumer(ticdc): update the logic of consumer. (pingcap#4129)
43a1d78 
close pingcap#4128
@buchuitoudegou @ti-chi-bot
fix: data race (pingcap#4419)
fd5bad5 
Co-authored-by: Ti Chi Robot <ti-community-prow-bot@tidb.io>
@glorv
dm: adjust lightning loader config (pingcap#4034)
990327d 
close pingcap#3519
@asddongmen
Revert "redo (ticdc): make FlushLogs in manager asynchronously (pingc…
180fa4e 
…ap#4301)" (pingcap#4475)

ref pingcap#4222
@overvenus
errors,version(ticdc): refine ErrCheckClusterVersionFromPD error (pin…
f51e410 
…gcap#4435)

close pingcap#4341
@dsdashun
relay/meta(dm): fix potential data races after saving GTID (pingcap#4455
72c5fab 
)

close pingcap#4166
@lance6716
*(dm): add --start-time in DM master (pingcap#4261)
9d7ae48 
ref pingcap#4106
@zhaoxinyu
tests(ticdc): Migrate several test cases to testify (pingcap#4199)
03c5232 
close pingcap#2901
@Rustin170506
docs(ticdc): add design doc for multiple topics (pingcap#4422)
462074a 
ref pingcap#4423
@Rustin170506
github(dm): add release-note-none for robot pull request (pingcap#4481)
c56c1c8 
ref pingcap#4478
overvenus and others added 30 commits February 9, 2022 09:43
@overvenus
.github(ticdc): request TiKV version (pingcap#4535)
7fd749e 
close pingcap#4534
@Ehco1996
metrics(dm): fix lag metrics for ddl never updated (pingcap#4491)
be7b742 
close pingcap#4471
@3AceShowHand
sink(cdc): kafka producer use default configuration. (pingcap#4359)
1c1015b 
close pingcap#4383, close pingcap#4499
@Ehco1996
openapi(dm): add error_msg in task status (pingcap#4421)
85116e4 
ref pingcap#3583
@overvenus
api,owner(ticdc): return error when api fails (pingcap#4494)
e2d529c 
close pingcap#1710, ref pingcap#3456
@sdojjy
Merge branch 'master' into sorter-go-routine-size
b38ef9c
@lance6716
syncer(dm): use DML library (pingcap#4313)
edad7da 
ref pingcap#3242
@Rustin170506
tests(ticdc): fix canal_json_basic charset issue (pingcap#4558)
7bcfae4 
ref pingcap#4555
@3AceShowHand
cdc/metrics: Integrate sarama producer metrics (pingcap#4520)
8a709d7 
close pingcap#4561
@lance6716
syncer(dm): fix startGTID is equal to endGTID (pingcap#4386)
c48f208 
ref pingcap#4287
@overvenus
docs/media(ticdc): update cdc architecture (pingcap#4536)
f69f168 
close pingcap#4564
@asddongmen
etcd (ticdc): fix a data race in unit test (pingcap#4551)
fc3b5ad 
close pingcap#4549
@okJiang
checker(dm): support concurrent check (pingcap#3975)
c3e8be9 
close pingcap#3974
@D3Hunter
validation(dm): manage validator life cycle (pingcap#4479)
52f0ade 
close pingcap#4459
@Ehco1996
test(dm): fix unsteable test TestRedirectRequestToLeader (pingcap#4556)
e230160 
close pingcap#4554
@Ehco1996
test(dm): fix unsteable test TestFailToStartLeader (pingcap#4566)
7ccbdff 
close pingcap#4565
@lance6716
syncer(dm): implement start-task --start-time (pingcap#4485)
803aa92 
close pingcap#4106
@okJiang
checker(dm): support optimistic checker (pingcap#4329)
1fb41f9 
close pingcap#4328
sink(ticdc): fix Unknown system variable 'transaction_isolation' fo…
67a3029 
…r some mysql versions (pingcap#4569)

close pingcap#4504
@3AceShowHand
cdc/sink: kafka sink integrate broker level metrics (pingcap#4517)
28fe713 
close pingcap#4588
@CharlesCheung96
test(ticdc): migrate test-infra to testify for cdc/kv pkg (pingcap#2899…
9ddc713 
…) (pingcap#4515)

close pingcap#2899
@overvenus
pkg/actor(ticdc): reduce metrics overhead (pingcap#4585)
87cfd44 
close pingcap#4584
@Rustin170506
ticdc(sink): remove useless AppendResolvedEvent (pingcap#4596)
2293dc2 
ref pingcap#4423
@buchuitoudegou
dm: new router compatible with regular expression (pingcap#4358)
536d8ae 
close pingcap#4256
@lance6716
loader(dm): skip feature for nil etcd client (pingcap#4599)
744a26e 
ref pingcap#4287
@Ehco1996
unit(dm): add Kill func for unit (pingcap#4035)
e7b0aae 
close pingcap#3737
@3AceShowHand
cdc/sink: decouple opt out of statistics (pingcap#4606)
1c3bf68 
close pingcap#4607
@Rustin170506
ticdc(mounter): remove unnecessary delete event decode logic (pingcap…
2f4b706 
…#4602)

close pingcap#4582
@sdojjy
Merge branch 'master' into sorter-go-routine-size
e9cb7ab
@sdojjy
revert client
2592e98
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers
No reviews
Assignees
No one assigned
Labels
None yet
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.