Skip to content
This repository has been archived by the owner on Dec 8, 2021. It is now read-only.

Try to create tables in parallel #434

Closed
kennytm opened this issue Oct 28, 2020 · 24 comments · Fixed by #502
Closed

Try to create tables in parallel #434

kennytm opened this issue Oct 28, 2020 · 24 comments · Fixed by #502
Labels
challenge-program difficulty/1-easy Easy issue feature-request This issue is a feature request high-performance picked priority/P3 Low priority issue

Comments

@kennytm
Copy link
Collaborator

kennytm commented Oct 28, 2020 •
edited
Loading

Feature Request

Is your feature request related to a problem? Please describe:

Currently we perform CREATE TABLE (tidbMgr.InitSchema) in sequence. But experiences in BR shows that running them in parallel is faster pingcap/br#377

Describe the feature you'd like:

Execute the CREATE TABLE in restoreSchema in parallel over 16 connections.

Benchmark that by importing 300 small tables.

Describe alternatives you've considered:

Don't do it.

Teachability, Documentation, Adoption, Optimization:

N/A

Score

600

SIG slack channel

sig-migrate

Mentor

@glorv @lance6716
@kennytm kennytm added feature-request This issue is a feature request priority/P3 Low priority issue difficulty/1-easy Easy issue challenge-program hptc labels Oct 28, 2020
@pingcap pingcap deleted a comment from ti-challenge-bot bot Oct 30, 2020
@pingcap pingcap deleted a comment from AndreMouche Oct 30, 2020
@hidehalo
Copy link
Contributor

/pick-up

@ti-challenge-bot
Copy link

This issue does not belong to any SIG.

Details

Tip : Currently, we only support sig labels starting with sig/, maybe you should add this type of label.

Warning: None

@hidehalo
Copy link
Contributor

hidehalo commented Dec 1, 2020

@kennytm shall we label it sig-migrate?

@lance6716
Copy link
Contributor

@kennytm shall we label it sig-migrate?

we're setting sig label for this whole repo in #498 😄

@Rustin170506
Copy link
Member

@kennytm shall we label it sig-migrate?

Could you please try pick up again?

@hidehalo
Copy link
Contributor

hidehalo commented Dec 1, 2020

/pick-up

@ti-challenge-bot
Copy link

This issue does not belong to any SIG.

Details

Tip : Currently, we only support sig labels starting with sig/, maybe you should add this type of label.

Warning: None

@hidehalo
Copy link
Contributor

hidehalo commented Dec 1, 2020

/cc @hi-rustin maybe later?

@ti-challenge-bot
Copy link

This issue does not belong to any SIG.

Details

Tip : Currently, we only support sig labels starting with sig/, maybe you should add this type of label.

Warning: None

@Rustin170506
Copy link
Member

/cc @hi-rustin maybe later?

Try it when #498 megred.

@glorv
Copy link
Contributor

glorv commented Dec 1, 2020

@hidehalo Please try again~

@lance6716
Copy link
Contributor

BTW, there's a similar issue in DM pingcap/dm#717 . We'll be happy if you could take a look after this one (will add learning material and score later)

@hidehalo
Copy link
Contributor

hidehalo commented Dec 1, 2020

/pick-up

@ti-challenge-bot
Copy link

Pick up success.

@hidehalo
Copy link
Contributor

hidehalo commented Dec 1, 2020

BTW, there's a similar issue in DM pingcap/dm#717 . We'll be happy if you could take a look after this one (will add learning material and score later)

Thx!

@hidehalo hidehalo mentioned this issue Dec 1, 2020
Merged
@ti-challenge-bot
Copy link

@hidehalo You did not submit PR within 7 days, so give up automatically.

@ti-challenge-bot ti-challenge-bot bot removed the picked label Dec 8, 2020
@hidehalo
Copy link
Contributor

hidehalo commented Dec 8, 2020

/pick-up

@ti-challenge-bot
Copy link

Pick up success.

@ti-challenge-bot
Copy link

@hidehalo You did not submit PR within 7 days, so give up automatically.

@ti-challenge-bot ti-challenge-bot bot removed the picked label Dec 15, 2020
@hidehalo
Copy link
Contributor

/pick-up

@ti-challenge-bot
Copy link

Pick up success.

@ti-challenge-bot
Copy link

@hidehalo You did not submit PR within 7 days, so give up automatically.

@ti-challenge-bot ti-challenge-bot bot removed the picked label Dec 22, 2020
@hidehalo
Copy link
Contributor

/pick-up

@ti-challenge-bot
Copy link

Pick up success.

Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Assignees
No one assigned
Labels
challenge-program difficulty/1-easy Easy issue feature-request This issue is a feature request high-performance picked priority/P3 Low priority issue
Projects
None yet
Milestone
No milestone
Development

Successfully merging a pull request may close this issue.

restore: Try to create tables in parallel hidehalo/tidb-lightning
6 participants
@kennytm @AndreMouche @lance6716 @glorv @hidehalo @Rustin170506