Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Split into multi-service architecture #5766

Open
18 tasks done
rleungx opened this issue Dec 8, 2022 · 2 comments
Open
18 tasks done

Split into multi-service architecture #5766

rleungx opened this issue Dec 8, 2022 · 2 comments
Labels
report/customer Customers have encountered this bug. type/feature-request Categorizes issue or PR as related to a new feature.

Comments

@rleungx
Copy link
Member

rleungx commented Dec 8, 2022 •
edited by lhy1024
Loading

Feature Request

Describe your feature request related problem

Currently, PD has more and more management services. As TiDB clusters are becoming larger and larger, it is increasingly important to ensure the quality of service on the PD. In most large cluster cases, the main bottleneck of PD is CPU resources, and it is difficult to effectively control and isolate the CPU resources of multiple services in the same golang program. Therefore, we need to split these services into individual ones and deploy them independently for management, which is better to control the use and isolation of resources. Also, the microservice architecture is also very suitable for the cloud environment.

Describe the feature you'd like

One of the most important services which PD provided is the TSO service. Deploy it through an independent binary will improve its QoS.

Task

PreWorks

Teachability, Documentation, Adoption, Migration Strategy
@rleungx rleungx added the type/feature-request Categorizes issue or PR as related to a new feature. label Dec 8, 2022
@nolouch nolouch mentioned this issue Dec 8, 2022
Merged
@rleungx rleungx mentioned this issue Dec 8, 2022
Merged
ti-chi-bot pushed a commit that referenced this issue Dec 8, 2022
@rleungx
pkg: move the election to pkg (#5767)
268c2b6 
ref #5766

Signed-off-by: Ryan Leung <rleungx@gmail.com>
ti-chi-bot pushed a commit that referenced this issue Dec 8, 2022
@nolouch
*: move all util relative package to utils (#5764)
8227cdb 
ref #5766

*: move all util relative package to utils

Signed-off-by: nolouch <nolouch@gmail.com>
This was referenced Dec 12, 2022
Merged
Merged
Merged
ti-chi-bot added a commit that referenced this issue Dec 12, 2022
@rleungx @ti-chi-bot
*: move kv to pkg (#5773)
7ab1c9c 
ref #5766

Signed-off-by: Ryan Leung <rleungx@gmail.com>

Co-authored-by: Ti Chi Robot <ti-community-prow-bot@tidb.io>
ti-chi-bot pushed a commit that referenced this issue Dec 13, 2022
@rleungx
*: move encryptionkm to pkg (#5772)
aa3ab28 
ref #5766

Signed-off-by: Ryan Leung <rleungx@gmail.com>
ti-chi-bot added a commit that referenced this issue Dec 13, 2022
@rleungx @ti-chi-bot
config: remove config dependency for tso (#5771)
65abc36 
ref #5766

Signed-off-by: Ryan Leung <rleungx@gmail.com>

Co-authored-by: Ti Chi Robot <ti-community-prow-bot@tidb.io>
@rleungx rleungx mentioned this issue Dec 13, 2022
Merged
ti-chi-bot added a commit that referenced this issue Dec 13, 2022
@rleungx @ti-chi-bot
*: move member to pkg (#5778)
ad1cc2d 
ref #5766

Signed-off-by: Ryan Leung <rleungx@gmail.com>

Co-authored-by: Ti Chi Robot <ti-community-prow-bot@tidb.io>
@nolouch nolouch mentioned this issue Dec 13, 2022
Merged
@rleungx rleungx mentioned this issue Dec 14, 2022
Merged
ti-chi-bot pushed a commit that referenced this issue Dec 14, 2022
@rleungx
*: move id and version to pkg (#5780)
84c0f82 
ref #5766

Signed-off-by: Ryan Leung <rleungx@gmail.com>
ti-chi-bot pushed a commit that referenced this issue Dec 14, 2022
@nolouch
*: Introduce the service registry (#5779)
6ada022 
ref #5766

Introduce the service registry:
- Provide a new way to initial external service
- Supports gRPC and HTTP API both

Signed-off-by: nolouch <nolouch@gmail.com>
This was referenced Dec 15, 2022
Merged
Merged
@nolouch
Copy link
Contributor

nolouch commented Dec 19, 2022

PTAL @rleungx, The region Informer and storage interface maybe high priority, then the service can decoupling rely on the pd.Server.

@nolouch nolouch changed the title Split into microservice architecture Split into multi-service architecture Dec 20, 2022
@rleungx rleungx pinned this issue Jan 4, 2023
@rleungx rleungx mentioned this issue Feb 23, 2023
Merged
ti-chi-bot pushed a commit that referenced this issue Feb 27, 2023
@rleungx
mcs: register resource manager to etcd (#6040)
002d6ca 
ref #5766

Signed-off-by: Ryan Leung <rleungx@gmail.com>
@JmPotato JmPotato mentioned this issue Mar 3, 2023
Merged
ti-chi-bot added a commit that referenced this issue Mar 6, 2023
@JmPotato @ti-chi-bot
resource_manager: implement the independent primary election (#6086)
bc5234c 
ref #5766

Implement the independent resource manager primary election.

Signed-off-by: JmPotato <ghzpotato@gmail.com>

Co-authored-by: Ti Chi Robot <ti-community-prow-bot@tidb.io>
@rleungx rleungx mentioned this issue Jul 10, 2023
Merged
ti-chi-bot bot pushed a commit that referenced this issue Jul 10, 2023
@rleungx
mcs: add log flags (#6777)
b2a9768 
ref #5766

Signed-off-by: Ryan Leung <rleungx@gmail.com>
rleungx added a commit to rleungx/pd that referenced this issue Aug 2, 2023
@rleungx
mcs: add log flags (tikv#6777)
0f3fdd4 
ref tikv#5766

Signed-off-by: Ryan Leung <rleungx@gmail.com>
@JmPotato JmPotato mentioned this issue Aug 31, 2023
Merged
ti-chi-bot bot pushed a commit that referenced this issue Aug 31, 2023
@JmPotato @rleungx
cmd, server: add env to manage the service mode (#7018)
7a4e1e2 
ref #5766

Add env `PD_SERVICE_MODE` to manage the service mode.

Signed-off-by: Ryan Leung <rleungx@gmail.com>

Co-authored-by: Ryan Leung <rleungx@gmail.com>
@rleungx rleungx unpinned this issue Oct 19, 2023
@HuSharp HuSharp mentioned this issue Dec 11, 2023
Closed
This was referenced May 8, 2024
Closed
Merged
@seiya-annie
Copy link

/found customer

@ti-chi-bot ti-chi-bot bot added the report/customer Customers have encountered this bug. label Jun 4, 2024
@rleungx rleungx mentioned this issue Jun 8, 2024
Merged
18 tasks
ti-chi-bot bot added a commit that referenced this issue Aug 13, 2024
@HuSharp @ti-chi-bot
pdms: support primary/transfer api for scheduling and tso (#8157)
1c1cd99 
ref #5766, close #7995

Signed-off-by: husharp <ihusharp@gmail.com>

Co-authored-by: ti-chi-bot[bot] <108142056+ti-chi-bot[bot]@users.noreply.github.com>
@HuSharp HuSharp mentioned this issue Aug 21, 2024
Open
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
report/customer Customers have encountered this bug. type/feature-request Categorizes issue or PR as related to a new feature.
Projects
None yet
Milestone
No milestone
Development

No branches or pull requests
3 participants
@nolouch @rleungx @seiya-annie