Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[fix][broker]Support to migrate topics from blue to green cluster per namespace #21367

Merged
merged 1 commit into from
Oct 16, 2023

Conversation

vraulji567
Copy link
Contributor

Motivation

Currently the blue-green migration functionality is available at the cluster level. Need a controlled strategy to migrate the topics at namespace level.

Modifications

Add config to namespace policy to enable topic migration at the namespace level.
./bin/pulsar-admin namespaces update-migration-state $namespace --migrated

Result

Once the namespace migration is enabled, all the topics within the namespace will be migrated to the green cluster.

Verifying this change

  • Make sure that the change passes the CI checks.

This change added tests and can be verified as follows:

(example:)

  • Added integration tests for end-to-end deployment with large payloads (10MB)
  • Extended integration test for recovery after broker failure

Does this pull request potentially affect one of the following parts:

If the box was checked, please highlight the changes

  • Dependencies (add or upgrade a dependency)
  • The public API
  • The schema
  • The default values of configurations
  • The threading model
  • The binary protocol
  • The REST endpoints
  • The admin CLI options
  • The metrics
  • Anything that affects deployment

Documentation

  • doc
  • doc-required
  • doc-not-needed
  • doc-complete

Matching PR in forked repository

PR in forked repository:

@github-actions github-actions bot added the doc-not-needed Your PR changes do not impact docs label Oct 14, 2023
@codecov-commenter
Copy link

Codecov Report

Merging #21367 (1658b5c) into master (d09642c) will decrease coverage by 0.13%.
The diff coverage is 76.74%.

Impacted file tree graph

@@             Coverage Diff              @@
##             master   #21367      +/-   ##
============================================
- Coverage     73.36%   73.24%   -0.13%     
- Complexity     3827    32564   +28737     
============================================
  Files          1888     1888              
  Lines        140340   140371      +31     
  Branches      15437    15438       +1     
============================================
- Hits         102958   102812     -146     
- Misses        29298    29475     +177     
  Partials       8084     8084              
Flag Coverage Δ
inttests 24.20% <13.95%> (-0.09%) ⬇️
systests 24.71% <13.95%> (-0.01%) ⬇️
unittests 72.55% <76.74%> (-0.10%) ⬇️

Flags with carried forward coverage won't be shown. Click here to find out more.

Files Coverage Δ
.../org/apache/pulsar/broker/admin/v2/Namespaces.java 68.38% <100.00%> (+0.09%) ⬆️
...rg/apache/pulsar/broker/service/AbstractTopic.java 87.73% <100.00%> (+0.52%) ⬆️
...ava/org/apache/pulsar/broker/service/Consumer.java 86.23% <100.00%> (ø)
...va/org/apache/pulsar/broker/service/ServerCnx.java 72.00% <100.00%> (ø)
...oker/service/nonpersistent/NonPersistentTopic.java 73.05% <100.00%> (ø)
...ava/org/apache/pulsar/client/admin/Namespaces.java 66.66% <ø> (ø)
...g/apache/pulsar/common/policies/data/Policies.java 59.09% <ø> (ø)
...e/pulsar/client/admin/internal/NamespacesImpl.java 95.65% <100.00%> (+0.03%) ⬆️
.../java/org/apache/pulsar/admin/cli/CmdClusters.java 74.07% <ø> (ø)
...sar/broker/service/persistent/PersistentTopic.java 79.61% <50.00%> (+0.15%) ⬆️
... and 3 more

... and 77 files with indirect coverage changes

@rdhabalia rdhabalia merged commit a10564d into apache:master Oct 16, 2023
48 checks passed
vraulji567 added a commit to vraulji567/pulsar that referenced this pull request Oct 16, 2023
@Technoboy- Technoboy- added this to the 3.2.0 milestone Mar 26, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
doc-not-needed Your PR changes do not impact docs ready-to-test
Projects
None yet
Development

Successfully merging this pull request may close these issues.

4 participants