[fix][broker] Fix avoid creating new topic after migration is started #21368
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
…apache#584) * [fix][broker] Fix avoid creating new topic after migration is started fix migration * add org.apache.pulsar.broker.service.ClusterMigrationTest test * fix test Co-authored-by: rdhabalia <rdhabalia@apache.org> clean up file fix formatting
rdhabalia
added
area/broker
doc-not-needed
vraulji567
pushed a commit
to vraulji567/pulsar
that referenced
this pull request
Oct 16, 2023
…apache#21368) Co-authored-by: Rajan Dhabalia <rdhabalia@oath.com>
|
Hi @rdhabalia , cluster migration is introduced by 3.0, so I think we need to cherry-pick this patch to branch-3.0 and branch-3.1 |
15 tasks
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Motivation
Right now, blue broker deletes the topic once all producers and subscriptions are migrated to green cluster. Blue cluster persist topic's migration state into zk with managed-ledger metadata. However, once topic fully migrates to green cluster, blue cluster broker deletes the topic and deletes the topic's managed-ledger metadata because of that next time if client connects to blue-broker on the same topic, blue broker recreates the same topic again which can cause incorrect connection and data loss during migration. Therefore, blue broker should not allow any new topic creation if migration is started for that namespace to avoid such data loss and incorrect connection. This PR depends on #21354
Modifications
Broker fails to create topic if migration is enabled for that namespace and sends migration error to client so, client can connect to green cluster.
Verifying this change
(Please pick either of the following options)
This change is a trivial rework / code cleanup without any test coverage.
(or)
This change is already covered by existing tests, such as (please describe tests).
(or)
This change added tests and can be verified as follows:
(example:)
Does this pull request potentially affect one of the following parts:
If the box was checked, please highlight the changes
Documentation
docdoc-requireddoc-not-neededdoc-completeMatching PR in forked repository
PR in forked repository: