Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

feat: support migrating an instance to sharding #48795

Open
wants to merge 1 commit into
base: master
Choose a base branch
from

Conversation

icewind1991
Copy link
Member

@icewind1991 icewind1991 commented Oct 18, 2024

Support migrating existing instances to sharding.

Enabled by setting the sharding configuration and setting the from_shard_key and from_primary to the sharding configuration with the next storage/file id (so the first storage/file that will be created after sharding is configured).

The schema from the main db will need to be copied to the shard db's before enabling the sharding (having a command to help with this would be a nice stretch-goal)

Any files on the existing storages will still go to the main db, only newly created storages will be sharded.

@tcitworld
Copy link
Member

Things seem to work, but in some cases, the "File entry could not be inserted but could also not be selected with getId() in order to perform an update. Please try again." error gets thrown. (creating a new user is the most reliable way I found).
While things seem to work otherwise, I suspect that this issue is a sign of a bigger connection handling issue.

I had tried #36313 a while ago

@icewind1991 icewind1991 force-pushed the sharding-existing branch 6 times, most recently from ec107dc to 756e35a Compare November 11, 2024 14:35
Signed-off-by: Robin Appelman <robin@icewind.nl>
@icewind1991 icewind1991 changed the title [wip] Support sharding migration feat: support migrating an instance to sharding Dec 16, 2024
@icewind1991 icewind1991 marked this pull request as ready for review December 16, 2024 19:23
@icewind1991 icewind1991 requested a review from come-nc December 20, 2024 13:09
@icewind1991 icewind1991 added the 3. to review Waiting for reviews label Dec 20, 2024
@icewind1991 icewind1991 added this to the Nextcloud 31 milestone Dec 20, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
3. to review Waiting for reviews
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants