Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[mdc][leo_storage] read-repair, data-compaction, data-recovery feature may replicate objects more than "mdc_replication.num_of_replicas_a_dc" #645

Closed
mocchira opened this issue Mar 17, 2017 · 3 comments · Fixed by leo-project/leo_redundant_manager#5

Comments

@mocchira
Copy link
Member

mocchira commented Mar 17, 2017

Because read_repair don't take care whether an object was replicated via MDC replication or via local replication.

For example,
say

mdc_replication.num_of_replicas_a_dc = 1 ### on the source cluster
consistency.num_of_replicas = 2 ### on the destination one

When a MDC replication happened, one replica is created on the destination.
Subsequently when a GET request to the replicated object on the destination happened, that cause read_repair and in turn one more replica is created (Now the destination have 2 replicas for the object).

@yosukehara
Copy link
Member

WIP. I've recognized that data-compaction, data-recovery are also affected.

@yosukehara yosukehara changed the title [mdc][leo_storage] read_repair may replicate objects more than mdc_replication.num_of_replicas_a_dc [mdc][leo_storage] read-repair, data-compaction, data-recovery feature may replicate objects more than "mdc_replication.num_of_replicas_a_dc" Apr 12, 2017
yosukehara added a commit to yosukehara/leo_redundant_manager that referenced this issue Apr 12, 2017
@yosukehara yosukehara reopened this Apr 17, 2017
yosukehara added a commit to yosukehara/leo_redundant_manager that referenced this issue Apr 18, 2017
mocchira pushed a commit to leo-project/leo_redundant_manager that referenced this issue Apr 19, 2017
@yosukehara yosukehara reopened this Apr 19, 2017
yosukehara added a commit to yosukehara/leofs that referenced this issue Apr 19, 2017
@mocchira
Copy link
Member Author

@yosukehara

Sorry for closing the issue unintentionally.
It seems now github automatically close the issue if the commit message includes Fix ${Organization}/${Project}#${IssueNumber} (yes actually there are much more cases).
AFAIK this auto issue closer worked only when Fix xxx part appeared in the head of a message, however now it seems to work regardless of its position.

We need to take care not to use such commit messages to avoid unintentional closes.

@yosukehara
Copy link
Member

@mocchira

It seems now GitHub automatically close the issue if the commit message includes Fix ${Organization}/${Project}#${IssueNumber} (yes actually there are much more cases).
AFAIK this auto issue closer worked only when Fix xxx part appeared in the head of a message, however now it seems to work regardless of its position.

I see.

yosukehara added a commit to leo-project/leo_object_storage that referenced this issue Apr 24, 2017
yosukehara added a commit to leo-project/leo_object_storage that referenced this issue Apr 24, 2017
yosukehara added a commit to yosukehara/leofs that referenced this issue Apr 25, 2017
yosukehara added a commit to yosukehara/leofs that referenced this issue Apr 25, 2017
yosukehara added a commit to yosukehara/leofs that referenced this issue Apr 26, 2017
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment