Revendor etcd/raft

[aaronlehmann]

Change Campaign logic to trigger after Advance, because now Campaign
won't do anything if there are uncommitted entries. Also, remove the
check on Progress, because now Progress is only filled in if the node is
already the leader.

Set a smaller WAL segment size to work around high I/O load on Mac when
running unit tests.

Fixes #1229

[codecov-io]

Current coverage is 55.35% (diff: 100%)

Merging #1353 into master will increase coverage by 0.14%

@@             master      #1353   diff @@
==========================================
  Files            80         80          
  Lines         12637      12632     -5   
  Methods           0          0          
  Messages          0          0          
  Branches          0          0          
==========================================
+ Hits           6977       6993    +16   
+ Misses         4699       4679    -20   
+ Partials        961        960     -1   

Powered by Codecov. Last update 77a6894...62e73cd

[dperny]

while running this branch, got this:

--- FAIL: TestRaftSnapshot (10.76s)
        Error Trace:    testutils.go:104
                        testutils.go:365
                        storage_test.go:67
        Error:          Received unexpected error polling failed: did not find leader in member list

FAIL
exit status 1

[aaronlehmann]

while running this branch, got this:

Was this on Mac? I also saw some issues with the tests due to I/O load, but I have an upstream PR pending that should solve the problem.

[dperny]

Negative. Ubuntu 12.04 LTS.

[aaronlehmann]

@dperny: I made a change to the test that may help. Can you check if the problem is still occurring?

[aaronlehmann]

My PR to coreos/etcd that allows tests to work around the preallocation issue was merged. Updated this PR and removed WIP.

[dperny]

@aaronlehmann, sorry, I should have specified that the failure was intermittent. I got it in one run out of maybe 10.

[LK4D4]

-22k LOC? I'm in.
LGTM

[aaronlehmann]

-22k LOC? I'm in.

Yeah, some of it is from removing dependencies like candiedyaml that we don't use any more. But also etcd switched to a better vendoring system where they aren't rewriting import paths. So we no longer vendor two copies of gogoprotobuf, for example.

[aaronlehmann]

Rebased after #1327 to adapt to new locking API in this upstream version, and adjust TestGCWAL for the smaller segment size. The test is now enabled in CI.

PTAL

[abronan]

Thanks @aaronlehmann, I'm going to test this with docker integration.

[abronan]

Haven't seen any issue running a cluster with docker integration, removing the progress check seems safe too since we now vendor v3.

On 3 Manager cluster, I tested:

• promotion/demotion (with self-demotion)
• stop whole cluster and restart
• stop whole cluster and restart a subset
• stop leader
• some service operations and stress-test while doing all of the above

We should think about using the leadership transfer on self-demotion now that we vendor v3.

LGTM

[aaronlehmann]

Cool. Did you have a chance to run the Docker integration tests? I haven't done this yet.

[aaronlehmann]

Docker integration tests passed. Here goes nothing.

[aaronlehmann]

@LK4D4: Actually it's -36k LOC - GitHub is not showing correct totals.