restart etcd panic with unexpected smaller revision in mvcc

[siddontang]

I use docker-compose to start pd cluster (mount pd data to local directory), and when I recreate the cluster with docker-compose up -d --force-recreate, I meet following panic:

2016/12/15 09:43:35 util.go:329: [info] I | membership: [set the cluster version to 3.0 from store]
2016/12/15 09:43:36 util.go:321: [fatal] C | mvcc: [store.keyindex: put with unexpected smaller revision [{65178 0} / {65178 0}]]

The embed etcd version is e53b995

[xiang90]

Do you still have the data dir around. Mind sharing that broken data dir?

[xiang90]

If you can reproduce the problem, can you try to update the etcd version to current master and see if you can still reproduce?

[siddontang]

@xiang90

pd.tar.gz

[siddontang]

We will update the etcd and try to reproduce it.

[xiang90]

Thanks

[xiang90]

@siddontang I can confirm this is a bug. Probably from upstream boltdb. We will investigate more.

[xiang90]

Opened an issue on upstream boltdb: boltdb/bolt#632

[xiang90]

Never hit this again. It seems like a bug outside etcd system really. Closing.

[gaddamas]

I got similar message "mvcc: store.keyindex: put with unexpected smaller revision [{200001 0} / {200001 0}]. what does it mean? could you please give technical description.

[pacoxu]

[fOO223Fr]

any update on this problem?

[seamounts]

any update on this problem?

[liangguohun]

how did you do to fix it， and keep your data..

[drtechco]

https://gist.github.com/drtechco/3773a8e0e7a692fabe8cefd3ca102180