TestBootstrap fails occasionally

[Kubuxu]

http://ci.ipfs.team:8111/viewLog.html?buildId=2045&buildTypeId=GoIpfs_CiTests&tab=buildLog#_focus=4136&state=4136

More here: #188

[whyrusleeping]

i hate this one. Its the 'misdial' issue, it affects a few different tests.

[whyrusleeping]

https://circleci.com/gh/ipfs/go-ipfs/4055

[Kubuxu]

My sanity tests caught it: https://travis-ci.org/ipfs/go-ipfs/jobs/136932615#L3812

[Kubuxu]

I am able to replicate failure in less than a minute:

cd routing/dht
while time go test -run TestBootstrap; do; done

Looks like failure about one in five.

And sundelly now it goes without a failure for a long time, yeah but it is repeatable.

[whyrusleeping]

https://travis-ci.org/ipfs/go-ipfs/jobs/137017648 heres another one

[Kubuxu]

It think I figured out what is happening, the SO_REUSE_PORT flag on a socket makes it possible for socket 127.0.0.1:0 to randomise to the same address.
https://github.com/ipfs/go-libp2p/blob/master/p2p/net/swarm/swarm_listen.go#L27

I am running now tests on my server with IPFS_REUSEPORT=false env variable. If I am sure that it is the solution (no failure for few hours) I will work on a solution.

[Kubuxu]

I found it pretty interesting that it is a problem but it makes sense, it is birthday paradox with calendar not of 365 days but 2^16. As it turns out for 10% failure rate binding 140 addresses is enough: https://www.wolframalpha.com/input/?i=y%3D+1+-+(((2%5E16)+nPr+n+)%2F(2%5E16)%5En)++from+0+to+200

I still will wait few hours to confirm that issues is gone.