various failures in multicore configuration

[btc]

Running go test --cpu 4 with or without --race results in lots of different failures.

@jbenet These cause clients to deadlock.

• [x]

goprocess (fix-424-blockservice-async-HasBlock) λ. go test ./... --cpu 4 --race                                                                                                     1
--- FAIL: TestCloseAfterChildren-4 (0.00s)
        goprocess_test.go:486: none should be closed
FAIL
FAIL    github.com/jbenet/go-ipfs/Godeps/_workspace/src/github.com/jbenet/goprocess     1.282s
?       github.com/jbenet/go-ipfs/Godeps/_workspace/src/github.com/jbenet/goprocess/context     [no test files]
ok      github.com/jbenet/go-ipfs/Godeps/_workspace/src/github.com/jbenet/goprocess/ratelimit   0.016s

• [x]

goprocess (fix-424-blockservice-async-HasBlock) λ. go test ./... --cpu 4 --race                                                                                                     1
--- FAIL: TestOnClosed-4 (0.00s)
        goprocess_test.go:498: context not in group: 0 [00 01 10 11]
        goprocess_test.go:498: context not in group: 11 [0 1]
--- FAIL: TestCloseAfterChildren-4 (0.00s)
        goprocess_test.go:486: none should be closed
FAIL
FAIL    github.com/jbenet/go-ipfs/Godeps/_workspace/src/github.com/jbenet/goprocess     1.283s
?       github.com/jbenet/go-ipfs/Godeps/_workspace/src/github.com/jbenet/goprocess/context     [no test files]
ok      github.com/jbenet/go-ipfs/Godeps/_workspace/src/github.com/jbenet/goprocess/ratelimit   0.017s

• [x]

--- FAIL: TestRateLimitGoDoesntBlock-4 (0.00s)
        ratelimit_test.go:65: create a rate limiter with limit of 3
        ratelimit_test.go:71: spawn 6 children with usual Process.Go.
        ratelimit_test.go:83: should not have blocked.
        ratelimit_test.go:78: spawned 0
        ratelimit_test.go:78: spawned 1
        ratelimit_test.go:78: spawned 2
        ratelimit_test.go:78: spawned 3
        ratelimit_test.go:78: spawned 4
        ratelimit_test.go:78: spawned 5
        ratelimit_test.go:88: process.Go blocked. it should not.
        ratelimit_test.go:91: drain children so they close
        ratelimit_test.go:94: closed 0
        ratelimit_test.go:94: closed 1
        ratelimit_test.go:94: closed 2
        ratelimit_test.go:94: closed 3
        ratelimit_test.go:94: closed 4
        ratelimit_test.go:94: closed 5
FAIL
FAIL    github.com/jbenet/go-ipfs/Godeps/_workspace/src/github.com/jbenet/goprocess/ratelimit   0.017s

[jbenet]

Hm, so all of these cases were actually issues in the tests.

@jbenet These cause clients to deadlock.

Can you give me an example (either in go-ipfs or an isolated test case)? i think you ran into a problem not covered by these failures that we need to find. (I run the tests with various permutations of -race -cpu=1-5 and everything is passing).

See: #3

[btc]

cd go-ipfs/blockservice/worker && go test ./... -run=TestStartClose --cpu=2

[jbenet]

What branch?

go test ./... -run=TestStartClose --cpu=2 -v
=== RUN TestStartClose-2
--- PASS: TestStartClose-2 (0.00s)
PASS
ok github.com/jbenet/go-ipfs/blockservice/worker 0.012s
? github.com/jbenet/go-ipfs/blockservice/worker/bench [no test files]

go test ./... -run=TestStartClose --cpu=4 -v
=== RUN TestStartClose-4
--- PASS: TestStartClose-4 (0.00s)
PASS
ok github.com/jbenet/go-ipfs/blockservice/worker 0.011s
? github.com/jbenet/go-ipfs/blockservice/worker/bench [no test files]

Note master now carries ipfs/kubo@c43f97d which are the changes from: 7f96033...c3e5920 (got merged with ipfs/kubo#583 -- it may have been: 2496b7c

[Stebalien]

I've fixed a bunch of close races in #23 as well so I'll assume this is fixed (can't reproduce locally).