Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Update BoomFilter to avoid Cayley panicing #744

Merged
merged 1 commit into from
Oct 20, 2018

Conversation

hypirion
Copy link
Contributor

Cayley uses a DeletableBloomFilter for kv stores, which may cause a panic when loading a database due to an off-by-one error. In particular, you will see something like this (using the latest release binary):

I1019 13:01:18.852254   15348 cayley.go:63] Cayley version: 0.7.4 (431ee2850fa29f8f7dfa0de6a03c07237871d504)
I1019 13:01:18.852716   15348 cayley.go:76] using config file: config.yml
I1019 13:01:18.852820   15348 database.go:187] using backend "bolt" (./boltey)
panic: runtime error: index out of range

goroutine 1 [running]:
github.com/cayleygraph/cayley/vendor/github.com/tylertreat/BoomFilters.(*Buckets).setBits(0xc4201c0000, 0x100000078, 0xc400000001)
	/go/src/github.com/cayleygraph/cayley/vendor/github.com/tylertreat/BoomFilters/buckets.go:101 +0x128
github.com/cayleygraph/cayley/vendor/github.com/tylertreat/BoomFilters.(*Buckets).Set(0xc4201c0000, 0x78, 0x1, 0x11106c0)
	/go/src/github.com/cayleygraph/cayley/vendor/github.com/tylertreat/BoomFilters/buckets.go:62 +0x53
github.com/cayleygraph/cayley/vendor/github.com/tylertreat/BoomFilters.(*DeletableBloomFilter).Add(0xc420292000, 0xc4200e0020, 0x18, 0x18, 0x0, 0x0)
	/go/src/github.com/cayleygraph/cayley/vendor/github.com/tylertreat/BoomFilters/deletable.go:96 +0x106
github.com/cayleygraph/cayley/graph/kv.(*QuadStore).initBloomFilter.func1(0x110cc80, 0xc420218000, 0x0, 0x0)
	/go/src/github.com/cayleygraph/cayley/graph/kv/indexing.go:927 +0x292
github.com/cayleygraph/cayley/graph/kv.View(0x1105a80, 0xc4200e2020, 0xc42009f9b0, 0x0, 0x0)
	/go/src/github.com/cayleygraph/cayley/graph/kv/kv.go:94 +0xa6
github.com/cayleygraph/cayley/graph/kv.(*QuadStore).initBloomFilter(0xc420112140, 0x110c900, 0xc4200de018, 0x2, 0x0)
	/go/src/github.com/cayleygraph/cayley/graph/kv/indexing.go:909 +0x10e
github.com/cayleygraph/cayley/graph/kv.New(0x1105a80, 0xc4200e2020, 0xc42028cb70, 0x1105a80, 0xc4200e2020, 0x0, 0x0)
	/go/src/github.com/cayleygraph/cayley/graph/kv/quadstore.go:142 +0x212
github.com/cayleygraph/cayley/graph/kv.Register.func2(0xc42024b608, 0x8, 0xc42028cb70, 0x4, 0xc4201c5208, 0x1, 0xc42028cb70)
	/go/src/github.com/cayleygraph/cayley/graph/kv/quadstore.go:69 +0x97
github.com/cayleygraph/cayley/graph.NewQuadStore(0xc42024b5f8, 0x4, 0xc42024b608, 0x8, 0xc42028cb70, 0x3600000000000001, 0xc420189ba0, 0x53b004, 0xc42010a5a0)
	/go/src/github.com/cayleygraph/cayley/graph/registry.go:60 +0x98
github.com/cayleygraph/cayley/cmd/cayley/command.openDatabase(0xc42010a5a0, 0x102d5b7, 0x4)
	/go/src/github.com/cayleygraph/cayley/cmd/cayley/command/database.go:201 +0xe6
github.com/cayleygraph/cayley/cmd/cayley/command.openForQueries(0xc4202a8b40, 0x1074f70, 0x0, 0x0)
	/go/src/github.com/cayleygraph/cayley/cmd/cayley/command/database.go:223 +0x88
github.com/cayleygraph/cayley/cmd/cayley/command.NewHttpCmd.func1(0xc4202a8b40, 0xc420219b40, 0x0, 0x2, 0x0, 0x0)
	/go/src/github.com/cayleygraph/cayley/cmd/cayley/command/http.go:24 +0x9f
github.com/cayleygraph/cayley/vendor/github.com/spf13/cobra.(*Command).execute(0xc4202a8b40, 0xc420219ae0, 0x2, 0x2, 0xc4202a8b40, 0xc420219ae0)
	/go/src/github.com/cayleygraph/cayley/vendor/github.com/spf13/cobra/command.go:620 +0x3e4
github.com/cayleygraph/cayley/vendor/github.com/spf13/cobra.(*Command).ExecuteC(0x1848de0, 0x1849020, 0xc420189f78, 0x405c3c)
	/go/src/github.com/cayleygraph/cayley/vendor/github.com/spf13/cobra/command.go:699 +0x2d4
github.com/cayleygraph/cayley/vendor/github.com/spf13/cobra.(*Command).Execute(0x1848de0, 0xc4202a8900, 0x0)
	/go/src/github.com/cayleygraph/cayley/vendor/github.com/spf13/cobra/command.go:658 +0x2b
main.main()
	/go/src/github.com/cayleygraph/cayley/cmd/cayley/cayley.go:187 +0x31

This was fixed in commit tylertreat/BoomFilters@e4c39d4, so it's sufficient to update the dependency on BoomFilters to resolve this issue.

It feels a bit silly to update my name to CONTRIBUTORS and AUTHORS for such a small change, so I decided not to add it for now. But if you really want me to put my name in those, then I can of course do so.

Cayley uses a DeletableBloomFilter for kv stores, which may cause a
panic when loading a database due to an off-by-one error. This was
fixed in commit
tylertreat/BoomFilters@e4c39d4
@hypirion hypirion requested a review from dennwc as a code owner October 19, 2018 21:10
@dennwc dennwc added this to the v0.7.5 milestone Oct 20, 2018
Copy link
Member

@dennwc dennwc left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thank for taking care of this! Merging.

@dennwc dennwc merged commit 6875ccd into cayleygraph:master Oct 20, 2018
@hypirion hypirion deleted the bump-boom-filters branch October 20, 2018 10:23
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants