We read every piece of feedback, and take your input very seriously.
To see all available qualifiers, see our documentation.
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Description
Cayley 0.7.4 fails with a panic when trying to load a Freebase dump.
Steps to reproduce the issue:
Received results:
$ ../cayley_0.7.4_linux_amd64/cayley --db leveldb --dbpath graphdb load --init freebase.pq.gz --verbose 2 --batch 100000 I0911 22:30:35.186468 15954 cayley.go:63] Cayley version: 0.7.4 (431ee2850fa29f8f7dfa0de6a03c07237871d504) I0911 22:30:35.186583 15954 database.go:187] using backend "leveldb" (graphdb) I0911 22:30:38.687174 15954 load.go:155] Wrote 100000 quads. [...snip...] I0912 03:02:06.054281 15954 load.go:155] Wrote 84600000 quads. I0912 03:02:16.663510 15954 load.go:155] Wrote 84700000 quads. panic: runtime error: index out of range goroutine 1 [running]: github.com/cayleygraph/cayley/vendor/github.com/tylertreat/BoomFilters.(*Buckets).setBits(0xc420280090, 0x100000078, 0xc400000001) /go/src/github.com/cayleygraph/cayley/vendor/github.com/tylertreat/BoomFilters/buckets.go:101 +0x128 github.com/cayleygraph/cayley/vendor/github.com/tylertreat/BoomFilters.(*Buckets).Set(0xc420280090, 0x78, 0x1, 0x11106c0) /go/src/github.com/cayleygraph/cayley/vendor/github.com/tylertreat/BoomFilters/buckets.go:62 +0x53 github.com/cayleygraph/cayley/vendor/github.com/tylertreat/BoomFilters.(*DeletableBloomFilter).Add(0xc420286000, 0xc4200de5a0, 0x18, 0x18, 0x8, 0x0) /go/src/github.com/cayleygraph/cayley/vendor/github.com/tylertreat/BoomFilters/deletable.go:96 +0x106 github.com/cayleygraph/cayley/graph/kv.(*QuadStore).bloomAdd(0xc4200fc6e0, 0xc425f64f88) /go/src/github.com/cayleygraph/cayley/graph/kv/indexing.go:951 +0xca github.com/cayleygraph/cayley/graph/kv.(*QuadStore).indexLink(0xc4200fc6e0, 0x110cc00, 0xc42ccdde00, 0xc425f64f88, 0x0, 0x0) /go/src/github.com/cayleygraph/cayley/graph/kv/indexing.go:535 +0x22d github.com/cayleygraph/cayley/graph/kv.(*QuadStore).indexLinks(0xc4200fc6e0, 0x110c900, 0xc4200c2018, 0x110cc00, 0xc42ccdde00, 0xc43d188000, 0x1864f, 0x1864f, 0x0, 0x0) /go/src/github.com/cayleygraph/cayley/graph/kv/indexing.go:518 +0x116 github.com/cayleygraph/cayley/graph/kv.(*QuadStore).ApplyDeltas(0xc4200fc6e0, 0xc43588c000, 0x1864f, 0x186a0, 0xc42f6b0001, 0x0, 0x0) /go/src/github.com/cayleygraph/cayley/graph/kv/indexing.go:408 +0xbe6 github.com/cayleygraph/cayley/writer.(*Single).AddQuadSet(0xc4202080a0, 0xc42071c000, 0x186a0, 0x186a0, 0xc42f6b3190, 0x1104b20) /go/src/github.com/cayleygraph/cayley/writer/single.go:69 +0x1cd github.com/cayleygraph/cayley/graph.(*batchWriter).WriteQuads(0xc4201b4000, 0xc42071c000, 0x186a0, 0x186a0, 0xc4200d2140, 0xc42071c000, 0x186a0) /go/src/github.com/cayleygraph/cayley/graph/quadwriter.go:226 +0x55 github.com/cayleygraph/cayley/internal.(*batchLogger).WriteQuads(0xc4200d2160, 0xc42071c000, 0x186a0, 0x186a0, 0x186a0, 0x0, 0x0) /go/src/github.com/cayleygraph/cayley/internal/load.go:152 +0x59 github.com/cayleygraph/cayley/quad.CopyBatch(0x10fd1c0, 0xc4200d2160, 0x7f3e366d8040, 0xc4200d2140, 0x186a0, 0xc4200d2140, 0x0, 0x0) /go/src/github.com/cayleygraph/cayley/quad/rw.go:142 +0x1d3 github.com/cayleygraph/cayley/internal.DecompressAndLoad(0x1110660, 0xc4202080a0, 0x186a0, 0x7ffc63d45565, 0xe, 0x0, 0x0, 0x0, 0x0, 0x0) /go/src/github.com/cayleygraph/cayley/internal/load.go:139 +0x1c7 github.com/cayleygraph/cayley/internal.Load(0x1110660, 0xc4202080a0, 0x186a0, 0x7ffc63d45565, 0xe, 0x0, 0x0, 0x102d3d7, 0x4) /go/src/github.com/cayleygraph/cayley/internal/load.go:22 +0x7d github.com/cayleygraph/cayley/cmd/cayley/command.NewLoadDatabaseCmd.func1(0xc42029e000, 0xc4202a10e0, 0x1, 0xa, 0x0, 0x0) /go/src/github.com/cayleygraph/cayley/cmd/cayley/command/database.go:116 +0x272 github.com/cayleygraph/cayley/vendor/github.com/spf13/cobra.(*Command).execute(0xc42029e000, 0xc4202a0fa0, 0xa, 0xa, 0xc42029e000, 0xc4202a0fa0) /go/src/github.com/cayleygraph/cayley/vendor/github.com/spf13/cobra/command.go:620 +0x3e4 github.com/cayleygraph/cayley/vendor/github.com/spf13/cobra.(*Command).ExecuteC(0x1848de0, 0x1849020, 0xc42016df78, 0x405c3c) /go/src/github.com/cayleygraph/cayley/vendor/github.com/spf13/cobra/command.go:699 +0x2d4 github.com/cayleygraph/cayley/vendor/github.com/spf13/cobra.(*Command).Execute(0x1848de0, 0xc42029e900, 0x0) /go/src/github.com/cayleygraph/cayley/vendor/github.com/spf13/cobra/command.go:658 +0x2b main.main() /go/src/github.com/cayleygraph/cayley/cmd/cayley/cayley.go:187 +0x31
The same error happens with the bolt KV backend. The error occurs sooner when using a smaller batch size.
bolt
Expected results:
The data is loaded successfully.
Output of cayley version or commit hash:
cayley version
Cayley version: 0.7.4 Git commit hash: 431ee2850fa29f8f7dfa0de6a03c07237871d504 Build date: 2018-06-03T15:44:12Z
Environment details:
Backend database: leveldb, bolt
leveldb
The text was updated successfully, but these errors were encountered:
This must have been solved with a new BoomFilter version. Will check it, thanks!
BoomFilter
Sorry, something went wrong.
Fixed in #744
dennwc
No branches or pull requests
Description
Cayley 0.7.4 fails with a panic when trying to load a Freebase dump.
Steps to reproduce the issue:
Received results:
The same error happens with the
bolt
KV backend. The error occurs sooner when using a smaller batch size.Expected results:
The data is loaded successfully.
Output of
cayley version
or commit hash:Environment details:
Backend database:
leveldb
,bolt
The text was updated successfully, but these errors were encountered: