boltdb panic on db.meta

[reich80]

We used the latest version etcd-io/boltdb in our products. Then we encounter the following panic:
goroutine 10254413 [running]:
runtime.throw(0x10153aa, 0x5)
/usr/local/go/src/runtime/panic.go:596 +0x95 fp=0xc420227be0 sp=0xc420227bc0
runtime.sigpanic()
/usr/local/go/src/runtime/signal_unix.go:287 +0xf4 fp=0xc420227c30 sp=0xc420227be0
go.etcd.io/bbolt.(*DB).meta(0xc43f58a400, 0xc435c95040)
/home/liuyawei/workspace/golang/go-work/src/go.etcd.io/bbolt/db.go:901 +0x34 fp=0xc420227c80 sp=0xc420227c30
go.etcd.io/bbolt.(*Tx).init(0xc439889ce0, 0xc43f58a400)
/home/liuyawei/workspace/golang/go-work/src/go.etcd.io/bbolt/tx.go:50 +0xba fp=0xc420227d68 sp=0xc420227c80
go.etcd.io/bbolt.(*DB).beginRWTx(0xc43f58a400, 0x0, 0x0, 0x0)
/home/liuyawei/workspace/golang/go-work/src/go.etcd.io/bbolt/db.go:608 +0xed fp=0xc420227da8 sp=0xc420227d68
go.etcd.io/bbolt.(*DB).Begin(0xc43f58a400, 0xc400000001, 0xc42fa99c00, 0x44945b, 0xc0b18d)
/home/liuyawei/workspace/golang/go-work/src/go.etcd.io/bbolt/db.go:542 +0x38 fp=0xc420227dd8 sp=0xc420227da8
go.etcd.io/bbolt.(*DB).Update(0xc43f58a400, 0xc420227e48, 0x0, 0x0)
/home/liuyawei/workspace/golang/go-work/src/go.etcd.io/bbolt/db.go:678 +0x46 fp=0xc420227e28 sp=0xc420227dd8
github.com/xtao/dsl-go/engine.(*BoltStore).Batch(0xc42f390360, 0x0, 0x0)
/home/liuyawei/workspace/golang/go-work/src/github.com/xtao/dsl-go/engine/BoltStore.go:280 +0xc6 fp=0xc420227e68 sp=0xc420227e28
github.com/xtao/dsl-go/engine.(*BoltStore).Update(0xc42f390360)
/home/liuyawei/workspace/golang/go-work/src/github.com/xtao/dsl-go/engine/BoltStore.go:319 +0x3c5 fp=0xc420227fb0 sp=0xc420227e68
github.com/xtao/dsl-go/engine.(*BoltStore).Load.func1(0xc42f390360)
/home/liuyawei/workspace/golang/go-work/src/github.com/xtao/dsl-go/engine/BoltStore.go:107 +0x51 fp=0xc420227fd8 sp=0xc420227fb0
runtime.goexit()
/usr/local/go/src/runtime/asm_amd64.s:2197 +0x1 fp=0xc420227fe0 sp=0xc420227fd8
created by github.com/xtao/dsl-go/engine.(*BoltStore).Load
/home/liuyawei/workspace/golang/go-work/src/github.com/xtao/dsl-go/engine/BoltStore.go:108 +0x263

it seems that the db.meta() panic on nil pointers. Could you help us on this problem?

[reich80]

The full stack dumped by go runtime is as follows:
unexpected fault address 0x7f1e5b001040
fatal error: fault
[signal SIGBUS: bus error code=0x2 addr=0x7f1e5b001040 pc=0xb607c4]

goroutine 10254413 [running]:
runtime.throw(0x10153aa, 0x5)
/usr/local/go/src/runtime/panic.go:596 +0x95 fp=0xc420227be0 sp=0xc420227bc0
runtime.sigpanic()
/usr/local/go/src/runtime/signal_unix.go:287 +0xf4 fp=0xc420227c30 sp=0xc420227be0
go.etcd.io/bbolt.(*DB).meta(0xc43f58a400, 0xc435c95040)
/home/liuyawei/workspace/golang/go-work/src/go.etcd.io/bbolt/db.go:901 +0x34 fp=0xc420227c80 sp=0xc420227c30
go.etcd.io/bbolt.(*Tx).init(0xc439889ce0, 0xc43f58a400)
/home/liuyawei/workspace/golang/go-work/src/go.etcd.io/bbolt/tx.go:50 +0xba fp=0xc420227d68 sp=0xc420227c80
go.etcd.io/bbolt.(*DB).beginRWTx(0xc43f58a400, 0x0, 0x0, 0x0)
/home/liuyawei/workspace/golang/go-work/src/go.etcd.io/bbolt/db.go:608 +0xed fp=0xc420227da8 sp=0xc420227d68
go.etcd.io/bbolt.(*DB).Begin(0xc43f58a400, 0xc400000001, 0xc42fa99c00, 0x44945b, 0xc0b18d)
/home/liuyawei/workspace/golang/go-work/src/go.etcd.io/bbolt/db.go:542 +0x38 fp=0xc420227dd8 sp=0xc420227da8
go.etcd.io/bbolt.(*DB).Update(0xc43f58a400, 0xc420227e48, 0x0, 0x0)
/home/liuyawei/workspace/golang/go-work/src/go.etcd.io/bbolt/db.go:678 +0x46 fp=0xc420227e28 sp=0xc420227dd8
github.com/xtao/dsl-go/engine.(*BoltStore).Batch(0xc42f390360, 0x0, 0x0)

It seems that the db meta pointer is invalid, cause a bus error

[horejsek]

I have the same problem. It failed with unexpected fault address and fatal error: fault during opening read transaction. I don't have reproducible code all the time, thought. It must be some special state of bbolt or something.

[xiang90]

what is the version of bbolt? can you reproduce this issue?

[horejsek]

We are using v1.3.3. Unfortunately no reproducible code.

[reich80]

I have root caused my problem. It is not a boltdb bug.

We use boltdb in a docker container and the db file is placed in a volume backed by our own distributed file system. Because boltdb read data via mmap a db file, so if the file system has IO issues, memory mapped read will fail and the process is sent a SIGBUS signal.

Thanks

[xiang90]

ok. closing it.