Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Telegraf Panics after start #3373

Closed
jackzampolin opened this issue Oct 22, 2017 · 7 comments
Closed

Telegraf Panics after start #3373

jackzampolin opened this issue Oct 22, 2017 · 7 comments
Labels
bug unexpected problem or unintended behavior panic issue that results in panics from Telegraf
Milestone
1.4.3

Comments

@jackzampolin
Copy link
Contributor

Bug report

Panic in logs

Relevant telegraf.conf:

gist

System info:

Telegraf v1.4.2 (git: release-1.4 0cc5fc0ce40b09467012fffddcea6985b2763a97)
Ubuntu 16.04
@danielnelson
Copy link
Contributor

This might be the same as #3299, how hard is it to reproduce the panic? It would be helpful if we could pin it down to a specific plugin.

@danielnelson danielnelson added bug unexpected problem or unintended behavior panic issue that results in panics from Telegraf labels Oct 23, 2017
@jackzampolin
Copy link
Contributor Author

I think its coming from the exec plugin. I've got it running a bitcoin-cli command. Those are the only instances where I'm seeing that panic.

@danielnelson
Copy link
Contributor

The exec plugin isn't doing anything fancy, if it is causing this panic it might be a Go bug. Any chance you can run it with all other plugins disabled to verify?

@jackzampolin
Copy link
Contributor Author

jackzampolin commented Oct 24, 2017 •
edited
Loading

So no panic after a day of just exec was having multiple per hour earlier. So prob not that. I'll try reenabling some of the other ones. Any guesses?

@danielnelson
Copy link
Contributor

Maybe diskio, are you running in docker?

@jackzampolin
Copy link
Contributor Author

jackzampolin commented Oct 24, 2017 •
edited
Loading

No. Just bitcoin and python app. Adding [[inputs.diskio]] and trying again. I've got the interval up to 1s so should reproduce earlier.

@jackzampolin
Copy link
Contributor Author

Ding ding ding! We have a winner. diskio it is.

Oct 24 18:27:50 testnet telegraf[44045]: 2017-10-24T18:27:50Z I! Loaded outputs: influxdb
Oct 24 18:27:50 testnet telegraf[44045]: 2017-10-24T18:27:50Z I! Loaded inputs: inputs.exec inputs.diskio
Oct 24 18:27:50 testnet telegraf[44045]: 2017-10-24T18:27:50Z I! Tags enabled: host=testnet service=bitcoind
Oct 24 18:27:50 testnet telegraf[44045]: 2017-10-24T18:27:50Z I! Agent Config: Interval:1s, Quiet:false, Hostname:"testnet", Flush Interval:10s
Oct 24 19:39:51 testnet telegraf[44045]: fatal error: workbuf is empty
Oct 24 19:39:51 testnet telegraf[44045]: fatal error: workbuf is empty
Oct 24 19:39:51 testnet telegraf[44045]: fatal error: workbuf is empty
Oct 24 19:39:51 testnet telegraf[44045]: fatal error: workbuf is empty
Oct 24 19:39:51 testnet telegraf[44045]: fatal error: workbuf is empty
Oct 24 19:39:51 testnet telegraf[44045]: fatal error: workbuf is empty
Oct 24 19:39:51 testnet telegraf[44045]: fatal error: workbuf is empty
Oct 24 19:39:51 testnet telegraf[44045]: runtime stack:
Oct 24 19:39:51 testnet telegraf[44045]: runtime.throw(0x144d989, 0x10)
Oct 24 19:39:51 testnet telegraf[44045]:         /usr/local/go/src/runtime/panic.go:596 +0x95
Oct 24 19:39:51 testnet telegraf[44045]: runtime.(*workbuf).checknonempty(0x7f2abb827800)
Oct 24 19:39:51 testnet telegraf[44045]:         /usr/local/go/src/runtime/mgcwork.go:314 +0x4e
Oct 24 19:39:51 testnet telegraf[44045]: runtime.trygetfull(0x7f2abb9400d8)
Oct 24 19:39:51 testnet telegraf[44045]:         /usr/local/go/src/runtime/mgcwork.go:364 +0x45
Oct 24 19:39:51 testnet telegraf[44045]: runtime.(*gcWork).tryGet(0xc42001d228, 0xc42001d228)
Oct 24 19:39:51 testnet telegraf[44045]:         /usr/local/go/src/runtime/mgcwork.go:165 +0x91
Oct 24 19:39:51 testnet telegraf[44045]: runtime.gcDrain(0xc42001d228, 0xd)
Oct 24 19:39:51 testnet telegraf[44045]:         /usr/local/go/src/runtime/mgcmark.go:1075 +0x2eb
Oct 24 19:39:51 testnet telegraf[44045]: runtime.gcBgMarkWorker.func2()
Oct 24 19:39:51 testnet telegraf[44045]:         /usr/local/go/src/runtime/mgc.go:1534 +0x86
Oct 24 19:39:51 testnet telegraf[44045]: runtime.systemstack(0xc4207bc6c0)
Oct 24 19:39:51 testnet telegraf[44045]:         /usr/local/go/src/runtime/asm_amd64.s:327 +0x79
Oct 24 19:39:51 testnet telegraf[44045]: runtime.mstart()
Oct 24 19:39:51 testnet telegraf[44045]:         /usr/local/go/src/runtime/proc.go:1132
Oct 24 19:39:51 testnet telegraf[44045]: goroutine 39 [GC worker (idle)]:
Oct 24 19:39:51 testnet telegraf[44045]: runtime.systemstack_switch()
Oct 24 19:39:51 testnet telegraf[44045]:         /usr/local/go/src/runtime/asm_amd64.s:281 fp=0xc42050f750 sp=0xc42050f748
Oct 24 19:39:51 testnet telegraf[44045]: runtime.gcBgMarkWorker(0xc42001c000)
Oct 24 19:39:51 testnet telegraf[44045]:         /usr/local/go/src/runtime/mgc.go:1537 +0x1fc fp=0xc42050f7d8 sp=0xc42050f750
Oct 24 19:39:51 testnet telegraf[44045]: runtime.goexit()
Oct 24 19:39:51 testnet telegraf[44045]:         /usr/local/go/src/runtime/asm_amd64.s:2197 +0x1 fp=0xc42050f7e0 sp=0xc42050f7d8
Oct 24 19:39:51 testnet telegraf[44045]: created by runtime.gcBgMarkStartWorkers
Oct 24 19:39:51 testnet telegraf[44045]:         /usr/local/go/src/runtime/mgc.go:1412 +0x98
Oct 24 19:39:51 testnet telegraf[44045]: goroutine 1 [semacquire, 72 minutes]:
Oct 24 19:39:51 testnet telegraf[44045]: sync.runtime_Semacquire(0xc420072fcc)
Oct 24 19:39:51 testnet telegraf[44045]:         /usr/local/go/src/runtime/sema.go:47 +0x34
Oct 24 19:39:51 testnet telegraf[44045]: sync.(*WaitGroup).Wait(0xc420072fc0)
Oct 24 19:39:51 testnet telegraf[44045]:         /usr/local/go/src/sync/waitgroup.go:131 +0x7a
Oct 24 19:39:51 testnet telegraf[44045]: github.com/influxdata/telegraf/agent.(*Agent).Run(0xc42000e030, 0xc42042c120, 0x0, 0x0)
Oct 24 19:39:51 testnet telegraf[44045]:         /home/ubuntu/telegraf-build/src/github.com/influxdata/telegraf/agent/agent.go:426 +0x54e
Oct 24 19:39:51 testnet telegraf[44045]: main.reloadLoop(0xc420010840, 0x1d37f10, 0x0, 0x0, 0x1d37f10, 0x0, 0x0, 0x1d37f10, 0x0, 0x0, ...)
Oct 24 19:39:51 testnet telegraf[44045]:         /home/ubuntu/telegraf-build/src/github.com/influxdata/telegraf/cmd/telegraf/telegraf.go:234 +0xa46
Oct 24 19:39:51 testnet telegraf[44045]: main.main()
Oct 24 19:39:51 testnet telegraf[44045]:         /home/ubuntu/telegraf-build/src/github.com/influxdata/telegraf/cmd/telegraf/telegraf.go:411 +0x65b
Oct 24 19:39:51 testnet telegraf[44045]: goroutine 17 [syscall, 72 minutes, locked to thread]:
Oct 24 19:39:51 testnet telegraf[44045]: runtime.goexit()
Oct 24 19:39:51 testnet telegraf[44045]:         /usr/local/go/src/runtime/asm_amd64.s:2197 +0x1
Oct 24 19:39:51 testnet telegraf[44045]: goroutine 3 [syscall, 72 minutes]:
Oct 24 19:39:51 testnet telegraf[44045]: os/signal.signal_recv(0x0)
Oct 24 19:39:51 testnet telegraf[44045]:         /usr/local/go/src/runtime/sigqueue.go:116 +0x104
Oct 24 19:39:51 testnet telegraf[44045]: os/signal.loop()
Oct 24 19:39:51 testnet telegraf[44045]:         /usr/local/go/src/os/signal/signal_unix.go:22 +0x22
Oct 24 19:39:51 testnet telegraf[44045]: created by os/signal.init.1
Oct 24 19:39:51 testnet telegraf[44045]:         /usr/local/go/src/os/signal/signal_unix.go:28 +0x41
Oct 24 19:39:51 testnet telegraf[44045]: goroutine 41 [select]:
Oct 24 19:39:51 testnet telegraf[44045]: github.com/influxdata/telegraf/agent.(*Agent).flusher(0xc42000e030, 0xc42042c120, 0xc42042c480, 0xc42042c4e0, 0xc420440000, 0xc42050c7a8)
Oct 24 19:39:51 testnet telegraf[44045]:         /home/ubuntu/telegraf-build/src/github.com/influxdata/telegraf/agent/agent.go:320 +0x444
Oct 24 19:39:51 testnet telegraf[44045]: github.com/influxdata/telegraf/agent.(*Agent).Run.func1(0xc420072fc0, 0xc42000e030, 0xc42042c120, 0xc42042c480, 0xc42042c4e0)
Oct 24 19:39:51 testnet telegraf[44045]:         /home/ubuntu/telegraf-build/src/github.com/influxdata/telegraf/agent/agent.go:396 +0x7a
Oct 24 19:39:51 testnet telegraf[44045]: created by github.com/influxdata/telegraf/agent.(*Agent).Run
Oct 24 19:39:51 testnet telegraf[44045]:         /home/ubuntu/telegraf-build/src/github.com/influxdata/telegraf/agent/agent.go:400 +0x34f
Oct 24 19:39:51 testnet telegraf[44045]: goroutine 62 [select]:
Oct 24 19:39:51 testnet telegraf[44045]: github.com/influxdata/telegraf/agent.(*Agent).flusher.func1(0xc42018e920, 0xc42042c120, 0xc4202b5200, 0xc42000e030)
Oct 24 19:39:51 testnet telegraf[44045]:         /home/ubuntu/telegraf-build/src/github.com/influxdata/telegraf/agent/agent.go:263 +0x2f3
Oct 24 19:39:51 testnet telegraf[44045]: created by github.com/influxdata/telegraf/agent.(*Agent).flusher
Oct 24 19:39:51 testnet telegraf[44045]:         /home/ubuntu/telegraf-build/src/github.com/influxdata/telegraf/agent/agent.go:292 +0xe5
Oct 24 19:39:51 testnet telegraf[44045]: goroutine 24 [select, 72 minutes, locked to thread]:
Oct 24 19:39:51 testnet telegraf[44045]: runtime.gopark(0x14bb4a8, 0x0, 0x143ae50, 0x6, 0x18, 0x2)
Oct 24 19:39:51 testnet telegraf[44045]:         /usr/local/go/src/runtime/proc.go:271 +0x13a
Oct 24 19:39:51 testnet telegraf[44045]: runtime.selectgoImpl(0xc42050cf50, 0x0, 0x18)
Oct 24 19:39:51 testnet telegraf[44045]:         /usr/local/go/src/runtime/select.go:423 +0x1364
Oct 24 19:39:51 testnet telegraf[44045]: runtime.selectgo(0xc42050cf50)
Oct 24 19:39:51 testnet telegraf[44045]:         /usr/local/go/src/runtime/select.go:238 +0x1c
Oct 24 19:39:51 testnet telegraf[44045]: runtime.ensureSigM.func1()
Oct 24 19:39:51 testnet telegraf[44045]:         /usr/local/go/src/runtime/signal_unix.go:434 +0x2dd
Oct 24 19:39:51 testnet telegraf[44045]: runtime.goexit()
Oct 24 19:39:51 testnet telegraf[44045]:         /usr/local/go/src/runtime/asm_amd64.s:2197 +0x1
Oct 24 19:39:51 testnet telegraf[44045]: goroutine 25 [select, 72 minutes]:
Oct 24 19:39:51 testnet telegraf[44045]: main.reloadLoop.func1(0xc42042c180, 0xc42042c120, 0xc420220a10, 0xc420010840)
Oct 24 19:39:51 testnet telegraf[44045]:         /home/ubuntu/telegraf-build/src/github.com/influxdata/telegraf/cmd/telegraf/telegraf.go:195 +0x24a
Oct 24 19:39:51 testnet telegraf[44045]: created by main.reloadLoop
Oct 24 19:39:51 testnet telegraf[44045]:         /home/ubuntu/telegraf-build/src/github.com/influxdata/telegraf/cmd/telegraf/telegraf.go:209 +0x6ec
Oct 24 19:39:51 testnet telegraf[44045]: goroutine 42 [select]:
Oct 24 19:39:51 testnet telegraf[44045]: github.com/influxdata/telegraf/agent.(*Agent).gatherer(0xc42000e030, 0xc42042c120, 0xc420192100, 0x3b9aca00, 0xc42042c480)
Oct 24 19:39:51 testnet telegraf[44045]:         /home/ubuntu/telegraf-build/src/github.com/influxdata/telegraf/agent/agent.go:129 +0x3a8
Oct 24 19:39:51 testnet telegraf[44045]: github.com/influxdata/telegraf/agent.(*Agent).Run.func3(0xc420072fc0, 0xc42000e030, 0xc42042c120, 0xc42042c480, 0xc420192100, 0x3b9aca00)
Oct 24 19:39:51 testnet telegraf[44045]:         /home/ubuntu/telegraf-build/src/github.com/influxdata/telegraf/agent/agent.go:422 +0x7d
Oct 24 19:39:51 testnet telegraf[44045]: created by github.com/influxdata/telegraf/agent.(*Agent).Run
Oct 24 19:39:51 testnet telegraf[44045]:         /home/ubuntu/telegraf-build/src/github.com/influxdata/telegraf/agent/agent.go:423 +0x505
Oct 24 19:39:51 testnet telegraf[44045]: goroutine 43 [select]:
Oct 24 19:39:51 testnet telegraf[44045]: github.com/influxdata/telegraf/agent.gatherWithTimeout(0xc42042c120, 0xc420192180, 0xc4201fc100, 0x3b9aca00)
Oct 24 19:39:51 testnet telegraf[44045]:         /home/ubuntu/telegraf-build/src/github.com/influxdata/telegraf/agent/agent.go:157 +0x28f
Oct 24 19:39:51 testnet telegraf[44045]: github.com/influxdata/telegraf/agent.(*Agent).gatherer(0xc42000e030, 0xc42042c120, 0xc420192180, 0x3b9aca00, 0xc42042c480)
Oct 24 19:39:51 testnet telegraf[44045]:         /home/ubuntu/telegraf-build/src/github.com/influxdata/telegraf/agent/agent.go:124 +0x2a4
Oct 24 19:39:51 testnet telegraf[44045]: github.com/influxdata/telegraf/agent.(*Agent).Run.func3(0xc420072fc0, 0xc42000e030, 0xc42042c120, 0xc42042c480, 0xc420192180, 0x3b9aca00)
Oct 24 19:39:51 testnet telegraf[44045]:         /home/ubuntu/telegraf-build/src/github.com/influxdata/telegraf/agent/agent.go:422 +0x7d
Oct 24 19:39:51 testnet telegraf[44045]: created by github.com/influxdata/telegraf/agent.(*Agent).Run
Oct 24 19:39:51 testnet telegraf[44045]:         /home/ubuntu/telegraf-build/src/github.com/influxdata/telegraf/agent/agent.go:423 +0x505
Oct 24 19:39:51 testnet telegraf[44045]: goroutine 63 [select, 72 minutes]:
Oct 24 19:39:51 testnet telegraf[44045]: github.com/influxdata/telegraf/agent.(*Agent).flusher.func2(0xc42018e920, 0xc42042c120, 0xc42042c4e0, 0xc42000e030, 0xc4202b5200)
Oct 24 19:39:51 testnet telegraf[44045]:         /home/ubuntu/telegraf-build/src/github.com/influxdata/telegraf/agent/agent.go:298 +0x2ae
Oct 24 19:39:51 testnet telegraf[44045]: created by github.com/influxdata/telegraf/agent.(*Agent).flusher
Oct 24 19:39:51 testnet telegraf[44045]:         /home/ubuntu/telegraf-build/src/github.com/influxdata/telegraf/agent/agent.go:315 +0x155
Oct 24 19:39:51 testnet telegraf[44045]: goroutine 83324 [select]:
Oct 24 19:39:51 testnet telegraf[44045]: github.com/shirou/gopsutil/internal/common.WaitTimeout(0xc4207c0000, 0xb2d05e00, 0x0, 0xc420aa32c0)
Oct 24 19:39:51 testnet telegraf[44045]:         /home/ubuntu/telegraf-build/src/github.com/shirou/gopsutil/internal/common/common.go:326 +0x279
Oct 24 19:39:51 testnet telegraf[44045]: github.com/shirou/gopsutil/internal/common.CombinedOutputTimeout(0xc4207c0000, 0xb2d05e00, 0xc4206e0b10, 0x3, 0x3, 0xc4207c0000, 0xc420263401)
Oct 24 19:39:51 testnet telegraf[44045]:         /home/ubuntu/telegraf-build/src/github.com/shirou/gopsutil/internal/common/common.go:314 +0xab
Oct 24 19:39:51 testnet telegraf[44045]: github.com/shirou/gopsutil/internal/common.Invoke.Command(0x14467c5, 0xd, 0xc4206e0b10, 0x3, 0x3, 0x411b18, 0x30, 0x1203cc0, 0xc420257d01, 0xc4206e0b10)
Oct 24 19:39:51 testnet telegraf[44045]:         /home/ubuntu/telegraf-build/src/github.com/shirou/gopsutil/internal/common/common.go:41 +0x71
Oct 24 19:39:51 testnet telegraf[44045]: github.com/shirou/gopsutil/internal/common.(*Invoke).Command(0x1d37f10, 0x14467c5, 0xd, 0xc4206e0b10, 0x3, 0x3, 0xb, 0x10000c4208284a8, 0xd, 0xe, ...)
Oct 24 19:39:51 testnet telegraf[44045]:         <autogenerated>:8 +0x81
Oct 24 19:39:51 testnet telegraf[44045]: github.com/shirou/gopsutil/disk.GetDiskSerialNumber(0xc4202e96dd, 0x4, 0x0, 0x40)
Oct 24 19:39:51 testnet telegraf[44045]:         /home/ubuntu/telegraf-build/src/github.com/shirou/gopsutil/disk/disk_linux.go:373 +0x1e5
Oct 24 19:39:51 testnet telegraf[44045]: github.com/shirou/gopsutil/disk.IOCounters(0x0, 0x0, 0x0, 0x47eb05, 0x0, 0x7)
Oct 24 19:39:51 testnet telegraf[44045]:         /home/ubuntu/telegraf-build/src/github.com/shirou/gopsutil/disk/disk_linux.go:358 +0x6b0
Oct 24 19:39:51 testnet telegraf[44045]: github.com/influxdata/telegraf/plugins/inputs/system.(*systemPS).DiskIO(0xc420014450, 0x0, 0x0, 0x0, 0x7, 0xc4203599ca, 0x436)
Oct 24 19:39:51 testnet telegraf[44045]:         /home/ubuntu/telegraf-build/src/github.com/influxdata/telegraf/plugins/inputs/system/ps.go:138 +0x43
Oct 24 19:39:51 testnet telegraf[44045]: github.com/influxdata/telegraf/plugins/inputs/system.(*DiskIOStats).Gather(0xc420238000, 0x1c2b1e0, 0xc4201fc100, 0x11e5d60, 0x0)
Oct 24 19:39:51 testnet telegraf[44045]:         /home/ubuntu/telegraf-build/src/github.com/influxdata/telegraf/plugins/inputs/system/disk.go:128 +0x66
Oct 24 19:39:51 testnet telegraf[44045]: github.com/influxdata/telegraf/agent.gatherWithTimeout.func1(0xc42076ade0, 0xc420192180, 0xc4201fc100)
Oct 24 19:39:51 testnet telegraf[44045]:         /home/ubuntu/telegraf-build/src/github.com/influxdata/telegraf/agent/agent.go:153 +0x49
Oct 24 19:39:51 testnet telegraf[44045]: created by github.com/influxdata/telegraf/agent.gatherWithTimeout
Oct 24 19:39:51 testnet telegraf[44045]:         /home/ubuntu/telegraf-build/src/github.com/influxdata/telegraf/agent/agent.go:154 +0xcc
Oct 24 19:39:51 testnet telegraf[44045]: goroutine 83508 [runnable]:
Oct 24 19:39:51 testnet telegraf[44045]: syscall.Syscall6(0xf7, 0x1, 0x14274, 0xc420936da0, 0x1000004, 0x0, 0x0, 0x0, 0xc420936da0, 0x0)
Oct 24 19:39:51 testnet telegraf[44045]:         /usr/local/go/src/syscall/asm_linux_amd64.s:44 +0x5
Oct 24 19:39:51 testnet telegraf[44045]: os.(*Process).blockUntilWaitable(0xc4206e0ba0, 0x1259660, 0xc420936e98, 0x41264c)
Oct 24 19:39:51 testnet telegraf[44045]:         /usr/local/go/src/os/wait_waitid.go:28 +0xa5
Oct 24 19:39:51 testnet telegraf[44045]: os.(*Process).wait(0xc4206e0ba0, 0x0, 0x0, 0x0)
Oct 24 19:39:51 testnet telegraf[44045]:         /usr/local/go/src/os/exec_unix.go:22 +0x4d
Oct 24 19:39:51 testnet telegraf[44045]: os.(*Process).Wait(0xc4206e0ba0, 0x0, 0x0, 0xc420936f88)
Oct 24 19:39:51 testnet telegraf[44045]:         /usr/local/go/src/os/exec.go:115 +0x2b
Oct 24 19:39:51 testnet telegraf[44045]: os/exec.(*Cmd).Wait(0xc4207c0000, 0xc420a61920, 0xc420936fb0)
Oct 24 19:39:51 testnet telegraf[44045]:         /usr/local/go/src/os/exec/exec.go:435 +0x62
Oct 24 19:39:51 testnet telegraf[44045]: github.com/shirou/gopsutil/internal/common.WaitTimeout.func1(0xc42076b3e0, 0xc4207c0000)
Oct 24 19:39:51 testnet telegraf[44045]:         /home/ubuntu/telegraf-build/src/github.com/shirou/gopsutil/internal/common/common.go:325 +0x2b
Oct 24 19:39:51 testnet telegraf[44045]: created by github.com/shirou/gopsutil/internal/common.WaitTimeout
Oct 24 19:39:51 testnet telegraf[44045]:         /home/ubuntu/telegraf-build/src/github.com/shirou/gopsutil/internal/common/common.go:325 +0x9a
Oct 24 19:39:51 testnet telegraf[44045]: goroutine 77339 [IO wait]:
Oct 24 19:39:51 testnet telegraf[44045]: net.runtime_pollWait(0x7f2abb868f88, 0x72, 0x3)
Oct 24 19:39:51 testnet telegraf[44045]:         /usr/local/go/src/runtime/netpoll.go:164 +0x59
Oct 24 19:39:51 testnet telegraf[44045]: net.(*pollDesc).wait(0xc4202bc378, 0x72, 0x1c1e8e0, 0x1c158a0)
Oct 24 19:39:51 testnet telegraf[44045]:         /usr/local/go/src/net/fd_poll_runtime.go:75 +0x38
Oct 24 19:39:51 testnet telegraf[44045]: net.(*pollDesc).waitRead(0xc4202bc378, 0xc42095f800, 0x400)
Oct 24 19:39:51 testnet telegraf[44045]:         /usr/local/go/src/net/fd_poll_runtime.go:80 +0x34
Oct 24 19:39:51 testnet telegraf[44045]: net.(*netFD).Read(0xc4202bc310, 0xc42095f800, 0x400, 0x400, 0x0, 0x1c1e8e0, 0x1c158a0)
Oct 24 19:39:51 testnet telegraf[44045]:         /usr/local/go/src/net/fd_unix.go:250 +0x1b7
Oct 24 19:39:51 testnet telegraf[44045]: net.(*conn).Read(0xc42007ec38, 0xc42095f800, 0x400, 0x400, 0x0, 0x0, 0x0)
Oct 24 19:39:51 testnet telegraf[44045]:         /usr/local/go/src/net/net.go:181 +0x70
Oct 24 19:39:51 testnet telegraf[44045]: crypto/tls.(*block).readFromUntil(0xc4206964b0, 0x1c1ab20, 0xc42007ec38, 0x5, 0xc42007ec38, 0x28)
Oct 24 19:39:51 testnet telegraf[44045]:         /usr/local/go/src/crypto/tls/conn.go:488 +0x98
Oct 24 19:39:51 testnet telegraf[44045]: crypto/tls.(*Conn).readRecord(0xc42038fc00, 0x14bb517, 0xc42038fd20, 0x0)
Oct 24 19:39:51 testnet telegraf[44045]:         /usr/local/go/src/crypto/tls/conn.go:590 +0xc4
Oct 24 19:39:51 testnet telegraf[44045]: crypto/tls.(*Conn).Read(0xc42038fc00, 0xc420889000, 0x1000, 0x1000, 0x0, 0x0, 0x0)
Oct 24 19:39:51 testnet telegraf[44045]:         /usr/local/go/src/crypto/tls/conn.go:1134 +0x11d
Oct 24 19:39:51 testnet telegraf[44045]: net/http.(*persistConn).Read(0xc42077f9e0, 0xc420889000, 0x1000, 0x1000, 0x5, 0xc4206be3c0, 0x21)
Oct 24 19:39:51 testnet telegraf[44045]:         /usr/local/go/src/net/http/transport.go:1316 +0x14b
Oct 24 19:39:51 testnet telegraf[44045]: bufio.(*Reader).fill(0xc4209652c0)
Oct 24 19:39:51 testnet telegraf[44045]:         /usr/local/go/src/bufio/bufio.go:97 +0x117
Oct 24 19:39:51 testnet telegraf[44045]: bufio.(*Reader).Peek(0xc4209652c0, 0x1, 0x0, 0x1, 0x0, 0xc420a609c0, 0x0)
Oct 24 19:39:51 testnet telegraf[44045]:         /usr/local/go/src/bufio/bufio.go:129 +0x67
Oct 24 19:39:51 testnet telegraf[44045]: net/http.(*persistConn).readLoop(0xc42077f9e0)
Oct 24 19:39:51 testnet telegraf[44045]:         /usr/local/go/src/net/http/transport.go:1474 +0x196
Oct 24 19:39:51 testnet telegraf[44045]: created by net/http.(*Transport).dialConn
Oct 24 19:39:51 testnet telegraf[44045]:         /usr/local/go/src/net/http/transport.go:1117 +0xa35
Oct 24 19:39:51 testnet telegraf[44045]: goroutine 83507 [runnable]:
Oct 24 19:39:51 testnet telegraf[44045]: os/exec.(*Cmd).Start.func1(0xc4207c0000, 0xc4206c4060)
Oct 24 19:39:51 testnet telegraf[44045]:         /usr/local/go/src/os/exec/exec.go:370
Oct 24 19:39:51 testnet telegraf[44045]: created by os/exec.(*Cmd).Start
Oct 24 19:39:51 testnet telegraf[44045]:         /usr/local/go/src/os/exec/exec.go:372 +0x4e4
Oct 24 19:39:51 testnet telegraf[44045]: goroutine 77340 [select]:
Oct 24 19:39:51 testnet telegraf[44045]: net/http.(*persistConn).writeLoop(0xc42077f9e0)
Oct 24 19:39:51 testnet telegraf[44045]:         /usr/local/go/src/net/http/transport.go:1704 +0x43a
Oct 24 19:39:51 testnet telegraf[44045]: created by net/http.(*Transport).dialConn
Oct 24 19:39:51 testnet telegraf[44045]:         /usr/local/go/src/net/http/transport.go:1118 +0xa5a
Oct 24 19:39:51 testnet telegraf[44045]: runtime stack:
Oct 24 19:39:51 testnet telegraf[44045]: runtime.throw(0x144d989, 0x10)
Oct 24 19:39:51 testnet telegraf[44045]:         /usr/local/go/src/runtime/panic.go:596 +0x95
Oct 24 19:39:51 testnet telegraf[44045]: runtime.(*workbuf).checknonempty(0x7f2abb943040)
Oct 24 19:39:51 testnet telegraf[44045]:         /usr/local/go/src/runtime/mgcwork.go:314 +0x4e
Oct 24 19:39:51 testnet telegraf[44045]: runtime.trygetfull(0x7f2abb869ae0)
Oct 24 19:39:51 testnet telegraf[44045]:         /usr/local/go/src/runtime/mgcwork.go:364 +0x45
Oct 24 19:39:51 testnet telegraf[44045]: runtime.(*gcWork).tryGet(0xc42001e528, 0xc42001e528)
Oct 24 19:39:51 testnet telegraf[44045]:         /usr/local/go/src/runtime/mgcwork.go:165 +0x91
Oct 24 19:39:51 testnet telegraf[44045]: runtime.gcDrain(0xc42001e528, 0xd)
Oct 24 19:39:51 testnet telegraf[44045]:         /usr/local/go/src/runtime/mgcmark.go:1075 +0x2eb
Oct 24 19:39:51 testnet telegraf[44045]: runtime.gcBgMarkWorker.func2()
Oct 24 19:39:51 testnet telegraf[44045]:         /usr/local/go/src/runtime/mgc.go:1534 +0x86
Oct 24 19:39:51 testnet telegraf[44045]: runtime.systemstack(0xc4205126c0)
Oct 24 19:39:51 testnet telegraf[44045]:         /usr/local/go/src/runtime/asm_amd64.s:327 +0x79
Oct 24 19:39:51 testnet telegraf[44045]: runtime.mstart()
Oct 24 19:39:51 testnet telegraf[44045]:         /usr/local/go/src/runtime/proc.go:1132
Oct 24 19:39:51 testnet telegraf[44045]: runtime stack:
Oct 24 19:39:51 testnet telegraf[44045]: runtime.throw(0x144d989, 0x10)
Oct 24 19:39:51 testnet telegraf[44045]:         /usr/local/go/src/runtime/panic.go:596 +0x95
Oct 24 19:39:51 testnet telegraf[44045]: runtime.(*workbuf).checknonempty(0x7f2abb8b8000)
Oct 24 19:39:51 testnet telegraf[44045]:         /usr/local/go/src/runtime/mgcwork.go:314 +0x4e
Oct 24 19:39:51 testnet telegraf[44045]: runtime.trygetfull(0x7f2abb937b48)
Oct 24 19:39:51 testnet telegraf[44045]:         /usr/local/go/src/runtime/mgcwork.go:364 +0x45
Oct 24 19:39:51 testnet telegraf[44045]: runtime.(*gcWork).tryGet(0xc420017228, 0xc420017228)
Oct 24 19:39:51 testnet telegraf[44045]:         /usr/local/go/src/runtime/mgcwork.go:165 +0x91
Oct 24 19:39:51 testnet telegraf[44045]: runtime.gcDrain(0xc420017228, 0x6)
Oct 24 19:39:51 testnet telegraf[44045]:         /usr/local/go/src/runtime/mgcmark.go:1075 +0x2eb
Oct 24 19:39:51 testnet telegraf[44045]: runtime.gcBgMarkWorker.func2()
Oct 24 19:39:51 testnet telegraf[44045]:         /usr/local/go/src/runtime/mgc.go:1530 +0xfa
Oct 24 19:39:51 testnet telegraf[44045]: runtime.systemstack(0xc420019900)
Oct 24 19:39:51 testnet telegraf[44045]:         /usr/local/go/src/runtime/asm_amd64.s:327 +0x79
Oct 24 19:39:51 testnet telegraf[44045]: runtime.mstart()
Oct 24 19:39:51 testnet telegraf[44045]:         /usr/local/go/src/runtime/proc.go:1132
Oct 24 19:39:51 testnet telegraf[44045]: runtime stack:
Oct 24 19:39:51 testnet telegraf[44045]: runtime.throw(0x144d989, 0x10)
Oct 24 19:39:51 testnet telegraf[44045]:         /usr/local/go/src/runtime/panic.go:596 +0x95
Oct 24 19:39:51 testnet telegraf[44045]: runtime.(*workbuf).checknonempty(0x7f2abb945840)
Oct 24 19:39:51 testnet telegraf[44045]:         /usr/local/go/src/runtime/mgcwork.go:314 +0x4e
Oct 24 19:39:51 testnet telegraf[44045]: runtime.trygetfull(0x7f2abb93f6c0)
Oct 24 19:39:51 testnet telegraf[44045]:         /usr/local/go/src/runtime/mgcwork.go:364 +0x45
Oct 24 19:39:51 testnet telegraf[44045]: runtime.(*gcWork).tryGet(0xc420018528, 0xc420018528)
Oct 24 19:39:51 testnet telegraf[44045]:         /usr/local/go/src/runtime/mgcwork.go:165 +0x91
Oct 24 19:39:51 testnet telegraf[44045]: runtime.gcDrain(0xc420018528, 0x6)
Oct 24 19:39:51 testnet telegraf[44045]:         /usr/local/go/src/runtime/mgcmark.go:1075 +0x2eb
Oct 24 19:39:51 testnet telegraf[44045]: runtime.gcBgMarkWorker.func2()
Oct 24 19:39:51 testnet telegraf[44045]:         /usr/local/go/src/runtime/mgc.go:1530 +0xfa
Oct 24 19:39:51 testnet telegraf[44045]: runtime.systemstack(0xc420016000)
Oct 24 19:39:51 testnet telegraf[44045]:         /usr/local/go/src/runtime/asm_amd64.s:327 +0x79
Oct 24 19:39:51 testnet telegraf[44045]: runtime.mstart()
Oct 24 19:39:51 testnet telegraf[44045]:         /usr/local/go/src/runtime/proc.go:1132
Oct 24 19:39:51 testnet telegraf[44045]: runtime stack:
Oct 24 19:39:51 testnet telegraf[44045]: runtime.throw(0x144d989, 0x10)
Oct 24 19:39:51 testnet telegraf[44045]:         /usr/local/go/src/runtime/panic.go:596 +0x95
Oct 24 19:39:51 testnet telegraf[44045]: runtime.(*workbuf).checknonempty(0x7f2abb943840)
Oct 24 19:39:51 testnet telegraf[44045]:         /usr/local/go/src/runtime/mgcwork.go:314 +0x4e
Oct 24 19:39:51 testnet telegraf[44045]: runtime.trygetfull(0x7f2abb940fb0)
Oct 24 19:39:51 testnet telegraf[44045]:         /usr/local/go/src/runtime/mgcwork.go:364 +0x45
Oct 24 19:39:51 testnet telegraf[44045]: runtime.(*gcWork).tryGet(0xc42001be28, 0xc42001be28)
Oct 24 19:39:51 testnet telegraf[44045]:         /usr/local/go/src/runtime/mgcwork.go:165 +0x91
Oct 24 19:39:51 testnet telegraf[44045]: runtime.gcDrain(0xc42001be28, 0xd)
Oct 24 19:39:51 testnet telegraf[44045]:         /usr/local/go/src/runtime/mgcmark.go:1075 +0x2eb
Oct 24 19:39:51 testnet telegraf[44045]: runtime.gcBgMarkWorker.func2()
Oct 24 19:39:51 testnet telegraf[44045]:         /usr/local/go/src/runtime/mgc.go:1534 +0x86
Oct 24 19:39:51 testnet telegraf[44045]: runtime.systemstack(0xc4207bc860)
Oct 24 19:39:51 testnet telegraf[44045]:         /usr/local/go/src/runtime/asm_amd64.s:327 +0x79
Oct 24 19:39:51 testnet telegraf[44045]: runtime.mstart()
Oct 24 19:39:51 testnet telegraf[44045]:         /usr/local/go/src/runtime/proc.go:1132
Oct 24 19:39:51 testnet telegraf[44045]: runtime stack:
Oct 24 19:39:51 testnet telegraf[44045]: runtime.throw(0x144d989, 0x10)
Oct 24 19:39:51 testnet telegraf[44045]:         /usr/local/go/src/runtime/panic.go:596 +0x95
Oct 24 19:39:51 testnet telegraf[44045]: runtime.(*workbuf).checknonempty(0x7f2abb981940)
Oct 24 19:39:51 testnet telegraf[44045]:         /usr/local/go/src/runtime/mgcwork.go:314 +0x4e
Oct 24 19:39:51 testnet telegraf[44045]: runtime.trygetfull(0x7f2abb9dcdb0)
Oct 24 19:39:51 testnet telegraf[44045]:         /usr/local/go/src/runtime/mgcwork.go:364 +0x45
Oct 24 19:39:51 testnet telegraf[44045]: runtime.(*gcWork).tryGet(0xc42001ab28, 0xc42001ab28)
Oct 24 19:39:51 testnet telegraf[44045]:         /usr/local/go/src/runtime/mgcwork.go:165 +0x91
Oct 24 19:39:51 testnet telegraf[44045]: runtime.gcDrain(0xc42001ab28, 0xd)
Oct 24 19:39:51 testnet telegraf[44045]:         /usr/local/go/src/runtime/mgcmark.go:1075 +0x2eb
Oct 24 19:39:51 testnet telegraf[44045]: runtime.gcBgMarkWorker.func2()
Oct 24 19:39:51 testnet telegraf[44045]:         /usr/local/go/src/runtime/mgc.go:1534 +0x86
Oct 24 19:39:51 testnet telegraf[44045]: runtime.systemstack(0xc42088a1e0)
Oct 24 19:39:51 testnet telegraf[44045]:         /usr/local/go/src/runtime/asm_amd64.s:327 +0x79
Oct 24 19:39:51 testnet telegraf[44045]: runtime.mstart()
Oct 24 19:39:51 testnet telegraf[44045]:         /usr/local/go/src/runtime/proc.go:1132
Oct 24 19:39:51 testnet telegraf[44045]: runtime stack:
Oct 24 19:39:51 testnet telegraf[44045]: runtime.throw(0x144d989, 0x10)
Oct 24 19:39:51 testnet telegraf[44045]:         /usr/local/go/src/runtime/panic.go:596 +0x95
Oct 24 19:39:51 testnet telegraf[44045]: runtime.(*workbuf).checknonempty(0x7f2abb97f140)
Oct 24 19:39:51 testnet telegraf[44045]:         /usr/local/go/src/runtime/mgcwork.go:314 +0x4e
Oct 24 19:39:51 testnet telegraf[44045]: runtime.trygetfull(0x7f2abb981140)
Oct 24 19:39:51 testnet telegraf[44045]:         /usr/local/go/src/runtime/mgcwork.go:364 +0x45
Oct 24 19:39:51 testnet telegraf[44045]: runtime.(*gcWork).init(0xc42001f828)
Oct 24 19:39:51 testnet telegraf[44045]:         /usr/local/go/src/runtime/mgcwork.go:91 +0x33
Oct 24 19:39:51 testnet telegraf[44045]: runtime.(*gcWork).tryGet(0xc42001f828, 0x0)
Oct 24 19:39:51 testnet telegraf[44045]:         /usr/local/go/src/runtime/mgcwork.go:156 +0xdb
Oct 24 19:39:51 testnet telegraf[44045]: runtime.gcDrain(0xc42001f828, 0xd)
Oct 24 19:39:51 testnet telegraf[44045]:         /usr/local/go/src/runtime/mgcmark.go:1075 +0x2eb
Oct 24 19:39:51 testnet telegraf[44045]: runtime.gcBgMarkWorker.func2()
Oct 24 19:39:51 testnet telegraf[44045]:         /usr/local/go/src/runtime/mgc.go:1534 +0x86
Oct 24 19:39:51 testnet telegraf[44045]: runtime.systemstack(0xc42081c1e0)
Oct 24 19:39:51 testnet telegraf[44045]:         /usr/local/go/src/runtime/asm_amd64.s:327 +0x79
Oct 24 19:39:51 testnet telegraf[44045]: runtime.mstart()
Oct 24 19:39:51 testnet telegraf[44045]:         /usr/local/go/src/runtime/proc.go:1132
Oct 24 19:39:51 testnet systemd[1]: telegraf.service: Main process exited, code=exited, status=2/INVALIDARGUMENT
Oct 24 19:39:51 testnet systemd[1]: telegraf.service: Unit entered failed state.
Oct 24 19:39:51 testnet systemd[1]: telegraf.service: Failed with result 'exit-code'.
Oct 24 19:39:51 testnet systemd[1]: telegraf.service: Service hold-off time over, scheduling restart.
Oct 24 19:39:51 testnet systemd[1]: Stopped The plugin-driven server agent for reporting metrics into InfluxDB.
Oct 24 19:39:51 testnet systemd[1]: Started The plugin-driven server agent for reporting metrics into InfluxDB.
Oct 24 19:39:51 testnet telegraf[82558]: 2017-10-24T19:39:51Z I! Starting Telegraf v1.4.2
Oct 24 19:39:51 testnet telegraf[82558]: 2017-10-24T19:39:51Z I! Loaded outputs: influxdb
Oct 24 19:39:51 testnet telegraf[82558]: 2017-10-24T19:39:51Z I! Loaded inputs: inputs.exec inputs.diskio
Oct 24 19:39:51 testnet telegraf[82558]: 2017-10-24T19:39:51Z I! Tags enabled: host=testnet service=bitcoind
Oct 24 19:39:51 testnet telegraf[82558]: 2017-10-24T19:39:51Z I! Agent Config: Interval:1s, Quiet:false, Hostname:"testnet", Flush Interval:10s
Oct 24 19:40:19 testnet telegraf[82558]: fatal error: unexpected signal during runtime execution
Oct 24 19:40:19 testnet telegraf[82558]: [signal SIGSEGV: segmentation violation code=0x1 addr=0x0 pc=0x40fb28]
Oct 24 19:40:19 testnet telegraf[82558]: runtime stack:
Oct 24 19:40:19 testnet telegraf[82558]: runtime.throw(0x14740ad, 0x2a)
Oct 24 19:40:19 testnet telegraf[82558]:         /usr/local/go/src/runtime/panic.go:596 +0x95
Oct 24 19:40:19 testnet telegraf[82558]: runtime.sigpanic()
Oct 24 19:40:19 testnet telegraf[82558]:         /usr/local/go/src/runtime/signal_unix.go:274 +0x2db
Oct 24 19:40:19 testnet telegraf[82558]: runtime.lfstackpop(0x1d15460, 0x170)
Oct 24 19:40:19 testnet telegraf[82558]:         /usr/local/go/src/runtime/lfstack.go:41 +0x18
Oct 24 19:40:19 testnet telegraf[82558]: runtime.trygetfull(0x7f8c723d5900)
Oct 24 19:40:19 testnet telegraf[82558]:         /usr/local/go/src/runtime/mgcwork.go:362 +0x2d
Oct 24 19:40:19 testnet telegraf[82558]: runtime.(*gcWork).tryGet(0xc420019828, 0xc420019828)
Oct 24 19:40:19 testnet telegraf[82558]:         /usr/local/go/src/runtime/mgcwork.go:165 +0x91
Oct 24 19:40:19 testnet telegraf[82558]: runtime.gcDrain(0xc420019828, 0xd)
Oct 24 19:40:19 testnet telegraf[82558]:         /usr/local/go/src/runtime/mgcmark.go:1075 +0x2eb
Oct 24 19:40:19 testnet telegraf[82558]: runtime.gcBgMarkWorker.func2()
Oct 24 19:40:19 testnet telegraf[82558]:         /usr/local/go/src/runtime/mgc.go:1534 +0x86
Oct 24 19:40:19 testnet telegraf[82558]: runtime.systemstack(0xc420016000)
Oct 24 19:40:19 testnet telegraf[82558]:         /usr/local/go/src/runtime/asm_amd64.s:327 +0x79
Oct 24 19:40:19 testnet telegraf[82558]: runtime.mstart()
Oct 24 19:40:19 testnet telegraf[82558]:         /usr/local/go/src/runtime/proc.go:1132
Oct 24 19:40:19 testnet telegraf[82558]: goroutine 22 [GC worker (idle)]:
Oct 24 19:40:19 testnet telegraf[82558]: runtime.systemstack_switch()
Oct 24 19:40:19 testnet telegraf[82558]:         /usr/local/go/src/runtime/asm_amd64.s:281 fp=0xc420023750 sp=0xc420023748
Oct 24 19:40:19 testnet telegraf[82558]: runtime.gcBgMarkWorker(0xc420018600)
Oct 24 19:40:19 testnet telegraf[82558]:         /usr/local/go/src/runtime/mgc.go:1537 +0x1fc fp=0xc4200237d8 sp=0xc420023750
Oct 24 19:40:19 testnet telegraf[82558]: runtime.goexit()
Oct 24 19:40:19 testnet telegraf[82558]:         /usr/local/go/src/runtime/asm_amd64.s:2197 +0x1 fp=0xc4200237e0 sp=0xc4200237d8
Oct 24 19:40:19 testnet telegraf[82558]: created by runtime.gcBgMarkStartWorkers
Oct 24 19:40:19 testnet telegraf[82558]:         /usr/local/go/src/runtime/mgc.go:1412 +0x98
Oct 24 19:40:19 testnet telegraf[82558]: goroutine 1 [semacquire]:
Oct 24 19:40:19 testnet telegraf[82558]: sync.runtime_Semacquire(0xc4200150ac)
Oct 24 19:40:19 testnet telegraf[82558]:         /usr/local/go/src/runtime/sema.go:47 +0x34
Oct 24 19:40:19 testnet telegraf[82558]: sync.(*WaitGroup).Wait(0xc4200150a0)
Oct 24 19:40:19 testnet telegraf[82558]:         /usr/local/go/src/sync/waitgroup.go:131 +0x7a
Oct 24 19:40:19 testnet telegraf[82558]: github.com/influxdata/telegraf/agent.(*Agent).Run(0xc420128118, 0xc420010360, 0x0, 0x0)
Oct 24 19:40:19 testnet telegraf[82558]:         /home/ubuntu/telegraf-build/src/github.com/influxdata/telegraf/agent/agent.go:426 +0x54e
Oct 24 19:40:19 testnet telegraf[82558]: main.reloadLoop(0xc420198960, 0x1d37f10, 0x0, 0x0, 0x1d37f10, 0x0, 0x0, 0x1d37f10, 0x0, 0x0, ...)
Oct 24 19:40:19 testnet telegraf[82558]:         /home/ubuntu/telegraf-build/src/github.com/influxdata/telegraf/cmd/telegraf/telegraf.go:234 +0xa46
Oct 24 19:40:19 testnet telegraf[82558]: main.main()
Oct 24 19:40:19 testnet telegraf[82558]:         /home/ubuntu/telegraf-build/src/github.com/influxdata/telegraf/cmd/telegraf/telegraf.go:411 +0x65b
Oct 24 19:40:19 testnet telegraf[82558]: goroutine 17 [syscall, locked to thread]:
Oct 24 19:40:19 testnet telegraf[82558]: runtime.goexit()
Oct 24 19:40:19 testnet telegraf[82558]:         /usr/local/go/src/runtime/asm_amd64.s:2197 +0x1
Oct 24 19:40:19 testnet telegraf[82558]: goroutine 18 [syscall]:
Oct 24 19:40:19 testnet telegraf[82558]: os/signal.signal_recv(0x0)
Oct 24 19:40:19 testnet telegraf[82558]:         /usr/local/go/src/runtime/sigqueue.go:116 +0x104
Oct 24 19:40:19 testnet telegraf[82558]: os/signal.loop()
Oct 24 19:40:19 testnet telegraf[82558]:         /usr/local/go/src/os/signal/signal_unix.go:22 +0x22
Oct 24 19:40:19 testnet telegraf[82558]: created by os/signal.init.1
Oct 24 19:40:19 testnet telegraf[82558]:         /usr/local/go/src/os/signal/signal_unix.go:28 +0x41
Oct 24 19:40:19 testnet telegraf[82558]: goroutine 66 [IO wait]:
Oct 24 19:40:19 testnet telegraf[82558]: net.runtime_pollWait(0x7f8c72220f00, 0x72, 0x3)
Oct 24 19:40:19 testnet telegraf[82558]:         /usr/local/go/src/runtime/netpoll.go:164 +0x59
Oct 24 19:40:19 testnet telegraf[82558]: net.(*pollDesc).wait(0xc42025a1b8, 0x72, 0x1c1e8e0, 0x1c158a0)
Oct 24 19:40:19 testnet telegraf[82558]:         /usr/local/go/src/net/fd_poll_runtime.go:75 +0x38
Oct 24 19:40:19 testnet telegraf[82558]: net.(*pollDesc).waitRead(0xc42025a1b8, 0xc4204eec00, 0x400)
Oct 24 19:40:19 testnet telegraf[82558]:         /usr/local/go/src/net/fd_poll_runtime.go:80 +0x34
Oct 24 19:40:19 testnet telegraf[82558]: net.(*netFD).Read(0xc42025a150, 0xc4204eec00, 0x400, 0x400, 0x0, 0x1c1e8e0, 0x1c158a0)
Oct 24 19:40:19 testnet telegraf[82558]:         /usr/local/go/src/net/fd_unix.go:250 +0x1b7
Oct 24 19:40:19 testnet telegraf[82558]: net.(*conn).Read(0xc420128058, 0xc4204eec00, 0x400, 0x400, 0x0, 0x0, 0x0)
Oct 24 19:40:19 testnet telegraf[82558]:         /usr/local/go/src/net/net.go:181 +0x70
Oct 24 19:40:19 testnet telegraf[82558]: crypto/tls.(*block).readFromUntil(0xc4204ec8d0, 0x1c1ab20, 0xc420128058, 0x5, 0xc420128058, 0x28)
Oct 24 19:40:19 testnet telegraf[82558]:         /usr/local/go/src/crypto/tls/conn.go:488 +0x98
Oct 24 19:40:19 testnet telegraf[82558]: crypto/tls.(*Conn).readRecord(0xc4201ef880, 0x14bb517, 0xc4201ef9a0, 0x0)
Oct 24 19:40:19 testnet telegraf[82558]:         /usr/local/go/src/crypto/tls/conn.go:590 +0xc4
Oct 24 19:40:19 testnet telegraf[82558]: crypto/tls.(*Conn).Read(0xc4201ef880, 0xc420179000, 0x1000, 0x1000, 0x0, 0x0, 0x0)
Oct 24 19:40:19 testnet telegraf[82558]:         /usr/local/go/src/crypto/tls/conn.go:1134 +0x11d
Oct 24 19:40:19 testnet telegraf[82558]: net/http.(*persistConn).Read(0xc4201f0000, 0xc420179000, 0x1000, 0x1000, 0x5, 0xc4204ec0c0, 0x21)
Oct 24 19:40:19 testnet telegraf[82558]:         /usr/local/go/src/net/http/transport.go:1316 +0x14b
Oct 24 19:40:19 testnet telegraf[82558]: bufio.(*Reader).fill(0xc420198d20)
Oct 24 19:40:19 testnet telegraf[82558]:         /usr/local/go/src/bufio/bufio.go:97 +0x117
Oct 24 19:40:19 testnet telegraf[82558]: bufio.(*Reader).Peek(0xc420198d20, 0x1, 0x0, 0x1, 0x0, 0xc4208d6b40, 0x0)
Oct 24 19:40:19 testnet telegraf[82558]:         /usr/local/go/src/bufio/bufio.go:129 +0x67
Oct 24 19:40:19 testnet telegraf[82558]: net/http.(*persistConn).readLoop(0xc4201f0000)
Oct 24 19:40:19 testnet telegraf[82558]:         /usr/local/go/src/net/http/transport.go:1474 +0x196
Oct 24 19:40:19 testnet telegraf[82558]: created by net/http.(*Transport).dialConn
Oct 24 19:40:19 testnet telegraf[82558]:         /usr/local/go/src/net/http/transport.go:1117 +0xa35
Oct 24 19:40:19 testnet telegraf[82558]: goroutine 41 [select]:
Oct 24 19:40:19 testnet telegraf[82558]: github.com/influxdata/telegraf/agent.(*Agent).flusher(0xc420128118, 0xc420010360, 0xc4200106c0, 0xc420010720, 0x45860c, 0x180001)
Oct 24 19:40:19 testnet telegraf[82558]:         /home/ubuntu/telegraf-build/src/github.com/influxdata/telegraf/agent/agent.go:320 +0x444
Oct 24 19:40:19 testnet systemd[1]: telegraf.service: Main process exited, code=exited, status=2/INVALIDARGUMENT
Oct 24 19:40:19 testnet telegraf[82558]: github.com/influxdata/telegraf/agent.(*Agent).Run.func1(0xc4200150a0, 0xc420128118, 0xc420010360, 0xc4200106c0, 0xc420010720)
Oct 24 19:40:19 testnet telegraf[82558]:         /home/ubuntu/telegraf-build/src/github.com/influxdata/telegraf/agent/agent.go:396 +0x7a
Oct 24 19:40:19 testnet telegraf[82558]: created by github.com/influxdata/telegraf/agent.(*Agent).Run
Oct 24 19:40:19 testnet telegraf[82558]:         /home/ubuntu/telegraf-build/src/github.com/influxdata/telegraf/agent/agent.go:400 +0x34f
Oct 24 19:40:19 testnet telegraf[82558]: goroutine 67 [select]:
Oct 24 19:40:19 testnet telegraf[82558]: net/http.(*persistConn).writeLoop(0xc4201f0000)
Oct 24 19:40:19 testnet telegraf[82558]:         /usr/local/go/src/net/http/transport.go:1704 +0x43a
Oct 24 19:40:19 testnet telegraf[82558]: created by net/http.(*Transport).dialConn
Oct 24 19:40:19 testnet telegraf[82558]:         /usr/local/go/src/net/http/transport.go:1118 +0xa5a
Oct 24 19:40:19 testnet telegraf[82558]: goroutine 23 [select, locked to thread]:
Oct 24 19:40:19 testnet telegraf[82558]: runtime.gopark(0x14bb4a8, 0x0, 0x143ae50, 0x6, 0x18, 0x2)
Oct 24 19:40:19 testnet telegraf[82558]:         /usr/local/go/src/runtime/proc.go:271 +0x13a
Oct 24 19:40:19 testnet telegraf[82558]: runtime.selectgoImpl(0xc420039f50, 0x0, 0x18)
Oct 24 19:40:19 testnet telegraf[82558]:         /usr/local/go/src/runtime/select.go:423 +0x1364
Oct 24 19:40:19 testnet telegraf[82558]: runtime.selectgo(0xc420039f50)
Oct 24 19:40:19 testnet telegraf[82558]:         /usr/local/go/src/runtime/select.go:238 +0x1c
Oct 24 19:40:19 testnet telegraf[82558]: runtime.ensureSigM.func1()
Oct 24 19:40:19 testnet telegraf[82558]:         /usr/local/go/src/runtime/signal_unix.go:434 +0x2dd
Oct 24 19:40:19 testnet telegraf[82558]: runtime.goexit()
Oct 24 19:40:19 testnet telegraf[82558]:         /usr/local/go/src/runtime/asm_amd64.s:2197 +0x1
Oct 24 19:40:19 testnet telegraf[82558]: goroutine 24 [select]:
Oct 24 19:40:19 testnet telegraf[82558]: main.reloadLoop.func1(0xc4200103c0, 0xc420010360, 0xc4202c82a0, 0xc420198960)
Oct 24 19:40:19 testnet telegraf[82558]:         /home/ubuntu/telegraf-build/src/github.com/influxdata/telegraf/cmd/telegraf/telegraf.go:195 +0x24a
Oct 24 19:40:19 testnet telegraf[82558]: created by main.reloadLoop
Oct 24 19:40:19 testnet telegraf[82558]:         /home/ubuntu/telegraf-build/src/github.com/influxdata/telegraf/cmd/telegraf/telegraf.go:209 +0x6ec
Oct 24 19:40:19 testnet telegraf[82558]: goroutine 42 [select]:
Oct 24 19:40:19 testnet telegraf[82558]: github.com/influxdata/telegraf/agent.(*Agent).gatherer(0xc420128118, 0xc420010360, 0xc420055d00, 0x3b9aca00, 0xc4200106c0)
Oct 24 19:40:19 testnet telegraf[82558]:         /home/ubuntu/telegraf-build/src/github.com/influxdata/telegraf/agent/agent.go:129 +0x3a8
Oct 24 19:40:19 testnet telegraf[82558]: github.com/influxdata/telegraf/agent.(*Agent).Run.func3(0xc4200150a0, 0xc420128118, 0xc420010360, 0xc4200106c0, 0xc420055d00, 0x3b9aca00)
Oct 24 19:40:19 testnet telegraf[82558]:         /home/ubuntu/telegraf-build/src/github.com/influxdata/telegraf/agent/agent.go:422 +0x7d
Oct 24 19:40:19 testnet telegraf[82558]: created by github.com/influxdata/telegraf/agent.(*Agent).Run
Oct 24 19:40:19 testnet telegraf[82558]:         /home/ubuntu/telegraf-build/src/github.com/influxdata/telegraf/agent/agent.go:423 +0x505
Oct 24 19:40:19 testnet telegraf[82558]: goroutine 43 [select]:
Oct 24 19:40:19 testnet systemd[1]: telegraf.service: Unit entered failed state.
Oct 24 19:40:19 testnet telegraf[82558]: github.com/influxdata/telegraf/agent.gatherWithTimeout(0xc420010360, 0xc420055d80, 0xc420126060, 0x3b9aca00)
Oct 24 19:40:19 testnet telegraf[82558]:         /home/ubuntu/telegraf-build/src/github.com/influxdata/telegraf/agent/agent.go:157 +0x28f
Oct 24 19:40:19 testnet telegraf[82558]: github.com/influxdata/telegraf/agent.(*Agent).gatherer(0xc420128118, 0xc420010360, 0xc420055d80, 0x3b9aca00, 0xc4200106c0)
Oct 24 19:40:19 testnet telegraf[82558]:         /home/ubuntu/telegraf-build/src/github.com/influxdata/telegraf/agent/agent.go:124 +0x2a4
Oct 24 19:40:19 testnet telegraf[82558]: github.com/influxdata/telegraf/agent.(*Agent).Run.func3(0xc4200150a0, 0xc420128118, 0xc420010360, 0xc4200106c0, 0xc420055d80, 0x3b9aca00)
Oct 24 19:40:19 testnet telegraf[82558]:         /home/ubuntu/telegraf-build/src/github.com/influxdata/telegraf/agent/agent.go:422 +0x7d
Oct 24 19:40:19 testnet telegraf[82558]: created by github.com/influxdata/telegraf/agent.(*Agent).Run
Oct 24 19:40:19 testnet telegraf[82558]:         /home/ubuntu/telegraf-build/src/github.com/influxdata/telegraf/agent/agent.go:423 +0x505
Oct 24 19:40:19 testnet telegraf[82558]: goroutine 30 [select]:
Oct 24 19:40:19 testnet telegraf[82558]: github.com/influxdata/telegraf/agent.(*Agent).flusher.func2(0xc420230040, 0xc420010360, 0xc420010720, 0xc420128118, 0xc420010a80)
Oct 24 19:40:19 testnet telegraf[82558]:         /home/ubuntu/telegraf-build/src/github.com/influxdata/telegraf/agent/agent.go:298 +0x2ae
Oct 24 19:40:19 testnet telegraf[82558]: created by github.com/influxdata/telegraf/agent.(*Agent).flusher
Oct 24 19:40:19 testnet telegraf[82558]:         /home/ubuntu/telegraf-build/src/github.com/influxdata/telegraf/agent/agent.go:315 +0x155
Oct 24 19:40:19 testnet telegraf[82558]: goroutine 29 [select]:
Oct 24 19:40:19 testnet telegraf[82558]: github.com/influxdata/telegraf/agent.(*Agent).flusher.func1(0xc420230040, 0xc420010360, 0xc420010a80, 0xc420128118)
Oct 24 19:40:19 testnet telegraf[82558]:         /home/ubuntu/telegraf-build/src/github.com/influxdata/telegraf/agent/agent.go:263 +0x2f3
Oct 24 19:40:19 testnet telegraf[82558]: created by github.com/influxdata/telegraf/agent.(*Agent).flusher
Oct 24 19:40:19 testnet telegraf[82558]:         /home/ubuntu/telegraf-build/src/github.com/influxdata/telegraf/agent/agent.go:292 +0xe5
Oct 24 19:40:19 testnet telegraf[82558]: goroutine 587 [runnable]:
Oct 24 19:40:19 testnet telegraf[82558]: github.com/shirou/gopsutil/internal/common.WaitTimeout(0xc420819760, 0xb2d05e00, 0x0, 0xc420124540)
Oct 24 19:40:19 testnet telegraf[82558]:         /home/ubuntu/telegraf-build/src/github.com/shirou/gopsutil/internal/common/common.go:326 +0x279
Oct 24 19:40:19 testnet telegraf[82558]: github.com/shirou/gopsutil/internal/common.CombinedOutputTimeout(0xc420819760, 0xb2d05e00, 0xc420970870, 0x3, 0x3, 0xc420819760, 0xc42081f808)
Oct 24 19:40:19 testnet telegraf[82558]:         /home/ubuntu/telegraf-build/src/github.com/shirou/gopsutil/internal/common/common.go:314 +0xab
Oct 24 19:40:19 testnet systemd[1]: telegraf.service: Failed with result 'exit-code'.
Oct 24 19:40:19 testnet telegraf[82558]: github.com/shirou/gopsutil/internal/common.Invoke.Command(0x14467c5, 0xd, 0xc420970870, 0x3, 0x3, 0xc420970890, 0xc4204c2130, 0x30, 0xc4204c2101, 0xc420970870)
Oct 24 19:40:19 testnet telegraf[82558]:         /home/ubuntu/telegraf-build/src/github.com/shirou/gopsutil/internal/common/common.go:41 +0x71
Oct 24 19:40:19 testnet telegraf[82558]: github.com/shirou/gopsutil/internal/common.(*Invoke).Command(0x1d37f10, 0x14467c5, 0xd, 0xc420970870, 0x3, 0x3, 0xb, 0x10000c42077d6d8, 0xd, 0xe, ...)
Oct 24 19:40:19 testnet telegraf[82558]:         <autogenerated>:8 +0x81
Oct 24 19:40:19 testnet telegraf[82558]: github.com/shirou/gopsutil/disk.GetDiskSerialNumber(0xc42039ea8d, 0x4, 0x0, 0x40)
Oct 24 19:40:19 testnet telegraf[82558]:         /home/ubuntu/telegraf-build/src/github.com/shirou/gopsutil/disk/disk_linux.go:373 +0x1e5
Oct 24 19:40:19 testnet telegraf[82558]: github.com/shirou/gopsutil/disk.IOCounters(0x0, 0x0, 0x0, 0x4a985ef260184a02, 0x7d33ea864b993ebb, 0x8b833fefb17979c1)
Oct 24 19:40:19 testnet telegraf[82558]:         /home/ubuntu/telegraf-build/src/github.com/shirou/gopsutil/disk/disk_linux.go:358 +0x6b0
Oct 24 19:40:19 testnet telegraf[82558]: github.com/influxdata/telegraf/plugins/inputs/system.(*systemPS).DiskIO(0xc420122420, 0x0, 0x0, 0x0, 0x1d35e21c43ddc96, 0xe46cb6e13be6411, 0x9e810f3f875549fb)
Oct 24 19:40:19 testnet telegraf[82558]:         /home/ubuntu/telegraf-build/src/github.com/influxdata/telegraf/plugins/inputs/system/ps.go:138 +0x43
Oct 24 19:40:19 testnet telegraf[82558]: github.com/influxdata/telegraf/plugins/inputs/system.(*DiskIOStats).Gather(0xc4202c8310, 0x1c2b1e0, 0xc420126060, 0x0, 0x0)
Oct 24 19:40:19 testnet telegraf[82558]:         /home/ubuntu/telegraf-build/src/github.com/influxdata/telegraf/plugins/inputs/system/disk.go:128 +0x66
Oct 24 19:40:19 testnet telegraf[82558]: github.com/influxdata/telegraf/agent.gatherWithTimeout.func1(0xc42098ef00, 0xc420055d80, 0xc420126060)
Oct 24 19:40:19 testnet telegraf[82558]:         /home/ubuntu/telegraf-build/src/github.com/influxdata/telegraf/agent/agent.go:153 +0x49
Oct 24 19:40:19 testnet telegraf[82558]: created by github.com/influxdata/telegraf/agent.gatherWithTimeout
Oct 24 19:40:19 testnet telegraf[82558]:         /home/ubuntu/telegraf-build/src/github.com/influxdata/telegraf/agent/agent.go:154 +0xcc
Oct 24 19:40:19 testnet systemd[1]: telegraf.service: Service hold-off time over, scheduling restart.
Oct 24 19:40:19 testnet systemd[1]: Stopped The plugin-driven server agent for reporting metrics into InfluxDB.
Oct 24 19:40:19 testnet systemd[1]: Started The plugin-driven server agent for reporting metrics into InfluxDB.

@danielnelson danielnelson mentioned this issue Oct 24, 2017
Merged
3 tasks
@danielnelson danielnelson added this to the 1.4.3 milestone Oct 24, 2017
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
bug unexpected problem or unintended behavior panic issue that results in panics from Telegraf
Projects
None yet
Milestone
1.4.3
Development

No branches or pull requests
2 participants
@danielnelson @jackzampolin