[feature] Instrument the API server(s) with pprof

[els0r]

Provides facilities to hook up a net/http/pprof server on /debug/pprof.

The majority of this commit will unify the server base across all binaries, which serve an API, in this case goProbe and global-query. This is primarily to deduplicate code for functionalit that is desired in both APIs.

To this end, and analogous to the client.DefaultClient a server.DefaultServer is introduced to cover observability middlewares as well as setting up the gin.Engine

Closes #142

[els0r]

@fako1024 : I wanted to do this first before providing the profiles for #138 . There's only one caveat: the profiles are empty if I pull them with wget http://host:port/debug/pprof/profile?seconds=300. However, if I hook into the server directly, using go tool pprof http://host:port/debug/pprof/profile, I do get profiling results.

Have you encountered this before? Am I missing something obvious?

[fako1024]

@fako1024 : I wanted to do this first before providing the profiles for #138 . There's only one caveat: the profiles are empty if I pull them with wget http://host:port/debug/pprof/profile?seconds=300. However, if I hook into the server directly, using go tool pprof http://host:port/debug/pprof/profile, I do get profiling results.

Have you encountered this before? Am I missing something obvious?

I do recall something like that. Maybe it has something to do with the way wget does the request. Can you try this (which seems to be the commonly referenced way of doing it):

curl -s http://host:port/debug/pprof/profile?seconds=60 > ./cpu.prof

[fako1024]

Aside from some minor mimimi, LGTM!

[els0r]

I do recall something like that. Maybe it has something to do with the way wget does the request. Can you try this (which seems to be the commonly referenced way of doing it):

curl -s http://host:port/debug/pprof/profile?seconds=60 > ./cpu.prof

Yup. That was it. Thanks! In that case, I'll wrap up this PR with the feedback and then deploy it on the sensors to collect profiles. This is getting better by the minute :)