pprof: create HTTP endpoint for setting MutexProfileFraction

[Kubuxu]

Allows to dynamically change the MutexProfileFraction to enable and
disable mutex profiling. It should be very useful for detecting
deadlocks, lock contention and general concurrency problems.

How to use:
To enable: curl -X POST -v 'localhost:5001/debug/pprof-mutex/?fraction=10
To disable: curl -X POST -v 'localhost:5001/debug/pprof-mutex/?fraction=0'

Fraction defines which fraction of events will be profiled. Higher it is
the lower performance impact but less reliable the result.

To fetch the result use:
go tool pprof $PATH_TO_IPFS_BIN http://localhost:5001/debug/pprof/mutex

License: MIT

[magik6k]

Code LGTM, but I wonder if it wouldn't be better to do this as a more generalized endpoint to multiple internal knobs / hardcoded values. Also, is there a reason this can't be just a command (like ipfs debug set)?

[Kubuxu]

It could be a command, but is there a reason for it to be a command? No normal user is expected to use it directly and if we are asking someone for pprof dumps we send them commands either way (in future we can create collection util).

As for more generic endpoint we can think about it but please, let's not block this PR on figuring out a general solution.

[Stebalien]

I actually prefer this over a command as commands tend to become a part of our API. This is really just a debugging endpoint.

Now, at some point in the future we may add a debug command. At that point, we can reconsider.

---

@Kubuxu this should have at least one sharness test (just turn it on and off and make sure nothing errors).

[Stebalien]

Otherwise, LGTM.

[Kubuxu]

Tests added.

[Kubuxu]

Fixed the tabs, not even sure where they were.

[Stebalien]

@magik6k are you fine with merging this for now (given that we already have a debug endpoint like this) or do you feel that it really needs to be a command?

[Kubuxu]

From another side, IMO, his shouldn't be an API. It is specific to Go only, and only because this option is not exposed under existing pprof endpoints.

It is the essence of implementation detail as an implementation detail can be.

[Stebalien]

t is specific to Go only, and only because this option is not exposed under existing pprof endpoints.

I agree that it's a go-ipfs specific implementation detail but that doesn't necessarily mean we can't expose it over the HTTP API (like our other ipfs diag commands).

[magik6k]

LGTM, It was more of a suggestion about the command.

[Stebalien]

Yeah, we should probably come up with a comprehensive story around debugging. I just kind of doubt it'll get done any time soon.