Deadlock in http.Client

[AyWa]

versions

• go version go1.16.3 darwin/amd64
• go.elastic.co/apm v1.12.0
• go.elastic.co/apm/module/apmhttp v1.12.0

Bug

When the default http client is wrap with apmhttp.WrapClient(http.DefaultClient, apmhttp.WithClientTrace()) there is some case where an http request, will never answer. Even if the request has a context deadline.

To Reproduce

It is little hard to reproduce so I made code to reproduce it: https://github.com/AyWa/goapmdeadlock
I guess the scenario is like that:

• make many sub request that timeout concurrently. In my example I do 50 sub request concurrently https://github.com/AyWa/goapmdeadlock/blob/main/main.go#L68 * 100 times.

Expected behavior

The http.Client should not be in deadlock when wrap by apmhttp ( I think the deadlock happen for a given trace id so new request might work fine)

[axw]

@AyWa thank you for the detailed issue and reproducer - I can reproduce the issue with your program. I haven't gotten to the bottom of it yet, but it appears to be related to the use of apmhttp.WithClientTrace(). At least, when I remove just that option it no longer reproduces.

[axw]

Sorry for the delay in getting back to you. I've found a couple of different concurrency bugs, both of which are addressed in #997.

[axw]

@AyWa we've just released v1.13.1 that should fix this issue. Please upgrade and let us know how it goes.

[AyWa]

@axw Thank you a lot for the quick fix. I will try it soon -> I can not reproduce it 👍