Allow specifying agent host in config

[bendemaree]

This allows setting the Jaeger agent host to something other than localhost in config. This scenario is also noted in #47 and this implementation follows the notes given there.

Fixes #47

[CLAassistant]

Thank you for your submission, we really appreciate it. Like many open source projects, we ask that you sign our Contributor License Agreement before we can accept your contribution.

---

Ben Demaree seems not to be a GitHub user. You need a GitHub account to be able to sign the CLA. If you have already a GitHub account, please add the email address used for this commit to your account.
_{You have signed the CLA already but the status is still pending? Let us recheck it.}

[coveralls]

Coverage decreased (-0.2%) to 94.184% when pulling 271c62e on bendemaree:local-agent-host into 3315f85 on uber:master.

[coveralls]

Coverage decreased (-0.2%) to 94.184% when pulling c749fb1 on bendemaree:local-agent-host into 3315f85 on uber:master.

[bharat-p]

Hey @bendemaree were you able to get it to work to send span/trace to agent running in docker all-in-one image running on a remote host. I am trying to get it to work but so far no luck.

[yurishkuro]

@bendemaree do you mind adding a unit test?

[bharat-p]

Update: So I was able to get traces sent to remote agent, when agent was running in a VM ( via virtualbox) on same mac where client was.

When Jaeger agent was on a different VM running elsewhere, I didn't get any traces and tcpdump kept showing UDP error, it could have been because of the our corp network, will debug it more to see if I can consistently reproduce the issue in a different network setup

[yurishkuro]

@bharat-p fyi, the main UDP sender was really not meant to be used over the network, the main assumption was that the agent would run on the same host and the UDP connection will be to 127.0.0.1

If you cannot make such deployment work, I think a better option would be to implement an HTTP sender. In fact, in that case you won't even need to run the agent, because we're adding HTTP endpoints to the collectors directly.

The Java and Go clients already have HTTP-based reporters.

[bendemaree]

@yurishkuro I'm happy to add a test, yes. Should I hold off? Like @bharat-p I had difficulties getting requests to actually hit the agent, even after the updated host. The localhost-only strategy is an impediment for me locally and in a potential deployment; I'd hope to have the Jaeger stack pulled in for local development with docker-compose and in a potential deployment, as a sidecar to a Kubernetes deployment (with N homogenous pods on a node talking to a Jaeger agent on that node).

How much effort do you think adding an HTTP sender is? I can attempt adding that but I could use some high-level implementation guidance.

[yurishkuro]

@bendemaree Given that using another host doesn't work too well, I think it makes sense not to merge this at all. Could you explain though why you cannot run the agent as a side car on the same bare metal host? E.g. NewRelic is using the same mechanism as Jaeger: https://github.com/kubernetes/kubernetes/tree/master/examples/newrelic

[bendemaree]

@yurishkuro Whoops, I messed up there; I see now that localhost should work fine under Kube. Thanks. 😁

I agree; if this enables broken configuration it should just be closed. ISTM the issue for folks here (myself included) is running the agent as part of a local Docker stack in which localhost isn't available for the agent. I just want to clarify that implementing the HTTP sender is the path forward here; is there some other option that should be considered?

[yurishkuro]

Technically there are many options of getting spans out of process, e.g. gRPC or Kafka, but we want to balance it against adding more dependencies to the library. An HTTP sender can be implemented with the dependencies we already have, and it works better than UDP when the agent on the same host is not available (e.g. we have a similar use case when running some services in the cloud).

[bharat-p]

@bendemaree / @yurishkuro Would you mind explaining how it works in K8S world ( I am not too familiar with it). How does a service running in K8S as container can reach agent ( which probably is running in another container) via localhost?

[yurishkuro]

@bharat-p you may want to take a look at https://github.com/jaegertracing/jaeger-kubernetes, and also open a ticket there if you have questions like ^^ (I'm not sure I know the correct answer as we're not running Jaeger on K8S internally).

[pavolloffay]

I'm syncing on this, k8s templates require you to configure the address to point to the agent/(or collector) as the agent/collector does not run on localhost.

In production k8s/openshift templates we decided to run agent as https://kubernetes.io/docs/concepts/workloads/controllers/daemonset/ so all pods on the same node will send spans to the same agent. The agent address is not a localhost but something like this jaeger-agent.${NAMESPACE}.svc.cluster.local. It means that data will go through some virtual interfaces to reach the right pod, I'm not sure if this can mean that some data can be lost compared to running it to localhost

cc @jpkrohling.

[bharat-p]

Thanks @pavolloffay , so I guess this PR is still relevant as without it we won't be able to configure client to send trace info to agent address like jaeger-agent.${NAMESPACE}.svc.cluster.local

[yurishkuro]

@bharat-p +1. Can you add a unit test for this property? Then we can merge.

[bharat-p]

Original PR was opened by @bendemaree , he will have to add UT for new config. If he's ok, I can open a new PR with his change + UT.

[yurishkuro]

oh, sorry - yeah, either way.

[bendemaree]

I'll get a test up today!

[23doors]

Any progress on that?

[bharat-p]

@bendemaree if you are tied up with other things, let me know and I will open a new PR with UT to get this merged in. Thanks.

[bharat-p]

Opened a new PR #51 with @bendemaree 's changes alsong with UT to test new config.

[yurishkuro]

Closing, done in #51