DIG from external network not working the same as dig from LAN

[tpietsch]

dig @[myhost].com zookeeper.service.consul SRV

on the local network consul is running on returns an answer

; (1 server found)
;; global options: +cmd
;; Got answer:
;; ->>HEADER<<- opcode: QUERY, status: NOERROR, id: 28494
;; flags: qr aa rd ra; QUERY: 1, ANSWER: 1, AUTHORITY: 0, ADDITIONAL: 1

;; QUESTION SECTION:
;zookeeper.service.consul. IN SRV

;; ANSWER SECTION:
zookeeper.service.consul. 0 IN SRV 1 1 2181 ubuntu.node.dc1.consul.

;; ADDITIONAL SECTION:
ubuntu.node.dc1.consul. 0 IN A [external public ip]

;; Query time: 5 msec
;; SERVER: [hostip]#53([hostip])
;; WHEN: Fri Nov 27 10:04:08 PST 2015
;; MSG SIZE rcvd: 146

but running the same command returns no answers when run on an external network.

[ryanbreen]

The external network needs to be explicitly configured to forward requests for the .consul TLD to your consul cluster. This generally isn't possible on external networks because odds are you don't control the DNS configuration for all clients.

The typical approach is to use Consul for DNS-based service discovery inside your network and expose services via a load balancer (hardware or software) behind a publicly resolvable hostname. So, you would point myapp.mydomain.com at your load balancer which would route traffic to zookeeper.service.consul.

[tpietsch]

I have consul running as my DNS server on 53
telnet [myhost] 53 resolves

I would think if i am requesting DNS records from consul it would already know how to resolve the .consul addresses

[ryanbreen]

What error do you get when you run from the external network?

[tpietsch]

I don't see an error i just get a different DNS response

; <<>> DiG 9.8.3-P1 <<>> @[myhost].com zookeeper.service.consul. A
; (1 server found)
;; global options: +cmd
;; Got answer:
;; ->>HEADER<<- opcode: QUERY, status: NXDOMAIN, id: 61000
;; flags: qr rd ra; QUERY: 1, ANSWER: 0, AUTHORITY: 1, ADDITIONAL: 0

;; QUESTION SECTION:
;zookeeper.service.consul. IN A

;; AUTHORITY SECTION:
. 9027 IN SOA a.root-servers.net. nstld.verisign-grs.com. 2015112701 1800 900 604800 86400

;; Query time: 2 msec
;; SERVER: [hostip]#53(hostip)
;; WHEN: Fri Nov 27 10:19:56 2015
;; MSG SIZE rcvd: 117

I have recurse also configured in my consul cluster so if it cant resolve something it would it will allow another Public DNS server to do so

[tpietsch]

From what i can see consul does not know how to resolve so it passes it to another DNS server which eventually resolve to a root authoritative one

[tpietsch]

I have also tried my service configured to both an internal/public IP and the same behavior was observed

[ryanbreen]

Please paste the dig command you're running externally and its results.

[tpietsch]

dig @treverpietsch.com zookeeper.service.consul SRV

; <<>> DiG 9.8.3-P1 <<>> @treverpietsch.com zookeeper.service.consul SRV
; (1 server found)
;; global options: +cmd
;; Got answer:
;; ->>HEADER<<- opcode: QUERY, status: NXDOMAIN, id: 37646
;; flags: qr rd ra; QUERY: 1, ANSWER: 0, AUTHORITY: 1, ADDITIONAL: 0

;; QUESTION SECTION:
;zookeeper.service.consul. IN SRV

;; AUTHORITY SECTION:
. 8119 IN SOA a.root-servers.net. nstld.verisign-grs.com. 2015112701 1800 900 604800 86400

;; Query time: 5 msec
;; SERVER: 67.188.113.121#53(67.188.113.121)
;; WHEN: Fri Nov 27 10:35:03 2015
;; MSG SIZE rcvd: 117

same command internal/external has different answers

my initial post has the response form a local network

[tpietsch]

also THANKS for the speedy response :)

[ryanbreen]

OK, and you are certain that port 53 on treverpietsch.com forwards to the DNS port on a node in your Consul cluster? I can't think of a reason why the Consul server would give a different answer to dig if the same command is being run, regardless of what network it originated from. dig is just making a DNS query to a service; there's no magic in there based on what network it's coming from.

Can you paste the config from your Consul server?

Also, I noticed that in your example output from dig on an external network, the response is for an A record, not a SRV record. I wouldn't expect that to be the case if the command you ran was dig @treverpietsch.com zookeeper.service.consul SRV.

[tpietsch]

{
"bind_addr" : "10.0.1.6",
"client_addr" : "10.0.1.6",
"ui_dir":"/etc/consul.d/dist/",
"data_dir": "/var/consul",

    "bootstrap" : true,

"server":true,
"recursor": "10.0.1.1",
"log_level": "INFO",
"ports":{
"dns": 53
},
"dns_config" : {

"enable_truncate" : true
}
}

[tpietsch]

{"service": {
"checks": [
{
"script": "/usr/share/zookeeper/bin/zkServer.sh status",
"interval": "10s"
}
],
"name": "zookeeper", "tags": ["zookeeper"], "address": "10.0.1.6", "port": 2181}}

[tpietsch]

i agree with you statement that is why i am confused.

im literally running dig using the same dns host from different machines
and getting different responses

[tpietsch]

the dig does return an SRV record in the initial post

[tpietsch]

;; ANSWER SECTION:
zookeeper.service.consul. 0 IN SRV 1 1 2181 ubuntu.node.dc1.consul.

[ryanbreen]

Right, I was referring to this follow-up:

I don't see an error i just get a different DNS response

; <<>> DiG 9.8.3-P1 <<>> @[myhost].com zookeeper.service.consul. A

[tpietsch]

maybe because there are no answers A records are like a fall back? I would need to do some research on that not super familiar with anything above the basic DNS stuff.

the dig versions are different on the machines? When i was on my LAN i was able to use consul as my DNS server tho so i doubt that is the issue.