[Feature request] wait_for_txt - show failed lookups

[Kariton]

SUMMARY

This is a request for the module wait_for_txt.

• Add the ability to define the "authoritative DNS server"
• Show each try - even failed ones like an until loop does

ISSUE TYPE

• Feature Idea

COMPONENT NAME

wait_for_txt

ADDITIONAL INFORMATION

I need to deal with split-DNS - and from my ansible host perspective the authoritative DNS server never knows the acme TXT record. Fast-forward: This is my problem with the awesome felixfontein.acme collection.

my solution for now:

- name: Wait for DNS entries to propagate
  ansible.builtin.command:
    argv:
      - dig
      - -t
      - TXT
      - +short
      - "@{{ acme_certificate_lookup_dns | d(ansible_facts['dns']['nameservers']) | list | random }}"
      - "{{ item.key }}"
  register: _acme_challenge_lookup
  until: _acme_challenge_lookup.stdout[1:-1] == item.value | first
  retries: 3
  delay: 300
  delegate_to: localhost
  run_once: true
  changed_when: false
  with_dict: "{{ acme_certificate_INTERNAL_challenge.challenge_data_dns }}"

BUT this just verify ONE random DNS server - not great, not terrible.
What I really like about this until is that it show me how often it fails while it is running. I really much appreciate this.

example for the fixed module:
var with real authoritative DNS server ;)

acme_certificate_lookup_dns: ['9.9.9.9', '1.1.1.1']
or
acme_certificate_lookup_dns: '9.9.9.9'

task

- name: Wait for DNS entries to propagate
  community.dns.wait_for_txt:
    dns_resolver: "{{ acme_certificate_lookup_dns | d(ansible_facts['dns']['nameservers']) | list }}"
    records: >-
      {{ acme_certificate_INTERNAL_challenge.challenge_data_dns | dict2items(key_name='name', value_name='values') | list }}
    timeout: 120
  delegate_to: localhost
  run_once: true

EDIT1:
There is now a better more complete workaround - which checks ALL defined (default) DNS server:
#72 (comment)

[felixfontein]

Makes sense to have such an option! I'll work on this later this week / next week (I'm pretty packed with other stuff right now).

About the number of failed tries: the result of the module already contains check_count which gives the number of checks performed in total (https://github.com/ansible-collections/community.dns/blob/main/plugins/modules/wait_for_txt.py#L277). This information is per domain, not per DNS server name though:

        {
            "check_count": 1,
            "done": true,
            "name": "github.io",
            "values": {
                "dns1.p05.nsone.net.": [
                    "v=spf1 a -all"
                ],
                "dns2.p05.nsone.net.": [
                    "v=spf1 a -all"
                ],
                "dns3.p05.nsone.net.": [
                    "v=spf1 a -all"
                ],
                "dns4.p05.nsone.net.": [
                    "v=spf1 a -all"
                ],
                "ns-1339.awsdns-39.org.": [
                    "v=spf1 a -all"
                ],
                "ns-1622.awsdns-10.co.uk.": [
                    "v=spf1 a -all"
                ],
                "ns-393.awsdns-49.com.": [
                    "v=spf1 a -all"
                ],
                "ns-692.awsdns-22.net.": [
                    "v=spf1 a -all"
                ]
            }
        }

(In this case check_count is 1 because it succeeded on the first try.)

[Kariton]

What you describe is just the fact that the module itself includes the check_count within the returned json.
What I mean is more of a debug output to the console.

Like this:

TASK [Wait until] *******************************************************************************************************************************************
FAILED - RETRYING: Wait until. (10 retries left).
FAILED - RETRYING: Wait until. (9 retries left).
FAILED - RETRYING: Wait until. (8 retries left).
FAILED - RETRYING: Wait until. (7 retries left).
FAILED - RETRYING: Wait until. (6 retries left).
FAILED - RETRYING: Wait until. (5 retries left).
FAILED - RETRYING: Wait until. (4 retries left).

[Kariton]

Just for anyone looking for this,
here is a proper check for my quiet incomplete workaround in the beginning.

This includes the DNS check as an own file and a variable to define alternative DNS server.
(if necessary - defaults to ansible_facts['dns']['nameservers'])

My assumption is that you use the felixfontein.acme.acme_certificate role.
But I think you will get the point.

var - filled with your authoritative DNS servers

acme_certificate_lookup_dns:
  - '9.9.9.9'
  - '149.112.112.112'

dns-check.yml - you may want to use another delay/retry configuration

---

- name: "Wait for DNS entries to propagate to {{ dns_resolver }}"
  ansible.builtin.command:
    argv:
      - dig
      - -t
      - TXT
      - +short
      - "@{{ dns_resolver }}"
      - "{{ item.0.key }}"
  register: _acme_challenge_lookup
  until: item.1 in _acme_challenge_lookup.stdout
  retries: 3
  delay: 120
  delegate_to: localhost
  run_once: true
  changed_when: false
  loop: "{{ acme_certificate_INTERNAL_challenge.challenge_data_dns | dict2items | subelements('value') }}"

And now loop that file for each DNS server:

dns-PROVIDER-create.yml

- name: check DNS propagation
  ansible.builtin.include: dns-check.yml
  vars:
    dns_resolver: "{{ _acme_certificate_lookup_dns }}"
  loop_control:
    loop_var: _acme_certificate_lookup_dns
  loop: "{{ acme_certificate_lookup_dns | d(ansible_facts['dns']['nameservers']) }}"

result:

TASK [acme_certificate : check DNS propagation] *******************************************************************************
included: tasks/dns-check.yml for foo.example.com => (item=9.9.9.9)
included: tasks/dns-check.yml for foo.example.com => (item=149.112.112.112)

TASK [acme_certificate : Wait for DNS entries to propagate to 9.9.9.9] *******************************************************************************
FAILED - RETRYING: Wait for DNS entries to propagate to 9.9.9.9 (3 retries left).
ok: [foo.example.com -> localhost] => (item=[{'key': '_acme-challenge.foo.example.com', 'value': ['XXX', 'YYY']}, 'XXX'])
ok: [foo.example.com -> localhost] => (item=[{'key': '_acme-challenge.foo.example.com', 'value': ['XXX', 'YYY']}, 'YYY'])

TASK [acme_certificate : Wait for DNS entries to propagate to 149.112.112.112] *******************************************************************************
ok: [foo.example.com -> localhost] => (item=[{'key': '_acme-challenge.foo.example.com', 'value': ['XXX', 'YYY']}, 'XXX'])
ok: [foo.example.com -> localhost] => (item=[{'key': '_acme-challenge.foo.example.com', 'value': ['XXX', 'YYY']}, 'YYY'])

@felixfontein
I highly appreciate that you use your spare time to advance this and your own collection.
Would it be sensible to relate to this issue on your felixfontein.acme collection - with the upcoming possibilities and further commits in mind?

[felixfontein]

What you describe is just the fact that the module itself includes the check_count within the returned json. What I mean is more of a debug output to the console.

Like this:

TASK [Wait until] *******************************************************************************************************************************************
FAILED - RETRYING: Wait until. (10 retries left).
FAILED - RETRYING: Wait until. (9 retries left).
FAILED - RETRYING: Wait until. (8 retries left).
FAILED - RETRYING: Wait until. (7 retries left).
FAILED - RETRYING: Wait until. (6 retries left).
FAILED - RETRYING: Wait until. (5 retries left).
FAILED - RETRYING: Wait until. (4 retries left).

That's probably not possible if I understand you correctly. A module can only output a result when it's done (resp. it can only pass something to the controller, and the callback plugin the user installed can decide whether to output it or not) and not yield any intermediate result. So you either see that output when the module is done (which depending on the propagation times can be a long time), or you don't see anything at all (because the callback decides not to output anything, for example the default callback with the default verbosity level).

[felixfontein]

There is a proposal for adding support for something like this to ansible-core (ansible/proposals#92). I haven't given up hope that we will eventually get that, but I don't expect it to happen anytime soon...

[Kariton]

Thank you for that detailed answer.

[felixfontein]

I adjusted the title since providing a list of servers is already possible (server option).