Some callback plugins can be used together with community.general.diy while others cannot

[tan-wei-xin-alez]

Summary

As mentioned in the title, when setting stdout_callback=community.general.diy in ansible.cfg, some callbacks can be enabled while others can't (e.g. callbacks_enabled=profile_tasks works but not callbacks_enabled=debug)

Issue Type

Bug Report

Component Name

community.general.diy

Ansible Version

ansible [core 2.13.7]
  config file = ./ansible.cfg
  configured module search path = ['~/.ansible/plugins/modules', '/usr/share/ansible/plugins/modules']
  ansible python module location = /usr/lib/python3/dist-packages/ansible
  ansible collection location = ~/.ansible/collections:/usr/share/ansible/collections
  executable location = /usr/bin/ansible
  python version = 3.10.6 (main, Nov 14 2022, 16:10:14) [GCC 11.3.0]
  jinja version = 3.0.1
  libyaml = True

Community.general Version

# ~/.ansible/collections/ansible_collections
Collection        Version
----------------- -------
community.general 6.1.0

OS / Environment

PRETTY_NAME="Ubuntu Jammy Jellyfish (development branch)"
NAME="Ubuntu"
VERSION_ID="22.04"
VERSION="22.04 (Jammy Jellyfish)"
VERSION_CODENAME=jammy
ID=ubuntu
ID_LIKE=debian
HOME_URL="https://www.ubuntu.com/"
SUPPORT_URL="https://help.ubuntu.com/"
BUG_REPORT_URL="https://bugs.launchpad.net/ubuntu/"
PRIVACY_POLICY_URL="https://www.ubuntu.com/legal/terms-and-policies/privacy-policy"
UBUNTU_CODENAME=jammy

Steps to Reproduce

The following ansible.cfg

[defaults]
callbacks_enabled=profile_tasks
stdout_callback=community.general.diy

behaves as expected with the following playbook.yml

# playbook.yml
---
- hosts: testserver
  tasks:
  - vars:
      block_name: 'Install vim and nginx packages'
    block:
    - name: "{{ block_name }}"
      ansible.builtin.include_tasks: include/install_apt_pkg.yml
      loop:
        - {
            block_name: "{{ block_name }}",
            pkg: 'vim'
          }
        - {
            block_name: "{{ block_name }}",
            pkg: 'nginx'
          }

and install_apt_pkg.yml

# install_apt_pkg.yml
---
- name: "{{ item.block_name ~ ' - i' if item.block_name is defined else 'I' }}nstall {{ item.pkg }}"
  become: true # need to be root for sudo commands
  ansible.builtin.apt:
    pkg: "{{ item.pkg }}"
    state: latest
    update_cache: true
  async: 10
  poll: 0
  register: apt_task

- name: "{{ item.block_name ~ ' - i' if item.block_name is defined else 'I' }}nstallation progress for {{ item.pkg }}"
  become: true # need to be root for tasks run with sudo
  ansible.builtin.async_status:
    jid: "{{ apt_task.ansible_job_id }}"
  register: apt_job
  until: apt_job.finished
  retries: 10
  vars:
    ansible_callback_diy_runner_retry_msg: "Installing...({{ ansible_callback_diy.result.output.attempts }}/{{ ansible_callback_diy.result.output.retries - 1 }})"

giving the expected output (truncated)

...
TASK [Install vim and nginx packages] ****************************************************************************************************************
Saturday 24 December 2022  17:06:30 +0100 (0:00:01.212)       0:00:01.252 ***** 
included: ~/commisioning/include/install_apt_pkg.yml for testserver => (item={'block_name': 'Install vim and nginx packages', 'pkg': 'vim'})
included: ~/commisioning/include/install_apt_pkg.yml for testserver => (item={'block_name': 'Install vim and nginx packages', 'pkg': 'nginx'})

TASK [Install vim and nginx packages - install vim] *******************************************************************************************
Saturday 24 December 2022  17:06:30 +0100 (0:00:00.177)       0:00:01.430 ***** 
changed: [testserver]

TASK [Install vim and nginx packages - installation progress for terminator] *************************************************************************
Saturday 24 December 2022  17:06:31 +0100 (0:00:00.699)       0:00:02.130 ***** 
Installing...(1/10)
Installing...(2/10)
Installing...(3/10)
fatal: [testserver]: FAILED! => {"ansible_job_id": "434472033082.48044", "attempts": 4, "changed": false, "child_pid": 48049, "finished": 1, "msg": "Timeout exceeded", "results_file": "/root/.ansible_async/434472033082.48044", "started": 1, "stderr": "", "stderr_lines": [], "stdout": "", "stdout_lines": []}
...

On the other hand, the following ansible.cfg

[defaults]
callbacks_enabled=debug
stdout_callback=community.general.diy

does not behave as expected as seen in the following output (truncated)

...
TASK [Install vim and nginx packages] ****************************************************************************************************************
included: ~/commisioning/include/install_apt_pkg.yml for testserver => (item={'block_name': 'Install vim and nginx packages', 'pkg': 'vim'})
included: ~/commisioning/include/install_apt_pkg.yml for testserver => (item={'block_name': 'Install vim and nginx packages', 'pkg': 'nginx'})

TASK [Install vim and nginx packages - install vim] *******************************************************************************************
changed: [testserver]

TASK [Install vim and nginx packages - installation progress for terminator] *************************************************************************
Installing...(1/10)
Installing...(2/10)
Installing...(3/10)
fatal: [testserver]: FAILED! => {"ansible_job_id": "434472033082.48044", "attempts": 4, "changed": false, "child_pid": 48049, "finished": 1, "msg": "Timeout exceeded", "results_file": "/root/.ansible_async/434472033082.48044", "started": 1, "stderr": "", "stderr_lines": [], "stdout": "", "stdout_lines": []}
...

Expected Results

They should work together or some sort of error should be thrown indicating that the callbacks are incompatible, otherwise the only way to figure out is via trial and error

Code of Conduct

• I agree to follow the Ansible Code of Conduct

[ansibullbot]

Files identified in the description:

• plugins/callback/diy.py

If these files are incorrect, please update the component name section of the description or use the !component bot command.

click here for bot help

[ansibullbot]

cc @theque5t
click here for bot help

[felixfontein]

ansible.posix.debug is a stdout callback, same as community.general.diy, so only one of the two can be active (https://docs.ansible.com/ansible/latest/plugins/callback.html#setting-a-callback-plugin-for-ansible-playbook). ansible.posix.profile_tasks is an aggregate callback, so it can work together with stdout callbacks.

[tan-wei-xin-alez]

ansible.posix.debug is a stdout callback, same as community.general.diy, so only one of the two can be active (https://docs.ansible.com/ansible/latest/plugins/callback.html#setting-a-callback-plugin-for-ansible-playbook). ansible.posix.profile_tasks is an aggregate callback, so it can work together with stdout callbacks.

ah that explains it, where is this documented?
As in, is there a list of which callbacks are aggregate and which are not?

Is it also possible to implement ansible.posix.debug as an aggregate callback?
I figured out how to replace the default callback with ansible.posix.debug at https://github.com/ansible-collections/community.general/blob/main/plugins/callback/diy.py#L795 but that's obviously a hacky way of doing it

[felixfontein]

As in, is there a list of which callbacks are aggregate and which are not?

I don't think there is. At least for the two plugins (debug and profile_tasks), the ansible-doc CLI utility will show that information (ansible-doc -t callback debug and ansible-doc -t callback profile_tasks have a TYPE entry). The HTML docsite (https://docs.ansible.com/ansible/devel/collections/ansible/posix/debug_callback.html, https://docs.ansible.com/ansible/devel/collections/ansible/posix/profile_tasks_callback.html) does not show that yet.

Is it also possible to implement ansible.posix.debug as an aggregate callback?

I never used the debug callback and have no idea what it does, so I cannot really answer that.