yamllint compatibility should be a fatal error

[kousu]

Summary

If .yamllint disagrees with ansible-lint, #4139 is supposed to make it quit early with return code 3:

ansible-lint/src/ansiblelint/yaml_utils.py

Lines 141 to 143 in 921aa83

	msg = f"Found incompatible custom yamllint configuration ({file}), please either remove the file or edit it to comply with:{nl} - {nl + ' - '.join(errors)}.{nl}{nl}Read https://ansible.readthedocs.io/projects/lint/rules/yaml/ for more details regarding why we have these requirements."
	logging.fatal(msg)
	sys.exit(RC.INVALID_CONFIG)

ansible-lint/src/ansiblelint/constants.py

Line 33 in 921aa83

INVALID_CONFIG = 3

but it does not.

Issue Type

• Bug Report

OS / ENVIRONMENT

$ ansible-lint --version
ansible-lint 24.6.0 using ansible-core:2.17.0 ansible-compat:24.6.0 ruamel-yaml:0.18.6 ruamel-yaml-clib:0.2.8

• OS: macOS 14.4.1
• ansible installation method: brew
• ansible-lint installation method: brew

STEPS TO REPRODUCE

# .yamllint.yml
---
extends: default
rules:
  braces:
    min-spaces-inside: 1
    max-spaces-inside: 1
  quoted-strings:
    quote-type: double
    required: only-when-needed

Without linting errors:

# site.yaml
---
- name: Base setup for all nodes
  hosts: all
  gather_facts: false
  tasks:
    - name: Install
      ansible.builtin.package:
        name:
          - firefox
          - 0ad

Or with:

# site.yaml
---
- name: Base setup for all nodes
  hosts: all
  gather_facts: false
  tasks:
    - name: Install
      ansible.builtin.package:
        name:
          - "firefox"
          - 0ad

Desired Behavior

With or without linting errors should both short-circuit to:

$ ansible-lint; printf "\nRC=%d\n" $?
ERROR  Found incompatible custom yamllint configuration (.yamllint.yml), please either remove the file or edit it to comply with:
  - comments.min-spaces-from-content must be 1
  - comments-indentation must be false
  - braces.min-spaces-inside must be 0
  - octal-values.forbid-implicit-octal must be true
  - octal-values.forbid-explicit-octal must be true.

Read https://ansible.readthedocs.io/projects/lint/rules/yaml/ for more details regarding why we have these requirements. Fix mode will not be available.

Failed: <some message about the short-circuit>. Last profile that met the validation criteria was 'production'.

RC=3

Actual Behavior

The linter still runs, instead of being skipped as promised.

Without linting errors:

$ ansible-lint; printf "\nRC=%d\n" $?
WARNING  Found incompatible custom yamllint configuration (.yamllint.yml), please either remove the file or edit it to comply with:
  - comments.min-spaces-from-content must be 1
  - comments-indentation must be false
  - braces.min-spaces-inside must be 0
  - octal-values.forbid-implicit-octal must be true
  - octal-values.forbid-explicit-octal must be true.

Read https://ansible.readthedocs.io/projects/lint/rules/yaml/ for more details regarding why we have these requirements. Fix mode will not be available.

Passed: 0 failure(s), 0 warning(s) on 3 files. Last profile that met the validation criteria was 'production'.

RC=0

With linting errors:

% ansible-lint; printf "\nRC=%d\n" $?  
WARNING  Found incompatible custom yamllint configuration (.yamllint.yml), please either remove the file or edit it to comply with:
  - comments.min-spaces-from-content must be 1
  - comments-indentation must be false
  - braces.min-spaces-inside must be 0
  - octal-values.forbid-implicit-octal must be true
  - octal-values.forbid-explicit-octal must be true.

Read https://ansible.readthedocs.io/projects/lint/rules/yaml/ for more details regarding why we have these requirements. Fix mode will not be available.
WARNING  Listing 1 violation(s) that are fatal
yaml[quoted-strings]: String value is redundantly quoted with double quotes
site.yml:9

Read documentation for instructions on how to ignore specific rule violations.

                 Rule Violation Summary                  
 count tag                  profile rule associated tags 
     1 yaml[quoted-strings] basic   formatting, yaml     

Failed: 1 failure(s), 0 warning(s) on 3 files. Last profile that met the validation criteria was 'min'.

RC=2

[kousu]

Oh I see what happened, the error was demoted to a warning in #415. But #4152 only requested it apply if warn_list: [yaml] or skip_list: [yaml] are set; otherwise it should still be an error.

I didn't set either of those in my test case above. But instead, setting

# .ansible-lint
write_list:
  - all

Gives

$ ansible-lint 
ERROR    Found incompatible custom yamllint configuration (.yamllint.yml), please either remove the file or edit it to comply with:
  - comments.min-spaces-from-content must be 1
  - comments-indentation must be false
  - braces.min-spaces-inside must be 0
  - octal-values.forbid-implicit-octal must be true
  - octal-values.forbid-explicit-octal must be true.

Read https://ansible.readthedocs.io/projects/lint/rules/yaml/ for more details regarding why we have these requirements. Fix mode will not be available.
$ echo $?
3

I don't see the connection between write_list and yamllint, they seem totally apart, does anyone have an explanation?

[ssbarnea]

Yes, these settings are required only if you use the --fix option, because we can only format using these values.

That is why for other cases we only display a warning. The user might decide to either fix ansible lint config.