ansible-lint on collection tries to install the collection itself

[falon]

Summary

If I develop a collection and I lint it, the linter try to install the same collection by Galaxy. After that it warn that multiple collection versions were found.

I don't understand. If I'm developing a collection, I don't expect that the linter modify this collection by installing it from Galaxy.

Issue Type

• Bug Report

OS / ENVIRONMENT

# ansible-lint --version
ansible-lint 24.6.0 using ansible-core:2.17.0 ansible-compat:24.6.1 ruamel-yaml:0.18.6 ruamel-yaml-clib:0.2.8

• ansible installation method: pip
• ansible-lint installation method: pip

STEPS TO REPRODUCE

Init a new fresh collection with ansible-galaxy.
When I run ansible-lint from the base path of the collection I see

INFO     Running from /root/.ansible/collections/ansible_collections/mynamespace/mycollection : ansible-galaxy collection install -vvv --force /root/.ansible/collections/ansible_collections/mynamespace/mycollection
WARNING  Multiple versions of 'mynamespace.mycollection' were found installed, only the first one will be used, 1.0.0 (/root/.ansible/collections/ansible_collections).

Why does ansible-lint want to install the collection which I'm linting?

My galaxy.cfg is

namespace: mynamespace
name: mycollection
version: 1.0.0
readme: README.md
authors:
  - Marco  <marco@example.com>
description: A general suite of customization.
license:
  - GPL-3.0-or-later
license_file: ""
tags:
  - linux
dependencies:
  community.general: ">=8.1.0"
  community.docker: ">=3.4.11"
repository: https://gitlab.example.com/mynamespace/mycollection
homepage: https://gitlab.example.com/mynamespace/mycollection
issues: https://gitlab.example.com/mynamespace/mycollection/-/issues

Desired Behavior

It's good that ansible-lint installs collection dependent by the collection. Ansible-lint shoud not try to install from ansible-galaxy the same collection that I'm linting.

Actual Behavior

[root@test mycollection]# ansible-lint
WARNING  Multiple versions of 'mynamespace.mycollection' were found installed, only the first one will be used, 1.0.0 (/root/.ansible/collections/ansible_collections).
WARNING  Multiple versions of 'mynamespace.mycollection' were found installed, only the first one will be used, 1.0.0 (/root/.ansible/collections/ansible_collections).

Passed: 0 failure(s), 0 warning(s) on 149 files. Last profile that met the validation criteria was 'production'.

[root@test mycollection]# ansible-galaxy collection list | grep mynamespace\.mycollection
mynamespace.mycollection         1.0.0

Finally, as I see above, it's not true that Multiple versions of 'mynamespace.mycollection' were found installed.

If I run the lint with --offline this warning doesn't appear.

[ssbarnea]

This should never happen and the only case where I would see it as happening is if you have your own collection mentioned as a dependency somewhere inside your codebase (or being listed as an indirect dependency of another collection).

Please run with -vvv and look at the operations that are happening to figure it out where it does get this collection as a dependency.

[falon]

Please run with -vvv and look at the operations that are happening to figure it out where it does get this collection as a dependency.

Hello @ssbarnea, I run ansible-lint in the base path of my collection: /root/.ansible/collections/ansible_collections/mynamespace/mycollection

The debug shows as when the dependencies are parsed, also the collection itself is tried to install.

^[[2mDEBUG    Logging initialized to level 10^[[0m
^[[2mINFO     Identified ^[[0m^[[2;34m/root/.ansible/collections/ansible_collections/mynamespace/mycollection^[[0m^[[2m as project root due ^[[0m^[[1;2m.git directory^[[0m^[[2m.^[[0m
^[[2mDEBUG    Options: Options(_skip_ansible_syntax_check=False, cache_dir=PosixPath('/root/.cache/ansible-compat/92a5dc'), colored=True, configured=True, cwd=PosixPath('/root/.ansible/collections/ansible_collections/mynamespace/mycollection'), display_relative_path=True, exclude_paths=['.cache', '.git', '.hg', '.svn', '.tox'], format=None, lintables=[], list_rules=False, list_tags=False, write_list=[], parseable=False, quiet=0, rulesdirs=[PosixPath('/usr/local/lib/python3.12/site-packages/ansiblelint/rules')], skip_list=[], tags=[], verbosity=3, warn_list=['experimental', 'jinja^[[0m^[[2m', 'fqcn^[[0m^[[2m'], mock_filters=[], mock_modules=[], mock_roles=[], loop_var_prefix=None, only_builtins_allow_collections=[], only_builtins_allow_modules=[], var_naming_pattern=None, offline=None, project_dir='.', extra_vars=None, enable_list=[], skip_action_validation=True, strict=False, rules={}, profile=None, task_name_prefix='{stem} | ', sarif_file=None, config_file=None, generate_ignore=False, rulesdir=[], use_default_rules=False, version=False, list_profiles=False, ignore_file=None, max_tasks=100, max_block_depth=20, supported_ansible_also=[])^[[0m
^[[2mDEBUG    CWD: /root/.ansible/collections/ansible_collections/mynamespace/mycollection^[[0m
^[[2mDEBUG    Logging initialized to level 10^[[0m
^[[2mDEBUG    Loading custom /root/.config/yamllint/config config file, this extends our internal yamllint config.^[[0m
^[[2mDEBUG    Effective yamllint rules used: {'anchors': {'level': 'error', 'forbid-undeclared-aliases': True, 'forbid-duplicated-anchors': False, 'forbid-unused-anchors': False}, 'braces': {'level': 'error', 'forbid': False, 'min-spaces-inside': 0, 'max-spaces-inside': 1, 'min-spaces-inside-empty': -1, 'max-spaces-inside-empty': -1}, 'brackets': {'level': 'error', 'forbid': False, 'min-spaces-inside': 0, 'max-spaces-inside': 0, 'min-spaces-inside-empty': -1, 'max-spaces-inside-empty': -1}, 'colons': {'level': 'error', 'max-spaces-before': 0, 'max-spaces-after': 1}, 'commas': {'level': 'error', 'max-spaces-before': 0, 'min-spaces-after': 1, 'max-spaces-after': 1}, 'comments': {'level': 'warning', 'require-starting-space': True, 'ignore-shebangs': True, 'min-spaces-from-content': 1}, 'comments-indentation': False, 'document-end': False, 'document-start': False, 'empty-lines': {'level': 'error', 'max': 2, 'max-start': 0, 'max-end': 0}, 'empty-values': False, 'float-values': False, 'hyphens': {'level': 'error', 'max-spaces-after': 1}, 'indentation': {'level': 'error', 'spaces': 'consistent', 'indent-sequences': True, 'check-multi-line-strings': False}, 'key-duplicates': {'level': 'error', 'forbid-duplicated-merge-keys': False}, 'key-ordering': False, 'line-length': {'level': 'error', 'max': 160, 'allow-non-breakable-words': True, 'allow-non-breakable-inline-mappings': False}, 'new-line-at-end-of-file': {'level': 'error'}, 'new-lines': {'level': 'error', 'type': 'unix'}, 'octal-values': {'forbid-implicit-octal': True, 'forbid-explicit-octal': True, 'level': 'error'}, 'quoted-strings': False, 'trailing-spaces': {'level': 'error'}, 'truthy': {'level': 'warning', 'allowed-values': ['true', 'false'], 'check-keys': True}}^[[0m
^[[2mINFO     Set ANSIBLE_LIBRARY=/root/.cache/ansible-compat/92a5dc/modules:/root/.ansible/plugins/modules:/usr/share/ansible/plugins/modules^[[0m
^[[2mINFO     Set ANSIBLE_COLLECTIONS_PATH=/root/.cache/ansible-compat/92a5dc/collections:/root/.ansible/collections:/usr/share/ansible/collections:/usr/local/lib/python3.12/site-packages:/root/.cache/ansible-compat/92a5dc/collections:/root/.ansible/collections:/usr/share/ansible/collections:/usr/local/lib/python3.12/site-packages^[[0m
^[[2mINFO     Set ANSIBLE_ROLES_PATH=/root/.cache/ansible-compat/92a5dc/roles:roles:/root/.ansible/roles^[[0m
^[[2mINFO     Provisioning collection community.general:>=8.1.0 from galaxy.yml^[[0m
^[[2mINFO     Running from /root/.ansible/collections/ansible_collections/mynamespace/mycollection : ansible-galaxy collection install -vvv community.general:>=8.1.0^[[0m
^[[2mINFO     Provisioning collection community.docker:>=3.4.11 from galaxy.yml^[[0m
^[[2mINFO     Running from /root/.ansible/collections/ansible_collections/mynamespace/mycollection : ansible-galaxy collection install -vvv community.docker:>=3.4.11^[[0m
^[[2mINFO     Running from /root/.ansible/collections/ansible_collections/mynamespace/mycollection : ansible-galaxy collection install -vvv --force /root/.ansible/collections/ansible_collections/mynamespace/mycollection^[[0m
^[[2mWARNING  Multiple versions of 'mynamespace.mycollection' were found installed, only the first one will be used, 1.0.0 (/root/.ansible/collections/ansible_collections).^[[0m
^[[2mDEBUG    Loading custom /root/.config/yamllint/config config file, this extends our internal yamllint config.^[[0m

I could attach the entire output if it could be useful. The check about dependecies run more times, I don't understand why.

[tanwigeetika1618]

Hi @falon , Since you originally reported this problem, there have been several bug fixes and releases. The current version of ansible-lint is now 24.7.0, and we have addressed multiple issues since version 24.6.0. I was unable to reproduce the problem you mentioned using the steps provided and the latest version of ansible-lint. Could you please check on your end with the most recent version and let me know if the issue still persists? If you are still experiencing the problem, it would be helpful if you could provide updated steps to reproduce it.
If everything works fine with the latest release, I will proceed to close this issue based on the current findings.

This is what I got upon reproducing the steps.

:~/ansible-lint/AllTests/mynamespace/mycollection$ ansible-lint

Passed: 0 failure(s), 0 warning(s) on 6 files. Last profile that met the validation criteria was 'production'.

[falon]

Hello @tanwigeetika1618 , I'm sorry for the delay, I was away for a while.

I still see the same problem even with the last version of ansible-lint:

# ansible-lint --version
ansible-lint 24.7.0 using ansible-core:2.16.9 ansible-compat:24.7.0 ruamel-yaml:0.18.6 ruamel-yaml-clib:0.2.8

My big question is that I can't understand why you can't reproduce the issue.

[tigattack]

I have the same problem:

❯ ansible-lint --version
ansible-lint 24.7.0 using ansible-core:2.17.1 ansible-compat:24.9.0 ruamel-yaml:0.18.6 ruamel-yaml-clib:0.2.8

❯ ansible-lint
WARNING  Multiple versions of 'community.general' were found installed, only the first one will be used, 9.2.0 (/Users/tigattack/.cache/ansible-compat/0cb87f/collections/ansible_collections).
WARNING  Multiple versions of 'community.vmware' were found installed, only the first one will be used, 4.5.0 (/Users/tigattack/.cache/ansible-compat/0cb87f/collections/ansible_collections).
WARNING  Multiple versions of 'community.general' were found installed, only the first one will be used, 9.2.0 (/Users/tigattack/.cache/ansible-compat/0cb87f/collections/ansible_collections).
WARNING  Multiple versions of 'community.vmware' were found installed, only the first one will be used, 4.5.0 (/Users/tigattack/.cache/ansible-compat/0cb87f/collections/ansible_collections).

Passed: 0 failure(s), 0 warning(s) on 583 files. Profile 'production' was required, and it passed.

❯ grep -A1 -E 'general|vmware' requirements.yml
  - name: community.general
    version: 9.2.0
--
  - name: community.vmware
    version: 4.5.0

Some interesting behaviour I noticed in verbose output - It appears to be doubling up the paths in the ANSIBLE_COLLECTIONS_PATH for some reason. Not the issue at hand, but felt noteworthy.

❯ ansible-lint -v
INFO     Identified /Users/tigattack/Documents/dev/infra/ansible as project root due config file /Users/tigattack/Documents/dev/infra/ansible/.ansible-lint.
INFO     Running ansible-galaxy role install -r /Users/tigattack/Documents/dev/infra/ansible/requirements.yml -v --roles-path /Users/tigattack/.cache/ansible-compat/0cb87f/roles
INFO     Running ansible-galaxy collection install -v -r /Users/tigattack/Documents/dev/infra/ansible/requirements.yml
[...]
INFO     Set ANSIBLE_COLLECTIONS_PATH=/Users/tigattack/.cache/ansible-compat/0cb87f/collections:/Users/tigattack/Documents/dev/infra/ansible/.dependencies:/Users/tigattack/.cache/ansible-compat/0cb87f/collections:/Users/tigattack/Documents/dev/infra/ansible/.dependencies

[oschroeder8]

Have been seeing this behavior as well with ansible-lint 24.9.0:

$ ansible-lint packer/ansible/
WARNING  Multiple versions of 'ansible.windows' were found installed, only the first one will be used, 2.5.0 (/root/.ansible/collections/ansible_collections).
WARNING  Multiple versions of 'community.general' were found installed, only the first one will be used, 9.4.0 (/root/.ansible/collections/ansible_collections).
WARNING  Multiple versions of 'community.windows' were found installed, only the first one will be used, 2.3.0 (/root/.ansible/collections/ansible_collections).
WARNING  Multiple versions of 'openstack.cloud' were found installed, only the first one will be used, 2.2.0 (/root/.ansible/collections/ansible_collections).
WARNING  Multiple versions of 'ansible.windows' were found installed, only the first one will be used, 2.5.0 (/root/.ansible/collections/ansible_collections).
WARNING  Multiple versions of 'community.general' were found installed, only the first one will be used, 9.4.0 (/root/.ansible/collections/ansible_collections).
WARNING  Multiple versions of 'community.windows' were found installed, only the first one will be used, 2.3.0 (/root/.ansible/collections/ansible_collections).
WARNING  Multiple versions of 'openstack.cloud' were found installed, only the first one will be used, 2.2.0 (/root/.ansible/collections/ansible_collections).
Passed: 0 failure(s), 0 warning(s) on 48 files. Last profile that met the validation criteria was 'production'

I have been seeing this for a while now. It is more of a cosmetic issue.

[ssbarnea]

Reopening as during debugging we found several issues we want to address.

[ssbarnea]

@falon Linter will try to install current collection unless run in offline mode, but it will do it from disk, not from galaxy. Exception would if there is this collection mentioned in a requirement file from within the repo.

We will downgrade that warning to info level as we might expect that a considerable number of people might already have it installed locally at user or system level. Still due to path manipulation done by the linter, its custom install path would take precedence and the already existing versions will not be used.

So once we do this you will no longer see these messages.

[ssbarnea]

Closing as with current release this no longer reproduces.

[falon]

Closing as with current release this no longer reproduces.

I agree, thank you @ssbarnea! Really now there is another warning:
WARNING Another version of 'mynamespace.mycollection' 0.0.1 was found installed in /root/.cache/ansible-compat/63641c/collections/ansible_collections, only the first one will be used, 0.0.1 (/root/.ansible/collections/ansible_collections).

# ansible-lint --version
ansible-lint 24.9.2 using ansible-core:2.16.11 ansible-compat:24.9.1 ruamel-yaml:0.18.6 ruamel-yaml-clib:0.2.8

Maybe do I have to do something with ansible-compat config?

Even if I remove /root/.cache/ansible-compat/* when I run ansible-lint a cache path is added and ansible-lint claims again.

I wonder if this could be related to

;collections_scan_sys_path=True in ansible.cfg

which it seems to be enabled by default. Maybe /root/.cache should not be a system path for a collection scan purpose.

Thank you

[DeadNews]

Same.