Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add sanity rule with check for bad and disallowed ignores #3102

Merged
merged 1 commit into from
Mar 2, 2023

Conversation

alisonlhart
Copy link
Contributor

@alisonlhart alisonlhart commented Feb 28, 2023

Note: This is an extremely opinionated rule that Partner Engineering enforces for certified content.

New rule sanity checks the tests/sanity/ignore-x.x.txt files for disallowed ignores. Commented-out entries are not evaluated.
Currently (broadly) allowed ignores are:

  • validate-modules:missing-gplv3-license
  • import-2.6
  • import-2.6!skip
  • import-2.7
  • import-2.7!skip
  • import-3.5
  • import-3.5!skip
  • compile-2.6
  • compile-2.6!skip
  • compile-2.7
  • compile-2.7!skip
  • compile-3.5
  • compile-3.5!skip

Error breakdown:

  • sanity[cannot-ignore] - Ignore file contains a test that is not on the allow list.
  • sanity[bad-ignore] - Ignore file entry is formatted incorrectly.

Other changes:

  • Added new file type: sanity-ignore-file to recognize files with glob pattern tests/sanity/ignore*.txt
  • Updated test count to account for 3 tests in sanity.py
  • Added rule under production profile

Fixes: #2121

@alisonlhart alisonlhart changed the title New rule: sanity-ignore-file (shared or production profile) New rule: sanity-ignore-file (production profile) Feb 28, 2023
@alisonlhart
Copy link
Contributor Author

alisonlhart commented Feb 28, 2023

Note: The validate-modules:missing-gplv3-license entry is throwing a cspell error for gplv.

@alisonlhart alisonlhart marked this pull request as ready for review March 1, 2023 15:37
@alisonlhart alisonlhart requested a review from a team as a code owner March 1, 2023 15:37
Copy link
Member

@thedoubl3j thedoubl3j left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

looks good and what we want to enforce from a certification stand point.

@ssbarnea ssbarnea changed the title New rule: sanity-ignore-file (production profile) Add sanity rule with check for bad and not-allowed ignores Mar 2, 2023
@ssbarnea ssbarnea changed the title Add sanity rule with check for bad and not-allowed ignores Add sanity rule with check for bad and disallowed ignores Mar 2, 2023
@ssbarnea
Copy link
Member

ssbarnea commented Mar 2, 2023

Eco pipeline is correctly failing as cisco.nxos collection will need some updates.

@ssbarnea ssbarnea merged commit b9d96df into ansible:main Mar 2, 2023
@alisonlhart
Copy link
Contributor Author

I'm going to open another PR today adding allow-lists for specific versions of ansible. E.g. validate-modules:invalid-documentation and validate-modules:deprecation-mismatch. Networking brought these rules to our attention after merge.

@alisonlhart alisonlhart deleted the check_ignore_files branch March 2, 2023 17:21
t-woerner added a commit to t-woerner/ansible-freeipa that referenced this pull request May 4, 2023
The old ignore file ignore-2.12.txt is not needed and used anymore. The
new files ignore-2.13.txt and ignore-2.14.txt are empty after
ansible-lint made nearly all ignores disallowed.

All the newly disallowed ignores need to be fixed.

See ansible/ansible-lint#3102
t-woerner added a commit to t-woerner/ansible-freeipa that referenced this pull request May 5, 2023
The old ignore file ignore-2.12.txt is not needed and used anymore. The
new files ignore-2.13.txt and ignore-2.14.txt are empty after
ansible-lint made nearly all ignores disallowed.

All the newly disallowed ignores need to be fixed.

See ansible/ansible-lint#3102
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Projects
None yet
Development

Successfully merging this pull request may close these issues.

disallowed-ignore[sanity]: disallow specific skips in ignore files
3 participants