Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Configure/Customize ruamel.yaml Loading and Dumping #1927

Merged
merged 32 commits into from
Feb 26, 2022

Conversation

cognifloyd
Copy link
Contributor

@cognifloyd cognifloyd commented Feb 17, 2022

This configures and improves ruamel.yaml based formatting. One goal is to be compatible with other YAML formatters like prettier. It requires extending several ruamel.yaml components in ansiblelint.yaml_utils:

ansiblelint.yaml_utils. extends ruamel.yaml. For Loading For Dumping
FormattedYAML YAML
CustomConstructor constructor.RoundTripConstructor
FormattedEmitter emitter.Emitter
OctalIntYAML11 scalarint.ScalarInt

Each of these components serves the following purposes:

  • FormattedYAML:
    • For Loading and Dumping:
      • Default to YAML 1.1 instead of YAML 1.2 (PyYAML does not support 1.2 so Ansible does not support it either)
        • this fixes boolean and octal identification and dumping (and possibly other issues)
    • For Loading:
      • Wire up CustomConstructor
      • preserve more newlines when loading YAML files (ruamel.yaml was discarding blank lines if they only had spaces)
      • preserve preamble/header (comments before ---) which ruamel.yaml does not preserve
    • For Dumping:
      • Wire up FormattedEmitter
      • Adjusts/configures ruamel.yaml's defaults
        • enable explicit start (---) and disable explicit end (...)
        • Max dump width = 120
        • indent maps with 2 spaces; sequences with 4 (including dash)
      • Drop blank lines at end of file (it should have only one \n)
  • CustomConstructor:
    • For Loading:
      • Identify YAML 1.1 style octals to preserve them with OctalIntYAML11. Without this, ruamel.yaml converted octals to ints in YAML 1.1.
  • OctalIntYAML11:
    • For Loading:
      • When Octals get loaded, the CustomConstructor builds this special int subclass to preserve its format as an Octal
    • For Dumping:
      • Represents itself as a YAML 1.1 formatted octal 0123.
      • For YAML 1.2, ruamel.yaml loads octals as OctalInt, but that gets represented as a YAML 1.2 style octal 0o123.
  • FormattedEmitter:
    • For Dumping:
      • Ensures that root-level sequences (playbooks, task files, etc) are not over-indented without sacrificing indents at other levels.
      • Changes ruamel.yaml scalar quoting preference from single- to double-quotes.
      • Drop extra spaces (eg at end of line)
      • Condense repeated blank lines so that there is only ever one blank line
      • Add white space in flow-style maps ({ key: value } instead of {key: value})

This builds on #1926, #1934, and #1936 (and others)
Extracted from #1828.

@cognifloyd cognifloyd force-pushed the write-yaml branch 3 times, most recently from 1ed8a52 to 853711a Compare February 17, 2022 22:38
@cognifloyd cognifloyd force-pushed the write-yaml branch 2 times, most recently from fa7ae27 to fc22cc6 Compare February 18, 2022 16:00
@cognifloyd cognifloyd marked this pull request as ready for review February 18, 2022 16:00
@cognifloyd cognifloyd requested a review from a team as a code owner February 18, 2022 16:00
@cognifloyd cognifloyd requested review from relrod, ssbarnea and ganeshrn and removed request for a team February 18, 2022 16:00
@ssbarnea ssbarnea added the skip-changelog Can be missed from the changelog. label Feb 18, 2022
@cognifloyd cognifloyd force-pushed the write-yaml branch 6 times, most recently from 710fa26 to 71ecb16 Compare February 21, 2022 22:40
@ssbarnea ssbarnea added this to the 6.0.0 milestone Feb 25, 2022
The Emitter ensures that root-level sequences (playbooks, task files, etc)
are not over-indented without sacrificing indents at other levels.

It also changes ruamel.yaml scalar quoting preference from single- to double-quotes.
Make it clear which elements are under test.
It was dedenting a second level sequence when the top-level was a map.
They are very rough (not final)
YAML 1.1 was not important for this test. Just drop it.
@cognifloyd cognifloyd changed the title Configure ruamel.yaml dumping with custom Emitter Configure/Customize ruamel.yaml Loading and Dumping Feb 25, 2022
@cognifloyd
Copy link
Contributor Author

cognifloyd commented Feb 25, 2022

OK. I think this is done and ready for review.

The changes to the formatting-after fixture files highlight the differences with prettier's formatting. And, since lint is passing, prettier is not trying to change the format created by FormattedYAML (in other words, FormattedYAML output is compatible with prettier's output).

@ssbarnea ssbarnea merged commit 842e05f into ansible:main Feb 26, 2022
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
skip-changelog Can be missed from the changelog.
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants