Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[PR #2037/65293909 backport][stable-2] New Filter plugin from_csv #2074

Conversation

patchback[bot]
Copy link

@patchback patchback bot commented Mar 21, 2021

This is a backport of PR #2037 as merged into main (6529390).

SUMMARY

This filter adapts the csv parsing features from the read_csv module to use directly with ansible text/variables

Fixes: #1881

ISSUE TYPE
  • Feature Pull Request
COMPONENT NAME

plugins/filter/from_csv.py
plugins/modules/read_csv.py
plugins/module_utils/csv.py

ADDITIONAL INFORMATION

Feature set and core implementation is pulled directly from read_csv and adapted to the filter structure.
Common functions have been refactored into module_utils/csv.py which both read_csv and from_csv utilize

Notable Changes:

  • Return of read_csv is a tuple of list and dict with populated result switching to dict if key is provided. For the purposes of a filter I have chosen to make the result a list of anonymous dicts which can be keyed using jinja e.g. some_var | community.general.from_csv | groupby('key') | flatten is equivalent to keyed output of read_csv
  • Since keys have been removed there is no need for the unique parameter
  • Updated instances of ansible.module_utils._text.to_text to ansible.module_utils._text.to_native

Integration tests were added which cover basic functions, but could be updated for better coverage.

Unit tests also added for csv.py.

Wasn't sure how to properly credit the author of read_csv so I updated copyright as if it was an update to the original module

* Added from_csv filter and integration tests

* Cleaning up whitespace

* Adding changelog fragment

* Updated changelog fragment name

* Removed temp fragment

* Refactoring csv functions Part 1

* Syncing refactored csv modules/filters

* Adding unit tests for csv Module_Util

* Updating changelog fragment

* Correcting whitespace in unit test

* Improving changelog fragment

Co-authored-by: Felix Fontein <felix@fontein.de>

* Update changelogs/fragments/2037-add-from-csv-filter.yml

Co-authored-by: Felix Fontein <felix@fontein.de>
(cherry picked from commit 6529390)
@ansibullbot
Copy link
Collaborator

@ansibullbot ansibullbot added affects_2.10 community_review feature This issue/PR relates to a feature request files integration tests/integration module module module_utils module_utils needs_triage new_contributor Help guide this first time contributor plugins plugin (any type) tests tests unit tests/unit labels Mar 21, 2021
@felixfontein felixfontein merged commit 81cef0b into stable-2 Mar 21, 2021
@felixfontein felixfontein deleted the patchback/backports/stable-2/652939090147b1389ce8270f5fdb136b21f637e7/pr-2037 branch March 21, 2021 12:56
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
community_review feature This issue/PR relates to a feature request files integration tests/integration module_utils module_utils module module needs_triage new_contributor Help guide this first time contributor plugins plugin (any type) tests tests unit tests/unit
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants