Add modules to manipulate additional types of Name Server Groups #56

badnetmask · 2021-04-13T21:12:46Z

This PR adds support for additional types of Name Server Groups.

The original nsgroup module only handles Authoritative type. This PR also modifies the description of the original nsgroup module to reflect that.

Each module's documentation contain examples.

nios_nsgroup_delegation – Configure Infoblox DNS Nameserver Delegation Groups
nios_nsgroup_forwardingmember – Configure Infoblox DNS Nameserver Forwarding Member Groups
nios_nsgroup_forwardstubserver – Configure Infoblox DNS Nameserver Forward/Stub Server Groups
nios_nsgroup_stubmember – Configure Infoblox DNS Nameserver Stub Member Groups```

anagha-infoblox · 2021-10-11T19:17:57Z

Hi @badnetmask,
Thank you for your contribution to the Infoblox NIOS modules. We had to restructure the directory as per the requirements. As a result, there are some merge conflicts that have to be resolved. Can you please consider the same? Sorry for the inconvenience caused.

badnetmask · 2021-10-11T19:31:35Z

Will be glad to review my changes as soon as I can find some available cycles.

…p_commands

badnetmask · 2021-10-18T20:34:14Z

WIP -- I have fixed the merge conflicts, but I believe the new standards require integration tests, so I will work on these later.

plugins/modules/nios_nsgroup_delegation.py

plugins/modules/nios_nsgroup_forwardingmember.py

plugins/modules/nios_nsgroup_forwardstubserver.py

plugins/modules/nios_nsgroup_stubmember.py

anagha-infoblox · 2021-10-19T04:31:08Z

WIP -- I have fixed the merge conflicts, but I believe the new standards require integration tests, so I will work on these later.

Thank you! Yes, according to the new standards there's a requirement for unit and integration tests. I've suggested a few changes to fix the sanity error.

…al system, wanna check what happens on github)

badnetmask · 2021-10-31T15:57:34Z

@anagha-infoblox I need help figuring out how to create the tests. For whatever reason, when I run a playbook that uses all these new objects, against a real Infoblox device, all the objects are created just fine. However, running the collection in integration tests mode, they don't due to an error on the "infoblox-client" library. You can see the error on here on GitHub, on the checks tab.

So I'm not sure what's happening here, since the integration tests docker image uses the same version of the library that I have installed on my system (0.5.0). Any help here would be appreciated.

badnetmask · 2021-10-31T16:04:54Z

Here's a quick example to demonstrate how it's working on a real Infoblox device:

❯ cat test_nsgroup.yml 
---
# playbook used to test: https://github.com/badnetmask/infoblox-ansible/tree/dtc
- hosts: localhost
  connection: local
  gather_facts: no
  vars:
    - nios_provider:
        host: <ip_address>
        username: admin
        password: infoblox
        wapi_version: "2.6"
  tasks:
    - name: configure a nsgroup delegation on the system
      infoblox.nios_modules.nios_nsgroup_delegation:
        name: ansible-nsgroup_delegation
        state: present
        provider: "{{ nios_provider }}"
        delegate_to:
          - name: ns1
            address: 192.168.0.1
      register: nsgroup_delegation_create1

❯ ansible-playbook -i localhost, test_nsgroup.yml

PLAY [localhost]

TASK [configure a nsgroup delegation on the system] 
changed: [localhost]

PLAY RECAP 
localhost                  : ok=1    changed=1    unreachable=0    failed=0    skipped=0    rescued=0    ignored=0

anagha-infoblox · 2021-11-01T15:05:46Z

@anagha-infoblox I need help figuring out how to create the tests. For whatever reason, when I run a playbook that uses all these new objects, against a real Infoblox device, all the objects are created just fine. However, running the collection in integration tests mode, they don't due to an error on the "infoblox-client" library. You can see the error on here on GitHub, on the checks tab.

So I'm not sure what's happening here, since the integration tests docker image uses the same version of the library that I have installed on my system (0.5.0). Any help here would be appreciated.

@badnetmask I see that the tests are failing because of the NoneType error. Before the tests were failing due to the error stated above for stable-2.12 and devel branches. It was fixed by adding this file under integration tests- https://github.com/infobloxopen/infoblox-ansible/blob/master/tests/integration/targets/prepare_nios_tests/tasks/prepare_nios_tests_idempotence.yml
If that's an issue then, I would suggest adding

- pip:
    name: infoblox-client

under the tasks.

badnetmask · 2021-11-01T15:49:59Z

Well, I basically copied the entire tree under the nios_zone, and modified the test tasks for this object, so I would expect all dependencies would be resolved. Nevertheless, I did add the pip task to my test, and it still fails the same way. I hope I am just missing something obvious, but I can't see what it is.

anagha-infoblox · 2021-11-19T07:59:38Z

@badnetmask Sorry for the delayed reply. The integration tests are failing because these objects are not added to the nios-simulator. The tests are being run with the help of a simulator that mimics the functionality of a NIOS grid. We will look into the procedure of adding new objects to the simulator.
FYI,
Code for nios-simulator- https://github.com/ansible/nios-test-container
Link to the docker image of nios-simulator: https://quay.io/repository/ansible/nios-test-container?tag=1.3.0&tab=tags

- install package 'cargo' on the image (required to build 'cryptography') - update flaskapp.py to add support for additional nsgroup:delegation (required for infobloxopen/infoblox-ansible#56)

NOTE: requires update on 'nios-test-container' which has not been pushed yet

NOTE: automated github tests in this repo will fail until this is merged, and a new image is pushed to quay.io: ansible/nios-test-container#8

Container updates: - update image to alpine 3.15 - install cryptography using apk Updates to support infobloxopen/infoblox-ansible#56: - update flaskapp.py to add support for additional nsgroup:delegation - add more nsgroup objects

badnetmask · 2021-12-02T22:19:58Z

@anagha-infoblox - good and bad news.
good news: the simulator image has been updated and is working (as you can see here: https://github.com/infobloxopen/infoblox-ansible/runs/4401477649?check_suite_focus=true).
bad news: it only works for the devel branch, because that's where the image version has been updated.
@mattclay any suggestions to help get us unstuck from here?

mattclay · 2021-12-02T22:35:03Z

@badnetmask The container used can be overridden by setting the ANSIBLE_NIOSSIM_CONTAINER environment variable before running ansible-test.

badnetmask · 2021-12-02T23:23:03Z

@anagha-infoblox I suppose the container override needs to be configured on the repo level, which I do not have access. Please, let me know your thoughts.

anagha-infoblox · 2021-12-10T08:12:13Z

@anagha-infoblox I suppose the container override needs to be configured on the repo level, which I do not have access. Please, let me know your thoughts.

@badnetmask I have configured the ANSIBLE_NIOSSIM_CONTAINER environment variable in the PR#99

badnetmask · 2021-12-10T14:43:50Z

Seems like the integration tests are not picking up the modified variable.

anagha-infoblox · 2021-12-13T11:16:57Z

Seems like the integration tests are not picking up the modified variable.

Yes. I will try this again and, the modules will be included in the next major release.

anagha-infoblox · 2022-01-07T11:41:45Z

@badnetmask, I have configured the ANSIBLE_NIOSSIM_CONTAINER environment variable in the PR#103. The tests are picking up the modified variable.

badnetmask · 2022-01-14T15:31:39Z

@anagha-infoblox looks like all checks passed now \o/

anagha-infoblox · 2022-01-21T15:00:18Z

Thank you for your contribution to the Infoblox NIOS modules. According to the new standards, there is a requirement for unit and integration tests. The PR will be merged once apt unit tests for the modules are added.

badnetmask · 2022-02-05T23:27:11Z

I'm having an issue here while trying to run the unit test locally (trying to avoid spamming GitHub checks).

This command:

$ ansible-test units --docker -v --color --coverage --python 3.9

Fails with this error:

ImportError while importing test module '/root/ansible_collections/infoblox/nios_modules/tests/unit/plugins/modules/test_nios_zone.py'.
Hint: make sure your test modules/packages have valid Python names.
Traceback:
/usr/lib/python3.9/importlib/__init__.py:127: in import_module
    return _bootstrap._gcd_import(name[level:], package, level)
/tmp/ansible-test-5dmd8vi7/ansible/utils/collection_loader/_collection_finder.py:434: in exec_module
    exec(code_obj, module.__dict__)
tests/unit/plugins/modules/test_nios_zone.py:25: in <module>
    from ansible_collections.infoblox.nios_modules.tests.unit.compat.mock import patch, MagicMock, Mock
E   ModuleNotFoundError: No module named 'ansible_collections.infoblox.nios_modules.tests.unit.compat'
--------------------------------------------------------------------- generated xml file: /root/ansible_collections/infoblox/nios_modules/tests/output/junit/python3.9-modules-units.xml

What I don't understand is how these tests run just fine from GitHub if this module does not exist in the repo.
Please advise.

anagha-infoblox · 2022-05-02T12:41:03Z

I'm having an issue here while trying to run the unit test locally (trying to avoid spamming GitHub checks).

This command:

$ ansible-test units --docker -v --color --coverage --python 3.9

Fails with this error:

ImportError while importing test module '/root/ansible_collections/infoblox/nios_modules/tests/unit/plugins/modules/test_nios_zone.py'.
Hint: make sure your test modules/packages have valid Python names.
Traceback:
/usr/lib/python3.9/importlib/__init__.py:127: in import_module
    return _bootstrap._gcd_import(name[level:], package, level)
/tmp/ansible-test-5dmd8vi7/ansible/utils/collection_loader/_collection_finder.py:434: in exec_module
    exec(code_obj, module.__dict__)
tests/unit/plugins/modules/test_nios_zone.py:25: in <module>
    from ansible_collections.infoblox.nios_modules.tests.unit.compat.mock import patch, MagicMock, Mock
E   ModuleNotFoundError: No module named 'ansible_collections.infoblox.nios_modules.tests.unit.compat'
--------------------------------------------------------------------- generated xml file: /root/ansible_collections/infoblox/nios_modules/tests/output/junit/python3.9-modules-units.xml

What I don't understand is how these tests run just fine from GitHub if this module does not exist in the repo. Please advise.

@badnetmask, Sorry for the delayed reply. The CI workflow is failing because of the same issue. Will make the required changes soon.

anagha-infoblox · 2022-05-02T13:01:40Z

@badnetmask, I have raised a Pull Request with the changes required to pass the unit tests. Once the unit tests are added, the PR: Add modules to manipulate additional types of Name Server Groups will be merged.
Thank you

anagha-infoblox · 2022-05-02T14:15:21Z

The Pull Request is merged. With the latest code

I'm having an issue here while trying to run the unit test locally (trying to avoid spamming GitHub checks).

This command:

$ ansible-test units --docker -v --color --coverage --python 3.9

Fails with this error:

ImportError while importing test module '/root/ansible_collections/infoblox/nios_modules/tests/unit/plugins/modules/test_nios_zone.py'.
Hint: make sure your test modules/packages have valid Python names.
Traceback:
/usr/lib/python3.9/importlib/__init__.py:127: in import_module
    return _bootstrap._gcd_import(name[level:], package, level)
/tmp/ansible-test-5dmd8vi7/ansible/utils/collection_loader/_collection_finder.py:434: in exec_module
    exec(code_obj, module.__dict__)
tests/unit/plugins/modules/test_nios_zone.py:25: in <module>
    from ansible_collections.infoblox.nios_modules.tests.unit.compat.mock import patch, MagicMock, Mock
E   ModuleNotFoundError: No module named 'ansible_collections.infoblox.nios_modules.tests.unit.compat'
--------------------------------------------------------------------- generated xml file: /root/ansible_collections/infoblox/nios_modules/tests/output/junit/python3.9-modules-units.xml

What I don't understand is how these tests run just fine from GitHub if this module does not exist in the repo. Please advise.

The Pull Request is merged. With the latest code, the above issue is fixed. Sorry for the delayed response.

…p_commands

badnetmask · 2022-05-03T20:25:57Z

Seems like I still have issues running the unittests on my end, but I'll spend some time figuring it out before complaining. 😄

anagha-infoblox · 2022-06-01T05:47:33Z

@badnetmask, can you please push the unit tests that you have written? We can verify the same if any issues persist.

badnetmask · 2024-04-03T03:07:36Z

@JchhatbarInfoblox @JkhatriInfobox - this PR has been pending for a few years, and ended up forgotten because I didn't fix some pending issues. Now that they are fixed, would you mind giving it a review?

JkhatriInfobox · 2024-07-25T09:49:58Z

@badnetmask - We’ve initiated the review process for your Pull Request. It appears that only Integration tests have been included, with Unit tests missing. Could you please commit the Unit test cases to the PR? This will ensure it passes both the Unit and Integration tests in the CI run.

Additionally, the modifications made to the file [tests/unit/compat/mock.py] have already been merged into the master branch. To maintain a clean commit history, please rebase your branch with the master. Thank you.

JkhatriInfobox

The addition of new objects with respect to nsgroup looks good.
Unable to verify the Continuous Integration (CI) for the PR as it is quite old.
Unit test cases are not included in the PR and need to be addressed by the internal development team.
Integration tests need to be monitored for any failures after the merge.

badnetmask added 7 commits April 13, 2021 12:01

clarify that nios_nsgroup creates authoritative groups

b1b6c8b

add nios_nsgroup_forwardingmember

c320faf

add nios_nsgroup_forwardingmember

a22515d

fix sanity tests

7494adc

fix module description

f297f0f

add nios_nsgroup_delegation

eef3741

add nios_nsgroup_stubmember

f49f3ab

badnetmask added 3 commits October 18, 2021 16:16

Merge remote-tracking branch 'upstream/master' into additional_nsgrou…

09faaa1

…p_commands

fix bad merge

b279c74

remove deprecated files

b7704b4

anagha-infoblox reviewed Oct 19, 2021

View reviewed changes

badnetmask added 3 commits October 19, 2021 10:27

modify ib_spec to fix sanity errors

de9750e

add missing import to fix sanity tests

3846ccd

include integration test for nios_nsgroup_delegation (fails on my loc…

c210854

…al system, wanna check what happens on github)

badnetmask added 2 commits November 20, 2021 19:27

fix incorrect object type for 'extattrs'

c368894

correct integration tests for nios_nsgroup_delegation_idempotence.yml

e416995

NOTE: requires update on 'nios-test-container' which has not been pushed yet

badnetmask added a commit to badnetmask/nios-test-container that referenced this pull request Nov 21, 2021

add more nsgroup objects, required for infobloxopen/infoblox-ansible#56

12b25a1

badnetmask mentioned this pull request Nov 21, 2021

Create additional nsgroup objects ansible/nios-test-container#8

Merged

badnetmask added 2 commits November 20, 2021 20:47

fix typos on nios_nsgroup_delegation_idempotence integration test

b6942f3

add remaining integration tests for all nsgroup types

b777f3b

NOTE: automated github tests in this repo will fail until this is merged, and a new image is pushed to quay.io: ansible/nios-test-container#8

fix sanity tests

202eb39

retrigger checks

5273224

retrigger checks

de2c26a

Merge branch 'master' into additional_nsgroup_commands

41069d5

Merge remote-tracking branch 'upstream/master' into additional_nsgrou…

ec1cf4f

…p_commands

badnetmask added 3 commits April 1, 2024 22:20

Merge branch 'master' into additional_nsgroup_commands

72f729e

fix integration and sanity tests

708653d

fix sanity tests for ansible devel branch

406edc0

JkhatriInfobox self-requested a review August 13, 2024 09:42

JkhatriInfobox reviewed Aug 22, 2024

View reviewed changes

JkhatriInfobox approved these changes Aug 22, 2024

View reviewed changes

JkhatriInfobox merged commit 10cb08a into infobloxopen:master Aug 22, 2024
25 checks passed

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Add modules to manipulate additional types of Name Server Groups #56

Add modules to manipulate additional types of Name Server Groups #56

badnetmask commented Apr 13, 2021

anagha-infoblox commented Oct 11, 2021

badnetmask commented Oct 11, 2021

badnetmask commented Oct 18, 2021

anagha-infoblox commented Oct 19, 2021

badnetmask commented Oct 31, 2021

badnetmask commented Oct 31, 2021

anagha-infoblox commented Nov 1, 2021 •

edited

Loading

badnetmask commented Nov 1, 2021

anagha-infoblox commented Nov 19, 2021

badnetmask commented Dec 2, 2021

mattclay commented Dec 2, 2021

badnetmask commented Dec 2, 2021

anagha-infoblox commented Dec 10, 2021

badnetmask commented Dec 10, 2021

anagha-infoblox commented Dec 13, 2021

anagha-infoblox commented Jan 7, 2022

badnetmask commented Jan 14, 2022

anagha-infoblox commented Jan 21, 2022

badnetmask commented Feb 5, 2022

anagha-infoblox commented May 2, 2022

anagha-infoblox commented May 2, 2022

anagha-infoblox commented May 2, 2022

badnetmask commented May 3, 2022

anagha-infoblox commented Jun 1, 2022

badnetmask commented Apr 3, 2024

JkhatriInfobox commented Jul 25, 2024

JkhatriInfobox left a comment

Add modules to manipulate additional types of Name Server Groups #56

Add modules to manipulate additional types of Name Server Groups #56

Conversation

badnetmask commented Apr 13, 2021

anagha-infoblox commented Oct 11, 2021

badnetmask commented Oct 11, 2021

badnetmask commented Oct 18, 2021

anagha-infoblox commented Oct 19, 2021

badnetmask commented Oct 31, 2021

badnetmask commented Oct 31, 2021

anagha-infoblox commented Nov 1, 2021 • edited Loading

badnetmask commented Nov 1, 2021

anagha-infoblox commented Nov 19, 2021

badnetmask commented Dec 2, 2021

mattclay commented Dec 2, 2021

badnetmask commented Dec 2, 2021

anagha-infoblox commented Dec 10, 2021

badnetmask commented Dec 10, 2021

anagha-infoblox commented Dec 13, 2021

anagha-infoblox commented Jan 7, 2022

badnetmask commented Jan 14, 2022

anagha-infoblox commented Jan 21, 2022

badnetmask commented Feb 5, 2022

anagha-infoblox commented May 2, 2022

anagha-infoblox commented May 2, 2022

anagha-infoblox commented May 2, 2022

badnetmask commented May 3, 2022

anagha-infoblox commented Jun 1, 2022

badnetmask commented Apr 3, 2024

JkhatriInfobox commented Jul 25, 2024

JkhatriInfobox left a comment

Choose a reason for hiding this comment

anagha-infoblox commented Nov 1, 2021 •

edited

Loading