Testinfra runs failing in Qubes staging

[rocodes]

Description

Differences in the Qubes staging environment (such as internal IP addresses used for app and mon qubes) lead to testinfra failures.

Steps to Reproduce

Set up Qubes Staging environment according to docs, build debs, make staging, then make testinfra.

Expected Behavior

Testinfra tests can be run successfully

Actual Behavior

testinfra tests on debs built from latest develop:

    =========================== short test summary info ============================
    FAILED ../testinfra/app-code/test_securedrop_app_code.py::test_securedrop_application_test_journalist_key[ansible:/app-staging]
    FAILED ../testinfra/mon/test_mon_network.py::test_listening_ports[ansible:/mon-staging-ossec_service1]
    FAILED ../testinfra/mon/test_mon_network.py::test_mon_iptables_rules[ansible:/mon-staging]
    FAILED ../testinfra/mon/test_ossec_server.py::test_ossec_connectivity[ansible:/mon-staging]
    FAILED ../testinfra/mon/test_ossec_server.py::test_hosts_files[ansible:/mon-staging]
    FAILED ../testinfra/app/test_apparmor.py::test_apparmor_enforced[ansible:/app-staging-/usr/sbin/tcpdump]
    FAILED ../testinfra/app/test_app_network.py::test_app_iptables_rules[ansible:/app-staging]
    FAILED ../testinfra/app/test_ossec_agent.py::test_hosts_files[ansible:/app-staging]
    FAILED ../testinfra/common/test_user_config.py::test_sudoers_tmux_env_deprecated[ansible:/app-staging]
    FAILED ../testinfra/common/test_user_config.py::test_sudoers_tmux_env_deprecated[ansible:/mon-staging]
    FAILED ../testinfra/common/test_system_hardening.py::test_swap_disabled[ansible:/mon-staging]
    = 11 failed, 449 passed, 13 skipped, 3 xfailed, 7 xpassed, 8 warnings in 76.34s (0:01:16) =
make: *** [Makefile:201: testinfra] Error 1

Comments

I will look into this as part of my ansible and Qubes staging environment learning.

[conorsch]

@rocodes Looks like this ticket is a duplicate of #5377. To my eye, 5377 has a bit more info, but the total number of failing tests hasn't changed much. Up to you which one we keep to track the work!

[zenmonkeykstop]

This should be relatively straightforward to implement: the testinfra tests load environmental information from a vars file - so if you can detect that you're in Qubes before loading the vars file you can just load a Qubes-specific one with the network values as per the docs. Also tests can be marked to be skipped if they're not env-appropriate - see the prod testinfra changes for an example.

[rocodes]

@rocodes Looks like this ticket is a duplicate of #5377. To my eye, 5377 has a bit more info, but the total number of failing tests hasn't changed much. Up to you which one we keep to track the work!

Not only a dupe, but my dupe, oh dear. I'll close this one.