Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

QACTL: Fix HTML report generation and update documentation #1728

Closed
4 tasks done
roronoasins opened this issue Aug 18, 2021 · 3 comments
Closed
4 tasks done

QACTL: Fix HTML report generation and update documentation #1728

roronoasins opened this issue Aug 18, 2021 · 3 comments
Assignees

Comments

@roronoasins
Copy link

roronoasins commented Aug 18, 2021

Description

When test are run, the HTML report file creation makes that program exits with File not found error

Also, the QACTL Test Execution documentation is outdated, we need to check and update it.

Tasks

  • HTML report is not generated correctly.
  • HTML report does not contain test info
  • Check and update documentation.

Additional task

  • Use a new directory inside tests execution dir, in order to not fill the directory with assets, plain and html reports
@roronoasins roronoasins self-assigned this Aug 18, 2021
@roronoasins roronoasins mentioned this issue Aug 18, 2021
43 tasks
@jmv74211 jmv74211 changed the title QACTL: Test stage need to be fixed QACTL: Fix HTML report generation and update documentation Aug 18, 2021
@roronoasins
Copy link
Author

roronoasins commented Aug 18, 2021

I have created the necessary environment in order to execute the tests and check the HTML report generation error using the following conf.yaml:

deployment:
        host1:
                provider:
                        vagrant:
                                enabled: True
                                vagrantfile_path: '/tmp'
                                vagrant_box: 'qactl/ubuntu_20_04'
                                vm_memory: 1024
                                vm_cpu: 2
                                vm_name: 'ubu20'
                                vm_system: 'Linux'
                                label: 'qactl'
                                vm_ip: '172.16.1.10'
                                quiet_out: False

provision:
        hosts:
                host1:
                        host_info:
                                connection_method: 'ssh'
                                host: '172.16.1.10'
                                user: 'vagrant'
                                password: 'vagrant'
                                connection_port: '22'
                                ansible_python_interpreter: '/usr/bin/python3'

                        wazuh_deployment:
                                wazuh_installation:
                                        type: 'package'
                                        target: 'manager'
                                        local_package_path: '/home/luisgr/work/packages/wazuh-manager_4.2.0-1_amd64.deb'
                                        installation_files_path: '/tmp'

                        qa_framework:
                                wazuh_qa_branch: '4.2'
                                qa_workdir: '/tmp'

 tests:
         test1:
                 hosts: '172.16.1.10'
                 type: pytest
                 path:
                         test_files_path: '/tmp/wazuh-qa/tests/integration/test_analysisd/test_all_syscheckd_configuration/test_check_rare_socket_responses.py'
                         run_tests_dir_path: '/tmp/wazuh-qa/tests/integration'
                         test_results_path: '/home/luisgr/Documents/test'
                 parameters:
                         # #tier: str
                         # stop_after_first_failure: boolean
                         # keyword_expression: str
                         # traceback: str
                         dry_run: True
                         # custom_args: list(str)
                         # verbose_level: boolean
                         # log_level: str
                         # markers: list(str)

At test execution phase, I got this error:

UnboundLocalError: local variable 'qa_provisioning' referenced before assignment

Need to fix this error and find out why the HTML report generation fails.
Update: in order to use the test execution phase, a previous provisioning is currently needed(#1734).

Also updated the Provisioning module documentation here: provisioning was missing.

@roronoasins
Copy link
Author

roronoasins commented Aug 19, 2021

When you run the tests, the following error is prompted:

image (1)

Changing the path solves the error, html report and assets are generated in /home/vagrant. So that was the problem, when using that path all works as expected and report is located correctly.

Now the HTML report does not have test info, but time is spent anyway:
image

Update: If you don't use --collect-only mannualy in vagrant machine, tests will not pass(maybe is cause I am using 4.2 branch)

Not using dry_run at tests section in conf.yaml or setting it to False, will generate a correct html report(not simulating the execution). If it is set to True, an empty html report will be generated instead, as showed above.

image

Also correctly located:

image

@roronoasins
Copy link
Author

provisioning and test execution yaml syntax updated #1533 (comment)

roronoasins pushed a commit that referenced this issue Aug 23, 2021
Now the tests execution path is used, not the user home dir.
roronoasins pushed a commit that referenced this issue Aug 23, 2021
Now all the reports(assets, plain and html reports) are stored in a dedicated directory.
@roronoasins roronoasins mentioned this issue Aug 25, 2021
8 tasks
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

No branches or pull requests

2 participants