Update API performance tests #1881

yanazaeva · 2021-09-13T11:57:00Z

Related issue
wazuh/wazuh#9920

Description

Hi team,

In this PR we have updated the API performance tests based on the latest changes made in the main repo.

Test result.

================================================== test session starts ===================================================
platform linux -- Python 3.8.8, pytest-6.2.3, py-1.10.0, pluggy-0.13.1 -- /root/anaconda3/bin/python3
cachedir: .pytest_cache
metadata: {'Python': '3.8.8', 'Platform': 'Linux-5.11.0-7633-generic-x86_64-with-glibc2.10', 'Packages': {'pytest': '6.2.3', 'py': '1.10.0', 'pluggy': '0.13.1'}, 'Plugins': {'html': '3.1.1', 'cov': '2.12.1', 'metadata': '1.11.0', 'anyio': '2.2.0', 'testinfra': '5.0.0'}}
rootdir: /home/yanazaeva/Documents/github/wazuh-qa
plugins: html-3.1.1, cov-2.12.1, metadata-1.11.0, anyio-2.2.0, testinfra-5.0.0
collected 4 items                                                                                                        

tests/system/test_cluster/test_integrity_sync/test_integrity_sync.py::test_missing_file PASSED
tests/system/test_cluster/test_integrity_sync/test_integrity_sync.py::test_shared_files PASSED
tests/system/test_cluster/test_integrity_sync/test_integrity_sync.py::test_extra_files PASSED
tests/system/test_cluster/test_integrity_sync/test_integrity_sync.py::test_extra_valid_files PASSED

============================================= 4 passed in 348.73s (0:05:48) ==============================================

Tests

Proven that tests pass when they have to pass.
Proven that tests fail when they have to fail.

Python codebase satisfies PEP-8 style style guide. pycodestyle --max-line-length=120 --show-source --show-pep8 file.py.
Python codebase is documented following the Google Style for Python docstrings.
The test is documented in wazuh-qa/docs.
provision_documentation.sh generate the docs without errors.

The base branch was changed.

juliamagan

When launching the test 1 of the 4 cases has failed in all three rounds:

	Local
R1	🔴
R2	🔴
R3	🔴

This is part of the output obtained:

DEBUG    testinfra:base.py:277 RUN CommandResult(command=b'ansible --tree /tmp/tmpbr92mzbl -i /home/juliamagan/Escritorio/wazuh-qa/tests/system/provisioning/agentless_cluster/inventory.yml -m command --args \'stat -c "%a" /var/ossec/etc/shared/test_group/test_file\' wazuh-worker1', exit_status=0, stdout=b'wazuh-worker1 | CHANGED | rc=0 >>\n660\n', stderr=None)
INFO     testinfra:ansible.py:58 RUN Ansible('command', 'stat -c "%a" /var/ossec/etc/shared/test_group/test_file', {'check': False, 'become': False}): {'changed': True,
 'cmd': ['stat', '-c', '%a', '/var/ossec/etc/shared/test_group/test_file'],
 'delta': '0:00:00.145148',
 'end': '2021-09-15 13:19:07.142385',
 'rc': 0,
 'start': '2021-09-15 13:19:06.997237',
 'stderr': '',
 'stderr_lines': [],
 'stdout': '660',
 'stdout_lines': ['660']}
DEBUG    testinfra:base.py:277 RUN CommandResult(command=b"ansible --tree /tmp/tmpu9kocaz_ -i /home/juliamagan/Escritorio/wazuh-qa/tests/system/provisioning/agentless_cluster/inventory.yml -m command --args 'ls /var/ossec/etc/test_file' wazuh-worker1", exit_status=2, stdout=b"wazuh-worker1 | FAILED | rc=2 >>\nls: cannot access '/var/ossec/etc/test_file': No such file or directorynon-zero return code\n", stderr=None)
INFO     testinfra:ansible.py:58 RUN Ansible('command', 'ls /var/ossec/etc/test_file', {'check': False, 'become': False}): {'changed': True,
 'cmd': ['ls', '/var/ossec/etc/test_file'],
 'delta': '0:00:00.149665',
 'end': '2021-09-15 13:19:09.009865',
 'msg': 'non-zero return code',
 'rc': 2,
 'start': '2021-09-15 13:19:08.860200',
 'stderr': "ls: cannot access '/var/ossec/etc/test_file': No such file or "
           'directory',
 'stderr_lines': ["ls: cannot access '/var/ossec/etc/test_file': No such file "
                  'or directory'],
 'stdout': '',
 'stdout_lines': []}
DEBUG    testinfra:base.py:277 RUN CommandResult(command=b"ansible --tree /tmp/tmpi1ddw6dl -i /home/juliamagan/Escritorio/wazuh-qa/tests/system/provisioning/agentless_cluster/inventory.yml -m command --args 'ls /var/ossec/etc/lists/ar.conf' wazuh-worker1", exit_status=2, stdout=b"wazuh-worker1 | FAILED | rc=2 >>\nls: cannot access '/var/ossec/etc/lists/ar.conf': No such file or directorynon-zero return code\n", stderr=None)
INFO     testinfra:ansible.py:58 RUN Ansible('command', 'ls /var/ossec/etc/lists/ar.conf', {'check': False, 'become': False}): {'changed': True,
 'cmd': ['ls', '/var/ossec/etc/lists/ar.conf'],
 'delta': '0:00:00.142498',
 'end': '2021-09-15 13:19:10.851457',
 'msg': 'non-zero return code',
 'rc': 2,
 'start': '2021-09-15 13:19:10.708959',
 'stderr': "ls: cannot access '/var/ossec/etc/lists/ar.conf': No such file or "
           'directory',
 'stderr_lines': ["ls: cannot access '/var/ossec/etc/lists/ar.conf': No such "
                  'file or directory'],
 'stdout': '',
 'stdout_lines': []}
DEBUG    testinfra:base.py:277 RUN CommandResult(command=b"ansible --tree /tmp/tmpihtw8yrd -i /home/juliamagan/Escritorio/wazuh-qa/tests/system/provisioning/agentless_cluster/inventory.yml -m command --args 'ls /var/ossec/etc/lists/ossec.conf' wazuh-worker1", exit_status=2, stdout=b"wazuh-worker1 | FAILED | rc=2 >>\nls: cannot access '/var/ossec/etc/lists/ossec.conf': No such file or directorynon-zero return code\n", stderr=None)
INFO     testinfra:ansible.py:58 RUN Ansible('command', 'ls /var/ossec/etc/lists/ossec.conf', {'check': False, 'become': False}): {'changed': True,
 'cmd': ['ls', '/var/ossec/etc/lists/ossec.conf'],
 'delta': '0:00:00.138224',
 'end': '2021-09-15 13:19:12.700646',
 'msg': 'non-zero return code',
 'rc': 2,
 'start': '2021-09-15 13:19:12.562422',
 'stderr': "ls: cannot access '/var/ossec/etc/lists/ossec.conf': No such file "
           'or directory',
 'stderr_lines': ["ls: cannot access '/var/ossec/etc/lists/ossec.conf': No "
                  'such file or directory'],
 'stdout': '',
 'stdout_lines': []}
DEBUG    testinfra:base.py:277 RUN CommandResult(command=b"ansible --tree /tmp/tmpsymi1tpl -i /home/juliamagan/Escritorio/wazuh-qa/tests/system/provisioning/agentless_cluster/inventory.yml -m command --args 'ls /var/ossec/etc/lists/test.tmp' wazuh-worker1", exit_status=2, stdout=b"wazuh-worker1 | FAILED | rc=2 >>\nls: cannot access '/var/ossec/etc/lists/test.tmp': No such file or directorynon-zero return code\n", stderr=None)
INFO     testinfra:ansible.py:58 RUN Ansible('command', 'ls /var/ossec/etc/lists/test.tmp', {'check': False, 'become': False}): {'changed': True,
 'cmd': ['ls', '/var/ossec/etc/lists/test.tmp'],
 'delta': '0:00:00.136152',
 'end': '2021-09-15 13:19:14.709767',
 'msg': 'non-zero return code',
 'rc': 2,
 'start': '2021-09-15 13:19:14.573615',
 'stderr': "ls: cannot access '/var/ossec/etc/lists/test.tmp': No such file or "
           'directory',
 'stderr_lines': ["ls: cannot access '/var/ossec/etc/lists/test.tmp': No such "
                  'file or directory'],
 'stdout': '',
 'stdout_lines': []}
DEBUG    testinfra:base.py:277 RUN CommandResult(command=b"ansible --tree /tmp/tmpmh_3qwe5 -i /home/juliamagan/Escritorio/wazuh-qa/tests/system/provisioning/agentless_cluster/inventory.yml -m command --args 'ls /var/ossec/etc/lists/test.lock' wazuh-worker1", exit_status=2, stdout=b"wazuh-worker1 | FAILED | rc=2 >>\nls: cannot access '/var/ossec/etc/lists/test.lock': No such file or directorynon-zero return code\n", stderr=None)
INFO     testinfra:ansible.py:58 RUN Ansible('command', 'ls /var/ossec/etc/lists/test.lock', {'check': False, 'become': False}): {'changed': True,
 'cmd': ['ls', '/var/ossec/etc/lists/test.lock'],
 'delta': '0:00:00.145096',
 'end': '2021-09-15 13:19:16.720374',
 'msg': 'non-zero return code',
 'rc': 2,
 'start': '2021-09-15 13:19:16.575278',
 'stderr': "ls: cannot access '/var/ossec/etc/lists/test.lock': No such file "
           'or directory',
 'stderr_lines': ["ls: cannot access '/var/ossec/etc/lists/test.lock': No such "
                  'file or directory'],
 'stdout': '',
 'stdout_lines': []}
DEBUG    testinfra:base.py:277 RUN CommandResult(command=b"ansible --tree /tmp/tmpj_g3ftb7 -i /home/juliamagan/Escritorio/wazuh-qa/tests/system/provisioning/agentless_cluster/inventory.yml -m command --args 'ls /var/ossec/etc/lists/test.swp' wazuh-worker1", exit_status=2, stdout=b"wazuh-worker1 | FAILED | rc=2 >>\nls: cannot access '/var/ossec/etc/lists/test.swp': No such file or directorynon-zero return code\n", stderr=None)
INFO     testinfra:ansible.py:58 RUN Ansible('command', 'ls /var/ossec/etc/lists/test.swp', {'check': False, 'become': False}): {'changed': True,
 'cmd': ['ls', '/var/ossec/etc/lists/test.swp'],
 'delta': '0:00:00.147286',
 'end': '2021-09-15 13:19:18.711873',
 'msg': 'non-zero return code',
 'rc': 2,
 'start': '2021-09-15 13:19:18.564587',
 'stderr': "ls: cannot access '/var/ossec/etc/lists/test.swp': No such file or "
           'directory',
 'stderr_lines': ["ls: cannot access '/var/ossec/etc/lists/test.swp': No such "
                  'file or directory'],
 'stdout': '',
 'stdout_lines': []}
DEBUG    testinfra:base.py:277 RUN CommandResult(command=b"ansible --tree /tmp/tmpv46m7g5o -i /home/juliamagan/Escritorio/wazuh-qa/tests/system/provisioning/agentless_cluster/inventory.yml -m command --args 'ls /var/ossec/var/multigroups/test_dir/test_file' wazuh-worker1", exit_status=2, stdout=b"wazuh-worker1 | FAILED | rc=2 >>\nls: cannot access '/var/ossec/var/multigroups/test_dir/test_file': No such file or directorynon-zero return code\n", stderr=None)
INFO     testinfra:ansible.py:58 RUN Ansible('command', 'ls /var/ossec/var/multigroups/test_dir/test_file', {'check': False, 'become': False}): {'changed': True,
 'cmd': ['ls', '/var/ossec/var/multigroups/test_dir/test_file'],
 'delta': '0:00:00.143837',
 'end': '2021-09-15 13:19:20.636571',
 'msg': 'non-zero return code',
 'rc': 2,
 'start': '2021-09-15 13:19:20.492734',
 'stderr': "ls: cannot access '/var/ossec/var/multigroups/test_dir/test_file': "
           'No such file or directory',
 'stderr_lines': ['ls: cannot access '
                  "'/var/ossec/var/multigroups/test_dir/test_file': No such "
                  'file or directory'],
 'stdout': '',
 'stdout_lines': []}
DEBUG    testinfra:base.py:277 RUN CommandResult(command=b"ansible --tree /tmp/tmpm19yizpd -i /home/juliamagan/Escritorio/wazuh-qa/tests/system/provisioning/agentless_cluster/inventory.yml -m command --args 'ls /var/ossec/etc/shared/test_group/agent.conf' wazuh-worker1", exit_status=0, stdout=b'wazuh-worker1 | CHANGED | rc=0 >>\n/var/ossec/etc/shared/test_group/agent.conf\n', stderr=None)
INFO     testinfra:ansible.py:58 RUN Ansible('command', 'ls /var/ossec/etc/shared/test_group/agent.conf', {'check': False, 'become': False}): {'changed': True,
 'cmd': ['ls', '/var/ossec/etc/shared/test_group/agent.conf'],
 'delta': '0:00:00.141136',
 'end': '2021-09-15 13:19:22.594065',
 'rc': 0,
 'start': '2021-09-15 13:19:22.452929',
 'stderr': '',
 'stderr_lines': [],
 'stdout': '/var/ossec/etc/shared/test_group/agent.conf',
 'stdout_lines': ['/var/ossec/etc/shared/test_group/agent.conf']}
-------------------------------- generated html file: file:///home/juliamagan/Escritorio/wazuh-qa/tests/system/test_cluster/test_integrity_sync/system_test_integrity_sync_r3.html --------------------------------
============================================================================================= short test summary info =============================================================================================
FAILED test_integrity_sync.py::test_missing_file - AssertionError: File /var/ossec/etc/shared/test_group/agent.conf was expected not to be copied in wazuh-worker1, but it was.

yanazaeva · 2021-09-16T09:24:32Z

PR update.

Added changes according to the base change.

QA system test output:

================================================================================================================================== test session starts ===================================================================================================================================
platform linux -- Python 3.8.10, pytest-6.2.3, py-1.10.0, pluggy-0.13.1 -- /usr/bin/python3
cachedir: .pytest_cache
metadata: {'Python': '3.8.10', 'Platform': 'Linux-5.11.0-7633-generic-x86_64-with-glibc2.29', 'Packages': {'pytest': '6.2.3', 'py': '1.10.0', 'pluggy': '0.13.1'}, 'Plugins': {'html': '3.1.1', 'metadata': '1.11.0', 'testinfra': '5.0.0'}}
rootdir: /home/yanazaeva/Documents/github/wazuh-qa/tests/system/test_cluster/test_integrity_sync
plugins: html-3.1.1, metadata-1.11.0, testinfra-5.0.0
collected 4 items                                                                                                                                                                                                                                                                        

test_integrity_sync.py::test_missing_file PASSED
test_integrity_sync.py::test_shared_files PASSED
test_integrity_sync.py::test_extra_files PASSED
test_integrity_sync.py::test_extra_valid_files PASSED

============================================================================================================================= 4 passed in 320.81s (0:05:20) ==============================================================================================================================

juliamagan · 2021-09-16T14:21:30Z

	Local
R1	🟢
R2	🟢
R3	🟢

yanazaeva requested review from AdriiiPRodri and Selutario September 13, 2021 11:57

yanazaeva changed the base branch from master to 4.2 September 13, 2021 12:03

yanazaeva requested review from davidjiglesias and removed request for AdriiiPRodri and Selutario September 13, 2021 12:07

juliamagan self-requested a review September 14, 2021 08:26

Selutario previously approved these changes Sep 14, 2021

View reviewed changes

yanazaeva self-assigned this Sep 14, 2021

Modify test_integrity_sync test according to the issue 9920

9ecce46

yanazaeva force-pushed the 9920-api-performance-test-update branch from b37d1e0 to 9ecce46 Compare September 14, 2021 14:57

yanazaeva changed the base branch from 4.2 to master September 14, 2021 14:57

yanazaeva requested a review from Selutario September 14, 2021 14:57

Selutario previously approved these changes Sep 15, 2021

View reviewed changes

juliamagan reviewed Sep 15, 2021

View reviewed changes

yanazaeva mentioned this pull request Sep 16, 2021

Newly added nodes do not replicate group files wazuh/wazuh#9920

Closed

Update tests according to the new base

ac1af70

yanazaeva dismissed Selutario’s stale review via ac1af70 September 16, 2021 09:20

yanazaeva requested review from juliamagan and Selutario September 16, 2021 09:32

juliamagan approved these changes Sep 16, 2021

View reviewed changes

Selutario approved these changes Sep 16, 2021

View reviewed changes

davidjiglesias approved these changes Sep 27, 2021

View reviewed changes

davidjiglesias requested a review from jmv74211 September 27, 2021 07:39

jmv74211 merged commit 475701a into master Sep 27, 2021

jmv74211 deleted the 9920-api-performance-test-update branch September 27, 2021 07:42

snaow mentioned this pull request Jan 27, 2022

QA Release - Rev 430031 #2500

Closed

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Update API performance tests #1881

Update API performance tests #1881

yanazaeva commented Sep 13, 2021 •

edited

Loading

juliamagan left a comment •

edited

Loading

yanazaeva commented Sep 16, 2021

juliamagan commented Sep 16, 2021

Update API performance tests #1881

Update API performance tests #1881

Conversation

yanazaeva commented Sep 13, 2021 • edited Loading

Description

Test result.

Tests

juliamagan left a comment • edited Loading

Choose a reason for hiding this comment

yanazaeva commented Sep 16, 2021

PR update.

juliamagan commented Sep 16, 2021

yanazaeva commented Sep 13, 2021 •

edited

Loading

juliamagan left a comment •

edited

Loading