Tests start before feeds are updated

[juliamagan]

Description

During the execution of the tests to check that the development of #5108 was correct, it was seen that the tests failed because they started before the feeds were updated.

Currently, the tests wait for the INFO: Vulnerability scanner module started in:

wazuh-qa/tests/end_to_end/test_vulnerability_detector/test_vulnerability_detector.py

Lines 150 to 152 in 96cf0b6

	# Wait until VD is updated
	logger.error("Wait until Vulnerability Detector has update all the feeds")
	wait_until_vd_is_updated(host_manager)

However, this message appears when the manager is restarted, even though the feeds have not been updated. It is necessary to investigate and find a log that indicates that the feeds have been updated in order to set this as a starting point.

[santipadilla]

Moved ETA and on hold due to 4.8.0 - beta 5 testing.

[santipadilla]

The log indicating the feed has been updated is as follows:
INFO: Feed update process completed

[santipadilla]

Changes made in the following branch:
5173-wait-until-vd-is-updated

In this PR: #5227

[santipadilla]

Feed start

We wait until the feed has finished updating.

(waitVD) santipa@santipa-Lenovo-Legion-5-15IMH05:~/Escritorio/Issue_Wait/wazuh-qa/tests$ python3 -m pytest end_to_end/test_vulnerability_detector --inventory-path=/home/santipa/Escritorio/Wazuh_QA_environment1098_testing_inventory.yaml --html=report.html
=========================================================================================== test session starts ============================================================================================
platform linux -- Python 3.10.12, pytest-7.1.2, pluggy-1.4.0
rootdir: /home/santipa/Escritorio/Issue_Wait/wazuh-qa/tests/end_to_end, configfile: pytest.ini
plugins: testinfra-5.0.0, metadata-3.1.1, html-3.1.1
collected 14 items                                                                                                                                                                                         

end_to_end/test_vulnerability_detector/test_vulnerability_detector.py::TestInitialScans::test_syscollector_first_scan 
---------------------------------------------------------------------------------------------- live log setup ----------------------------------------------------------------------------------------------
2024-04-16 13:19:58 Init setup of environment (test_vulnerability_detector.py:153)
2024-04-16 13:19:58 Getting backup of current configurations (test_vulnerability_detector.py:156)
2024-04-16 13:20:06 Configuring environment (test_vulnerability_detector.py:159)
2024-04-16 13:20:19 Save the Wazuh indexer username and password into the Wazuh manager keystore (test_vulnerability_detector.py:161)
2024-04-16 13:20:24 Truncate managers and agents logs (test_vulnerability_detector.py:165)
2024-04-16 13:20:38 Stopping agents (test_vulnerability_detector.py:169)
2024-04-16 13:20:42 Restarting managers (test_vulnerability_detector.py:177)
2024-04-16 13:21:02 Wait until Vulnerability Detector has update all the feeds (test_vulnerability_detector.py:181)

Manager

2024/04/16 11:34:37 wazuh-modulesd:vulnerability-scanner: INFO: Initiating update feed process
2024/04/16 11:34:37 wazuh-modulesd:vulnerability-scanner: INFO: Processing file: queue/vd_updater/tmp/contents/431557-api_file.json
2024/04/16 11:35:03 wazuh-modulesd:vulnerability-scanner: INFO: Processing file: queue/vd_updater/tmp/contents/432557-api_file.json
2024/04/16 11:35:31 wazuh-modulesd:vulnerability-scanner: INFO: Processing file: queue/vd_updater/tmp/contents/433557-api_file.json
...

Feed completed

(waitVD) santipa@santipa-Lenovo-Legion-5-15IMH05:~/Escritorio/Issue_Wait/wazuh-qa/tests$ python3 -m pytest end_to_end/test_vulnerability_detector --inventory-path=/home/santipa/Escritorio/Wazuh_QA_environment1098_testing_inventory.yaml --html=report.html
=========================================================================================== test session starts ============================================================================================
platform linux -- Python 3.10.12, pytest-7.1.2, pluggy-1.4.0
rootdir: /home/santipa/Escritorio/Issue_Wait/wazuh-qa/tests/end_to_end, configfile: pytest.ini
plugins: testinfra-5.0.0, metadata-3.1.1, html-3.1.1
collected 14 items                                                                                                                                                                                         

end_to_end/test_vulnerability_detector/test_vulnerability_detector.py::TestInitialScans::test_syscollector_first_scan 
---------------------------------------------------------------------------------------------- live log setup ----------------------------------------------------------------------------------------------
2024-04-16 13:19:58 Init setup of environment (test_vulnerability_detector.py:153)
2024-04-16 13:19:58 Getting backup of current configurations (test_vulnerability_detector.py:156)
2024-04-16 13:20:06 Configuring environment (test_vulnerability_detector.py:159)
2024-04-16 13:20:19 Save the Wazuh indexer username and password into the Wazuh manager keystore (test_vulnerability_detector.py:161)
2024-04-16 13:20:24 Truncate managers and agents logs (test_vulnerability_detector.py:165)
2024-04-16 13:20:38 Stopping agents (test_vulnerability_detector.py:169)
2024-04-16 13:20:42 Restarting managers (test_vulnerability_detector.py:177)
2024-04-16 13:21:02 Wait until Vulnerability Detector has update all the feeds (test_vulnerability_detector.py:181)
---------------------------------------------------------------------------------------------- live log call -----------------------------------------------------------------------------------------------
2024-04-16 14:36:37 Monitoring syscollector first scan (test_vulnerability_detector.py:253)
2024-04-16 14:36:45 Value of monitoring results is: {'agent1': {'not_found': [], 'found': ['2024/04/16 12:36:34']}} (test_vulnerability_detector.py:262)
2024-04-16 14:36:45 All agents has been scanned (test_vulnerability_detector.py:277)
PASSED

Manager

2024/04/16 12:36:03 wazuh-modulesd:vulnerability-scanner: INFO: Processing file: queue/vd_updater/tmp/contents/574276-api_file.json
2024/04/16 12:36:24 wazuh-modulesd:vulnerability-scanner: INFO: Feed update process completed

[santipadilla]

Moved to on hold due to 4.8.0 - beta 6 testing.

[santipadilla]

Update

When launching the VD test for the first time we find this log in the manager "INFO: Vulnerability scanner module started" followed by "INFO: Initiating update feed process" and when it is finished "INFO: Feed update process completed".

When we launch it after the first time the only log we get is "INFO: Vulnerability scanner module started" since the feed has been updated and completed previously.

What we see in common is that the log "INFO: Vulnerability scanner module started" appears in both.

So to carry out the development when this log comes out we wait a while to see if "INFO: Initiating feed update process" appears and then wait for the feed to complete "INFO: Feed update process completed".
And if this "INFO: Initiating update feed process" does not appear after that time, it means that the feed was completed previously.

[santipadilla]

Testing

Running the VD test for the first time

VD test (Display logs via print for debugging)

(waitVD) santipa@santipa-Lenovo-Legion-5-15IMH05:~/Escritorio/Issue_Wait/wazuh-qa/tests$ python3 -m pytest -s end_to_end/test_vulnerability_detector --inventory-path=/home/santipa/Escritorio/Wazuh_QA_environment1133_testing_inventory.yaml --html=report.html
=========================================================================================== test session starts ============================================================================================
platform linux -- Python 3.10.12, pytest-7.1.2, pluggy-1.4.0
rootdir: /home/santipa/Escritorio/Issue_Wait/wazuh-qa/tests/end_to_end, configfile: pytest.ini
plugins: testinfra-5.0.0, metadata-3.1.1, html-3.1.1
collected 14 items                                                                                                                                                                                         

end_to_end/test_vulnerability_detector/test_vulnerability_detector.py::TestInitialScans::test_syscollector_first_scan 
---------------------------------------------------------------------------------------------- live log setup ----------------------------------------------------------------------------------------------
2024-04-26 14:04:31 Init setup of environment (test_vulnerability_detector.py:153)
2024-04-26 14:04:31 Getting backup of current configurations (test_vulnerability_detector.py:156)
2024-04-26 14:04:53 Configuring environment (test_vulnerability_detector.py:159)
2024-04-26 14:05:06 Save the Wazuh indexer username and password into the Wazuh manager keystore (test_vulnerability_detector.py:161)
2024-04-26 14:05:18 Truncate managers and agents logs (test_vulnerability_detector.py:165)
2024-04-26 14:06:00 Stopping agents (test_vulnerability_detector.py:169)
2024-04-26 14:06:19 Restarting managers (test_vulnerability_detector.py:177)
2024-04-26 14:07:03 Wait until Vulnerability Detector has update all the feeds (test_vulnerability_detector.py:181)
Vulnerability scanner has started. Waiting for feed update completion...
Feed update process initiated successfully. Waiting for feed update completion...
Feed update process completed successfully
---------------------------------------------------------------------------------------------- live log call -----------------------------------------------------------------------------------------------
2024-04-26 14:14:05 Monitoring syscollector first scan (test_vulnerability_detector.py:253)
2024-04-26 14:14:14 Value of monitoring results is: {'agent5': {'not_found': [], 'found': ['2024/04/26 12:06:14', '2024/04/26 12:13:55']}, 'agent4': {'not_found': [], 'found': ['2024/04/26 12:06:10', '2024/04/26 12:13:43']}, 'agent1': {'not_found': [], 'found': ['2024/04/26 12:13:22']}, 'agent3': {'not_found': [], 'found': ['2024/04/26 12:13:32']}, 'agent2': {'not_found': [], 'found': ['2024/04/26 12:14:03']}} (test_vulnerability_detector.py:262)
2024-04-26 14:14:14 All agents has been scanned (test_vulnerability_detector.py:277)
PASSED

Manager

2024/04/26 12:06:35 indexer-connector: INFO: IndexerConnector initialized successfully for index: wazuh-states-vulnerabilities.
2024/04/26 12:08:42 wazuh-modulesd:vulnerability-scanner: INFO: Database decompression finished.
2024/04/26 12:08:43 wazuh-modulesd:vulnerability-scanner: INFO: Initiating update feed process
2024/04/26 12:08:43 wazuh-modulesd:vulnerability-scanner: INFO: Vulnerability scanner module started
2024/04/26 12:11:13 wazuh-modulesd:vulnerability-scanner: INFO: Feed update process completed

Running the VD test from the first time

VD test (Display logs via print for debugging)

(waitVD) santipa@santipa-Lenovo-Legion-5-15IMH05:~/Escritorio/Issue_Wait/wazuh-qa/tests$ python3 -m pytest -s end_to_end/test_vulnerability_detector --inventory-path=/home/santipa/Escritorio/Wazuh_QA_environment1133_testing_inventory.yaml --html=report.html
=========================================================================================== test session starts ============================================================================================
platform linux -- Python 3.10.12, pytest-7.1.2, pluggy-1.4.0
rootdir: /home/santipa/Escritorio/Issue_Wait/wazuh-qa/tests/end_to_end, configfile: pytest.ini
plugins: testinfra-5.0.0, metadata-3.1.1, html-3.1.1
collected 14 items                                                                                                                                                                                         

end_to_end/test_vulnerability_detector/test_vulnerability_detector.py::TestInitialScans::test_syscollector_first_scan 
---------------------------------------------------------------------------------------------- live log setup ----------------------------------------------------------------------------------------------
2024-04-26 14:19:15 Init setup of environment (test_vulnerability_detector.py:153)
2024-04-26 14:19:15 Getting backup of current configurations (test_vulnerability_detector.py:156)
2024-04-26 14:19:40 Configuring environment (test_vulnerability_detector.py:159)
2024-04-26 14:19:52 Save the Wazuh indexer username and password into the Wazuh manager keystore (test_vulnerability_detector.py:161)
2024-04-26 14:20:04 Truncate managers and agents logs (test_vulnerability_detector.py:165)
2024-04-26 14:20:46 Stopping agents (test_vulnerability_detector.py:169)
2024-04-26 14:21:05 Restarting managers (test_vulnerability_detector.py:177)
2024-04-26 14:21:49 Wait until Vulnerability Detector has update all the feeds (test_vulnerability_detector.py:181)
Vulnerability scanner has started. Waiting for feed update completion...
Feed update initiated log not found, the feed has already been updated previously
---------------------------------------------------------------------------------------------- live log call -----------------------------------------------------------------------------------------------
2024-04-26 14:24:23 Monitoring syscollector first scan (test_vulnerability_detector.py:253)
2024-04-26 14:24:31 Value of monitoring results is: {'agent5': {'not_found': [], 'found': ['2024/04/26 12:24:13']}, 'agent4': {'not_found': [], 'found': ['2024/04/26 12:24:01']}, 'agent3': {'not_found': [], 'found': ['2024/04/26 12:23:49']}, 'agent1': {'not_found': [], 'found': ['2024/04/26 12:23:39']}, 'agent2': {'not_found': [], 'found': ['2024/04/26 12:24:19']}} (test_vulnerability_detector.py:262)
2024-04-26 14:24:31 All agents has been scanned (test_vulnerability_detector.py:277)
PASSED

Manager

2024/04/26 12:21:20 indexer-connector: INFO: IndexerConnector initialized successfully for index: wazuh-states-vulnerabilities.
2024/04/26 12:21:20 wazuh-modulesd:vulnerability-scanner: INFO: Vulnerability scanner module started

[santipadilla]

Moved ETA due to pending final review