From 9f3420fa0ded5bb4927c627fa2591688dd885a31 Mon Sep 17 00:00:00 2001 From: Alvaro Carvajal Date: Tue, 15 Oct 2024 13:39:43 +0200 Subject: [PATCH] Add `_angi` to shared directory name when testing HanaSR angi Cluster tests on `svirt` and `pvm_hmc` backends rely on a directory created on a NFS share, where nodes can share information regarding node's IP addresses, iSCSI LUNs and other details. This is implemented in: https://github.com/os-autoinst/os-autoinst-distri-opensuse/blob/master/tests/ha/setup_hosts_and_luns.pm#L43 This means that if running 2 sets of cluster tests at the same time, on the same build, same version and same architecture on either of these 2 backends, it can lead to a race condition if both clusters have the same `CLUSTER_NAME` setting. This is currently happening on the pre-release job group which has 2 HANA clusters (one testing `SAPHanaSR` resources, and the other testing `SAPHanaSR-angi` resources) running on `ppc64le-hmc-sap` workers both with setting `CLUSTER_NAME: hana`. As can be seen, as the directory name is composed of the cluster name, the version under test, the architecture and the build, this means that if tests for 2 clusters with the same name run at the same time for any given build, both clusters will try to use the same directory to exchange information introducing a race condition which can lead to failures such as the one in https://openqa.suse.de/tests/15631415#step/ha_cluster_join/7. There, the other cluster overwrote the information of the running test, leading to a failure to add `hana-node01` to `/etc/hosts`. A first attempt to avoid the race condition was made by replacing the value of `CLUSTER_NAME: hana` for `CLUSTER_NAME: angi` in the SAPHanaSR-angi tests in the job group level (MR#801 in openqa_ha_sap and MR#793 in qam-openqa-yml), however this did not work as there are several test modules and libraries which rely on `CLUSTER_NAME: hana` to perform HANA specific actions, such as the takeover. For example: * tests/ha/check_after_reboot.pm: line 213 (and others) * tests/ha/fencing.pm: line 70 * lib/hacluster.pm: line 669 So for the time being, cluster tests for hana require the `CLUSTER_NAME` setting to be set to `hana`. This commit is a second attempt to avoid the race condition. It adds the string `_angi` to the shared directory name if running with the `USE_SAP_HANA_SR_ANGI` setting set to true. This would help us avoid the existing race condition as the 2 sets of HanaSR tests currently under test are a classic `SAPHanaSR` and a `SAPHanaSR-angi` one. Eventually when adding more cluster scenarios on pvm_hmc, further changes will be required. We will track these in the ticket https://jira.suse.com/browse/TEAM-9750 --- tests/ha/setup_hosts_and_luns.pm | 1 + 1 file changed, 1 insertion(+) diff --git a/tests/ha/setup_hosts_and_luns.pm b/tests/ha/setup_hosts_and_luns.pm index 686a9d053f3f..b19d90e7cf53 100644 --- a/tests/ha/setup_hosts_and_luns.pm +++ b/tests/ha/setup_hosts_and_luns.pm @@ -49,6 +49,7 @@ sub run { $testname =~ s/@.+$//; $testname =~ s/_node\d+$//; $dir_id .= "_$testname" if get_var('HDDVERSION', ''); + $dir_id .= '_angi' if (get_var('USE_SAP_HANA_SR_ANGI', ''); set_var('NFS_SUPPORT_DIR', "$mountpt/$dir_id"); assert_script_run "mkdir -p $mountpt";