Enable feature autorestart before test_encap_with_mirror_session #12909
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
echuawu
approved these changes
May 21, 2024
mrkcmo
pushed a commit
to Azarack/sonic-mgmt
that referenced
this pull request
Jul 8, 2024
[sdn_tests]: Adding results testhelper to pins_ondatra. (sonic-net#12675) * [sdn_tests]: Adding Build support for pins_ondatra * [sdn_tests]: Adding binding infra to pins_ondatra. * [sdn_tests]: Adding cert generation script and certs folder to pins_ondatra. * [sdn_tests]: Adding Data Infra to pins_ondatra. * [sdn_tests]: Adding gnmi testhelper to pins_ondatra. * [sdn_tests]: Adding GNOI testhelper to pins_ondatra. * [sdn_tests]: Adding LACP testhelper to pins_ondatra. * [sdn_tests]: Adding P4RT testhelper to pins_ondatra. * [sdn_tests]: Adding Platform and port_management testhelper to pins_ondatra. * [sdn_tests]: Adding testhelper to pins_ondatra. * [sdn_tests]: Adding Augment testhelper Infra to pins_ondatra. * [sdn_tests]: Adding results testhelper to pins_ondatra. [Feat] Enhance upgrade image script to support chassis devices (sonic-net#12871) Description of PR Summary: Enhance upgrade image script to support chassis device. For chassis device, we need to firstly upgrade the image for supervisor cards, then upgrade the image for line cards. Approach What is the motivation for this PR? Enhance the upgrade_image script to support chassis devices. How did you do it? Upgrade image on the supervisor cards, then wait 900s for the supervisor card to be ready. Upgrade image on the line cards, then wait 300s for the line cards to be ready. The sonichosts defautly run commands on all supervisor cards and line cards at the same time, enhance the framework to be able to upgrade specific hosts. How did you verify/test it? Run upgrade image script on the chassis device and pizzbox device, both of them works well co-authorized by: jianquanye@microsoft.com increase sleep time for packet send/receive (sonic-net#12720) [sdn_tests]: Adding ssh testhelper to pins_ondatra. (sonic-net#12676) * [sdn_tests]: Adding Build support for pins_ondatra * [sdn_tests]: Adding binding infra to pins_ondatra. * [sdn_tests]: Adding cert generation script and certs folder to pins_ondatra. * [sdn_tests]: Adding Data Infra to pins_ondatra. * [sdn_tests]: Adding gnmi testhelper to pins_ondatra. * [sdn_tests]: Adding GNOI testhelper to pins_ondatra. * [sdn_tests]: Adding LACP testhelper to pins_ondatra. * [sdn_tests]: Adding P4RT testhelper to pins_ondatra. * [sdn_tests]: Adding Platform and port_management testhelper to pins_ondatra. * [sdn_tests]: Adding testhelper to pins_ondatra. * [sdn_tests]: Adding Augment testhelper Infra to pins_ondatra. * [sdn_tests]: Adding results testhelper to pins_ondatra. * [sdn_tests]: Adding ssh testhelper to pins_ondatra. Fix qos/test_qos_sai.py::TestQosSai::testQosSaiPfcXonLimit (sonic-net#12649) * Fix qos/test_qos_sai.py::TestQosSai::testQosSaiPfcXonLimit [TACACS] Improve TACACS run command on IPV6 failed issue. (sonic-net#12819) Improve TACACS run command on IPV6 failed issue. Tacplus server crash when receive authorization request from IPV6 address. Check TACACS server status after run command with IPV6 address. Pass all test case. Improve TACACS run command on IPV6 failed issue. [sdn_tests]: Adding ondatra tests bazel to pins_ondatra. (sonic-net#12677) * [sdn_tests]: Adding Build support for pins_ondatra * [sdn_tests]: Adding binding infra to pins_ondatra. * [sdn_tests]: Adding cert generation script and certs folder to pins_ondatra. * [sdn_tests]: Adding Data Infra to pins_ondatra. * [sdn_tests]: Adding gnmi testhelper to pins_ondatra. * [sdn_tests]: Adding GNOI testhelper to pins_ondatra. * [sdn_tests]: Adding LACP testhelper to pins_ondatra. * [sdn_tests]: Adding P4RT testhelper to pins_ondatra. * [sdn_tests]: Adding Platform and port_management testhelper to pins_ondatra. * [sdn_tests]: Adding testhelper to pins_ondatra. * [sdn_tests]: Adding Augment testhelper Infra to pins_ondatra. * [sdn_tests]: Adding results testhelper to pins_ondatra. * [sdn_tests]: Adding ssh testhelper to pins_ondatra. * [sdn_tests]: Adding ondatra bazel tests to pins_ondatra. imporve qos log readability (sonic-net#12526) Log message wrapper: All the messages were outputted to PTF logging by default, and can set flag to output to stderr of PTF console for specific message. so it can avoid lots of message flush on console and "test summary", easy to identify failure when triage. and also can check PTF logging which include all the message when rootcause failure. CounterCollector Class provide general interface for counter collecting, comparing, and displaying. Diagnostic Counter Wrapper so far, we can read 8 kinds of counter: port_counter, queue_counter_counter, queue_share_wm_counter, pg_share_wm_counter, pg_headroom_wm_counter, pg_counter_couner, pg_drop_counter and ptf_tx_rx_counter Although CounterCollector provides a common API to collect, compare and display these counters, if you use countercollect directly, the code of the test case will still become confusing. After all, at least one line of code for each counter. If the types of counter queries are subsequently increased, more code unrelated to the test steps will be exposed in the testcase. Therefore, the diag coutner wrapper is used to include all types of counter activities, so that the code in the test case is more inclined to reflect the test steps and logic rather than these diagnostic codes. assert wrapper By default, we will display the counter difference between the first and last step of this case on both normal and abnormal exits. but using python build-in assert instruction make it difficult to show counter diff. so we implement a assert wrapper to show counter diff when assert exception occur. TextTable Class This is not newly added class, in befor, it help to output counters in table format like well-known python library prettytable. in this PR, add a new class static method "merge_table())" to merge two table which need to show their difference. example case: not applied this feature to all qos testcase. only applied above changes to xoff, xon, lossyqueue cases as a example first. Monitor for long time to collect the feedback, and then enhance. already cover various sku/topo see below test record table skip chassis device since test have not covered chassis yet, skip chassis device support so far. How did you verify/test it? pass verification in lab testbed Set mux mode to manual to disable port toggle in cacl scripts on dualtor testbed (sonic-net#12883) Description of PR GCU test_cacl case failed on dualtor testbed randomly. The failed message: failed on teardown with "Failed: iptable rules are not suppose to change after test. diff: ['+ -A DHCP -m mark --mark 0x67008 -j DROP', '+ -A DHCP -m mark --mark 0x67024 -j DROP', '+ -A DHCP -m mark --mark 0x67022 -j DROP', '+ -A DHCP -m mark --mark 0x67018 -j DROP', '+ -A DHCP -m mark --mark 0x67014 -j DROP', '+ -A DHCP -m mark --mark 0x67013 -j DROP', '+ -A DHCP -m mark --mark 0x67010 -j DROP', '+ -A DHCP -m mark --mark 0x67009 -j DROP', '+ -A DHCP -m mark --mark 0x67004 -j DROP', '+ -A DHCP -m mark --mark 0x67007 -j DROP', '+ -A DHCP -m mark --mark 0x67006 -j DROP', '+ -A DHCP -m mark --mark 0x67002 -j DROP', '+ -A DHCP -m mark --mark 0x67001 -j DROP', '+ -A DHCP -m mark --mark 0x67005 -j DROP']" Summary: Fixes # (issue) The failed reason is dualtor oscillation feature is enabled by default in 202311/mater/internal branch. In nightly test environment, it's easy to trigger oscillation between standby and active ports. In this case, before running test case, some ports are active on DUT, but after test case, when do iptables verification, some ports turn to be standby, that causes iptables DHCP rules to change. The consequence is there is diff between original and latest iptables, which will fail the test case in teardown phase. We have to disable oscillation by set mux to manual mode on both TORs. Enhance the change in sonic-net#11878, set mux mode to manual on both TORs to avoid port state change during test. fix: add topology mark for test_bgp_peer_shutdown.py (sonic-net#12891) Description of PR Add topology mark for test_bgp_peer_shutdown.py so it won't get skipped [CI/CD] Add topo and device_type args for PR test scripts (sonic-net#12731) What is the motivation for this PR? Add "--topology" and "--device_type" arguments to PR test scripts. Fix the issue of "--device_type" skipped on test scripts without "device_type" mark. How did you do it? Add "--topology" and "--device_type" arguments to PR test scripts. Fix the issue of "--device_type" skipped on test scripts without "device_type" mark. How did you verify/test it? Raised Draft PR to test the template script: https://github.com/sonic-net/sonic-mgmt/pull/12645/files Signed-off-by: chunangli <chunangli@microsoft.com> [sdn_tests]: Adding testrunner to pins_ondatra. (sonic-net#12678) * [sdn_tests]: Adding Build support for pins_ondatra * [sdn_tests]: Adding binding infra to pins_ondatra. * [sdn_tests]: Adding cert generation script and certs folder to pins_ondatra. * [sdn_tests]: Adding Data Infra to pins_ondatra. * [sdn_tests]: Adding gnmi testhelper to pins_ondatra. * [sdn_tests]: Adding GNOI testhelper to pins_ondatra. * [sdn_tests]: Adding LACP testhelper to pins_ondatra. * [sdn_tests]: Adding P4RT testhelper to pins_ondatra. * [sdn_tests]: Adding Platform and port_management testhelper to pins_ondatra. * [sdn_tests]: Adding testhelper to pins_ondatra. * [sdn_tests]: Adding Augment testhelper Infra to pins_ondatra. * [sdn_tests]: Adding results testhelper to pins_ondatra. * [sdn_tests]: Adding ssh testhelper to pins_ondatra. * [sdn_tests]: Adding ondatra bazel tests to pins_ondatra. * [sdn_tests]: Adding testrunner to pins_ondatra. Skip DSCP mapping test for test issue (sonic-net#12907) Fix the Keyerror issue caused by none `conn_graph_fact` in VS set-up (sonic-net#12751) What is the motivation for this PR? To adapt to kvm testbed, we need to fix the issue -- conn_graph_fact is none in VS set-up. In this PR, we use method get of a dict to get the value instead of [] to avoid key error when getting value from an empty dict conn_graph_fact. Also, in this PR, after fixing this issue, we can add three platform tests into PR test. [sdn_tests]:Adding Dual switch EthernetCounter Test to pins_ondatra. (sonic-net#12901) [sdn_tests]: Adding GNMI long Stress Test to pins_ondatra. (sonic-net#12902) * [sdn_tests]:Adding Dual switch EthernetCounter Test to pins_ondatra. * [sdn_tests]: Adding GNMI long Stress Test to pins_ondatra. Remove deprecated PR testing pipeline yaml files (sonic-net#12780) PR testing of current repo has been migrated to Elastictest, the legacy PR testing pipeline yaml files are no longer required. This change removed the deprecated PR testing pipeline yaml files. Signed-off-by: Xin Wang <xiwang5@microsoft.com> Remove pre-test and GIT_SECRETS from baseline test (sonic-net#12778) * Remove pre-test and GIT_SECRETS from baseline test Baseline test is to run PR tests multiple times to surface flaky issues. The pre-test step is to find out if there is any importing dependency issue between test scripts. This dependency issue only could happen with new PRs. During baseline test, the dependency issue won't happen. It is unnecessary for baseline test. The GIT_SECRETS variable is not really used by baseline tests. This change also deleted the reference to GIT_SECRETS variable group. Signed-off-by: Xin Wang <xiwang5@microsoft.com> * Fix depends on conditions --------- Signed-off-by: Xin Wang <xiwang5@microsoft.com> [dhcp_server_test] Add multiple vlans test and config change test (sonic-net#12775) Fix m0-2vlan topo prefix_v6 value that causes route_check fail (sonic-net#12851) Explicit zeros in prefix_v6 value causes issue in route_check. Need to use simplified form instead [sdn_tests]:Adding GNOI File Test to pins_ondatra. (sonic-net#12905) * [sdn_tests]:Adding Dual switch EthernetCounter Test to pins_ondatra. * [sdn_tests]: Adding GNMI long Stress Test to pins_ondatra. * [sdn_tests]:Adding GNOI File Test to pins_ondatra. Added temporary skip to system_health TC for MLX platfroms (sonic-net#12798) Update test_ro_disk.py (sonic-net#12913) Fix ro disk test case generate garbled syslog and break loganalyzer issue. Log rotate during ro disk may cause syslog file contains garbled characters. These characters will break loganalyzer, to fix this issue, rotate again to cleanup syslog file. log rotate again after ro disk test case finish, Pass all test case. Fix ro disk test case generate garbled syslog and break loganalyzer issue. Revert "Put back sonic-net#11934, fix qos/test_qos_sai.py teardown for dualtor. (sonic-net#12503)" (sonic-net#12920) This reverts commit 5f6b931. [Snappi] Adding Lossless Response to External pause storm test case (sonic-net#8024) * adding lossless response to external pause storm test case Co-authored-by: selldinesh <dinesh.sellappan@keysight.com> [sdn_tests]: Adding LACP time-out test to pins_ondatra. (sonic-net#12917) [sdn_tests]: Adding Link event damping test to pins_ondatra. (sonic-net#12919) * [sdn_tests]: Adding LACP time-out test to pins_ondatra. * [sdn_tests]: Adding Link event damping test to pins_ondatra. [sdn_tests]: Adding port debug data test to pins_ondatra (sonic-net#12921) * [sdn_tests]: Adding LACP time-out test to pins_ondatra. * [sdn_tests]: Adding Link event damping test to pins_ondatra. * [sdn_tests]: Adding port debug data test to pins_ondatra. [sdn_tests]: Adding module reset test to pins_ondatra. (sonic-net#12928) * [sdn_tests]: Adding LACP time-out test to pins_ondatra. * [sdn_tests]: Adding Link event damping test to pins_ondatra. * [sdn_tests]: Adding port debug data test to pins_ondatra. * [sdn_tests]: Adding module reset test to pins_ondatra. Pin requests to version 2.31.0 (sonic-net#12910) Short-term workaround to avoid issue where testbed-cli.sh ... add-topo errored out with 'Not supported URL scheme http+docker' due to the update of the transitive dependency python requests through docker-py. co-authorized by: jianquanye@microsoft.com Changed all references to 'is_dualtor_aa' to just 'is_dualtor' (sonic-net#12927) [sdn_tests]: Adding Installation test to pins_ondatra. (sonic-net#12943) [sdn_tests]: Adding inband sw interface dual switch test to pins_ondatra. (sonic-net#12946) * [sdn_tests]: Adding Installation test to pins_ondatra. * [sdn_tests]: Adding inband sw interface dual switch test to pins_ondatra. [TACACS] Increase check local log wait time and print debug info. (sonic-net#12822) Increase check local log wait time and print debug info. test_accounting failed because local log does not exist. Increase check local log wait time and print debug info. Pass all test case. Increase check local log wait time and print debug info. [dhcp_server_test] support dhcp server container auto restarted test on mx topo (sonic-net#12897) What is the motivation for this PR? We need to test if dhcp_server container auto restart will work. How did you do it? Enable dhcp_server feature before the test. How did you verify/test it? Run test on mx topo Fix sonic-mgmt testbed-cli interface_to_vms undefined failure. (sonic-net#12890) What is the motivation for this PR? This change is to fix the error above, when VM list is empty. How did you do it? The fix is straightforward - to give it a default value that is an empty list when enumerating. How did you verify/test it? The change is run locally with topology with empty VMs, and it works. Fix add-topo failure when deploying topology with empty VM list. (sonic-net#12932) What is the motivation for this PR? There are 2 issues are hitting, when deploying any topology without the VM list: Ansible Bind topology to VMs task task will fail, because the vm list is empty, but it is checking the VM base in the VM list. The generated minigraph will fail to load, because minigraph.py is not expecting any ACL interface with empty attached to list. How did you do it? This change fixes both issue with adding the empty list checks in these 2 places. How did you verify/test it? Run the add-topo and deploy-mg locally, both works. Any platform specific information? No. [sdn_tests]: Adding GNMI inband sw interface dual switch LoopbackInCnts test. (sonic-net#12951) * [sdn_tests]: Adding Installation test to pins_ondatra. * [sdn_tests]: Adding inband sw interface dual switch test to pins_ondatra. * [sdn_tests]: Adding GNMI inband sw interface dual switch LoopbackInCnts test. [tests/wr_arp] Migrate warm reboot mgmt plane script ferret to py3 (sonic-net#12898) What is the motivation for this PR? The warm reboot test scripts have been migrated to Python 3. The ferret script still runs in Python 2 environment. This PR migrates that to Py3. How did you do it? Run 2to3 Modified script as struct packing handles bytes instead of str. Modified other affected parts to handle bytes directly How did you verify/test it? Ran warm reboot and warm reboot advanced tests on Dell S6100. Tests have passed successfully. [CI/CD] Update elastictest pr test template file, use long term maintained token instead of personal access token (sonic-net#12940) [CI/CD] Update elastictest pr test template file, use long term maintained token instead of personal access token. Compatibility update, in order to ensure that all PRtests that use this template file can transition smoothly. Signed-off-by: Chun'ang Li <chunangli@microsoft.com> Pin requests to version 2.31.0 in main.yml (sonic-net#12959) What is the motivation for this PR? In latest requests python package, would got error in some steps like add-topo and restart-ptf with message 'Not supported URL scheme http+docker' We already have fix PR sonic-net#12910 in master branch to install stable requests version in docker.yml, but once package_installation was set to false like restart-ptf, docker.yml would skip and report error again How did you do it? Install stable requests version in main.yml after executing docker.yml to cover all scenario Enable container autorestart before test_encap_with_mirror_session (sonic-net#12909) Fix qos/test_pfc_pause.py for dualtor-aa (sonic-net#12530) This test was enabled on T0. This test assigns an ip to Eth0 and (sonic-net#12952) checks if MAC resolution happens properly or not. The same functionality is tested by test_neighbor_mac_noptf.py This test fails on T0s because on T0s the Eth0 may b e part of a port channel. As a result the test fails to assign an IP to the Interface. Disablign this test on T0 is the right approach as the same functionality is bening tested on T0 by test_neighbor_mac_noptf.py Support new topology t1-32-lag in acl test (sonic-net#12965) Limit IP addr flushes to global scope IPs (sonic-net#12556) What is the motivation for this PR? Applying the fix in sonic-net#11779 to all tests How did you do it? How did you verify/test it? Include advanced-reboot ptf exception in pytest assertion (sonic-net#12941) What is the motivation for this PR? Surface more information in our test reporting infrastructure to aid in debugging How did you do it? Added more information to a pytest assertion. How did you verify/test it? Artificially added an exception to the try-except block and saw that it was printed out in the pytest assertions. [sdn_tests]: Adding GNMI inband sw interface dual switch LoopbackOutCnts test. (sonic-net#12955) * [sdn_tests]: Adding Installation test to pins_ondatra. * [sdn_tests]: Adding inband sw interface dual switch test to pins_ondatra. * [sdn_tests]: Adding GNMI inband sw interface dual switch LoopbackInCnts test. * [sdn_tests]: Adding GNMI inband sw interface dual switch LoopbackOutCnts test. [Baseline] Fix 1 testplan fail may cancel others issue (sonic-net#12978) What is the motivation for this PR? Baseline test will run a batch of PR tests, and in baseline test, test plan in same platform will run in 1 test job in azure devops. In current logic, once a test plan fails, the test job would catch the sys.exit code and cancel following steps, but we want it only cancel following steps when all test plans fail in the test job How did you do it? Use set -o to continue bash command when fails, get the return code of polling test plan status, if all polling results fail, raise exit code to cancel following steps Add detailed reason for assert failure (sonic-net#12997) What is the motivation for this PR? Add useful information about failed reason. How did you do it? Add the message parameter in pytest_assert How did you verify/test it? Run ssh.test_ssh_limit. [sonic-mgmt]: Skipping pre-commit checks for pins_ondatra patches. (sonic-net#12999) fix: fix bfd static route tests (sonic-net#12995) Add fixes and improvements to bfd/test_bfd_static_route.py. [sdn-tests]: Adding patches to pins_ondatra. (sonic-net#13012) [sdn_tests]:Adding GNMI Get Modes test to pins_ondatra. (sonic-net#12983) [sdn_tests]: Adding inband sw interface test to pins_ondatra. (sonic-net#12984) [sdn_tests]: Adding Software Platform Components Tests to pins_ondatra. (sonic-net#12985)
|
@bingwang-ms PR conflicts with 202405 branch |
mssonicbld
pushed a commit
to mssonicbld/sonic-mgmt
that referenced
this pull request
Jul 18, 2024
|
Cherry-pick PR to 202311: #13712 |
8 tasks
mssonicbld
pushed a commit
that referenced
this pull request
Jul 18, 2024
mssonicbld
added
Included in 202311 branch
and removed
Created PR to 202311 branch
labels
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Description of PR
Summary:
Fixes #11744
test_encap_with_mirror_sessionwould invoke fixturesetup_mirror_session, and it would invoke fixturesetup_uplink. There are steps in fixturesetup_uplink, which could restart teamd docker, it would leadswssandsyncddocker exits.If the autorestart is disabled in
test_pretest.py, thenswssandsyncdwill fail to restart, and then leads to test failure.This PR fixed the issue by adding a fixture to enable container autorestart before the test running.
Type of change
Back port request
Approach
What is the motivation for this PR?
This PR is to enable feature autorestart before
test_encap_with_mirror_session.How did you do it?
This PR fixed the issue by adding a fixture to enable container autorestart before the test running.
How did you verify/test it?
The change is verified by running
test_encap_with_mirror_sessionaftertest_disable_container_autorestart.Any platform specific information?
No.
Supported testbed topology if it's a new test case?
Not a new test case.
Documentation