[crm]: Implement test for CRM #473

volodymyrsamotiy · 2018-02-19T18:35:36Z

Signed-off-by: Volodymyr Samotiy volodymyrs@mellanox.com

Description of PR

Implementation of new ansible test for CRM

Type of change

New test

Approach

Implementation of new ansible test for CRM

Documentation

https://github.com/Azure/SONiC/wiki/CRM-test-plan

Signed-off-by: Volodymyr Samotiy <volodymyrs@mellanox.com>

prsunny · 2018-02-22T00:42:28Z

ansible/roles/test/tasks/crm/crm_test_ipv4_route.yml

+    - set_fact: crm_stats_ipv4_route_available={{out.stdout}}
+
+    - name: Add IPv4 route 
+      command: ip route add 2.2.2.0/24 via 10.0.0.1


This command would need sudo privilege. Can you add become: yes here? Need to do this all the other places where you execute a command in DUT

prsunny · 2018-02-22T00:43:12Z

ansible/roles/test/vars/testcases.yml

+
+    crm:
+      filename: crm.yml
+      topologies: [t1]


Can you add this for t0, t1-lag as well?

If it takes more time to extend the test for t0, we can plan this in a future commit.

prsunny · 2018-02-23T03:12:49Z

ansible/roles/test/tasks/crm/crm_test_acl_counter.yml

+      command: acl-loader update full /tmp/acl.json
+
+    - name: Get ACL table key
+      command: bash -c "docker exec -i database redis-cli --raw -n 2 KEYS *CRM:ACL_TABLE_STATS* | head -1"


This is fetching the first table entry and checking for values. But if there are multiple tables, and if the acl-loader is updating entries to a table other than the first entry, then this test fails.

prsunny · 2018-02-23T03:15:41Z

ansible/roles/test/tasks/crm/crm_test_fdb_entry.yml

+      command: docker exec -i swss swssconfig /fdb.json
+
+    - name: Add VLAN required for FDB entry
+      command: config vlan add 2


This require root privileges. Please add become: yes

prsunny · 2018-02-23T03:16:15Z

ansible/roles/test/tasks/crm/crm_test_fdb_entry.yml

+      command: config vlan add 2
+
+    - name: Add VLAN member required for FDB entry
+      command: config vlan member add 2 Ethernet0


Same here and other places in file where "config vlan" is invoked.

prsunny · 2018-02-23T04:03:26Z

ansible/roles/test/tasks/crm/crm_test_acl_counter.yml

+      copy: src=roles/test/tasks/crm/acl.json dest=/tmp
+
+    - name: Add ACL
+      command: acl-loader update full /tmp/acl.json


Just to confirm, @lguohan, is DATAACL table always present in the DUTs or is there any case where it may be missing. If so, the acl-loader will fail since it adds the entry to DATAACL

prsunny

Please check the review comments!

* Add support of t0 and t1-lag topologies * Add option to run commands with sudo privileges * Fix problem with fetching ACL table ID Signed-off-by: Volodymyr Samotiy <volodymyrs@mellanox.com>

volodymyrsamotiy · 2018-03-05T15:39:40Z

Fixed review comments

prsunny · 2018-03-05T20:24:26Z

ansible/roles/test/tasks/crm/crm_test_fdb_entry.yml


    - name: Remove VLAN required for FDB entry
      command: config vlan del 2
+      become: yes

    - name: Remove FDB JSON config from switch.


How are we deleting the FDB entry that got added to the switch?

* Update FDB entry test case Signed-off-by: Volodymyr Samotiy <volodymyrs@mellanox.com>

volodymyrsamotiy · 2018-03-06T12:40:50Z

Updated FDB test case.

prsunny · 2018-03-07T06:09:26Z

ansible/roles/test/tasks/crm/crm_test_fdb_entry.yml

+      become: yes
+
+    - name: Remove FDB entry
+      command: fdbclear


This is good. But I think, we must clear only for Vlan 2 else it might clear any other leaned MAC address not configured as part of this test.

I agree, but the problem is that currently such functionality is not supported.
So, for now the only way to clear FDB entry is to call "fdbclear".

# fdbclear -p Ethernet0 command not supported yet. # fdbclear -p Ethernet0 -v 2 command not supported yet.

Ok. We can fix this in future commit.

* msft_github/master: (111 commits) add disconnect/connect vm to testbed-cli.sh (sonic-net#566) [dhcp_relay] Increase sleep duration to allow LAG and BGP to come up (sonic-net#565) [pfcwd]: support t0-116 and clean up the code (sonic-net#563) [fanout]: remove vrf management in arista fanout deploy templates (sonic-net#562) [fanout-switch-deploy] Support multiple speeds and port breakout (sonic-net#561) [extract_log] Improve extract_log script (sonic-net#559) Disable upgrade_sonic retry (sonic-net#560) Support for Sonic fanout (sonic-net#555) [Fanout deploy template] enable root user on fanout switches (sonic-net#557) [link state] match exact dut name in the link list (sonic-net#556) [pfcwd]: cache the ansible facts (sonic-net#554) Fix kernel version check (sonic-net#553) [pfcwd]: increase the pause waiting time and ingore snmp errors (sonic-net#551) fix typo (sonic-net#552) [dir_bcast] enable dir_bcast test on t0-116 topology (sonic-net#550) use command to gather host distribution, kernel version facts (sonic-net#549) [minigraph templte] use consistent VLAN subnet (sonic-net#546) [VM config] skip podset 0 tor 0 routing entry (sonic-net#545) Remove job minigraph_facts from boot_onie (sonic-net#548) [fast-reboot] pass VM IP in as ASCII strings (sonic-net#547) [fast-reboot test] fix syslog reading issue (sonic-net#543) add dataacl to minigraph template (sonic-net#544) [service_acl] Make test reliable when testing Arista service ACL solution (sonic-net#542) [pfcwd]: Iterate functional test over all ports (sonic-net#490) [service_acl] Detect expected output message even if it is followed by other text (sonic-net#541) [testbed]: Remove connection local for port_alias module (sonic-net#540) Revert "[minigraph-gen] fix AclInterface entries in minigraph (sonic-net#538)" (sonic-net#539) [minigraph-gen] fix AclInterface entries in minigraph (sonic-net#538) add retries in onie installation (sonic-net#537) Use connection plugin to install sonic image in ONIE. (sonic-net#536) [dhcp_relay]: Add --relax flag to ptf command (sonic-net#535) [minigraph_facts] use minigraph on DUT (sonic-net#534) Fix minigraph_facts: mkdir recursively (sonic-net#533) [minigraph_facts] use mingraph on DUT to test (sonic-net#532) generate minigraph based on topology file (sonic-net#531) Need to double-escape when using 'args' syntax (sonic-net#529) Fix improper 'local_action' syntax (sonic-net#528) Unify style of 'wait_for' actions across playbooks (sonic-net#527) [minigraph_facts] retrieving dhcp server list from vlan configuration instead of DhcpResources (sonic-net#526) [snmp_facts] increase get command timeout to fix cpu test failure (sonic-net#525) [everflow_test]: Add copy ptftests folder to use the remote.py file (sonic-net#522) Fix snmp_facts on PSU oid (sonic-net#520) Fix snmp queue test (sonic-net#519) [lag_test]: Remove the unnecessary testbed_type check (sonic-net#518) [ip_decap_test]: Support t0-64 topology (sonic-net#517) [acl_test]: Copy ptftests folder for the remote.py file (sonic-net#516) Add test case for PSU (sonic-net#514) [mtu]: Add t1-64-lag topology support for MTU test (sonic-net#513) [ip_decap]: Add t1-64-lag support in the script for the list of source port (sonic-net#512) [everflow]: Add missing spaces in ptf command (sonic-net#511) [acl_test]: Add ptf_platform_dir: ptftests to use customized platform code to support 64 ports (sonic-net#510) [crm]: Implement test for CRM (sonic-net#473) [everflow]: Add support for t1-64-lag topology (sonic-net#502) Fix sonic_image_version: get from sonic_version.yml, no dependency on grub (sonic-net#508) [topology]: Update t1-64-lag topology template to add AclInterfaces piece (sonic-net#505) Pull syncd-rpc with sonic version tag (sonic-net#507) Remove leading and trailing whitespaces when reading veos file (sonic-net#506) [lag_2] remove hard coded interval_count so it can be set by test (sonic-net#503) ptf_runner: Add one line comment for ptf_platform_dir (sonic-net#501) [testbed]: add port speed and fec configuration in sonic fanout (sonic-net#498) [typo]: Replace string t1-lag-64 with t1-64-lag (sonic-net#499) Adding sensor data for S6100 (sonic-net#496) [fib_test]: Add t1-64-lag src_ports in FIB test (sonic-net#497) Fix typo in acl test case name (sonic-net#494) Add one more Mellanox SKU string in everflow_tb_test script (sonic-net#495) Adding sensor data for Z9100 (sonic-net#492) [service_acl] Make test more robust and efficient (sonic-net#489) [dhcp relay test] adding more test scenarios (sonic-net#440) fix sanity check failed to recover (sonic-net#488) Fix PFC_WD test (sonic-net#479) add sonic fanout support (sonic-net#485) Update README.test.md Update README.test.md Fix table caption in testbed.csv and documentation (sonic-net#482) [test case] Add test: restart swss service (sonic-net#483) Add test case port toggle (sonic-net#484) [test infrastructure] allow overriding recover system actioin (sonic-net#480) [SNMP]add new SNMP counters tests to snmp.yml (sonic-net#477) add t0-52 topology (sonic-net#476) add command line option for creategraph.py (sonic-net#475) Add support for additional timestamp format (sonic-net#474) Ignore ansible output in extract_log (sonic-net#472) Add extract_logs action to concatenate logs after log rotate (sonic-net#471) Add ACL ICMP test (sonic-net#2) (sonic-net#465) [pfcwd]:add docker exec to avoid tty error (sonic-net#470) [test_tag]remove pfc_wd test tag from test by tag main yaml (sonic-net#469) [ansible]gather fact by default (sonic-net#468) [sensors]fix Dell S6000 sensors test fail (sonic-net#462) [vlan]improve test to wait a bit longer for config reload (sonic-net#463) one more place for hwsku of new Mellanox 2700 (sonic-net#464) [sensors]sensors test add new hwsku for Mellanox 2700 (sonic-net#461) [PFCWD]: test enhancement (sonic-net#456) [sensors] remove redundant sensor data definitin for sku MSN2100 (sonic-net#460) when call testcase by name, fetch all vms management info from testbed_facts (sonic-net#457) [acl test] error in ACL rule json file for destination ip (sonic-net#434) [sensor data] add sensor data for Mellanox MSN2410 and MSN2100 (sonic-net#449) Fix sku-sensors-data for 7050-QX32 (sonic-net#454) [deploy minigraph]add default enable BGP to deploy minigraph step (sonic-net#455) [upgrade]save bgp UP state after they are brough up (sonic-net#453) [reboot test] call sudo reboot to reboot dut (sonic-net#452) ...

[crm]: Implement test for CRM

1e07522

Signed-off-by: Volodymyr Samotiy <volodymyrs@mellanox.com>

prsunny reviewed Feb 22, 2018

View reviewed changes

prsunny reviewed Feb 23, 2018

View reviewed changes

prsunny requested changes Feb 28, 2018

View reviewed changes

[crm]: Implement test for CRM

545631e

* Add support of t0 and t1-lag topologies * Add option to run commands with sudo privileges * Fix problem with fetching ACL table ID Signed-off-by: Volodymyr Samotiy <volodymyrs@mellanox.com>

prsunny reviewed Mar 5, 2018

View reviewed changes

[crm]: Implement test for CRM

51f940a

* Update FDB entry test case Signed-off-by: Volodymyr Samotiy <volodymyrs@mellanox.com>

prsunny reviewed Mar 7, 2018

View reviewed changes

prsunny approved these changes Mar 7, 2018

View reviewed changes

lguohan merged commit d53e060 into sonic-net:master Mar 14, 2018

stcheng added the Enhancement label Mar 15, 2018

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

[crm]: Implement test for CRM #473

[crm]: Implement test for CRM #473

volodymyrsamotiy commented Feb 19, 2018

prsunny Feb 22, 2018

prsunny Feb 22, 2018

prsunny Feb 28, 2018

prsunny Feb 23, 2018

prsunny Feb 23, 2018

prsunny Feb 23, 2018

prsunny Feb 23, 2018

prsunny left a comment

volodymyrsamotiy commented Mar 5, 2018

prsunny Mar 5, 2018

volodymyrsamotiy commented Mar 6, 2018

prsunny Mar 7, 2018

volodymyrsamotiy Mar 7, 2018

prsunny Mar 7, 2018

[crm]: Implement test for CRM #473

[crm]: Implement test for CRM #473

Conversation

volodymyrsamotiy commented Feb 19, 2018

Description of PR

Type of change

Approach

Documentation

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

prsunny left a comment

Choose a reason for hiding this comment

volodymyrsamotiy commented Mar 5, 2018

Choose a reason for hiding this comment

volodymyrsamotiy commented Mar 6, 2018

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment