Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[ceos]: check ceos testbed health via snmp and restart ceos if necessary #2855

Merged
merged 4 commits into from
Jan 26, 2021

Conversation

lguohan
Copy link
Contributor

@lguohan lguohan commented Jan 24, 2021

Description of PR

Summary:
Fixes # (issue)

Type of change

  • Bug fix
  • Testbed and Framework(new/improvement)
  • Test case(new/improvement)

Approach

What is the motivation for this PR?

there are quite a few testbed failure causing kvm test fails. I login into those ceos devices and found management ip disappear although the configuration is good. restart the ceos container make the nbr healthy. as a mitigation efforts, this pr check the nbr health via snmp and restart the ceos testbed if neccessary.

https://dev.azure.com/sonicswitch/be1b070f-be15-4154-aade-b1d3bfb17054/_apis/build/builds/1811/logs/39

How did you do it?

  • check snmp facts for refresh-dut, restart ceos container if snmp facts is not avaiable.
  • also consolidate refresh-dut and add_topo playbook since they are essentially the same.

How did you verify/test it?

  • run refresh-dut without ceos error
  • run refresh-dut with ceos error
  • run remove-topo
  • run add-topo

Any platform specific information?

Supported testbed topology if it's a new test case?

Documentation

refresh-dut and add_vm_topology are essentially the same.
consolidate the playbook

Signed-off-by: Guohan Lu <lguohan@gmail.com>
check if ceos container responds to snmp query, force
restart ceos container if it does not.

Signed-off-by: Guohan Lu <lguohan@gmail.com>
@lguohan lguohan requested a review from a team January 24, 2021 20:35
Signed-off-by: Guohan Lu <lguohan@gmail.com>
check for both ceos original and derived image.
no need to download the ceos image if any of them
exists

Signed-off-by: Guohan Lu <lguohan@gmail.com>
@lguohan lguohan merged commit 1e12790 into sonic-net:master Jan 26, 2021
@lguohan lguohan deleted the ceos_health branch January 26, 2021 05:13
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants