Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Update container_checker for multi-asic devices when state is 'always_enabled' #10067

Merged
merged 2 commits into from
Feb 24, 2022
Merged

Conversation

wenyiz2021
Copy link
Contributor

@wenyiz2021 wenyiz2021 commented Feb 23, 2022

Why I did it

This PR of finding running containers via feature table does not consider 'database' containers as feature in multi-asic devices: #7474

How I did it

Change in this PR is to add database containers for multi-asic, to always_running_containers, as their status is 'always_enabled'

How to verify it

Before this change on a multi-asic device, container_status says database containers are not running even if they are:
Program 'container_checker'
status Status ok
monitoring status Monitored
monitoring mode active
on reboot start
last exit value 3
last output Expected containers not running: database0, database5, database1, database2, database, database4, database3
data collected Wed, 09 Feb 2022 19:15:35

After change, result of 'monit status':
Program 'container_checker'
status Status ok
monitoring status Monitored
monitoring mode active
on reboot start
last exit value 0
last output -
data collected Wed, 23 Feb 2022 02:31:39

Which release branch to backport (provide reason below if selected)

  • 201811
  • 201911
  • 202006
  • 202012
  • 202106
  • 202111

Description for the changelog

Link to config_db schema for YANG module changes

A picture of a cute animal (not mandatory but encouraged)

Update container_checker for multi-asic devices to add database containers in always_running_containers. 
Previous change was made for single-asic, and that database containers were not considered as feature when writing to state_db.
Update an indent
@wenyiz2021 wenyiz2021 requested a review from lguohan as a code owner February 23, 2022 02:30
@wenyiz2021 wenyiz2021 requested review from renukamanavalan and arlakshm and removed request for lguohan February 23, 2022 02:30
else:
expected_running_containers.add(container_name)

if feature_table[container_name]["state"] == 'always_enabled':
if multi_asic.is_multi_asic():
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

has_global_scope -- Is this true only for Database container?

Copy link
Contributor Author

@wenyiz2021 wenyiz2021 Feb 23, 2022

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

has_global_scope is True for other containers e.g. lldp, dhcp_relay, mux, etc
but here the if statement is under "if feature_table[container_name]["state"] == 'always_enabled'", which is only True for mux and databse, while mux doesn't seem to be a container as shown in 'docker ps -a'

Feature State AutoRestart SystemState UpdateTime ContainerId Version SetOwner CurrentOwner RemoteState


acms enabled enabled up 2022-02-22 19:20:47 acms local local none
bgp enabled enabled local
database always_enabled always_enabled local
dhcp_relay enabled enabled up 2022-02-22 19:21:10 dhcp_relay 20201231.54 kube local
lldp enabled enabled up 2022-02-22 19:20:53 lldp 20201231.54 kube local
mux always_disabled enabled local
pmon enabled enabled up 2022-02-22 19:20:48 pmon 20201231.54 kube local
radv enabled enabled up 2022-02-22 19:21:06 radv 20201231.54 kube local
restapi enabled enabled up 2022-02-22 19:20:47 restapi local local none
snmp enabled enabled up 2022-02-22 19:23:56 snmp 20201231.54 kube local
swss enabled enabled local
syncd enabled enabled local
teamd enabled enabled local
telemetry enabled enabled up 2022-02-22 19:23:59 telemetry 20201231.54 kube local

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The previous code on line 54 and 55 will add container with 'state' as 'always_enabled' to always_running_containers only if single-asic. I moved that condition one upper level, and let it decide separately for multi-asic and single-asic.

if feature_table[container_name]["has_per_asic_scope"] == "True":
num_asics = multi_asic.get_num_asics()
for asic_id in range(num_asics):
always_running_containers.add(container_name + str(asic_id))
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Can you please verify that indeed the container is running?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Hi Renuka,
f2a9c49b7857 docker-database:latest "/usr/local/bin/dock…" 23 hours ago Up 21 hours database4
bca510cdfd1c docker-database:latest "/usr/local/bin/dock…" 23 hours ago Up 21 hours database5
f109ac36ee86 docker-database:latest "/usr/local/bin/dock…" 23 hours ago Up 21 hours database1
8ba15862639d docker-database:latest "/usr/local/bin/dock…" 23 hours ago Up 21 hours database2
c669bf6e465f docker-database:latest "/usr/local/bin/dock…" 23 hours ago Up 21 hours database3
95801b05ada1 docker-database:latest "/usr/local/bin/dock…" 23 hours ago Up 21 hours database0
df566813a493 docker-database:latest "/usr/local/bin/dock…" 23 hours ago Up 21 hours database

Copy link
Contributor

@arlakshm arlakshm left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

lgtm

@wenyiz2021 wenyiz2021 merged commit 2d0b063 into sonic-net:master Feb 24, 2022
@arlakshm arlakshm added Request for 202106 Branch Request for 202111 Branch For PRs being requested for 202111 branch labels Mar 4, 2022
arlakshm pushed a commit that referenced this pull request Mar 4, 2022
…_enabled' (#10067)

* Update container_checker for multi-asic devices 

Update container_checker for multi-asic devices to add database containers in always_running_containers. 
Previous change was made for single-asic, and that database containers were not considered as feature when writing to state_db.

* Update container_checker

Update an indent
judyjoseph pushed a commit that referenced this pull request Mar 7, 2022
…_enabled' (#10067)

* Update container_checker for multi-asic devices 

Update container_checker for multi-asic devices to add database containers in always_running_containers. 
Previous change was made for single-asic, and that database containers were not considered as feature when writing to state_db.

* Update container_checker

Update an indent
qiluo-msft pushed a commit that referenced this pull request Mar 14, 2022
…_enabled' (#10067)

* Update container_checker for multi-asic devices 

Update container_checker for multi-asic devices to add database containers in always_running_containers. 
Previous change was made for single-asic, and that database containers were not considered as feature when writing to state_db.

* Update container_checker

Update an indent
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging this pull request may close these issues.

5 participants