Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[202205][Nokia][pmon[ledd] PMON ledd FATAL on Nokia_IXR7250E platform after PR18609 which cherry-pick PR10960 #18742

Closed
mlok-nokia opened this issue Apr 22, 2024 · 1 comment

Comments

@mlok-nokia
Copy link
Contributor

Description

On Nokia Platform Nokia_IXR7250E, ledd show as FATAL on PMON after PR #18609 which cherry-pick PR #10960.

admin@ixre-egl-board25:~$ docker exec -it pmon bash
root@ixre-egl-board25:/# supervisorctl status
chassis_db_init                  EXITED    Apr 22 07:46 AM
chassisd                         RUNNING   pid 21, uptime 10:45:32
dependent-startup                EXITED    Apr 22 07:46 AM
ledd                             FATAL     Exited too quickly (process log may have details)
pcied                            RUNNING   pid 26, uptime 10:45:32
rsyslogd                         RUNNING   pid 16, uptime 10:45:34
supervisor-proc-exit-listener    RUNNING   pid 15, uptime 10:45:35
syseepromd                       RUNNING   pid 24, uptime 10:45:32
thermalctld                      RUNNING   pid 25, uptime 10:45:32
xcvrd                            RUNNING   pid 23, uptime 10:45:32
root@ixre-egl-board25:/# 

Also, the following error message are shown in syslog

Apr 22 07:41:28.169278 ixre-egl-board25 ERR pmon#led-mgmt: :- validateNamespace: Initialize global DB config using API SonicDBConfig::initializeGlobalConfig
Apr 22 07:41:28.169596 ixre-egl-board25 ERR pmon#led-mgmt: Failed to load ledutil: :- validateNamespace: Initialize global DB config using API SonicDBConfig::initializeGlobalConfig
Apr 22 07:41:28.169868 ixre-egl-board25 INFO pmon#supervisord: ledd Failed to load ledutil: :- validateNamespace: Initialize global DB config using API SonicDBConfig::initializeGlobalConfig

Steps to reproduce the issue:

  1. Running the latest 202205 image on Nokia Platform Nokia_IXR7250E
  2. Log into the PMON docker, and execute the 'supervisorctl status'. The following output will be shown
admin@ixre-egl-board25:~$ docker exec -it pmon bash
root@ixre-egl-board25:/# supervisorctl status
chassis_db_init                  EXITED    Apr 22 07:46 AM
chassisd                         RUNNING   pid 21, uptime 10:45:32
dependent-startup                EXITED    Apr 22 07:46 AM
ledd                             FATAL     Exited too quickly (process log may have details)
pcied                            RUNNING   pid 26, uptime 10:45:32
rsyslogd                         RUNNING   pid 16, uptime 10:45:34
supervisor-proc-exit-listener    RUNNING   pid 15, uptime 10:45:35
syseepromd                       RUNNING   pid 24, uptime 10:45:32
thermalctld                      RUNNING   pid 25, uptime 10:45:32
xcvrd                            RUNNING   pid 23, uptime 10:45:32
root@ixre-egl-board25:/# 

Describe the results you received:

The ledd process is shown as FATAL.

admin@ixre-egl-board25:~$ docker exec -it pmon bash
root@ixre-egl-board25:/# supervisorctl status
chassis_db_init                  EXITED    Apr 22 07:46 AM
chassisd                         RUNNING   pid 21, uptime 10:45:32
dependent-startup                EXITED    Apr 22 07:46 AM
ledd                             FATAL     Exited too quickly (process log may have details)
pcied                            RUNNING   pid 26, uptime 10:45:32
rsyslogd                         RUNNING   pid 16, uptime 10:45:34
supervisor-proc-exit-listener    RUNNING   pid 15, uptime 10:45:35
syseepromd                       RUNNING   pid 24, uptime 10:45:32
thermalctld                      RUNNING   pid 25, uptime 10:45:32
xcvrd                            RUNNING   pid 23, uptime 10:45:32

Describe the results you expected:

The output should be shown as below. Ledd should be in running state.

admin@ixre-egl-board7:~$ docker exec -it pmon bash
root@ixre-egl-board7:/# supervisorctl status
chassis_db_init                  EXITED    Apr 22 06:27 PM
chassisd                         RUNNING   pid 26, uptime 0:13:22
dependent-startup                EXITED    Apr 22 06:27 PM
ledd                             RUNNING   pid 27, uptime 0:13:22
pcied                            RUNNING   pid 31, uptime 0:13:22
rsyslogd                         RUNNING   pid 21, uptime 0:13:24
supervisor-proc-exit-listener    RUNNING   pid 18, uptime 0:13:26
syseepromd                       RUNNING   pid 29, uptime 0:13:22
thermalctld                      RUNNING   pid 30, uptime 0:13:22
xcvrd                            RUNNING   pid 28, uptime 0:13:22
root@ixre-egl-board7:/# 

Output of show version:

The latest 202205 branch.

(paste your output here)

Output of show techsupport:

(paste your output here or download and attach the file here )

Additional information you deem important (e.g. issue happens only occasionally):

@mlok-nokia
Copy link
Contributor Author

This is closed and wait for further discussion and decision.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

1 participant