Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

lldp_syncd: continuous complaints of Failed to get system capabilities #5912

Closed
yxieca opened this issue Nov 12, 2020 · 3 comments · Fixed by sonic-net/sonic-dbsyncd#30
Closed
Assignees

Comments

@yxieca
Copy link
Contributor

yxieca commented Nov 12, 2020

Description

Steps to reproduce the issue:

  1. run nightly test on arista 7050cx3 platform

Describe the results you received:
Throughout the whole test period, lldp_syncd has been complaining with following ERROR level message and caused various testcases to fail due to loganalyzer caught this complaint.

syslog.58.gz:Nov 12 08:13:48.506123 str2-7050cx3-acs-06 ERR lldp#lldp-syncd message repeated 3 times: [ [lldp_syncd] ERROR: Failed to get system capabilities]
syslog.58.gz:Nov 12 08:13:53.185800 str2-7050cx3-acs-06 ERR lldp#lldp-syncd [lldp_syncd] ERROR: Failed to get system capabilities
syslog.30.gz:Nov 12 08:48:45.298817 str2-7050cx3-acs-06 ERR lldp#lldp-syncd message repeated 207 times: [ [lldp_syncd] ERROR: Failed to get system capabilities]
syslog.30.gz:Nov 12 08:49:33.204235 str2-7050cx3-acs-06 ERR lldp#lldp-syncd [lldp_syncd] ERROR: Failed to get system capabilities
syslog.30.gz:Nov 12 08:51:44.095083 str2-7050cx3-acs-06 ERR lldp#lldp-syncd message repeated 12 times: [ [lldp_syncd] ERROR: Failed to get system capabilities]
syslog.30.gz:Nov 12 08:52:04.194876 str2-7050cx3-acs-06 ERR lldp#lldp-syncd [lldp_syncd] ERROR: Failed to get system capabilities
syslog.30.gz:Nov 12 08:52:34.582895 str2-7050cx3-acs-06 ERR lldp#lldp-syncd message repeated 3 times: [ [lldp_syncd] ERROR: Failed to get system capabilities]
syslog.30.gz:Nov 12 08:53:22.892214 str2-7050cx3-acs-06 ERR lldp#lldp-syncd [lldp_syncd] ERROR: Failed to get system capabilities
syslog.6.gz:Nov 12 09:52:37.383686 str2-7050cx3-acs-06 ERR lldp#lldp-syncd message repeated 353 times: [ [lldp_syncd] ERROR: Failed to get system capabilities]
syslog.6.gz:Nov 12 09:52:44.960146 str2-7050cx3-acs-06 ERR lldp#lldp-syncd [lldp_syncd] ERROR: Failed to get system capabilities
syslog.6.gz:Nov 12 09:52:55.013824 str2-7050cx3-acs-06 ERR lldp#lldp-syncd [lldp_syncd] ERROR: Failed to get system capabilities
syslog.6.gz:Nov 12 09:55:12.202408 str2-7050cx3-acs-06 ERR lldp#lldp-syncd message repeated 13 times: [ [lldp_syncd] ERROR: Failed to get system capabilities]
syslog.6.gz:Nov 12 09:55:15.792351 str2-7050cx3-acs-06 ERR lldp#lldp-syncd [lldp_syncd] ERROR: Failed to get system capabilities
syslog.3.gz:Nov 12 09:57:16.158855 str2-7050cx3-acs-06 ERR lldp#lldp-syncd message repeated 11 times: [ [lldp_syncd] ERROR: Failed to get system capabilities]
syslog.3.gz:Nov 12 09:58:05.243887 str2-7050cx3-acs-06 ERR lldp#lldp-syncd [lldp_syncd] ERROR: Failed to get system capabilities
syslog.3.gz:Nov 12 09:58:15.824529 str2-7050cx3-acs-06 ERR lldp#lldp-syncd [lldp_syncd] ERROR: Failed to get system capabilities
syslog.3.gz:Nov 12 09:58:25.876357 str2-7050cx3-acs-06 ERR lldp#lldp-syncd [lldp_syncd] ERROR: Failed to get system capabilities
syslog:Nov 12 10:09:14.192442 str2-7050cx3-acs-06 ERR lldp#lldp-syncd [lldp_syncd] ERROR: Failed to get system capabilities
syslog:Nov 12 10:09:24.230264 str2-7050cx3-acs-06 ERR lldp#lldp-syncd [lldp_syncd] ERROR: Failed to get system capabilities

Describe the results you expected:

Additional information you deem important (e.g. issue happens only occasionally):

**Output of `show version`:**

```

SONiC Software Version: SONiC.master.481-b5121dcf
Distribution: Debian 10.6
Kernel: 4.19.0-9-2-amd64
Build commit: b5121dc
Build date: Tue Nov 10 10:27:57 UTC 2020
Built by: johnar@jenkins-worker-8

Platform: x86_64-arista_7050cx3_32s
HwSKU: Arista-7050CX3-32S-C32
ASIC: broadcom
Serial Number: JPW20281061
Uptime: 17:02:17 up 6:48, 1 user, load average: 1.19, 1.40, 1.44 ```

**Attach debug file `sudo generate_dump`:**

```
(paste your output here)
```
@yxieca
Copy link
Contributor Author

yxieca commented Nov 12, 2020

Further log analysis shows that this error message was associated with either dut reboot or config reload.

@lguohan
Copy link
Collaborator

lguohan commented Nov 12, 2020

debug a little bit. seems like some neighbor device on eth0 send invalid lldp messages.

Interface:    eth0, via: LLDP, RID: 1, Time: 0 day, 05:33:06
  Chassis:     
    ChassisID:    mac 50:2f:a8:a5:03:d5
  Port:        
    PortID:       ifname Ethernet1/14
    TTL:          120
  Unknown TLVs:
    TLV:          OUI: 00,01,42, SubType: 1, Len: 1 01
    TLV:          OUI: 00,01,42, SubType: 2, Len: 16 05,DC,05,DC,05,DC,05,DC,05,DC,05,DC,05,DC,05,DC

@lguohan
Copy link
Collaborator

lguohan commented Nov 12, 2020

tcpdump

admin@str2-7050cx3-acs-03:~$ sudo tcpdump -vvvni eth0 -Qin ether proto 0x88cc 
tcpdump: listening on eth0, link-type EN10MB (Ethernet), capture size 262144 bytes
18:48:33.118967 LLDP, length 59
        Chassis ID TLV (1), length 7
          Subtype MAC address (4): 50:2f:a8:a5:03:d5
          0x0000:  0450 2fa8 a503 d5
        Port ID TLV (2), length 13
          Subtype Interface Name (5): Ethernet1/14
          0x0000:  0545 7468 6572 6e65 7431 2f31 34
        Time to Live TLV (3), length 2: TTL 120s
          0x0000:  0078
        Organization specific TLV (127), length 5: OUI Unknown (0x000142)
          0x0000:  0001 4201 01
        Organization specific TLV (127), length 20: OUI Unknown (0x000142)
          0x0000:  0001 4202 05dc 05dc 05dc 05dc 05dc 05dc
          0x0010:  05dc 05dc
        End TLV (0), length 0

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
2 participants