[teamd][warmreboot] teamd fails to restore PortChannel state sporadically after warm reboot

[stepanblyschak]

teamd.txt

Description

Steps to reproduce the issue:

1. sudo warm-reboot -v
2. repeat until use see issue with LAG flapping
3. zgrep ERR teamd.log

Part of teamd log

Oct 22 14:04:03.827944 arc-switch1004 ERR teamd#teamd_PortChannel0002[36]: Ethernet116: Failed to find "enabled" option.
Oct 22 14:04:03.827984 arc-switch1004 ERR teamd#teamd_PortChannel0002[36]: get_ifinfo_list: check_call_change_handers failed
Oct 22 14:04:03.828018 arc-switch1004 ERR teamd#teamd_PortChannel0002[36]: Failed to get interface information list.
Oct 22 14:04:03.828056 arc-switch1004 ERR teamd#teamd_PortChannel0002[36]: Failed to refresh interface information list.
Oct 22 14:04:03.828096 arc-switch1004 ERR teamd#teamd_PortChannel0002[36]: Ethernet116: Team refresh failed.
Oct 22 14:04:03.829959 arc-switch1004 ERR teamd#teamd_PortChannel0002[36]: Failed to init port priv.
Oct 22 14:04:03.833389 arc-switch1004 ERR teamd#teamd_PortChannel0002[36]: ioctl SIOCDELMULTI failed.
Oct 22 14:04:03.839081 arc-switch1004 ERR teamd#teamd_PortChannel0002[36]: Port with interface index "76" is not part of this device.

Describe the results you received:

LAG flaps sometimes after warm reboot

Describe the results you expected:

teamd restores LAG

Additional information you deem important (e.g. issue happens only occasionally):

**Output of `show version`:**

```

SONiC Software Version: SONiC.HEAD.106-6cb445cb
Distribution: Debian 9.11
Kernel: 4.9.0-9-2-amd64
Build commit: 6cb445c
Build date: Mon Oct 21 08:11:44 UTC 2019
Built by: johnar@jenkins-worker-4

Platform: x86_64-mlnx_msn2700-r0
HwSKU: ACS-MSN2700
ASIC: mellanox
Serial Number: MT1822K07823
Uptime: 15:32:25 up 11 min, 1 user, load average: 3.37, 3.45, 2.38

```

**Attach debug file `sudo generate_dump`:**

```
(paste your output here)
```

[pavel-shirshov]

Please check #3725

[pavel-shirshov]

Please check #3753
I still can't reproduce the issue with/or without the latest change

[stephenxs]

Hi @pavel-shirshov,
We found that this issue can also take place after a cold reboot. When it takes place after a cold reboot, the error message is the same.