-
Notifications
You must be signed in to change notification settings - Fork 1.4k
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
[Debian9-Z9100] config save is hanging indefinitely #1566
Comments
taoyl will introduce timeout and then you can see if you can repro. |
Verification Logs: root@sonic:~# show ver Docker images: |
fff4051 Fixing serial number read to get from DB if it is populated (sonic-net#1580) be974bf [neighbor_advertiser] Use existing tunnel if present for creating tunnel mappings (sonic-net#1589) 9492eab Use swsscommon instead of swsssdk (sonic-net#1510) 0f4988b Add pg-drop script to sonic filesystem (sonic-net#1583) cbe2159 [vnet] Add "vnet_route_check" script (sonic-net#1300) 9120766 Relax the install_requires, no need to exact version as long as there are no broken changes with future versions (sonic-net#1530) 2e09b22 Handle the new db version which mellanox_buffer_migrator isn't interested (sonic-net#1566)
* Add pg-drop script to sonic filesystem (sonic-net#1583) * Fixing serial number read to get from DB if it is populated (sonic-net#1580) * Handle the new db version which mellanox_buffer_migrator isn't interested (sonic-net#1566) * Relax the install_requires, no need to exact version as long as there are no broken changes with future versions (sonic-net#1530) (sonic-net#1592) Signed-off-by: Danny Allen <daall@microsoft.com>
68ea9efc Add pg-drop script to sonic filesystem (sonic-net#1583) b216bf0a Fixing serial number read to get from DB if it is populated (sonic-net#1580) fa7230c6 Handle the new db version which mellanox_buffer_migrator isn't interested (sonic-net#1566) Signed-off-by: Stephen Sun <stephens@nvidia.com>
9a88cb6 [sonic_installer] dont fail package migration (sonic-net#1591) 615e531 [show][config] Add new snmp commands (sonic-net#1347) fff4051 Fixing serial number read to get from DB if it is populated (sonic-net#1580) be974bf [neighbor_advertiser] Use existing tunnel if present for creating tunnel mappings (sonic-net#1589) 9492eab Use swsscommon instead of swsssdk (sonic-net#1510) 0f4988b Add pg-drop script to sonic filesystem (sonic-net#1583) cbe2159 [vnet] Add "vnet_route_check" script (sonic-net#1300) 9120766 Relax the install_requires, no need to exact version as long as there are no broken changes with future versions (sonic-net#1530) 2e09b22 Handle the new db version which mellanox_buffer_migrator isn't interested (sonic-net#1566)
9a88cb6 [sonic_installer] dont fail package migration (#1591) 615e531 [show][config] Add new snmp commands (#1347) fff4051 Fixing serial number read to get from DB if it is populated (#1580) be974bf [neighbor_advertiser] Use existing tunnel if present for creating tunnel mappings (#1589) 9492eab Use swsscommon instead of swsssdk (#1510) 0f4988b Add pg-drop script to sonic filesystem (#1583) cbe2159 [vnet] Add "vnet_route_check" script (#1300) 9120766 Relax the install_requires, no need to exact version as long as there are no broken changes with future versions (#1530) 2e09b22 Handle the new db version which mellanox_buffer_migrator isn't interested (#1566)
Includes below commits 9a88cb6 2021-05-06 | [sonic_installer] dont fail package migration (#1591) [Stepan Blyshchak] 615e531 2021-05-05 | [show][config] Add new snmp commands (#1347) [Travis Van Duyn] fff4051 2021-05-05 | Fixing serial number read to get from DB if it is populated (#1580) [Sudharsan Dhamal Gopalarathnam] be974bf 2021-05-05 | [neighbor_advertiser] Use existing tunnel if present for creating tunnel mappings (#1589) [Sumukha Tumkur Vani] 9492eab 2021-05-04 | Use swsscommon instead of swsssdk (#1510) [Andriy Yurkiv] 0f4988b 2021-05-04 | Add pg-drop script to sonic filesystem (#1583) [Andriy Yurkiv] cbe2159 2021-05-04 | [vnet] Add "vnet_route_check" script (#1300) [Volodymyr Samotiy] 9120766 2021-05-03 | Relax the install_requires, no need to exact version as long as there are no broken changes with future versions (#1530) [Qi Luo] 2e09b22 2021-05-03 | Handle the new db version which mellanox_buffer_migrator isn't interested (#1566) [Stephen Sun]
…sted (sonic-net#1566) Enhancement: handle the case that no buffer change in the latest database version Current, the following two versions are the same: - The latest version changed by mellanox_buffer_migrator - The latest version in CONFIG_DB That won't be true if another part in CONFIG_DB is updated. In that case, the latest version in CONFIG_DB will be greater than the latest version in mellanox_buffer_migrator. However, this can break the buffer migrator unit test: - The db_migrator will always migrate the database to the latest version - The config database version check will fail in case the latest version in the config database doesn't match that defined in the buffer migrator. This is to support this case. Signed-off-by: Stephen Sun <stephens@nvidia.com>
9a88cb6 [sonic_installer] dont fail package migration (sonic-net#1591) 615e531 [show][config] Add new snmp commands (sonic-net#1347) fff4051 Fixing serial number read to get from DB if it is populated (sonic-net#1580) be974bf [neighbor_advertiser] Use existing tunnel if present for creating tunnel mappings (sonic-net#1589) 9492eab Use swsscommon instead of swsssdk (sonic-net#1510) 0f4988b Add pg-drop script to sonic filesystem (sonic-net#1583) cbe2159 [vnet] Add "vnet_route_check" script (sonic-net#1300) 9120766 Relax the install_requires, no need to exact version as long as there are no broken changes with future versions (sonic-net#1530) 2e09b22 Handle the new db version which mellanox_buffer_migrator isn't interested (sonic-net#1566)
9a88cb6 [sonic_installer] dont fail package migration (sonic-net#1591) 615e531 [show][config] Add new snmp commands (sonic-net#1347) fff4051 Fixing serial number read to get from DB if it is populated (sonic-net#1580) be974bf [neighbor_advertiser] Use existing tunnel if present for creating tunnel mappings (sonic-net#1589) 9492eab Use swsscommon instead of swsssdk (sonic-net#1510) 0f4988b Add pg-drop script to sonic filesystem (sonic-net#1583) cbe2159 [vnet] Add "vnet_route_check" script (sonic-net#1300) 9120766 Relax the install_requires, no need to exact version as long as there are no broken changes with future versions (sonic-net#1530) 2e09b22 Handle the new db version which mellanox_buffer_migrator isn't interested (sonic-net#1566)
…rator test cases as well (sonic-net#1614) - What I did Originally, the method advance_version_for_expected_database was introduced (in sonic-net#1566) to handle the case the latest version in CONFIG_DB is greater than the latest version in mellanox_buffer_migrator. Now there are other database migrators whose test cases can also encounter this situation, like port auto-negotiation (sonic-net#1568) and port-channel for LACP key (sonic-net#1473). So I would like to make the method public, available for all database migrators. Related database migrator test cases have been updated accordingly. - How to verify it Run the unit test. Signed-off-by: Stephen Sun <stephens@nvidia.com>
What I did The port-channel key migrator was introduced in version 2_0_2 so the expected database version of the test case should be 2_0_2. It was modified to 2_0_3 when the new version was introduced by mistake. This won't fail the test but disable the require its database version to be updated every time a new version is introduced. (Refer sonic-net#1566 and sonic-net#1614 for details) This is to correct it by changing it back to 2_0_2. Signed-off-by: Stephen Sun <stephens@nvidia.com>
Description
config save is hangs indefinitely after deploying t0 topology minigraph from server
cannot take o/p of "show techsupport" as that cmd also hangs indefinitely
using the label 161
https://sonic-jenkins.westus2.cloudapp.azure.com/job/broadcom/job/buildimage-brcm-stretch-all/lastSuccessfulBuild/artifact/target/sonic-broadcom.bin
Steps to reproduce the issue:
Describe the results you received:
<<<<<<<<<<<<<<<<<<<<>>>>>>>>>>>>>>>>>>>
root@sonic-z9100-02:~# config save
Existing file will be overwritten, continue? [y/N]: y
Running command: sonic-cfggen -d --print-data > /etc/sonic/config_db.json
<<<<<<<<<<<<<<<<<<<<>>>>>>>>>>>>>>>>>>>
the cmd hangs indefinitely
Describe the results you expected:
Additional information you deem important (e.g. issue happens only occasionally):
<<<<<<<<<<<<<<<<<<<<>>>>>>>>>>>>>>>>>>>
dmin@sonic-z9100-02:~$ show ver
SONiC Software Version: SONiC.HEAD.161-7c8a635
Distribution: Debian 9.4
Kernel: 4.9.0-5-amd64
Build commit: 7c8a635
Build date: Tue Apr 3 11:25:06 UTC 2018
Built by: johnar@jenkins-worker-4
Docker images:
REPOSITORY TAG IMAGE ID SIZE
docker-syncd-brcm HEAD.161-7c8a635 263bcbed3b14 358.1 MB
docker-syncd-brcm latest 263bcbed3b14 358.1 MB
docker-orchagent-brcm HEAD.161-7c8a635 f5326eae6970 287 MB
docker-orchagent-brcm latest f5326eae6970 287 MB
docker-lldp-sv2 HEAD.161-7c8a635 d77ff6995334 297.1 MB
docker-lldp-sv2 latest d77ff6995334 297.1 MB
docker-dhcp-relay HEAD.161-7c8a635 92ada5e29818 280.1 MB
docker-dhcp-relay latest 92ada5e29818 280.1 MB
docker-database HEAD.161-7c8a635 746e6563e61f 278.7 MB
docker-database latest 746e6563e61f 278.7 MB
docker-teamd HEAD.161-7c8a635 b7ac2067fd06 284.1 MB
docker-teamd latest b7ac2067fd06 284.1 MB
docker-snmp-sv2 HEAD.161-7c8a635 7096ad720bf0 319.2 MB
docker-snmp-sv2 latest 7096ad720bf0 319.2 MB
docker-router-advertiser HEAD.161-7c8a635 a78cf51c9c00 276.4 MB
docker-router-advertiser latest a78cf51c9c00 276.4 MB
docker-platform-monitor HEAD.161-7c8a635 072d237209c4 298.3 MB
docker-platform-monitor latest 072d237209c4 298.3 MB
docker-fpm-quagga HEAD.161-7c8a635 02f60ab8b7b5 290.6 MB
docker-fpm-quagga latest 02f60ab8b7b5 290.6 MB
admin@sonic-z9100-02:~$
<<<<<<<<<<<<<<<<<<<<>>>>>>>>>>>>>>>>>>>
"show techsupport" cli also hangs indefinitely
The text was updated successfully, but these errors were encountered: