DB_MIGRATOR misses FEATURE table during warm upgrade from 201811 to 201911

[vaibhavhd]

Description

Warm reboot upgrade from 201811 to 201911 fails to update FEATURE table in the target config_db
Possible cause of the issue - FEATURE table is not found in 201811 image and in 201911 image, db_migrator failed to update this new information in config_db.
However, with cold reboot, the error gets fixed. So there might be a difference in db_migrator functionality in WARM vs COLD reboot.

root@sonic-device:~# show feature status 
Feature    State    AutoRestart
---------  -------  -------------
root@sonic-device:~#

Also, FEATURE table access errors are seen in syslog consistently:

Oct 28 07:44:24.955625 sonic-device ERR syncd#supervisor-proc-exit-listener: Unable to retrieve features table from Config DB. Exiting…
Oct 28 07:44:28.146142 sonic-device ERR lldp#supervisor-proc-exit-listener: Unable to retrieve features table from Config DB. Exiting...
Oct 28 07:44:28.254954 sonic-device ERR swss#supervisor-proc-exit-listener: Unable to retrieve features table from Config DB. Exiting...
Oct 28 07:44:30.499858 sonic-device ERR radv#supervisor-proc-exit-listener: Unable to retrieve features table from Config DB. Exiting...

Note that after cold reboot, FEATURE table is found and works:

root@sonic-device:~# show feature status 
Feature     State    AutoRestart
----------  -------  -------------
acms        enabled  enabled
bgp         enabled  enabled
database    enabled  disabled
dhcp_relay  enabled  enabled
lldp        enabled  enabled
pmon        enabled  enabled
radv        enabled  enabled
snmp        enabled  enabled
swss        enabled  enabled
syncd       enabled  enabled
teamd       enabled  enabled
telemetry   enabled  enabled
root@sonic-device:~# 

Steps to reproduce the issue:

1. With base image - 201811, install a new 201911 image on DUT.
2. Warm boot the DUT
3. Errors will be reproduced in syslog and via show feature status
4. COLD reboot the DUT, and the issue will be fixed.

Describe the results you received:

No features displayed in show feature status and errors seen in syslog

Describe the results you expected:
Successful warm boot upgrade with FEATURE table updated in new image.

[qiluo-msft]

Should we cherry-pick to fix commit to 201911?

[vaibhavhd]

Should we cherry-pick to fix commit to 201911?

I think it should be cherry-picked. There is also an ongoing discussion regarding this in #6659

[prsunny]

@vaibhavhd , i think we may need to test and verify before we do cherry-pick. What do you say?

[vaibhavhd]

Sure Prince. I think that @shi-su has already tested the fix (sonic-net/sonic-utilities#1209) for 201811 to 201911 upgrade. @shi-su can you confirm please?

[shi-su]

Sure Prince. I think that @shi-su has already tested the fix (Azure/sonic-utilities#1209) for 201811 to 201911 upgrade. @shi-su can you confirm please?

Yes, I checked that the fix addresses the problem in #6659.