[dhcp6_relay] [counters] Counters show statistics for deleted vlans during config reload

[vivekrnv]

Description

dhcpv6 show statistics for even deleted vlans

Steps to reproduce the issue:

1. Add 2 Vlans with IP addresses
2. Configure IPv6 address on port to dhcpv6 server (it can be configured without dhcp server itself)
3. Configre IPv6 on Client vlans
4. enable dhcp_relay feature (config feature state dhcp_relay enabled)
5. configre dhcp_relay ipv6 destination for client vlans (config dhcp_relay ipv6 destination add 10 2000:100::2)
6. Check dhcp_relay ipv6 counters (show dhcp_relay ipv6 counters)
7. Remove one of client vlans
8. Check dhcp_relay ipv6 counters

Describe the results you received:

root@qa-eth-vt04-1-3700v:/home/admin# show vlan brief
+-----------+--------------+-----------+----------------+-------------+-----------------------+
|   VLAN ID | IP Address   | Ports     | Port Tagging   | Proxy ARP   | DHCP Helper Address   |
+===========+==============+===========+================+=============+=======================+
|      3207 | 2000::1:1/64 | Ethernet4 | tagged         | disabled    |                       |
+-----------+--------------+-----------+----------------+-------------+-----------------------+

root@qa-eth-vt04-1-3700v:/home/admin# show dhcp_relay ipv6  counters
       Message Type    Vlan1570
-------------------  ----------
            Unknown           0
            Solicit           2
          Advertise           2
            Request           1
            Confirm           0
              Renew           1
             Rebind           1
              Reply           3
            Release           0
            Decline           0
        Reconfigure           8
Information-Request           0
      Relay-Forward           5
        Relay-Reply          13
          Malformed           0

 

       Message Type    Vlan3207
-------------------  ----------
            Unknown           0
            Solicit           1
          Advertise           1
            Request           1
            Confirm           0
              Renew           0
             Rebind           0
              Reply           1
            Release           0
            Decline           0
        Reconfigure           0
Information-Request           0
      Relay-Forward           2
        Relay-Reply           2
          Malformed           0

root@qa-eth-vt04-1-3700v:/home/admin#

Describe the results you expected:

Output of show techsupport:

sonic_dump_qa-eth-vt04-1-3700v_20230511_123606.tar.gz

[vivekrnv]

@yxieca @jcaiMR FYI

[jcaiMR]

@yaqiangz Add yaqiang as well.

[yaqiangz]

@yaqiangz Add yaqiang as well.

Will check.

[jcaiMR]

We are working on this, ETA 6.9

[vivekrnv]

Fixed by sonic-net/sonic-utilities#2852

[vivekrnv]

Seen again on latest labels,. This time, the counters for member interfaces of downstream Vlan's are still seen.

config vlan member del 10 Ethernet0
config vlan member del 10 Ethernet252
config vlan del 10
Restarting dhcp_relay service..........
root@r-tigon-15:/home/admin# show dhcp_relay ipv6 counters -i Ethernet0
       Message Type    Ethernet0(RX)
-------------------  ---------------
            Unknown                0
            Solicit                0
          Advertise                0
            Request                0
            Confirm                0
              Renew                0
             Rebind                0
              Reply                0
            Release                0
            Decline                0
        Reconfigure                0
Information-Request                0
      Relay-Forward                0
        Relay-Reply                0
          Malformed                0

       Message Type    Ethernet0(TX)
-------------------  ---------------
          Advertise                0
            Confirm                0
            Decline                0
Information-Request                0
          Malformed                0
             Rebind                0
        Reconfigure                0
      Relay-Forward                0
        Relay-Reply                0
            Release                0
              Renew                0
              Reply                0
            Request                0
            Solicit                1
            Unknown                0

root@r-tigon-15:/home/admin# show dhcp_relay ipv6 counters -i Ethernet252
       Message Type    Ethernet252(RX)
-------------------  -----------------
            Unknown                  0
            Solicit                  0
          Advertise                  0
            Request                  0
            Confirm                  0
              Renew                  0
             Rebind                  0
              Reply                  0
            Release                  0
            Decline                  0
        Reconfigure                  0
Information-Request                  0
      Relay-Forward                  0
        Relay-Reply                  0
          Malformed                  0

       Message Type    Ethernet252(TX)
-------------------  -----------------
          Advertise                  0
            Confirm                  0
            Decline                  0
Information-Request                  0
          Malformed                  0
             Rebind                  0
        Reconfigure                  0
      Relay-Forward                  0
        Relay-Reply                  0
            Release                  0
              Renew                  0
              Reply                  0
            Request                  0
            Solicit                  1
            Unknown                  0

root@r-tigon-15:/home/admin# show dhcp_relay ipv6 counters -i Vlan10
  Message Type    Vlan10(RX)
--------------  ------------

  Message Type    Vlan10(TX)
--------------  ------------

Related to the new interface counters feature added in dhcp6relay. Vlan counters are cleared but member interfaces are not

[vivekrnv]

@yxieca @yaqiangz @kellyyeh Please Check

[yaqiangz]

@yxieca @yaqiangz @kellyyeh Please Check

Acked

[yaqiangz]

sonic-net/sonic-utilities#3054
Fixed

[vivekrnv]

CLI issue is fixed. Thanks for the quick response. One problem is remaining.

If the config_db is updated (vlan is removed) and config reload is done, the stale entries in state_db are not deleted. Fix has to be in dhcp6relay (or container start script) to delete all the entries of the counter table during the process/container start

[dgsudharsan]

@yxieca @yaqiangz @kellyyeh We have reopened this since we are seeing the issue with config reload scenario as stated above. Can you please raise a fix for this?

[yaqiangz]

@yxieca @yaqiangz @kellyyeh We have reopened this since we are seeing the issue with config reload scenario as stated above. Can you please raise a fix for this?

Will check

[jcaiMR]

@yaqiangz
In dhcp6_relay, we do counter initialize job in following functions.

update_vlan_mapping
update_portchannel_mapping
update_loopback_mapping

So if some interfaces are not in current Vlan/Port-channel member group or not Loopback0, we will skip counter initialize job for them. It won't impact service and once the interfaces added back to Vlan/portchannel groups their counter will be right initialized.

But for this cosmetic issue fix, you can add a function to go through all Ethernet interfaces, and delete "DHCPv6_COUNTER_TABLE|Ethernet*" in state_db. If want fix the issue in dhcp6_relay repo the diff may something like following

void delete_counter(std::shared_ptrswss::DBConnector state_db, std::string &ifname) {
std::string table_name = counter_table + ifname;
state_db->del(table_name);
}

relay.cpp
1514 sockets.push_back(out_filter);
1515 +
+ // walk through all Ethernet interface and call delete_counter()
+
1516 int lo_sock = -1;

[yaqiangz]

@yaqiangz In dhcp6_relay, we do counter initialize job in following functions.

update_vlan_mapping update_portchannel_mapping update_loopback_mapping

So if some interfaces are not in current Vlan/Port-channel member group or not Loopback0, we will skip counter initialize job for them. It won't impact service and once the interfaces added back to Vlan/portchannel groups their counter will be right initialized.

But for this cosmetic issue fix, you can add a function to go through all Ethernet interfaces, and delete "DHCPv6_COUNTER_TABLE|Ethernet*" in state_db. If want fix the issue in dhcp6_relay repo the diff may something like following

void delete_counter(std::shared_ptrswss::DBConnector state_db, std::string &ifname) { std::string table_name = counter_table + ifname; state_db->del(table_name); }

relay.cpp 1514 sockets.push_back(out_filter); 1515 + + // walk through all Ethernet interface and call delete_counter() + 1516 int lo_sock = -1;

Thanks @jcaiMR

[dgsudharsan]

Hi @yaqiangz Any ETA for fix?

[yaqiangz]

Hi @yaqiangz Any ETA for fix?

ETA 12.29

[yaqiangz]

IPv6 fix PR (merged): sonic-net/sonic-dhcp-relay#45

IPv4 fix PR (waiting review): sonic-net/sonic-dhcpmon#14

[vivekrnv]

Issue fixed in 202305 & 202311.

@yaqiangz Will this fix (sonic-net/sonic-dhcp-relay@5ae186f) be ported back to 202205?

[vivekrnv]

Issue fixed in 202305 & 202311.

@yaqiangz Will this fix (sonic-net/sonic-dhcp-relay@5ae186f) be ported back to 202205?

++ @yxieca

[yaqiangz]

Issue fixed in 202305 & 202311.

@yaqiangz Will this fix (sonic-net/sonic-dhcp-relay@5ae186f) be ported back to 202205?

@vivekrnv This PR is to fix counter issue in IPv6 per interface counter, but IPv6 per interface counter support haven't been backport to 202205

Hi @kellyyeh Seems this PR is to update to latest and backport #17841. May I know why we abandon it?

[vivekrnv]

@yaqiangz, This is the problem i'm seeing. "If the config_db is updated (vlan is removed) and config reload is done, the stale entries in state_db are not deleted. Fix has to be in dhcp6relay (or container start script) to delete all the entries of the counter table during the process/container start"

I understand that the per-interface counter fix is not present in 202205. But the other problem exists. Will this be fixed

[vivekrnv]

Hi @yaqiangz , @yxieca This issue in seen again. With this revert sonic-net/sonic-dhcp-relay@84e4419, we've also removed the changes to clear all the counters in state_db during dhcp init.

Please FYI

[yaqiangz]

Confirmed that clear counters has been removed with reverting per-interface counter