-
Notifications
You must be signed in to change notification settings - Fork 531
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Add basic fabric link monitoring counters and states handling. #2988
Add basic fabric link monitoring counters and states handling. #2988
Conversation
Signed-off-by: Jie Feng <jfeng@arista.com>
Signed-off-by: Jie Feng <jfeng@arista.com>
it looks like none of the test_fabric* run |
I checked again and find that those fabric related tests in the log, and the log@1 from https://dev.azure.com/mssonic/build/_build/results?buildId=431467&view=artifacts&pathAsName=false&type=publishedArtifacts from the syslog of all test_fabric_* , I can see FabricPortsOrch get construct and at least generatePortStats getting called and etc. so at least it seems the orchagent.fabricportsorch.cpp get covered, should not get 0% coverage here. Would anyone please help check this ? and pointing what actually missed ? thanks @arlakshm do you mind help check if this is some known behavior ? or can help direct this question to someone might be able to help ? |
/Azpw run Azure.sonic-swss |
/AzurePipelines run Azure.sonic-swss |
Azure Pipelines successfully started running 1 pipeline(s). |
test_add_remove_port failed (1 runs remaining out of 2). It is not caused by this change . |
/Azpw run Azure.sonic-swss |
/AzurePipelines run Azure.sonic-swss |
Azure Pipelines successfully started running 1 pipeline(s). |
three bfd related tests failed |
/Azpw run Azure.sonic-swss |
/AzurePipelines run Azure.sonic-swss |
Azure Pipelines successfully started running 1 pipeline(s). |
@jfeng-arista , will merge after @arlakshm sign-off |
thank you
…On Thu, Jan 4, 2024 at 5:06 PM Prince Sunny ***@***.***> wrote:
@jfeng-arista <https://github.com/jfeng-arista> , will merge after
@arlakshm <https://github.com/arlakshm> sign-off
—
Reply to this email directly, view it on GitHub
<#2988 (comment)>,
or unsubscribe
<https://github.com/notifications/unsubscribe-auth/AXO4THQXE6HW55FPFURGLY3YM5GZRAVCNFSM6AAAAABAQK26I2VHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMYTQNZXHE3TGNBXGQ>
.
You are receiving this because you were mentioned.Message ID:
***@***.***>
|
test_NeighborAddRemoveIpv6LinkLocal failed, does not related to this change |
/azp run |
Azure Pipelines successfully started running 1 pipeline(s). |
/azp run |
Azure Pipelines successfully started running 1 pipeline(s). |
For code coverage, please skip the check coverage.Azure.sonic-swss.amd64, only need to pass coverage.Azure.sonic-swss.vstest. |
@vmittal-msft @saksarav-nokia please review |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
lgtm, few minor comments
orchagent/fabricportsorch.cpp
Outdated
if (!m_getFabricPortListDone) | ||
{ | ||
getFabricPortList(); | ||
} |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
why we need to get the fabric port list again?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
why we need to get the fabric port list again?
we try to catch up the case if getFabricPortList did not get called before or not successfully get the done so m_getFabricPortListDone get set. Here we only try to call getFabricPortList again if m_getFabricPortListDone is false. thank you
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
why we need to get the fabric port list again?
Discussed with Arvind, remove this getFabricPortList call here. thank you
orchagent/fabricportsorch.cpp
Outdated
bool exist = m_stateTable->get(key, values); | ||
if(!exist) | ||
{ | ||
SWSS_LOG_INFO("No state infor for port %s", key.c_str()); |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
fix identation
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
fix identation
fixed. thank you!
orchagent/fabricportsorch.cpp
Outdated
valuePt = fvValue(val); | ||
if (fvField(val) == "STATUS") | ||
{ | ||
lnkStatus = valuePt; | ||
continue; | ||
} | ||
if (fvField(val) == "POLL_WITH_ERRORS") | ||
{ | ||
consecutivePollsWithErrors = to_uint<uint8_t>(valuePt); | ||
continue; | ||
} | ||
if (fvField(val) == "POLL_WITH_NO_ERRORS") | ||
{ | ||
consecutivePollsWithNoErrors = to_uint<uint8_t>(valuePt); | ||
continue; | ||
} | ||
if (fvField(val) == "POLL_WITH_FEC_ERRORS") | ||
{ | ||
consecutivePollsWithFecErrs = to_uint<uint8_t>(valuePt); | ||
continue; | ||
} | ||
if (fvField(val) == "POLL_WITH_NOFEC_ERRORS") | ||
{ | ||
consecutivePollsWithNoFecErrs = to_uint<uint8_t>(valuePt); | ||
continue; | ||
} | ||
if (fvField(val) == "SKIP_CRC_ERR_ON_LNKUP_CNT") | ||
{ | ||
skipCrcErrorsOnLinkupCount = to_uint<uint8_t>(valuePt); | ||
continue; | ||
} | ||
if (fvField(val) == "SKIP_FEC_ERR_ON_LNKUP_CNT") | ||
{ | ||
skipFecErrorsOnLinkupCount = to_uint<uint8_t>(valuePt); | ||
continue; | ||
} | ||
if (fvField(val) == "RX_CELLS") | ||
{ | ||
prevRxCells = to_uint<uint64_t>(valuePt); | ||
continue; | ||
} | ||
if (fvField(val) == "CRC_ERRORS") | ||
{ | ||
prevCrcErrors = to_uint<uint64_t>(valuePt); | ||
continue; | ||
} | ||
if (fvField(val) == "CODE_ERRORS") | ||
{ | ||
prevCodeErrors = to_uint<uint64_t>(valuePt); | ||
continue; | ||
} | ||
if (fvField(val) == "AUTO_ISOLATED") | ||
{ | ||
autoIsolated = to_uint<uint8_t>(valuePt); | ||
SWSS_LOG_INFO("port %s currently isolated: %s", key.c_str(),valuePt.c_str()); | ||
continue; | ||
} | ||
if (fvField(val) == "TEST_CRC_ERRORS") | ||
{ | ||
testCrcErrors = to_uint<uint64_t>(valuePt); | ||
continue; | ||
} | ||
if (fvField(val) == "TEST_CODE_ERRORS") | ||
{ | ||
testCodeErrors = to_uint<uint64_t>(valuePt); | ||
continue; | ||
} | ||
if (fvField(val) == "TEST") | ||
{ | ||
testState = valuePt; | ||
continue; | ||
} |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Fix identation
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Fix identation
fixed. thank you !
orchagent/fabricportsorch.cpp
Outdated
valuePt = to_string(autoIsolated); | ||
m_stateTable->hset(key, "AUTO_ISOLATED", valuePt); | ||
SWSS_LOG_INFO("port %s set AUTO_ISOLATED %s", key.c_str(), valuePt.c_str()); | ||
// Call SAI api here to actually isolated the link |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
This is will be subsequent PR?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
This is will be subsequent PR?
Yes. There will be subsequent PR of this actual sai call when it is available . thank you
orchagent/fabricportsorch.cpp
Outdated
if (lnkStatus == "up") | ||
{ | ||
// debug information | ||
SWSS_LOG_INFO("port %s status up autoIsolated %d", |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
can we make this log as NOTICE
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
can we make this log as NOTICE
Here is the debug information for all links and that might be too chatty. BUT I did change the log level to NOTICE several lines down where we decide to isolate the link due to the problem or unisolate due to the link healthy again, so we will have the information for links that in trouble.
{
// Link needs to be isolated.
SWSS_LOG_INFO("port %s auto isolated", key.c_str());
autoIsolated = 1;
valuePt = to_string(autoIsolated);
m_stateTable->hset(key, "AUTO_ISOLATED", valuePt);
**SWSS_LOG_NOTICE**("port %s set AUTO_ISOLATED %s", key.c_str(), valuePt.c_str());
// Call SAI api here to actually isolated the link
}
{
// Link is isolated, but no longer needs to be.
SWSS_LOG_INFO("port %s healthy again", key.c_str());
autoIsolated = 0;
valuePt = to_string(autoIsolated);
m_stateTable->hset(key, "AUTO_ISOLATED", valuePt);
**SWSS_LOG_NOTICE**("port %s set AUTO_ISOLATED %s", key.c_str(), valuePt.c_str());
// Can we call SAI api here to unisolate the link?
}
…-net#2988) * Add basic fabric link monitoring counters and states handling.
Description of PR Summary: Add a management test for fabric link monitoring infrastructure. Fixes # (issue) Type of change Bug fix Testbed and Framework(new/improvement) [ x] Test case(new/improvement) Back port request 201911 202012 202205 202305 202311 Approach What is the motivation for this PR? This PR adds a test for the fabric link monitoring infrastructure, the main PR is at sonic-net/sonic-swss#2988
* Fixes mock test failure * Fixes mock test run failure fixes pipeline run failure FAIL: p4orch_tests_usan ======================= ../../../orchagent/vrforch.cpp:113:41: runtime error: member call on null pointer of type 'struct RouteOrch' ../../../orchagent/vrforch.cpp:113:41: runtime error: member access within null pointer of type 'struct RouteOrch' FAIL p4orch_tests_usan (exit status: 139) * Fixed orchagent crash in VM with the Qos BUFFER_QUEUE|system-port|Queue-id-range config (sonic-net#3050) * Fixed orchagent crash in VM with the Qos BUFFER_QUEUE|system-port|Queue-id-range config * [intfsorch] Enable ipv6 proxy ndp along with proxy arp (sonic-net#3045) * [intfsorch] Enable ipv6 proxy ndp along with proxy arp setting SAI_VLAN_ATTR_UNKNOWN_MULTICAST_FLOOD_CONTROL_TYPE to SAI_VLAN_FLOOD_CONTROL_TYPE_NONE when proxy arp is enabled. This fixes a bug where ipv6 NS packets were flooding ports with duplicate packets. We now set multicast flood type to none. * Fix multi VLAN neighbor learning (sonic-net#3049) What I did When adding a new neighbor, check if the neighbor IP has already been learned on a different VLAN. If it has, remove the old neighbor entry before adding the new one. Why I did it On Gemini devices, if a neighbor IP moves from an active port in one VLAN to a second VLAN, then back to the first VLAN (with 3 different MAC addresses), orchagent will crash. Even though the MAC address of the last move is different from the first MAC address, orchagent believes the last MAC address to already be programmed in the hardware and tries to set an attribute of the entry which doesn't exist. * [asan] Disable the "maybe-uninitialized" warning when compiled with ASAN enabled. * Set HOST_TX_READY_NOTIFY attribute only after query capabilities(sonic-net#3070) *Set HOST_TX_READY_NOTIFY attribute only after query capabilities * [EVPN] Skip EVPN routes with invalid VNI or router mac field (sonic-net#3073) * Skip EVPN routes with invalid VNI or router mac field * Add port flap count and last flap timestamp to APPL_DB (sonic-net#3052) * Add port flap count and last flap timestamp * Add basic fabric link monitoring counters and states handling. (sonic-net#2988) * Add basic fabric link monitoring counters and states handling. * [Mellanox] Fix inconsistence in the shared headroom pool initialization (sonic-net#3057) * Fix inconsistence in the shared headroom pool initialization * Why I did it During initialization, if SHP is enabled the buffer pool sizes, xoff have initialized to 0, which means SHP is disabled but the buffer profiles already indicate SHP later on the buffer pool sizes are updated with off being non-zero In case the orchagent starts handling buffer configuration between 2 and 3, it is inconsistent between buffer pools and profiles, which fails Mellanox SAI sanity check. To avoid it, it indicates SHP enabled by setting a very small buffer pool and SHP sizes * [acl] Add IN_PORTS qualifier for L3 table (sonic-net#3078) * Apply IN_PORTS qualifiier for L3 table Why I did it IN_PORTS qualifier was allowed for L3 table in 202012 release and below. Changes in sonic-net#1982 removed that support leading to regression in some of our testcases. The following error was observed ERR swss#orchagent: :- validateAclRuleMatch: Match SAI_ACL_ENTRY_ATTR_FIELD_IN_PORTS in rule RULE_1 is not supported by table DATAACL * [bulker] add support for neighbor bulking (sonic-net#2768) Adding support for sai_neighbor_api_t bulking in bulker.h * [buffermgrd] Move switch-statement outside of if-statement in BufferMgr::doTask (sonic-net#3055) * [buffermgr] Moved switch statement outside of if-statmement in Buffermgr::doTask The switch statement which would normally erase buffer events was moved to be inside the if-statement which would only enter if the event is a SET event. This was introduced in commit e5329c39. This would cause an infinite loop, since non-set events would never be erased. The switch statement has now been moved to occur outside the if, allowing for non-set commands to be processed. * [portsorch] process only updated APP_DB fields when port is already created (sonic-net#3025) * [portsorch] process only updated APP_DB fields when port is already created What I did Fixing an issue when setting some port attribute in APPL_DB triggers serdes parameters to be re-programmed with port toggling. Made portsorch to handle only those attributes that were pushed to APPL_DB, so that serdes programming happens only by xcvrd's request to do so. * [Copp]Refactor coppmgr tests (sonic-net#3093) What I did Refactoring coppmgr mock tests Why I did it After migration to bookworm, coppmgr tests started failing due to the use of sudo commands. * Revert "[acl] Add IN_PORTS qualifier for L3 table (sonic-net#3078)" (sonic-net#3092) This reverts commit 9d4a3ad. *Revert "[acl] Add IN_PORTS qualifier for L3 table" * [orchagent] TWAMP Light orchagent implementation (sonic-net#2927) * [orchagent] TWAMP Light orchagent implementation. (sonic-net#2927) * What I did Implemented the TWAMP Light feature according to the SONiC TWAMP Light HLD(sonic-net/SONiC#1320). * Clang format change. (sonic-net#3080) What I did This PR has no real code change. It is purely clang formatting. It only applies to the P4Orch codes. Commands that I run: find orchagent/p4orch -name *.h -o -name .cpp | xargs clang-format -i -style="{BasedOnStyle: Microsoft, DerivePointerAlignment: false}" find orchagent -name response_publisher -o -name return_code.h | xargs clang-format -i -style="{BasedOnStyle: Microsoft, DerivePointerAlignment: false}" * T2-VOQ-VS: Fix iBGP bringup issue (sonic-net#3053) * Fix iBGP bringup issue T2-vswitch * On T2-VOQ chassis Emulation with multi-asic linecards, iBGP sessions dont come up. Related Issue: sonic-net/sonic-buildimage#18129 * [Fdbsyncd] Adding extern_learn flag with fdb entry so Kernel doesn't age out (sonic-net#2985) * Adding extern_learn flag with fdb entry so that Kernel doesn't age out the MAC * [Fdbsyncd] Adding extern_learn flag with fdb entry so Kernel doesn't age out What I did extern_learn flag is added while programming the fdb entry into the Kernel. This will make sure that kernel doesn't age out the fdb entry. (#15004) How I did it A flag extern_learn will be passed while programing the fdb entry. (#15004) How to verify it Tested MAC add/del to the Kernel from the local FDB entry. (#15004) Signed-off-by: kishore.kunal@broadcom.com --------- Signed-off-by: kishore.kunal@broadcom.com Co-authored-by: Sudharsan Dhamal Gopalarathnam <sudharsand@nvidia.com> * Fix oper FEC retrieval after warmboot (sonic-net#3100) Updating oper FEC status in state_db after warm-reboot as part of refresh port status call * [EVPN]Fix fpmsyncd crash when EVPN type5 is received with bgp fib suppression enabled (sonic-net#3101) * [EVPN]Fix fpmsyncd crash when EVPN type5 is received with bgp fib suppression enabled * [portsorch] Handle TRANSCEIVER_INFO table on warm boot (sonic-net#3087) * Add existing data from TRANSCEIVER_INFO table * Introduce a new role for DPU-NPU Interconnect Signed-off-by: Vivek Reddy Karri <vkarri@nvidia.com> Co-authored-by: Sudharsan Dhamal Gopalarathnam <sudharsand@nvidia.com> * [p4orch] Clang format change. (sonic-net#3096) What I did [p4orch] This PR has no real code change. It is purely clang formatting. It does the same as sonic-net#3080. * [dash] fix ENI admin state update (sonic-net#3081) * [dash] fix ENI admin state update * Add force option for fabric port unisolate command (sonic-net#3089) What I did Add force option to the unisolate link command, so users can make the links not isolate if they want. depends on sonic-net/sonic-buildimage#18447 * [twamporch] Explicitly initialize local variable (sonic-net#3115) What I did Explicitly initialized local variable. Why I did it We met below error message in sonic-buildimage armhf build (sonic-net/sonic-buildimage#18334) * Add bookworm build to the PR checkers (sonic-net#3114) What I did Add a Bookworm build to the PR checkers. Also fix some Bookworm build errors that crept in. Why I did it Buildimage now builds swss for Bookworm, so the build needs to succeed. * [ACL] Remove flex counter when updating ACL rule (sonic-net#3118) What I did This PR is to fix sonic-net/sonic-buildimage#18719 When ACL rule is created for the first time, a flex counter is created and registered. When the same ACL rule is being updated, the FlexCounter created before is not removed, and another FlexCounter is created and registered. Why I did it Fix the issue that FlexCounter is duplicated when updating existing ACL rule. --------- Signed-off-by: kishore.kunal@broadcom.com Signed-off-by: Vivek Reddy Karri <vkarri@nvidia.com> Co-authored-by: saksarav-nokia <sakthivadivu.saravanaraj@nokia.com> Co-authored-by: Nikola Dancejic <26731235+Ndancejic@users.noreply.github.com> Co-authored-by: Lawrence Lee <lawlee@microsoft.com> Co-authored-by: Oleksandr Ivantsiv <oivantsiv@nvidia.com> Co-authored-by: noaOrMlnx <58519608+noaOrMlnx@users.noreply.github.com> Co-authored-by: Lior Avramov <73036155+liorghub@users.noreply.github.com> Co-authored-by: Prince George <45705344+prgeor@users.noreply.github.com> Co-authored-by: jfeng-arista <98421150+jfeng-arista@users.noreply.github.com> Co-authored-by: Stephen Sun <5379172+stephenxs@users.noreply.github.com> Co-authored-by: Neetha John <nejo@microsoft.com> Co-authored-by: Amir <mazora@marvell.com> Co-authored-by: Stepan Blyshchak <38952541+stepanblyschak@users.noreply.github.com> Co-authored-by: Sudharsan Dhamal Gopalarathnam <sudharsand@nvidia.com> Co-authored-by: xiaodong hu <32903206+huseratgithub@users.noreply.github.com> Co-authored-by: mint570 <70396898+mint570@users.noreply.github.com> Co-authored-by: Deepak Singhal <115033986+deepak-singhal0408@users.noreply.github.com> Co-authored-by: KISHORE KUNAL <64033340+kishorekunal01@users.noreply.github.com> Co-authored-by: Vivek <vivekreddykarri98@gmail.com> Co-authored-by: Yakiv Huryk <62013282+Yakiv-Huryk@users.noreply.github.com> Co-authored-by: Saikrishna Arcot <sarcot@microsoft.com> Co-authored-by: bingwang-ms <66248323+bingwang-ms@users.noreply.github.com>
What I did
Added basic fabric link monitoring counters and states handling, and the test for this.
Why I did it
How I verified it
Details if related