[Mellanox] Support DSCP remapping in dual ToR topo on T0 switch #12605

stephenxs · 2022-11-04T11:03:47Z

Why I did it

Support DSCP remapping in dual ToR topo on T0 switch for SKU Mellanox-SN4600c-C64, Mellanox-SN4600c-D48C40, Mellanox-SN2700, Mellanox-SN2700-D48C8.

How I did it

Regarding buffer settings, originally, there are two lossless PGs and queues 3, 4. In dual ToR scenario, the lossless traffic from the leaf switch to the uplink of the ToR switch can be bounced back.
To avoid PFC deadlock, we need to map the bounce-back lossless traffic to different PGs and queues. Therefore, 2 additional lossless PGs and queues are allocated on uplink ports on ToR switches.

On uplink ports, map DSCP 2/6 to TC 2/6 respectively
On downlink ports, both DSCP 2/6 are still mapped to TC 1
Buffer adjusted according to the ports information:
- Mellanox-SN4600c-C64:
  - 56 downlinks 50G + 8 uplinks 100G
- Mellanox-SN4600c-D48C40, Mellanox-SN2700, Mellanox-SN2700-D48C8:
  - 24 downlinks 50G + 8 uplinks 100G

How to verify it

Unit test.

Which release branch to backport (provide reason below if selected)

Description for the changelog

Ensure to add label/tag for the feature raised. example - PR#2174 under sonic-utilities repo. where, Generic Config and Update feature has been labelled as GCU.

Link to config_db schema for YANG module changes

A picture of a cute animal (not mandatory but encouraged)

stephenxs · 2022-11-11T01:54:03Z

Building vs failed due to the following error. Retrying.

fatal: No names found, cannot describe anything.
fatal: No names found, cannot describe anything.
fatal: No names found, cannot describe anything.
fatal: No names found, cannot describe anything.
fatal: No names found, cannot describe anything.

stephenxs · 2022-11-11T01:54:15Z

/azpw run azure.sonic-buildimage

mssonicbld · 2022-11-11T01:54:17Z

/AzurePipelines run azure.sonic-buildimage

azure-pipelines · 2022-11-11T01:54:29Z

Azure Pipelines successfully started running 1 pipeline(s).

stephenxs · 2022-11-30T23:58:30Z

Failed due to environmental issue. Retriggering.

Traceback (most recent call last):
  File "/usr/lib/python3.9/shutil.py", line 806, in move
    os.rename(src, real_dst)
FileNotFoundError: [Errno 2] No such file or directory: '/var/AzDevOps/.local/lib/python3.9/site-packages/tests/__init__.py' -> '/tmp/pip-uninstall-ypwbu40b/__init__.py'

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/usr/local/lib/python3.9/dist-packages/pip/_internal/cli/base_command.py", line 160, in exc_logging_wrapper
    status = run_func(*args)
  File "/usr/local/lib/python3.9/dist-packages/pip/_internal/commands/uninstall.py", line 98, in run
    uninstall_pathset = req.uninstall(
  File "/usr/local/lib/python3.9/dist-packages/pip/_internal/req/req_install.py", line 660, in uninstall
    uninstalled_pathset.remove(auto_confirm, verbose)
  File "/usr/local/lib/python3.9/dist-packages/pip/_internal/req/req_uninstall.py", line 373, in remove
    moved.stash(path)
  File "/usr/local/lib/python3.9/dist-packages/pip/_internal/req/req_uninstall.py", line 271, in stash
    renames(path, new_path)
  File "/usr/local/lib/python3.9/dist-packages/pip/_internal/utils/misc.py", line 311, in renames
    shutil.move(old, new)
  File "/usr/lib/python3.9/shutil.py", line 820, in move
    copy_function(src, real_dst)
  File "/usr/lib/python3.9/shutil.py", line 435, in copy2
    copyfile(src, dst, follow_symlinks=follow_symlinks)
  File "/usr/lib/python3.9/shutil.py", line 264, in copyfile
    with open(src, 'rb') as fsrc, open(dst, 'wb') as fdst:
FileNotFoundError: [Errno 2] No such file or directory: '/var/AzDevOps/.local/lib/python3.9/site-packages/tests/__init__.py'
make: *** [slave.mk:817: target/python-wheels/bullseye/sonic_utilities-1.2-py3-none-any.whl] Error 2
make: *** Waiting for unfinished jobs....
[ finished ] [ target/python-wheels/bullseye/sonic_thermalctld-1.0-py3-none-any.whl ] 
[ finished ] [ target/python-wheels/bullseye/sonic_psud-1.0-py3-none-any.whl ] 
e9ac64c622531679b1968c428d1c98ed7ee618bd97cc59d7c711b7ebf3f57ef0
docker-config-engine-bullseye
[ finished ] [ target/docker-config-engine-bullseye.gz ] 
WARNING: Ignoring invalid distribution -onic-utilities (/var/AzDevOps/.local/lib/python3.9/site-packages)
make[1]: *** [Makefile.work:535: target/sonic-aboot-broadcom.swi] Error 2
make[1]: Leaving directory '/agent/_work/2/s'
make: *** [Makefile:41: target/sonic-aboot-broadcom.swi] Error 2

stephenxs · 2022-11-30T23:58:42Z

/azpw run azure.sonic-buildimage

mssonicbld · 2022-11-30T23:58:44Z

/AzurePipelines run azure.sonic-buildimage

azure-pipelines · 2022-11-30T23:58:56Z

Azure Pipelines successfully started running 1 pipeline(s).

stephenxs · 2022-12-01T10:53:39Z

/azpw run azure.sonic-buildimage

mssonicbld · 2022-12-01T10:53:41Z

/AzurePipelines run azure.sonic-buildimage

azure-pipelines · 2022-12-01T10:53:54Z

Azure Pipelines successfully started running 1 pipeline(s).

stephenxs · 2022-12-08T00:52:13Z

/azpw run azure.sonic-buildimage

mssonicbld · 2022-12-08T00:52:15Z

/AzurePipelines run azure.sonic-buildimage

azure-pipelines · 2022-12-08T00:52:28Z

Azure Pipelines successfully started running 1 pipeline(s).

Signed-off-by: Stephen Sun <stephens@nvidia.com>

Mellanox-SN4600c-C64: - 56 downlinks 50G + 8 uplinks 100G Mellanox-SN4600c-D48C40/Mellanox-SN2700/Mellanox-SN2700-D48C8: - 24 downlinks 50G + 8 uplinks 100G Signed-off-by: Stephen Sun <stephens@nvidia.com>

Signed-off-by: Stephen Sun <stephens@nvidia.com>

…l ToR t0 test cases Signed-off-by: Stephen Sun <stephens@nvidia.com>

Signed-off-by: Stephen Sun <stephens@nvidia.com>

azure-pipelines · 2023-02-06T18:00:34Z

Azure Pipelines successfully started running 1 pipeline(s).

stephenxs · 2023-02-07T00:12:14Z

/azpw run azure.sonic-buildimage

mssonicbld · 2023-02-07T00:12:16Z

/AzurePipelines run azure.sonic-buildimage

azure-pipelines · 2023-02-07T00:12:29Z

Azure Pipelines successfully started running 1 pipeline(s).

…c-net#12605) - Why I did it Support DSCP remapping in dual ToR topo on T0 switch for SKU Mellanox-SN4600c-C64, Mellanox-SN4600c-D48C40, Mellanox-SN2700, Mellanox-SN2700-D48C8. - How I did it Regarding buffer settings, originally, there are two lossless PGs and queues 3, 4. In dual ToR scenario, the lossless traffic from the leaf switch to the uplink of the ToR switch can be bounced back. To avoid PFC deadlock, we need to map the bounce-back lossless traffic to different PGs and queues. Therefore, 2 additional lossless PGs and queues are allocated on uplink ports on ToR switches. On uplink ports, map DSCP 2/6 to TC 2/6 respectively On downlink ports, both DSCP 2/6 are still mapped to TC 1 Buffer adjusted according to the ports information: Mellanox-SN4600c-C64: 56 downlinks 50G + 8 uplinks 100G Mellanox-SN4600c-D48C40, Mellanox-SN2700, Mellanox-SN2700-D48C8: 24 downlinks 50G + 8 uplinks 100G - How to verify it Unit test. Signed-off-by: Stephen Sun <stephens@nvidia.com>

mssonicbld · 2023-02-09T21:35:05Z

Cherry-pick PR to 202205: #13745

…) (#13745) - Why I did it Support DSCP remapping in dual ToR topo on T0 switch for SKU Mellanox-SN4600c-C64, Mellanox-SN4600c-D48C40, Mellanox-SN2700, Mellanox-SN2700-D48C8. - How I did it Regarding buffer settings, originally, there are two lossless PGs and queues 3, 4. In dual ToR scenario, the lossless traffic from the leaf switch to the uplink of the ToR switch can be bounced back. To avoid PFC deadlock, we need to map the bounce-back lossless traffic to different PGs and queues. Therefore, 2 additional lossless PGs and queues are allocated on uplink ports on ToR switches. On uplink ports, map DSCP 2/6 to TC 2/6 respectively On downlink ports, both DSCP 2/6 are still mapped to TC 1 Buffer adjusted according to the ports information: Mellanox-SN4600c-C64: 56 downlinks 50G + 8 uplinks 100G Mellanox-SN4600c-D48C40, Mellanox-SN2700, Mellanox-SN2700-D48C8: 24 downlinks 50G + 8 uplinks 100G - How to verify it Unit test. Signed-off-by: Stephen Sun <stephens@nvidia.com> Co-authored-by: Stephen Sun <5379172+stephenxs@users.noreply.github.com>

stephenxs · 2023-02-11T03:10:12Z

@StormLiangMS Can you help cherry-pick this PR to 202211? Thanks

…c-net#12605) - Why I did it Support DSCP remapping in dual ToR topo on T0 switch for SKU Mellanox-SN4600c-C64, Mellanox-SN4600c-D48C40, Mellanox-SN2700, Mellanox-SN2700-D48C8. - How I did it Regarding buffer settings, originally, there are two lossless PGs and queues 3, 4. In dual ToR scenario, the lossless traffic from the leaf switch to the uplink of the ToR switch can be bounced back. To avoid PFC deadlock, we need to map the bounce-back lossless traffic to different PGs and queues. Therefore, 2 additional lossless PGs and queues are allocated on uplink ports on ToR switches. On uplink ports, map DSCP 2/6 to TC 2/6 respectively On downlink ports, both DSCP 2/6 are still mapped to TC 1 Buffer adjusted according to the ports information: Mellanox-SN4600c-C64: 56 downlinks 50G + 8 uplinks 100G Mellanox-SN4600c-D48C40, Mellanox-SN2700, Mellanox-SN2700-D48C8: 24 downlinks 50G + 8 uplinks 100G - How to verify it Unit test. Signed-off-by: Stephen Sun <stephens@nvidia.com>

mssonicbld · 2023-02-13T07:01:11Z

Cherry-pick PR to 202211: #13787

…) (#13787)

stephenxs requested review from bingwang-ms, baiwei0427 and neethajohn November 4, 2022 11:04

stephenxs mentioned this pull request Nov 4, 2022

[PCBB] Add testcases to verify separated DSCP_TO_TC_MAP on leaf router sonic-net/sonic-mgmt#6393

Merged

5 tasks

stephenxs force-pushed the dual-tor-dscp-remapping branch from 0bb04f8 to 4dc91f3 Compare November 23, 2022 06:02

stephenxs force-pushed the dual-tor-dscp-remapping branch from d08ad84 to 50936a4 Compare December 27, 2022 04:36

stephenxs mentioned this pull request Dec 28, 2022

[Mellanox] Enable DSCP remapping by using SAI attribute sonic-net/sonic-sairedis#1188

Merged

stephenxs added 9 commits January 17, 2023 13:04

Calculate 4600C-C64 t0 with 48 downlinks and 16 uplinks

38ad4fa

Signed-off-by: Stephen Sun <stephens@nvidia.com>

Support t0 SKUs

0c5da22

Signed-off-by: Stephen Sun <stephens@nvidia.com>

Support DSCP remapping for t0 switch

df0274d

Signed-off-by: Stephen Sun <stephens@nvidia.com>

Unit test cases for dual ToR DSCP remapping t0

f821d52

Signed-off-by: Stephen Sun <stephens@nvidia.com>

Update pool size for dual ToR t0 topo

aae310b

Mellanox-SN4600c-C64: - 56 downlinks 50G + 8 uplinks 100G Mellanox-SN4600c-D48C40/Mellanox-SN2700/Mellanox-SN2700-D48C8: - 24 downlinks 50G + 8 uplinks 100G Signed-off-by: Stephen Sun <stephens@nvidia.com>

Unit test for DSCP remapping buffer part

035e269

Signed-off-by: Stephen Sun <stephens@nvidia.com>

Add queue 7 in SCHEDULER table

d1c5c7f

Signed-off-by: Stephen Sun <stephens@nvidia.com>

Remove buffer test of 2700 since the template has been covered by dua…

01df9e6

…l ToR t0 test cases Signed-off-by: Stephen Sun <stephens@nvidia.com>

Fix bug: remapping wasn't disabled in 4600c-t0 "remap disable" minigraph

659e66f

Signed-off-by: Stephen Sun <stephens@nvidia.com>

Merge branch 'sonic-net:master' into dual-tor-dscp-remapping

30cba4d

liat-grozovik merged commit e3ff088 into sonic-net:master Feb 7, 2023

stephenxs deleted the dual-tor-dscp-remapping branch February 7, 2023 14:50

ayurkiv-nvda mentioned this pull request Feb 7, 2023

[Dual-ToR] add 'additional_mac' attr to swss vars #13546

Closed

8 tasks

yxieca added the Approved for 202205 Branch label Feb 9, 2023

mssonicbld added the Created PR to 202205 Branch label Feb 9, 2023

mssonicbld mentioned this pull request Feb 9, 2023

[action] [PR:12605] [Mellanox] Support DSCP remapping in dual ToR topo on T0 switch #13745

Merged

mssonicbld added Included in 202205 Branch and removed Approved for 202205 Branch Created PR to 202205 Branch labels Feb 10, 2023

StormLiangMS added the Approved for 202211 Branch label Feb 13, 2023

mssonicbld added the Created PR to 202211 Branch label Feb 13, 2023

mssonicbld mentioned this pull request Feb 13, 2023

[action] [PR:12605] [Mellanox] Support DSCP remapping in dual ToR topo on T0 switch #13787

Merged

mssonicbld added a commit that referenced this pull request Feb 14, 2023

[Mellanox] Support DSCP remapping in dual ToR topo on T0 switch (#12605…

d1de964

…) (#13787)

mssonicbld added Included in 202211 Branch and removed Approved for 202211 Branch Created PR to 202211 Branch labels Feb 14, 2023

congh-nvidia mentioned this pull request May 16, 2023

[Mellanox] Update dscp remapping cases for Nvidia platforms sonic-net/sonic-mgmt#8317

Merged

6 tasks

stephenxs mentioned this pull request May 7, 2024

Avoid attaching lossless buffer profiles for internal ports vivekrnv/sonic-buildimage#97

Closed

11 tasks

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

[Mellanox] Support DSCP remapping in dual ToR topo on T0 switch #12605

[Mellanox] Support DSCP remapping in dual ToR topo on T0 switch #12605

stephenxs commented Nov 4, 2022 •

edited by liat-grozovik

Loading

stephenxs commented Nov 11, 2022

stephenxs commented Nov 11, 2022

mssonicbld commented Nov 11, 2022

azure-pipelines bot commented Nov 11, 2022

stephenxs commented Nov 30, 2022

stephenxs commented Nov 30, 2022

mssonicbld commented Nov 30, 2022

azure-pipelines bot commented Nov 30, 2022

stephenxs commented Dec 1, 2022

mssonicbld commented Dec 1, 2022

azure-pipelines bot commented Dec 1, 2022

stephenxs commented Dec 8, 2022

mssonicbld commented Dec 8, 2022

azure-pipelines bot commented Dec 8, 2022

azure-pipelines bot commented Feb 6, 2023

stephenxs commented Feb 7, 2023

mssonicbld commented Feb 7, 2023

azure-pipelines bot commented Feb 7, 2023

mssonicbld commented Feb 9, 2023

stephenxs commented Feb 11, 2023

mssonicbld commented Feb 13, 2023

[Mellanox] Support DSCP remapping in dual ToR topo on T0 switch #12605

[Mellanox] Support DSCP remapping in dual ToR topo on T0 switch #12605

Conversation

stephenxs commented Nov 4, 2022 • edited by liat-grozovik Loading

Why I did it

How I did it

How to verify it

Which release branch to backport (provide reason below if selected)

Description for the changelog

Ensure to add label/tag for the feature raised. example - PR#2174 under sonic-utilities repo. where, Generic Config and Update feature has been labelled as GCU.

Link to config_db schema for YANG module changes

A picture of a cute animal (not mandatory but encouraged)

stephenxs commented Nov 11, 2022

stephenxs commented Nov 11, 2022

mssonicbld commented Nov 11, 2022

azure-pipelines bot commented Nov 11, 2022

stephenxs commented Nov 30, 2022

stephenxs commented Nov 30, 2022

mssonicbld commented Nov 30, 2022

azure-pipelines bot commented Nov 30, 2022

stephenxs commented Dec 1, 2022

mssonicbld commented Dec 1, 2022

azure-pipelines bot commented Dec 1, 2022

stephenxs commented Dec 8, 2022

mssonicbld commented Dec 8, 2022

azure-pipelines bot commented Dec 8, 2022

azure-pipelines bot commented Feb 6, 2023

stephenxs commented Feb 7, 2023

mssonicbld commented Feb 7, 2023

azure-pipelines bot commented Feb 7, 2023

mssonicbld commented Feb 9, 2023

stephenxs commented Feb 11, 2023

mssonicbld commented Feb 13, 2023

stephenxs commented Nov 4, 2022 •

edited by liat-grozovik

Loading