Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

QOS fieldvalue reference ABNF format to string changes #1754

Merged
merged 2 commits into from
Sep 24, 2021

Conversation

AshokDaparthi
Copy link
Contributor

@AshokDaparthi AshokDaparthi commented May 20, 2021

Depends on sonic-net/sonic-utilities#1626
Depends on sonic-net/sonic-buildimage#7752

Qos tables in config db and app db used ABNF format i.e "[TABLE_NAME|name] to refer fieldvalue other qos tables.

Example:
Config DB:
"Ethernet92|3": {
"scheduler": "[SCHEDULER|scheduler.1]",
"wred_profile": "[WRED_PROFILE|AZURE_LOSSLESS]"
},
"Ethernet0|0": {
"profile": "[BUFFER_PROFILE|ingress_lossy_profile]"
},
"Ethernet0": {
"dscp_to_tc_map": "[DSCP_TO_TC_MAP|AZURE]",
"pfc_enable": "3,4",
"pfc_to_queue_map": "[MAP_PFC_PRIORITY_TO_QUEUE|AZURE]",
"tc_to_pg_map": "[TC_TO_PRIORITY_GROUP_MAP|AZURE]",
"tc_to_queue_map": "[TC_TO_QUEUE_MAP|AZURE]"
},

AppDB:
"BUFFER_QUEUE_TABLE:Ethernet88:3-4": {
"profile": "[BUFFER_PROFILE_TABLE:egress_lossless_profile]"
},

This format is not consistent with other DB schema followed in sonic.
And also this reference in DB is not required, This is taken care by YANG "leafref".

Removed this format and made it as string.
Example:
"Ethernet92|3": {
"scheduler": "scheduler.1",
"wred_profile": "AZURE_LOSSLESS"
},

Dependent pul; requests:
sonic-net/sonic-buildimage#7752 - To modify platfrom files
sonic-net/sonic-buildimage#7281 - Yang model
sonic-net/sonic-utilities#1626 - DB migration

Copy link
Collaborator

@stephenxs stephenxs left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Hi @AshokDaparthi
I haven't got time to go over the entire PR. Some comments for now:

  • Need to update the vs test cases as well, like test_buffer_dynamic.py, test_speed.py, and test_qos_map.py
    I believe the current vs test is failing without these changes.
  • Need to update the db schema description.
  • Can you fill the fields in the PR description?
    By doing so reviewers can have a better understanding of this PR.

Thanks.

@AshokDaparthi
Copy link
Contributor Author

@stephenxs - Thanks for information, i will dig to test cases. i compiled only for sonic-broadcom.bin , i do not see any failure at build time.

@stephenxs
Copy link
Collaborator

@stephenxs - Thanks for information, i will dig to test cases. i compiled only for sonic-broadcom.bin , i do not see any failure at build time.

Yes. The vs test won't run when the sonic image (like sonic-broadcom.bin) is being built. It runs as part of the checks of the PR (Azure.sonic-swss (Test vstest)). To set up a local environment for vs test, you can refer this guide

@AshokDaparthi
Copy link
Contributor Author

/azpw run

@AshokDaparthi
Copy link
Contributor Author

/azpw run

1 similar comment
@AshokDaparthi
Copy link
Contributor Author

/azpw run

@@ -284,14 +277,6 @@ void BufferMgr::doBufferTableTask(Consumer &consumer, ProducerStateTable &applTa

for (auto i : kfvFieldsValues(t))
{
SWSS_LOG_INFO("Inserting field %s value %s", fvField(i).c_str(), fvValue(i).c_str());
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

transformSeperator which is called in L269 can be removed as well. There is no separator now.
The function transformSeperator and transformReference can be removed as they are no longer called

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

transformSeperator is used to convert from config_db to app_db

@@ -1951,8 +1939,7 @@ task_process_status BufferMgrDynamic::handleOneBufferPgEntry(const string &key,
{
// Headroom override
pureDynamic = false;
transformReference(value);
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

transformReference is no longer called and can be removed.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Done

// Deem it as a valid format
// clear both type_name and object_name
// as an indication to the caller that
// such a case has been encountered
type_name.clear();
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Can an empty string be passed by redis-db? If no and this is a valid flow, we need to find another way to indicate clearing reference. Probably we can use "NULL".

@@ -301,7 +301,7 @@ bool Orch::bake()
}

/*
- Validates reference has proper format which is [table_name:object_name]
- Validates reference has proper format which is not ABNF [table_name:object_name]
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Emphasizing not ABNF looks strange. I think Validates reference has proper format which is object_name should be good.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Done

tokens = tokenize(ref_content, delimiter);
if (tokens.size() != 2)

if ((ref_in[0] == ref_start) || (ref_in[ref_in.size()-1] == ref_end))
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Not sure whether we need to check whether it's surrounded by [] here.

stephenxs
stephenxs previously approved these changes Jul 30, 2021
@lguohan
Copy link
Contributor

lguohan commented Aug 26, 2021

/azp run

@azure-pipelines
Copy link

Azure Pipelines successfully started running 1 pipeline(s).

@zhangyanzhao
Copy link
Collaborator

@AshokDaparthi please help to monitor the build progress and check the result.

@rathnasabapathyv
Copy link

rathnasabapathyv commented Aug 26, 2021

As per the discussion in yang-subgroup-meeting (8/26) with @lguohan , we have decided to merge sonic-utilities PR (sonic-net/sonic-utilities#1626) 1st followed by sub-module update in sonic-buildimage. Then this sonic-swss PR (#1754) should be merged.

Merge can happen in this order:
sonic-utilities PR for DB-migration logic : sonic-net/sonic-utilities#1626
sonic-buildimage PR for new config-DB changes : sonic-net/sonic-buildimage#7752
sonic-swss PR : #1754

@AshokDaparthi
Copy link
Contributor Author

/azpw run

@mssonicbld
Copy link
Collaborator

/AzurePipelines run

@azure-pipelines
Copy link

Azure Pipelines successfully started running 1 pipeline(s).

@prsunny
Copy link
Collaborator

prsunny commented Sep 23, 2021

/azp run

@azure-pipelines
Copy link

Azure Pipelines successfully started running 1 pipeline(s).

@AshokDaparthi
Copy link
Contributor Author

@stephenxs, @neethajohn - Please relook, i did modification to pass swss test cases until sonic-buildimage check in happens. Plan is to push ASAP and update sub modules.

@AshokDaparthi
Copy link
Contributor Author

/azpw run

@mssonicbld
Copy link
Collaborator

/AzurePipelines run

@azure-pipelines
Copy link

Azure Pipelines successfully started running 1 pipeline(s).

AshokDaparthi and others added 2 commits September 23, 2021 13:33
…ests

Skip test cases until sonic-buildimage changes in

Skip test cases until sonic-buildimage changes in
@AshokDaparthi
Copy link
Contributor Author

@neethajohn @stephenxs - Could you please help to review and merge, so that i can push other PR's

@prsunny
Copy link
Collaborator

prsunny commented Sep 24, 2021

@stephenxs , could you please sign-off?

Copy link
Collaborator

@stephenxs stephenxs left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM.

@stephenxs
Copy link
Collaborator

@stephenxs , could you please sign-off?

@prsunny approved.

@prsunny prsunny merged commit a3fdaf4 into sonic-net:master Sep 24, 2021
neethajohn pushed a commit to sonic-net/sonic-buildimage that referenced this pull request Sep 28, 2021
Depends on sonic-net/sonic-utilities#1626
Depends on sonic-net/sonic-swss#1754

QOS tables in config db used ABNF format i.e "[TABLE_NAME|name] to refer fieldvalue to other qos tables.

Example:
Config DB:
"Ethernet92|3": {
"scheduler": "[SCHEDULER|scheduler.1]",
"wred_profile": "[WRED_PROFILE|AZURE_LOSSLESS]"
},
"Ethernet0|0": {
"profile": "[BUFFER_PROFILE|ingress_lossy_profile]"
},
"Ethernet0": {
"dscp_to_tc_map": "[DSCP_TO_TC_MAP|AZURE]",
"pfc_enable": "3,4",
"pfc_to_queue_map": "[MAP_PFC_PRIORITY_TO_QUEUE|AZURE]",
"tc_to_pg_map": "[TC_TO_PRIORITY_GROUP_MAP|AZURE]",
"tc_to_queue_map": "[TC_TO_QUEUE_MAP|AZURE]"
},

This format is not consistent with other DB schema followed in sonic.
And also this reference in DB is not required, This is taken care by YANG "leafref".

Removed this format from all platform files to consistent with other sonic db schema.
Example:
"Ethernet92|3": {
"scheduler": "scheduler.1",
"wred_profile": "AZURE_LOSSLESS"
},

Dependent pull requests:
#7752 - To modify platfrom files
#7281 - Yang model
sonic-net/sonic-utilities#1626 - DB migration
sonic-net/sonic-swss#1754 - swss change to remove ABNF format
@stephenxs
Copy link
Collaborator

@stephenxs, @neethajohn - Please relook, i did modification to pass swss test cases until sonic-buildimage check in happens. Plan is to push ASAP and update sub modules.

Hi @AshokDaparthi
Now the PR sonic-net/sonic-buildimage#7752 has been merged. Can you open a PR to remove the code added for passing the CI?
Thanks

@AshokDaparthi
Copy link
Contributor Author

@stephenxs - Yes, i can do. I am waiting for build to be ready with today sonic-buildimage changes. i did not see this yet in artifacts. only docker-sonic-vs.gz comes out of today image only checks will pass. So i will rise it tomorrow.

neethajohn pushed a commit that referenced this pull request Sep 30, 2021
What I did
Reverted skipped test_buffer_dynamic as part of #1754

Why I did it

How I verified it sudo pytest --dvsname=vs --forcedvs -sv --keeptb test_buffer_dynamic.py
======================================================= test session starts ========================================================
platform linux -- Python 3.6.9, pytest-6.2.4, py-1.10.0, pluggy-0.13.1 -- /usr/bin/python3
cachedir: .pytest_cache
rootdir: /home/ashokd/swss-vs/ashok-swss/sonic-swss/tests
plugins: flaky-3.7.0
collected 9 items

test_buffer_dynamic.py::TestBufferMgrDyn::test_changeSpeed remove extra link dummy PASSED
test_buffer_dynamic.py::TestBufferMgrDyn::test_changeCableLen PASSED
test_buffer_dynamic.py::TestBufferMgrDyn::test_MultipleLosslessPg PASSED
test_buffer_dynamic.py::TestBufferMgrDyn::test_headroomOverride PASSED
test_buffer_dynamic.py::TestBufferMgrDyn::test_mtuUpdate PASSED
test_buffer_dynamic.py::TestBufferMgrDyn::test_nonDefaultAlpha PASSED
test_buffer_dynamic.py::TestBufferMgrDyn::test_sharedHeadroomPool PASSED
test_buffer_dynamic.py::TestBufferMgrDyn::test_shutdownPort PASSED
test_buffer_dynamic.py::TestBufferMgrDyn::test_autoNegPort PASSED
raphaelt-nvidia pushed a commit to raphaelt-nvidia/sonic-swss that referenced this pull request Oct 5, 2021
* QOS field reference ABNF format to string changes
raphaelt-nvidia pushed a commit to raphaelt-nvidia/sonic-swss that referenced this pull request Oct 5, 2021
What I did
Reverted skipped test_buffer_dynamic as part of sonic-net#1754

Why I did it

How I verified it sudo pytest --dvsname=vs --forcedvs -sv --keeptb test_buffer_dynamic.py
======================================================= test session starts ========================================================
platform linux -- Python 3.6.9, pytest-6.2.4, py-1.10.0, pluggy-0.13.1 -- /usr/bin/python3
cachedir: .pytest_cache
rootdir: /home/ashokd/swss-vs/ashok-swss/sonic-swss/tests
plugins: flaky-3.7.0
collected 9 items

test_buffer_dynamic.py::TestBufferMgrDyn::test_changeSpeed remove extra link dummy PASSED
test_buffer_dynamic.py::TestBufferMgrDyn::test_changeCableLen PASSED
test_buffer_dynamic.py::TestBufferMgrDyn::test_MultipleLosslessPg PASSED
test_buffer_dynamic.py::TestBufferMgrDyn::test_headroomOverride PASSED
test_buffer_dynamic.py::TestBufferMgrDyn::test_mtuUpdate PASSED
test_buffer_dynamic.py::TestBufferMgrDyn::test_nonDefaultAlpha PASSED
test_buffer_dynamic.py::TestBufferMgrDyn::test_sharedHeadroomPool PASSED
test_buffer_dynamic.py::TestBufferMgrDyn::test_shutdownPort PASSED
test_buffer_dynamic.py::TestBufferMgrDyn::test_autoNegPort PASSED
EdenGri pushed a commit to EdenGri/sonic-swss that referenced this pull request Feb 28, 2022
…nic-net#1754)

What I did
Allow system with no ports in config db run without errors.
This is needed for modular system which should boot properly without line cards.

How I did it
Do not raise warning if ports dictionary is empty.

How to verify it
Run show interfaces status
malletvapid23 added a commit to malletvapid23/Sonic-Utility that referenced this pull request Aug 3, 2023
Qos tables in config db and app db used ABNF format i.e "[TABLE_NAME|name] to refer fieldvalue other qos tables.

Example:
Config DB:
"Ethernet92|3": {
"scheduler": "[SCHEDULER|scheduler.1]",
"wred_profile": "[WRED_PROFILE|AZURE_LOSSLESS]"
},
"Ethernet0|0": {
"profile": "[BUFFER_PROFILE|ingress_lossy_profile]"
},
"Ethernet0": {
"dscp_to_tc_map": "[DSCP_TO_TC_MAP|AZURE]",
"pfc_enable": "3,4",
"pfc_to_queue_map": "[MAP_PFC_PRIORITY_TO_QUEUE|AZURE]",
"tc_to_pg_map": "[TC_TO_PRIORITY_GROUP_MAP|AZURE]",
"tc_to_queue_map": "[TC_TO_QUEUE_MAP|AZURE]"
},

AppDB:
"BUFFER_QUEUE_TABLE:Ethernet88:3-4": {
"profile": "[BUFFER_PROFILE_TABLE:egress_lossless_profile]"
},

1#This format is not consistent with other DB schema followed in sonic.
2# Added db_migrator.py case to  change from old format in config_db and appl_db  to new format. 
3#Modified the test case 

Dependent pull requests: 
sonic-net/sonic-buildimage#7752  - To modify platfrom files 
sonic-net/sonic-buildimage#7281 - Yang model 
sonic-net/sonic-swss#1754    - swss change to remove ABNF format
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

8 participants