Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Use device loopback IP address to send SNMP query from neighboring ceos or vsonic #8802

Merged
merged 13 commits into from
Jul 10, 2023

Conversation

SuvarnaMeenakshi
Copy link
Contributor

@SuvarnaMeenakshi SuvarnaMeenakshi commented Jul 3, 2023

Description of PR

Summary:
Fixes # (issue)

Type of change

  • Bug fix
  • Testbed and Framework(new/improvement)
  • Test case(new/improvement)

Back port request

  • 201911
  • 202012
  • 202205

Approach

What is the motivation for this PR?

sonic-net/sonic-buildimage#15487 modifies snmpd in SONiC to listen on management and loopback ip by default instead of listening on any IP.
test_interop_protocol.py::test_snmp sends a query to the neighbor device using the link ip address.
After the above change to listen on management and loopback ip, the snmp query will fail.
Hence, modifying the test to send SNMP query to SONiC DUT from neighboring vsonic/eos using Loopback IP of SONiC DUT.

How did you do it?

Get loopback IPv4 address DUT and use that to query from neighbor device.
Added a generic helped function to send SNMP query to DUT from a neighbor.

How did you verify/test it?

Any platform specific information?

Supported testbed topology if it's a new test case?

Documentation

Signed-off-by: Suvarna Meenakshi <sumeenak@microsoft.com>
@mssonicbld
Copy link
Collaborator

The pre-commit check detected issues in the files touched by this pull request.
The pre-commit check is a mandatory check, please fix detected issues.

Detailed pre-commit check results:
trim trailing whitespace.................................................Failed
- hook id: trailing-whitespace
- exit code: 1
- files were modified by this hook

Fixing tests/macsec/test_interop_protocol.py

fix end of files.........................................................Passed
check yaml...........................................(no files to check)Skipped
check for added large files..............................................Passed
check python ast.........................................................Passed
flake8...................................................................Failed
- hook id: flake8
- exit code: 1

tests/macsec/test_interop_protocol.py:136:26: F821 undefined name 'up_link'
tests/macsec/test_interop_protocol.py:141:38: E128 continuation line under-indented for visual indent
tests/macsec/test_interop_protocol.py:150:13: F841 local variable 'up_link' is assigned to but never used

check conditional mark sort..........................(no files to check)Skipped

To run the pre-commit checks locally, you can follow below steps:

  1. Ensure that default python is python3. In sonic-mgmt docker container, default python is python2. You can run
    the check by activating the python3 virtual environment in sonic-mgmt docker container or outside of sonic-mgmt
    docker container.
  2. Ensure that the pre-commit package is installed:
sudo pip install pre-commit
  1. Go to repository root folder
  2. Install the pre-commit hooks:
pre-commit install
  1. Use pre-commit to check staged file:
pre-commit
  1. Alternatively, you can check committed files using:
pre-commit run --from-ref <commit_id> --to-ref <commit_id>

Signed-off-by: Suvarna Meenakshi <sumeenak@microsoft.com>
@mssonicbld
Copy link
Collaborator

The pre-commit check detected issues in the files touched by this pull request.
The pre-commit check is a mandatory check, please fix detected issues.

Detailed pre-commit check results:
trim trailing whitespace.................................................Passed
fix end of files.........................................................Passed
check yaml...........................................(no files to check)Skipped
check for added large files..............................................Passed
check python ast.........................................................Passed
flake8...................................................................Failed
- hook id: flake8
- exit code: 1

tests/macsec/test_interop_protocol.py:142:68: E211 whitespace before '['
tests/macsec/test_interop_protocol.py:143:51: E127 continuation line over-indented for visual indent

check conditional mark sort..........................(no files to check)Skipped

To run the pre-commit checks locally, you can follow below steps:

  1. Ensure that default python is python3. In sonic-mgmt docker container, default python is python2. You can run
    the check by activating the python3 virtual environment in sonic-mgmt docker container or outside of sonic-mgmt
    docker container.
  2. Ensure that the pre-commit package is installed:
sudo pip install pre-commit
  1. Go to repository root folder
  2. Install the pre-commit hooks:
pre-commit install
  1. Use pre-commit to check staged file:
pre-commit
  1. Alternatively, you can check committed files using:
pre-commit run --from-ref <commit_id> --to-ref <commit_id>

Signed-off-by: Suvarna Meenakshi <sumeenak@microsoft.com>
loopback ip

Signed-off-by: Suvarna Meenakshi <sumeenak@microsoft.com>
@mssonicbld
Copy link
Collaborator

The pre-commit check detected issues in the files touched by this pull request.
The pre-commit check is a mandatory check, please fix detected issues.

Detailed pre-commit check results:
trim trailing whitespace.................................................Passed
fix end of files.........................................................Passed
check yaml...........................................(no files to check)Skipped
check for added large files..............................................Passed
check python ast.........................................................Failed
- hook id: check-ast
- exit code: 1

tests/macsec/test_interop_protocol.py: failed parsing with CPython 3.8.10:

Traceback (most recent call last):
File "/home/AzDevOps/.cache/pre-commit/repoyaq743u2/py_env-python3/lib/python3.8/site-packages/pre_commit_hooks/check_ast.py", line 21, in main
ast.parse(f.read(), filename=filename)
File "/usr/lib/python3.8/ast.py", line 47, in parse
return compile(source, filename, mode, flags,
File "tests/macsec/test_interop_protocol.py", line 162
assert not duthost.command(command)["failed"]
^
SyntaxError: invalid syntax
...
[truncated extra lines, please run pre-commit locally to view full check results]

To run the pre-commit checks locally, you can follow below steps:

  1. Ensure that default python is python3. In sonic-mgmt docker container, default python is python2. You can run
    the check by activating the python3 virtual environment in sonic-mgmt docker container or outside of sonic-mgmt
    docker container.
  2. Ensure that the pre-commit package is installed:
sudo pip install pre-commit
  1. Go to repository root folder
  2. Install the pre-commit hooks:
pre-commit install
  1. Use pre-commit to check staged file:
pre-commit
  1. Alternatively, you can check committed files using:
pre-commit run --from-ref <commit_id> --to-ref <commit_id>

Signed-off-by: Suvarna Meenakshi <sumeenak@microsoft.com>
@mssonicbld
Copy link
Collaborator

The pre-commit check detected issues in the files touched by this pull request.
The pre-commit check is a mandatory check, please fix detected issues.

Detailed pre-commit check results:
trim trailing whitespace.................................................Passed
fix end of files.........................................................Passed
check yaml...........................................(no files to check)Skipped
check for added large files..............................................Passed
check python ast.........................................................Passed
flake8...................................................................Failed
- hook id: flake8
- exit code: 1

tests/macsec/test_interop_protocol.py:158:40: F821 undefined name 'SonicHost'
tests/macsec/test_interop_protocol.py:161:38: E127 continuation line over-indented for visual indent

check conditional mark sort..........................(no files to check)Skipped

To run the pre-commit checks locally, you can follow below steps:

  1. Ensure that default python is python3. In sonic-mgmt docker container, default python is python2. You can run
    the check by activating the python3 virtual environment in sonic-mgmt docker container or outside of sonic-mgmt
    docker container.
  2. Ensure that the pre-commit package is installed:
sudo pip install pre-commit
  1. Go to repository root folder
  2. Install the pre-commit hooks:
pre-commit install
  1. Use pre-commit to check staged file:
pre-commit
  1. Alternatively, you can check committed files using:
pre-commit run --from-ref <commit_id> --to-ref <commit_id>

Signed-off-by: Suvarna Meenakshi <sumeenak@microsoft.com>
@mssonicbld
Copy link
Collaborator

The pre-commit check detected issues in the files touched by this pull request.
The pre-commit check is a mandatory check, please fix detected issues.

Detailed pre-commit check results:
trim trailing whitespace.................................................Passed
fix end of files.........................................................Passed
check yaml...........................................(no files to check)Skipped
check for added large files..............................................Passed
check python ast.........................................................Passed
flake8...................................................................Failed
- hook id: flake8
- exit code: 1

tests/macsec/test_interop_protocol.py:161:38: E127 continuation line over-indented for visual indent

check conditional mark sort..........................(no files to check)Skipped

To run the pre-commit checks locally, you can follow below steps:

  1. Ensure that default python is python3. In sonic-mgmt docker container, default python is python2. You can run
    the check by activating the python3 virtual environment in sonic-mgmt docker container or outside of sonic-mgmt
    docker container.
  2. Ensure that the pre-commit package is installed:
sudo pip install pre-commit
  1. Go to repository root folder
  2. Install the pre-commit hooks:
pre-commit install
  1. Use pre-commit to check staged file:
pre-commit
  1. Alternatively, you can check committed files using:
pre-commit run --from-ref <commit_id> --to-ref <commit_id>

Signed-off-by: Suvarna Meenakshi <sumeenak@microsoft.com>
Signed-off-by: Suvarna Meenakshi <sumeenak@microsoft.com>
@abdosi
Copy link
Contributor

abdosi commented Jul 5, 2023

@SuvarnaMeenakshi can we remove this check now ?

if duthost.is_multi_asic: pytest.skip("The test is for Single ASIC devices")

I wanted to make sure it run on multi-asic platforms also. Since we are using loobpack ip i think it should work

@SuvarnaMeenakshi
Copy link
Contributor Author

@SuvarnaMeenakshi can we remove this check now ?

if duthost.is_multi_asic: pytest.skip("The test is for Single ASIC devices")

I wanted to make sure it run on multi-asic platforms also. Since we are using loobpack ip i think it should work

Sure, will test this out on multi-asic and make the change.
snmp should have worked even with link ip on multi-asic, not sure why the skip was added.

@SuvarnaMeenakshi
Copy link
Contributor Author

@SuvarnaMeenakshi can we remove this check now ?
if duthost.is_multi_asic: pytest.skip("The test is for Single ASIC devices")
I wanted to make sure it run on multi-asic platforms also. Since we are using loobpack ip i think it should work

Sure, will test this out on multi-asic and make the change. snmp should have worked even with link ip on multi-asic, not sure why the skip was added.

@abdosi this will require additional work as this test is sending snmp query from multi-asic dut to neighbor vsonic.
From multi-asic DUT, the loopback or link ip to neighbor vsonic/ceos is not reachable that is why it is skipped here, it is reachable from front-end asic namespace.

admin@vlab-08:~$ ping 100.1.0.1
PING 100.1.0.1 (100.1.0.1) 56(84) bytes of data.
^C
--- 100.1.0.1 ping statistics ---
2 packets transmitted, 0 received, 100% packet loss, time 1012ms

admin@vlab-08:~$ sudo ip netns exec asic0 ping 100.1.0.1
PING 100.1.0.1 (100.1.0.1) 56(84) bytes of data.
64 bytes from 100.1.0.1: icmp_seq=1 ttl=64 time=1.12 ms
64 bytes from 100.1.0.1: icmp_seq=2 ttl=64 time=1.11 ms

--- 100.1.0.1 ping statistics ---
2 packets transmitted, 2 received, 0% packet loss, time 1001ms
rtt min/avg/max/mdev = 1.107/1.111/1.115/0.004 ms

I will raise a github issue to fix this and enable this test on multi-asic/
Can we keep this PR scope to ensure current test case works as is.

@SuvarnaMeenakshi
Copy link
Contributor Author

SuvarnaMeenakshi commented Jul 5, 2023

test is sending snmp query from multi-asic dut to neighbor vsonic. From multi-asic DUT, the loopback or link ip to neighbor vsonic/ceos is not reachable that is why it is skipped here, it is reachable from front-end asic namespace.

admin@vlab-08:~$ ping 100.1.0.1
PING 100.1.0.1 (100.1.0.1) 56(84) bytes of data.
^C
--- 100.1.0.1 ping statistics ---
2 packets transmitted, 0 received, 100% packet loss, time 1012ms

admin@vlab-08:~$ sudo ip netns exec asic0 ping 100.1.0.1
PING 100.1.0.1 (100.1.0.1) 56(84) bytes of data.
64 bytes from 100.1.0.1: icmp_seq=1 ttl=64 time=1.12 ms
64 bytes from 100.1.0.1: icmp_seq=2 ttl=64 time=1.11 ms

--- 100.1.0.1 ping statistics ---
2 packets transmitted, 2 received, 0% packet loss, time 1001ms
rtt min/avg/max/mdev = 1.107/1.111/1.115/0.004 ms

Raised issue: #8817

else: # vsonic neighbour
community = creds['snmp_rocommunity']
# Use LoopbackIP to query from vsonic neighbor
Copy link
Contributor

@qiluo-msft qiluo-msft Jul 6, 2023

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

In the code comment, could you be more specific about ipv4 or ipv6? #Closed

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Fixed

"LOOPBACK_INTERFACE",
{}).get('Loopback0', {})
for ip in result:
if isinstance(ipaddress.ip_address(ip.split('/')[0]),
Copy link
Contributor

@qiluo-msft qiluo-msft Jul 6, 2023

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

No need to split, you can directly parse ipaddress.ip_network #Closed

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Fixed

"ansible_facts"].get(
"LOOPBACK_INTERFACE",
{}).get('Loopback0', {})
for ip in result:
Copy link
Contributor

@qiluo-msft qiluo-msft Jul 6, 2023

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

for

You may use for-else to capture the situation that no ip satisfy the condition. In the else block, you can explicitly fail the test. #Closed

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Fixed

Signed-off-by: Suvarna Meenakshi <sumeenak@microsoft.com>
from nbr device over loopback ipv4 IP.

Signed-off-by: Suvarna Meenakshi <sumeenak@microsoft.com>
@mssonicbld
Copy link
Collaborator

The pre-commit check detected issues in the files touched by this pull request.
The pre-commit check is a mandatory check, please fix detected issues.

Detailed pre-commit check results:
trim trailing whitespace.................................................Failed
- hook id: trailing-whitespace
- exit code: 1
- files were modified by this hook

Fixing tests/common/helpers/snmp_helpers.py
Fixing tests/macsec/test_interop_protocol.py

fix end of files.........................................................Passed
check yaml...........................................(no files to check)Skipped
check for added large files..............................................Passed
check python ast.........................................................Passed
flake8...................................................................Failed
- hook id: flake8
- exit code: 1

tests/common/helpers/snmp_helpers.py:77:5: E303 too many blank lines (2)
tests/macsec/test_interop_protocol.py:3:1: F401 're' imported but unused
tests/macsec/test_interop_protocol.py:131:9: E303 too many blank lines (2)
tests/macsec/test_interop_protocol.py:132:43: E128 continuation line under-indented for visual indent
...
[truncated extra lines, please run pre-commit locally to view full check results]

To run the pre-commit checks locally, you can follow below steps:

  1. Ensure that default python is python3. In sonic-mgmt docker container, default python is python2. You can run
    the check by activating the python3 virtual environment in sonic-mgmt docker container or outside of sonic-mgmt
    docker container.
  2. Ensure that the pre-commit package is installed:
sudo pip install pre-commit
  1. Go to repository root folder
  2. Install the pre-commit hooks:
pre-commit install
  1. Use pre-commit to check staged file:
pre-commit
  1. Alternatively, you can check committed files using:
pre-commit run --from-ref <commit_id> --to-ref <commit_id>

@yejianquan
Copy link
Collaborator

LGTM, @Pterosaur , can you also confirm?

@yejianquan yejianquan requested a review from Pterosaur July 7, 2023 02:28
@Pterosaur
Copy link
Contributor

lgtm

Signed-off-by: Suvarna Meenakshi <sumeenak@microsoft.com>
@mssonicbld
Copy link
Collaborator

The pre-commit check detected issues in the files touched by this pull request.
The pre-commit check is a mandatory check, please fix detected issues.

Detailed pre-commit check results:
trim trailing whitespace.................................................Failed
- hook id: trailing-whitespace
- exit code: 1
- files were modified by this hook

Fixing tests/common/helpers/snmp_helpers.py

fix end of files.........................................................Passed
check yaml...........................................(no files to check)Skipped
check for added large files..............................................Passed
check python ast.........................................................Passed
flake8...................................................................Passed
check conditional mark sort..........................(no files to check)Skipped

To run the pre-commit checks locally, you can follow below steps:

  1. Ensure that default python is python3. In sonic-mgmt docker container, default python is python2. You can run
    the check by activating the python3 virtual environment in sonic-mgmt docker container or outside of sonic-mgmt
    docker container.
  2. Ensure that the pre-commit package is installed:
sudo pip install pre-commit
  1. Go to repository root folder
  2. Install the pre-commit hooks:
pre-commit install
  1. Use pre-commit to check staged file:
pre-commit
  1. Alternatively, you can check committed files using:
pre-commit run --from-ref <commit_id> --to-ref <commit_id>

Signed-off-by: Suvarna Meenakshi <sumeenak@microsoft.com>
@SuvarnaMeenakshi
Copy link
Contributor Author

@abdosi modified the macsec/test_interop_protocol.py test_snmp to send SNMP query from neighboring vsonic/ceos devices to SONiC DUT. This way we can have the test running on multi-asic device as well.

While testing out on multi-asic device, found that there might be additional changes required.
So will keep the skip for multi-asic as is in this PR and raise another PR with changes required for multi-asic.

@SuvarnaMeenakshi SuvarnaMeenakshi changed the title Use loopback IP address to send SNMP query to vsonic neighbor Use device loopback IP address to send SNMP query from neighboring ceos or vsonic Jul 7, 2023
@mssonicbld
Copy link
Collaborator

@SuvarnaMeenakshi PR conflicts with 202205 branch

SuvarnaMeenakshi added a commit to SuvarnaMeenakshi/sonic-mgmt that referenced this pull request Jul 13, 2023
…os or vsonic (sonic-net#8802)

What is the motivation for this PR?
sonic-net/sonic-buildimage#15487 modifies snmpd in SONiC to listen on management and loopback ip by default instead of listening on any IP.
test_interop_protocol.py::test_snmp sends a query to the neighbor device using the link ip address.
After the above change to listen on management and loopback ip, the snmp query will fail.
Hence, modifying the test to send SNMP query to SONiC DUT from neighboring vsonic/eos using Loopback IP of SONiC DUT.

How did you do it?
Get loopback IPv4 address DUT and use that to query from neighbor device.
Added a generic helped function to send SNMP query to DUT from a neighbor.

(cherry picked from commit d66965a)
wangxin pushed a commit that referenced this pull request Jul 14, 2023
…oring ceos or vsonic (#8802) (#8972)

What is the motivation for this PR?
cherry-pick of #8802
sonic-net/sonic-buildimage#15487 modifies snmpd in SONiC to listen on management and loopback ip by default instead of listening on any IP. test_interop_protocol.py::test_snmp sends a query to the neighbor device using the link ip address. After the above change to listen on management and loopback ip, the snmp query will fail. Hence, modifying the test to send SNMP query to SONiC DUT from neighboring vsonic/eos using Loopback IP of SONiC DUT.

How did you do it?
Get loopback IPv4 address DUT and use that to query from neighbor device. Added a generic helped function to send SNMP query to DUT from a neighbor.

(cherry picked from commit d66965a)

How did you verify/test it?
Tested on single asic VS testbed.

Signed-off-by: Suvarna Meenakshi <sumeenak@microsoft.com>
AharonMalkin pushed a commit to AharonMalkin/sonic-mgmt that referenced this pull request Jan 25, 2024
…os or vsonic (sonic-net#8802)

What is the motivation for this PR?
sonic-net/sonic-buildimage#15487 modifies snmpd in SONiC to listen on management and loopback ip by default instead of listening on any IP.
test_interop_protocol.py::test_snmp sends a query to the neighbor device using the link ip address.
After the above change to listen on management and loopback ip, the snmp query will fail.
Hence, modifying the test to send SNMP query to SONiC DUT from neighboring vsonic/eos using Loopback IP of SONiC DUT.

How did you do it?
Get loopback IPv4 address DUT and use that to query from neighbor device.
Added a generic helped function to send SNMP query to DUT from a neighbor.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging this pull request may close these issues.

7 participants