Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[sonic-utilities] Update submodule; Build and install as a Python 3 wheel #5926

Merged
merged 24 commits into from
Nov 25, 2020
Merged

[sonic-utilities] Update submodule; Build and install as a Python 3 wheel #5926

merged 24 commits into from
Nov 25, 2020

Conversation

jleveque
Copy link
Contributor

@jleveque jleveque commented Nov 15, 2020

Submodule updates include the following commits:

  • Also fix Python 3-related issues:
    • Use integer (floor) division in config_samples.py (sonic-config-engine)
    • Replace print statement with print function in eeprom.py plugin for x86_64-kvm_x86_64-r0 platform
    • Update all platform plugins to be compatible with both Python 2 and Python 3
    • Remove shebangs from plugins files which are not intended to be executable
    • Replace tabs with spaces in Python plugin files and fix alignment, because Python 3 is more strict
    • Remove trailing whitespace from plugins files

@jleveque jleveque self-assigned this Nov 15, 2020
@jleveque jleveque changed the title [sonic-utilities] Update submodule; Build and install sonic-utilities as a Python 3 wheel [sonic-utilities] Update submodule; Build and install as a Python 3 wheel Nov 16, 2020
lguohan
lguohan previously approved these changes Nov 16, 2020
@lgtm-com

This comment has been minimized.

@jleveque
Copy link
Contributor Author

@tahmed-dev: Please review. A few fixes were needed in sonic-config-engine.

lguohan
lguohan previously approved these changes Nov 18, 2020
@lguohan lguohan linked an issue Nov 18, 2020 that may be closed by this pull request
tahmed-dev
tahmed-dev previously approved these changes Nov 18, 2020
@lguohan
Copy link
Collaborator

lguohan commented Nov 18, 2020

looks like error for load_minigraph, can you check?

01:26:20 TASK [execute cli "config load_minigraph -y" to apply new minigraph] ***********
01:26:20 Wednesday 18 November 2020 09:26:20 +0000 (0:00:00.062) 0:00:23.834 ****
01:26:48 fatal: [vlab-01]: FAILED! => {"changed": true, "cmd": "config load_minigraph -y", "delta": "0:00:25.419355", "end": "2020-11-18 09:26:46.044793", "msg": "non-zero return code", "rc": 1, "start": "2020-11-18 09:26:20.625438", "stderr": "Warning: Stopping telemetry.service, but it can still be activated by:\n telemetry.timer\nTraceback (most recent call last):\n File "/usr/local/bin/sonic-cfggen", line 432, in \n main()\n File "/usr/local/bin/sonic-cfggen", line 361, in main\n deep_update(data, FormatConverter.db_to_output(configdb.get_config()))\n File "/usr/local/lib/python3.7/dist-packages/swsssdk/configdb.py", line 443, in get_config\n cur = self.__get_config(client, pipe, data, 0)\n File "/usr/local/lib/python3.7/dist-packages/swsssdk/configdb.py", line 416, in __get_config\n keys = [key.decode() for key in keys if key != self.INIT_INDICATOR]\n File "/usr/local/lib/python3.7/dist-packages/swsssdk/configdb.py", line 416, in \n keys = [key.decode() for key in keys if key != self.INIT_INDICATOR]\nAttributeError: 'str' object has no attribute 'decode'", "stderr_lines": ["Warning: Stopping telemetry.service, but it can still be activated by:", " telemetry.timer", "Traceback (most recent call last):", " File "/usr/local/bin/sonic-cfggen", line 432, in ", " main()", " File "/usr/local/bin/sonic-cfggen", line 361, in main", " deep_update(data, FormatConverter.db_to_output(configdb.get_config()))", " File "/usr/local/lib/python3.7/dist-packages/swsssdk/configdb.py", line 443, in get_config", " cur = self.__get_config(client, pipe, data, 0)", " File "/usr/local/lib/python3.7/dist-packages/swsssdk/configdb.py", line 416, in __get_config", " keys = [key.decode() for key in keys if key != self.INIT_INDICATOR]", " File "/usr/local/lib/python3.7/dist-packages/swsssdk/configdb.py", line 416, in ", " keys = [key.decode() for key in keys if key != self.INIT_INDICATOR]", "AttributeError: 'str' object has no attribute 'decode'"], "stdout": "Executing stop of service telemetry...\nExecuting stop of service swss...\nExecuting stop of service lldp...\nExecuting stop of service pmon...\nExecuting stop of service bgp...\nRunning command: /usr/local/bin/sonic-cfggen -H -m -j /etc/sonic/init_cfg.json --write-to-db\nRunning command: pfcwd start_default\nRunning command: /usr/local/bin/sonic-cfggen -d -y /etc/sonic/sonic_version.yml -t /usr/share/sonic/templates/sonic-environment.j2,/etc/sonic/sonic-environment", "stdout_lines": ["Executing stop of service telemetry...", "Executing stop of service swss...", "Executing stop of service lldp...", "Executing stop of service pmon...", "Executing stop of service bgp...", "Running command: /usr/local/bin/sonic-cfggen -H -m -j /etc/sonic/init_cfg.json --write-to-db", "Running command: pfcwd start_default", "Running command: /usr/local/bin/sonic-cfggen -d -y /etc/sonic/sonic_version.yml -t /usr/share/sonic/templates/sonic-environment.j2,/etc/sonic/sonic-environment"]}

@jleveque
Copy link
Contributor Author

@qiluo-msft: vsimage build is failing with the following swsssdk error. Can you please take a look?

Traceback (most recent call last):
  File "/usr/local/bin/sonic-cfggen", line 432, in <module>
    main()
  File "/usr/local/bin/sonic-cfggen", line 361, in main
    deep_update(data, FormatConverter.db_to_output(configdb.get_config()))
  File "/usr/local/lib/python3.7/dist-packages/swsssdk/configdb.py", line 443, in get_config
    cur = self.__get_config(client, pipe, data, 0)
  File "/usr/local/lib/python3.7/dist-packages/swsssdk/configdb.py", line 416, in __get_config
    keys = [key.decode() for key in keys if key != self.INIT_INDICATOR]
  File "/usr/local/lib/python3.7/dist-packages/swsssdk/configdb.py", line 416, in <listcomp>
    keys = [key.decode() for key in keys if key != self.INIT_INDICATOR]
AttributeError: 'str' object has no attribute 'decode'

@lguohan
Copy link
Collaborator

lguohan commented Nov 18, 2020

must be some python3 issue, str does not have decode method

@jleveque
Copy link
Contributor Author

must be some python3 issue, str does not have decode method

Yes. It appears to be a Python 3 swsssdk error.

@jleveque jleveque dismissed stale reviews from tahmed-dev and lguohan via fefcddf November 18, 2020 20:55
@lgtm-com

This comment has been minimized.

@lgtm-com

This comment has been minimized.

@qiluo-msft
Copy link
Collaborator

AttributeError: 'str' object has no attribute 'decode'

It is fixed by sonic-net/sonic-py-swsssdk#96

lguohan pushed a commit to sonic-net/sonic-utilities that referenced this pull request Nov 20, 2020
In Python 3, sort() takes no positional arguments, only keyword arguments. Fix acl-loader to comply.

Fixes vsimage build failure in sonic-net/sonic-buildimage#5926
@lgtm-com

This comment has been minimized.

@lgtm-com

This comment has been minimized.

This reverts commit 1a3233c106c4028f2c23bdbe145b02d196906ab2.
* src/sonic-utilities f46c27e...9d55082 (3):
  > [configlet][portconfig] Remove calls to dict.has_key() which is not available in Python 3 (#1247)
  > Remove unnecessary conversions to list() and calls to dict.keys() (#1243)
  > Clean up LGTM alerts (#1239)
@lgtm-com
Copy link

lgtm-com bot commented Nov 25, 2020

This pull request fixes 69 alerts when merging 3fab9f4 into dc15fbc - view on LGTM.com

fixed alerts:

  • 63 for Unused import
  • 4 for Result of integer division may be truncated
  • 2 for Unused local variable

@jleveque jleveque merged commit 7f4ab8f into sonic-net:master Nov 25, 2020
@jleveque jleveque deleted the utilities_py3 branch November 25, 2020 18:28
santhosh-kt pushed a commit to santhosh-kt/sonic-buildimage that referenced this pull request Feb 25, 2021
…heel (sonic-net#5926)

Submodule updates include the following commits:

* src/sonic-utilities 9dc58ea...f9eb739 (18):
  > Remove unnecessary calls to str.encode() now that the package is Python 3; Fix deprecation warning (sonic-net#1260)
  > [generate_dump] Ignoring file/directory not found Errors (sonic-net#1201)
  > Fixed porstat rate and util issues (sonic-net#1140)
  > fix error: interface counters is mismatch after warm-reboot (sonic-net#1099)
  > Remove unnecessary calls to str.decode() now that the package is Python 3 (sonic-net#1255)
  > [acl-loader] Make list sorting compliant with Python 3 (sonic-net#1257)
  > Replace hard-coded fast-reboot with variable. And some typo corrections (sonic-net#1254)
  > [configlet][portconfig] Remove calls to dict.has_key() which is not available in Python 3 (sonic-net#1247)
  > Remove unnecessary conversions to list() and calls to dict.keys() (sonic-net#1243)
  > Clean up LGTM alerts (sonic-net#1239)
  > Add 'requests' as install dependency in setup.py (sonic-net#1240)
  > Convert to Python 3 (sonic-net#1128)
  > Fix mock SonicV2Connector in python3: use decode_responses mode so caller code will be the same as python2 (sonic-net#1238)
  > [tests] Do not trim from PATH if we did not append to it; Clean up/fix shebangs in scripts (sonic-net#1233)
  > Updates to bgp config and show commands with BGP_INTERNAL_NEIGHBOR table (sonic-net#1224)
  > [cli]: NAT show commands newline issue after migrated to Python3 (sonic-net#1204)
  > [doc]: Update Command-Reference.md (sonic-net#1231)
  > Added 'import sys' in feature.py file (sonic-net#1232)

* src/sonic-py-swsssdk 9d9f0c6...1664be9 (2):
  > Fix: no need to decode() after redis client scan, so it will work for both python2 and python3 (sonic-net#96)
  > FieldValueMap `contains`(`in`)  will also work when migrated to libswsscommon(C++ with SWIG wrapper) (sonic-net#94)

- Also fix Python 3-related issues:
    - Use integer (floor) division in config_samples.py (sonic-config-engine)
    - Replace print statement with print function in eeprom.py plugin for x86_64-kvm_x86_64-r0 platform
    - Update all platform plugins to be compatible with both Python 2 and Python 3
    - Remove shebangs from plugins files which are not intended to be executable
    - Replace tabs with spaces in Python plugin files and fix alignment, because Python 3 is more strict
    - Remove trailing whitespace from plugins files
malletvapid23 added a commit to malletvapid23/Sonic-Utility that referenced this pull request Aug 3, 2023
In Python 3, sort() takes no positional arguments, only keyword arguments. Fix acl-loader to comply.

Fixes vsimage build failure in sonic-net/sonic-buildimage#5926
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging this pull request may close these issues.

Unable to install sonic_utilities-1.2-py3-none-any.whl with Build 492
4 participants