Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Update pylint version in order to support python 3.12 #5338

Merged
merged 2 commits into from
May 28, 2024

Conversation

ani-sinha
Copy link
Contributor

Fedora 39 and above comes with python version 3.12. When running tox -e pylint on cloud-init, we may experience issue such as the one reported here: pylint-dev/pylint#8782

Minimum version of pylint required in order to support python 3.12 is 3.0.2. Please see pylint-dev/astroid#2201 . Upon further experimentation, it is seen that we need minimum pylint version 3.2.0 for cloud-init. Update tox.ini in order to use this pylint version.

@ani-sinha
Copy link
Contributor Author

@aciba90 This is the other issue I was talking about on IRC

@TheRealFalcon
Copy link
Member

This change makes sense, though we should fix the pylint errors before merging this. It can be done as an extra commit here or as a separate PR.

@ani-sinha
Copy link
Contributor Author

ERROR: InvocationError for command /home/runner/work/cloud-init/cloud-init/.tox/pylint/bin/python -m pylint cloudinit/ tests/ tools/ conftest.py setup.py (exited with code 6)

I do not see the actual error that returns code 6.

@holmanb
Copy link
Member

holmanb commented May 28, 2024

ERROR: InvocationError for command /home/runner/work/cloud-init/cloud-init/.tox/pylint/bin/python -m pylint cloudinit/ tests/ tools/ conftest.py setup.py (exited with code 6)

I do not see the actual error that returns code 6.

I think that's caused by the new pylint warnings that are introduced by bumping versions.

This doesn't fix all of them, but it's a start:

diff --git a/cloudinit/config/cc_mounts.py b/cloudinit/config/cc_mounts.py
index a12b2aec2..c75895c98 100644
--- a/cloudinit/config/cc_mounts.py
+++ b/cloudinit/config/cc_mounts.py
@@ -304,6 +304,10 @@ def create_swapfile(fname: str, size: str) -> None:
                 "bs=1M",
                 "count=%s" % size,
             ]
+        else:
+            raise subp.ProcessExecutionError(
+                "Missing dependency: 'dd' and 'fallocate' are not available"
+            )
 
         try:
             subp.subp(cmd, capture=True)
diff --git a/cloudinit/distros/bsd.py b/cloudinit/distros/bsd.py
index 3d4c37e32..8f4b0a80a 100644
--- a/cloudinit/distros/bsd.py
+++ b/cloudinit/distros/bsd.py
@@ -121,6 +121,8 @@ class BSD(distros.Distro):
             if not self.pkg_cmd_upgrade_prefix:
                 return
             cmd = self.pkg_cmd_upgrade_prefix
+        else:
+            cmd = []
 
         if args and isinstance(args, str):
             cmd.append(args)
diff --git a/cloudinit/distros/netbsd.py b/cloudinit/distros/netbsd.py
index e8b9bcd5b..972528c6d 100644
--- a/cloudinit/distros/netbsd.py
+++ b/cloudinit/distros/netbsd.py
@@ -12,7 +12,7 @@ import cloudinit.distros.bsd
 from cloudinit import subp, util
 
 try:
-    import crypt
+    import crypt  # pylint: disable=W4901
 
     salt = crypt.METHOD_BLOWFISH  # pylint: disable=E1101
     blowfish_hash: Any = functools.partial(
diff --git a/cloudinit/sources/DataSourceAzure.py b/cloudinit/sources/DataSourceAzure.py
index 259905db6..c6cbf4187 100644
--- a/cloudinit/sources/DataSourceAzure.py
+++ b/cloudinit/sources/DataSourceAzure.py
@@ -49,7 +49,7 @@ from cloudinit.sources.helpers.azure import (
 from cloudinit.url_helper import UrlError
 
 try:
-    import crypt
+    import crypt  # pylint: disable=W4901
 
     blowfish_hash: Any = functools.partial(
         crypt.crypt, salt=f"$6${util.rand_str(strlen=16)}"
diff --git a/cloudinit/sources/DataSourceEc2.py b/cloudinit/sources/DataSourceEc2.py
index dca60768c..f2a00f119 100644
--- a/cloudinit/sources/DataSourceEc2.py
+++ b/cloudinit/sources/DataSourceEc2.py
@@ -336,6 +336,8 @@ class DataSourceEc2(sources.DataSource):
         return None
 
     def wait_for_metadata_service(self):
+        urls = []
+        start_time = 0
         mcfg = self.ds_cfg
 
         url_params = self.get_url_params()
@@ -369,7 +371,6 @@ class DataSourceEc2(sources.DataSource):
             and self.cloud_name not in IDMSV2_SUPPORTED_CLOUD_PLATFORMS
         ):
             # if we can't get a token, use instance-id path
-            urls = []
             url2base = {}
             url_path = "{ver}/meta-data/instance-id".format(
                 ver=self.min_metadata_version
diff --git a/cloudinit/sources/DataSourceWSL.py b/cloudinit/sources/DataSourceWSL.py
index 52b0b24b9..b81298927 100644
--- a/cloudinit/sources/DataSourceWSL.py
+++ b/cloudinit/sources/DataSourceWSL.py
@@ -284,6 +284,7 @@ class DataSourceWSL(sources.DataSource):
             return False
 
         seed_dir = cloud_init_data_dir(user_home)
+        agent_data = None
         user_data: Optional[Union[dict, bytes]] = None
 
         # Load any metadata

Feel free to incorporate it.

@TheRealFalcon
Copy link
Member

I'm referring to these errors:

pylint run-test: commands[0] | /home/runner/work/cloud-init/cloud-init/.tox/pylint/bin/python -m pylint cloudinit/ tests/ tools/ conftest.py setup.py
************* Module cloudinit.distros.bsd
cloudinit/distros/bsd.py:126: [E0606(possibly-used-before-assignment), BSD.package_command] Possibly using variable 'cmd' before assignment
************* Module cloudinit.sources.DataSourceWSL
cloudinit/sources/DataSourceWSL.py:3[15](https://github.com/canonical/cloud-init/actions/runs/9269004518/job/25498897391?pr=5338#step:6:16): [E0606(possibly-used-before-assignment), DataSourceWSL._get_data] Possibly using variable 'agent_data' before assignment
************* Module cloudinit.sources.DataSourceLXD
cloudinit/sources/DataSourceLXD.py:335: [E1131(unsupported-binary-operation), MetaDataKeys] unsupported operand type(s) for |
************* Module cloudinit.sources.DataSourceEc2
cloudinit/sources/DataSourceEc2.py:406: [E0606(possibly-used-before-assignment), DataSourceEc2.wait_for_metadata_service] Possibly using variable 'start_time' before assignment
************* Module cloudinit.config.cc_mounts
cloudinit/config/cc_mounts.py:309: [E0606(possibly-used-before-assignment), create_swapfile.create_swap] Possibly using variable 'cmd' before assignment
************* Module tests.unittests.test_util
tests/unittests/test_util.py:[17](https://github.com/canonical/cloud-init/actions/runs/9269004518/job/25498897391?pr=5338#step:6:18)29: [E0606(possibly-used-before-assignment), TestRedirectOutputPreexecFn.preexec_fn] Possibly using variable 'args' before assignment
************* Module tests.unittests.sources.test_gce
tests/unittests/sources/test_gce.py:113: [E0606(possibly-used-before-assignment), TestDataSourceGCE._set_mock_metadata._request_callback] Possibly using variable 'recursive' before assignment
************* Module tests.unittests.config.test_cc_ntp
tests/unittests/config/test_cc_ntp.py:264: [E0606(possibly-used-before-assignment), TestNtp._get_expected_pools] Possibly using variable 'expected_pools' before assignment
tests/unittests/config/test_cc_ntp.py:281: [E0606(possibly-used-before-assignment), TestNtp._get_expected_servers] Possibly using variable 'expected_servers' before assignment
************* Module tests.integration_tests.conftest
tests/integration_tests/conftest.py:321: [W0135(contextmanager-generator-missing-cleanup), client] The context used in function 'client' will not be exited.
tests/integration_tests/conftest.py:330: [W0135(contextmanager-generator-missing-cleanup), module_client] The context used in function 'module_client' will not be exited.
tests/integration_tests/conftest.py:339: [W0135(contextmanager-generator-missing-cleanup), class_client] The context used in function 'class_client' will not be exited.
************* Module tests.integration_tests.util
tests/integration_tests/util.py:322: [W07[19](https://github.com/canonical/cloud-init/actions/runs/9269004518/job/25498897391?pr=5338#step:6:20)(broad-exception-raised), wait_for_cloud_init] Raising too general exception: Exception

Exit code 6 means at least one error and at least one warning

@ani-sinha
Copy link
Contributor Author

ERROR: InvocationError for command /home/runner/work/cloud-init/cloud-init/.tox/pylint/bin/python -m pylint cloudinit/ tests/ tools/ conftest.py setup.py (exited with code 6)

I do not see the actual error that returns code 6.

I think that's caused by the new pylint warnings that are introduced by bumping versions.

This doesn't fix all of them, but it's a start:

diff --git a/cloudinit/config/cc_mounts.py b/cloudinit/config/cc_mounts.py
index a12b2aec2..c75895c98 100644
--- a/cloudinit/config/cc_mounts.py
+++ b/cloudinit/config/cc_mounts.py
@@ -304,6 +304,10 @@ def create_swapfile(fname: str, size: str) -> None:
                 "bs=1M",
                 "count=%s" % size,
             ]
+        else:
+            raise subp.ProcessExecutionError(
+                "Missing dependency: 'dd' and 'fallocate' are not available"
+            )
 
         try:
             subp.subp(cmd, capture=True)
diff --git a/cloudinit/distros/bsd.py b/cloudinit/distros/bsd.py
index 3d4c37e32..8f4b0a80a 100644
--- a/cloudinit/distros/bsd.py
+++ b/cloudinit/distros/bsd.py
@@ -121,6 +121,8 @@ class BSD(distros.Distro):
             if not self.pkg_cmd_upgrade_prefix:
                 return
             cmd = self.pkg_cmd_upgrade_prefix
+        else:
+            cmd = []
 
         if args and isinstance(args, str):
             cmd.append(args)
diff --git a/cloudinit/distros/netbsd.py b/cloudinit/distros/netbsd.py
index e8b9bcd5b..972528c6d 100644
--- a/cloudinit/distros/netbsd.py
+++ b/cloudinit/distros/netbsd.py
@@ -12,7 +12,7 @@ import cloudinit.distros.bsd
 from cloudinit import subp, util
 
 try:
-    import crypt
+    import crypt  # pylint: disable=W4901
 
     salt = crypt.METHOD_BLOWFISH  # pylint: disable=E1101
     blowfish_hash: Any = functools.partial(
diff --git a/cloudinit/sources/DataSourceAzure.py b/cloudinit/sources/DataSourceAzure.py
index 259905db6..c6cbf4187 100644
--- a/cloudinit/sources/DataSourceAzure.py
+++ b/cloudinit/sources/DataSourceAzure.py
@@ -49,7 +49,7 @@ from cloudinit.sources.helpers.azure import (
 from cloudinit.url_helper import UrlError
 
 try:
-    import crypt
+    import crypt  # pylint: disable=W4901
 
     blowfish_hash: Any = functools.partial(
         crypt.crypt, salt=f"$6${util.rand_str(strlen=16)}"
diff --git a/cloudinit/sources/DataSourceEc2.py b/cloudinit/sources/DataSourceEc2.py
index dca60768c..f2a00f119 100644
--- a/cloudinit/sources/DataSourceEc2.py
+++ b/cloudinit/sources/DataSourceEc2.py
@@ -336,6 +336,8 @@ class DataSourceEc2(sources.DataSource):
         return None
 
     def wait_for_metadata_service(self):
+        urls = []
+        start_time = 0
         mcfg = self.ds_cfg
 
         url_params = self.get_url_params()
@@ -369,7 +371,6 @@ class DataSourceEc2(sources.DataSource):
             and self.cloud_name not in IDMSV2_SUPPORTED_CLOUD_PLATFORMS
         ):
             # if we can't get a token, use instance-id path
-            urls = []
             url2base = {}
             url_path = "{ver}/meta-data/instance-id".format(
                 ver=self.min_metadata_version
diff --git a/cloudinit/sources/DataSourceWSL.py b/cloudinit/sources/DataSourceWSL.py
index 52b0b24b9..b81298927 100644
--- a/cloudinit/sources/DataSourceWSL.py
+++ b/cloudinit/sources/DataSourceWSL.py
@@ -284,6 +284,7 @@ class DataSourceWSL(sources.DataSource):
             return False
 
         seed_dir = cloud_init_data_dir(user_home)
+        agent_data = None
         user_data: Optional[Union[dict, bytes]] = None
 
         # Load any metadata

Feel free to incorporate it.

Done.

ani-sinha added 2 commits May 28, 2024 20:33
Fedora 39 and above comes with python version 3.12. When running `tox -e pylint`
on cloud-init, we may experience issue such as the one reported here:
pylint-dev/pylint#8782

Minimum version of pylint required in order to support python 3.12 is 3.0.2.
Please see pylint-dev/astroid#2201 . Upon further
experimentation, it is seen that we need minimum pylint version 3.2.0 for
cloud-init. Update tox.ini in order to use this pylint version.

Signed-off-by: Ani Sinha <anisinha@redhat.com>
Pylint fixes in various files.

Signed-off-by: Ani Sinha <anisinha@redhat.com>
Copy link
Member

@TheRealFalcon TheRealFalcon left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks!

@TheRealFalcon TheRealFalcon merged commit 5ad609f into canonical:main May 28, 2024
28 of 29 checks passed
holmanb pushed a commit that referenced this pull request Jun 28, 2024
Fedora 39 and above comes with python version 3.12. When running `tox -e pylint`
on cloud-init, we may experience issue such as the one reported here:
pylint-dev/pylint#8782

Minimum version of pylint required in order to support python 3.12 is 3.0.2.
Please see pylint-dev/astroid#2201 . Upon further
experimentation, it is seen that we need minimum pylint version 3.2.0 for
cloud-init. Update tox.ini in order to use this pylint version.

Signed-off-by: Ani Sinha <anisinha@redhat.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants