Updated source for Pylint 1.5. #1248

dhermes · 2015-11-30T19:08:51Z

Pylint 1.5 brought with it much more than

    pylint.extensions.check_docs

The changes for this PR are

checking import order
checking self._foo is set in __init__
complaining about methods that don't use self
complaining about not calling super().__init__ (for
the virtual base class we used)
complaining about not (self == other) in __ne__,
suggested (self != other) instead
complaining about unit tests checking None result from methods
which are known (by static analysis?) to return None
complaining about not over-riding virtual method in storage
_PropertyMixin
aggressively checking redefined-variable-type when variables
can be optionally float /int / Entity / Key / etc.
checking that GCloudError constructor is called correctly
in unit tests
checking that subclass method overrides use the same arguments
(this occurred in BigQuery test_job with _makeResource)

dhermes · 2015-11-30T19:11:04Z

Pylint 1.5 was released yesterday (Nov. 29, 2015). This broke our latest build:
https://travis-ci.org/GoogleCloudPlatform/gcloud-python/builds/93984109

Pylint 1.4.4 was released on June 30, 2015 and the maintainers mentioned

My plan is to have 1.5 released on 15-17 july.

gcloud/streaming/http_wrapper.py

@@ -340,7 +340,7 @@ def handle_http_exceptions(retry_args):
        logging.debug('Response returned a retry-after header, retrying')
        retry_after = retry_args.exc.retry_after
    else:
-        raise
+        raise  # pylint: disable=misplaced-bare-raise


dhermes · 2015-11-30T20:08:03Z

@tseaver I rebased this on top of #1249. PTAL

gcloud/datastore/entity.py

@@ -106,7 +106,11 @@ def __ne__(self, other):
        :rtype: boolean
        :returns: False if the entities compare equal, else True.
        """
-        return not self == other
+        eq_val = self.__eq__(other)


dhermes · 2015-11-30T20:49:44Z

@tseaver I addressed the __init__ business for _access_grants and _schema and folded it into the original commit.

tseaver · 2015-11-30T21:08:23Z

Seems like we're good except the issue of returning False vs. NotImplemented from __eq__ (entity and key).

Pylint 1.5 brought with it much more than pylint.extensions.check_docs The changes for this PR are - checking import order - checking self._foo is set in __init__ - complaining about methods that don't use self - complaining about not calling super().__init__ (for the virtual base class we used) - complaining about not (self == other) in __ne__, suggested (self != other) instead - complaining about unit tests checking None result from methods which are known (by static analysis?) to return None - complaining about not over-riding virtual method in storage _PropertyMixin - aggressively checking `redefined-variable-type` when variables can be optionally float/int/Entity/Key/etc. - checking that GCloudError constructor is called correctly in unit tests - checking that subclass method overrides use the same arguments (this occurred in BigQuery test_job with _makeResource)

dhermes · 2015-11-30T21:17:37Z

@tseaver PTAL. Folded in the assignment-from-no-return and ditched NotImplemented as return value from __eq__.

Only open question left is local vs. global disabling of wrong-import-position. We can open an issue and discuss if you don't want to block on this PR?

tseaver · 2015-11-30T21:48:53Z

We can let the current local disabling go, I think: I don't know what it would take to get a "conforming" version in place (one which wouldn't require the local disable). So, LGTM to merge.

Updated source for Pylint 1.5.

dhermes · 2015-11-30T22:03:11Z

It's impossible to get a conforming version in place. You've inspired me to file an issue with the Pylint folks.

The issue was that using

try:
    import foo
except ImportError:
    ...

import bar

makes the bar import invalid no matter what, since the try/except is considered code (i.e. imports should be over by then).

But for some pairs of foo / bar (like if one is in the stdlib and the other is local to our library), swapping them

import bar

try:
    import foo
except ImportError:
    ...

makes Pylint angry that the import of bar comes before the import of foo.

dhermes · 2015-11-30T22:13:17Z

@tseaver Maybe our rcfile is out of date? I'm going to check it out.

dhermes · 2015-11-30T22:18:59Z

With the current order (and no ignores):

diff --git a/gcloud/_helpers.py b/gcloud/_helpers.py
index b784042..2d44b99 100644
--- a/gcloud/_helpers.py
+++ b/gcloud/_helpers.py
@@ -27,12 +27,10 @@ from six.moves.http_client import HTTPConnection  # pylint: disable=F0401

 from gcloud.environment_vars import PROJECT

-# pylint: disable=wrong-import-position
 try:
     from google.appengine.api import app_identity
 except ImportError:
     app_identity = None
-# pylint: enable=wrong-import-position


 _NOW = datetime.datetime.utcnow  # To be replaced by tests.
diff --git a/gcloud/credentials.py b/gcloud/credentials.py
index bfe8926..cd61b7f 100644
--- a/gcloud/credentials.py
+++ b/gcloud/credentials.py
@@ -32,7 +32,6 @@ except ImportError:
     class _GAECreds(object):
         """Dummy class if not in App Engine environment."""

-# pylint: disable=wrong-import-position
 try:
     from google.appengine.api import app_identity
 except ImportError:
@@ -41,7 +40,6 @@ except ImportError:
 from gcloud._helpers import UTC
 from gcloud._helpers import _NOW
 from gcloud._helpers import _microseconds_from_datetime
-# pylint: enable=wrong-import-position


 def get_credentials():

we get the following errors:

C: 36, 4: Import "from google.appengine.api import app_identity" should be placed at the top of the module (wrong-import-position)
C: 40, 0: Import "from gcloud._helpers import UTC" should be placed at the top of the module (wrong-import-position)
C: 41, 0: Import "from gcloud._helpers import _NOW" should be placed at the top of the module (wrong-import-position)
C: 42, 0: Import "from gcloud._helpers import _microseconds_from_datetime" should be placed at the top of the module (wrong-import-position)

Trying to swap the order:

diff --git a/gcloud/_helpers.py b/gcloud/_helpers.py
index b784042..2d44b99 100644
--- a/gcloud/_helpers.py
+++ b/gcloud/_helpers.py
@@ -27,12 +27,10 @@ from six.moves.http_client import HTTPConnection  # pylint: disable=F0401

 from gcloud.environment_vars import PROJECT

-# pylint: disable=wrong-import-position
 try:
     from google.appengine.api import app_identity
 except ImportError:
     app_identity = None
-# pylint: enable=wrong-import-position


 _NOW = datetime.datetime.utcnow  # To be replaced by tests.
diff --git a/gcloud/credentials.py b/gcloud/credentials.py
index bfe8926..11d7227 100644
--- a/gcloud/credentials.py
+++ b/gcloud/credentials.py
@@ -26,23 +26,22 @@ from oauth2client import client
 from oauth2client.client import _get_application_default_credential_from_file
 from oauth2client import crypt
 from oauth2client import service_account
+
+from gcloud._helpers import UTC
+from gcloud._helpers import _NOW
+from gcloud._helpers import _microseconds_from_datetime
+
 try:
     from oauth2client.appengine import AppAssertionCredentials as _GAECreds
 except ImportError:
     class _GAECreds(object):
         """Dummy class if not in App Engine environment."""

-# pylint: disable=wrong-import-position
 try:
     from google.appengine.api import app_identity
 except ImportError:
     app_identity = None

-from gcloud._helpers import UTC
-from gcloud._helpers import _NOW
-from gcloud._helpers import _microseconds_from_datetime
-# pylint: enable=wrong-import-position
-

 def get_credentials():
     """Gets credentials implicitly from the current environment.

we get the following errors

C: 41, 4: Import "from google.appengine.api import app_identity" should be placed at the top of the module (wrong-import-position)
C: 35, 4: external import "from oauth2client.appengine import AppAssertionCredentials as _GAECreds" comes before "from gcloud._helpers import UTC" (wrong-import-order)

dhermes · 2015-11-30T22:32:52Z

Filed: https://bitbucket.org/logilab/pylint/issues/714/

It seems if our except blocks are more vanilla (i.e. don't define a class right then and there) then it is still considered an import.

tox.ini

@@ -75,8 +75,7 @@ commands =
    python run_pylint.py
 deps =
    pep8
-    -ehg+https://bitbucket.org/logilab/pylint@33e334be064c#egg=pylint
-    logilab-common<=0.63.0  # Hack for pylint upstream
+    pylint
    unittest2


dhermes · 2015-12-01T04:54:12Z

Also filed: https://bitbucket.org/logilab/pylint/issues/716/

dhermes added testing hygiene labels Nov 30, 2015

googlebot added the cla: yes This human has signed the Contributor License Agreement. label Nov 30, 2015

dhermes reviewed Nov 30, 2015
View reviewed changes

dhermes mentioned this pull request Nov 30, 2015

Refactor 'gcloud.streaming.http_wrapper' exception handling #1249

Merged

tseaver mentioned this pull request Nov 30, 2015

Fix handling of timestamps in BigQuery insert data method #1250

Merged

dhermes force-pushed the update-for-new-pylint branch from ae1518f to 1944ab5 Compare November 30, 2015 19:56

dhermes reviewed Nov 30, 2015
View reviewed changes

dhermes force-pushed the update-for-new-pylint branch from 1944ab5 to 3073d32 Compare November 30, 2015 20:43

dhermes force-pushed the update-for-new-pylint branch from 3073d32 to 0c88da4 Compare November 30, 2015 21:13

dhermes added a commit that referenced this pull request Nov 30, 2015

Merge pull request #1248 from dhermes/update-for-new-pylint

0f8bd8e

Updated source for Pylint 1.5.

dhermes merged commit 0f8bd8e into googleapis:master Nov 30, 2015

dhermes deleted the update-for-new-pylint branch November 30, 2015 21:58

PCManticore reviewed Nov 30, 2015
View reviewed changes

dhermes mentioned this pull request Dec 1, 2015

Removing wrong-import-position Pylint ignore statements. #1255

Merged

dhermes mentioned this pull request Jan 25, 2016

Upgrading version to 0.9.0. #1418

Merged

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Updated source for Pylint 1.5. #1248

Updated source for Pylint 1.5. #1248

dhermes commented Nov 30, 2015

dhermes commented Nov 30, 2015

This comment was marked as spam.

This comment was marked as spam.

This comment was marked as spam.

dhermes commented Nov 30, 2015

This comment was marked as spam.

This comment was marked as spam.

dhermes commented Nov 30, 2015

tseaver commented Nov 30, 2015

dhermes commented Nov 30, 2015

tseaver commented Nov 30, 2015

dhermes commented Nov 30, 2015

dhermes commented Nov 30, 2015

dhermes commented Nov 30, 2015

dhermes commented Nov 30, 2015

This comment was marked as spam.

This comment was marked as spam.

This comment was marked as spam.

dhermes commented Dec 1, 2015

Updated source for Pylint 1.5. #1248

Updated source for Pylint 1.5. #1248

Conversation

dhermes commented Nov 30, 2015

dhermes commented Nov 30, 2015

This comment was marked as spam.

This comment was marked as spam.

This comment was marked as spam.

dhermes commented Nov 30, 2015

This comment was marked as spam.

This comment was marked as spam.

dhermes commented Nov 30, 2015

tseaver commented Nov 30, 2015

dhermes commented Nov 30, 2015

tseaver commented Nov 30, 2015

dhermes commented Nov 30, 2015

dhermes commented Nov 30, 2015

dhermes commented Nov 30, 2015

dhermes commented Nov 30, 2015

This comment was marked as spam.

This comment was marked as spam.

This comment was marked as spam.

dhermes commented Dec 1, 2015