astroid error when using flask dependencies

[johanngomes]

Hi,

On the latest prospector version (1.1.3), I noticed that I get an error when trying to use flask dependencies

prospector --uses flask results in that error:

Traceback (most recent call last):
  File "/usr/local/bin/prospector", line 11, in <module>
    sys.exit(main())
  File "/usr/local/lib/python3.6/site-packages/prospector/run.py", line 173, in main
    prospector.execute()
  File "/usr/local/lib/python3.6/site-packages/prospector/run.py", line 53, in execute
    for tool in self.config.get_tools(found_files):
  File "/usr/local/lib/python3.6/site-packages/prospector/config/__init__.py", line 41, in get_tools
    config_result = tool.configure(self, found_files)
  File "/usr/local/lib/python3.6/site-packages/prospector/tools/pylint/__init__.py", line 189, in configure
    self._prospector_configure(prospector_config, linter)
  File "/usr/local/lib/python3.6/site-packages/prospector/tools/pylint/__init__.py", line 37, in _prospector_configure
    linter.load_plugin_modules(['pylint_flask'])
  File "/usr/local/lib/python3.6/site-packages/pylint/lint.py", line 519, in load_plugin_modules
    module = modutils.load_module_from_name(modname)
  File "/usr/local/lib/python3.6/site-packages/astroid/modutils.py", line 196, in load_module_from_name
    return load_module_from_modpath(dotted_name.split('.'), path, use_sys)
  File "/usr/local/lib/python3.6/site-packages/astroid/modutils.py", line 239, in load_module_from_modpath
    module = imp.load_module(curname, mp_file, mp_filename, mp_desc)
  File "/usr/local/lib/python3.6/imp.py", line 245, in load_module
    return load_package(name, filename)
  File "/usr/local/lib/python3.6/imp.py", line 217, in load_package
    return _load(spec)
  File "<frozen importlib._bootstrap>", line 684, in _load
  File "<frozen importlib._bootstrap>", line 665, in _load_unlocked
  File "<frozen importlib._bootstrap_external>", line 678, in exec_module
  File "<frozen importlib._bootstrap>", line 219, in _call_with_frames_removed
  File "/usr/local/lib/python3.6/site-packages/pylint_flask/__init__.py", line 79, in <module>
    MANAGER.register_transform(nodes.From,
AttributeError: module 'astroid.nodes' has no attribute 'From'

Here is my requirements.txt:

prospector == 1.1.3
pylint-flask == 0.5
astroid == 2.0.4
bandit == 1.4.0
safety==1.4.0

Please let me know if you need more details. I needed to specify pylint-flask directly, because even with --uses flask, the prospector did not downloaded pylint-flask automatically (not sure if it was meant to be downloaded automatically in that case).

[carlio]

I suspect this is related to #285 - I think we will need a new release with stricter dependencies as they are quite open right now.

[chocoelho]

Not really about the issue itself, but we've removed the automatic installation of pylint-django, pylint-flask and pylint-celery due to the reason that it install unnecessary dependencies to projects. One thing we can come up with @carlio is offer as an option, such as with_flask, like we do for other tools like vulture, mypy...

[carlio]

@chocoelho Actually for that I think maybe an addition to autodetect would work. It'll find 'django' or whatever in the requirements for the project, so it'll think that pylint-django should be imported. But once it finds that, it should try 'import django' and on an import error, raise an error/warning saying 'Oi user. Install Django'.

[ramnes]

Just tried to give prospector a shot today, but I got this error.

[carlio]

@ramnes which error? There are two reported, one is that django-flask is not installed which should be fixed in the latest release, the other is that stacktrace which I am looking into.

[carlio]

And the cause is the pylint-flask library unfortunately: jschaf/pylint-flask#8

[ramnes]

OP's error. Thanks for the hint; downgrading pylint to 1.9.3 fixes this.

[johanngomes]

You can also just not use python-flask dependency option and you should be ok. Prospector will then, run with the default modules.

On Sat, 24 Nov 2018 at 11:07 Guillaume Gelin ***@***.***> wrote: OP's error. Thanks for the hint; downgrading pylint to 1.9.3 fixes this. — You are receiving this because you authored the thread. Reply to this email directly, view it on GitHub <#286 (comment)>, or mute the thread <https://github.com/notifications/unsubscribe-auth/AAWSfgNRPtuWSbdNYdbt388xWvSfa-aeks5uyVK6gaJpZM4X4wJx> .

-- * Johann Gomes Barros Lima Software Developer Consultant ** Aware of our own privilege, we strive to see the world from the perspective of the oppressed, the powerless and the invisible.*

[johanngomes]

*pylint-flask, not python-flask. Sorry for the typo.

On Sat, 24 Nov 2018 at 14:22 Johann Gomes ***@***.***> wrote: You can also just not use python-flask dependency option and you should be ok. Prospector will then, run with the default modules. On Sat, 24 Nov 2018 at 11:07 Guillaume Gelin ***@***.***> wrote: > OP's error. Thanks for the hint; downgrading pylint to 1.9.3 fixes this. > > — > You are receiving this because you authored the thread. > Reply to this email directly, view it on GitHub > <#286 (comment)>, > or mute the thread > <https://github.com/notifications/unsubscribe-auth/AAWSfgNRPtuWSbdNYdbt388xWvSfa-aeks5uyVK6gaJpZM4X4wJx> > . > -- * Johann Gomes Barros Lima Software Developer Consultant ** Aware of our own privilege, we strive to see the world from the perspective of the oppressed, the powerless and the invisible.* --

* Johann Gomes Barros Lima Software Developer Consultant ** Aware of our own privilege, we strive to see the world from the perspective of the oppressed, the powerless and the invisible.*

[ramnes]

Well the thing is that all of those were bundled by default with a simple pip install prospector. It's a bit cumbersome that I have to play with the dependencies pulled in by prospector itself afterwards.

[carlio]

@ramnes can you link to your repo or show the requirements/setup.py? pylint-flask is installed by default but only turned on if prospector thinks you are using flask, which it does by running requirements-detector. I agree it's rubbish right now as pylint-flask is broken. You can also also turn that off with the --no-autodetect / -A flag to prospector and pylint-flask will never be called.

[richardhanson]

Had the same issue and used the -A flag to fix it. Here's a copy of my Pipfile. I think the ideal way to deal with an issue with pylint like this would be to expose a flag the prevents pylint from loading the extension. That way it's configurable from the command line and from .prospector.yaml.

Pipenv file

[[source]]
url = "https://pypi.org/simple"
name = "pypi"
verify_ssl = true

[packages]
alembic = "==0.9.10"
atomicwrites = "==1.2.1"
attrs = "==18.2.0"
"boto3" = "==1.9.0"
botocore = "==1.12.18"
certifi = "==2018.4.16"
chardet = "==3.0.4"
click = "==6.7"
coverage = "==4.5.1"
docutils = "==0.14"
expiringdict = "==1.1.4"
idna = "==2.7"
itsdangerous = "==0.24"
jmespath = "==0.9.3"
more-itertools = "==4.3.0"
numpy = "==1.14.5"
pandas = "==0.23.3"
pluggy = "==0.7.1"
"psycopg2-binary" = "==2.7.5"
py = "==1.6.0"
pytest = "==3.8.0"
pytest-cov = "==2.6.0"
pytest-mock = "==1.10.0"
python-dateutil = "==2.7.3"
python-editor = "==1.0.3"
pytz = "==2018.5"
requests = "==2.20.0"
requests-mock = "==1.5.2"
"s3transfer" = "==0.1.13"
six = "==1.11.0"
sortedcontainers = "==2.0.5"
"urllib3" = "==1.23"
Flask = "==1.0.2"
Flask-BasicAuth = "*"
"Jinja2" = "==2.10"
Mako = "==1.0.7"
MarkupSafe = "==1.0"
PyYAML = "==3.13"
SQLAlchemy = "==1.2.9"
SQLAlchemy-Utils = "==0.33.4"
uWSGI = "==2.0.17.1"
Werkzeug = "==0.14.1"
XlsxWriter = "==1.0.5"
numexpr = "*"
bottleneck = "*"

[requires]
python_version = "3.7"

[dev-packages]
flake8 = "<3.6.0"
prospector = {version = "*", extras = ["with_vulture"]}
bandit = "*"
mypy = "*"
pylint = "*"

pipenv graph for prospector

prospector==1.1.6.2
  - astroid [required: ==2.0.4, installed: 2.0.4]
    - lazy-object-proxy [required: Any, installed: 1.3.1]
    - six [required: Any, installed: 1.11.0]
    - wrapt [required: Any, installed: 1.10.11]
  - dodgy [required: >=0.1.9, installed: 0.1.9]
  - mccabe [required: >=0.5.0, installed: 0.6.1]
  - pep8-naming [required: >=0.3.3,<=0.4.1, installed: 0.4.1]
  - pycodestyle [required: >=2.0.0,<=2.4.0, installed: 2.3.1]
  - pydocstyle [required: >=2.0.0, installed: 3.0.0]
    - six [required: Any, installed: 1.11.0]
    - snowballstemmer [required: Any, installed: 1.2.1]
  - pyflakes [required: >=0.8.1,<2.0.0, installed: 1.6.0]
  - pylint [required: ==2.1.1, installed: 2.1.1]
    - astroid [required: >=2.0.0, installed: 2.0.4]
      - lazy-object-proxy [required: Any, installed: 1.3.1]
      - six [required: Any, installed: 1.11.0]
      - wrapt [required: Any, installed: 1.10.11]
    - isort [required: >=4.2.5, installed: 4.3.4]
    - mccabe [required: Any, installed: 0.6.1]
  - pylint-celery [required: ==0.3, installed: 0.3]
    - astroid [required: >=1.0, installed: 2.0.4]
      - lazy-object-proxy [required: Any, installed: 1.3.1]
      - six [required: Any, installed: 1.11.0]
      - wrapt [required: Any, installed: 1.10.11]
    - pylint [required: >=1.0, installed: 2.1.1]
      - astroid [required: >=2.0.0, installed: 2.0.4]
        - lazy-object-proxy [required: Any, installed: 1.3.1]
        - six [required: Any, installed: 1.11.0]
        - wrapt [required: Any, installed: 1.10.11]
      - isort [required: >=4.2.5, installed: 4.3.4]
      - mccabe [required: Any, installed: 0.6.1]
    - pylint-plugin-utils [required: >=0.2.1, installed: 0.4]
      - pylint [required: Any, installed: 2.1.1]
        - astroid [required: >=2.0.0, installed: 2.0.4]
          - lazy-object-proxy [required: Any, installed: 1.3.1]
          - six [required: Any, installed: 1.11.0]
          - wrapt [required: Any, installed: 1.10.11]
        - isort [required: >=4.2.5, installed: 4.3.4]
        - mccabe [required: Any, installed: 0.6.1]
  - pylint-django [required: ==2.0.2, installed: 2.0.2]
    - pylint [required: >=2.0, installed: 2.1.1]
      - astroid [required: >=2.0.0, installed: 2.0.4]
        - lazy-object-proxy [required: Any, installed: 1.3.1]
        - six [required: Any, installed: 1.11.0]
        - wrapt [required: Any, installed: 1.10.11]
      - isort [required: >=4.2.5, installed: 4.3.4]
      - mccabe [required: Any, installed: 0.6.1]
    - pylint-plugin-utils [required: >=0.4, installed: 0.4]
      - pylint [required: Any, installed: 2.1.1]
        - astroid [required: >=2.0.0, installed: 2.0.4]
          - lazy-object-proxy [required: Any, installed: 1.3.1]
          - six [required: Any, installed: 1.11.0]
          - wrapt [required: Any, installed: 1.10.11]
        - isort [required: >=4.2.5, installed: 4.3.4]
        - mccabe [required: Any, installed: 0.6.1]
  - pylint-flask [required: ==0.5, installed: 0.5]
    - astroid [required: >=1.0, installed: 2.0.4]
      - lazy-object-proxy [required: Any, installed: 1.3.1]
      - six [required: Any, installed: 1.11.0]
      - wrapt [required: Any, installed: 1.10.11]
    - pylint [required: >=1.0, installed: 2.1.1]
      - astroid [required: >=2.0.0, installed: 2.0.4]
        - lazy-object-proxy [required: Any, installed: 1.3.1]
        - six [required: Any, installed: 1.11.0]
        - wrapt [required: Any, installed: 1.10.11]
      - isort [required: >=4.2.5, installed: 4.3.4]
      - mccabe [required: Any, installed: 0.6.1]
    - pylint-plugin-utils [required: >=0.2.1, installed: 0.4]
      - pylint [required: Any, installed: 2.1.1]
        - astroid [required: >=2.0.0, installed: 2.0.4]
          - lazy-object-proxy [required: Any, installed: 1.3.1]
          - six [required: Any, installed: 1.11.0]
          - wrapt [required: Any, installed: 1.10.11]
        - isort [required: >=4.2.5, installed: 4.3.4]
        - mccabe [required: Any, installed: 0.6.1]
  - pylint-plugin-utils [required: >=0.2.6, installed: 0.4]
    - pylint [required: Any, installed: 2.1.1]
      - astroid [required: >=2.0.0, installed: 2.0.4]
        - lazy-object-proxy [required: Any, installed: 1.3.1]
        - six [required: Any, installed: 1.11.0]
        - wrapt [required: Any, installed: 1.10.11]
      - isort [required: >=4.2.5, installed: 4.3.4]
      - mccabe [required: Any, installed: 0.6.1]
  - pyyaml [required: Any, installed: 3.13]
  - requirements-detector [required: >=0.6, installed: 0.6]
    - astroid [required: >=1.4, installed: 2.0.4]
      - lazy-object-proxy [required: Any, installed: 1.3.1]
      - six [required: Any, installed: 1.11.0]
      - wrapt [required: Any, installed: 1.10.11]
  - setoptconf [required: >=0.2.0, installed: 0.2.0]

[carlio]

I shall close this as "very old" and likely out of date, please re-open if the issue persists but I suspect that either new versions have fixed things or that you're no longer using prospector ;-)