Incorrect inference of ancestors of Enum class with six.with_metaclass

[tkukushkin]

Steps to reproduce

import astroid

code = '''import six
from enum import Enum, EnumMeta

class FooMeta(EnumMeta):
    pass

class Foo(six.with_metaclass(FooMeta, Enum)):
    bar = 1'''

module = astroid.parse(code)
print(list(module.body[3].ancestors()))

Current behavior

Prints [<ClassDef.NoneType l.0 at 0x102bb4d68>, <ClassDef.object l.0 at 0x102bbe1d0>]

Expected behavior

Prints [<ClassDef.Enum l.10 at 0x10d0617b8>, <ClassDef.object l.0 at 0x10c4650f0>]

With another simple example without enums works ok:

import astroid

code = '''import six

class FooMeta(type):
    pass

class BaseFoo(object):
    pass

class Foo(six.with_metaclass(FooMeta, BaseFoo)):
    bar = 1'''

module = astroid.parse(code)
print(list(module.body[3].ancestors()))

Prints: [<ClassDef.BaseFoo l.6 at 0x1091531d0>, <ClassDef.object l.0 at 0x10853d128>]

python -c "from astroid import __pkginfo__; print(__pkginfo__.version)" output

2.3.2

This behaviour breaks pylint with exception:

Traceback (most recent call last):
  File "bin/pylint", line 8, in <module>
    sys.exit(run_pylint())
  File "lib/python3.7/site-packages/pylint/__init__.py", line 19, in run_pylint
    Run(sys.argv[1:])
  File "lib/python3.7/site-packages/pylint/lint.py", line 1394, in __init__
    linter.check(args)
  File "lib/python3.7/site-packages/pylint/lint.py", line 801, in check
    self._do_check(files_or_modules)
  File "lib/python3.7/site-packages/pylint/lint.py", line 938, in _do_check
    self.check_astroid_module(ast_node, walker, rawcheckers, tokencheckers)
  File "lib/python3.7/site-packages/pylint/lint.py", line 1018, in check_astroid_module
    walker.walk(ast_node)
  File "lib/python3.7/site-packages/pylint/utils.py", line 1162, in walk
    self.walk(child)
  File "lib/python3.7/site-packages/pylint/utils.py", line 1162, in walk
    self.walk(child)
  File "lib/python3.7/site-packages/pylint/utils.py", line 1162, in walk
    self.walk(child)
  File "lib/python3.7/site-packages/pylint/utils.py", line 1159, in walk
    cb(astroid)
  File "lib/python3.7/site-packages/pylint/checkers/base.py", line 1509, in visit_assignname
    if not list(frame.local_attr_ancestors(node.name)):
  File "lib/python3.7/site-packages/astroid/scoped_nodes.py", line 2197, in local_attr_ancestors
    if name in astroid:
TypeError: argument of type 'Const' is not iterable

[tkukushkin]

After some debug I see, that infer() method of <Name.Enum l.2 at 0x1062eb5c0> returns [<Const.NoneType l.119 at 0x1068aceb8>, <ClassDef.Enum l.531 at 0x106750550>].
Hack for with_metaclass uses class_bases = [next(arg.infer(context)) for arg in caller.args[1:]] which uses only first result of infer()

[tkukushkin]

Maybe need to register transform for six.with_metaclass like transform for six.add_metaclass?

P.S.: six.add_metaclass doesn't work with Enum.

[tkukushkin]

Possible implementation:

def _looks_like_nested_from_six_with_metaclass(node):
    if len(node.bases) != 1:
        return False
    base = node.bases[0]
    if not isinstance(base, Call):
        return False
    call = base
    try:
        func = next(call.func.infer())
    except InferenceError:
        return False
    return func.qname() == 'six.with_metaclass'

def _transform_six_with_metaclass(node):
    call = node.bases[0]
    node._metaclass = call.args[0]
    node.bases = call.args[1:]

MANAGER.register_transform(
    nodes.ClassDef,
    _transform_six_with_metaclass,
    _looks_like_nested_from_six_with_metaclass,
)

[PCManticore]

@tkukushkin Just managed to check your issue. I believe it makes sense, would you be able to submit a PR with that transform, tests and all the relevant boilerplate? Let me know if you need any help with that.

[tkukushkin]

@PCManticore, I have an issue, that after removing six.with_metaclass from base classes import six remains in file. Pylint shows an error: [W0611 unused-import] Unused import six.

[liamgens]

Any update on this? I'm experiencing the same issue

[fmigneault]

In earlier version (eg pylint 2.4.4), only a message was returned in the form: argument of type 'Const' is not iterable, which we could live with. With pylint 2.5+, this is breaking pylint execution with following tracelog:

multiprocessing.pool.RemoteTraceback: 
"""
Traceback (most recent call last):
  File "/home/francis/DEV/conda/envs/magpie-py38/lib/python3.8/multiprocessing/pool.py", line 125, in worker
    result = (True, func(*args, **kwds))
  File "/home/francis/DEV/conda/envs/magpie-py38/lib/python3.8/site-packages/pylint/lint/check_parallel.py", line 69, in _worker_check_single_file
    _worker_linter.check_single_file(name, filepath, modname)
  File "/home/francis/DEV/conda/envs/magpie-py38/lib/python3.8/site-packages/pylint/lint/pylinter.py", line 881, in check_single_file
    self._check_file(
  File "/home/francis/DEV/conda/envs/magpie-py38/lib/python3.8/site-packages/pylint/lint/pylinter.py", line 922, in _check_file
    check_astroid_module(ast_node)
  File "/home/francis/DEV/conda/envs/magpie-py38/lib/python3.8/site-packages/pylint/lint/pylinter.py", line 1054, in check_astroid_module
    retval = self._check_astroid_module(
  File "/home/francis/DEV/conda/envs/magpie-py38/lib/python3.8/site-packages/pylint/lint/pylinter.py", line 1099, in _check_astroid_module
    walker.walk(ast_node)
  File "/home/francis/DEV/conda/envs/magpie-py38/lib/python3.8/site-packages/pylint/utils/ast_walker.py", line 75, in walk
    self.walk(child)
  File "/home/francis/DEV/conda/envs/magpie-py38/lib/python3.8/site-packages/pylint/utils/ast_walker.py", line 75, in walk
    self.walk(child)
  File "/home/francis/DEV/conda/envs/magpie-py38/lib/python3.8/site-packages/pylint/utils/ast_walker.py", line 75, in walk
    self.walk(child)
  File "/home/francis/DEV/conda/envs/magpie-py38/lib/python3.8/site-packages/pylint/utils/ast_walker.py", line 72, in walk
    callback(astroid)
  File "/home/francis/DEV/conda/envs/magpie-py38/lib/python3.8/site-packages/pylint/checkers/base.py", line 1992, in visit_assignname
    if not list(frame.local_attr_ancestors(node.name)):
  File "/home/francis/DEV/conda/envs/magpie-py38/lib/python3.8/site-packages/astroid/scoped_nodes.py", line 2314, in local_attr_ancestors
    if name in astroid:
TypeError: argument of type 'Const' is not iterable
"""

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
  File "/home/francis/DEV/conda/envs/magpie-py38/bin/pylint", line 8, in <module>
    sys.exit(run_pylint())
  File "/home/francis/DEV/conda/envs/magpie-py38/lib/python3.8/site-packages/pylint/__init__.py", line 22, in run_pylint
    PylintRun(sys.argv[1:])
  File "/home/francis/DEV/conda/envs/magpie-py38/lib/python3.8/site-packages/pylint/lint/run.py", line 349, in __init__
    linter.check(args)
  File "/home/francis/DEV/conda/envs/magpie-py38/lib/python3.8/site-packages/pylint/lint/pylinter.py", line 866, in check
    check_parallel(
  File "/home/francis/DEV/conda/envs/magpie-py38/lib/python3.8/site-packages/pylint/lint/check_parallel.py", line 102, in check_parallel
    for (
  File "/home/francis/DEV/conda/envs/magpie-py38/lib/python3.8/multiprocessing/pool.py", line 868, in next
    raise value
TypeError: argument of type 'Const' is not iterable

[tkukushkin]

@PCManticore, @fmigneault original issue is reproduced again after #893.