Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Fix crash with overload and callable object decorators #11630

Merged
merged 3 commits into from
Nov 28, 2021
Merged

Fix crash with overload and callable object decorators #11630

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
3 commits
Select commit Hold shift + click to select a range
085b88f
Fix crash with overload and callable object decorators
Nov 27, 2021
e600b0e
handle dumb decorators better
Nov 27, 2021
3f135c3
also fix 9967
Nov 27, 2021
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
12 changes: 10 additions & 2 deletions mypy/checker.py
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -507,8 +507,16 @@ def check_overlapping_overloads(self, defn: OverloadedFuncDef) -> None:
# decorator or if the implementation is untyped -- we gave up on the types.
inner_type = get_proper_type(inner_type)
if inner_type is not None and not isinstance(inner_type, AnyType):
assert isinstance(inner_type, CallableType)
impl_type = inner_type
if isinstance(inner_type, CallableType):
impl_type = inner_type
elif isinstance(inner_type, Instance):
inner_call = get_proper_type(
find_member('__call__', inner_type, inner_type, is_operator=True)
Copy link
Member

@sobolevn sobolevn Nov 28, 2021 •
edited
Loading

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

find_member won't call any plugins. And might not be as accurate as analyze_member_access.

Any specific reason to use find_member here?

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I didn't have any specific reason, feel free to change! Just make sure that the getattr handling for the call operator matches runtime handling, e.g. with code like:

class X:
    def __getattr__(self, attr):
        if attr == "__call__":
            return lambda *a, **kw: print(a, kw)
        raise AttributeError

@X()
def f(): ...

In general, I'd love for member access to get cleaned up. I last tried a little bit in #8438 but that was swiftly reverted in #8500

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Related #3832

I will send a PR in a moment 🙂

Copy link
Member

@sobolevn sobolevn Nov 29, 2021 •
edited
Loading

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Just make sure that the getattr handling for the call operator matches runtime handling

Right now it does not work this way with both find_member and analyze_member_access. I will have to add this feature.

Copy link
Collaborator Author

@hauntsaninja hauntsaninja Nov 29, 2021 •
edited
Loading

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think find_member does the right thing when passed is_operator=True? This is what I meant: #11637 (the current code does what I'd expect)

)
if isinstance(inner_call, CallableType):
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

One more problem:

from typing import Callable, Union, Any, overload, Protocol

class DCall(Protocol):
    def __call__(self, arg: Union[int, str]) -> None:
        ...

class D:
    def __getattr__(self, attr: str) -> DCall:
        ...   # Will return `__call__` in runtime

def dec_d(f: Callable[..., Any]) -> D:
    return D()

@overload
def f_d(arg: int) -> None: ...
@overload
def f_d(arg: str) -> None: ...
@dec_d
def f_d(arg: Union[int, str]) -> None: ...

This now reports: out/ex.py:18: error: "D" not callable, because the return type of __getattr__ is not CallableType, but Instance.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I am going to create a new issue out of it. Sadly, I don't have the time right now to fix it.

Copy link
Collaborator Author

@hauntsaninja hauntsaninja Nov 29, 2021 •
edited
Loading

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Sorry, it's late for me, so I might be missing something, but that seems to be the correct behaviour?

λ cat ex.py
from typing import Callable, Union, Any, overload, Protocol

class DCall(Protocol):
    def __call__(self, arg: Union[int, str]) -> None:
        ...

class D:
    def __getattr__(self, attr: str) -> DCall:
        if attr == "__call__":
            return lambda *a, **kw: print(a, kw)
        raise AttributeError

def dec_d(f: Callable[..., Any]) -> D:
    return D()

@overload
def f_d(arg: int) -> None: ...
@overload
def f_d(arg: str) -> None: ...
@dec_d
def f_d(arg: Union[int, str]) -> None: ...

import pytest

D().__call__(1)
with pytest.raises(TypeError):
    print(D()(1))
with pytest.raises(TypeError):
    print(f_d(1))
print("correct")

λ python3 ex.py
(1,) {}
correct

λ mypy ex.py
ex.py:20: error: "D" not callable
ex.py:27: error: "D" not callable
Found 2 errors in 1 file (checked 1 source file)

impl_type = inner_call
if impl_type is None:
self.msg.not_callable(inner_type, defn.impl)

is_descriptor_get = defn.info and defn.name == "__get__"
for i, item in enumerate(defn.items):
Expand Down
68 changes: 68 additions & 0 deletions test-data/unit/check-overloading.test
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -5339,3 +5339,71 @@ def register(cls: Any) -> Any: return None
x = register(Foo)
reveal_type(x) # N: Revealed type is "builtins.int"
[builtins fixtures/dict.pyi]


[case testOverloadWithObjectDecorator]
from typing import Any, Callable, Union, overload

class A:
def __call__(self, *arg, **kwargs) -> None: ...

def dec_a(f: Callable[..., Any]) -> A:
return A()

@overload
def f_a(arg: int) -> None: ...
@overload
def f_a(arg: str) -> None: ...
@dec_a
def f_a(arg): ...

class B:
def __call__(self, arg: Union[int, str]) -> None: ...

def dec_b(f: Callable[..., Any]) -> B:
return B()

@overload
def f_b(arg: int) -> None: ...
@overload
def f_b(arg: str) -> None: ...
@dec_b
def f_b(arg): ...

class C:
def __call__(self, arg: int) -> None: ...

def dec_c(f: Callable[..., Any]) -> C:
return C()

@overload
def f_c(arg: int) -> None: ...
@overload
def f_c(arg: str) -> None: ...
@dec_c # E: Overloaded function implementation does not accept all possible arguments of signature 2
def f_c(arg): ...
[builtins fixtures/dict.pyi]

[case testOverloadWithErrorDecorator]
from typing import Any, Callable, TypeVar, overload

def dec_d(f: Callable[..., Any]) -> int: ...

@overload
def f_d(arg: int) -> None: ...
@overload
def f_d(arg: str) -> None: ...
@dec_d # E: "int" not callable
def f_d(arg): ...

Bad = TypeVar('Good') # type: ignore

def dec_e(f: Bad) -> Bad: ... # type: ignore

@overload
def f_e(arg: int) -> None: ...
@overload
def f_e(arg: str) -> None: ...
@dec_e # E: Bad? not callable
def f_e(arg): ...
[builtins fixtures/dict.pyi]