Cannot assign to field of Callable type

[spkersten]

Consider this example:

from typing import Callable

class A:
    f = None  # type: Callable[[int], int]

    def __init__(self, g: Callable[[int], int]) -> None:
        self.f = g

mypy reports:

trial.py: In member "__init__" of class "A":
trial.py, line 7: Cannot assign to a method
trial.py, line 7: Invalid method type
trial.py, line 7: Incompatible types in assignment (expression has type Callable[[int], int], variable has type Callable[[], int])

while I'd expect this to work.

Note that there is a second error in the reporting of the type of the variable.

Also, this passes type checking (correctly):

f = None  # type: Callable[[int], int]

def g(x: int) -> int: return 0

f = g

[spkersten]

The mismatching types is probably because mypy thinks f is a method and therefore strips off the first argument (self).

[JukkaL]

Hmm.. actually not sure what's the right thing to do here. Based on a quick experiment, the desired behavior seems to depend on what kind of a callable g is in your original example.

Consider this:

class A:
    f = None
def m(x):
    return x + 1
print(m(1))  # 2
A.f = m
print(A().f(1))  # Error (2 args given, expects 1)

But now look at this:

class A:
    f = None
class M:
    def __call__(self, x):
        return x + 1
m = M()
print(m(1) # 2
A.f = m
print(A().f(1))  # Okay => 2

[JukkaL]

Taking off priority label because the fix doesn't seem obvious.

[spkersten]

Before reporting it I also thought for a while that I could fix it quickly 😄.

[tharvik]

Is there any update? The issue is still present. For some real world example, it is used in the ctypes module (in the examples).

from typing import Any, Callable

class A():
    a = lambda: None  # type: Callable[..., None]

def f() -> None: return None

A().a = f

gives

/tmp/assign_callable.py:8: error: Cannot assign to a method
/tmp/assign_callable.py:8: error: Incompatible types in assignment (expression has type Callable[[], None], variable has type Callable[..., None])

Having Callable[..., None] or Callable[[], None] do not resolve it, having Callable[..., Any] remove the incompatible types line.

[gvanrossum]

I'm sorry, I don't have any further updates.

[gnprice]

Thanks for bumping this with the new repro!

It's possible these two errors are different issues. The second looks like it may be a case of the Void/NoneTyp distinction (recently eliminated in our --strict-optional future: #1975). Would you try reproducing it on master and with --strict-optional?

The first error looks like the same thing as the original issue.

[gnprice]

@tharvik, would you also link more specifically to the bit in ctypes that refers to this? It's good to know about that use case, and it'd be interesting to see it more concretely.

[tharvik]

Seems that I'm unable to reproduce 'Incompatible types in assignment' on master with or without --strict-optional.

@gnprice in ctypes, when declaring a foreign function, one can redefine errcheck, which should be a callable. It allows processing of the return value (and raising on failure).

Example from the documentation

def errcheck(result, func, args):
    if not result:
        raise WinError()
     return args

GetWindowRect.errcheck = errcheck

This way, we can load a library, in which, every attribute is a foreign function, then assign to some an errcheck for freeing some pointer or other.

For another example, here goes libLAS, where the library is loaded then it is assigned some checks.

[gvanrossum]

To be clear, the example still produces "Cannot assign to a method". The problem seems to be that mypy disallows assigning to a method (which I think is reasonable) but thinks that a class variable declared with a Callable type (and initialized with a lambda) is also a method. The latter is wrong, I think.

[Stiivi]

Just for the completeness of cases: same error with Python 3.6 variable annotations:

MyFunction = Callable[[int], int]                     
                                                      
class Thing:                                          
    function: MyFunction                              
                                                      
    def __init__(self, function: MyFunction) -> None: 
        self.function = function                      

[hendi]

This affects basically every Django app that uses django.contrib.admin. The following is an often-used pattern there:

class FooAdmin(admin.ModelAdmin):
    list_display = ("id", "custom")

    def custom(self, obj):
        return obj.bar
    custom.short_description = "My Custom Field"  # error in mypy

[gvanrossum]

I guess mypy needs to introduce a notion of a function object which is a subclass of Callable that also supports other operations like `__getattr__` and `__setattr__` (though we probably won't be able to type-check such attributes).

[pkch]

Just an example I ran into today:

from collections import defaultdict
d: DefaultDict[int, int] = defaultdict(int)

d.default_factory = None

fails with --strict-optional with 3 errors, and fails without --strict-optional with 2 errors.

When I changed typeshed to make default_factory an Optional[Callable...], --strict-optional type check passed without errors, and --no-strict-optional was unaffected.

[gvanrossum]

To reiterate, assigning a callable to a class variable is fraught with problems in Python itself too. The reason is descriptors (things with a __get__ method) -- if the object is a descriptor then its __get__ method will be called when you use e.g. C.f so what you get back is a bound method. (Regular function objects defined with def or lambda are descriptors that work this way, but builtin functions and other callables typucally don't).

We should really teach mypy about all this (after all it now does know about descriptors!) but that would be a complicated change.

[vegolasse]

Being a bit of a Python beginner and also ran into this which might or might not have more implications than I realise.

My workaround (since all I wanted was a "selfless" function pointer as a member variable) was to put the Callable in a List like: _get_time_ms: List[Callable[[], int]]. It means an extra list lookup every use, but is in my case negligible and the type check is worth more.

[gvanrossum]

That's not a mypy problem per se -- you can't initialize a class variable to a "selfless" function pointer at runtime either (or rather, it will be interpreted as a method and the first argument will be passed the instance). Your workaround is valid.

[AlexWaygood]

This issue was recently discussed in-depth in python/typing#1113, where it was noted that mypy's behaviour differs from most other type-checkers

[altendky]

It seems that hinting with a callable protocol instead of Callable manages to evade mypy's method detection.

https://mypy-play.net/?mypy=latest&python=3.10&gist=389004dcf62cd710bec35920fdce4b68

import typing


class CallableProtocol(typing.Protocol):
    def __call__(self) -> None:
        pass


def a() -> None:
    return

TypingCallable = typing.Callable[[], None]

class C:
    f: TypingCallable
    g: CallableProtocol
    
    def __init__(self, f: TypingCallable, g: CallableProtocol) -> None:
        self.f = f
        self.g = g

    def call(self) -> None:
        self.f()
        self.g()

main.py:19: error: Cannot assign to a method
main.py:19: error: Attribute function "f" with type "Callable[[], None]" does not accept self argument
main.py:23: error: Attribute function "f" with type "Callable[[], None]" does not accept self argument
Found 3 errors in 1 file (checked 1 source file)

[kerimovkhikmet]

There was issue #5062, which was closed as a duplicate in favor of this issue, which is shown as fixed. However, I am currently using Python 3.11.4 with mypy 1.4.1 and still getting the errors with ConfigParser, described in this comment - #5062 (comment). Is this expected, or duplicate issue was closed by mistake, and the error with ConfigParser is not fixed?