List concatenation doesn't use type context

[JukkaL]

Code like this doesn't type check correctly (Python 2 example):

from typing import List, Optional
result = range(10) + [None]  # type: List[Optional[int]]

It generates these errors:

t.py:2: error: Incompatible types in assignment (expression has type "List[int]", variable has type "List[Optional[int]]")
t.py:2: error: List item 0 has incompatible type "None"; expected "int"

Issues like these seem somewhat frequent. There are also similar examples in the mypy codebase.

The errors have two causes:

1. list concatenation cannot vary the return type based on context
2. the types of the list items must match in concatenation.

A quick way to fix this particular example would be to change the signature of list.__add__ to something like this:

class list(Generic[_T]):
    ...
    def __add__(self, x: List[_S]) -> List[Union[_T, _S]]: ...
    ...

This signature might cause other problems, though. This wouldn't solve the problem in general. For example, if the annotation would say # type: List[Union[int, str, None]] the error would persist, even though the code would be safe.

With some type system extensions we could fix this more generally. We'd need type variable lower bounds (we currently only have upper bounds), and to allow type variable bounds to refer to other type variables. I think that this would be a good signature:

T = TypeVar('T')
S = TypeVar('S')
U = TypeVar('U', lower_bound=Union[T, S])

class list(Generic[T]):
    def __add__(self, other: List[S]) -> List[U]: ...

Implementing this could be difficult.

Instead, we could special case list.__add__ in the type checker, since it's a very common operation. This would be ugly and potentially confusing, since the effective signature wouldn't match typeshed.

[ilevkivskyi]

Isn't this just a duplicate of #5492? (with a replacement: a common ancestor -> a union). I believe python/typeshed#2404 (or similar) can solve this as well (the downside however that there are several known issues with generic self-types).

[JukkaL]

Yeah, this is the same as #5492, but with different examples and discussion. I'll close this and move the discussion to the original issue.

[JukkaL]

Closed in favor of #5492.