-
-
Notifications
You must be signed in to change notification settings - Fork 2.8k
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
strict_optional=False makes checking scripts using xarray to fail #9437
Comments
Thanks for the report. I have reproduced this using Python 3.8, in 0.782 as well as on master. One needs to I'm pasting the traceback from the master here:
|
Maybe this is a new regression? Someone with som time on their hands could perhaps bisect to a specific commit that introduced this. |
I checked, since it seemed like a good use case for mypy_primer. It does appear to be a regression in mypy, but an old one: #6442. I didn't find this too enlightening (though you'll notice the stacktrace goes through How to bisect (somewhat) easily: I added the following to primer.py, where xarray_bug.py just contains
And then ran:
Edit: I added the The somewhat* is because bisecting over large ranges of (old) mypy revisions proved to be fraught, so I had to narrow it down a little with a couple other mypy_primer commands, hence the |
Thanks; I agree #6442 doesn't look like a great starting point. Sorry, this is one of my long rambling crash narratives. I used pdb a lot. TL;DR: Root cause I reproduced this on the master of xarray, the crash line is here: Here The offending line is this: for name, coord in self.coords.items():
... I replaced it with this: c = self.coords
for name in c:
coord = c[name] and then the crash moved to a later place in the same file: for dim, coord in self.coords.items(): So it seems there's something funny about After "fixing" the second case in a similar way, the crash moved to this line in if isinstance(other, Dataset): Here After commenting out that block and the next ( Now let's focus on mypy. In all cases the crash is here: Lines 466 to 468 in 4fb5a21
That's this function in class def visit_partial_type(self, left: PartialType) -> bool:
# This is indeterminate as we don't really know the complete type yet.
raise RuntimeError But that function wasn't actually changed by #6442, so maybe this is a dead end. (It is ancient, from 2015, commit fc72019, when partial types were introduced; the commit log claims the work is "still very incomplete".) So what's going on? We're in a subtype check, which is implemented using (Quick intro to partial types: these are used to avoid "Need type annotation" errors in cases like this:
From the last line we infer that Are partial types also involved with forward references? I dunno.) If I simply change the function to And... Aha! This leads here: # mutable objects should not be hashable
# https://github.com/python/mypy/issues/4266
__hash__ = None # type: ignore So now at least I have a root cause. This also leads to the workaround: __hash__: Optional[Callable[[], int]] = None # type: ignore # UPDATE: added type: ignore ( UPDATE: Probably better workaround: __hash__: None = None # type: ignore Note that the comment leads to #4266, which seems relevant (it's about Alas, there's a little more to it, because this code doesn't tickle the crash. Maybe it's the import cycle between dataarray.py and coordinates.py? class C:
__hash__ = None # type: ignore More maybe later, or someone can just come up with a fix based on this idea. |
At least I now have a short repro.
|
Hmm... The short repro helped because the debug cycle is quicker, but I'm still not sure how to fix it. What happens is that
But the partial type just stays there. Here's something weird. The repro still crashes without the And what's the connection with |
This is great! I was able to repro^ in a single file, order doesn't matter. I think the |
Okay, here's a shorter repro (but using lists, not tuples):
|
Seems like the trouble is when we try to join with something hashable, inside the class definition. That is, the following does not crash:
|
Okay, it looks like we special case this already, but five lines too late: Line 555 in 4fb5a21
PR coming soon! |
Actually, we shouldn't just tweak that check, the problem is a little more general. This fails with no special flags:
You can also set |
Good finds! And quite surprising, given where this started... |
In particular, this affected hashables. Fixes #9437 Co-authored-by: hauntsaninja <>
python/mypy#9437 still affect mypy 0.800
🐛 Bug Report
When using mypy with
strict_optional = False
in the config file or using the--no-strict-optional
flag in the terminal to check a file using thexarray
package, it fails with an scary INTERNAL ERROR message.This might be a bug better suited for the
xarray
team, but thought to check with you first.To Reproduce
Write a python script just importing
xarray
:And check it with mypy using
mypy --no-strict-optional script.py
Expected Behavior
Everything works fine and I get:
Success: no issues found in 1 source file
. This is indeed the behavior if the--no-strict-optional
flag is NOT used.Actual Behavior
I get the following output (personal info replaced by XXXXX):
I've tried also the development version of mypy, as sugested, with identical results, and also used the
--show-traceback
option, with results in:Your Environment
--no-strict-optional
mypy.ini
(and other config files):strict_optional = False
has the same effect.The text was updated successfully, but these errors were encountered: