Unbound local due to missing "partial\n" in py.typed

[wcooley]

• What are the versions of mypy and Python you are using?
  mypy: 0.641
  Python: 3.6.5 built with pyenv on macOS 10.13.6
• What is the actual behavior/output?
  Crash with traceback.
• What is the behavior/output you expect?
  Either not crashing or exiting with a helpful error message.
  • Do you see the same issue after installing mypy from Git master?
    I have not tried but seeing where the obvious error in the code is, I do not see any difference in master at this time.
• What are the mypy flags you are using? (For example --strict-optional)
  No flags are needed to cause the failure.

I am attempting to create a stub-only package for a 3rd party package. In attempting to figure out why it is not working, I created an empty py.typed alongside the installed stubfiles (I know PEP 561 says it is not necessary... but it does not forbid it either). Afterwards, I get the following UnboundLocalError exception when I run mypy, which goes away when I create it with printf "partial\n" > .../py.typed:

Traceback (most recent call last):
  File "/my/virtualenv/bin/mypy", line 11, in <module>
    sys.exit(console_entry())
  File "/my/virtualenv/lib/python3.6/site-packages/mypy/__main__.py", line 7, in console_entry
    main(None)
  File "/my/virtualenv/lib/python3.6/site-packages/mypy/main.py", line 92, in main
    res = build.build(sources, options, None, flush_errors, fscache)
  File "/my/virtualenv/lib/python3.6/site-packages/mypy/build.py", line 155, in build
    result = _build(sources, options, alt_lib_path, flush_errors, fscache)
  File "/my/virtualenv/lib/python3.6/site-packages/mypy/build.py", line 204, in _build
    graph = dispatch(sources, manager)
  File "/my/virtualenv/lib/python3.6/site-packages/mypy/build.py", line 2118, in dispatch
    graph = load_graph(sources, manager)
            if pkg_id not in self.ns_ancestors and self.fscache.isdir(path):
  File "/my/virtualenv/lib/python3.6/site-packages/mypy/build.py", line 2320, in load_graph
    ancestor_for=st)
  File "/my/virtualenv/lib/python3.6/site-packages/mypy/build.py", line 1402, in __init__
    ancestor_for, root_source, skip_diagnose=temporary)
  File "/my/virtualenv/lib/python3.6/site-packages/mypy/build.py", line 1941, in find_module_and_diagnose
    path = manager.find_module_cache.find_module(file_id)
  File "/my/virtualenv/lib/python3.6/site-packages/mypy/modulefinder.py", line 108, in find_module
    self.results[id] = self._find_module(id)
  File "/my/virtualenv/lib/python3.6/site-packages/mypy/modulefinder.py", line 159, in _find_module
    third_party_inline_dirs.append((runtime_path, True))
UnboundLocalError: local variable 'runtime_path' referenced before assignment

The cause of the error is obvious:

 157                       if fscache.read(stub_typed_file).decode().strip() == 'partial':
 158                           runtime_path = os.path.join(pkg_dir, dir_chain)
 159                       third_party_inline_dirs.append((runtime_path, True))

runtime_path is not assigned outside of the if conditional on line 157 and not checked before the append on line 159.

It seems like there should be an else clause that at least exits without a traceback with a helpful error message, like 'Either {stub_typed_file} should not exist or it should have "partial\n" at the beginning.' Alternatively, the py.typed file could be ignored as it is redundant.

But I am confused about this behavior with the py.typed marker file in a "<...>-stubs" package: PEP 561 says that if the stubs package is partial, it must include py.typed with "partial\n", which implies to me that if the py.typed file exists without "partial\n", then it indicates the package is not partial (which might be the default assumption and therefore redundant). Perhaps this is an ambiguity I should take up with the PEP 561 author?

[gvanrossum]

@ethanhs Can you triage this? This sounds like a release blocker. #5960

[wcooley]

This sounds like a release blocker.

That's obviously your call, but I should emphasize that this only happened because I was blundering about, trying something that was expressly outside the defined scope of PEP-561, in an attempt to get a stub-only distribution & package for a 3rd party distribution & package recognized by mypy (OTOH, the question of why my stub-only it isn't being recognized might itself be....)

[ethanhs]

This sounds like a release blocker.

That's obviously your call, but I should emphasize that this only happened because I was blundering about, trying something that was expressly outside the defined scope of PEP-561

The fix is easy enough that it should get cherrypicked before it matters. PR incoming :)

[ethanhs]

Also @wcooley feel free to open a new issue if you think mypy isn't finding a stub package it should.