Error messages don't tell you where the problem originates

[cjw296]

I'm trying to use this package when generating the API docs for https://github.com/simplistix/sybil.

I've hit an issue, but it's made more tricky because the error messages give little clue as to where the problem originates:

WARNING: Cannot resolve forward reference in type annotations of "sybil.Sybil": name 'sybil' is not defined
WARNING: Cannot resolve forward reference in type annotations of "sybil.Document.parse": name 'sybil' is not defined
WARNING: Cannot resolve forward reference in type annotations of "sybil.Document.push_evaluator": name 'sybil' is not defined
WARNING: Cannot resolve forward reference in type annotations of "sybil.Document.pop_evaluator": name 'sybil' is not defined
WARNING: Cannot resolve forward reference in type annotations of "sybil.document.PythonDocStringDocument.parse": name 'sybil' is not defined
WARNING: Cannot resolve forward reference in type annotations of "sybil.Region": name 'sybil' is not defined
WARNING: Cannot resolve forward reference in type annotations of "sybil.parsers.abstract.codeblock.AbstractCodeBlockParser": name 'sybil' is not defined
WARNING: Cannot resolve forward reference in type annotations of "sybil.parsers.abstract.skip.AbstractSkipParser": name 'sybil' is not defined
WARNING: Cannot resolve forward reference in type annotations of "sybil.parsers.abstract.clear.AbstractClearNamespaceParser": name 'sybil' is not defined
WARNING: Cannot resolve forward reference in type annotations of "sybil.parsers.rest.CodeBlockParser": name 'sybil' is not defined
WARNING: Cannot resolve forward reference in type annotations of "sybil.parsers.markdown.CodeBlockParser": name 'sybil' is not defined
WARNING: Cannot resolve forward reference in type annotations of "sybil.parsers.myst.CodeBlockParser": name 'sybil' is not defined

These problems actually appear to be in the sybil.typing module due to #402, but there's no mention of that module or any filenames in the above.

[hoodmane]

The problem here actually has nothing to do with TYPECHECKING, you would get exactly the same error if TYPE_CHECKING was True. Strings in types are like macros: you need all names to be in scope everywhere that you use them. This is a pretty fundamental limitation of runtime types in Python. The reason mypy is okay with it is that it doesn't use runtime types at all it does a custom analysis of the ast. I guess for us to fix it we'd have to switch to using mypy to query the types or something.

The following is a minimal reproduction:

Minimal example

a.py

from typing import Callable
    
class A:
    pass

T =  Callable[['A'], None]

b.py

from a import T
from typing import get_type_hints

def f(t: T):
    pass

# The following fails:
get_type_hints(f)

sphinx-autodoc-typehints tries to use get_type_hints to resolve the type of f and fail to do so because the Python interpreter has not annotated T with any information about the file in which it is defined. Likewise, because T has no information about where it was defined, we cannot tell you in an error because the information about where it came from is not easily available. There is probably a way to invoke mypy and ask it about the type, but short of that we're pretty helpless here.

I think in your case you can fix it with:

if TYPE_CHECKING:
    from sybil import Example, Document, LexedRegion, Region


#: The signature for an evaluator. See :ref:`developing-parsers`.
Evaluator = Callable[['Example'], Optional[str]]

#: The signature for a lexer. See :ref:`developing-parsers`.
Lexer = Callable[['Document'], Iterable['LexedRegion']]

#: The signature for a parser. See :ref:`developing-parsers`.
Parser = Callable[['Document'], Iterable['Region']]

Python 3.12 gives a way out if people use the new type keyword:

type Evaluator = Callable[['Sybil.Example'], Optional[str]]

Then Evaluator has a __module__ attribute and a __value__ attribute so if one wants to expand it out the information about its site of definition is available. Once you get to this point, though, you'd start running into problems with TYPE_CHECKING.

[cjw296]

Yeah, I don't think your suggestion works as we just end up back at #22

[hoodmane]

I don't think you can hit #22 with your code because the import guard is only in typing.py but you only use the annotations in other files. Either those other files have Example, Document, and LexedRegion and it's fine or they don't and it's not fine.

[hoodmane]

In any case, the errors are confusing, the problem is difficult to understand, and when you understand the problem it is difficult to work around. It would be nice if we could improve the situation in some way.

[hoodmane]

Ah I see do you hit #22 for example with Document in https://github.com/simplistix/sybil/blob/master/sybil/example.py