Help text of builtin functions – missing signatures

[frenzymadness]

I don't understand the deep details here but I think something went wrong in commit bdfb694 when some of the builtin functions where transferred to argument clinic.

The problem I see is in the help texts of those functions. Here are a few examples of help(<function>):

iter

Python 3.11.4

Help on built-in function iter in module builtins:

iter(...)
    iter(iterable) -> iterator
    iter(callable, sentinel) -> iterator
    
    Get an iterator from an object.  In the first form, the argument must
    supply its own iterator, or be a sequence.
    In the second form, the callable is called until it returns the sentinel.

Python 3.12.0b4

Help on built-in function iter in module builtins:

iter(...)
    Get an iterator from an object.

    In the first form, the argument must supply its own iterator, or be a sequence.
    In the second form, the callable is called until it returns the sentinel.

As you can see, the help in 3.12 is talking about two forms but there are no signatures so it doesn't make sense.

next

Python 3.11.4

Help on built-in function next in module builtins:

next(...)
    next(iterator[, default])
    
    Return the next item from the iterator. If default is given and the iterator
    is exhausted, it is returned instead of raising StopIteration.

Python 3.12.0b4

Help on built-in function next in module builtins:

next(...)
    Return the next item from the iterator.

    If default is given and the iterator is exhausted,
    it is returned instead of raising StopIteration.

Again, the help text talks about iterator and default but missing signature means that the reader might not know what those are.

getattr

Python 3.11.4

Help on built-in function getattr in module builtins:

getattr(...)
    getattr(object, name[, default]) -> value
    
    Get a named attribute from an object; getattr(x, 'y') is equivalent to x.y.
    When a default argument is given, it is returned when the attribute doesn't
    exist; without it, an exception is raised in that case.

Python 3.12.0b4

Help on built-in function getattr in module builtins:

getattr(...)
    Get a named attribute from an object.

    getattr(x, 'y') is equivalent to x.y
    When a default argument is given, it is returned when the attribute doesn't
    exist; without it, an exception is raised in that case.

Again, a similar case here.

Linked PRs

• gh-107526: Revert "gh-100357: Convert several functions in bltinsmodule to AC" #107542
• gh-107526: Fix test_module_level_callable_unrepresentable_default #108187

[sobolevn]

Yes, this is because of NULL default :(

arg: object = NULL

[frenzymadness]

Is there any way to fix it?

[sobolevn]

I've submitted https://discuss.python.org/t/ac-null-defaults-prevent-correct-signatures-lets-add-inspect-unrepresentable-to-fix-this/30753

I wanted to do it for quite a long time, sorry for ignoring this item in my backlog :)

[encukou]

Is this feasible for 3.11?
If not, should we add the signatures to the doc text manually, in these cases where AC doesn't generate anything?

FWIW, this breaks Jedi, which currently parses the manually-formatted signatures in docstrings: davidhalter/jedi#1952
They would be happy to move to inspect.signature once that works. Until it does, parsing __doc__ seems like a sensible workaround, so this might be considered a regression.
So I'm flagging this as a potential blocker.

[sobolevn]

I will send a PR with doc-based signatures later today.

Today I broke a Jedi! The force is strong in me 😆

[encukou]

I meant 3.12, not 3.11. Got my version numbers mixed up, sorry!

[JelleZijlstra]

The fix @sobolevn proposes doesn't seem feasible for 3.12, as it requires introducing a new object in the inspect module. As a less invasive fix, maybe we can directly add these signatures to the docstrings?

[sobolevn]

@JelleZijlstra that's exactly what I am proposing for 3.11 and 3.12 in #107526 (comment) :)

[sobolevn]

Ok, I think that we would need to revert these changes. Here's why:

1. We cannot simply add signatures back in AC. Here's what is generated:

/*[clinic input]
dir as builtin_dir

    arg: object = NULL
    /

dir([object]) -> list of strings

Show attributes of an object.

...
[clinic start generated code]*/

static PyObject *
builtin_dir_impl(PyObject *module, PyObject *arg)
/*[clinic end generated code: output=24f2c7a52c1e3b08 input=2e2cb290445c88f3]*/

outputs:

PyDoc_STRVAR(builtin_dir__doc__,
"dir($module, arg=<unrepresentable>, /)\n"
"--\n"
"\n"
"dir([object]) -> list of strings\n"
"\n"
"Show attributes of an object.\n"
"\n"
"...");

But, that's not what we want!

2. Argument clinic has only_docstring option for functions, but I will need to rework how it sets this value to True to be able to use it. This would require extra testing and stuff
3. I hope that https://discuss.python.org/t/ac-null-defaults-prevent-correct-signatures-lets-add-inspect-unrepresentable-to-fix-this/30753 will land for new code, so I won't have to change the clinic code

[davidhalter]

FYI I don't think the effects on Jedi are extreme (speaking as its author). It's just that it's a bit unfortunate that the signatures won't be shown anymore in that case for 3.12. So yes, it's a regression, but not a problematic regression. So if argument clinic helps you here with other issues, it might just be worth keeping it. IMO the more problematic change here might be that the docstrings are just not as good as they were previously. Especially for iter the docstring talks about In the first form, [...]. What is the first form? In the other docstrings there are weird comments as well.

In the end it would be really nice if these functions were eventually supported by inspect.signature, but I don't think you need to hurry there either.

[serhiy-storchaka]

#107782 can help.

[JelleZijlstra]

The immediate issue was fixed here. We still have the broader issue that inspect.signature can't handle `', but that's tracked in #87233.