Adding Script.documentation.

[davidhalter]

Emerged from #340:

I've wanted a Script.documentation() for quite some time now. This function would return a Documentation or Doc object that contains all the gathered docstrings along the line, including rendered call signatures (look at parser.representation). This would also include Attribute Docstrings.

The return class of Script.documentation could look something like this:

class Documentation(object):
    def __init__(self, *definitions):
        self._definitions = definitions

    def __str__(self):
        return '\n\n'.join(self._list_strings())

    def _list_strings(self, raw=False):
        strings = []
        for definition in self._definitions:
            if raw:
                try:
                    strings.append(self._definition.doc)
                    continue
                except AttributeError:
                    pass
            try:
                strings.append(self._definition.raw_doc)
            except AttributeError:
                pass
        return strings

    def raw(self):
        """raw differs from the normal documenation in that it returns the original 
        docstrings without call signatures attached."""
        return '\n\n'.join(self._list_strings(raw=True))

Of course it's not planned to only have a raw and __str__ method available. We could do more. This is an early example.

There are a couple of reasons why Documentation is necessary:

1. I have disabled pydoc keywords and operators, because it conflicted with goto_definition. Basically the pydoc stuff that I disabled in Jedi was what you get with help('except'). This can be really practical, especially for beginners. A keyword is really no definition. It's not "specified" somewhere. So I removed the hacks used to return it in Script.goto_definition.
2. I have also disabled attribute docstrings support (introduced with Attribute docstring support #141), because it wasn't really working properly (it caused some bugs). However, attribute docstrings are still available in the parser. We just have to find ways to evaluate them properly. We still have to figure out if we want statements with attribute docstrings also in get_definition as a separate definition, but I think that's a different discussion (and not such an important one, we could also add a bool param to get_definition).
   It is undoubted that those attribute docstrings add relevant documentation to the code and should be available within Jedi.
3. There are a few things that almost all Python editors can do:
   • Autocompletion
   • Goto
   • Show documentation
     Except for displaying documentation, we have a good and broad support for the other ones. I think I don't need to explain to you guys how complicated it is to get documentation with Jedi as of now (and how confusing).

[tkf]

I think we would like to make sure the name of the class makes sense for some long time.

How about having Help class? It's a bit more abstract and so extensible. For example, getting source code may help you sometime. In that case, Help.code() makes sense but Documentation.code() does not. It is even possible to mix docstring and source code like .doc(with_code=True). I think other possible features I mentioned in #340 fit in the range of Help.

A keyword is really no definition. It's not "specified" somewhere. So I removed the hacks used to return it in Script.goto_definition.

I think you could implement it correctly rather than removing the hack. How about subclassing definition class which is only for keywords (e.g., that subclass does not have .line or it is set to None)? I guess you still need definition class for keyword completion, right? Then having such subclass makes sense.

Script.documentation() is fine. But please make it possible to do the same with goto_definitions because I'll not use Script.documentation. I need file path, line, column etc. with documentation and I want to get keyword help with the same function.

[davidhalter]

I changed it to Help for now, I think that makes more sense in the long term.

However, I have to come to a conclusion for the Definition.documentation() function. I renamed it to docstring(raw=False) and it returns unicode. Always. I think this makes sense, because looking up a docstring is such a basic action. In the future, we might add a Definition.help() method. Now however, Help doesn't have any meaning and I will rename it to _Help.

These actions don't resolve the discussions about Script.documentation() or Script.help(). My plan is to still add such a function.

But please make it possible to do the same with goto_definitions because I'll not use Script.documentation. I need file path, line, column etc. with documentation and I want to get keyword help with the same function.

My plan (far away from starting to implementing it) is to create a Script.help() function that returns a Help object on which you can call definitions (which would return both definitions and keywords). But we could also add it to goto_assignments or goto_definitions. Not sure yet, doesn't really matter, yet. Will happen in 6 months or so.

[davidhalter]

Implemented Script.help(), which is essentially a wrapper around goto with additional keyword information. Might include other stuff in the future as well.