-
Notifications
You must be signed in to change notification settings - Fork 768
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
[READY] Use a http wrapper for jedi #246
Conversation
Have it running, again with changing the following change (since I really need the python3 support; I fully agree it makes sense for now to not add that to the PR; one step at a time)
|
@reinhrst thanks for testing this out! So at this point the review is needed :) @Valloric @micbou @puremourning |
@vheon you want us to take a look at the jedihttp code itself? |
if you have the time would be great. |
""" Check if JediHTTP server is ready. """ | ||
try: | ||
return bool( self._GetResponse( '/ready' ) ) | ||
except: |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I'm not sure if you're aware or not, but usually you don't want a "bare" except:
.
A bare
except:
clause will catch SystemExit and KeyboardInterrupt exceptions, making it harder to interrupt a program with Control-C, and can disguise other problems. If you want to catch all exceptions that signal program errors, useexcept Exception:
.
https://www.python.org/dev/peps/pep-0008/#programming-recommendations
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
This applies to a lot of places in this pull request.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
No I didn't know, but I did like we did in the cs_completer
:S
Of course! No problem :) |
@Renstrom I highly recommend making line-level comments on the Reviewable.io review (link in PR description), not on github. :) |
A question taken from @reinhrst comment in #160 (comment):
So the question here is clear, should the new JediHTTP server have a secret shared with ycmd so we are sure that we answer only to ycmd requests? |
Reviewed 11 of 12 files at r1, 1 of 1 files at r3. ycmd/completers/python/jedi_completer.py, line 252 [r3] (raw file): try:
...
except Exception:
pass
try:
... ycmd/tests/test_utils.py, line 113 [r3] (raw file): Comments from the review on Reviewable.io |
@vheon It wouldn't hurt. Getting MAC auth right is tricky though; look at how ycmd does it. That code has been security reviewed by professional security researchers. BTW sorry for the delay in reviewing this; this + the JediHTTP code itself is pretty big and I don't have a solid chunk of time to look at it yet. I'll try to find time this weekend. Review status: 10 of 12 files reviewed at latest revision, all discussions resolved, all commit checks successful. Comments from the review on Reviewable.io |
Do you think is reasonable to add the HMAC feature later as another PR? That way the review will be much simpler and focused. Anyway don't worry about the delay ;) Review status: 10 of 12 files reviewed at latest revision, all discussions resolved, all commit checks successful. Comments from the review on Reviewable.io |
Would be nice If I could use the same hmac_plugin for bottle as ycmd use without basically duplicate the code :S Review status: 10 of 12 files reviewed at latest revision, all discussions resolved, all commit checks successful. Comments from the review on Reviewable.io |
Few idle thoughts:
Reviewed 10 of 12 files at r1, 2 of 2 files at r4. ycmd/completers/python/jedi_completer.py, line 100 [r4] (raw file): ycmd/completers/python/jedi_completer.py, line 117 [r4] (raw file): ycmd/completers/python/jedi_completer.py, line 121 [r4] (raw file): ycmd/completers/python/jedi_completer.py, line 189 [r4] (raw file):
? ycmd/completers/python/jedi_completer.py, line 194 [r4] (raw file): Comments from the review on Reviewable.io |
Review status: 11 of 12 files reviewed at latest revision, 2 unresolved discussions, some commit checks pending. ycmd/completers/python/jedi_completer.py, line 117 [r4] (raw file):
Any more light on this is welcome though :) ycmd/completers/python/jedi_completer.py, line 189 [r4] (raw file): Comments from the review on Reviewable.io |
It could be a thing to think about. We could do it later though if we decide to go that route. The thing that right now concerns me about it is the windows support.
That is a thing to consider, specially looking forward to the multiple JediHTTP solution for
I will see if I can do something about it :) Review status: 11 of 12 files reviewed at latest revision, 1 unresolved discussion, some commit checks pending. Comments from the review on Reviewable.io |
Review status: 11 of 12 files reviewed at latest revision, 1 unresolved discussion, some commit checks pending. Comments from the review on Reviewable.io |
Reviewed 1 of 1 files at r5. Comments from the review on Reviewable.io |
So with a little abstraction we could do it cross-platform. If that is the case then we could discuss the pro/cons about this and the cost of maintaining it in the future. I think it is better discussing this is ycm-dev or in another issue 👍 Review status: all files reviewed at latest revision, 1 unresolved discussion, some commit checks pending. Comments from the review on Reviewable.io |
Anyway I made a PR of the master branch against an empty branch of JediHTTP vheon/JediHTTP#1 so you can use reviewable for that as well 👍 Review status: all files reviewed at latest revision, 1 unresolved discussion, some commit checks pending. Comments from the review on Reviewable.io |
I think this makes a lot of sense to consider (was on my todo list to check why this path wasn't chosen in the first place). I can also imagine it would make sense to communicate over stdin/stdout instead of starting a server all together. I don't think we have any situation where anyone else would benefit from connecting to a live jedihttp process. Probably better indeed to discuss this ycmd-wide.
It certainly is; I indeed had the feeling that I was missing some completions in vim, and this is exactly the reason why (technically, it's not so much the
As far as I could quickly see, this (plus the directory of the file itself) is searched for a module. This means that in the example the module |
@reinhrst ok so having
in the process path is not ideal because it would bring completion that the user do not actually want/use. How should I use them then? I believe the same is true for |
Indeed. I guess it should be possible to remove everything you don't want from the |
I like this a lot. We have a plan to do a general purpose server for Jedi: davidhalter/jedi#385. So this would be a very good place for us to gather some experience with it. As for the sys.path modifications, I don't think there's anything that can be done right now. We're working on davidhalter/jedi#562 which will solve modifying the sys path. However, you can do that right now as well, just by replacing the sys.path (it's not very "clean" though). |
@davidhalter just to be clear: davidhalter/jedi#562 would allow to pass the |
@vheon Pretty soon. Maybe a month? I'm discussing a split of the issue with the maintainer of the PR. However, I don't see a release coming very soon. But you could just use the latest master. |
Regardless of the way we take forward, we still need to know the current directory (actually the directory of the entrypoint for your python program) to either add to I'm just wondering whether anyone knows what's in the sys.path of the ycmd process (which is what jedi currently uses). If the (current dir/entry point dir) is not in there, then we might decide not solving that issue until a later date. |
Just to know if it worth it to merge this in and then do the
If I'm not mistaken we already pass the vim current directory with every request, @puremourning did a PR recently, right?
I don't think that that is going to be an automatic thing. I see that more like a I think I'll have some time this weekend to look at the "directory-thing" |
I think we do on completion requests, but not all subcommands, etc. Wouldn't be hard to add, and I think using that is the right way to go. |
so, we should be raising an exception with def _BuildGoToResponse( self, definition_list ):
if len( definition_list ) == 1:
definition = definition_list[ 0 ]
if definition[ 'in_builtin_module' ]:
if definition[ 'is_keyword' ]:
raise RuntimeError( 'Cannot get the definition of Python keywords.' )
else:
raise RuntimeError( 'Builtin modules cannot be displayed.' )
else:
return responses.BuildGoToResponse( definition[ 'module_path' ],
definition[ 'line' ],
definition[ 'column' ] + 1 )
else:
# multiple definitions
defs = []
for definition in definition_list:
if definition[ 'in_builtin_module' ]:
defs.append( responses.BuildDescriptionOnlyGoToResponse(
'Builtin ' + definition[ 'description' ] ) )
else:
defs.append(
responses.BuildGoToResponse( definition[ 'module_path' ],
definition[ 'line' ],
definition[ 'column' ] + 1,
definition[ 'description' ] ) )
return defs
I do see the exception text in vanilla YCM, but not this fork (get the empty QF list). So something in the above is not working right. Review status: all files reviewed at latest revision, 8 unresolved discussions. Comments from the review on Reviewable.io |
BTW i checked and the Jedi commit on |
I'm doing more inspection and there are more than one thing that are not right:
content = """class Class():
propertyA = 1
"""
script = jedi.Script( content, 2, 2, '/foo' )
script.goto_definitions()
[]
script = jedi.Script( content, 2, 3, '/foo' )
script.goto_definitions()
[<Definition class int>] gives two distinct results if the col is 2 or 3, now considering that jedi has column 0-based then it seems that jedi is returning different results depending if we are on the
Review status: all files reviewed at latest revision, 8 unresolved discussions. Comments from the review on Reviewable.io |
So basically with the example above, if I'm am on the Review status: all files reviewed at latest revision, 8 unresolved discussions. Comments from the review on Reviewable.io |
Interesting, and agree that the behaviour of Jedi is dodgy However, we still have an issue our side, because with this test program: class A():
propertyA = 1 And current ycmd, running
With this fork:
So the behaviour is regressed either way (independent of Jedi bug, which is invariant). Review status: all files reviewed at latest revision, 8 unresolved discussions. Comments from the review on Reviewable.io |
Yes, I have to check for empty list of definitions which means that we cannot jump to a definition. I probably have to revisit the JediHTTP API for this and make sure to test this on this end and on JediHTTP itself Review status: all files reviewed at latest revision, 8 unresolved discussions. Comments from the review on Reviewable.io |
bbfa285
to
c4d07a5
Compare
I confirm that this is now fixed. However, there is still a little regression in that less informations is given to the user about the error. See the testcase from @puremourning. In the new version, only the Reviewed 1 of 4 files at r24. ycmd/completers/python/jedi_completer.py, line 262 [r24] (raw file): ycmd/completers/python/jedi_completer.py, line 290 [r24] (raw file): ycmd/completers/python/jedi_completer.py, line 295 [r24] (raw file): Comments from the review on Reviewable.io |
I think is enough to let the more specific exception prevail and we would get the old behaviour :) Thanks! Review status: 16 of 19 files reviewed at latest revision, 11 unresolved discussions. Comments from the review on Reviewable.io |
Review status: 16 of 19 files reviewed at latest revision, 10 unresolved discussions. ycmd/completers/python/jedi_completer.py, line 290 [r24] (raw file): ycmd/completers/python/jedi_completer.py, line 295 [r24] (raw file): Comments from the review on Reviewable.io |
Review status: 16 of 19 files reviewed at latest revision, 10 unresolved discussions. ycmd/completers/python/jedi_completer.py, line 253 [r25] (raw file): if not definitions:
raise ...
return ... ycmd/completers/python/jedi_completer.py, line 261 [r25] (raw file): Comments from the review on Reviewable.io |
Reviewed 4 of 4 files at r24, 1 of 1 files at r25. Comments from the review on Reviewable.io |
Review status: 18 of 19 files reviewed at latest revision, 10 unresolved discussions. Comments from the review on Reviewable.io |
Reviewed 1 of 1 files at r26. Comments from the review on Reviewable.io |
. Waiting for the Note: this PR worsens the shutdown issue on Windows. I really need to work on PR #282! Reviewed 2 of 4 files at r24, 1 of 1 files at r26. Comments from the review on Reviewable.io |
Awesome work that has earned its badge. :) Did someone file a bug with Jedi upstream? Let's not lose that, we seem to have a nice test case. Review status: all files reviewed at latest revision, 3 unresolved discussions. ycmd/tests/handlers_test.py, line 45 [r18] (raw file): Not in this PR, but we should come up with a better way of doing this than patching private variables. What we have leads to tight coupling and breaks encapsulation. Someone renames Comments from the review on Reviewable.io |
One last thing, could you rebase and squash the commits into one before we merge? I don't think we need 32 commits of history here. Review status: all files reviewed at latest revision, 3 unresolved discussions. Comments from the review on Reviewable.io |
7b6abd2
to
c42739a
Compare
Done
I already did; you can find it at davidhalter/jedi#671. If you read the response one can now understand why jedi has 0-based column, or at least it make some sense now.
Done Review status: all files reviewed at latest revision, 3 unresolved discussions. Comments from the review on Reviewable.io |
@homu r+ Review status: all files reviewed at latest revision, 3 unresolved discussions. Comments from the review on Reviewable.io |
📌 Commit c42739a has been approved by |
⚡ Test exempted - status |
[READY] Use a http wrapper for jedi This is the first step to support python completions for python version different from the one ycmd is currently running. # What this change does The jedi library is still used, but instead of calling it directly we now wrapped in a http+JSON server. # Testing ~~The test have been updated to solicit the completer first with a `FileReadyToParse` event, which would start a JediHTTP server if one is not running (similarly to the `OmnisharpServer` tests). The tests for the `GoTo` subcommand family have been updated to test all three of the subcommands.~~ # Notes Two subcommand have been added: `RestartServer` and `StopServer` but only the `RestartServer` is exposed through the user by the `DefinedSubcommand` method, this is because I think that the `StopServer` command is not useful for the user but is useful in the testing phase for shutting down the JediHTTP. I have one question: would make sense to rename any reference from `jedi_completer` to `python_completer`? If it is I will make another PR. @reinhrst If you could test this would be awesome :) Fixes ycm-core/YouCompleteMe#1827 <!-- Reviewable:start --> [<img src="https://reviewable.io/review_button.png" height=40 alt="Review on Reviewable"/>](https://reviewable.io/reviews/valloric/ycmd/246) <!-- Reviewable:end -->
This is the first step to support python completions for python version different from the one ycmd is currently running.
What this change does
The jedi library is still used, but instead of calling it directly we now wrapped in a http+JSON server.
Testing
The test have been updated to solicit the completer first with aFileReadyToParse
event, which would start a JediHTTP server if one is not running (similarly to theOmnisharpServer
tests). The tests for theGoTo
subcommand family have been updated to test all three of the subcommands.Notes
Two subcommand have been added:
RestartServer
andStopServer
but only theRestartServer
is exposed through the user by theDefinedSubcommand
method, this is because I think that theStopServer
command is not useful for the user but is useful in the testing phase for shutting down the JediHTTP.I have one question: would make sense to rename any reference from
jedi_completer
topython_completer
? If it is I will make another PR.@reinhrst If you could test this would be awesome :)
Fixes ycm-core/YouCompleteMe#1827