slow in large files

[davidhalter]

When I install the plugin and open a file in python + - 400 lines, when it gets slow when I start typing. And when I type the point, it takes about 2 seconds to appear oa listing.

Remembering that this was a project in Django.

also in: davidhalter/jedi-vim#20

from @gilsondev (in another issue)

[davidhalter]

If you want a quick solution for the typing issue, just disable: let g:jedi#show_function_definition = 0 (in jedi-vim).
The completion is slow in certain places, I really don't like it in certain places. I'm working on solutions.

If you can give me examples of slow code I would really be interested. (I haven't seen to much, beside my Jedi library, which is awfully recursive).

[gilsondev]

It happened in the cases that I was encoding a query with the Django ORM.

Happens is that as it has several methods (taking the slow typing), it takes about 1 second to display the list.

Another situation that makes it slow is charging the docstring of some class and method being entered. It has an option to disable for a while, until you see the reason for this slowness?

[davidhalter]

Huge files should be much faster now, at least for the function definition display. Completion may still be a problem, but I'm working on it (It remains in the dev branch for now, because I don't know how stable it is, but you may try it of course).

Basically you can disable:

let g:jedi#show_function_definition = 0
let g:jedi#popup_on_dot = 0

for the time being. This way, nothing starts automatically. Your vim will only be slow if you explicitly start the completion.

[davidhalter]

Awesome! The last few changes really improved this situation a lot! I'm typing/completing in a 14k file (wx._core) and it's just really really fast!

@gilsondev Could you please checkout the current revision (dev branch) and tell me if it's still slow?

[gilsondev]

Yes ;)

2012/12/17 David Halter notifications@github.com

Awesome! The last few changes really improved this situation a lot! I'm
typing/completing in a 14k file (wx._core) and it's just really really fast!

@gilsondev https://github.com/gilsondev Could you please checkout the
current revision (dev branch) and tell me if it's still slow?

—
Reply to this email directly or view it on GitHubhttps://github.com//issues/34#issuecomment-11469036.

---

Gilson Filho
Desenvolvedor Web
http://gilsondev.com

[davidhalter]

@gilsondev Could please send me a "slow" file? Doesn't have to be public, you also send me an email.

Otherwise I need to know the structure of this file. This means how big is your file, how big are your classes, functions, where you are trying to use completion, etc.

[edmondburnett]

I'm getting 1-2 second pauses when in insert mode (Vim) and moving over the isupper(), lower(), and upper() Python string methods. The file does not have to be large. See example code:

https://gist.github.com/4383325

[davidhalter]

@edburnett What do you mean by "moving over"? I cannot reproduce the slowness.

[edmondburnett]

By 'moving over' I mean using the arrow keys while in insert mode to navigate through the line. When the cursor arrives at the parentheses of those methods then it pauses/slows down.

I know navigating while in insert mode is not considered proper usage of Vim but it sounded similar to this bug so I thought they might be related.

[davidhalter]

I know now what you mean (and there is a certain lag sometimes [not on every move, though]). I'll try to fix it, but you have to give me some time. I know where it comes from and it's not that easy to fix :-)

[edmondburnett]

No worries, thanks for the great plugin. :)

[wladston]

I'm also affected by the slowness in many situations. Even if the file I'm editing is small, if the function comes from an external lib, it usually will slow down while on insert mode, right after I type in the "(".

doing the "let g:jedi#show_function_definition = 0" for now.

Btw, that's a great plugin, congratz!

[vext01]

If i type "sys." for the first time, I have to wait about 5 seconds. This is on a 2GHz machine.

Have you considered some kind of pre-compiled index of the common modules? os, sys, ...

Maybe such an index could be built on first run.

[davidhalter]

@vext01 You're on a Windows machine?

This index already exists. But that version (feature) is not available with pip, yet. Just try to use the master branch of github (but it's not as stable as I'd expect it to be).

[vext01]

On Sun, Mar 03, 2013 at 08:27:08PM -0800, David Halter wrote:

@vext01 You're on a Windows machine?

I'm using git head on OpenBSD.

Best Regards
Edd Barrett

http://www.theunixzoo.co.uk

[davidhalter]

@vext01 Is it faster on dev head?

[vext01]

Hi David,

On Mon, Mar 04, 2013 at 02:45:27AM -0800, David Halter wrote:

@vext01 Is it faster on dev head?

Unfortunately I don't notice a difference using the dev branch.

It is not too bad after the first time a large module is scanned for
completions. Just the first time with os or sys for example.

I tried it on a 3GHz machine and it was a bit faster. I guess we would
expect that.

Best Regards
Edd Barrett

http://www.theunixzoo.co.uk

[ocehugo]

Hello,

i can confirm @vext01 problem too, @davidhalter. The problem is worse if you are closing the file and opening other vim (quit vim and reopening) session...probably some missing cache feature around some packages.
open a file test.py:

import numpy as np
a = np. here it takes a good 9s on my laptop for options to appear and vim usage goes 100% cpu
a = np.zeros( here it show the dialog stuff

if we close the file and open again, +9 seconds to reload the numpy methods and etc.

Same for os, system, scipy, etc...

my laptop is a i3-350m, and sometimes take a very long time to load all the imports.

Maybe a favorite option to cache import classes and etc...or better an auto-adjust base on usage...or faster loadings :P

PS: tested on both master and dev.

[davidhalter]

@ocehugo @vext01 This issue is now my main work again.

[davidhalter]

@gilsondev Could you please check the latest master branch (jedi version 0.6) and check if it's still slow?

@ocehugo: The following should now be pretty fast: Loading VIM, completing numpy (as in your examples) closing VIM and then opening/completing again. The second time it should be fast, something like 1.6 seconds on my 4 year old notebook.

@vext01: sys loads on my 4 year old notebook very fast (< 0.5s). But this is probably only true after the second time reopening VIM.

[gilsondev]

sorry for the delay. I will update the library and I'll get in touch ok?

2013/5/7 David Halter notifications@github.com

@gilsondev https://github.com/gilsondev Could you please check the
latest master branch (jedi version 0.6) and check if it's still slow?

@ocehugo https://github.com/ocehugo: The following should now be pretty
fast: Loading VIM, completing numpy (as in @ocehugohttps://github.com/ocehugo

—
Reply to this email directly or view it on GitHubhttps://github.com//issues/34#issuecomment-17554716
.

---

Gilson Filho
Desenvolvedor Web
http://gilsondev.com

[davidhalter]

@gilsondev Sure, no problem!

[gilsondev]

@davidhalter It's ok 👍

[davidhalter]

From what I've heard, this doesn't seem to be a problem anymore. Jedi seems to be a lot faster now. If anybody doesn't agree, I can still reopen. But I'll close for now.

[wladston]

It's indeed way faster now, but the problem still occurs for me when editing source files with over 3k+ lines.

[davidhalter]

@wladston Oh, that might be true. Could you send me one? Because that seems awfully unpythonic :-). I still have room for a few optimizations, but I don't really have a "slow" file at hand, which is being edited by humans and not just computer generated (real life use case).

[ocehugo]

@davidhalter, thankz for that. The autocomplete loading are pretty fast now.

Actually some of my problems was inside the filesystem container. After i start a preload/cache service of some python/vim files the performance improves a lot (Including vim startup with a lot of bundles).

[wladston]

I think this would be a nice test case :

https://github.com/django/django/blob/master/django/forms/fields.py

it has over 1k size, it should make typing slow if using the dot popup and the function definition helper.

[davidhalter]

@wladston Well that's the thing, this file (and others of this size) is pretty fast. I don't have any lag problems. Do you have problems in there?

If so: What operating system, what hardware?

[wladston]

Indeed, I tried out with the django sources and it worked very well. I'm on a Macbook air / Mac mini, I don't think hardware is the issue ... I'll try again with source files from work and will see if I can produce a test case.

[davidhalter]

@wladston Look out for either: Long files or long sections without breaking functions/classes. That's what I suspect.

[wladston]

We have classes with over 1k lines, definition of lists with over 200 lines, source files with over 3k lines.... many includes, some includes have sources with over 2/3k lines... I wasn't able to produce a similar test file that slows down jedi, but it is still slow to the point of being unusable with these sources...

I noticed that the greatest slow downs happen when I type dot after typing an included object. It's fast with the builtins.

[ocehugo]

@wladson, did you see if your HD I/O are runing up to 100% when loading this files!? I say that because 50% of my problem with jedi auto loading was related to I/O issues over large files and a lot of small ones( and of course an already know little bug with my file system). Maybe you should do a test with preload of the request files via a init.d service or before editing the sources.

Example:
my_packages ::: array of paths for preloading

my_packages=("$HOME/.vim" "/usr/lib64/python2.7/site-packages/numpy" "yourprojectfolder" "yoursourcepy")
for i in ${my_packages[@]}; do
echo "Preloading ${i}"
find ${i} -xdev -type f -exec cat '{}' ; > /dev/null
done

As long you dont clean the cache, the files will remains on ram. Try this before casting vim & your source file...after that if typing "." for methods is still slow, well, probably it's jedi-vim fault :P.

PS: It's important to preload all packages that are imported in ur python files to make sure that I/O it's not bounding your performance.
PS2: you can put this tiny hack on a startup script or use preload from sf.net.

[wladston]

@ocehugo I ended up quitting my job at the company (I dislike working with huge files), so currently I don't have access to those files to test anymore ...

Anyways, thanks a lot, your post will be a good guidance for other programmers that might run into this problem.

[gilsondev]

It's easy to get fit for summer!

^--- spam by email (removed the link http://huff.to/y2RlQg) ~ Dave

[The-Compiler]

With http://git.the-compiler.org/qutebrowser/tree/qutebrowser/config/configdata.py?id=de2f26760711655ad2c6e8faddbec702b5da3874 vim gets unusably slow, especially when typing inside some of the description strings. This is only 1000 lines though, and a rather fast machine (i7, SSD).

[gilsondev]

Oi Davidhalter/jedi,

Estamos felizes em anunciar que a Talentoday agora está disponível em português do Brasil.
Queremos que o Brasil seja o país da América do Sul mais representado na nossa plataforma. Até o momento, temos 76.000 brasileiros registrados nosite. Aceita o desafio? Compartilhe com seus amigos. Vamos, Brasil!
Espalhe por aí: a Talentoday agora tem sotaque brasileiro :)
Time Talentoday

https://www.talentoday.com/lang/pt-BR

Não quero mais estar inscrito
https://www.talentoday.com/contacts/unsubscribe/bd35196df6561afb647532355a97c1ec

Copyright © 2014 Talentoday, Todos os direitos reservados