libgit2 git_status_list_new is really slow.

[Stebalien]

For me, exa takes ~3 seconds to run on the rust repository and most of this time is spent in libgit2's git_status_list_new function (specifically, it appears that most of this time is spent calculating SHA1 hashes). However, git status --porcelain runs in about .1 seconds. The correct fix would be to fix libgit2 however, in the meantime, you might consider using git directly.

Note git2::Repository::statuses(opts) is a wrapper for libgit2's git_status_list_new function.

[ogham]

Yeah, I've had reports of the Git functionality being really slow on large Git repositories. I'm looking into what I can do with libgit2 to mitigate this. As you say, it seems to process a lot of SHA1 hashes before it even starts doing anything, and I think that I'm asking it to do a lot more work than I need to.

I'd rather not call Git directly, as it's hard to get it working accurately when you're outside of the repository directory (you have to find the root). If I can't find a way to speed it up in the near future, I'll at least include a --no-git stopgap so you can disable the column on a per-invocation basis.

[Stebalien]

Asking libgit2 to update the index shaves about a half of a second off of subsequent runs but none of the other options helped (except passing StatusOptions::new().show(ShowOptions::Index) but that also gets rid of most of the useful information so it's a non-starter). I'd also rather not call git directly but I don't know of a better fix. You could find the git root using libgit2 and then operate on it using the git command.

[lilydjwg]

exa takes quite a few seconds in my rustc repository, even without the --git option. I don't understand why exa tries to read git information when it doesn't show them.

[ogham]

It shouldn't do that! Let me test this.

[ogham]

Confirmed - that's definitely wrong!

[ogham]

d547c3f should fix it. Thanks for noticing the problem!

[ogham]

I think this might be related: libgit2/libgit2#4230

[dickshaydle]

I think the problem is that it searches in all subfolders, although only the current folder is needed.

You can see that if you take any larger folder that is not yet a git repo, make it git repo with
git init
and than look at the strace output
strace -f exa -l --git

The -f for strace means, it follow all threads / forks of that process.

[ariasuni]

exa is getting all the statuses of every file though the libgit2 and store them so it only had to query them once. Now I guess we could try to ask just about the file we find if it’s faster (or use that strategy only when we don’t recurse or something like that), or just parse the output of git like many other projects did.

Anyway, it’s a non-trivial amount of work for something that already works, so any help is welcome.