stack exec is unreasonably slow

[cocreature]

I primarily use stack exec to execute the executables I have just built. In the vast majory of cases (probably also for other people) all stack exec needs to do is set the PATH correctly since my executables don’t depend on any other settings. However the overhead stack exec adds has slowly grown to the point where it becomes annoying:

stack exec shake-reve -- --help  0.91s user 0.19s system 99% cpu 1.101 total

vs

./.stack-work/dist/x86_64-linux/Cabal-1.22.5.0/build/shake-reve/shake-reve   0.01s user 0.00s system 72% cpu 0.009 total

I realize that to completely set up the environment stack exec has to do more than just set the path correctly but if that causes such a drastic slowdown, could we get a --fast (probably needs a better name) option for the common case where it just needs to set the path?

[luigy]

Yes, it would be great if the performance could be improved.
In the meantime, I've been doing stack exec $SHELL to take the hit only once since I only need my PATH to be set

[mgsloan]

I've annotated the log with some notes:

2016-03-08 20:42:06.393949: [debug] Checking for project config at: /home/mgsloan/fpco/stack/stack.yaml @(stack_H2uuVvppPbeAJ1uiFxrFbt:Stack.Config src/Stack/Config.hs:761:9)
2016-03-08 20:42:06.394302: [debug] Loading project config file stack.yaml @(stack_H2uuVvppPbeAJ1uiFxrFbt:Stack.Config src/Stack/Config.hs:779:13)

Determining if stack is linked with gmp4
2016-03-08 20:42:06.396023: [debug] Run process: ldd /home/mgsloan/.local/bin/stack @(stack_H2uuVvppPbeAJ1uiFxrFbt:System.Process.Read src/System/Process/Read.hs:269:3)

Decoding the resolver's set of packages (~0.02s)
2016-03-08 20:42:06.407513: [debug] Trying to decode /home/mgsloan/.stack/build-plan-cache/x86_64-linux/lts-5.3.cache @(stack_H2uuVvppPbeAJ1uiFxrFbt:Data.Binary.VersionTagged src/Data/Binary/VersionTagged.hs:55:5)
2016-03-08 20:42:06.424316: [debug] Success decoding /home/mgsloan/.stack/build-plan-cache/x86_64-linux/lts-5.3.cache @(stack_H2uuVvppPbeAJ1uiFxrFbt:Data.Binary.VersionTagged src/Data/Binary/VersionTagged.hs:64:13)

Decoding the hackage index (~0.3s)
2016-03-08 20:42:06.425239: [debug] Trying to decode /home/mgsloan/.stack/indices/Hackage/00-index.cache @(stack_H2uuVvppPbeAJ1uiFxrFbt:Data.Binary.VersionTagged src/Data/Binary/VersionTagged.hs:55:5)
2016-03-08 20:42:06.722030: [debug] Success decoding /home/mgsloan/.stack/indices/Hackage/00-index.cache @(stack_H2uuVvppPbeAJ1uiFxrFbt:Data.Binary.VersionTagged src/Data/Binary/VersionTagged.hs:64:13)

Checking if the system compiler can be used (~0.06s)
2016-03-08 20:42:06.742570: [debug] Run process: ghc --info @(stack_H2uuVvppPbeAJ1uiFxrFbt:System.Process.Read src/System/Process/Read.hs:269:3)

Getting the exact ghc version (~0.04s)
2016-03-08 20:42:06.806545: [debug] Run process: ghc --numeric-version @(stack_H2uuVvppPbeAJ1uiFxrFbt:System.Process.Read src/System/Process/Read.hs:269:3)

Getting the cabal version (~0.03s)
2016-03-08 20:42:06.845667: [debug] Run process: ghc-pkg --no-user-package-db field --simple-output Cabal version @(stack_H2uuVvppPbeAJ1uiFxrFbt:System.Process.Read src/System/Process/Read.hs:269:3)

Getting the location of the global package DB (~0.03s)
2016-03-08 20:42:06.869424: [debug] Run process: ghc-pkg --no-user-package-db list --global @(stack_H2uuVvppPbeAJ1uiFxrFbt:System.Process.Read src/System/Process/Read.hs:269:3)
2016-03-08 20:42:06.891551: [debug] Run process: echo hi @(stack_H2uuVvppPbeAJ1uiFxrFbt:Stack.Exec src/Stack/Exec.hs:61:5)

So, about 3/5 of the time is being spent decoding the hackage index, which isn't being used.

It's tricky to do anything about the remaining 2/5, because in order to populate the PATH env var, we need to know the ghc and cabal version being used. For system installations managed by stack, we could trust that the version is accurate - we're still asking ghc for its version in that case.

You'll save something like 0.06s by doing --no-system-ghc, skipping the system ghc check.

[mgsloan]

I've pushed a commit which only loads up the hackage index when necessary. Now it takes 50% less time or so! There's still some work to do here, but not relevant to stack exec: #1892

[cocreature]

@mgsloan This is a huge improvement! Thanks a lot. It’s now below the point where it starts to be painful for me, so feel free to close this if there is no way to improve it further (with reasonable effort).

[mgsloan]

Welcome, glad it's below the point of being painful!

Yeah, I think optimizing it further is more effort than it's worth at this point. I've written down a misc TODO note to maybe revisit the ghc version checking stuff for stack installed ghcs.