Possible to shorten (hamlet) compile times in development?

[smichel17]

Background

I'm somewhat new to Haskell. Technically I've been touching it here or there for a few years, but I never invested any time in learning the language and ecosystem until recently. So, please don't assume I know things which should be obvious to a seasoned Haskeller.

Use Case / Problem

Writing css usually involves making tons of tiny iterations until your interface looks right. Having to wait for your code to compile in order to see the result of each iteration is painful. To help with this, yesod offers Reload mode (scroll down slightly), which allows css/js changes to be reflected ~instantly. 🎉

Unfortunately, that page seems to indicate that reload mode is not available for Hamlet… and if you build sites using any type of pre-built framework, classes are mainly applied directly in your html, which means you still need to wait for a full (incremental) recompile. It's particularly bad in Tailwind CSS (which I'd like to use), because with utility classes, nearly every minor tweak happens in the hamlet, so you get almost no benefit from reload mode.

Goal

I had a feeling that the project I was trying to optimize had some quirks which were slowing down builds. Before optimizing that particular code further, or migrating off of yesod, I wanted to know: "Is there any ergonomic way to use tailwind + yesod? Are quick builds/reloads of hamlet changes possible in yesod?

So, I decided to start from the standard scaffolded site and see if I'll be able to get a setup where changes to the html eqivalent are reflected ~instantaneously in the browser during development. I'm willing to give up features like type-safe routes, even in production, if that's necessary to achieve this; I still need splices, but I'm willing to give up type-checking on them.

Attempts

A simple file in its own module

Since right now we're looking for whether instant reloads are even possible, we only need to try the simplest file, so I added one, in its own module: b15a6ac.

hamletFileReload

Despite what the yesod book says, it seems that reload mode is actually available for Hamlet, via hamletFileReload… with a much smaller subset of features, but that's OK for now.

Unfortunately, naively replacing the only usage of hamletFile with hamletFileReload (6e25e0c) had no effect on compile times. Although, maybe yesodweb/yesod#665 means that I need to do something more involved to get hamletFileReload working?

Current performance

1. Get things ready for an incremental rebuild. I doubt the ghc options are needed, but I wanted to duplicate the command we care about (3) exactly.

   stack build --fast --ghc-options "-ddump-to-file -ddump-timings" yesod-perf-test:lib --flag yesod-perf-test:dev --flag yesod-perf-test:library-only

2. Make a trivial change to the hamlet file (append '1' to the text)

   sed -i 's/$/1/' templates/test.hamlet

3. Run the incremental rebuild (similar to what yesod devel runs)

   stack build --fast --ghc-options "-ddump-to-file -ddump-timings" yesod-perf-test:lib --flag yesod-perf-test:dev --flag yesod-perf-test:library-only

4. Analyze the timing (with https://github.com/codedownio/time-ghc-modules)

   time-ghc-modules

Result

2.8 seconds isn't terrible, but when I'm actually running stack exec -- yesod devel, it's closer to 5 seconds by the time I'm able to get the page reloaded, and this is still quite a bit to wait each time I want to tweak some margin/padding, etc.

Questions

• I'm still using defaultLayout— maybe ripping that out will help?
• Why are Model and Handler.Home being rebuilt when I only changed test.hamlet?
• Where should I open an issue about this? Depending on whether this is an unsolved problem (so it's a feature request) or whether it is possible but I just don't know how yet (so it's about documentation)…
  • The main yesod repo?
  • The shakespeare library?
  • Wherever the template for stack new comes from?
  • yesod-cookbooks?
  • The yesod book (website)?

[smichel17]

Someone pointed out that this issue applies to any off-the-shelf css framework, if you're you're happy with the default style and are mostly applying pre-made styles rather than writing your own; while it's worse with Tailwind's larger number of classes, the real problem I'm trying to fix is the speed of hamlet reloads, and focusing too much on Tailwind distracts from that. I agree, and have edited the title and first post to de-emphasize tailwind a little bit.

[smichel17]

Courtesy of @charukiewicz (the same "someone" from the previous comment), it turns out that ghcid is quite a bit faster than yesod devel.
In one terminal:

ghcid --command="stack ghci --ghci-options='-fobject-code -odir .ghci-build-artifacts -hidir .ghci-build-artifacts -O0 -ilib -fno-break-on-exception -fno-break-on-error -v1 -ferror-spans -j -isrc app/main.hs'" --reload="src/**" --run="Main.main"

And in another:

find ./ -type f \( -iname \*.hamlet \) | entr -s 'touch src/Foundation.hs'

I am not sure exactly how much faster, but it seems like ~3 seconds from when I save the .hamlet file to when the new page is rendered on my screen, as I spam-reload. Based on the timing analysis above, perhaps this is the same speed that it was already compiling at and it's just faster at detecting changes / restarting the dev server. But whatever the case, it feels significantly faster.

[smichel17]

In the process of getting ghcid going, I noticed app/DevelMain.hs which promises faster incremental builds, at the cost of no automatic rebuilds. Tomorrow, I'll look into whether ghcid can work with DevelMain in order to get the best of both worlds.

[smichel17]

Goal

I hope that whatever I end up with here can be incorporated into the yesod scaffolding and/or book, so that other people can get quick reloads out of the box, instead of having to research as much as I did. To that end, I'm trying to (semi-retroactively) document this in as much painstaking detail as I can remember. Hopefully this will help identify the pitfalls that a newcomer runs into when trying to get this working, so we can put that information front-and-center.

Since a few days ago…

The command from two posts up was mostly copy-pasted, from Christian's suggestion, and then added in a bunch of flags that ghcid sets by default (found in the ghcid source code), without really understanding them. However, at this point I was having real trouble stringing these different tools together (stack → ghcid → stack → ghci). The difficulty here is that both stack and ghcid add ghc/ghci arguments, which makes it difficult to find out what you're actually running, in the end.

So, I decided I decided I needed to understand how to use all these tools. Since stack was the tool I was most familiar with up until this point (I had skimmed and read sections of its docs), I started there. I read the stack guide, build command reference, yaml configuration reference, and ghci command documentation pages in their entirety. I also pulled up the ghc flag reference, although I haven't read it in full. There's still a lot of "known unknowns" that I don't understand, but I think there's fewer "unknown unknowns".

One thing was now clear: my goal is to answer, "How quickly can ghci reload?"

I should have documented the evolution of the commands I tried as I did them; since I didn't, this is a bit of a recreation. But I think it's close to what actually happened.

Getting the functionality running

Note: Throughout all of this, I have the find | entr command from above running in the background in a different terminal: find ./ -type f \( -iname \*.hamlet \) | entr -s 'touch src/Foundation.hs'

First: Minimal command that works

Turns out ghcid alone runs without crashing, hooray for good defaults! But it doesn't start the web server. The ghcid repo readme says that the hardest part is getting ghci itself working, so let's try that instead. DevelMain.hs has instructions for that.

stack ghci yesod-perf-test:lib --no-load --work-dir .stack-work-devel

That works, but I have to :l app/DevelMain.hs and :r and DevelMain.update manually. So, on to the next step.

Next: ghcid working with stack

I decided I wanted everything contained locally, so I removed the executable from stack install ghcid earlier and ran ghcid through stack, using the command that worked above:

stack exec -- ghcid --command="stack ghci yesod-perf-test:lib --no-load --work-dir .stack-work-devel"

Predictably, this failed miserably.

No files loaded, GHCi is not working properly.

Let's remove the --no-load tag, and load DevelMain, since I know we'll need that.

stack exec -- ghcid --command="stack ghci --work-dir .stack-work-devel yesod-perf-test:lib app/DevelMain.hs"

Cannot use 'stack ghci' with both file targets and package targets

Huh. Okay. Let's try it with just DevelMain, I guess…

stack exec -- ghcid --command="stack ghci app/DevelMain.hs"

This worked. Well, it launched without crashing; the webserver was still not running. But you may notice, that when I was typing this command, I forgot the --work-dir. Oops! I noticed it and added it back, and…

Side-track: some odd build system failure

stack exec -- ghcid --command="stack ghci --work-dir .stack-work-devel app/DevelMain.hs"

stack : cannot satisfy -package yesod-perf-test-0.0.0

wat. It was working a minute ago without the --work-dir, and I had definitely already run stack build --work-dir .stack-work-devel. In fact, I tried running that again, and still got the same error. At this point, I was stumped, so I turned to the internet. I found commercialhaskell/stack#980. I don't have Haskell Platform installed, but for this person the fix was to do a full reinstall of stack. I didn't want to do that, so, on a hunch, I tried something less extreme:

rm -rf .stack-work .stack-work-devel
stack build --work-dir .stack-work-devel

…and somehow this fixed the issue; the command above worked. I have no answers, but the roadblock was gone so I didn't dig into it further.

--only-main

I omitted it from the commands above, but I think I might have actually swapped out --no-load for --only-main, not removed it entirely. In any case, by this point I added it back in, and it still loaded

stack exec -- ghcid --command="stack ghci --work-dir .stack-work-devel --only-main app/DevelMain.hs"

Getting the DevelMain sever working in ghcid

At this point I was pretty sure I had app/DevelMain.hs loaded inside ghci, so I just needed to run it:

stack exec -- ghcid --command="stack ghci --work-dir .stack-work-devel --only-main app/DevelMain.hs" --run="DevelMain.update"

🎉 Success! 🎉

Finally: Automatic rebuilds

The while the command above ran the server, it did not trigger automatic rebuilds/reloads. Easy enough to add that back from earlier…

stack exec -- ghcid --command="stack ghci --work-dir .stack-work-devel --only-main app/DevelMain.hs" --reload="src/**" --run="DevelMain.update"

…nope. I don't know why this doesn't work. Turns out we need to pass -isrc to ghc, too. Actually, --reload="src/**" doesn't appear to do anything, so let's leave it off.
…sh
stack exec -- ghcid --command="stack ghci --work-dir .stack-work-devel --only-main app/DevelMain.hs --ghc-options='-isrc'" --run="DevelMain.update"

Automatic rebuilds are now working!

Note: By this point I had looked up most of the original options, and was generally much more comfortable with the tools.

## Speed
First, adding back in the flags which are obviously optimizations: `-fobject-code -O0`
```sh
stack exec -- ghcid --command="stack ghci --work-dir .stack-work-devel --only-main app/DevelMain.hs --ghc-options='-fobject-code -O0 -isrc'" --run="DevelMain.update"

This felt a bit snappier, so I wanted to see what the actual time was compared to the original 2.8 seconds.

Issues dumping timings

I tried adding the same flags I used for profiling earlier, -ddump-splices -ddump-timings.

stack exec -- ghcid --command="stack ghci --work-dir .stack-work-devel --only-main app/DevelMain.hs --ghc-options='-fobject-code -O0 -isrc -ddump-splices -ddump-timings'" --run="DevelMain.update"

However, I guess because ghci or ghcid runs in a loop, this constantly dumps new timings, which meant that I couldn't find a way to manually run time-ghc-modules and get just the rebuild timings.
I had the idea to use ghcid's --lint option to automatically run it after the build. (Note: If you're following along, you'll have to get the tool and change your path)

stack exec -- ghcid --command="stack ghci --work-dir .stack-work-devel --only-main app/DevelMain.hs --ghc-options='-fobject-code -O0 -isrc -ddump-splices -ddump-timings'" --run="DevelMain.update" --lint=/data/repo/time-ghc-modules/time-ghc-modules

However, this also ran in the aforementioned loop (each time timings were dumped), and produced at least 20 different timing pages in the course of testing one change-and-rebuild; I wasn't sure which one was correct, so I temporarily gave up on this.

Issues timing manually

I didn't have precise timings, but I could still get a manual estimate. I figured I'd write the changes to my hamlet file, switch over to my browser and spam-refresh until the changes showed (since there's still no automatic refresh in the browser). Unfortunately, this didn't work, because Spamming refresh prevents a rebuild until you stop. I took another look at DevelMain.hs and noticed this comment:

yesod-perf-test/app/DevelMain.hs

Lines 83 to 86 in 6e25e0c

	-- Note that this implies concurrency
	-- between shutdownApp and the next app that is starting.
	-- Normally this should be fine
	(\_ -> putMVar done () >> shutdownApp site)

So, my guess is that ghci is running single threaded, and won't start rebuilding until the previous server is shut down, which it won't do while answering refresh requests. But I could be totally wrong here, too.

Workaround: back to just running app/main.hs— faster?

The original long command didn't seem to have this issue, so I decided to try running app/main.hs again instead of DevelMain:

stack exec -- ghcid --command="stack ghci --work-dir .stack-work-devel --only-main app/main.hs --ghc-options='-fobject-code -O0 -isrc'" --run="Main.main"

Not only did this solve the reloading problem, it actually seemed to be slightly faster than using DevelMain. So, I decided to stick with this and see what I could optimize further.

More selective rebuilds

In the background of all this, we've been running a find | entr command to touch Foundation.hs each time a hamlet file changes. But, we're not really changing the foundation, just the Handler.Test module. Let's try touch-ing that instead:

find ./ -type f \( -iname \*.hamlet \) | entr -s 'touch src/Handler/Test.hs'

This made a perceptible improvement! I'm still estimating manually, but I'd guess we're under 2 seconds, now. Obviously we can't assume I'm only going to be changing one file at a time, so let's make it more generic: if there's an equivalently-named Handler module, touch that; otherwise fall back to Foundation.hs. The code is ugly and only works in bash, but those can both be fixed later.

find ./ -type f \( -iname \*.hamlet \) | entr -s 'A="$(basename $0)"; B="${A^}"; C="src/Handler/${B:0:-6}hs"; if test -f "$C"; then touch "$C"; else touch src/Foundation.hs; fi'

Yesod's cabal flags

The yesod scaffolding comes with two cabal flags that it says are only for use with yesod devel, but we're basically building our own equivalent, so let's turn them on:

stack exec -- ghcid --command="stack ghci --work-dir .stack-work-devel --only-main app/main.hs --flag yesod-perf-test:dev --flag yesod-perf-test:library-only --ghc-options='-fobject-code -O0 -isrc'" --run="Main.main"

🎉 WE'VE HIT GOLD!" 🎉

Compile times are now down to UNDER 1 SECOND! It's faster than I can alt-tab to my browser and hit reload, so it's effectively instantaneous— no slowdown in my development workflow.

Of course, this will go up once we get back to a real site, but it's really promising compared to the technically-2.8 but effectively 4-5 seconds that we started out with.

edit: Got timings working, the actual time is around 66 MILLISECONDS!

Further research

• I came across this repo, from back in 2014: https://github.com/chrisdone/ghci-reload-demo
  • It has a snippet for automatic browser reloads. I don't totally understand how it's triggered, but that would be nice to set up.
  • It also talks about a safe vs unsafe way to update (killing the webserver thread vs updating in place). I think DevelMain does the safe way. I'm not sure what my current ghcid command is doing.
• The yesod wiki page on ghci has a note about DevelMain and threads. Not sure how important it is.

Misc

• I found DevelMain.update in GHCi seems to compile model everytime yesodweb/yesod-scaffold#195, which might be related.
• Also, a reddit thread about yesod devel being slow, including a comment from snoyman which mentions a Google Summer of Code project to rewrite it to be faster. Did that go anywhere?
• The yesod book probably could use a section on tooling or a prominent link to some external document on tooling. This process was definitely not beginner-friendly; I could easily imagine someone giving up before figuring out that these compile times can be fixed with tooling changes, not just code changes.
  • Actually, I don't have to imagine: Snowdrift.coop actually did decide to migrate to a different front-end (in elm) and likely still will, since the work is in-progress. I only looked into this because I wanted to look for ways to make incremental progress instead of one big cut-over to a new system. Specifically, I looked into yesod build times as due-diligence before putting in the effort to migrate to IHP, to make sure there wasn't an easier solution.

Open questions

Nested bullets are edits.

• What's the difference between ghc's -odir and -hidir options and stack's --work-dir option? Are the ghc options still worth setting if I'm setting the work-dir from stack?
  • Answered
• How does stack still know to build & include yesod-perf-test even though it's not specified on the command line?
  • I think I answered my own question
• How to get good data (e.g. via -ddump-splices -ddump-timings) on an incremental rebuilds in ghci ?
  • Answered
• Is there any reason to use DevelMain over the regular app/main.hs?
  Similarly, at some point I tried pointing ghcid at app/devel.hs. This didn't work; hamlet changes are not recompiled. I'm not sure why. I'm not sure if it matters, given how fast running app/main.hs currently is.
  • I need to learn Haskell better before I can answer this
• Are there any important optimizations that I'm missing?
  • At this point, I think build times are low enough that I am not going to worry about shaving off more milliseconds. When I bring these changes to my real-world application, if build times are not improved enough, then I'll return to optimization. For now, I'm content with a 42x speedup.

[smichel17]

On irc, someone clarified that the -odir and -hidir options (which are are really just two parts of the -outputdir option) are not related to stack's work-dir. "Stack saves packages, not individual files; it's more closely related to using ghc-pkg to register a new package."

Using stack -v, the actual command that gets run (substantially edited by myself — I changed all paths to relative and removed a ton of -package-id=blah-0.1.2 arguments) is this:

ghc-8.8.4
    --interactive
    -i
    -odir=.stack-work-devel/odir
    -hidir=.stack-work-devel/odir
    -hide-all-packages
    -i.stack-work-devel/dist/x86_64-linux-tinfo6/Cabal-3.0.1.0/build/yesod-perf-test
    -iapp
    -i.stack-work-devel/dist/x86_64-linux-tinfo6/Cabal-3.0.1.0/build/yesod-perf-test/autogen
    -i.stack-work-devel/dist/x86_64-linux-tinfo6/Cabal-3.0.1.0/build/global-autogen
    -i.stack-work-devel/dist/x86_64-linux-tinfo6/Cabal-3.0.1.0/build/yesod-perf-test/yesod-perf-test-tmp
    -stubdir=.stack-work-devel/dist/x86_64-linux-tinfo6/Cabal-3.0.1.0/build
    -rtsopts
    -with-rtsopts=-N
    -optP-include
    -optP.stack-work-devel/ghci/e6338138/cabal_macros.h
    -ghci-script=/tmp/haskell-stack-ghci/edd82e5b/ghci-script
    -fobject-code
    -O0
    -isrc

So, it sets -odir and -hidir to be the odir folder inside whatever stack-work directory you choose.

[smichel17]

How does stack still know to build & include yesod-perf-test even though it's not specified on the command line?

I suspect it's because app/main.hs is specified as the main for the executable, in package.yaml (and therefore also in yesod-perf-test.cabal).

[smichel17]

How to get good data (e.g. via -ddump-splices -ddump-timings) on an incremental rebuilds in ghci?

Well, the first thing that would be good is to re-read the dang docs. I'm not sure how -ddump-splices snuck in there, when it should have been -ddump-to-file all along.

So, now I have a way to do this:

1. Start ghci (not with ghcid)

   stack ghci --work-dir .stack-work-devel --only-main app/main.hs --ghci-options='-fobject-code -O0 -isrc -ddump-timings -ddump-to-file'

2. Clear any ddump timings which this generates, so the analysis will only check the timings of the incremental rebuild.

   find . -name "*.dump-timings" -exec rm '{}' \;

3. Make some trivial change to the .hamlet file
4. Manually reload ghci to do the incremental rebuild
   • type :reload (or :r) in ghci, then press enter
5. Analyze the timings (and open them to view in browser)

   xdg-open "$(./time-ghc-modules/time-ghc-modules)"

   • Assuming you have time-ghc-modules cloned to a subdirectory and are using X11. If not, get the script however and run it manually.

Result

…apparently I was severely overestimating how long the incremental rebuilds took. Of course, this doesn't include restarting the server, but 66.4ms is pretty wild, considering where we started.

[smichel17]

Interestingly, the QuasiQuoted version actually appears to be a few ms slower.

And using DevelMain is about twice as slow:

Using DevelMain while running touch src/Foundation.hs instead of just the corresponding handler module 😬

[smichel17]

Debugging tip: stack -v will print the actual ghc command that gets run eventually… along with half a megabyte (508KB) of other output you'll have to sort through; it's near the end.

[smichel17]

Is there any reason to use DevelMain over the regular app/main.hs?

I think the main reason is that DevelMain persists some information between reloads, which allows things like persisting a Channel, to trigger automatic browser reloads.

There may be other reasons this is useful— this is the point where I run into a wall with my ability to read/write basic Haskell and my (in)ability to really understand what's going on. I might be able to hack automatic reloads onto the scaffolding… or the mechanism to do that might already be in place. It's hard for me to tell, because I don't have the ability to compare the scaffolding's appHttpManager (Manager?) with the Chan used in the "automatic browser reloads" link above.

So, I'm going to take a break from this rabbit hole to go learn Haskell For Real™, and I'll be back… whenever I get back.