Add SILE Writer

[alerque]

This PR aims to cherry pick bits of my working but very messy fork and make them ship-shape for inclusion upstream in Pandoc, per discussion in #6087. For now this will just be the Writer.

Feedback (and straight up help) welcome!

[KurtPfeifle]

Having only cursorily looked over the new Manual.txt I remember to have seen at least one instance of 'SIEL' misspelling (instead of 'SILE').

[alerque]

Thanks @KurtPfeifle, fixed.

[jmaibaum]

Hi @alerque , I would really like to see SILE support in pandoc. Anything that I could help with here?

[alerque]

@jmaibaum How is your Haskell? Mine is is all MacGyver & Duct Tape, which is why this is (still) not done. Also I have it working well enough to get my production work done, so it's been hard to stay motivated. If there was somebody that could help me out of the dark corners I get stuck in in Haskell I could probably get the bulk of the work done.

[jmaibaum]

Unfortunately "my Haskell" is next to non-existent apart from following a few tutorials and skimming a shorter book about it. I cannot claim having written any Haskell apart from example code, yet.

I managed to build this PR using stack and ran a few simple test documents successfully afterwards. That's basically it.

But I understand that you got pretty far by basically following what is done in similar writers like the LaTeX writer, so I won't consider this case hopeless just yet.

Can you pinpoint any obvious "good first issue" with this PR that I could take as a starting point?

I was thinking about trying to rebase this on current pandoc master as a first exercise (or maybe on the 2.10 head) and to see where I end up, but perhaps there is something else that would be more helpful and sufficiently isolated to start with?

[alerque]

Basically I've been working backwards from the LaTeX example. In retrospect that was a bad place to start. In spite of the superficial syntax similarities, SILE is an order of magnitude simpler and more regular than LaTeX and Pandoc's writer for LaTeX is full of workarounds for its idiosyncrasies. There is still a lot of cruft in my SILE writer that is actually LaTeX related not SILE.

However since at this point the basics are functional, the remaining work is really just a matter of working through every possible AST element and making sure the output is sane and also working through the Haskell code and making sure the implementation is sane.

Why don't you let me rebase on a newer Pandoc. That part is very tricky to do but I've done it so many times I kind of have a system (I started on 1.5 or something and it's based on 2.9 now). Once it's fresh again feel free to file PR's against my fork & branch and I'll get them merged into my fork so they make it into the PR here.

By the way I have two branches types I keep updated: a dirty working branch (named after whatever Pandoc version I rebased on last) that I hack in that has very messy but linear history, and the other a version of the same thing with debugging bits excluded and broken up into a few logical commits. The former is probably what you should branch from and hack on, the latter is what my draft PR #6088 is based on.

[alerque]

@jmaibaum I just rebased my sile-head branch onto the latest Git master. The tests are broken but building just the executable works (there is a temporary commit adding a make sile target to do a quick build skipping tests). I was able to build it and get working SILE support based on the current Pandoc 2.11 development code.

Note this doesn't mean the SILE code is up to speed with changes from 2.9.2 to 2.10.0 — one thing I do periodically is check out diffs in the LaTeX writing between versions and port any relevant coding style or general libary refactoring to the SILE writer code. I haven't done that since 2.9.2 yet.

Feel free to branch off of that work and hack on it. I'd encourage small incremental commits, however messy. For the actual PR I'm reconstructing a new history with clean logical commits by cherry picking code from the final diff anyway so the dev commit history is only relevant to hacking on ideas anyway. The more the merrier. If you do start a branch let me know and I'll keep an eye on it and maybe we can collaborate and get this cleaned up!

[alerque]

The sile-writer-pr branch this PR is based on has also now been rebased and compiles. Note that this doesn't have any of the reader or test components I was working on, so please do hack on top of the sile-head branch if you work on this.

[alerque]

For anyone still following this, my Haskell chops are still not great and getting this through testing phases is going to be hard for me. The work isn't abandoned and I do keep this rebased and working on newer versions of Pandoc (currently 2.11.2). I also tinker with it on HEAD trying to pick away at the problems facing any potential merge. If anybody steps up to help with Haskell stuff I'd be happy to jump in to help make this happen.

[alerque]

On the SILE side of this, I started a project board to organize issues relating to this: whether new commands to handle AST elements we didn't have good support for before, ways of handling different document classes that fits better with document conversion from other formats, or whatever. Some might find it useful to see at a glance what issues directly or indirectly relate to Pandoc support.

[yurivict]

Does this patch have a chance of being finished and merged into Pandoc?

[alerque]

I sure hope so. This branch isn't the latest work either, but I still need someone with a little bit of Haskell chops to help me clean it up for submission. I still maintain the fork on every Pandoc release and use it in production every day.

Here is a quick preview of how -t pdf --pdf-engine sile output looks (top left) compared to other engine defaults:

[alerque]

I just cherry picked from my freshest development branch and squashed the bits into the most relevant logical commits. Besides 1 commit dealing with some GHC version issues for my dev environment this now represents about where things are at.

@tarleb Do you think there is any chance I could talk you into helping out a bit with the Haskell end of things to sanitize things so this can move forward? We're at 9 years now of my maintaining a Pandoc fork so I can use this writer!

[tarleb]

Gladly, but probably not before the Christmas break.