Consider changing static site generator

[certik]

I have great experience with Hugo for site generation --- a single binary download on all platforms, it's very fast to run, and overall seems robust.

The issue with Jekyll is that it's harder to install and you typically run into issue such as this one:

#77 (comment)

This is relatively lower priority, since this website was primarily developed by @milancurcic and @LKedward and so I let them choose the technology. I opened this issue just as an alternative to know about.

[milancurcic]

Indeed Hugo is easier to install and run because it's a single binary. The only Jekyll issue that I see is actually nothing about Jekyll but with gem and bundler, which IMO just plain suck. Especially when you start using specific versions. Tracebacks are inscrutable.

Hugo was actually what I started working with at first and then gave up after few hours. I was somewhat familiar with Jekyll already, and Hugo is overall more complex so there was more to learn.

So setting up the tooling to run Jekyll is worse than Hugo. But after you do that once, I think Jekyll is easier to work with than Hugo. Personal opinion of course.

Build previews that @LKedward made mostly solves this. If you know how to submit a PR, there's no need to run locally. For writing markdown pages and submitting content this is good enough. If you need to make structural changes to the website, chances are that setting up Jekyll is not an issue.

[certik]

That's fine, you two are the main maintainers, so if you are comfortable with Jekyll, that is fine with me. If we start getting more users struggling to install Jekyll, we can reconsider.

[LKedward]

I agree Jekyll isn't great for contributors to setup - I'm also not sure if it supports Windows or not?
I find the Jekyll templating language very rudimentary at times, but this also means that it isn't too complicated to get started. I'm happy to stick with Jekyll for now, but we can reconsider later if need be.

[LKedward]

As discussed in #201, Jekyll isn't well-suited to robust internationalization, most notably it isn't possible to use a gettext-based approach which would offer a safe and automated workflow. Hugo similarly doesn't have support for using gettext.

Based on this, we should re-open discussion of changing static site generator with a mind to support gettext-based translation.

As suggested by @certik, the sympy site uses jinja2 templating with an i18n extension that supports gettext. The generate script demonstrates this nicely.

Some points for discussion/comment:

1. Are there any objections to switching static site generator in order to robustly implement internationalization?
2. Does anyone have an objection to switching to jinja2?
3. Does anyone have any other suggestions for alternative site generators with gettext support?
4. What are the specific requirements for site generators in order to port all of fortran-lang.org?

[awvwgk]

Switching from jekyll to plain HTML + jinja2 seems like a major step back. jinja2 templating is not so much different to liquid templating in jekyll and we would loose the markdown support to gain gettext support.

I'm fine with loosing markdown support only if we are able to replace it with something similarly like reStructuredText or Asciidoc in our static page generation, we need support for code-blocks and I'm not too eager writing those in plain HTML or some ugly template block.

[LKedward]

I'm fine with loosing markdown support only if we are able to replace it with something similarly like reStructuredText or Asciidoc in our static page generation, we need support for code-blocks and I'm not too eager writing those in plain HTML or some ugly template block.

Yes agreed, we definitely want to keep easy support for code-blocks.

[certik]

It looks like our website is a combination of html files and markdown. For html, moving to jinja is a step up.

For Markdown, we need to figure out a way how to translate Markdown documents in general. What are the options / alternatives? How is this done in Jekyll?

[awvwgk]

Jekyll is using a Markdown to HTML converter, you have the choice between several vendors, usually kramdown is used (which we are using as well), but you can plugin other converters as well, e.g. if you want to use Asciidoc with Jekyll.

[milancurcic]

Beside being able to write content in a human-readable markup, it's also important for me that GitHub can render it in the browser (in file views and PRs). I know GitHub renders md and rst but I don't know about adoc. Consider also how many people are familiar with these markups. Markdown is probably the most ubiquitous, and would impose the least friction to contributing writers.

[awvwgk]

GitHub can basically render any known markup language: md, rst, adoc, textile, mediawiki, ... This shouldn't be a limitation. I'm not too attached to a particular markup language, but I have a slight preference for reStructuredText and Asciidoc because I found them easier to use in bigger technical documentations, especially when it comes to code-blocks in admonitions. We currently create those with a liquid capture block in the minibooks which is quite ugly.

reStructuredText would be a natural choice when using sphinx as static side generator which has a gettext based i18n module

[certik]

I strongly prefer Markdown. Let's keep using it.

As a start, we can simply require that the whole Markdown file must be translated, and perhaps the translation can keep a comment somewhere which exact English version was used. That way you can manually check if there was some change in the English version. We can even write a tool for this. So in some sense, it would be like gettext, but for the whole Markdown document at once. I think that's fine with me.

My main objection was for the html documents, and those would be fixed with jinja+gettext.

[awvwgk]

I'll pin this issue for better visibility.

I had a look into SymPy's HTML + jinja2 and the generate script. While we won't be able to use gettext here, I'm quite confident that I can make localization work with jekyll and the plugin used in #201 almost as if we were using gettext. It might become a bit heavy on Ruby for the “generate” script the way I'm envision it, but I haven't yet started on this project yet due to time constraints.

The general idea would be:

1. use the translate / t to translate sentences like in SymPy's jinja2 syntax, this works for HTML just like markdown.

{% t "High-performance parallel programming language" %}

2. use a “generate” script to extract all sentences from the liquid templates and create the _i18n/en.yml automatically

# "index.html" 10
"High-performance parallel programming language": >-
 High-performance parallel programming language

3. translators copy the _i18n/en.yml and translate all sentences, i.e. _i18n/fr.yml would contain

# "index.html" 10
"High-performance parallel programming language": >-
 Un langage de programmation parallèle haute performance

4. the plugin will safely fall back for missing translations to the English version if translations are not yet available.

We will need some tooling to work with yml files and merge translations, but those are hash maps which require some fuzzy string comparison at best, nothing too difficult in the Ruby world.

[certik]

Of course, the generate script and the other tooling will be written in Fortran! :)

Is the fr.yml format standardized somehow? It sucks to write our own tool for this, but I remember the tools are just not super great anyway. I would only suggest to use more common formats, as I think there are tools that can read those files and help translators by providing GUIs.

[awvwgk]

It's YAML, not sure if I would call it “standardized” in any way ;).

Of course, I would drop the whole effort of creating our own tools without a second thought if there were actually tools available for our problem.

[LKedward]

Your proposed solution sounds good @awvwgk considering the various limitations we are dealing with. I have no experience with Ruby so I can't comment on how complex/time-consuming it might be to write and maintain the "generate" script, but if it's simple and robust, and it makes our life easier then it seems like a good approach.

[awvwgk]

I have been experimenting a bit with sphinx and myst recently while working on my teaching resources at https://qc2-teaching.readthedocs.io. While hacking an existing sphinx theme with CSS overwrites is incredible tedious, we could just create our own fortran-lang theme for this purpose (this would be the HTML + jinja2 part discussed above). Also, the myst parser does allow us to use markdown almost on par with reStructuredText which is really a step up from the kramdown parser. As a plus point we could use reStructuredText as well to write minibooks.

It is still a huge effort, but we would be using Python rather than Ruby, which most of you are probably more familiar with. Another plus point is that fortran90.org is using sphinx as well, so we could easily migrate the whole fortran90.org content as reStructuredText into fortran-lang.org at some point.

[aslozada]

sphinx
An extension for internationalization (i18n) and localization (l10n) using i18n API is presented here

[awvwgk]

Exactly, with sphinx we would get first class translation support via gettext.

[awvwgk]

I started working on sphinx based version of the webpage at my fork https://github.com/awvwgk/fortran-lang.org in the sphinx branch. It is still quite experimental at the moment, but so far sphinx looks really promising.

[awvwgk]

Coming back to this issue after a while, it seems like there is some development in the sphinx themes I checked previously. Styling with CSS is now quite straight-forward, also the sphinx-panels has been superseded by the sphinx-design project, which allows usage of almost all bootstrap we need. And we can translate it (there are a few rough edges).

[awvwgk]

Closing this in favor of #415 and https://github.com/fortran-lang/webpage