Improve documentation hosting

[starkos]

What problem will this solve?

The GitHub wiki, where the documentation is currently stored, has been subject to spam and is currently locked down.

Ideally, the documentation would just be a collection of Markdown files in a docs/ folder, something like this; that would allow us to update the docs right alongside the code and submit it all together in a single pull request. Downside is that we lose GitHub's wiki page search feature.

What might be a solution?

Set up an open source documentation website package like Docusaurus or MkDocs to format and serve the documentation.

What other alternatives have you already considered?

That's pretty much it. Don't think re-opening the wiki is a viable option.

Anything else we should know?

Bonus points if we can serve documentation for different versions of Premake from the same site, i.e 5.x vs. 6.x.

Update: We're bring a new Docusaurus-based site online now. Left to do:

• Bootstrap new site (@KyrietS)
• Port reference pages (@KyrietS)
• Port user guide pages (@starkos)
• Fill in home and download pages (@starkos)
• Set up publishing action (@KyrietS)
• Set up search (@KyrietS)
• Redirect all wiki content to new locations

[nickclark2016]

@starkos if you decide to keep with GitHub wiki, I'd be willing to help maintain it as new functionality is added.

[samsinsane]

Surely there's a way to deploy to the GitHub wiki in the GitHub actions? Also, maybe deploying the docs to premake.github.io would be better. Assuming the pages are indexed by Google, you can just search <term> site:premake.github.io and get all of the pages that reference that term, instead of just the page that contains that term in the title.

[starkos]

@samsinsane Hmm…an action that pushes all changes from a folder in the main repo into the wiki would solve the biggest issues, that's true. And it avoids adding another dependency to the project, that's a bonus. The search/navigation isn't nearly as nice as a dedicated package—for sake of comparison: Docusaurus generated docs for Redux and Babel. And I'm not sure how we'd juggle v5 vs. v6 docs if there are significant changes (which is looking pretty likely)? Here's how Jest does it (also Docusaurus).

@nickclark2016 I'm going to put a shout out for more reviewers/approvers shortly. Would be very happy to have the help.

Update: Looks like CopyCat action might do it?

[redorav]

I actually like the search-as-you-type feature of the current wiki. The example docs you've put forward sure look nice though...

[KyrietS]

GitHub Wiki is just a bunch of files with simple search by filename. There is no structure, no versioning, etc. It was always a pain for me to use it. It's just as well that it was indexed in Google. Jest is a good example of how Premake's docs could be done and maintained.

Docusaurus supports docs versioning on one branch. As far as I know MkDocs needs every version on separate branch so it introduces additional complexity and a lot of redundancy.

[KyrietS]

I've created a simple draft with Premake documentation using Docusaurus v2. It's quite easy because most wiki markdown pages work out of the box (some urls need to be fixed).

I could contribute into it more in my spare time but I need some kind of approval (from @starkos ?) that this is the direction that wiki of Premake will go, so my time won't be completely wasted if new wiki isn't going to be used anyway.

Pros of new wiki

• Easy to maintain versioning
• Nice looking docs
• Markdown
• Can be automatically built and published using Github Actions after every commit
• Search box search-as-you-type style is supported (link)
• People can actually contribute into wiki (currently it's not possible 😞)

Things to consider before wiki development is started:

Maybe we should not create separate pages for items like: os.mkdir, os.remove, os.rmdir, os.stat, etc. I think that it should be gathered in one descriptive page (see example here). So the question is: how functions should be grouped?

Note: you will still be able to link directly to particular function after it's grouped in one file, eg. premake.github.io/docs/os#mkdir

Contribution

It would be great if I could publish this draft wiki on a branch in this repo so I'm not the only one that can help with moving whole wiki of Premake4 and Premake5 🙃

[starkos]

Awesome…this was starting to near the top of my list, so great timing.

It would be great if I could publish this draft wiki on a branch in this repo so I'm not the only one that can help with moving whole wiki of Premake4 and Premake5

Sure would! You'll need to submit a pull request with a few pages in place for us to check it out. Feel free to ask questions about the process here.

I'll be happy to help with migrating content once it hits the top of my list.

[starkos]

Maybe we should not create separate pages for items like: os.mkdir, os.remove, os.rmdir, os.stat

Forgot to answer this bit…IMHO I'd prefer to keep the separate pages. Again, happy to help with porting those over.

[KyrietS]

Thanks!

You'll need to submit a pull request with a few pages in place for us to check it out.

In the following days I'll prepare functional and configured draft with some examples so it'll be clear how to add new pages, sections, versions etc.

Forgot to answer this bit…IMHO I'd prefer to keep the separate pages. Again, happy to help with porting those over.

Sure, maybe it's better idea since many functions in premake are not easily grouped into descriptive categories. Just having 300+ items in the sidebar on the left of docs page seemed to me uncommon.

[KyrietS]

@starkos I published simple draft on my Github Pages: https://kyriets.github.io/premake-core/.

Please check it out. What do you think?

Here is the source code of the website alone: https://github.com/KyrietS/premake-core/tree/docusaurus-wiki/website

I also added Github Workflow for building and publishing the website after every commit/PR to the branch with docs. See: https://github.com/KyrietS/premake-core/blob/docusaurus-wiki/.github/workflows/docs.yml

Whole website with CI is currently configured for my repo. I'll change it before PR of course.

Now we have to make a decision if we want to keep website source files in this repo (premake-core) or somewhere else (https://github.com/premake/premake.github.io ?)

• Docs can be stored close to the source code (in premake-core on master). Some people would say that it's unnecessary weight for the repo.
• Docs can be stored in premake-core but on a separate branch. Now cloners wouldn't have to download the whole website with premake source code (in theory 😉). Some people would say that having that one branch that is never going to be merged with master is distractive.
• Docs can be stored in a separate repo, eg. premake.github.io. It'll be separated from the code and easier to maintain. Some people would say that maintaining docs that far from the source code is not a good idea, even though there are big projects that do exactly that (eg. here)

Where should I make Pull Request to? Which repo? Please consider points above 🙃
IMHO it should be separate repo: premake.github.io

Oh, and adding searchbox 🔍 to the docs needs them to be published. It's essential feature and I know that. Don't worry 😸

[starkos]

Sorry, swamped, hoping to get to this soon. PR should go to the premake-core repository.

[starkos]

#1587 is now merged! 🎉

I have some client work that needs to be checked down this morning. Later this afternoon & evening I'm planning to dive into migrating the narrative content from the wiki—getting started, tutorials, guides, that kind of thing. If someone beats me to it, let me know and I'll tackle reference pages instead. (If we get more than 1–2 people interested in helping I'll open a Discord for it).

[KyrietS]

As it is a main thread for docs migration I'll paste here my Editor notes as a reminder what can/should be done:

Migration / editor notes

• All supported Markdown features can be found in: /docs/Style-Guide.md
• Lua syntax highlight is supported (```lua).
• Shell code blocks can (should?) be marked with ```bash (Especially when #comments are used).
• Every occurrence of **inlineCode()** should be changed to `inlineCode()`.
• When linking to another document ALWAYS use relative path with *.md extension. Docusaurus will generate valid urls, and links will also work when browsing on GitHub. Additionally, you will get a warning message when link to a markdown file can't be resolved. (there are many warnings like this at this moment).
• Avoid using markdown headers if it's possible. id: defaults to filename and title: defaults to id if not provided. So instead of creating markdown header in file os.mkdir.md like this:

---
id: os.mkdir
title: os.mkdir
---

it's better not to provide markdown header at all.

• Files' ids are used in sidebars.js. Every file must be added to sidebars.js manually.
• Sidebar items are not automatically sorted, but we'll try to keep them in order.
• All assets (like images) must be placed in website/static folder so they can be referenced with global path, eg. we have an image in /website/assets/img/image.png and if you want to include this image in docs page you should write: ![alt text](/img/image.png). There is no /assets/ before image path! It'll be added by Docusaurus automatically.
• You can highlight lines in a code block, eg. ```lua {3}.

---

Milestones:

1. Getting started, tutorials, guides migrated
2. Reference pages migrated
3. Download page created
4. Github Action for publishing the website automatically
5. Adding search to the website

Most reference pages only need links correction (missing *.md extension).

I would be happy to help. If anyone has questions related to the website please don't hesitate to ping me anytime.

[starkos]

It occurred to me after the fact that in my enthusiasm I stepped on your toes. So let me turn this around and ask: how can I help? I was thinking the narrative guides could be moved in parallel with the API reference pages?

[KyrietS]

It occurred to me after the fact that in my enthusiasm I stepped on your toes.

You didn't. Please keep working on your part and coordinate the whole process.

I want to be sure that nobody's work is done twice, so I won't do anything without being explicitly assigned 🙃

I was thinking the narrative guides could be moved in parallel with the API reference pages?

Of course it could.

[starkos]

Cool cool. If you're up for it, maybe stake a claim on a section of the API references?

[KyrietS]

Okay, so I take all API reference pages.

[starkos]

FYI, I'd like to have the ability to create index pages like this one, as they are handy for answering support requests and communication in general. Thinking to create more modules like sidebar.js that we can include into different pages, and into the sidebar, so we only have to list out each section once. Open to other ideas.

[KyrietS]

Seems useful.
Index pages like this can be automatically generated from sidebar.js files, but I'm not sure what you mean by including different sidebars to different pages. It would be very confusing. If we split sidebars.js into sections (separate files) then merging them into one list would be tricky because of alphabetical order. It's still possible though, but we'd need to extend Docusaurus build process a little bit.

If I understand you correctly. Let's say we have "sections" like:

sidebars/lua-library-additions.js
sidebars/api.js
sidebars/others.js

And you'd like to have index page for every section and also file sidebars.js would be dynamically generated from sidebars/*.js?

[starkos]

Ah, good point…I can just include sidebar.js as-is and then index the category I want to list. Okay, problem solved, carry on… 😄

[starkos]

Just submitted #1597 to port the user guide pages. @KyrietS Let me know if you spot anything obvious, otherwise I'll merge it later today and then start on the index and download pages.

[starkos]

#1600 is merged and the site is online! 🎉 Thanks for taking the initiative on this one @KyrietS , that was a big help!

I'm going to pull together the community-related pages (contributing, getting help, etc.) into their own section, copying from the Docusaurus website, clean up a few things. Once that's done I'll have a go at redirecting the wiki pages to their new locations.

@KyrietS Can you flip whatever bits are needed to enable search?

[KyrietS]

Great news! 🎉

@KyrietS Can you flip whatever bits are needed to enable search?

I've already started my research towards search functionality. Just give me couple of days and I'll notify you what should be done to set it up. I need to do some local testing.

[starkos]

Search is online and wiki pages have been redirected. Done and dusted.