Add Sphinx documentation (incl. built html files) and deploy workflow

[MarcoHuebner]

Rohversion einer Sphinx Doku, dementsprechend ebenfalls pyproject.toml dev-dependencies angepasst

tbd:

• (Done) Design (aktuell: Sphinx Standard)
• (Done) Testen des Bauens der Dokumentation in CI/CD (und vllt. lokal)
• "Hosting" z.B. auf readthedocs -> CorrelAid Account?
  • Automatisches Bauen der Dokumentation bei Code Updates/ PRs?
• Copyright (CorrelAid...?), Autoren und weitere Metadaten?

resolves #10

[pmayd]

Das sind ne Menge roter Tests XD Schön, dass hier noch weiter gearbeitet wird, mir war nicht bewusst, das jemand daran arbeitet aber ich schaue es mir gerne an!

[MarcoHuebner]

Oje - werde mir in den nächsten zwei Wochen mal anschauen, woher die grundlegenden Fehler kommen. Die Rohversion sollte eigentlich keine Tests - außer vllt. code quality - beeinflussen, ich wollte mit dem PR erstmal mögliche feature-Wünsche abklappern :)

[pmayd]

Relativ einfach die angegebenen Versionen von Python existieren wohl nicht mehr, warum auch immer, sollte also schnell behoben sein

[MarcoHuebner]

Current issues seem to be related to changes/ issues in urllib3 together with cachecontrol. Working on a fix.

[pmayd]

nice!

[pmayd]

not sure about this one but I think Google stylesheet says to intend the line if the string belongs to the previous line so Defaults to True is part of the parameter rename_classifying_variables. But not important and if it works with Sphinx fine

[MarcoHuebner]

Another alternative would be to have it (Defaults...) all in one line. At least locally this solution complies with black & pylint. The problem with sphinx is, that the indentations (without clear, return-separated blocks) are "unexpected" (warning, no error).

[pmayd]

LGTM if it works ;)

[pmayd]

Rohversion einer Sphinx Doku, dementsprechend ebenfalls pyproject.toml dev-dependencies angepasst

tbd:

* (Done) Design (aktuell: Sphinx Standard)

* (Done) Testen des Bauens der Dokumentation in CI/CD (und vllt. lokal)

* "Hosting" z.B. auf [readthedocs](https://docs.readthedocs.io/en/stable/tutorial/index.html) -> CorrelAid Account?
  
  * Automatisches Bauen der Dokumentation bei Code Updates/ [PRs](https://docs.readthedocs.io/en/stable/tutorial/index.html#trigger-a-build-from-a-pull-request)?

* Copyright (CorrelAid...?), Autoren und weitere Metadaten?

yea we could definitely publish to readthedocs, but let's first have the local version working and deployed

[MarcoHuebner]

Should I add more info on sphinx in the ReadMe? Testing whether the build works is already integrated into the CI/CD (afaik/ afai can test), and local deployment - as far as I understand - would be cd docs && make html. That could be clarified/ stated in the ReadMe since I did not do so already.

Opinion/ vision for aforementioned local deployment?

[pmayd]

Yes I think a few sentences or a reference would be helpful to understand what is done here and how to write docu in sphinx.

Regarding local deployment I guess the make command is enough so just mention how to create the docu locally. Should also add a custom pre-commit hook.

And add long as we don't deploy it maybe create an artifact in the workflow, does GitHub support ? In precious projects on work I created docu with tools in cicd and made the final docu available as artifacts so that a push automatically build the latest docu and make it available as artifacts

[MarcoHuebner]

• Updated ReadMe with a "full documentation" subsection
• Added custom pre-commit hook (but thus had to change the docs package version to a hard-coded one to avoid importing the pystatis package. ReadTheDocs also seems to do it that way... therefore added another version test...
• Uploaded docs as artifact - are the standard 90 days storage time ok?

[MarcoHuebner]

Detailed walkthrough: https://pages.github.com/

TODOs:

• Create gh-pages branch
• Add index.html (copy from build/html) and maybe .nojekyll to publish branch
• Add deploy SSH Key -> requires owner permissions
• set up GitHub Pages in the settings
• After that, the documentation should automatically be hosted at http://correlaid.github.io/pystatis

Note: code-quality should be fixed after #9 / feature/T9-update-dependencies branch is addressed

[pmayd]

How is this PR related to #4 ?

[MarcoHuebner]

How is this PR related to #4 ?

Did I write that in a commit message here? If so sorry, I meant that the T9-branch and #9 have addressed #4 IMO

[pmayd]

I updated the dev branch so you can rebase/merge again.

Is this PR ready for merge?

[MarcoHuebner]

Is this PR ready for merge?

No, since it already has a deploy workflow - which will likely fail without the existence of a gh-pages branch to push the built documentation to. If we want to merge it now and reopen another branch later, we would have to remove at least the deploy-docs.yaml

[MarcoHuebner]

Managed to remove manual updating of the .rst files to only having to update the headers and references to the rendered section of the README.md (and package metadata).

One minor flaw remains: Images are not rendered correctly (yet).

[MarcoHuebner]

The documentation can already be found under Actions / Workflow Runs / / Artifacts -> docs. This makes the documentation directory downloadable as a .zip which can be viewed locally after unpacking.

Next steps:

1. Adjust the artifact storage time if necessary
2. Set up GitHub pages. Steps:

• Create a gh-pages branch (or any other name)
• Add an index.html (copy from build/html) and maybe .nojekyll or _config.yml (see here) to the documentation publish (e.g. gh-pages) branch
• Decide whether to use GitHub Token or deploy SSH Key ->
• After that, the documentation should automatically be hosted at http://correlaid.github.io/pystatis

[pmayd]

I have done the necessary setup steps for the deploy SSH key, you can use it now as it is a repository secret:

- name: Deploy
  uses: peaceiris/actions-gh-pages@v3
  with:
    deploy_key: ${{ secrets.ACTIONS_DEPLOY_KEY }}
    publish_dir: ./public

[MarcoHuebner]

Alright, I updated the deploy key now. Things to consider - maybe in the next meeting on Tuesday:

• When should this action be triggered? Only on push/ merge to main? Already on the pull request? What about dev?
• Should we use force_orphan: true to only keep the newest documents
• Maybe we'll need enable_jekyll: false if we experience problems with file/ folder names starting with a underscore _

[pmayd]

Alright, I updated the deploy key now. Things to consider - maybe in the next meeting on Tuesday:

• When should this action be triggered? Only on push/ merge to main? Already on the pull request? What about dev?
• Should we use force_orphan: true to only keep the newest documents
• Maybe we'll need enable_jekyll: false if we experience problems with file/ folder names starting with a underscore _

Thanks! Have to test it out but it sounds perfect.

No, main is to limited, at least for dev and main you have to check the docu before deploying it to main. However, dev should be enough as a PR against dev already triggers the workflow so I always use PR against dev and main as trigger and I think that is perfect for this, too.

Yes, we should only keep the latest docu, we are nowhere near a first public version and there will be breaking changes all the time ...well I exaggerate but still I don't think we need to keep every version. But we should invest time in a proper release workflow next with proper Git tags etc

[MarcoHuebner]

The first deployment attempt failed with Error: Action failed with "not found deploy key or tokens". If I understand correctly, the SSH Key is personal, so I (and everyone else) would have to set up their own key.
A GITHUB_TOKEN however, should allow to automatically authenticate the runner/ workflow run: "A GitHub Actions runner automatically creates a GITHUB_TOKEN secret to use in your workflow. You can use the GITHUB_TOKEN to authenticate in a workflow run."

[pmayd]

That should not be the case the deploy key is not created under user settings but repo settings and it is a repository secret so no

[MarcoHuebner]

True... any obvious errors I'm making in calling the key? Typos in with: deploy_key: ${{ secrets.ACTIONS_DEPLOY_KEY }}? 🤔

[pmayd]

That should not be the case the deploy key is not created under user settings but repo settings and it is a repository secret so no .

From the docu: If the user who created the deploy key is removed from the repository, the deploy key will still be active as it isn't tied to the specific user, but rather to the repository.

[pmayd]

I don't think so, according to the doc it should be fairly simple: create the ssh key as public and the secret as private key and that's it...I will have a look later

[pmayd]

I found the problem which is logical if you think about it: you are on your own fork and that does not trigger the REPO secrets, you have to call from a branch in this repo.

Here the docu:
"Anyone with collaborator access to this repository can use these secrets and variables for actions. They are not passed to workflows that are triggered by a pull request from a fork."

[MarcoHuebner]

Moved this to #27