Merge pull request #70 from uktrade/docs/move-to-eleventy

docs: move to eleventy

.github/workflows/deploy-docs-to-github-pages.yml

@@ -6,17 +6,21 @@ on:
       - main
 
 jobs:
+
   build:
     runs-on: ubuntu-latest
     steps:
-      - uses: actions/checkout@v3
-      - uses: actions/setup-python@v4
-        with:
-          python-version: 3.11
-      - run: pip install -r requirements-docs.txt
-      - run: mkdocs build --site-dir _site
-      - name: Upload Pages artifact
-        uses: actions/upload-pages-artifact@v2
+    - name: Checkout
+      uses: actions/checkout@v3
+    - name: Setup Pages
+      uses: actions/configure-pages@v3
+    - name: Install dependencies
+      run: npm ci
+    - name: Build with Eleventy
+      run: npx eleventy
+    - name: Upload artifact
+      uses: actions/upload-pages-artifact@v2
+
   deploy:
     runs-on: ubuntu-latest
     needs: build

docs/contributing.md

@@ -1 +1,142 @@
---8<-- "CONTRIBUTING.md"
+---
+layout: sub-navigation
+order: 4
+title: How to contribute
+---
+
+
+In most cases to contribute you will need a [GitHub account](https://github.com/join).
+
+
+## Issues
+
+Suspected issues with stream-unzip can be submitted at [the stream-unzip Issues page](https://github.com/uktrade/stream-unzip/issues).
+
+An issue that contains a [minimal, reproducible example](https://stackoverflow.com/help/minimal-reproducible-example) stands the best chance of being resolved. However, it is understood that this is not possible in all circumstances.
+
+
+## Feature requests
+
+A feature request can be submitted using the [Ideas category in the stream-unzip discussions](https://github.com/uktrade/stream-unzip/discussions/categories/ideas).
+
+
+## Getting the source code
+
+To contribute changes to documentation or code, you will need the source of stream-unzip locally. The instructions for this depend on if you are a member of the [uktrade GitHub organisation](https://github.com/uktrade). In both cases, experience of working with source code, working on the command line, and working with git is helpful.
+
+
+### If you're a member of uktrade
+
+1. [Setup an SSH key and associate it with your GitHub account](https://docs.github.com/en/authentication/connecting-to-github-with-ssh/adding-a-new-ssh-key-to-your-github-account)
+
+2. Clone the repository
+
+    ```bash
+    git clone git@github.com:uktrade/stream-unzip.git
+    cd stream-zup
+    ```
+
+You should not fork the repository if you're a member of uktrade.
+
+### If you're not a member of uktrade
+
+1. [Setup an SSH key and associate it with your GitHub account](https://docs.github.com/en/authentication/connecting-to-github-with-ssh/adding-a-new-ssh-key-to-your-github-account).
+
+2. [Fork the repository](https://github.com/uktrade/stream-unzip/fork). Make a note of the "Owner" that you fork to. This is usually your username.
+
+    There is more documentation on forking in [GitHub's guide on contributing to projects](https://docs.github.com/en/get-started/quickstart/contributing-to-projects).
+
+3. Clone the forked repository. In the following, replace `my-username` with the owner that you forked to in step 2.
+
+    ```bash
+    git clone git@github.com:my-username/stream-unzip.git
+    cd stream-zup
+    ```
+
+## Documentation
+
+The source of the documentation is in the [docs/](https://github.com/uktrade/stream-unzip/tree/main/docs) directory of the source code, and is written using [Material for mkdocs](https://squidfunk.github.io/mkdocs-material/).
+
+Changes are then submitted via a Pull Request (PR). To do this:
+
+1. Decide on a short hyphen-separated descriptive name for your change, prefixed with `docs/` for example `docs/add-example`.
+
+2. Make a branch using this descriptive name.
+
+    ```bash
+    git checkout -b docs/add-example
+    cd stream-unzip
+    ```
+
+3. Make your changes in a text editor.
+
+4. Preview your changes locally.
+
+    ```bash
+    pip install -r requirements-docs.txt  # Only needed once
+    mkdocs serve
+    ```
+
+5. Commit your change and push to your fork. Ideally the commit message will follow the [Conventional Commit specification](https://www.conventionalcommits.org/).
+
+    ```bash
+    git add docs/getting-started.md  # Repeat for each file changed
+    git commit -m "docs: add an example"
+    gir push origin docs/add-example
+    ```
+
+6. Raise a PR at [https://github.com/uktrade/stream-unzip/pulls](https://github.com/uktrade/stream-unzip/pulls) against the `main` branch in stream-unzip.
+
+7. Wait for the PR to be approved and merged, and respond to any questions or suggested changes.
+
+When the PR is merged, the documentation is deployed automatically to [https://stream-unzip.docs.trade.gov.uk/](https://stream-unzip.docs.trade.gov.uk/).
+
+
+## Code
+
+To contribute most code changes:
+
+- Knowledge of Python is required. Python iterables, and specifically generators, are used heavily in stream-unzip.
+
+- Understanding the low-level properties of the ZIP file format is required. These are covered in detail in the specification of the ZIP file format, known as [APPNOTE](https://support.pkware.com/home/pkzip/developer-tools/appnote).
+
+APPNOTE can be difficult to read, and contains a lot of information that isn't needed for stream-unzip. A more concise introduction is in the [Wikipedia page on the ZIP file format](https://en.wikipedia.org/wiki/ZIP_(file_format)). However the Wikipedia page is less authoritative.
+
+In both APPNOTE and the Wikipedia page, the most relevant parts are about the "local file header" and the "data descriptor". These are sections of metadata that go before and after the contents of each file respectively.
+
+---
+
+Changes are then submitted via a Pull Request (PR). To do this:
+
+1. Decide on a short hyphen-separated descriptive name for your change, prefixed with the type of change. For example `fix/the-bug-description`.
+
+2. Make a branch using this descriptive name.
+
+    ```bash
+    git checkout -b fix-a-bug-description
+    ```
+
+3. Make sure you can run existing tests locally
+
+    ```bash
+    pip install -e ".[dev]"
+    pytest
+    ```
+
+4. Make your changes in a text editor. In the cases of changing behaviour, this would usually include changing or adding at least one test in [test_stream_zip.py](https://github.com/uktrade/stream-unzip/blob/main/test_stream_zip.py), and running them.
+
+    ```bash
+    pytest
+    ```
+
+5. Commit your changes and push to your fork. Ideally the commit message will follow the [Conventional Commit specification](https://www.conventionalcommits.org/).
+
+    ```bash
+    git add stream_zip.py  # Repeat for each file changed
+    git commit -m "feat: the bug description"
+    gir push origin fix/the-bug-description
+    ```
+
+6. Raise a PR at [https://github.com/uktrade/stream-unzip/pulls](https://github.com/uktrade/stream-unzip/pulls) against the `main` branch in stream-unzip.
+
+7. Wait for the PR to be approved and merged, and respond to any questions or suggested changes.

docs/exceptions.md

@@ -1,4 +1,9 @@
-# Exceptions
+---
+layout: sub-navigation
+order: 3
+title: Exceptions
+---
+
 
 Exceptions raised by the source iterable are passed through `stream_unzip` unchanged. Other exceptions are in the `stream_unzip` module, and derive from its `UnzipError`.
 

docs/features.md

@@ -0,0 +1,22 @@
+---
+layout: sub-navigation
+order: 2
+title: Features
+---
+
+
+In addition to being memory efficient, stream-unzip supports:
+
+- Deflate-compressed ZIPs. The is the historical standard for ZIP files.
+
+- Deflate64-compressed ZIPs. These are created by certain versions of Windows Explorer in some circumstances. Python's zipfile module cannot open Deflate64-compressed ZIPs.
+
+- Zip64 ZIP files. These are ZIP files that allow sizes far beyond the approximate 4GiB limit of the original ZIP format.
+
+- WinZip-style AES-encrypted ZIPs. Python's zipfile module cannot open AES-encrypted ZIPs.
+
+- Legacy-encrypted ZIP files. This is also known as ZipCrypto/Zip 2.0.
+
+- ZIP files created by Java's ZipOutputStream that are larger than 4GiB. At the time of writing libarchive-based stream readers cannot read these without error.
+
+- BZip2-compressed ZIPs.

docs/getting-started.md → docs/get-started.md

@@ -1,3 +1,10 @@
+---
+layout: sub-navigation
+order: 1
+title: Get started
+---
+
+
 ## Prerequisites
 
 Python 3.6.7+
@@ -7,7 +14,7 @@ Python 3.6.7+
 
 You can install stream-unzip and its dependencies from [PyPI](https://pypi.org/project/stream-unzip/) using pip.
 
-```bash
+```shell
 pip install stream-unzip
 ```
 

docs/index.md

@@ -1,9 +1,33 @@
---8<-- "README.md:intro"
-
-To create ZIP files on the fly try [stream-zip](https://stream-zip.docs.trade.gov.uk/).
+---
+homepage: true
+layout: product
+title: Decompress ZIP files while downloading them
+description: Use this Python package to unZIP all the files in a ZIP archive, without needing the entire archive in memory or disk at once.
+startButton:
+  href: "get-started"
+  text: Get started
+---
 
---8<-- "README.md:features"
+<div class="govuk-grid-row">
+  <section class="govuk-grid-column-one-third-from-desktop govuk-!-margin-bottom-7">
+    <h2 class="govuk-heading-m govuk-!-font-size-27">Memory</h2>
+    <p class="govuk-body">Only a small amount of the compressed or decompressed data is in memory at any one time.</p>
+  </section>
+  <section class="govuk-grid-column-one-third-from-desktop govuk-!-margin-bottom-7">
+    <h2 class="govuk-heading-m govuk-!-font-size-27">Encryption</h2>
+    <p class="govuk-body">Supports WinZIP-style AES-encryption, as well legacy ZipCrypto/Zip 2.0.</p>
+  </section>
+  <section class="govuk-grid-column-one-third-from-desktop govuk-!-margin-bottom-7">
+    <h2 class="govuk-heading-m govuk-!-font-size-27">Large files</h2>
+    <p class="govuk-body">Decompresses Zip64 and Deflate64 ZIPs that are used for larger amounts of data.</p>
+  </section>
+</div>
 
----
+<hr class="govuk-section-break govuk-section-break--visible govuk-section-break--xl govuk-!-margin-top-0">
 
-Visit [Getting started](getting-started.md) to get started.
+<div class="govuk-grid-row">
+  <section class="govuk-grid-column-two-thirds">
+    <h2 class="govuk-heading-m govuk-!-font-size-27">Contributions</h2>
+    <p class="govuk-body">The code for stream-unzip is public and contributions are welcome though the <a class="govuk-link govuk-!-font-weight-bold" href="https://github.com/uktrade/stream-unzip">stream-unzip repository on GitHub</a>.</p>
+  </section>
+</div>

docs/publish-a-release.md

@@ -1,4 +1,9 @@
-# How to publish a release
+---
+layout: sub-navigation
+order: 5
+title: How to publish a release
+---
+
 
 Only mambers of the uktrade GitHub organisation may publish a release. If you are a member of the uktrade GitHub organsation:
 

eleventy.config.js

@@ -0,0 +1,31 @@
+const govukEleventyPlugin = require('@x-govuk/govuk-eleventy-plugin')
+const fs = require('fs')
+
+module.exports = function(eleventyConfig) {
+  // Register the plugin
+  eleventyConfig.addPlugin(govukEleventyPlugin, {
+    fontFamily: 'system-ui, sans-serif',
+    icons: {
+      shortcut: '/assets/dit-favicon.png'
+    },
+    header: {
+      organisationName: 'DBT',
+      organisationLogo: fs.readFileSync('./docs/assets/dit-logo.svg', {encoding: 'utf8'}),
+      productName: 'stream-unzip',
+    }
+  })
+
+  eleventyConfig.addPassthroughCopy('./docs/assets')
+  eleventyConfig.addPassthroughCopy('./docs/CNAME')
+
+  return {
+    dataTemplateEngine: 'njk',
+    htmlTemplateEngine: 'njk',
+    markdownTemplateEngine: 'njk',
+    dir: {
+      // Use layouts from the plugin
+      input: 'docs',
+      layouts: '../node_modules/@x-govuk/govuk-eleventy-plugin/layouts'
+    }
+  }
+};