build: use podman for image creation

.github/workflows/publish.yml

@@ -34,19 +34,11 @@ jobs:
             --username ${{ secrets.PYPI_USERNAME }} \
             --password ${{ secrets.PYPI_PASSWORD }}
 
-      - name: Docker meta server
-        id: meta_server
+      - name: Docker meta
+        id: meta
         uses: docker/metadata-action@v3
         with:
-          images: aparcar/asu-server
-          tags: |
-            type=semver,pattern={{version}}
-
-      - name: Docker meta worker
-        id: meta_worker
-        uses: docker/metadata-action@v3
-        with:
-          images: aparcar/asu-worker
+          images: aparcar/asu
           tags: |
             type=semver,pattern={{version}}
 
@@ -56,18 +48,10 @@ jobs:
           username: ${{ secrets.DOCKER_USERNAME }}
           password: ${{ secrets.DOCKER_PASSWORD }}
 
-      - name: Build and push ASU server to Docker Hub
-        uses: docker/build-push-action@v2
-        with:
-          file: Dockerfile.server
-          push: true
-          tags: ${{ steps.meta_server.outputs.tags }}
-          labels: ${{ steps.meta_server.outputs.labels }}
-
-      - name: Build and push ASU worker to Docker Hub
+      - name: Build and push ASU to Docker Hub
         uses: docker/build-push-action@v2
         with:
-          file: Dockerfile.worker
+          file: Containerfile
           push: true
-          tags: ${{ steps.meta_worker.outputs.tags }}
-          labels: ${{ steps.meta_worker.outputs.labels }}
+          tags: ${{ steps.meta.outputs.tags }}
+          labels: ${{ steps.meta.outputs.labels }}

.github/workflows/test.yml

@@ -48,6 +48,8 @@ jobs:
 
     - name: Test with pytest
       run: |
+        podman system service --time=0 unix://tmp/podman.sock &
+        export CONTAINER_HOST="unix:///tmp/podman.sock"
         poetry run coverage run -m pytest --runslow
         poetry run coverage xml
 

.gitignore

@@ -22,6 +22,7 @@ instance/
 json/
 poetry.lock
 public/
+redis/
 site/
 store/
 var/

Containerfile

@@ -0,0 +1,16 @@
+FROM python:3.10-slim
+
+WORKDIR /app/
+
+RUN pip install poetry
+
+COPY poetry.lock pyproject.toml ./
+
+RUN poetry config virtualenvs.create false \
+    && poetry install --only main --no-interaction --no-ansi
+
+COPY ./asu/ ./asu/
+
+COPY ./misc/config.py /etc/asu/config.py
+
+CMD gunicorn 'asu.asu:create_app()' --bind 0.0.0.0:8000

README.md

@@ -1,4 +1,4 @@
-# Attendedsysupgrade Server for OpenWrt (GSoC 2017)
+# Attendedsysupgrade Server (GSoC 2017)
 
 [![codecov](https://codecov.io/gh/aparcar/asu/branch/master/graph/badge.svg)](https://codecov.io/gh/aparcar/asu)
 [![Code style: black](https://img.shields.io/badge/code%20style-black-000000.svg)](https://github.com/psf/black)
@@ -9,11 +9,8 @@ devices running OpenWrt or distributions based on it. These tools offer an easy
 way to reflash the router with a new firmware version
 (including all packages) without the need to use `opkg`.
 
-It's called Attended SysUpgrade (ASU) because the upgrade process is not started
-automatically, but is initiated by a user who waits until it's done.
-
-ASU is based on an API (described below) to request custom firmware images with
-any selection of packages pre-installed. This avoids the need to set up a build
+ASU is based on an [API](#api) to request custom firmware images with any
+selection of packages pre-installed. This avoids the need to set up a build
 environment, and makes it possible to create a custom firmware image even using
 a mobile device.
 
@@ -59,81 +56,56 @@ immediately without rebuilding.
 ### Active server
 
 - [sysupgrade.openwrt.org](https://sysupgrade.openwrt.org)
-- [asu.aparcar.org](https://asu.aparcar.org)
-- ~~[chef.libremesh.org](https://chef.libremesh.org)~~ (`CNAME` to
-  asu.aparcar.org)
+- [asu.aparcar.org](http://asu.aparcar.org:8000)
+- [asu.hauke-m.de](http://asu.hauke-m.de:8000)
 
 ## Run your own server
 
-Redis is required to store image requests:
-
-    sudo apt install redis-server tar
-
-Install _asu_:
-
-    pip install asu
-
-Create a `config.py`.
-You can use `misc/config.py` as an example.
-
-Start the server via the following commands:
+For security reasons each build happens inside a container so that one build
+can't affect another build. For this to work a Podman container runs an API
+service so workers can themselfs execute builds inside containers.
 
-    export FLASK_APP=asu.asu  # set Flask app to asu
-    flask janitor update      # download upstream profiles/packages - this runs forever
-    flask run                 # run development server - this runs forever
+Please install Podman and test if it works:
 
-Start the worker via the following comand:
+    podman run --rm -it docker.io/library/alpine:latest
 
-    rq worker                 # this runs forever
+Once Podman works, install `podman-compose`:
 
-### Docker
+    pip install podman-compose
 
-Run the service inside multiple Docker containers. The services include the _
-ASU_ server itself, a _janitor_ service which fills the Redis database with
-known packages and profiles as well as a `rqworker` which actually builds
-images.
-
-Currently all services share the same folder and therefore a very "open" access
-is required. Suggestions on how to improve this setup are welcome.
+Now it's possible to run all services via `podman-compose`:
 
     mkdir -p ./asu-service/public/
-    chmod -R 777 ./asu-service/
-    cp ./misc/config.py ./asu-service/
-    docker-compose up
+    podman-compose up -d
 
-A webserver should proxy API calls to port 8000 of the `server` service while
-the `asu/` folder should be file hosted as-is.
+This will start the server, the Podman API container and two workers. The first
+run needs a few minutes since available packages are parsed from teh upstream
+server. Once the server is running, it's possible to request images via the API
+on `http://localhost:8000`. Modify `podman-compose.yml` to change the port.
 
 ### Production
 
-It is recommended to run _ASU_ via `gunicorn` proxied by `nginx` or
-`caddyserver`. Find a possible server configurations in the `misc/` folder.
-
-The _ASU_ server will try `$PWD/config.py` and `/etc/asu/config.py` to find a
-configuration. Find an example configuration in the `misc/` folder.
-
-    pip install gunicorn
-    gunicorn "asu.asu:create_app()"
+For production it's recommended to use a reverse proxy like `nginx` or `caddy`.
 
-Ideally use the tool `squid` to cache package indexes, which are reloaded every
-time an image is built. Find a basic configuration in at `misc/squid.conf`
-which should be copied to `/etc/squid/squid.conf`.
-
-If you want to use `systemd` find the service files `asu.service` and
-`worker@.service` in the `misc` folder as well.
+#### System requirements
 
+- 2 GB RAM (4 GB recommended)
+- 2 CPU cores (4 cores recommended)
+- 50 GB disk space (200 GB recommended)
+
 ### Development
 
 After cloning this repository, create a Python virtual environment and install
 the dependencies:
 
-    python3 -m venv .direnv
-    source .direnv/bin/activate
-    pip install -r requirements.txt
-    export FLASK_APP=asu.asu  # set Flask app to asu
-    export FLASK_APP=tests.conftest:mock_app FLASK_DEBUG=1 # run Flask in debug mode with mock data
-    flask run
+#### Running the server
+
+    poetry install
+    poetry run flask run
+
+#### Running a worker
 
+    poetry run rq worker
 ### API
 
 The API is documented via _OpenAPI_ and can be viewed interactively on the