Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Funkwhale new flist #305

Open
wants to merge 4 commits into
base: development
Choose a base branch
from
Open
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
55 changes: 55 additions & 0 deletions tfgrid3/funkwhale/Dockerfile
Original file line number Diff line number Diff line change
@@ -0,0 +1,55 @@
FROM ubuntu:22.04

Check failure

Code scanning / Trivy

Image user should not be 'root' High

Artifact: tfgrid3/funkwhale/Dockerfile
Type: dockerfile
Vulnerability DS002
Severity: HIGH
Message: Specify at least 1 USER command in Dockerfile with non-root user as argument
Link: DS002
ENV DEBIAN_FRONTEND=noninteractive

# Install necessary packages
RUN apt-get update && \
apt-get upgrade -y && \
apt-get install -y --no-install-recommends \
wget \
ca-certificates \
curl \
gettext \
gnupg \
openssh-server \
ufw \
nginx \
certbot \
python3-certbot-nginx && \
apt-get clean && \
rm -rf /var/lib/apt/lists/* /tmp/* /var/tmp/*

# Download and install the latest Zinit
RUN curl -s https://api.github.com/repos/threefoldtech/zinit/releases/latest | \
grep "browser_download_url" | \
cut -d '"' -f 4 | \
wget -qi - -O /sbin/zinit && \
chmod +x /sbin/zinit

# Copy configuration scripts
COPY scripts/ufw.sh /usr/local/bin/
COPY scripts/dockerd.sh /usr/local/bin/
COPY scripts/start-fk.sh /usr/local/bin/
RUN chmod +x /usr/local/bin/ufw.sh /usr/local/bin/dockerd.sh /usr/local/bin/start-fk.sh

# Add Zinit configuration files
ADD zinit /etc/zinit/

# Install Docker, Buildx, and Docker Compose
RUN apt-get update && \
apt-get install -y apt-transport-https ca-certificates curl software-properties-common && \
apt-get update && \
apt-get install -y docker.io && \
curl -L "https://github.com/docker/compose/releases/download/$(curl -s https://api.github.com/repos/docker/compose/releases/latest | grep -Po '"tag_name": "\K.*?(?=")')/docker-compose-$(uname -s)-$(uname -m)" -o /usr/local/bin/docker-compose && \
chmod +x /usr/local/bin/docker-compose && \
ln -s /usr/local/bin/docker-compose /usr/bin/docker-compose
Comment on lines +38 to +44

Check failure

Code scanning / Trivy

'apt-get' missing '--no-install-recommends' High

Artifact: tfgrid3/funkwhale/Dockerfile
Type: dockerfile
Vulnerability DS029
Severity: HIGH
Message: '--no-install-recommends' flag is missed: 'apt-get update && apt-get install -y apt-transport-https ca-certificates curl software-properties-common && apt-get update && apt-get install -y docker.io && curl -L "https://github.com/docker/compose/releases/download/$(curl -s https://api.github.com/repos/docker/compose/releases/latest | grep -Po '"tag_name": "\K.*?(?=")')/docker-compose-$(uname -s)-$(uname -m)" -o /usr/local/bin/docker-compose && chmod +x /usr/local/bin/docker-compose && ln -s /usr/local/bin/docker-compose /usr/bin/docker-compose'
Link: DS029


# Clean up any remaining temporary files
RUN apt-get clean && \
rm -rf /var/lib/apt/lists/* /tmp/* /var/tmp/*

# Expose ports for HTTP and SSH access
EXPOSE 5000 222

# Command to run Zinit in the container
CMD ["/sbin/zinit", "init", "--container"]
41 changes: 41 additions & 0 deletions tfgrid3/funkwhale/README.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,41 @@

# Funkwhale for ThreeFold Grid

This Directory contains the configuration files and setup required to deploy **Funkwhale**, a self-hosted, decentralized audio platform. The directory includes Docker Compose files, service scripts, and configurations necessary for running Funkwhale.


## Building the Funkwhale Image

- ### To build the Docker image for Funkwhale, you can follow these steps:

#### Build the Docker image and tag it for the threefolddev repository::
```
cd tf-image/tfgrid3/funkwhale/
docker build -t threefolddev/funkwhale:$FUNKWHALE_VERSION .
```
#### Log in to the Docker registry:
```docker login ```
#### Push the image to the threefolddev account:
```docker push threefolddev/funkwhale:$FUNKWHALE_VERSION```

#### Convert the docker image to Zero-OS flist
Convert the docker image to Flist is using [Docker Hub Converter tool](https://hub.grid.tf/docker-convert), make sure you already built and pushed the docker image to docker hub before using this tool.


## Environment Variables

Several environment variables need to be configured before deploying Funkwhale:

- `FUNKWHALE_SUPERUSER_NAME`: The username for the Funkwhale superuser.
- `FUNKWHALE_SUPERUSER_EMAIL`: The email for the superuser account.
- `FUNKWHALE_SUPERUSER_PASSWORD`: Password for the superuser account.
- `FUNKWHALE_VERSION`: The version of Funkwhale to use (default: `1.4.0`).
- `Domain`: The domain name for the Funkwhale instance (e.g., `funkwhale.example.com`).
- `DJANGO_SECRET_KEY`: A unique secret key for Django. This environment variable is generated automatically by the **start script**.

These variables are passed via `.env` files or Zinit service configurations. Ensure they are set properly before deployment.

## SSL Configuration
This setup uses Certbot to generate and configure SSL certificates for your domain. Ensure the domain is properly configured with DNS settings that point to your server. The certificates are generated automatically during the deployment process.


3 changes: 3 additions & 0 deletions tfgrid3/funkwhale/scripts/dockerd.sh
Original file line number Diff line number Diff line change
@@ -0,0 +1,3 @@
#!/bin/bash
echo -500 > /proc/self/oom_score_adj
exec /usr/bin/dockerd -H unix:// --containerd=/run/containerd/containerd.sock
82 changes: 82 additions & 0 deletions tfgrid3/funkwhale/scripts/start-fk.sh
Original file line number Diff line number Diff line change
@@ -0,0 +1,82 @@
#!/bin/bash
set -ex

# Check if FUNKWHALE_VERSION is set, if not, default to 1.4.0
FUNKWHALE_VERSION=${FUNKWHALE_VERSION:-1.4.0}

# Change to Funkwhale directory
cd /srv/funkwhale/

# Download the Docker Compose and environment files for the specified version
curl -L -o docker-compose.yml "https://dev.funkwhale.audio/funkwhale/funkwhale/raw/${FUNKWHALE_VERSION}/deploy/docker-compose.yml"
curl -L -o .env "https://dev.funkwhale.audio/funkwhale/funkwhale/raw/${FUNKWHALE_VERSION}/deploy/env.prod.sample"

# Set appropriate permissions for the .env file
chmod 600 .env

# Generate a random Django secret key and update the .env file
DJANGO_SECRET=$(openssl rand -base64 45)
sed -i "s#^DJANGO_SECRET_KEY=.*#DJANGO_SECRET_KEY=$DJANGO_SECRET#" .env

# Add environment variables to the .env file
sed -i "s#^FUNKWHALE_VERSION=.*#FUNKWHALE_VERSION=$FUNKWHALE_VERSION#" .env
sed -i "s#^FUNKWHALE_HOSTNAME=.*#FUNKWHALE_HOSTNAME=$Domain#" .env

# Pull the latest Docker images for Funkwhale
docker-compose pull

# Start PSQL service
docker-compose up -d postgres

# Run database migrations
docker-compose run --rm api funkwhale-manage migrate

# Create a superuser using the provided credentials
docker-compose run --rm -T api funkwhale-manage fw users create --superuser <<EOF
$FUNKWHALE_SUPERUSER_NAME
$FUNKWHALE_SUPERUSER_PASSWORD
$FUNKWHALE_SUPERUSER_EMAIL
EOF

# Start Funkwhale services
docker-compose up -d

# Check if $Domain is set before proceeding with SSL generation
if [ -z "$Domain" ]; then
echo "No Domain provided. Skipping SSL setup and exiting."
exit 0
fi

# Download and apply the Funkwhale Nginx proxy configuration
curl -L -o /etc/nginx/funkwhale_proxy.conf "https://dev.funkwhale.audio/funkwhale/funkwhale/raw/$FUNKWHALE_VERSION/deploy/funkwhale_proxy.conf"
curl -L -o /etc/nginx/sites-available/funkwhale.template "https://dev.funkwhale.audio/funkwhale/funkwhale/raw/$FUNKWHALE_VERSION/deploy/docker.proxy.template"

# Apply environment variables to the Nginx template and create the final Nginx configuration
set -a && source /srv/funkwhale/.env && set +a
envsubst "`env | awk -F = '{printf \" $%s\", $$1}'`" < /etc/nginx/sites-available/funkwhale.template > /etc/nginx/sites-available/funkwhale.conf

# Comment out SSL cert lines temporarily to allow HTTP access before SSL setup
sed -i "s/listen 443 ssl http2\;/#listen 443 ssl http2\;/" /etc/nginx/sites-available/funkwhale.conf
sed -i "s/listen \[::\]:443 ssl http2;/#listen \[::\]:443 ssl http2;/" /etc/nginx/sites-available/funkwhale.conf
sed -i "s/ssl_certificate/\#ssl_certificate/" /etc/nginx/sites-available/funkwhale.conf

# Enable the Funkwhale Nginx site and create a backup of the current configuration
ln -s /etc/nginx/sites-available/funkwhale.conf /etc/nginx/sites-enabled/
cp /etc/nginx/sites-available/funkwhale.conf /etc/nginx/sites-available/funkwhale.conf.bak

# Generate SSL certificate using Certbot
certbot --nginx -d $Domain --non-interactive --agree-tos --register-unsafely-without-email

# Restore the original Nginx configuration from the backup
mv /etc/nginx/sites-available/funkwhale.conf.bak /etc/nginx/sites-available/funkwhale.conf

# Uncomment the SSL lines after Certbot finishes
sed -i "s/\#ssl_certificate/ssl_certificate/" /etc/nginx/sites-available/funkwhale.conf
sed -i "s/\#listen/listen/" /etc/nginx/sites-available/funkwhale.conf

# Add HTTP to HTTPS redirection in the Nginx configuration
sed -i "11a \ if (\$host = $Domain) {\n return 301 https://\$host\$request_uri;\n }" /etc/nginx/sites-available/funkwhale.conf

# Restart Nginx to apply the New SSL configuration
zinit stop nginx
zinit start nginx
9 changes: 9 additions & 0 deletions tfgrid3/funkwhale/scripts/ufw.sh
Original file line number Diff line number Diff line change
@@ -0,0 +1,9 @@
#!/bin/bash
set -x

ufw default deny incoming
ufw default allow outgoing
ufw allow ssh
ufw limit ssh
ufw allow 443
ufw allow 80
3 changes: 3 additions & 0 deletions tfgrid3/funkwhale/zinit/containerd.yaml
Original file line number Diff line number Diff line change
@@ -0,0 +1,3 @@
exec: /usr/bin/containerd
after:
- ufw
4 changes: 4 additions & 0 deletions tfgrid3/funkwhale/zinit/dockerd.yaml
Original file line number Diff line number Diff line change
@@ -0,0 +1,4 @@
exec: /usr/local/bin/dockerd.sh
after:
- containerd
test: docker ps
4 changes: 4 additions & 0 deletions tfgrid3/funkwhale/zinit/funkwhale.yaml
Original file line number Diff line number Diff line change
@@ -0,0 +1,4 @@
exec: /bin/bash /usr/local/bin/start-fk.sh
after:
- dockerd
oneshot: true
4 changes: 4 additions & 0 deletions tfgrid3/funkwhale/zinit/nginx.yaml
Original file line number Diff line number Diff line change
@@ -0,0 +1,4 @@
exec: nginx -g "daemon off;"
after:
- dockerd
restart: true
3 changes: 3 additions & 0 deletions tfgrid3/funkwhale/zinit/sshd.yaml
Original file line number Diff line number Diff line change
@@ -0,0 +1,3 @@
exec: /usr/sbin/sshd -D
after:
- sshkey
12 changes: 12 additions & 0 deletions tfgrid3/funkwhale/zinit/sshkey.yaml
Original file line number Diff line number Diff line change
@@ -0,0 +1,12 @@
exec: |
bash -c '
if [ ! -z "$SSH_KEY" ]; then
mkdir -p /var/run/sshd
mkdir -p /root/.ssh
touch /root/.ssh/authorized_keys

chmod 700 /root/.ssh
chmod 600 /root/.ssh/authorized_keys
fi
'
oneshot: true
4 changes: 4 additions & 0 deletions tfgrid3/funkwhale/zinit/ufw.yaml
Original file line number Diff line number Diff line change
@@ -0,0 +1,4 @@
exec: ufw --force enable
oneshot: true
after:
- ufw_init
3 changes: 3 additions & 0 deletions tfgrid3/funkwhale/zinit/ufw_init.yaml
Original file line number Diff line number Diff line change
@@ -0,0 +1,3 @@
exec: /bin/sh -eux -c "
/usr/local/bin/ufw.sh;"
oneshot: true
Loading