Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Clarify the horizontal scaling documentation #2984

Merged
merged 2 commits into from
Jul 7, 2023
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 1 addition & 1 deletion .github/workflows/audit.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -7,7 +7,7 @@ on:

jobs:
audit:
runs-on: ubuntu-20.04
runs-on: ubuntu-22.04
name: Audit
timeout-minutes: 20

Expand Down
2 changes: 1 addition & 1 deletion .github/workflows/auto-review.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -11,7 +11,7 @@ on:
jobs:
auto-merge:
name: Auto reviews updates
runs-on: ubuntu-20.04
runs-on: ubuntu-22.04
timeout-minutes: 5

steps:
Expand Down
2 changes: 1 addition & 1 deletion .github/workflows/backport.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -12,7 +12,7 @@ env:

jobs:
backport:
runs-on: ubuntu-20.04
runs-on: ubuntu-22.04
name: Backport
timeout-minutes: 5

Expand Down
2 changes: 1 addition & 1 deletion .github/workflows/clean.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -7,7 +7,7 @@ on:
- closed
jobs:
clean:
runs-on: ubuntu-20.04
runs-on: ubuntu-22.04
name: Clean docker hub tags
timeout-minutes: 5

Expand Down
45 changes: 0 additions & 45 deletions .github/workflows/codeql.yaml

This file was deleted.

2 changes: 1 addition & 1 deletion .github/workflows/delete-old-workflows-run.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -9,7 +9,7 @@ env:

jobs:
build:
runs-on: ubuntu-20.04
runs-on: ubuntu-22.04
timeout-minutes: 25
name: Delete old workflow runs

Expand Down
2 changes: 1 addition & 1 deletion .github/workflows/main.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -17,7 +17,7 @@ permissions:

jobs:
build:
runs-on: ubuntu-20.04
runs-on: ubuntu-22.04
timeout-minutes: 30
name: Continuous integration
if: "!startsWith(github.event.head_commit.message, '[skip ci] ')"
Expand Down
2 changes: 1 addition & 1 deletion .github/workflows/pr-checks.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -13,7 +13,7 @@ on:
jobs:
build:
name: Pull request check
runs-on: ubuntu-20.04
runs-on: ubuntu-22.04
timeout-minutes: 5

steps:
Expand Down
2 changes: 1 addition & 1 deletion .github/workflows/rebuild.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -6,7 +6,7 @@ on:

jobs:
rebuild:
runs-on: ubuntu-20.04
runs-on: ubuntu-22.04
timeout-minutes: 25
name: Rebuild

Expand Down
4 changes: 4 additions & 0 deletions docs/src/main/resources/templates/_main.html
Original file line number Diff line number Diff line change
Expand Up @@ -93,6 +93,10 @@
<li class="toctree-l1 {{current_docker}}">
<a class="reference internal {{current_docker}}" href="docker.html">Docker</a>
</li>

<li class="toctree-l1 {{current_scaling}}">
<a class="reference internal {{current_scaling}}" href="scaling.html">Horizontal scaling</a>
</li>
</ul>
</div>
</div>
Expand Down
45 changes: 0 additions & 45 deletions docs/src/main/resources/templates/docker.html
Original file line number Diff line number Diff line change
Expand Up @@ -104,48 +104,3 @@ <h4 id="docker_sentry">
Other parameters are documented here:
<a href="https://docs.sentry.io/clients/java/config/">https://docs.sentry.io/clients/java/config/</a>
</p>

<h4 id="multi">
Multi-instance
<a class="headerlink" href="#multi" title="Permalink to this headline">¶</a>
</h4>

<p>
In this mode several instances of the print will collaborate together to handle the print jobs. The state is
stored in a database. You need to add the following parameters to CATALINA_OPTS:
</p>
<ul>
<li><code>-Ddb.username=...</code>: The PostgresQL username</li>
<li><code>-Ddb.password=...</code>: The PostgresQL password</li>
<li><code>-Ddb.host=...</code>: The PostgresQL host name</li>
<li><code>-Ddb.name=...</code>: The PostgresQL database name</li>
<li><code>-Ddb.port=...</code>: The PostgresQL database port (defaults to <code>5432</code>)</li>
</ul>

<p>Example:</p>

<pre><code>
docker run --name=mapfish-print-test --publish=8080:8080
--env=TOMCAT_LOG_TYPE=json
--env=CATALINA_OPTS="-Ddb.name=mydb -Ddb.host=myserver -Ddb.username=myuser -Ddb.password=mypwd -Ddb.port=5432"
mydockerhub/mapfish-print:latest
</code></pre>

<p>
The necessary tables are created automatically (print_accountings, print_job_results, print_job_statuses).
In this mode the container will wait for the database to be reachable before actually starting the tomcat
server.
</p>

<p>The DB polling can be tuned with those two environment variables:</p>
<ul>
<li>
<code>PRINT_CANCEL_OLD_POLL_INTERVAL</code>: How often in seconds the DB is polled for jobs to be canceled
(default=<code>60</code>s)
</li>

<li>
<code>PRINT_POLL_INTERVAL</code>: How often in seconds the DB is polled for new jobs
(default=<code>0.5</code>s)
</li>
</ul>
115 changes: 115 additions & 0 deletions docs/src/main/resources/templates/scaling.html
Original file line number Diff line number Diff line change
@@ -0,0 +1,115 @@
<h4 id="scaling_intro">
Introduction
<a class="headerlink" href="#scaling_intro" title="Permalink to this headline">¶</a>
</h4>
<p>
The print has a state containing the job queue and each job's state. To enable horizontal scaling, these
states must be persisted.
</p>
<h4 id="scaling_solus">
Solution
<a class="headerlink" href="#scaling_solus" title="Permalink to this headline">¶</a>
</h4>
<p>
To store the state we use a PostgreSQL database, the database connection should be configured with the
following Java system properties:
</p>

<ul>
<li><code>db.host</code>: The database server host name</li>
<li><code>db.port</code>: The database server port (defaults to <code>5432</code>)</li>
<li><code>db.username</code>: The username to connect to the database</li>
<li><code>db.password</code>: The password to connect to the database</li>
<li><code>db.name</code>: The name of the database</li>
<li><code>db.schema</code>: The schema to use (defaults to <code>public</code>)</li>
</ul>

<p>
The schema should exist, and the necessary tables are created automatically (<code>print_accountings</code>,
<code>print_job_results</code>, <code>print_job_statuses</code>). In this mode, the container will wait for
the database to be reachable before actually starting the tomcat server.
</p>

<p>The DB polling can be tuned with these two environment variables:</p>
<ul>
<li>
<code>PRINT_CANCEL_OLD_POLL_INTERVAL</code>: How often in seconds the DB is polled for jobs to be canceled
(default=<code>60</code>s)
</li>

<li>
<code>PRINT_POLL_INTERVAL</code>: How often in seconds the DB is polled for new jobs
(default=<code>0.5</code>s)
</li>
</ul>

<h5 id="scaling_docker">
Docker
<a class="headerlink" href="#scaling_docker" title="Permalink to this headline">¶</a>
</h5>
<p>
In a Docker environment, the system properties should be added in the <code>CATALINA_OPTS</code> environment
variable Like that: <code>-D&lt;property name&gt;=&lt;property value&gt;</code>.
</p>

<h5 id="scaling_kubernetes">
Kubernetes
<a class="headerlink" href="#scaling_kubernetes" title="Permalink to this headline">¶</a>
</h5>
<p>In Kubernetes, you can reuse an environment variable with:</p>
<pre><code>
env:
- name: PROPERTY_VALUE
value: test
- name: CATALINA_OPTS
value: -D&lt;property name&gt;==$(PROPERTY_VALUE)
</code></pre>

<p>The order is important.</p>

<p>Full example where we get the database credentials from a secret:</p>

<pre>
<code>
env:
- name: PGHOST
valueFrom:
secretKeyRef:
key: hostname
name: database-credential-secret
- name: PGPORT
valueFrom:
secretKeyRef:
key: port
name: database-credential-secret
- name: PGUSER
valueFrom:
secretKeyRef:
key: username
name: database-credential-secret
- name: PGPASSWORD
valueFrom:
secretKeyRef:
key: password
name: database-credential-secret
- name: PGDATABASE
valueFrom:
secretKeyRef:
key: database
name: database-credential-secret
- name: PGSCHEMA
value: print
- name: PGOPTIONS
value: '-c statement_timeout=30000'
- name: PRINT_POLL_INTERVAL
value: '1'
- name: CATALINA_OPTS
value: >-
-Ddb.host=$(PGHOST)
-Ddb.port=$(PGPORT)
-Ddb.username=$(PGUSER)
-Ddb.password=$(PGPASSWORD)
-Ddb.name=$(PGDATABASE)
-Ddb.schema=$(PGSCHEMA)
</code>
</pre>
Loading