-
Notifications
You must be signed in to change notification settings - Fork 19
/
architecture.html.md.erb
114 lines (84 loc) · 6.98 KB
/
architecture.html.md.erb
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
---
title: Concourse Architecture
owner: Concourse
---
<strong><%= modified_date %></strong>
This topic describes the VMs and software components that make up a Concourse installation.
## <a id="diagram"></a> Component Diagram
Concourse software components can be logically split into two main subsystems: a web (or server) subsystem, and one or more workers. In a simple setup, both subsystems can run on a single VM. In more complex systems, Concourse web and worker subsystems run on multiple VMs.
This diagram shows a Concourse deployment with web and server subsystems on separate VMs. You can deploy each subsystem on a single VM, as shown, or on multiple VMs in parallel. A dedicated database VM runs Concourse’s PostgreSQL server. If needed, an optional load balancer points external traffic to web VMs.
The [Component Descriptions](#description) below explain what each component does.
The [Internal vs. External Workers](#comms) section below describes the two setups that enable the
Air Traffic Controller (ATC) to communicate with workers approved by the
Transit Security Administration (TSA), depending on whether or not it can access the worker VMs directly.
## <a id="description"></a> Component Descriptions
The following table describes the function of each Concourse component:
<table>
<tr>
<th>Subsystem</th>
<th>Component</th>
<th>Function</th>
</tr><tr>
<td rowspan=2><b>Web</b></td>
<td><b>ATC</b></td>
<td>Runs an HTTP server that:
<ul><li>Runs the Concourse web UI.</li>
<li>Serves the Concourse API on port 8080, to support the Fly CLI and other tools.</li>
<li>Schedules pipelines to run on worker VMs.</li></ul></td>
</tr><tr>
<td><b>TSA</b></td>
<td>Runs an SSH server that:
<ul><li>Listens on port 2222.</li>
<li>Securely <a href="https://concourse-ci.org/concepts.html#component-tsa">registers worker VMs</a> for use by the ATC.</li>
<li>For <a href="#comms-external">external worker</a> VMs, handshakes with Beacon to open a reverse SSH tunnel for ongoing communication between the ATC and the worker.</li>
<li>Pings workers for heartbeat every 30 seconds, updates ATC with worker metric changes, and,
if there is no response, notifies ATC that the worker is in a <code>stalled</code> state.</li></ul>
</td>
</tr><tr>
<td><b>Database</b></td>
<td><b>PostgreSQL</td>
<td>Stores pipeline data, such as build configurations, build logs, and resource versions, for use by worker VMs.</td>
</tr><tr>
<td rowspan=3><b>Worker</b></td>
<td><b>Beacon</b></td>
<td><ul><li>Sends Garden and Baggageclaim addresses to the ATC, to register that the worker is ready to run Concourse build pipelines</li>
<li>For <a href="#comms-external">external worker</a> VMs, handshakes with TSA to open a reverse SSH tunnel
for ongoing communication between the ATC and the worker.</li></ul>
</tr><tr>
<td><b>Garden</b></td>
<td>Runs an HTTP server that:
<ul><li>Listens on port 7777.</li>
<li>Provides an API for the ATC to manage <b>containers</b> to do things like:
<ul><li>Discover new versions of <a href="https://concourse-ci.org/resources.html">resources</a>.</li>
<li>Run the tasks in a pipeline <a href="https://concourse-ci.org/builds.html">build plan.</a></li></ul></li></ul></td>
</tr><tr>
<td><b>Baggageclaim</b></td>
<td>Runs an HTTP server that:
<ul><li>Listens on port 7788.</li>
<li>Provides an API for the ATC to manage disk resources, or <b>volumes</b>, mounted to containers.</li>
<li>Uses volumes to store persistent caches of resource versions and to propagate artifacts such as caches and task outputs through the containers used for a build.</li></ul></td>
</tr>
</table>
For more information, see [Concepts](https://concourse-ci.org/concepts.html) in the Concourse documentation.
## <a id="comms"></a> Internal vs. External Workers
Concourse workers can be either _internal_, also called _local_; or _external_, also called _remote_. These two types of worker VMs differ as follows:
* A worker VM is **internal** when the web VM, which hosts the ATC, can make direct HTTP calls to the worker VM. This is typical when web and worker VMs are all on the same private network.
* A worker VM is **external** when the web VM cannot directly connect to the worker VM through HTTP or SSH.
This is typical when the worker VMs are behind a firewall in a separate network from the web VM
or when security groups prevent communication from web VM to worker.
You might use both internal and external workers in the same deployment. For example, if your main Concourse installation runs on Linux and you need Windows workers to run tests in a remote, Windows-based infrastructure.
Both internal and external workers can use SSH to securely log in to the web VM.
Concourse sets up secure communications from the ATC to worker VMs in two ways, depending on whether the worker VMs are internal or external, as described below.
### <a id="comms-internal"></a> Internal Worker Communications
Concourse sets up communications between the ATC and internal worker VMs as follows:
1. The worker's Beacon connects to the TSA through SSH and runs the `register-worker` command to the TSA, passing in the worker payload. For more information, see [payload](https://github.com/concourse/concourse/blob/master/atc/worker.go) in GitHub.
1. The TSA starts heartbeating every 30 seconds, pinging the worker for its current number of Garden containers and Baggageclaim volumes, and sending updated information to the ATC. If the worker fails to respond, the TSA notifies ATC that worker is <code>stalled</code>.
1. ATC makes direct HTTP calls to Garden and Baggageclaim APIs on the worker, to create containers and volumes for running containerized processes.
### <a id="comms-external"></a> External Worker Communications
Concourse sets up communications between the ATC and external worker VMs as follows:
1. The worker's Beacon connects to the TSA through SSH and runs the `forward-worker` command to the TSA, passing in the worker payload. For more information, see [payload](https://github.com/concourse/concourse/blob/master/atc/worker.go) in GitHub.
1. The `forward-worker` command establishes two unique hidden local ports on the web VM, and creates a reverse-SSH tunnel from these local ports to the external worker's Garden and Baggageclaim ports.
1. The TSA replaces the ATC's Garden and Baggageclaim addresses for the worker to the hidden local port addresses.
1. The TSA starts heartbeating every 30 seconds, pinging the worker for its current number of Garden containers and Baggageclaim volumes, and sending updated information to the ATC. If the worker fails to respond, the TSA notifies ATC that worker is <code>stalled</code>.
1. The ATC calls Garden and Baggageclaim APIs on the worker by sending the HTTP requests to its own hidden local ports. The SSH daemon on the web VM then forwards the requests to the worker VM through the reverse-SSH tunnel.