Skip to content

Commit

Permalink
[WIP, SQUASH] README section tweak
Browse files Browse the repository at this point in the history
  • Loading branch information
SwooshyCueb committed Jul 27, 2022
1 parent a161291 commit 42b4af4
Showing 1 changed file with 8 additions and 10 deletions.
18 changes: 8 additions & 10 deletions irods_audit_elk_stack/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -15,9 +15,7 @@ Case in point:
Elasticstack security is explicitly disabled,
and the Python script standing in for Logstash was not written with performance or resilience in mind.

## Container Contents

### Overview
## Container Overview

This Ubuntu Focal-based container contains Elasticsearch 8, Kibana 8, RabbitMQ (with AMQP 1.0 plugin and management plugin), and a Python daemon that was specifically written for this demonstration to stand in for an AMQP 1.0-capable Logstash.

Expand Down Expand Up @@ -45,9 +43,9 @@ The entrypoint takes a single optional argument, `--es-java-heap-size`, to set t
| `15672` | `TCP`/`HTTP` | RabbitMQ management plugin listens on this port for web browsers and HTTP API clients |
| `5601` | `TCP`/`HTTP` | Kibana listens on this port for web browsers and REST API clients |

### Details
## Container Details

#### JVM
### JVM

The JDK/JRE used in this container is [Temurin](https://adoptium.net/temurin) 17 with the Hotspot JVM.

Expand All @@ -65,11 +63,11 @@ Instead of using the [Eclipse-provided Focal-based Temurin 17 docker image](http

[^1]: At present, the full JDK is installed (minus the dpkg excludes). We are investigating using `jlink` to construct a JRE that includes only the components we need for the demonstration.

#### RabbitMQ
### RabbitMQ

The [`rabbitmq_amqp1_0`](https://github.com/rabbitmq/rabbitmq-server/tree/master/deps/rabbitmq_amqp1_0) and [`rabbitmq_management`](https://github.com/rabbitmq/rabbitmq-server/tree/master/deps/rabbitmq_management) plugins are enabled. The `test` administrator account is created in the Dockerfile.

#### Elasticsearch
### Elasticsearch

Elasticsearch is configured for a single-node cluster. Security is explicitly disabled, as are machine learning APIs. Both the transport and HTTP ports are configured to specific ports instead of a port range (`9200` and `9300`, respectively).

Expand All @@ -81,7 +79,7 @@ The Elasticsearch JVM is configured to not dump its heap on an out-of-memory err

`dpkg` is configured to drop the bundled JVM from the Elasticsearch package, so it is not installed in the container.

#### Kibana
### Kibana

Kibana is initialized with a sample dashboard useful for demonstrating how one might use Kibana to aggregate metrics from audit data.

Expand All @@ -90,13 +88,13 @@ Compared to other `init.d` script implementations for Kibana (and the systemd un

`dpkg` is configured to drop includes and manpages from Kibana's bundled `nodejs`, so they are not installed in the container.

#### Logstash Stand-In Python Script
### Logstash Stand-In Python Script

We have written a Python script that uses [Qpid Proton](http://web.archive.org/web/20130717085741/http://qpid.apache.org/releases/qpid-0.22/messaging-api/python/api/index.html) to pull AMQP 1.0 messages from RabbitMQ, perform a few transformations on the message (see the following subsection and the script itself for more info on this), and then push the data to Elasticsearch.

The `init.d` script that daemonizes this script is based on the `init.d` script provided by the Elasticsearch 7 packages.

##### Why not Logstash?
#### Why not Logstash?

Previously, we used Logstash to move data from RabbitMQ to Elasticsearch. This worked well enough for demonstration purposes before 4.3.0 was released, but the fact that it worked *at all* was pure coincidence.

Expand Down

0 comments on commit 42b4af4

Please sign in to comment.