Skip to content

Commit

Permalink
README.md update (#90)
Browse files Browse the repository at this point in the history
* update README.md

Signed-off-by: see-quick <maros.orsak159@gmail.com>

* update to be more consistent

Signed-off-by: see-quick <maros.orsak159@gmail.com>

---------

Signed-off-by: see-quick <maros.orsak159@gmail.com>
  • Loading branch information
see-quick authored Oct 16, 2024
1 parent b46bd0e commit 477e1e9
Show file tree
Hide file tree
Showing 3 changed files with 52 additions and 22 deletions.
64 changes: 47 additions & 17 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -3,8 +3,8 @@

# Test container repository

The test container repository primarily relates to developing and maintaining test container code using an [Apache Kafka®](https://kafka.apache.org) image from the [strimzi/test-container-images](https://github.com/strimzi/test-container-images) repository.
The main dependency is the test container framework, which lets you control the lifecycle of the Docker container.
The test container repository primarily relates to developing and maintaining test container code using an [Apache Kafka®](https://kafka.apache.org) image from the [strimzi/test-container-images](https://github.com/strimzi/test-container-images) repository.
The main dependency is the Testcontainers framework, which lets you control the lifecycle of Docker containers in your tests.

## Why use a Test container?

Expand All @@ -15,17 +15,17 @@ The most important classes are described here::
You can use it in two ways:
1. As an embedded ZooKeeper to run inside a Kafka container.
2. As an external ZooKeeper.
An additional configuration for Kafka brokers can be injected through the constructor.
The first one is using an embedded ZooKeeper which will run inside Kafka container.
Another option is to use @StrimziZookeeperContainer as an external ZooKeeper.
An additional configuration for Kafka brokers can be injected through the constructor.
This container is a good fit for integration testing, but for more comprehensive testing, we suggest using @StrimziKafkaCluster.
3. In KRaft mode [KIP-500](https://github.com/apache/kafka/blob/trunk/config/kraft/README.md), which allows running Apache Kafka without Apache ZooKeeper.

Additional configuration for Kafka brokers can be injected through methods such as withKafkaConfigurationMap.
This container is a good fit for integration testing, but for more comprehensive testing, we suggest using StrimziKafkaCluster.

- `StrimziZookeeperContainer` is an instance of ZooKeeper encapsulated inside a Docker container using an image from quay.io/strimzi-test-container/test-container with the given version.
It can be combined with @StrimziKafkaContainer, but we suggest using directly @StrimziKafkaCluster for more complicated testing.
- `StrimziKafkaCluster` is a multi-node instance of Kafka and ZooKeeper using the latest image from quay.io/strimzi-test-container/test-container with the given version.
It's a perfect fit for integration or system testing.
We always deploy one ZooKeeper with a specified number of Kafka instances, running as a separate container inside Docker.
The additional configuration for Kafka brokers can be passed to the constructor.
Additional configuration for Kafka brokers can be passed using the builder pattern.

In summary, you can use `StrimziKafkaContainer` to use a Kafka cluster with a single Kafka broker.
Or you can use `StrimziKafkaCluster` for a Kafka cluster with multiple Kafka brokers.
Expand All @@ -41,11 +41,11 @@ Using Strimzi test container takes two simple steps:
Add the Strimzi test container to your project as dependency, for example with Maven:

```
// in case of 0.102.0 version
// in case of 0.108.0 version
<dependency>
<groupId>io.strimzi</groupId>
<artifactId>strimzi-test-container</artifactId>
<version>0.102.0</version>
<version>0.108.0</version>
</dependency>
```

Expand Down Expand Up @@ -88,7 +88,7 @@ strimziKafkaContainer.start();
```java
StrimziKafkaContainer strimziKafkaContainer = new StrimziKafkaContainer()
.withBrokerId(1)
.withKraft(true);
.withKraft();

strimziKafkaContainer.start();
```
Expand All @@ -98,7 +98,8 @@ strimziKafkaContainer.start();
By default, the Kafka container will be exposed on a random host port. To expose Kafka on a fixed port:

```java
StrimziKafkaContainer strimziKafkaContainer = strimziKafkaContainer.withFixedPort(9092);
StrimziKafkaContainer strimziKafkaContainer = new StrimziKafkaContainer()
.withFixedPort(9092);

strimziKafkaContainer.start();
```
Expand All @@ -108,7 +109,7 @@ strimziKafkaContainer.start();
You can configure Kafka by providing a `server.properties` file:

```java
StrimziKafkaContainer strimziKafkaContainer = strimziKafkaContainer
StrimziKafkaContainer strimziKafkaContainer = new StrimziKafkaContainer()
.withServerProperties(MountableFile.forClasspathResource("server.properties"));
strimziKafkaContainer.start();
```
Expand All @@ -122,7 +123,7 @@ Properties configured through `withKafkaConfigurationMap` will also precede thos
You can customize the bootstrap servers, thus the advertised listeners property by:

```java
StrimziKafkaContainer strimziKafkaContainer = strimziKafkaContainer
StrimziKafkaContainer strimziKafkaContainer = new StrimziKafkaContainer()
.withBootstrapServers(container -> String.format("SSL://%s:%s", container.getHost(), container.getMappedPort(9092)));
strimziKafkaContainer.start();
```
Expand All @@ -137,7 +138,7 @@ Before starting the container, use the following code configuring Test Container
```java
StrimziKafkaContainer strimziKafkaContainer = new StrimziKafkaContainer()
.withBrokerId(1)
.withKraft(true)
.withKraft()
.waitForRunning();

strimziKafkaContainer.start();
Expand All @@ -149,9 +150,9 @@ Strimzi test container supported versions can be find in `src/main/java/resource

```java
StrimziKafkaContainer strimziKafkaContainer = new StrimziKafkaContainer()
.withKafkaVersion("2.8.0")
.withKafkaVersion("3.8.0")
.withBrokerId(1)
.withKraft(true)
.withKraft()
.waitForRunning();

strimziKafkaContainer.start();
Expand Down Expand Up @@ -205,6 +206,35 @@ systemUnderTest.start();
strimziKafkaContainer.getProxy().setConnectionCut(true);
```

#### xi) Run a multi-node Kafka cluster

To run a multi-node Kafka cluster, you can use the StrimziKafkaCluster class with the builder pattern.

1. Default configuration (ZK-based)

```java
StrimziKafkaCluster kafkaCluster = new StrimziKafkaCluster.StrimziKafkaClusterBuilder()
.withNumberOfBrokers(3)
.build();

kafkaCluster.start();
```

2. Specified Kafka version, KRaft-enabled and additional Kafka configuration

```java
StrimziKafkaCluster kafkaCluster = new StrimziKafkaCluster.StrimziKafkaClusterBuilder()
.withNumberOfBrokers(3)
.withKafkaVersion("3.8.0") // if not specified then latest Kafka version is selected
.withKraft() // if not specified ZK-based is used
.withAdditionalKafkaConfiguration(Map.of(
"log.cleaner.enable", "false"
))
.build();

kafkaCluster.start();
```

### Additional tips

1. In case you are using `Azure pipelines` Ryuk needs to be turned off, since Azure does not allow starting privileged containers.
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -308,7 +308,7 @@ public StrimziKafkaClusterBuilder withKafkaVersion(String kafkaVersion) {
return this;
}

public StrimziKafkaClusterBuilder withKraftEnabled() {
public StrimziKafkaClusterBuilder withKraft() {
this.enableKRaft = true;
return this;
}
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -69,7 +69,7 @@ void testKafkaClusterStartupWithSharedNetwork() throws InterruptedException, Exe
systemUnderTest = new StrimziKafkaCluster.StrimziKafkaClusterBuilder()
.withNumberOfBrokers(NUMBER_OF_REPLICAS)
.withSharedNetwork()
.withKraftEnabled()
.withKraft()
.build();
systemUnderTest.start();

Expand All @@ -96,7 +96,7 @@ void testKafkaClusterWithSharedNetworkFunctionality() throws ExecutionException,
systemUnderTest = new StrimziKafkaCluster.StrimziKafkaClusterBuilder()
.withNumberOfBrokers(NUMBER_OF_REPLICAS)
.withSharedNetwork()
.withKraftEnabled()
.withKraft()
.build();
systemUnderTest.start();

Expand All @@ -120,7 +120,7 @@ void testStartClusterWithProxyContainer() {
kafkaCluster = new StrimziKafkaCluster.StrimziKafkaClusterBuilder()
.withNumberOfBrokers(NUMBER_OF_REPLICAS)
.withProxyContainer(proxyContainer)
.withKraftEnabled()
.withKraft()
.build();

kafkaCluster.start();
Expand All @@ -142,7 +142,7 @@ void testStartClusterWithProxyContainer() {
private void setUpKafkaKRaftCluster() {
systemUnderTest = new StrimziKafkaCluster.StrimziKafkaClusterBuilder()
.withNumberOfBrokers(NUMBER_OF_REPLICAS)
.withKraftEnabled()
.withKraft()
.build();
systemUnderTest.start();
}
Expand Down

0 comments on commit 477e1e9

Please sign in to comment.