Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[WFLY-19359] define kafka yaml file for OpenShift and update the README. #929

Merged
merged 2 commits into from
Jun 17, 2024
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
Original file line number Diff line number Diff line change
Expand Up @@ -26,103 +26,36 @@ function applicationName() {
# 1 - application name
function installPrerequisites()
{
application="${1}"
echo "Creating amq-streams-operator-group"
maxWaitingTimeInSeconds=900 # 15 minutes

oc apply -f - <<EOF
apiVersion: operators.coreos.com/v1
kind: OperatorGroup
metadata:
name: amq-streams-operator-group
namespace: default
spec: {}
EOF
echo "Subscribe the AMQ stream operator"
oc apply -f ./charts/amq-operator-on-openshift.yaml --wait --timeout=10m0s

echo "Creating amq-streams-subscription"
oc apply -f - <<EOF
apiVersion: operators.coreos.com/v1alpha1
kind: Subscription
metadata:
name: amq-streams-subscription
namespace: default
spec:
channel: stable
installPlanApproval: Automatic
name: amq-streams
source: redhat-operators
sourceNamespace: openshift-marketplace
startingCSV: amqstreams.v2.5.0-0
EOF


seconds=120
echo "Looping for 15 minutes until the AMQ stream Operator is ready to use"
now=$(date +%s)
end=$(($seconds + $now))

echo "Looping for 2 minutes until the 'kafka' CRD is available "
end=$((maxWaitingTimeInSeconds + $now))
while [ $now -lt $end ]; do
# It takes a while for the kafka CRD to be ready
sleep 5
echo "Trying to create my-cluster"
oc apply -f - <<EOF
apiVersion: kafka.strimzi.io/v1beta2
kind: Kafka
metadata:
name: my-cluster
spec:
kafka:
replicas: 3
listeners:
- name: plain
port: 9092
type: internal
tls: false
storage:
type: ephemeral
zookeeper:
replicas: 3
storage:
type: ephemeral
entityOperator:
topicOperator: {}
EOF
if [ "$?" = "0" ]; then
break
fi
now=$(date +%s)
echo "Checking if The AMQ operator is ready"
# Check the entity operator exists. If Phase is "Succeeded" it is installed and we can continue
oc get ClusterServiceVersion | grep "Red Hat Streams for Apache Kafka" | grep "Succeeded" || continue
echo "The AMQ operator is ready!"
break
done

echo "Creating testing topic"
oc apply -f - <<EOF
apiVersion: kafka.strimzi.io/v1beta2
kind: KafkaTopic
metadata:
name: testing
labels:
strimzi.io/cluster: my-cluster
spec:
partitions: 3
replicas: 3
EOF

echo "Create a AMQ streams instance and testing topic"
oc apply -f ./charts/kafka-on-openshift.yaml --wait --timeout=10m0s

# Wait for the pods to come up
seconds=900
now=$(date +%s)
end=$(($seconds + $now))

echo "Looping for 15 minutes until the Kafka cluster is ready"
end=$((maxWaitingTimeInSeconds + $now))
while [ $now -lt $end ]; do
sleep 15
echo "Checking if pods are ready"

# Check the entity operator exists. It will have a name like my-cluster-entity-operator-<pod suffix>
# We do this check first because it takes a while to appear
oc get pods -l app.kubernetes.io/instance='my-cluster',app.kubernetes.io/name='entity-operator' | grep "my-cluster-entity-operator" || continue

# Wait 10 seconds for all pods to come up, and renter the loop if not
oc wait pod -l app.kubernetes.io/instance='my-cluster' --for=condition=Ready --timeout=10s || continue

# If we got here, everything is up, so we can proceed
sleep 5
echo "Checking if \"my-cluster-entity-operator\" pod is ready"
# Check the entity operator exists. And 1/1 instance is ready to use
oc get pod | grep "my-cluster-entity-operator" | grep "1/1" || continue
echo "The AMQ stream instance ready!"
break
done
}
Expand All @@ -135,10 +68,7 @@ EOF
# 1 - application name
function cleanPrerequisites()
{
# TODO There are a few topics created that need cleaning up

oc delete kafka my-cluster
oc delete subscription amq-streams-subscription
oc delete operatorgroup amq-streams-operator-group
oc delete deployment amq-streams-cluster-operator-v2.5.0-1
echo "Deleting all AMQ streams resources"
oc delete -f ./charts/amq-operator-on-openshift.yaml --wait --timeout=10m0s
oc delete -f ./charts/kafka-on-openshift.yaml --wait --timeout=10m0s
}
4 changes: 4 additions & 0 deletions microprofile-reactive-messaging-kafka/README-source.adoc
Original file line number Diff line number Diff line change
Expand Up @@ -987,6 +987,10 @@ bin/kafka-topics.sh --create --topic testing --bootstrap-server localhost:9092

// OpenShift
include::../shared-doc/build-and-run-the-quickstart-with-openshift.adoc[leveloffset=+1]
----
$ oc delete -f ./charts/kafka-on-openshift.yaml --wait --timeout=10m0s
emmartins marked this conversation as resolved.
Show resolved Hide resolved
$ oc delete -f ./charts/amq-operator-on-openshift.yaml --wait --timeout=10m0s
----

== Conclusion

Expand Down
Original file line number Diff line number Diff line change
@@ -0,0 +1,13 @@
# Subscribe the Red Hat Streams for Apache Kafka operator
apiVersion: operators.coreos.com/v1alpha1
kind: Subscription
metadata:
generation: 1
name: amq-streams
namespace: openshift-operators
spec:
channel: stable
installPlanApproval: Automatic
name: amq-streams
source: redhat-operators
sourceNamespace: openshift-marketplace
Original file line number Diff line number Diff line change
@@ -0,0 +1,35 @@
# This is the YAML needed to install Kafka provided by Strimzi on OpenShift.

# create a Kafka Stream instance
apiVersion: kafka.strimzi.io/v1beta2
kind: Kafka
metadata:
name: my-cluster
spec:
kafka:
replicas: 3
listeners:
- name: plain
port: 9092
type: internal
tls: false
storage:
type: ephemeral
zookeeper:
replicas: 3
storage:
type: ephemeral
entityOperator:
topicOperator: {}

---
# create a topic in Kafka Stream instance
apiVersion: kafka.strimzi.io/v1beta2
kind: KafkaTopic
metadata:
name: testing
labels:
strimzi.io/cluster: my-cluster
spec:
partitions: 3
replicas: 3
Original file line number Diff line number Diff line change
Expand Up @@ -4,56 +4,32 @@ The functionality of this quickstart depends on a running instance of the
https://access.redhat.com/products/red-hat-amq#streams[AMQ Streams] Operator. AMQ Streams is a Red Hat project based on Apache Kafka. To deploy AMQ Streams in the Openshift environment:

. Log in into the Openshift console as `kubeadmin` user (or any cluster administrator).
. Navigate to `Operators` -> `OperatorHub`.
. Search for `AMQ Streams` - click on the 'AMQ Streams' operator.
+
. Install the `Red Hat Streams for Apache Kafka` operator
. Create an instance of `Red Hat Streams for Apache Kafka`
. Create a topic in the `Red Hat Streams for Apache Kafka`

Install it with the default values and wait for the message telling you it has been installed and is ready for use.
. In your terminal, run the following command to set up a Kafka cluster called `my-cluster` in your project:
+

In your terminal, run the following command to subscribe the `Red Hat Streams for Apache Kafka` operator.
[options="nowrap",subs="+attributes"]
----
$ oc apply -f - <<EOF
apiVersion: kafka.strimzi.io/{strimzi-version}
kind: Kafka
metadata:
name: my-cluster
spec:
kafka:
replicas: 3
listeners:
- name: plain
port: 9092
type: internal
tls: false
storage:
type: ephemeral
zookeeper:
replicas: 3
storage:
type: ephemeral
entityOperator:
topicOperator: {}
EOF
$ oc apply -f ./charts/amq-operator-on-openshift.yaml --wait --timeout=10m0s
----

NOTE: If you see errors along the lines of _no matches for kind "Kafka" in version "kafka.strimzi.io/{strimzi-version}"_, execute the command `oc get crd kafkas.kafka.strimzi.io -o jsonpath="{.spec.versions[*].name}"` and update `apiVersion` to the returned version.
To verify the operator is ready to use, you can run the following command. You should see the following output.
The `PHASE` needs to be `Succeeded`
[options="nowrap",subs="+attributes"]
----
$ oc get ClusterServiceVersion

NAME DISPLAY VERSION REPLACES PHASE
amqstreams.v2.7.0-1 Red Hat Streams for Apache Kafka 2.7.0-1 amqstreams.v2.7.0-0 Succeeded
----

. Next set up a topic called `testing` in the `my-cluster` cluster we created:
+
Then you can set up a Kafka cluster called `my-cluster` with topic `testing` in your project:
[options="nowrap",subs="+attributes"]
----
oc apply -f - <<EOF
apiVersion: kafka.strimzi.io/{strimzi-version}
kind: KafkaTopic
metadata:
name: testing
labels:
strimzi.io/cluster: my-cluster
spec:
partitions: 3
replicas: 3
EOF
$ oc apply -f ./charts/kafka-on-openshift.yaml --wait --timeout=10m0s
----

Although the above commands will return pretty immediately, your AMQ Streams instance will not be available until its entity operator is up and running. The name of the pod will be of the format `my-cluster-entity-operator-xxxxxxxxx-yyyyy`.
Expand All @@ -62,6 +38,7 @@ To be on the safe side, wait until this pod is ready, as shown in this example:
[options="nowrap",subs="+attributes"]
----
oc get pods -w

NAME READY STATUS RESTARTS AGE
my-cluster-zookeeper-0 0/1 Running 0 29s
...
Expand Down
Loading