Monitor Spring Kafka Applications with Kpow

Integrated Confluent Order Service Spring Kafka example application with the Kpow Streams Agent.

Run this project with the instructions below, we have integrated the Kpow Agent. You will see log-lines like:

Kpow: sent [370] streams metrics for application.id OrdersService

Once started, run Kpow with the target cluster and navigate to 'Streams' to view the live topology and metrics.

Quickstart

1. Start a 3-Node Kafka Cluster and Kpow with Kpow Local.
2. Build the Order Service JAR with make test
3. Run the Order Service JAR with java -jar build/libs/orders-service-10.0.8.jar
4. Navigate to localhost:3000 > Streams > OrdersService (can take 1-2 minutes to appear)

How We Integrated WordCount Streams with the Kpow Agent

Get the Kpow Streams Dependency

Include the Kpow Streams Agent library in your application:

implementation 'io.operatr:kpow-streams-agent:0.2.8'

Integrate the Agent

We define a new Component that adds an event-listener to the StreamsBuilderFactoryBean, registering the Streams and Topology once the Kafka Streams application has been started.

package io.confluent.kafkadevops.microservicesorders.ordersservice;

import io.operatr.kpow.StreamsRegistry;
import org.apache.kafka.streams.KafkaStreams;
import org.apache.kafka.streams.Topology;
import org.springframework.beans.factory.annotation.Autowired;
import org.springframework.kafka.config.StreamsBuilderFactoryBean;
import org.springframework.lang.NonNull;
import org.springframework.stereotype.Component;

@Component
public class KpowAgentService {

  @Autowired
  public KpowAgentService(final StreamsBuilderFactoryBean factory) {

    factory.addListener(new StreamsBuilderFactoryBean.Listener() {
      @Override
      public void streamsAdded(@NonNull String id, @NonNull KafkaStreams streams) {
        Topology topology = factory.getTopology();

        // Create a Kpow StreamsRegistry connecting to the same Kafka Cluster as Kafka Streams.
        // Note: In a multi-cluster Kpow setup the StreamsRegistry must be configured with your Primary cluster.
        StreamsRegistry registry = new StreamsRegistry(factory.getStreamsConfiguration());

        // Register your KafkaStreams and Topology instances with the StreamsRegistry
        registry.register(streams, topology);
      }
    });
  }
}

---

Original Project Readme Follows

Orders Service

This is a re-write of the OrdersService found in the Confluent Microservices Demos examples using the Spring Framework.

Building

The project contains a Makefile for managing build and package.

make clean	will clean build artifacts
make test	will run provide unit and integration tests
make build	will build a single shadow jar for running the application
make package	will build a Docker image
make publish	will build and publish the Docker image to the cnfldemos Confluent Docker Hub repository (proper credentials required)

Configuring

This application builds on the Spring Framework and uses Spring Application Property Files.

Configuration can be overridden using many methods.