-
Notifications
You must be signed in to change notification settings - Fork 98
Home
This page has a very quick overview of ducktape.
For more details on how to install and use, see the readme: https://github.com/confluentinc/ducktape
ducktape is a lightweight tool written in python which helps with writing and running system tests. It intentionally looks a lot like a *unit framework, but is designed to work well for system tests. A lot of that just boils down to providing a bunch of utilities and some structure that makes writing and running those tests easy.
The core of the design is just that you get simple SSH access to machines in a cluster (which the framework doles out) and the framework has support for "services" in addition to tests. The services are intended to be reusable components that have commonly used functionality, e.g. for kafka tests we wrote zookeeper and kafka services which make it easy to bring up the entire cluster in one line, have extra functionality like getting leaders for topics, bouncing individual nodes, etc.
Beyond that, the table on https://cwiki.apache.org/confluence/display/KAFKA/System+Test+Improvements probably has the most relevant listing of features -- nothing ground breaking, but a few of them are particularly important for a workable solution for kafka (running on different types of infrastructure, automatable, and the promise of being parallelizable).
Cluster
Cluster is a pluggable interface representing a pool of nodes that can be allocated (or deallocated) to services on demand. Currently in our system tests we use the VagrantCluster implementation, which uses Vagrant to manage a group of virtual machines for us. The nice thing about VagrantCluster is that the virtual machines can be run locally or on AWS. Other cluster implementations to suit other setups are possible.
Service
A service is an abstraction for (possibly distributed) services which can run on one or more nodes. It has a few overrideable lifecycle methods which the ducktape framework uses during automatic setup and teardown. For example, here is the KafkaService: https://github.com/confluentinc/muckrake/blob/master/muckrake/services/kafka.py
Test
A test is a method that can be discovered and run by the ducktape command-line tool. Currently, these are methods named "test." or ".test" on a leaf subclass of Test.
Most test examples are in the muckrake repository:
Here is an example of a group of Kafka performance tests:
This group of tests fails Kafka nodes in various ways while schemas are registered to the schema registry in a background thread:
Ducktape repo:
Muckrake (Confluent system tests):
KIP for developing Kafka system tests with ducktape:
This has a table comparing ducktape's features agains another system we considered called Zopkio: