Home
The goal of the framework is to provide an extensible platform that can be used to benchmark and compare different Cloud Pub/Sub client libraries against each other and against other messaging services such as Kafka. This includes different language implementations as well as different APIs, therefore the flexibility and extensibility of the framework is critical.
The main runner of the load test framework is the Driver. It is run on the user’s local machine and configures the remote environment that will generate the load. We currently support the Compute Engine environment. The Driver is responsible for bringing up the jobs in the remote environment, and for performing health checks and basic statistic aggregation from the Clients. In the case of load tests, where we want to run multiple tests until we hit some threshold, the Driver is responsible for reading metrics from Stackdriver. The Driver communicates via gRPC with the Clients.
Each virtual machine runs a single Client. The Client runs a gRPC server that accepts Start and Check RPCs from the Driver. The Client is responsible for exporting statistics to Stackdriver, and controlling the number of concurrent requests and the QPS. The load test client receives a Start RPC from the Driver, after which it waits until the specified start time. It then responds to Check RPCs with responses that signal that the Client is healthy, along with providing the latency distributions and signal the Client has finished. Each publish or subscribe is performed by running a Task.
For Java clients, the Tasks implement Runnable and are called on each publish or subscribe. Each has a constructor that takes a StartRequest, and each overrides run() to perform their action (publishing a message, subscribing). Each is also responsible for the timing and reporting of its latencies to the Client. For clients in languages other than Java, an AdapterTask instead communicates via gRPC to a worker, and proxies latency information up from its Worker.
The worker is responsible for running a gRPC server that implements Start and Execute. Start is received from the AdapterTask, which proxies the Start Request for initialization. The ExecuteRequest tells the Worker to run an action a single time, and then responds with the latencies it took to run the action. (The Worker returns multiple latencies because a single Pull request will contain multiple and potentially different end-to-end latency measurements.)