Event Grid Benchmarking Tools

This repository contains the source code for the two tools needed to do end-to-end benchmarking of eventgrid and observe performance metrics. (1) an eventgrid load generator, and (2) a webhook sink.

Both of them have basic instrumentation to report success/failure rates, request latencies, end to end latencies. Right now reporting can happen either to the console (default) or to Azure AppInsights.

Latency calculation is pretty pedestrian and only manages to report the average latencies (no percentiles/mins/maxes/medians/coordinated omission handling).

These tools are shipped as (cross-platform) docker container images and run anywhere .net core runs.

See the packages section of this repo to get details on this docker image.

To generate load:

Usage: dotnet egbench.dll publisher start [options]

Options:
  -u|--topic-url                              REQUIRED. URL to which events should be posted to. Can specify multiple destinations like -u "http://..." -u
                                              "http://..." -u "http://...". Make sure number of URLs and number of topic names (-n) match up.
  -n|--topic-name                             REQUIRED. String that should be used for stamping eventgrid event envelope's Topic field or cloud event envelope's
                                              source field. Can specify multiple values. Each value corresponds in order to the topic url.
  -s|--topic-schema                           Defaults to CloudEvent10. Possible values: EventGrid / CloudEvent10 / Custom. Specify the -d|--data-payload
                                              property for Custom topic schema.
  -d|--data-payload                           Specify the data payload when -s|--topic-schema=Custom. Either give inline json or a file path.
  -p|--publishers                             Number of concurrent publishing "threads" per topic, defaults to 100. Each publisher sends to all the topic
                                              urls/names specified in -u / -n.
  -r|--rps-per-publisher                      Requests per second generated by each publish "thread", defaults to 1. Must be between 1 and 1000, inclusive.
  -c|--max-concurrent-requests-per-publisher  Max requests kept in flight by each publisher before waiting for older requests to end, defaults to 10.
  -e|--events-per-request                     Number of events in each request, defaults to 1.
  -b|--event-size-in-bytes                    Number of bytes per event, defaults to 1024, doesn't take effect if -s|--topic-schema==Custom. Total request
                                              payload size = 2 + (EventsPerRequest * (EventSizeInBytes + 1) - 1)
  -t|--runtime-in-minutes                     Time after which the publisher auto-shuts down, defaults to 0 minutes. Set to 0 to run forever.
  -v|--protocol-version                       The protocol version to use, defaults to 1.1.
  --skip-ssl-validation                       Skip SSL Server Certificate validation, defaults to false.
  --log-errors                                Log Status code, reason, and response content of all non-200 responses. Defaults to false.
  --total-requests                            Total number of requests to be made at the configured rate. When done, the publisher process will exit. Defaults
                                              to 0.
  -?|-h|--help                                Show help information.
  --runtag                                    Used as a context for metrics reports. Defaults to EGBench.
  --app-insights-key                          Azure Application Insights key. If null, metrics are written to console in influxdb single lineformat.
  --metrics-interval-seconds                  Frequency of reporting metrics out to console/azmonitor. Defaults to 60

To setup a listener:

Usage: dotnet egbench.dll subscriber start [options]

Options:
  -p|--port                   REQUIRED. Port on which to listen
  -t|--runtime-in-minutes     Time after which the subscriber auto-shuts down, defaults to 0 minutes. Set to 0 to run forever.
  -m|--mean-delay-ms          Fixed subscriber delay (in milliseconds). Defaults to 0. If stdDevDelayInMs is specified, delays are generated via a
                              normal/gaussian distribution, Specify the mean of the distribution here.
  -r|--return-code            HTTP Status code to be returned, formatted as (%,HttpCode). Defaults to 100% HTTP 200. For instance ... -r "10,400" -r "90:200"
                              would result in 10% HTTP 400 responses and 90% HTTP 200. All entries must sum to 100%. Valid separators: , : ; | _ <space>
  --log-payloads              Log all the payloads received, defaults to false.
  --event-time-property-name  Defaults to time (for Cloud Event Schema 1.0). Use 'eventTime' for EventGrid schema.
  -?|-h|--help                Show help information.
  --runtag                    Used as a context for metrics reports. Defaults to EGBench.
  --app-insights-key          Azure Application Insights key. If null, metrics are written to console in influxdb single lineformat.
  --metrics-interval-seconds  Frequency of reporting metrics out to console/azmonitor. Defaults to 60