Skip to content

Commit

Permalink
added plot_data.go to visualize read-write benchmark result
Browse files Browse the repository at this point in the history
Reasons:
1. The original plot_data.py is out of maintainance.
2. Remove the dependency on python and related python modules.
3. It isn't a best practice to compare benchmark results of two
   difference branches (e.g. main vs dev) in two charts. Instead,
   it's better to display the benchmarks to be compared in one chart.

Signed-off-by: Benjamin Wang <wachao@vmware.com>
  • Loading branch information
ahrtr committed Jan 12, 2023
1 parent 3306639 commit 2146477
Show file tree
Hide file tree
Showing 15 changed files with 2,180 additions and 309 deletions.
3 changes: 3 additions & 0 deletions .gitignore
Original file line number Diff line number Diff line change
Expand Up @@ -33,5 +33,8 @@ hack/tls-setup/certs
/tools/local-tester/bridge/bridge
/tools/proto-annotations/proto-annotations
/tools/benchmark/benchmark
/tools/rw-benchmark/rw-benchmark
/tools/rw-benchmark/rw_benchmark.html
/tools/rw-benchmark/*.csv
/out
/etcd-dump-logs
5 changes: 3 additions & 2 deletions CHANGELOG/CHANGELOG-3.6.md
Original file line number Diff line number Diff line change
Expand Up @@ -74,9 +74,10 @@ See [code changes](https://github.com/etcd-io/etcd/compare/v3.5.0...v3.6.0).
- Add [`etcd grpc-proxy start --endpoints-auto-sync-interval`](https://github.com/etcd-io/etcd/pull/14354) flag to enable and configure interval of auto sync of endpoints with server.
- Add [`etcd grpc-proxy start --listen-cipher-suites`](https://github.com/etcd-io/etcd/pull/14308) flag to support adding configurable cipher list.

### tools/benchmark
### tools

- [Add etcd client autoSync flag](https://github.com/etcd-io/etcd/pull/13416)
- [Add etcd client autoSync flag for benchmark](https://github.com/etcd-io/etcd/pull/13416)
- [Reimplement the rw-heatmaps using Golang and rename it to rw-benchmark](https://github.com/etcd-io/etcd/pull/15060)

### Metrics, Monitoring

Expand Down
4 changes: 4 additions & 0 deletions Makefile
Original file line number Diff line number Diff line change
Expand Up @@ -5,6 +5,10 @@ build:
./bin/etcdctl version
./bin/etcdutl version

.PHONY: tools
tools:
GO_BUILD_FLAGS="${GO_BUILD_FLAGS} -v" ./scripts/build_tools.sh

# Tests

GO_TEST_FLAGS?=
Expand Down
18 changes: 18 additions & 0 deletions bill-of-materials.json
Original file line number Diff line number Diff line change
Expand Up @@ -8,6 +8,15 @@
}
]
},
{
"project": "github.com/ahrtr/gocontainer",
"licenses": [
{
"type": "MIT License",
"confidence": 1
}
]
},
{
"project": "github.com/anishathalye/porcupine",
"licenses": [
Expand Down Expand Up @@ -125,6 +134,15 @@
}
]
},
{
"project": "github.com/go-echarts/go-echarts/v2",
"licenses": [
{
"type": "MIT License",
"confidence": 1
}
]
},
{
"project": "github.com/go-logr/logr",
"licenses": [
Expand Down
2 changes: 2 additions & 0 deletions go.mod
Original file line number Diff line number Diff line change
Expand Up @@ -15,10 +15,12 @@ replace (
)

require (
github.com/ahrtr/gocontainer v0.3.0
github.com/bgentry/speakeasy v0.1.0
github.com/cheggaaa/pb/v3 v3.1.0
github.com/coreos/go-semver v0.3.0
github.com/dustin/go-humanize v1.0.0
github.com/go-echarts/go-echarts/v2 v2.2.5-0.20211021024243-33ae1aa415d6
github.com/spf13/cobra v1.6.1
github.com/stretchr/testify v1.8.1
go.etcd.io/bbolt v1.3.6
Expand Down
5 changes: 5 additions & 0 deletions go.sum
Original file line number Diff line number Diff line change
Expand Up @@ -43,6 +43,8 @@ github.com/BurntSushi/xgb v0.0.0-20160522181843-27f122750802/go.mod h1:IVnqGOEym
github.com/OneOfOne/xxhash v1.2.2/go.mod h1:HSdplMjZKSmBqAxg5vPj2TmRDmfkzw+cTzAElWljhcU=
github.com/VividCortex/ewma v1.1.1 h1:MnEK4VOv6n0RSY4vtRe3h11qjxL3+t0B8yOL8iMXdcM=
github.com/VividCortex/ewma v1.1.1/go.mod h1:2Tkkvm3sRDVXaiyucHiACn4cqf7DpdyLvmxzcbUokwA=
github.com/ahrtr/gocontainer v0.3.0 h1:/4wM0VhaLEYZMoF6WT8ZHUmf2n9BVpCD3uMaKrA0iHY=
github.com/ahrtr/gocontainer v0.3.0/go.mod h1:cQoR5/JTMoDNEkk5vGaohPZ+nnTQVB2nk2Y012WJWsM=
github.com/alecthomas/template v0.0.0-20160405071501-a0175ee3bccc/go.mod h1:LOuyumcjzFXgccqObfd/Ljyb9UuFJ6TxHnclSeseNhc=
github.com/alecthomas/template v0.0.0-20190718012654-fb15b899a751/go.mod h1:LOuyumcjzFXgccqObfd/Ljyb9UuFJ6TxHnclSeseNhc=
github.com/alecthomas/units v0.0.0-20151022065526-2efee857e7cf/go.mod h1:ybxpYRFXyAe+OPACYpWeL0wqObRcbAqCMya13uyzqw0=
Expand Down Expand Up @@ -105,6 +107,8 @@ github.com/fatih/color v1.13.0 h1:8LOYc1KYPPmyKMuN8QV2DNRWNbLo6LZ0iLs8+mlH53w=
github.com/fatih/color v1.13.0/go.mod h1:kLAiJbzzSOZDVNGyDpeOxJ47H46qBXwg5ILebYFFOfk=
github.com/getsentry/raven-go v0.2.0 h1:no+xWJRb5ZI7eE8TWgIq1jLulQiIoLG0IfYxv5JYMGs=
github.com/ghodss/yaml v1.0.0/go.mod h1:4dBDuWmgqj2HViK6kFavaiC9ZROes6MMH2rRYeMEF04=
github.com/go-echarts/go-echarts/v2 v2.2.5-0.20211021024243-33ae1aa415d6 h1:+p0u+1svKoBC2xS6GzpmcDHShkAGqD+wUQLpxIpygM0=
github.com/go-echarts/go-echarts/v2 v2.2.5-0.20211021024243-33ae1aa415d6/go.mod h1:6TOomEztzGDVDkOSCFBq3ed7xOYfbOqhaBzD0YV771A=
github.com/go-gl/glfw v0.0.0-20190409004039-e6da0acd62b1/go.mod h1:vR7hzQXu2zJy9AVAgeJqvqgH9Q5CA+iKCZ2gyEVpxRU=
github.com/go-gl/glfw/v3.3/glfw v0.0.0-20191125211704-12ad95a8df72/go.mod h1:tQ2UAYgL5IevRw8kRxooKSPJfGvJ9fJQFa0TUsXzTg8=
github.com/go-gl/glfw/v3.3/glfw v0.0.0-20200222043503-6f7a984d4dc4/go.mod h1:tQ2UAYgL5IevRw8kRxooKSPJfGvJ9fJQFa0TUsXzTg8=
Expand Down Expand Up @@ -316,6 +320,7 @@ github.com/stretchr/testify v1.2.2/go.mod h1:a8OnRcib4nhh0OaRAV+Yts87kKdq0PP7pXf
github.com/stretchr/testify v1.3.0/go.mod h1:M5WIy9Dh21IEIfnGCwXGc5bZfKNJtfHm1UVUgZn+9EI=
github.com/stretchr/testify v1.4.0/go.mod h1:j7eGeouHqKxXV5pUuKE4zz7dFj8WfuZ+81PSLYec5m4=
github.com/stretchr/testify v1.5.1/go.mod h1:5W2xD1RspED5o8YsWQXVCued0rvSQ+mT+I5cxcmMvtA=
github.com/stretchr/testify v1.6.0/go.mod h1:6Fq8oRcR53rry900zMqJjRRixrwX3KX962/h/Wwjteg=
github.com/stretchr/testify v1.6.1/go.mod h1:6Fq8oRcR53rry900zMqJjRRixrwX3KX962/h/Wwjteg=
github.com/stretchr/testify v1.7.0/go.mod h1:6Fq8oRcR53rry900zMqJjRRixrwX3KX962/h/Wwjteg=
github.com/stretchr/testify v1.7.1/go.mod h1:6Fq8oRcR53rry900zMqJjRRixrwX3KX962/h/Wwjteg=
Expand Down
1 change: 1 addition & 0 deletions scripts/build_lib.sh
Original file line number Diff line number Diff line change
Expand Up @@ -64,6 +64,7 @@ tools_build() {
out="bin"
if [[ -n "${BINDIR}" ]]; then out="${BINDIR}"; fi
tools_path="tools/benchmark
tools/rw-benchmark
tools/etcd-dump-db
tools/etcd-dump-logs
tools/local-tester/bridge"
Expand Down
54 changes: 54 additions & 0 deletions tools/rw-benchmark/README.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,54 @@
# etcd/tools/rw-benchmark

`etcd/tools/rw-benchmark` is the mixed read/write performance evaluation tool for etcd clusters.

## Execute

### Benchmark
To get a mixed read/write performance evaluation result:
```sh
# run with default configurations and specify the working directory
./rw-benchmark.sh -w ${WORKING_DIR}
```
`rw-benchmark.sh` will automatically use the etcd binary compiled under `etcd/bin/tools` directory.

Note: the result csv file will be saved to current working directory. The working directory is where etcd database is saved. The working directory is designed for scenarios where a different mounted disk is preferred.

### Plot Graphs
The tool `rw-benchmark` can generate an HTML page including all the line charts based on the benchmark result csv files. See usage below,
```sh
$ ./rw-benchmark -h
rw-benchmark is a tool for visualize etcd read-write performance result.

Usage:
rw-benchmark [options] result-file1.csv [result-file2.csv]

Additional options:
-legend: Comma separated names of legends, such as "main,pr", defaults to "1" or "1,2" depending on the number of CSV files provided.
-layout: The layout of the page, valid values: none, center and flex, defaults to "flex".
-width: The width(pixel) of the each line chart, defaults to 600.
-height: The height(pixel) of the each line chart, defaults to 300.
-o: The HTML file name in which the benchmark data will be rendered, defaults to "rw_benchmark.html".
-h: Print usage.
```

See examples below,
```sh
# To generate a HTML page with each chart including one pair of read & write
# benchmark results from one data csv file.
./rw-benchmark ${FIRST_CSV_FILE}

# To generate a HTML page with each chart including two pair of read & write
# benchmark results from two data csv files respectively.
./rw-benchmark ${FIRST_CSV_FILE} ${SECOND_CSV_FILE}

# Set the legend to "main,dev"
./rw-benchmark -legend "main,dev" ${FIRST_CSV_FILE} ${SECOND_CSV_FILE}

# Set the width and height of each line chart to 800 and 400px respectively
./rw-benchmark -width 800 -height 400 ${FIRST_CSV_FILE} ${SECOND_CSV_FILE}
```

The read QPS is displayed as <span style="color:blue">blue</span>, and write QPS is displayed as <span style="color:red">red</span>.
The data in the second CSV file is rendered as dashed line if present. See example in [example/rw_benchmark.html](example/rw_benchmark.html).
Note each line in the line chart can be hidden or displayed by clicking on the related legend.
Loading

0 comments on commit 2146477

Please sign in to comment.