Skip to content

Commit

Permalink
added plot_data.go to visualize read-write benchmark result
Browse files Browse the repository at this point in the history
Reasons:
1. The original plot_data.py is out of maintainance.
2. Remove the dependency on python and related python modules.
3. It isn't a best practice to compare benchmark results of two
   difference branches (e.g. main vs dev) in two charts. Instead,
   it's better to display the benchmarks to be compared in one chart.

Signed-off-by: Benjamin Wang <wachao@vmware.com>
  • Loading branch information
ahrtr committed Jan 12, 2023
1 parent 3306639 commit 9c27b86
Show file tree
Hide file tree
Showing 15 changed files with 2,124 additions and 309 deletions.
3 changes: 3 additions & 0 deletions .gitignore
Original file line number Diff line number Diff line change
Expand Up @@ -33,5 +33,8 @@ hack/tls-setup/certs
/tools/local-tester/bridge/bridge
/tools/proto-annotations/proto-annotations
/tools/benchmark/benchmark
/tools/rw-benchmark/rw-benchmark
/tools/rw-benchmark/rw_benchmark.html
/tools/rw-benchmark/*.csv
/out
/etcd-dump-logs
5 changes: 3 additions & 2 deletions CHANGELOG/CHANGELOG-3.6.md
Original file line number Diff line number Diff line change
Expand Up @@ -74,9 +74,10 @@ See [code changes](https://github.com/etcd-io/etcd/compare/v3.5.0...v3.6.0).
- Add [`etcd grpc-proxy start --endpoints-auto-sync-interval`](https://github.com/etcd-io/etcd/pull/14354) flag to enable and configure interval of auto sync of endpoints with server.
- Add [`etcd grpc-proxy start --listen-cipher-suites`](https://github.com/etcd-io/etcd/pull/14308) flag to support adding configurable cipher list.

### tools/benchmark
### tools

- [Add etcd client autoSync flag](https://github.com/etcd-io/etcd/pull/13416)
- [Add etcd client autoSync flag for benchmark](https://github.com/etcd-io/etcd/pull/13416)
- [Reimplement the rw-heatmaps using Golang and rename it to rw-benchmark](https://github.com/etcd-io/etcd/pull/15060)

### Metrics, Monitoring

Expand Down
4 changes: 4 additions & 0 deletions Makefile
Original file line number Diff line number Diff line change
Expand Up @@ -5,6 +5,10 @@ build:
./bin/etcdctl version
./bin/etcdutl version

.PHONY: tools
tools:
GO_BUILD_FLAGS="${GO_BUILD_FLAGS} -v" ./scripts/build_tools.sh

# Tests

GO_TEST_FLAGS?=
Expand Down
18 changes: 18 additions & 0 deletions bill-of-materials.json
Original file line number Diff line number Diff line change
Expand Up @@ -8,6 +8,15 @@
}
]
},
{
"project": "github.com/ahrtr/gocontainer",
"licenses": [
{
"type": "MIT License",
"confidence": 1
}
]
},
{
"project": "github.com/anishathalye/porcupine",
"licenses": [
Expand Down Expand Up @@ -125,6 +134,15 @@
}
]
},
{
"project": "github.com/go-echarts/go-echarts/v2",
"licenses": [
{
"type": "MIT License",
"confidence": 1
}
]
},
{
"project": "github.com/go-logr/logr",
"licenses": [
Expand Down
2 changes: 2 additions & 0 deletions go.mod
Original file line number Diff line number Diff line change
Expand Up @@ -15,10 +15,12 @@ replace (
)

require (
github.com/ahrtr/gocontainer v0.3.0
github.com/bgentry/speakeasy v0.1.0
github.com/cheggaaa/pb/v3 v3.1.0
github.com/coreos/go-semver v0.3.0
github.com/dustin/go-humanize v1.0.0
github.com/go-echarts/go-echarts/v2 v2.2.5-0.20211021024243-33ae1aa415d6
github.com/spf13/cobra v1.6.1
github.com/stretchr/testify v1.8.1
go.etcd.io/bbolt v1.3.6
Expand Down
5 changes: 5 additions & 0 deletions go.sum
Original file line number Diff line number Diff line change
Expand Up @@ -43,6 +43,8 @@ github.com/BurntSushi/xgb v0.0.0-20160522181843-27f122750802/go.mod h1:IVnqGOEym
github.com/OneOfOne/xxhash v1.2.2/go.mod h1:HSdplMjZKSmBqAxg5vPj2TmRDmfkzw+cTzAElWljhcU=
github.com/VividCortex/ewma v1.1.1 h1:MnEK4VOv6n0RSY4vtRe3h11qjxL3+t0B8yOL8iMXdcM=
github.com/VividCortex/ewma v1.1.1/go.mod h1:2Tkkvm3sRDVXaiyucHiACn4cqf7DpdyLvmxzcbUokwA=
github.com/ahrtr/gocontainer v0.3.0 h1:/4wM0VhaLEYZMoF6WT8ZHUmf2n9BVpCD3uMaKrA0iHY=
github.com/ahrtr/gocontainer v0.3.0/go.mod h1:cQoR5/JTMoDNEkk5vGaohPZ+nnTQVB2nk2Y012WJWsM=
github.com/alecthomas/template v0.0.0-20160405071501-a0175ee3bccc/go.mod h1:LOuyumcjzFXgccqObfd/Ljyb9UuFJ6TxHnclSeseNhc=
github.com/alecthomas/template v0.0.0-20190718012654-fb15b899a751/go.mod h1:LOuyumcjzFXgccqObfd/Ljyb9UuFJ6TxHnclSeseNhc=
github.com/alecthomas/units v0.0.0-20151022065526-2efee857e7cf/go.mod h1:ybxpYRFXyAe+OPACYpWeL0wqObRcbAqCMya13uyzqw0=
Expand Down Expand Up @@ -105,6 +107,8 @@ github.com/fatih/color v1.13.0 h1:8LOYc1KYPPmyKMuN8QV2DNRWNbLo6LZ0iLs8+mlH53w=
github.com/fatih/color v1.13.0/go.mod h1:kLAiJbzzSOZDVNGyDpeOxJ47H46qBXwg5ILebYFFOfk=
github.com/getsentry/raven-go v0.2.0 h1:no+xWJRb5ZI7eE8TWgIq1jLulQiIoLG0IfYxv5JYMGs=
github.com/ghodss/yaml v1.0.0/go.mod h1:4dBDuWmgqj2HViK6kFavaiC9ZROes6MMH2rRYeMEF04=
github.com/go-echarts/go-echarts/v2 v2.2.5-0.20211021024243-33ae1aa415d6 h1:+p0u+1svKoBC2xS6GzpmcDHShkAGqD+wUQLpxIpygM0=
github.com/go-echarts/go-echarts/v2 v2.2.5-0.20211021024243-33ae1aa415d6/go.mod h1:6TOomEztzGDVDkOSCFBq3ed7xOYfbOqhaBzD0YV771A=
github.com/go-gl/glfw v0.0.0-20190409004039-e6da0acd62b1/go.mod h1:vR7hzQXu2zJy9AVAgeJqvqgH9Q5CA+iKCZ2gyEVpxRU=
github.com/go-gl/glfw/v3.3/glfw v0.0.0-20191125211704-12ad95a8df72/go.mod h1:tQ2UAYgL5IevRw8kRxooKSPJfGvJ9fJQFa0TUsXzTg8=
github.com/go-gl/glfw/v3.3/glfw v0.0.0-20200222043503-6f7a984d4dc4/go.mod h1:tQ2UAYgL5IevRw8kRxooKSPJfGvJ9fJQFa0TUsXzTg8=
Expand Down Expand Up @@ -316,6 +320,7 @@ github.com/stretchr/testify v1.2.2/go.mod h1:a8OnRcib4nhh0OaRAV+Yts87kKdq0PP7pXf
github.com/stretchr/testify v1.3.0/go.mod h1:M5WIy9Dh21IEIfnGCwXGc5bZfKNJtfHm1UVUgZn+9EI=
github.com/stretchr/testify v1.4.0/go.mod h1:j7eGeouHqKxXV5pUuKE4zz7dFj8WfuZ+81PSLYec5m4=
github.com/stretchr/testify v1.5.1/go.mod h1:5W2xD1RspED5o8YsWQXVCued0rvSQ+mT+I5cxcmMvtA=
github.com/stretchr/testify v1.6.0/go.mod h1:6Fq8oRcR53rry900zMqJjRRixrwX3KX962/h/Wwjteg=
github.com/stretchr/testify v1.6.1/go.mod h1:6Fq8oRcR53rry900zMqJjRRixrwX3KX962/h/Wwjteg=
github.com/stretchr/testify v1.7.0/go.mod h1:6Fq8oRcR53rry900zMqJjRRixrwX3KX962/h/Wwjteg=
github.com/stretchr/testify v1.7.1/go.mod h1:6Fq8oRcR53rry900zMqJjRRixrwX3KX962/h/Wwjteg=
Expand Down
1 change: 1 addition & 0 deletions scripts/build_lib.sh
Original file line number Diff line number Diff line change
Expand Up @@ -64,6 +64,7 @@ tools_build() {
out="bin"
if [[ -n "${BINDIR}" ]]; then out="${BINDIR}"; fi
tools_path="tools/benchmark
tools/rw-benchmark
tools/etcd-dump-db
tools/etcd-dump-logs
tools/local-tester/bridge"
Expand Down
31 changes: 31 additions & 0 deletions tools/rw-benchmark/README.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,31 @@
# etcd/tools/rw-benchmark

`etcd/tools/rw-benchmark` is the mixed read/write performance evaluation tool for etcd clusters.

## Execute

### Benchmark
To get a mixed read/write performance evaluation result:
```sh
# run with default configurations and specify the working directory
./rw-benchmark.sh -w ${WORKING_DIR}
```
`rw-benchmark.sh` will automatically use the etcd binary compiled under `etcd/bin/tools` directory.

Note: the result csv file will be saved to current working directory. The working directory is where etcd database is saved. The working directory is designed for scenarios where a different mounted disk is preferred.

### Plot Graphs
To generate a HTML page including all the line charts based on the benchmark result csv files:
```sh
# To generate a HTML page with each chart including one pair of read & write
# benchmark results from one data csv file.
./rw-benchmark -l "main" ${FIRST_CSV_FILE}

# To generate a HTML page with each chart including two pair of read & write
# benchmark results from two data csv files respectively.
./rw-benchmark -l "main:dev" ${FIRST_CSV_FILE} ${SECOND_CSV_FILE}
```

The read QPS is displayed as <span style="color:blue">blue</span>, and write QPS is displayed as <span style="color:red">red</span>.
The data in the second CSV file is rendered as dashed line if present. See example in [example/rw_benchmark.html](example/rw_benchmark.html).
Note each line in the line chart can be hidden or displayed by clicking on the related legend.
Loading

0 comments on commit 9c27b86

Please sign in to comment.