diff --git a/docs/datasource_api_userguide.md b/docs/datasource_api_userguide.md index cf95191b13..c9e4774c13 100644 --- a/docs/datasource_api_userguide.md +++ b/docs/datasource_api_userguide.md @@ -341,3 +341,48 @@ The full conversion metrics is as follows. | LONGBLOB | :white_check_mark: | :white_check_mark: | :white_check_mark: | :white_check_mark: | :white_check_mark: | :x: | :x: | :white_check_mark: | :x: | :x: | :x: | | ENUM | :white_check_mark: | :white_check_mark: | :white_check_mark: | :white_check_mark: | :white_check_mark: | :white_check_mark: | :white_check_mark: | :white_check_mark: | :x: | :x: | :x: | | SET | :x: | :x: | :x: | :x: | :x: | :x: | :x: | :x: | :x: | :x: | :x: | + +## Write Benchmark +Tested on 4 machines as follows: + +``` +Intel(R) Xeon(R) CPU E5-2630 v4 @ 2.20GHz * 2 = 40Vu +12 * 16G = 188G +``` + +`FIO` test result: + +``` +WRITE: bw=705MiB/s (740MB/s), 705MiB/s-705MiB/s (740MB/s-740MB/s), io=20.0GiB (21.5GB), run=29034-29034msec +``` + +The table schema is: +``` +CREATE TABLE ORDERS (O_ORDERKEY INTEGER NOT NULL, + O_CUSTKEY INTEGER NOT NULL, + O_ORDERSTATUS CHAR(1) NOT NULL, + O_TOTALPRICE DECIMAL(15,2) NOT NULL, + O_ORDERDATE DATE NOT NULL, + O_ORDERPRIORITY CHAR(15) NOT NULL, + O_CLERK CHAR(15) NOT NULL, + O_SHIPPRIORITY INTEGER NOT NULL, + O_COMMENT VARCHAR(79) NOT NULL); + +``` + +### TiSpark Write Benchmark + +| count(*) | data size | parallel number | prepare(s) | prewrite (s) | commit (s) | total (s) | +| ----------- | --------- | --------------- | ---------- | ------------ | ---------- | --------- | +| 1,500,000 | 165M | 2 | 17 | 68 | 62 | 148 | +| 15,000,000 | 1.7G | 24 | 49 | 157 | 119 | 326 | +| 150,000,000 | 17G | 120 | 630 | 1236 | 1098 | 2964 | + + +## Spark with JDBC Benchmark + +| count(*) | data size | parallel number | spark jdbc write (s) | comments | +| ----------- | --------- | --------------- | -------------------- | ----------------------------------- | +| 1,500,000 | 165M | 24 | 22 | | +| 15,000,000 | 1.7G | 24 | 411 | use 120 parallel will cause KV Busy | +| 150,000,000 | 17G | 24 | 2936 | use 120 parallel will cause KV Busy |