Decrease amount of transfered data with aggregation by Path

[Felixoid]

Here is a code for #60

With our 10Gbit network, there is no significant acceleration. But the amount of transferred data is decreased 6 times:

# upstream
[2019-09-02T06:39:34.284Z] INFO finder {"request_id": "6221247fed568342ef8eb6dc365588d5", "metrics": 25836}
[2019-09-02T06:39:49.508Z] INFO [query] query {"query": "SELECT Path, Time, Value, Timestamp FROM graphite.carbon PREWHERE Date >='2019-08-30' AND Date <= '2019-08-30' WHERE (Path IN (<...>)) AND (Time >= 1567134864 AND Time <= 1567178099) FORMAT RowBinary", "request_id": "6221247fed568342ef8eb6dc365588d5", "time": 14.247405276}
[2019-09-02T06:39:49.508Z] INFO render {"request_id": "6221247fed568342ef8eb6dc365588d5", "read_bytes": 1893367324, "read_points": 18549891}
[2019-09-02T06:39:49.508Z] DEBUG parse {"request_id": "6221247fed568342ef8eb6dc365588d5", "runtime": "9.496039402s", "runtime_ns": 9.496039402}
[2019-09-02T06:39:53.068Z] DEBUG sort {"request_id": "6221247fed568342ef8eb6dc365588d5", "runtime": "3.560176051s", "runtime_ns": 3.560176051}
[2019-09-02T06:39:59.478Z] DEBUG rollup {"request_id": "6221247fed568342ef8eb6dc365588d5", "runtime": "1.401853138s", "runtime_ns": 1.401853138}
[2019-09-02T06:39:59.478Z] DEBUG pickle {"request_id": "6221247fed568342ef8eb6dc365588d5", "runtime": "4.645392012s", "runtime_ns": 4.645392012}
[2019-09-02T06:39:59.478Z] DEBUG reply {"request_id": "6221247fed568342ef8eb6dc365588d5", "runtime": "6.306056204s", "runtime_ns": 6.306056204}
[2019-09-02T06:39:59.478Z] INFO access {"request_id": "6221247fed568342ef8eb6dc365588d5", "time": 25.38388494, "method": "GET", "url": "/render/?format=pickle&local=1&noCache=1&from=1567134864&until=1567178064&target=<...>&now=1567178064", "peer": "127.0.0.1:60430", "status": 200}


# with patch
[2019-09-02T06:39:05.454Z] INFO finder {"request_id": "2d90045ec404d7f057753948489015d3", "metrics": 25836}
[2019-09-02T06:39:20.845Z] INFO [query] query {"query": "SELECT Path, groupArray(Time), groupArray(Value), groupArray(Timestamp) FROM graphite.carbon PREWHERE Date >='2019-08-30' AND Date <= '2019-08-30' WHERE (Path IN (<...>)) AND (Time >= 1567134864 AND Time <= 1567178099) GROUP BY Path FORMAT RowBinary", "request_id": "2d90045ec404d7f057753948489015d3", "time": 14.383664671}
[2019-09-02T06:39:20.845Z] INFO render {"request_id": "2d90045ec404d7f057753948489015d3", "read_bytes": 299176954, "read_points": 18549891}
[2019-09-02T06:39:20.845Z] DEBUG parse {"request_id": "2d90045ec404d7f057753948489015d3", "runtime": "5.312247309s", "runtime_ns": 5.312247309}
[2019-09-02T06:39:23.788Z] DEBUG sort {"request_id": "2d90045ec404d7f057753948489015d3", "runtime": "2.942685954s", "runtime_ns": 2.942685954}
[2019-09-02T06:39:30.036Z] DEBUG rollup {"request_id": "2d90045ec404d7f057753948489015d3", "runtime": "1.304685963s", "runtime_ns": 1.304685963}
[2019-09-02T06:39:30.036Z] DEBUG pickle {"request_id": "2d90045ec404d7f057753948489015d3", "runtime": "4.461564452s", "runtime_ns": 4.461564452}
[2019-09-02T06:39:30.036Z] DEBUG reply {"request_id": "2d90045ec404d7f057753948489015d3", "runtime": "5.991442156s", "runtime_ns": 5.991442156}
[2019-09-02T06:39:30.036Z] INFO access {"request_id": "2d90045ec404d7f057753948489015d3", "time": 24.759430571, "method": "GET", "url": "/render/?format=pickle&local=1&noCache=1&from=1567134864&until=1567178064&target=<...>&now=1567178064", "peer": "127.0.0.1:42114", "status": 200}

[Felixoid]

Oh no, there is still failing tests. I'm working on it now, will send new pr soon

…

On Mon, 2 Sep 2019, 11:58 Roman Lomonosov, ***@***.***> wrote: Merged #61 <#61> into master. — You are receiving this because you authored the thread. Reply to this email directly, view it on GitHub <#61>, or mute the thread <https://github.com/notifications/unsubscribe-auth/AAXCVAJTIG4GJC7VUH5R5H3QHTIT7ANCNFSM4IS2FNGA> .