Not able to generate high load

[orzilca]

Hi, great tool!

Having said that, there are some issues to be solved :)

• it seems like its impossible to generate heavy load. no matter which setting I try (clients, duration, doc size), the maximum load of docs/second I can reach is 2000.
  i have tried setting the clients count to big numbers such as 5000, but I dont see indexing rate changes on Marvel (its never passes to 2000 docs/s).
  maybe I am missing something?
• Missing testing of non-bulk indexing. (es.index(...))

[orzilca]

Actually it seems like I'm not able generate load of more than 100 docs/s.
It shows 2000 because I use 1 replica.

[roiravhon]

Wierd, i was always able to generate load with this.
But this tool was written a long time ago, and frankly, for real load testing we are using this tool which we also wrote - https://github.com/logzio/elasticsearch-benchmark-tool
As it is a lot more comprehensive and can generate more "real-life" scenarios

[roiravhon]

Which version of ES are you testing it against?

[orzilca]

6.3.1

I'll take a look at the new version.
There's also Rally by elastic which seems like a good benchmarking tool.

[roiravhon]

Yep, also a good tool. It was just too comprehensive and complicated for our benchmark needs.
This tool was written for ES 2.x, so might need some adjustments for 6.x needs :)