Metricbeat overview dashboard for Zookeeper

[sayden]

I realized that we didn't have any dashboard for Zookeeper so I took advantage that I had all setup done to build one.

Just very key metrics were included, a total of 5 charts using 7 metrics:

• Alive connections
• Percentaje between open file descriptors and max file descriptors
• Latency
• Approximate data size
• Packets received / sent

[kaiyan-sheng]

@sayden just curious :-) why the alive connections diagram is not just lines connecting each data point(like the packet received/sent)? What does the bottom green part mean?

[ruflin]

Great to see a dashboard for zookeeper. Seems like this needs a make update.

[ruflin]

The json file is in the wrong directory. I moved it locally for testing and then everything loads as expected and graphs are shown.

Could you also add the screenshot you have above to the module docs like we do in other modules?

[ruflin]

This file belongs into the dashboard directory inside 7, otherwise import does not work.

[sayden]

@sayden just curious :-) why the alive connections diagram is not just lines connecting each data point(like the packet received/sent)? What does the bottom green part mean?

When there's a single value, using an Area chart instead of a line chart make it looks a bit more dynamically beautiful 😄 Some charting libraries and products uses Area charts by default because of this.

When using 2 values like packets sent / received, this is not an option.

[jsoriano]

@sayden awesome dashboard. Only a couple of small suggestions.
You may want to rearrange the graphs to put three graphs in a row so there is not a third row with a single graph.
For the file descriptors visualization, would it be possible to show the percentage in left axis and the absolute number in the right one? It can be useful to know the exact number of descriptors used.

[ruflin]

LGTM. My focus on testing was to check if dashboards are loaded correctly and if some data shows up which works.

[sayden]

For the file descriptors visualization, would it be possible to show the percentage in left axis and the absolute number in the right one? It can be useful to know the exact number of descriptors used.

@jsoriano As I wrote in the first message, the percentaje of file descriptors is a ratio between the max file descriptors and the used ones. If you want, for example, used file descriptors on the right axis, it is just an absolute number with no real meaning because you'll have to rely on the percentaje anyways. My question is: isn't it going to be too much information? I mean that I even did the visualization for the used file descriptors but then I realized that to see 1000 or 100000 wasn't very useful without knowing the max file descriptors. Just to clarify, Zookeeper only return max and used file descriptors.

[sayden]

This is what I mean: 26k file descriptors.... oh! that's a lot! Actually only a 0.003% of max configured 😄

[jsoriano]

it is just an absolute number with no real meaning because you'll have to rely on the percentage anyways.

For this very reason any other value in this dashboard would have no real meaning 😃

Percentages are useful for capacity reasons, agree, but absolute values are also useful for other things, e.g. to identify problems like "with X file descriptors or Y active connections average latency increases", this is harder to do with percentages. Also think for example that you want to reduce the maximum number of file descriptors, it is also easier to decide a proper value if you have absolute values, and it is also easier to compare two different configurations if you have the absolute values.

This is what I mean: 26k file descriptors.... oh! that's a lot! Actually only a 0.003% of max configured smile

26k in an idle zookeeper? is this value correct? what does zookeeper do so many file descriptors? 🤔

And please start both scales from zero.

[jsoriano]

Oh, and a cosmetic detail, please make the three graphs in the second row to have the same width.

[sayden]

I have adjusted the size and set the number format to 2 decimal places (by default it was 3, that's why I thought it had 26.000 opened file descriptors)

I didn't understand what do you mean with And please start both scales from zero. They are automatically scaled, do you want to fix the lower bounds to zero?

[kaiyan-sheng]

looks great 👍

[ruflin]

The dashboard screenshots keep getting nicer and nicer. @sayden Can you make sure that the screenshot used in the docs is from the "final" dashboard?

[jsoriano]

I didn't understand what do you mean with And please start both scales from zero. They are automatically scaled, do you want to fix the lower bounds to zero?

Yes, please.

[sayden]

Updated screenshot with changes addressed :)

[jsoriano]

Great 👍