Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

HAProxy: gauge missing statuses with zero #940

Merged
merged 8 commits into from
May 19, 2014

Conversation

tmichelet
Copy link
Contributor

About count_per_status:
For each service, we currently only gauge the current status of the service.
This means that if the service was previously down and is up again, we are going to gauge(count, [up]) but not gauge(0, [down]).
Since on the UI the missing data is regressed, we can see this kind of bug:
screen shot 2014-05-13 at 11 48 55 am
The second chart keeps displaying 2, instead of displaying 0 where the value is missing.

This PR is about solving that, by always gauging each status (either count or 0, depending on the current status).
Pros: avoid these bugs
Cons: more data pushed, charts with only zeros appear

New charts:
screen shot 2014-05-13 at 11 47 18 am

screen shot 2014-05-13 at 11 47 27 am

@@ -19,6 +19,10 @@ class Services(object):
BACKEND = 'BACKEND'
FRONTEND = 'FRONTEND'
ALL = (BACKEND, FRONTEND)
ALL_STATUSES = (
'up', 'up_1/3', 'up_2/3', 'open', 'no_check', 'down', 'down_1/2',
Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

do not consider up_x/y nor down_x/y

@tmichelet
Copy link
Contributor Author

  • may want to change the default dogweb haproxy dashboard, and remove count_per_status feature flag (since it's very useful)
  • may not want to send events regarding aggregated lines (BACKEND)

@tmichelet
Copy link
Contributor Author

might want to remove "front-end" from the event name, that's confusing

@tmichelet
Copy link
Contributor Author

  • updated events titles
    screen shot 2014-05-16 at 12 22 09 pm
  • Process events and host metrics only for expected services (don't process BACKEND if not aggregates only)
    screen shot 2014-05-16 at 12 22 28 pm

@tmichelet
Copy link
Contributor Author

@remh should we default count_per_status to True?

@conorbranagan
Copy link
Member

Do we need to have "HAProxy" in the event title? Seems a bit redundant with the haproxy logo right next to it

@tmichelet
Copy link
Contributor Author

good point, I'll remove it

# Send the list of data to the metric and event callbacks
self._process_metrics(data_dict, url)
if process_events:
self._process_event(data_dict, url)
if process_events:
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Is that intentional ?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

yes, I thought that we don't want to have events on aggregates if we use the detail otherwise, and vice versa

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

this way:

  • use aggregates -> metrics and events for the aggregates
  • don't use aggregates -> metrics and events detailed, without the noise of the aggregates

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Great

@remh
Copy link
Contributor

remh commented May 19, 2014

Thanks @tmichelet !

Looks like the tests are failing though! Could you investigate and fix ?
Then feel free to merge!

tmichelet added a commit that referenced this pull request May 19, 2014
HAProxy: gauge missing statuses with zero
@tmichelet tmichelet merged commit 5045ea0 into master May 19, 2014
@tmichelet tmichelet deleted the tristan/haproxy-count-per-status branch May 19, 2014 20:01
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants