Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Print the warning if amount of events in incoming queue is too big #75

Merged

Conversation

Jimilian
Copy link
Contributor

It should simplify troubleshooting in case of slowness in event
processing logic.

Copy link
Collaborator

@rsandell rsandell left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

FYI there already is an admin monitor showing this warning in the gerrit trigger.

if (queueSize >= WORK_QUEUE_SIZE_WARNING_THRESHOLD) {
logger.warn("The Gerrit-trigger incoming events queue contains {} items!"
+ " Something might be stuck, or your system can't process the commands fast enough."
+ " Try to increase the number of receiving worker threads on the Gerrit configuration page."
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Strange to be talking about configuration pages here. Other consumers are not necessarily a trigger and might not even have a configuration page.

Copy link
Contributor Author

@Jimilian Jimilian Jan 30, 2018

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

To be honest, I just copy-pasted it from

Can fix in both (initially I tried to generalise this two classes, but change grew too fast and did not stop).

Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

ah my memory bad :) the old warning was only on the out queue I guess, not the in queue .

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@rsandell should I remove mentioning of "configuration pages" from both?

Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yes, we can rephrase it a bit I think. Something like:

The Gerrit incoming events queue contains {} items!
Something might be stuck, or your system can't process the commands fast enough.
Try to increase the number of receiving worker threads.

/**
* The minimum size of the job-queue before monitors should begin to warn the administrator(s).
*/
public static final int WORK_QUEUE_SIZE_WARNING_THRESHOLD = 40;
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Would be nice if this could be set via for example a system property (negative value could be for turning it off), the default is probably OK, there is another default in the gerrit trigger shown via an admin monitor iirc.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It should simplify troubleshooting in case of slowness in event
processing logic.
@Jimilian Jimilian force-pushed the print_warning_in_case_of_big_queue branch from fd6815e to 9e1a872 Compare February 6, 2018 09:31
@Jimilian
Copy link
Contributor Author

Jimilian commented Feb 6, 2018

Done. @rsandell, please, take a look one more time.

@rsandell rsandell merged commit d129433 into sonyxperiadev:master Feb 6, 2018
@Jimilian Jimilian deleted the print_warning_in_case_of_big_queue branch February 6, 2018 14:12
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants