Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[monit] Restart rsyslog service if rsyslogd consumes > 800 MB memory #3117

Merged
merged 1 commit into from
Jul 4, 2019
Merged

[monit] Restart rsyslog service if rsyslogd consumes > 800 MB memory #3117

merged 1 commit into from
Jul 4, 2019

Conversation

jleveque
Copy link
Contributor

@jleveque jleveque commented Jul 3, 2019

Configure monit to monitor the resident memory consumption of rsyslogd. If memory usage is > 800 MB for 5 out of 10 checks (2-minute cycle interval, so 10 out of 20 minutes), restart the rsyslog service, because rsyslogd is most likely leaking memory. This is the same change that was committed to the 201803 branch here: #2963.

I tested an alternative solution using systemd (which in turn uses memory cgroups, which are now supported and enabled by default in Stretch). However, now that we have configured panic_on_oom = 2 (link), this will cause ALL OOM situations (even within memory cgroups) to trigger kernel panic. This defeats the purpose, as this solution is meant to prevent OOM situations, so I have decided to utilize monit for this purpose in the master branch (Stretch) as well as the 201803 (Jessie) branch.

@jleveque jleveque merged commit f14354f into sonic-net:master Jul 4, 2019
@jleveque jleveque deleted the rsyslog_monit branch July 4, 2019 01:21
@yxieca
Copy link
Contributor

yxieca commented Jul 10, 2019

Separate commit has been made to 201811 branch

@jleveque
Copy link
Contributor Author

@yxieca: You had temporarily cherry-picked from the 201803 branch until we had a solution on the master branch. You should probably replace it with this commit, even though they are identical for all intents and purposes.

@yxieca
Copy link
Contributor

yxieca commented Jul 10, 2019

Thanks Joe for keeping me honest! :-)

mssonicbld added a commit that referenced this pull request May 29, 2024
…lly (#19120)

#### Why I did it
src/sonic-swss
```
* 835e576d - (HEAD -> master, origin/master, origin/HEAD) [tests] mark suppress-fib-pending VS test as expected to fail (6 hours ago) [Stepan Blyshchak]
* 353ab92c - [subnet_decap] Add subnet decap (#3117) (10 hours ago) [Longxiang Lyu]
```
#### How I did it
#### How to verify it
#### Description for the changelog
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants