Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[FIX] Performance issues when using new Oplog implementation #19181

Merged
merged 2 commits into from
Oct 9, 2020

Conversation

rodrigok
Copy link
Member

@rodrigok rodrigok commented Oct 8, 2020

Proposed changes

Issue(s)

Closes #19082

How to test or reproduce

Screenshots

Types of changes

  • Bugfix (non-breaking change which fixes an issue)
  • Improvement (non-breaking change which improves a current function)
  • New feature (non-breaking change which adds functionality)
  • Breaking change (fix or feature that would cause existing functionality to not work as expected)
  • Hotfix (a major bugfix that has to be merged asap)
  • Documentation Update (if none of the other choices apply)

Checklist

  • I have read the CONTRIBUTING doc
  • I have signed the CLA
  • Lint and unit tests pass locally with my changes
  • I have added tests that prove my fix is effective or that my feature works (if applicable)
  • I have added necessary documentation (if applicable)
  • Any dependent changes have been merged and published in downstream modules

Changelog

A missing configuration was not limiting the new oplog tailing to pool the database frequently even when no data was available, leading to both node and mongodb process been consuming high CPU even with low usage. This case was happening for installations using mmapv1 database engine or when no admin access was granted to the database user, both preventing the usage of the new Change Streams implementation and fallbacking to our custom oplog implementation in replacement to the Meteor's one what was able to be disabled and use the native implementation via the environmental variable USE_NATIVE_OPLOG=true.

Further comments

@rodrigok rodrigok added this to the 3.7.2 milestone Oct 8, 2020
@rodrigok rodrigok requested a review from sampaiodiego October 8, 2020 17:00
@sampaiodiego sampaiodiego modified the milestones: 3.7.2, 3.7.1 Oct 8, 2020
@sampaiodiego sampaiodiego changed the title [FIX] Performance issue with mmapv1 databases [FIX] Performance issues when using new Oplog implementation Oct 8, 2020
@sampaiodiego sampaiodiego merged commit ce1f4a1 into develop Oct 9, 2020
@sampaiodiego sampaiodiego deleted the fix-oplog-performance-issue branch October 9, 2020 03:07
sampaiodiego added a commit that referenced this pull request Oct 9, 2020
Co-authored-by: Diego Sampaio <chinello@gmail.com>
@sampaiodiego sampaiodiego mentioned this pull request Oct 9, 2020
@sampaiodiego sampaiodiego mentioned this pull request Nov 14, 2020
@codeneno
Copy link

codeneno commented Jan 2, 2021

Does this only fix "mmapv1 database engine or when no admin access was granted to the database user,"?
we use 3.3.3 version,and it will have a high cpu load sometimes,does this work for wiredTiger ?thank you.

@codeneno
Copy link

codeneno commented Jan 3, 2021

when we use rocket.chat 3.9.4 version,without (USE_NATIVE_OPLOG=true),our mongodb will have much getmore operation,
and the mongodb's cpu is 99%......

@codeneno
Copy link

codeneno commented Jan 3, 2021

@sampaiodiego @rodrigok
All of us suggest there is something wrong with your new method (own oplog implementation)
#20027

@introspection3
Copy link

this bug is still hear

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

Node slamming CPU since update to 3.7
4 participants