River issuing too many queries to oplog.rs #164
Comments
|
It could be related to #123. I change the oplog.rs filter following the recommendations. This change will be available in release 1.7.2 |
|
Thanks for the prompt response! Will wait for 1.7.2. Great plugin! |
|
@ramv |
|
will do. I will push the update today. On Fri, Nov 15, 2013 at 2:15 AM, Richard Louapre
Best Regards, Ram Viswanadha |
|
hmm .. can't get authenticated I tried logging on to the server with same creds and was able to |
|
Can you please increase the log level [1]? Please provide the full ES log file. [1] -https://github.com/richardwilly98/elasticsearch-river-mongodb/wiki#troubleshooting |
|
|
It does not look like an authentication issue. How did you come to that conclusion? Initial import fails if you try to use an index with existing data. So that's your case please use the new parameter skip_initial_import (see On Friday, November 15, 2013, Ram Viswanadha wrote:
|
|
my apologies. In the first instance the log stopped after trying to authenticate. So I presumed that the authentication has failed |
|
still having issues with mongodb servers hosted on MongoLab. We had an issue yesterday after one of the ElasticSearch nodes was restarted. The following query was causing a huge spike in the CPU load {
"opid" : 55243449,
"active" : true,
"secs_running" : 2400,
"op" : "query",
"ns" : "local.oplog.rs",
"query" : {
"$query" : {
"$and" : [
{
"$or" : [
{
"ns" : "savvy.comments"
},
{
"ns" : "savvy.$cmd"
}
]
},
{
"ts" : {
"$gt" : Timestamp(1385145242, 3)
}
}
]
},
"$orderby" : {
"$natural" : 1
}
},
"client" : "10.76.227.130:25779",
"desc" : "conn4463008",
"threadId" : "0x7f2a15174700",
"connectionId" : 4463008,
"locks" : {
"^" : "r",
"^local" : "R"
},
"waitingForLock" : false,
"numYields" : 7824,
"lockStats" : {
"timeLockedMicros" : {
"r" : NumberLong("4250384961"),
"w" : NumberLong(0)
},
"timeAcquiringMicros" : {
"r" : NumberLong("2400856653"),
"w" : NumberLong(0)
}
}
}MongoLab support says the following:
Any pointers on fixing this? |
|
@ramv So you should see somethinkg like: {
"$query": {
"$and": [
{
"ns": {
"$in": ["savvy.collections", "savvy.$cmd"]
}
},
{
"ts": {
"$gt": "Timestamp1385115577000|2"
}
}
]
},
"$orderby": {
"$natural": 1
}
}Could you please confirm you are currently running river 1.7.2? It should be in ES log file when the river is started.
|
- The filter will only use ts with OPLOG_REPLAY - Other filtering will be handled within the river
|
@ramv |
|
will do . i will install the new version today. Thank you very much for On Fri, Dec 6, 2013 at 2:48 AM, Richard Louapre notifications@github.comwrote:
Best Regards, Ram Viswanadha |
|
@ramv |
|
currently testing So far no issues faced. |
|
Sounds like this was fixed for you 1.7.3, so I'm going to close this issue |
We have an issue with the river mongodb. Every 2 weeks the river issues queries to Oplog.rs that essentially brings the system to a grinding halt. We are running our servers on MongoLab's dedicated cluster. The ops support engineers at MongoLab indicated that the queries being issued to oplog.sys are the ones that seem to be causing this. Anyone experienced this? Any pointers? Is there a recommended configuration that is better suited for MongoLab?
Our River config
The text was updated successfully, but these errors were encountered: