Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

docker-compose up with multiple instances at same time will meet a bug : ' MongoError: ns not found', 'Errors like this can cause oplog processing errors.' #17020

Closed
introspection3 opened this issue Mar 26, 2020 · 20 comments
Labels
stat: stale Stale issues will be automatically closed if no activity

Comments

@introspection3
Copy link

introspection3 commented Mar 26, 2020

Description:

rocketchat-8080_1 | Updating process.env.MAIL_URL
rocketchat-3000_1 | === UnHandledPromiseRejection ===
rocketchat-3000_1 | { MongoError: ns not found
rocketchat-3000_1 | at Connection. (/app/bundle/programs/server/npm/node_modules/meteor/npm-mongo/node_modules/mongodb-core/lib/connection/pool.js:443:61)
rocketchat-3000_1 | at emitTwo (events.js:126:13)
rocketchat-3000_1 | at Connection.emit (events.js:214:7)
rocketchat-3000_1 | at processMessage (/app/bundle/programs/server/npm/node_modules/meteor/npm-mongo/node_modules/mongodb-core/lib/connection/connection.js:364:10)
rocketchat-3000_1 | at Socket. (/app/bundle/programs/server/npm/node_modules/meteor/npm-mongo/node_modules/mongodb-core/lib/connection/connection.js:533:15)
rocketchat-3000_1 | at emitOne (events.js:116:13)
rocketchat-3000_1 | at Socket.emit (events.js:211:7)
rocketchat-3000_1 | at addChunk (_stream_readable.js:263:12)
rocketchat-3000_1 | at readableAddChunk (stream_readable.js:250:11)
rocketchat-3000_1 | at Socket.Readable.push (stream_readable.js:208:10)
rocketchat-3000_1 | at TCP.onread (net.js:601:20)
rocketchat-3000_1 | operationTime: Timestamp { bsontype: 'Timestamp', low: 102, high
: 1585198643 },
rocketchat-3000_1 | ok: 0,
rocketchat-3000_1 | errmsg: 'ns not found',
rocketchat-3000_1 | code: 26,
rocketchat-3000_1 | codeName: 'NamespaceNotFound',
rocketchat-3000_1 | '$clusterTime':
rocketchat-3000_1 | { clusterTime: Timestamp { bsontype: 'Timestamp', low: 102, high
: 1585198643 },
rocketchat-3000_1 | signature: { hash: [Object], keyId: [Object] } },
rocketchat-3000_1 | name: 'MongoError',
rocketchat-3000_1 | [Symbol(mongoErrorContextSymbol)]: {} }
rocketchat-3000_1 | ---------------------------------
rocketchat-3000_1 | Errors like this can cause oplog processing errors.
rocketchat-3000_1 | Setting EXIT_UNHANDLEDPROMISEREJECTION will cause the process to exit allowing your service to automatically restart the process
rocketchat-3000_1 | Future node.js versions will automatically exit the process

Steps to reproduce:

Expected behavior:
No mongo errors should be presented

Actual behavior:
MongoError: ns not found

Server Setup Information:
Version of Rocket.Chat Server: 2.4.1
Operating System: docker-compose ubuntu

image

image

we have many Rocket Chat instances running on this server.

my docker-compose just like

**
version: "3.3"
services:
rocketchat-3000:
image: rocket.chat:2.4.11
environment:
- PORT=3000
- INSTANCE_IP=172.17.117.62
- ROOT_URL=http://nginxall.local:7777
- MONGO_URL=mongodb://mongoserver.com:27017/rocketchat?replicaSet=rs0
- MONGO_OPLOG_URL=mongodb://mongoserver.com:27017/local?replicaSet=rs0
ports:
- 3000:3000
depends_on:
- "rocketchat-7777"
extra_hosts:
- "mongoserver.com:172.17.117.61"
- "rocketchat-7777:172.17.117.62"
- "rocketchat-3000:172.17.117.62"
rocketchat-7777:
image: rocket.chat:2.4.11
environment:
- PORT=7777
- INSTANCE_IP=172.17.117.62
- ROOT_URL=http://nginxall.local:7777
- MONGO_URL=mongodb://mongoserver.com:27017/rocketchat?replicaSet=rs0
- MONGO_OPLOG_URL=mongodb://mongoserver.com:27017/local?replicaSet=rs0
ports:
- 7777:7777
extra_hosts:
- "mongoserver.com:172.17.117.61"
- "rocketchat-7777:172.17.117.62"
- "rocketchat-3000:172.17.117.62"**

@introspection3
Copy link
Author

any body can help me?

@introspection3
Copy link
Author

I just want to know is it fatal in this situation

@introspection3
Copy link
Author

any good man can help me?

@introspection3 introspection3 changed the title run docker-compose with multiple instances will meet a strange but ' MongoError: ns not found', 'Errors like this can cause oplog processing errors.' docker-compose with multiple instances at same time will meet a bug : ' MongoError: ns not found', 'Errors like this can cause oplog processing errors.' Mar 29, 2020
@introspection3
Copy link
Author

@AmShaegar13 @sampaiodiego Sir, could you give me a hand

@introspection3
Copy link
Author

introspection3 commented Mar 29, 2020

someone met the same bug ,He fixed with #11692
but his code seems not working....
his code(916d440)

@introspection3 introspection3 changed the title docker-compose with multiple instances at same time will meet a bug : ' MongoError: ns not found', 'Errors like this can cause oplog processing errors.' docker-compose up with multiple instances at same time will meet a bug : ' MongoError: ns not found', 'Errors like this can cause oplog processing errors.' Mar 29, 2020
@introspection3
Copy link
Author

please help me (a poor man)

@introspection3
Copy link
Author

@gabriellsh could you help me sir?

@magicbelette
Copy link
Contributor

Are you sure Rocket.Chat instance can connect to mongod ?

@introspection3
Copy link
Author

@magicbelette yes. it works. but when our
docker-compose up with multiple instances at same time will meet a bug : ' MongoError: ns not found', 'Errors like this can cause oplog processing errors.'
I just want to know the reason and did you met this error when you run rocketchat with multiple instances,did you try it with docker?

@magicbelette
Copy link
Contributor

We don't use docker but I saw those errors when starting a lot of instances at the same time. For me it's a non blocking error...

@introspection3
Copy link
Author

introspection3 commented Mar 30, 2020

you also met this error? and it is not a fatal error?
can you sure?

@magicbelette
Copy link
Contributor

Yes I met this error starting new instances and i wasn't fatal in previous release (v1.1.1).
It refers to pool.js, maybe we reach a limit in the pool connection but I didn't dig to check if we can increase this pool.

@introspection3
Copy link
Author

I suspect it is a concurrency issue caused by multiple instances. did you met this bug in recent version(above 1.1.1)

@magicbelette
Copy link
Contributor

I agree with you.

I met this error while starting 3.0.4 instances too but for me this version is unstable by now.

@introspection3
Copy link
Author

introspection3 commented Mar 30, 2020

@magicbelette yes.I think you should not use the recent version . I met so many bugs,and nobody can fix it as soon as possible。I think using recent version just look like a white mouse

@introspection3
Copy link
Author

@magicbelette we are using 2.4.11 version,and I am very worried about this error.

@introspection3
Copy link
Author

@magicbelette this error is catched by node process (https://github.com/RocketChat/Rocket.Chat/blob/0b4ec3b0b8f1cbb90f8688695558ce2f58257e75/app/lib/server/lib/meteorFixes.js)
someone met the same bug ,He fixed with #11692
but his code seems not working....
his code(916d440)
can you read it?

@magicbelette
Copy link
Contributor

Maybe it's a new problem elsewhere.
By now, I'm trying to figure out how mongodb-core pool works and how many connections it handles in Rocket.Chat

@gabriellsh
Copy link
Member

Hey @564064202, I recommend you try some help at our Support Channel. It's our way of helping community members.

You can also try support@rocket.chat, but will probably take a little longer. Have a good day!

@github-actions
Copy link
Contributor

This issue has been automatically marked as stale because it has not had recent activity. It will be closed if no further activity occurs. Thank you for your contributions.

@github-actions github-actions bot added the stat: stale Stale issues will be automatically closed if no activity label Jun 19, 2020
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
stat: stale Stale issues will be automatically closed if no activity
Projects
None yet
Development

No branches or pull requests

3 participants