-
-
Notifications
You must be signed in to change notification settings - Fork 3.8k
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Mongoose converting undefined to null when sending the query to MongoDB #5776
Comments
Unfortunately the mongodb node driver does not support the |
Hi @vkarpov15. Even though the mongodb node driver is not supposed to support this type, it still works. I am able to run through the mongodb drivers and I am receiving the correct results. Why would mongoose attempt to manipulate the structure of the query? Shouldn't it let the user's query fail if that is what it is designed to do? |
That's strange, I'll re-open this issue and investigate. |
I'm not able to reproduce this: const mongoose = require('mongoose');
const co = require('co');
mongoose.Promise = global.Promise;
const GITHUB_ISSUE = `gh-5776`;
mongoose.set('debug', true);
exec()
.then(() => {
console.log('successfully ran program');
process.exit(0);
})
.catch(error => {
console.error(`Error: ${error}\n${error.stack}`);
});
function exec() {
return co(function* () {
yield mongoose.connect(`mongodb://localhost:27017/${GITHUB_ISSUE}`, { useMongoClient: true });
const schema = new mongoose.Schema({
name: String
});
const Model = mongoose.model('Model', schema);
yield Model.aggregate([
{ $match: { _id: 0123 } },
{
$group: {
_id: "$_id",
validCount: {
$sum: {
$cond: [
{ $ne: ["$myField", undefined] },
1,
0
]
}
}
}
}])
});
} The mongoose debug output:
I am testing this with node 6.11.5. Are you sure your version is 3.6.10? That's not a valid node version - they went directly from 0.x to 4.0: https://nodejs.org/en/download/releases/ |
@varunjayaraman If you notice in the original question, I mentioned how you can only see in the MongoDB logs that the incorrect information is being passed. The debugger shows the correct query, but Mongoose changes the query before sending it to MongoDB. You can only see the problem when you start a MongoDB instance in verbose mode, when it will print off every query that it receives. To put MongoDB into verbose mode, you need to type open the Mongo Shell and run the command db.setProfilingLevel(0,-1). You should see the issue now using your test script |
ah i see, yup i was able to reproduce by setting the profiling level correctly. Looking at the mongo logs, i see:
|
Mongoose doesn't transform the data from the debug output before sending it to the server, so I don't think this is a mongoose issue. I get the same output by running the below script using the mongodb driver directly: const mongodb = require('mongodb');
const co = require('co');
const GITHUB_ISSUE = `gh5776`;
exec()
.then(() => {
console.log('successfully ran program');
process.exit(0);
})
.catch(error => {
console.error(`Error: ${error}\n${error.stack}`);
});
function exec() {
return co(function* () {
const db = yield mongodb.MongoClient.connect(`mongodb://localhost:27017/${GITHUB_ISSUE}`);
yield db.collection('Model').aggregate([
{ $match: { _id: 0123 } },
{
$group: {
_id: "$_id",
validCount: {
$sum: {
$cond: [
{ $ne: ["$myField", undefined] },
1,
0
]
}
}
}
}]).toArray();
});
} Can you clarify your comment "Even though the mongodb node driver is not supposed to support this type, it still works. I am able to run through the mongodb drivers and I am receiving the correct results"? Above script uses the driver and it still transforms to |
I bumped into this nasty bug today exactly the same way. I can now confirm this bug. @vkarpov15, the debug output is correct. But after the output at some point, e.g. when stringify JSON object and send to mongodb, the following code It appears that undefined in array will be converted to null. This issue has to be reopened. btw, I am using the latest mongoose 5.4.12. |
Apart from this bug, an alternative query inside $cond should be changing |
@PhantomRay that's because the underlying mongodb driver doesn't support the |
When running a complex aggregate query, if there is a conditional check for a field to be undefined, mongoose debugger will print out the query containing undefined as the value to check against. When pointing mongoose to a mongodb instance in verbose mode, it can be seen that the value is converted to null. Because of this, conditional checks for non-existent fields do not work in Mongoose.
Example Input: MongooseModel.aggregate([{$match:{_id:0123}},{$group: {_id:"$_id", validCount:{$sum:{$cond: [{$ne: ["$myField", undefined]}, 1, 0]}}])
Example Query MongoDB Receives: MongooseModel.aggregate([{$match:{_id:0123}},{$group: {_id:"$_id", validCount:{$sum:{$cond: [{$ne: ["$myField", null]}, 1, 0]}}])
In the event that the field myField does not exist in the document, the check will fail as mongoDB knows that the field is undefined and not null, however mongoose will pass in null, resulting in incorrect results.
MongoDB Version - 3.4
Node.js Version - 3.6.10
Mongoose Version -Able to replicate with all versions. Versions tried - 4.5.5, 4.7.1, 4.12.5
The text was updated successfully, but these errors were encountered: