`insertMany` ignore duplicate

I’ve a collection say Tweets and each tweet has _id and another unique id id_str (added when creating schema).

When running insertMany, is it possible to just ignore tweets with duplicate id_str without throwing error? I tried following which does not work and still throw error:

try {
  const tweets = fetchedFromApi()

  await Tweet.insertMany(tweets, { ordered: false })
} catch (error) {
  console.log(error) // -> MongoError: write operation failed
}

Node: v10.14.0 Mongoose: v5.3.13 MongoDB: v4.0.2

[mhombach]

@atulmy This issue came up in my head when doing bulkInsert myself now and i think what you were doing from the beginning on was technical correct. Setting ordered: false in the insertMany will insert ALL documents that are possible and “skip” those that throw an error. But when finished, the function itself will report/throw an error with all information on how many documents had been saved and how many not (see the docs: https://docs.mongodb.com/manual/reference/method/db.collection.insertMany ). So… i guess you were just thinking “it throws an error, so no documents were stored” maybe? But infact setting ordered: false will NOT abort the insert if errors happen, it will just report them at the end. So this should be the exact thing you wanted, am i right? Can you check if this works and you maybe misread the error-message? 😃 edit: Still, if you have the possibility to delete duplicated entries before the insert, that would be more clean. But sometimes, if you insert 10 million documents, checking duplicated ISd before the insert takes way longer than just inserting and ignoring the errros of duplucated entries.

[mhombach]

!!Need a help.I have added 1000000 numbers and make ordered:false if it have duplicate value in every 1000 items less than 200 it work fine.But if more than 500 duplicates in 1000 it gave a error and stop the insert other values.

Please send a working code-example of what is working and what is not. I don’t fully understand your problem and the mistake might be somewhere else 😃