add if no duplicate in collection mongodb python

Question

i'm writing a script that puts a large number of xml files into mongodb, thus when i execute the script multiple times the same object is added many times to the same collection.

I checked out for a way to stop this behavior by checkinng the existance of the object before adding it, but can't find a way.

help!

shx2 · Accepted Answer · 2014-09-12 20:50:10Z

1

The term for the operation you're describing is "upsert".

In mongodb, the way to upsert is to use the update functionality with upsert=True.

answered Sep 12, 2014 at 20:50

shx2

64.8k17 gold badges139 silver badges166 bronze badges

Sign up to request clarification or add additional context in comments.

Comments

vmr · Accepted Answer · 2014-09-12 11:55:24Z

0

You can index on one or more fields(not _id) of the document/xml structure. Then make use of find operator to check if a document containing that indexed_field:value is present in the collection. If it returns nothing then you can insert new documents into your collection. This will ensure only new docs are inserted when you re-run the script.

edited Sep 12, 2014 at 11:55

answered Sep 12, 2014 at 11:33

vmr

1,94514 silver badges24 bronze badges

2 Comments

Oussama L. Over a year ago

actualy i found out something like db.collection.find(object).limit(1) returns the first object it finds matching "object", i can't use $exist like that : db.collection.find(o, {$exist:true}), can i?

vmr Over a year ago

Apologies for the confusion caused, "$exists" does not make sens in this scenario. Just use "find" but on an indexed field so that the query is faster.

Collectives™ on Stack Overflow

add if no duplicate in collection mongodb python

2 Answers 2

Comments

2 Comments

Hot Network Questions

Collectives™ on Stack Overflow

2 Answers 2

Comments

2 Comments

Related