Upsert / Update if exist, create if not


how do I use upsert, to create or update entries = documents via the output-plugin to elasticsearch, dependent on the document with the given id is already in elasticsearch or even not.

My data will come from two different logstash-instances A and B and it will not be clear if A comes before B or vice versa. The document-id is clear and known and should be as it is, but all other attributes may be different (disjunct) and should be added to the document.
I tried out with a simple doc_as_upsert statement on B with action = update. This will only work, if B comes after A.

What might be a solution of that ?

Thank you

The whole point of doc_as_upsert is that if the document does not exist it will get created. You can use doc_as_upsert on both. If A sends the document first it will insert and B will update. If B sends the document first then B will insert and A will update.

If they both send it at the same time then I would trust Elastic to have locking in place to resolve the race so that one is an insert and the other an update.

Thank you for your reply.
Your answer is that what i did expect from doc_as_upsert.

However in my case it does not work, maybe it is about action=>update, so that limits the doing just to update. I will try out without action => update.

This is how it looks now for Logstash-Instance A and B. The output is kinda same


hosts => ["localhost:1234:9200"]
index => "customer"
document_type => "testdata"
document_id = "%{costomer_id}%{invoice_nr}"  
doc_as_upsert => true
action => "update" 
manage_template => true

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.