Prevent new document with the same docuemnt id replacing the old one


(Tao Liu) #1

Hi,

I am using a custom document id to avoid duplicate in Logstash. However, I notice that if the same document with the same document id gets parsed to the same index, it will replace the original one, which means @timestamp will be updated. Is there a way to prevent new document with the same document id flowing into the same index? Thank you so much.

--Tao


(Mark Walkom) #2

You cannot do that if the documents are in the same index, the ID is really the unique key.


(Tao Liu) #3

Hi warkolm,

Thank you so much for your reply. I just want to keep the original and ignore duplicate. Are there any ways to do that?

--Tao


(Mark Walkom) #4

Not within Elasticsearch, it will accept the document. I can't immediately think of another way though sorry.


(Tao Liu) #5

Thank you so much. At least I get rid of duplicate for now. :slight_smile:


(system) #6

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.