Prevent new document with the same docuemnt id replacing the old one

Elastic Stack Logstash
1

Hi,

I am using a custom document id to avoid duplicate in Logstash. However, I notice that if the same document with the same document id gets parsed to the same index, it will replace the original one, which means @timestamp will be updated. Is there a way to prevent new document with the same document id flowing into the same index? Thank you so much.

--Tao

2

You cannot do that if the documents are in the same index, the ID is really the unique key.

3

Hi warkolm,

Thank you so much for your reply. I just want to keep the original and ignore duplicate. Are there any ways to do that?

--Tao

4

Not within Elasticsearch, it will accept the document. I can't immediately think of another way though sorry.

5

Thank you so much. At least I get rid of duplicate for now. :slight_smile:

6

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.