How not to overwrite duplicates? save old documents

Elastic Stack Logstash
1

Hi all!

I use this filter to prevent duplicates:

fingerprint {
    source => "name"
    target => "[@metadata][fingerprint]"
    method => "MURMUR3"
}

And it works. I don't see documents with the same 'name' field in Kibana.

But if a document already exists with the same fingerprint, Elasticsearch will replace the existing contents with the one me just provided - the last indexed document wins.

How can I configure Logstash not to overwrite an existing document, but simply not add a new one if there is already a similar 'name' field?

Please tell me what to do. I'm desperate.

2

If you use the action "create", the document won't be updated:
https://www.elastic.co/guide/en/logstash/current/plugins-outputs-elasticsearch.html#plugins-outputs-elasticsearch-action

1 Like
3

Thanks you! You saved me. This is what I need :+1:

4

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.