I have documents coming in from Logstash to Elasticsearch. I would like the following functionality. whenever a new document comes, it gets indexed in ES only if it is already not present in ES. As of now, I was trying to use fingerprint plugin of logstash, but it updates/ overwrites the original document. But I would like if the new document gets dropped. Is there a way to do this?
Hi,
You need to change the op_type from "index" to "create". Look here:
https://www.elastic.co/guide/en/elasticsearch/reference/current/docs-bulk.html#docs-bulk
https://www.elastic.co/guide/en/elasticsearch/reference/current/docs-index_.html#operation-type
https://www.elastic.co/guide/en/logstash/current/plugins-outputs-elasticsearch.html#plugins-outputs-elasticsearch-action
This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.