How to use "op_type" with logstash to prevent duplicate entries?

Elastic Stack Logstash
1

Hi,

I would like to prevent the same data log indexed twice.

I found out from the documentation that it can be achieved using "op_type" parameter.

$ curl -XPUT 'http://localhost:9200/twitter/tweet/1?op_type=create' -d '{
"user" : "kimchy",
"post_date" : "2009-11-15T14:12:12",
"message" : "trying out Elasticsearch"
}'

However, how do you use "op_type" in Logstash?

I am using filebeats to send data to logstash.
In logstash, my data is parsed into JSON and sent to elastic search via the elasticsearch output plugin:

output {
elasticsearch { hosts => ["localhost:9200"] }
stdout { codec => rubydebug }
}

Can I include op_type=create in the output plugin? How, what is the syntax?

Thanks,
Gunay

2

I think https://www.elastic.co/guide/en/logstash/current/plugins-outputs-elasticsearch.html#plugins-outputs-elasticsearch-action is what you are after.

1 Like
3

Thank you very very much Mark. Indeed, it is it the action setting what I was looking for.