dadoonet
December 19, 2017, 5:41am
10
I can see that you asked the same question at
want to delete the duplicates the below code is correct
the below code is written in the logstash file under config file.
file type is conf file.
output {
elasticsearch {
hosts => ["localhost:9200"]
manage_template => false
index => "test" }
stdout { codec => rubydebug }
document_id => "%{[@metadata][_mrdno]}"
}
want to remove the duplicates of mrdno column.
please let me know
So the answer is: there is no way to remove duplicates in one single call.
As I said, you need to:
Remove all docs
Add back one of the docs you removed
Is it a one time operation or something you want to do in the long run? If the later, what is the usecase of allowing duplicates ?