I would like to segregate duplicated records in elastic -kibana by using a field with flag value like 'yes ' or ' no'. So the idea is to add a field from logstash to each documents and value of the field will be 'yes' if the document is duplicated and 'No' for opposite case. This duplication of records will be determined by using an existing filed value which is an hexadecimal value(This is nor document_id and don't want to make it as document_id).I have to filter out documents from kibana based on this new field added (with value as 'yes' or 'no') to get count with and without duplication.
I have idea to add a new field based on condition, but i need help for identifying documents having duplicated field value. I tried with 'Fingerprint filter plugin' but not worked as per my requirement.
Thanks in Advance
Prathibha.M