How to remove duplicate records from result set ( with _id as tracking code )

Elastic Stack Kibana
1

Hello All,
I am new to elastic stack ecosystem, and recently I face an issue,

Inside Kibana I am using index pattern to retrieve documents from multiple Index, but I don't know how do remove those duplicate records from result set ( I am using _id as tracking field, within the same index it cannot be duplicate, but kibana index pattern can be grep multiple index )

or I should put every document into same index when I using logstash to push mysql record to elasticsearch cloud.

Following is the Output config in my logstash

    elasticsearch {
        hosts => ["http://elasticsearch-host:9200"]
        user => "es-username"
        password => "es-password"
        index => "stock_action-%{+YYYY.MM.dd}"
        document_type => "stock_action"
        document_id => "%{action_id}"
    }

Any suggestion?

2

Hello @namsouth

Which version of the Elastic stack are you running?

How are you populating Elasticsearch with documents? You mention logstash - are you using the Elastic stack to monitor MySQL?

3

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.