# How to remove duplicate records from result set ( with \_id as tracking code )

**URL:** <https://discuss.elastic.co/t/how-to-remove-duplicate-records-from-result-set-with-id-as-tracking-code/220749>\
**Category:** Kibana\
**Tags:** ilm-index-lifecycle-management\
**Created:** [February 25, 2020, 1:44am UTC](https://discuss.elastic.co/t/how-to-remove-duplicate-records-from-result-set-with-id-as-tracking-code/220749 "2020-02-25T01:44:34Z")\
**Posts on this page:** 3\
**Page:** 1

<div class="post-metadata">

**Author:** ![namsouth](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/namsouth/32/63226_2.png) [@namsouth](https://discuss.elastic.co/u/namsouth)\
**Post date:** [February 25, 2020, 1:44am UTC](https://discuss.elastic.co/t/how-to-remove-duplicate-records-from-result-set-with-id-as-tracking-code/220749/1 "2020-02-25T01:44:34Z")

</div>

Hello All,  
I am new to elastic stack ecosystem, and recently I face an issue,

Inside Kibana I am using index pattern to retrieve documents from multiple Index, but I don't know how do remove those duplicate records from result set ( I am using \_id as tracking field, within the same index it cannot be duplicate, but kibana index pattern can be grep multiple index )

or I should put every document into same index when I using logstash to push mysql record to elasticsearch cloud.

Following is the Output config in my logstash

```
    elasticsearch {
        hosts => ["http://elasticsearch-host:9200"]
        user => "es-username"
        password => "es-password"
        index => "stock_action-%{+YYYY.MM.dd}"
        document_type => "stock_action"
        document_id => "%{action_id}"
    }

```

Any suggestion?

---

<div class="post-metadata">

**Author:** ![mattkime](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/mattkime/32/43522_2.png) [@mattkime](https://discuss.elastic.co/u/mattkime)\
**Post date:** [February 26, 2020, 5:21pm UTC](https://discuss.elastic.co/t/how-to-remove-duplicate-records-from-result-set-with-id-as-tracking-code/220749/2 "2020-02-26T17:21:56Z")

</div>

Hello @namsouth

Which version of the Elastic stack are you running?

How are you populating Elasticsearch with documents? You mention logstash - are you using the Elastic stack to monitor MySQL?

---

<div class="post-metadata">

**Author:** ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)\
**Post date:** [March 25, 2020, 5:22pm UTC](https://discuss.elastic.co/t/how-to-remove-duplicate-records-from-result-set-with-id-as-tracking-code/220749/3 "2020-03-25T17:22:20Z")

</div>

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.