# Elasticsearch and logstash problem

**URL:** <https://discuss.elastic.co/t/elasticsearch-and-logstash-problem/285540>\
**Category:** Elasticsearch\
**Created:** [September 30, 2021, 6:30am UTC](https://discuss.elastic.co/t/elasticsearch-and-logstash-problem/285540 "2021-09-30T06:30:42Z")\
**Posts on this page:** 4\
**Page:** 1

<div class="post-metadata">

**Author:** ![Busra\_Duygu](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/busra_duygu/32/90462_2.png) [@Busra\_Duygu](https://discuss.elastic.co/u/Busra_Duygu)\
**Post date:** [September 30, 2021, 6:30am UTC](https://discuss.elastic.co/t/elasticsearch-and-logstash-problem/285540/1 "2021-09-30T06:30:42Z")

</div>

Hello everyone. I have a problem with logstash and Elasticsearch. I would like to explain my problem with an example. I have two conf files, first example\_A.conf and second example\_B.conf. I am loading data from example\_A.conf file with logstash to Elasticsearch. The example\_A.conf file loads successfully. Then I load example\_B and all data in example\_A is deleted. The total count of the index does not change either, but there is no error while loading. Shard number of index is 0 and size = 537.4gb . If anyone has an idea what the problem could be and share it with me I would be very happy.

---

<div class="post-metadata">

**Author:** ![stephenb](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/stephenb/32/40856_2.png) [@stephenb](https://discuss.elastic.co/u/stephenb)\
**Post date:** [September 30, 2021, 1:40pm UTC](https://discuss.elastic.co/t/elasticsearch-and-logstash-problem/285540/2 "2021-09-30T13:40:52Z")

</div>

Hi @Busra_Duygu

You will need to share both the A and B conf files And perhaps sample data.

If I remember you were working with the fingerprint filter before to me it sounds like perhaps you're overwriting the data with the document ID but that's without seeing any actual configuration or data So that's just a guess.

Show us your configurations and the data and perhaps we can help.

---

<div class="post-metadata">

**Author:** ![Busra\_Duygu](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/busra_duygu/32/90462_2.png) [@Busra\_Duygu](https://discuss.elastic.co/u/Busra_Duygu)\
**Post date:** [October 1, 2021, 11:12am UTC](https://discuss.elastic.co/t/elasticsearch-and-logstash-problem/285540/3 "2021-10-01T11:12:22Z")

</div>

Hello @stephenb , the problem was in the fingerprint as you said. I solved the problem. Actually there are over 1.5 billion documents in the index and the index has only one shard so I thought the problem was in the shards or in the memory. I didn't think it would overwrite. Thank you so much for reminding me of this. You helped me a lot to improve myself on Elasticsearch and logstash, thank you for everything and all help 🤗 🤗

---

<div class="post-metadata">

**Author:** ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)\
**Post date:** [October 29, 2021, 11:12am UTC](https://discuss.elastic.co/t/elasticsearch-and-logstash-problem/285540/4 "2021-10-29T11:12:49Z")

</div>

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.
