# Overwriting new document?

**URL:** <https://discuss.elastic.co/t/overwriting-new-document/242729>\
**Category:** Elasticsearch\
**Created:** [July 27, 2020, 10:00am UTC](https://discuss.elastic.co/t/overwriting-new-document/242729 "2020-07-27T10:00:46Z")\
**Posts on this page:** 6\
**Page:** 1

<div class="post-metadata">

**Author:** ![Tuckson](https://avatars.discourse-cdn.com/v4/letter/t/f14d63/32.png) [@Tuckson](https://discuss.elastic.co/u/Tuckson)\
**Post date:** [July 27, 2020, 10:00am UTC](https://discuss.elastic.co/t/overwriting-new-document/242729/1 "2020-07-27T10:00:46Z")

</div>

Hi,

Recently I did a fresh install of elk 7.7 . I did a successfull attempt of creaing a POC and now try to build up a full environment. However, When I send a log via filebeatand logstash to ES, somehow every new document overwrites the previous one. What am I doing wrong?  
Have not seen this in my POC now on my previous production environment of ELK 6.7

Any suggestions where to look?

---

<div class="post-metadata">

**Author:** ![Christian\_Dahlqvist](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/christian_dahlqvist/32/4617_2.png) [@Christian\_Dahlqvist](https://discuss.elastic.co/u/Christian_Dahlqvist)\
**Post date:** [July 27, 2020, 10:03am UTC](https://discuss.elastic.co/t/overwriting-new-document/242729/2 "2020-07-27T10:03:10Z")

</div>

That is often caused by you setting the document\_id in the Elasticsearch output in Logstash and that the field you are using is wrong or not defined.

---

<div class="post-metadata">

**Author:** ![Tuckson](https://avatars.discourse-cdn.com/v4/letter/t/f14d63/32.png) [@Tuckson](https://discuss.elastic.co/u/Tuckson)\
**Post date:** [July 27, 2020, 10:07am UTC](https://discuss.elastic.co/t/overwriting-new-document/242729/3 "2020-07-27T10:07:14Z")

</div>

Thnx for the quick reponse. WIll have a look at that, however, I am surprised that that was not the case than with te POC install, which used the same output definitions.

---

<div class="post-metadata">

**Author:** ![Christian\_Dahlqvist](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/christian_dahlqvist/32/4617_2.png) [@Christian\_Dahlqvist](https://discuss.elastic.co/u/Christian_Dahlqvist)\
**Post date:** [July 27, 2020, 10:09am UTC](https://discuss.elastic.co/t/overwriting-new-document/242729/4 "2020-07-27T10:09:24Z")

</div>

Maybe the input has changed and the field is no longer present? Have a look at the ID of the document being indexed. That should show if it is this causing it or not.

---

<div class="post-metadata">

**Author:** ![Tuckson](https://avatars.discourse-cdn.com/v4/letter/t/f14d63/32.png) [@Tuckson](https://discuss.elastic.co/u/Tuckson)\
**Post date:** [July 27, 2020, 10:19am UTC](https://discuss.elastic.co/t/overwriting-new-document/242729/5 "2020-07-27T10:19:27Z")

</div>

It really DOES help if one puts the appropriate filter conf file in the conf.d as well ☹😉

Thnx for pointing me in te right direction.

---

<div class="post-metadata">

**Author:** ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)\
**Post date:** [August 24, 2020, 10:22am UTC](https://discuss.elastic.co/t/overwriting-new-document/242729/6 "2020-08-24T10:22:13Z")

</div>

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.