# Filebeat only partly updates documents

**URL:** <https://discuss.elastic.co/t/filebeat-only-partly-updates-documents/380510>\
**Category:** Beats\
**Tags:** filebeat\
**Created:** [July 28, 2025, 4:48pm UTC](https://discuss.elastic.co/t/filebeat-only-partly-updates-documents/380510 "2025-07-28T16:48:39Z")\
**Posts on this page:** 3\
**Page:** 1

<div class="post-metadata">

**Author:** ![Rick\_V](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/rick_v/32/131683_2.png) [@Rick\_V](https://discuss.elastic.co/u/Rick_V)\
**Post date:** [July 28, 2025, 4:48pm UTC](https://discuss.elastic.co/t/filebeat-only-partly-updates-documents/380510/1 "2025-07-28T16:48:40Z")

</div>

Hi all,

Using Elastic I want to create a single source of thruth. In an index I would want to store a single record of an entity and update that record when its status changes.

The status of a entity gets shared via a .ndjson file which is read by Filebeat. In Filebeat a processor is put in the configuration which sets op\_type to 'index' in the document metadata to make sure documents can be updated, also the documents are sent to an index, not a datastream.

All entities are already in Elastic with the entity ID being the document \_id, I want to update their status which is stored in two seperate fields: 'ready' and 'blockers'.

Now the following strange things happen:

- Whenever I add a single line to the .ndjson file, the @timestamp of the file being inested is changed in Elastic, but the other two fields are not.
- Whenever I add a second line, similar as the previous line, to the .ndjson file, the entire document suddenly does get updated.

The document \_id is set in an ingest pipeline, which works because the document gets updated, it being partly or completely.

Thanks in advance,  
Rick

---

<div class="post-metadata">

**Author:** ![stephenb](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/stephenb/32/40856_2.png) [@stephenb](https://discuss.elastic.co/u/stephenb)\
**Post date:** [July 28, 2025, 5:30pm UTC](https://discuss.elastic.co/t/filebeat-only-partly-updates-documents/380510/2 "2025-07-28T17:30:21Z")

</div>

Hi @Rick_V

> [@Rick\_V](#):
>
> Whenever I add a single line to the .ndjson file, the @timestamp of the file being inested is changed in Elastic, but the other two fields are not.

You will need to show us more... but filebeat is line oriented so if you add a line, it will not read it until there is a newline after it.. i.e. the next line... perhaps that is what you are seeing...

Add a few lines or in your test make sure you add a newline / empty line... check and see

---

<div class="post-metadata">

**Author:** ![Rick\_V](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/rick_v/32/131683_2.png) [@Rick\_V](https://discuss.elastic.co/u/Rick_V)\
**Post date:** [July 28, 2025, 5:53pm UTC](https://discuss.elastic.co/t/filebeat-only-partly-updates-documents/380510/3 "2025-07-28T17:53:16Z")

</div>

Hi Stephen,

Thanks for your quick reply, and solution... As I was manually adding new lines to the file, I didn't add an empty newline and thus the records didn't get updated on the first time, but always on the second.

I was thinking more in de area of filebeat settings than actually checking whether the file I submitted was valid.

Thanks for your help.