# Filebeat filestream: Duplicate ingestion after "key with no value" in log.json registry file

**URL:** <https://discuss.elastic.co/t/filebeat-filestream-duplicate-ingestion-after-key-with-no-value-in-log-json-registry-file/378377>\
**Category:** Beats\
**Tags:** filebeat\
**Created:** [May 21, 2025, 9:18am UTC](https://discuss.elastic.co/t/filebeat-filestream-duplicate-ingestion-after-key-with-no-value-in-log-json-registry-file/378377 "2025-05-21T09:18:53Z")\
**Posts on this page:** 1\
**Page:** 1

<div class="post-metadata">

**Author:** ![anpama](https://avatars.discourse-cdn.com/v4/letter/a/97f17d/32.png) [@anpama](https://discuss.elastic.co/u/anpama)\
**Post date:** [May 21, 2025, 9:18am UTC](https://discuss.elastic.co/t/filebeat-filestream-duplicate-ingestion-after-key-with-no-value-in-log-json-registry-file/378377/1 "2025-05-21T09:18:54Z")

</div>

Hi everyone,

I'm seeing a duplicate ingestion issue with Filebeat 8.14 using the filestream input and default settings. For some files, Filebeat suddenly starts reading the whole file again, leading to duplicate events.

While investigating, I noticed something odd in the `log.json` registry file. Sometimes I see entries like this:

```json
{"k":"filestream::a-b-c::native::528020-161","v":{"ttl":1800000000000,"updated":[516247465673,1747805249],"cursor":{"offset":261852},"meta":null}}                                                                                                                                                                                                                                                                      
{"k":"filestream::a-b-c::native::528020-161"}                                                                                                                                                                                                                                                                                                                                                                               
{"k":"filestream::a-b-c::native::528020-161","v":{"meta":null,"ttl":1800000000000,"updated":[516144192881,1747808794],"cursor":null}}                                                                                                                                                                                                                                                                                      
{"k":"filestream::a-b-c::native::528020-161","v":{"ttl":1800000000000,"updated":[516170057952,1747808794],"cursor":{"offset":262191},"meta":null}}                                                     

```

Here’s what I observed:

- Sometimes, there appears an entry with just a key and no value, like the second line above.
- The `updated` field in the preceding and following lines holds a timestamp. I saw that each of these timestamps matches exactly when a complete re-ingestion happens (the first timestamp for the original ingestion, the second one for the duplicate).
- Right after the "key with no value", the `cursor` field (which I believe tracks position in the file) gets reset to `null`. This seems to cause Filebeat to start from the beginning and re-ingest the whole file.

I checked in Kibana and the timing of these changes matches perfectly with duplicate ingestion spikes :

 ![image](https://us1.discourse-cdn.com/elastic/original/3X/3/b/3bdc3abfbd2f1ebaf4736d5c492f49352224964f.png)

Does this make sense as a possible cause, or should I be looking elsewhere? Has anyone else seen this, or know what could trigger a "key with no value" in the registry?