# Filebeat not sending all lines for a overwritten file

**URL:** <https://discuss.elastic.co/t/filebeat-not-sending-all-lines-for-a-overwritten-file/361786>\
**Category:** Beats\
**Tags:** filebeat\
**Created:** [June 20, 2024, 10:42am UTC](https://discuss.elastic.co/t/filebeat-not-sending-all-lines-for-a-overwritten-file/361786 "2024-06-20T10:42:13Z")\
**Posts on this page:** 3\
**Page:** 1

<div class="post-metadata">

**Author:** ![immortel](https://avatars.discourse-cdn.com/v4/letter/i/e99b99/32.png) [@immortel](https://discuss.elastic.co/u/immortel)\
**Post date:** [June 20, 2024, 10:42am UTC](https://discuss.elastic.co/t/filebeat-not-sending-all-lines-for-a-overwritten-file/361786/1 "2024-06-20T10:42:13Z")

</div>

Hello,  
On a daily basis, we are exporting 4 configurations files. We are overwriting the same file with the new data that is pretty much the same every time.  
My problem is that filebeat does not realize it is a new file and I'd like it to send the whole file on a daily basis.  
Some times, it sends all the data, some times just a subset of it. Right now, I'm deleting the existing file to recreate one (happening in few seconds). I also try to erase the old file and create a new file with a new name (filebeat sees it as a file being renamed).  
My latest configuration file looks like this:

```auto
  - type: filestream
    id: ConfigurationFile_1
    paths:
      - /var/config/ConfigurationFile_1.csv
    clean_removed: true
    prospector:
      scanner:
        resend_on_touch: true
        fingerprint:
          enabled: true
          offset: 0
          length: 1024
# file_identity:
# fingerprint:
# enabled: true
# offset: 0
# length: 1024
    ignore_older: 3m
    fields:
      log_topic: configuration_kafka_topic
    processors:
      - add_fields:
          target: ""
          fields:
            system_name: as002
      - dissect:
          tokenizer: "%{enterpriseId}|%{enterpriseName}|%{groupId}|%{groupName}|%{domainName}|"
          field: "message"
          target_prefix: "SystemConfig"
          trim_values: "all"
      - script:
          lang: javascript
          id: lowercase
          source: >
            function process(event) {
              var groupId = event.Get("SystemConfig.groupId");
              if (groupId != null) {
                event.Put("SystemConfig.groupIdLowerCase", groupId.toLowerCase());
              }
            }

```

---

<div class="post-metadata">

**Author:** ![ashishtiwari1993](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/ashishtiwari1993/32/135241_2.png) [@ashishtiwari1993](https://discuss.elastic.co/u/ashishtiwari1993)\
**Post date:** [June 24, 2024, 9:04am UTC](https://discuss.elastic.co/t/filebeat-not-sending-all-lines-for-a-overwritten-file/361786/2 "2024-06-24T09:04:55Z")

</div>

Hi @immortel, Welcome to Elastic community.

You can try to set [close\_inactive](https://www.elastic.co/guide/en/beats/filebeat/current/filebeat-input-log.html#filebeat-input-log-close-inactive).

_When this option is enabled, Filebeat closes the file handle if a file has not been harvested for the specified duration. If the closed file changes again, a new harvester is started and the latest changes will be picked up after scan\_frequency has elapsed._

---

<div class="post-metadata">

**Author:** ![immortel](https://avatars.discourse-cdn.com/v4/letter/i/e99b99/32.png) [@immortel](https://discuss.elastic.co/u/immortel)\
**Post date:** [June 25, 2024, 10:27am UTC](https://discuss.elastic.co/t/filebeat-not-sending-all-lines-for-a-overwritten-file/361786/3 "2024-06-25T10:27:30Z")

</div>

Hello @ashishtiwari1993,  
The default value for this option is 5 minutes and my file is being updated every 24 hours. I don't know what other value I could put in there that could improve the 5 minutes.