# Force Filebeat to re process a file

**URL:** <https://discuss.elastic.co/t/force-filebeat-to-re-process-a-file/247901>\
**Category:** Beats\
**Tags:** filebeat\
**Created:** [September 8, 2020, 1:59pm UTC](https://discuss.elastic.co/t/force-filebeat-to-re-process-a-file/247901 "2020-09-08T13:59:08Z")\
**Posts on this page:** 2\
**Page:** 1

<div class="post-metadata">

**Author:** ![Jackson](https://avatars.discourse-cdn.com/v4/letter/j/e68b1a/32.png) [@Jackson](https://discuss.elastic.co/u/Jackson)\
**Post date:** [September 8, 2020, 1:59pm UTC](https://discuss.elastic.co/t/force-filebeat-to-re-process-a-file/247901/1 "2020-09-08T13:59:08Z")

</div>

Hello,

I added a new file on the filebeat.yml. It went into elastic but some of the fields were not parsed correctly. This file is only updated once per day at 9AM.

I modified the filebeat.yml to properly parse the file but now filebeat refuses to send the file to elastic a second time.

I started by removing the bad data from my index.

```auto
POST filebeat-*/_delete_by_query
{
  "query": {
    "match": {
      "log.file.path":"/var/log/qrsender/qrsender.log"
    }
  }
}

```

I stopped the filebeat service and I edited the .json file in the registry by fully removing the only line that referenced "/var/log/qrsender/qrsender.log"

After restarting the service, I got nothing in elastic from this file.

Next, I modified "/var/log/qrsender/qrsender.log" by adding a line, saving, and then deleting the line so that the OS shows it was updated.

This didn't work.

I checked the .json file in the registry and there was a new line pointing to "/var/log/qrsender/qrsender.log" with the offset of 20K+.

I changed the offset to 0 and saved the file.

This didn't work.

I ran filebeat setup -e

This didn't work

`filebeat test config` returns Config OK

I checked the .json file and it again has the offset for "/var/log/qrsender/qrsender.log" at 22K+

I've been working on this for hours and it's quite frustrating. How can I get this file into elastic? I don't want to delete the entire registry as I have millions of documents.

Thanks in advance.

Here is the relevant parts of my filebeat.yml

```auto
- type: log
  enabled: true
  paths:
    - /var/log/qrsender/*.log
  json.keys_under_root: true
  json.add_error_key: true

```

and

```auto
processors:
  - add_host_metadata:
      when.not.contains.tags: forwarded
  - add_cloud_metadata: ~
  - add_docker_metadata: ~
  - add_kubernetes_metadata: ~
  - decode_json_fields:
      fields: ["error"]
      process_array: false
      max_depth: 1
      target: ""
      overwrite_keys: true
      add_error_key: false
  - rename:
      fields:
      - from: "error"
        to: "error.message"
      ignore_missing: true

```

---

<div class="post-metadata">

**Author:** ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)\
**Post date:** [October 6, 2020, 3:59pm UTC](https://discuss.elastic.co/t/force-filebeat-to-re-process-a-file/247901/2 "2020-10-06T15:59:22Z")

</div>

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.