# Filebeat not reading the already processed log file again

**URL:** <https://discuss.elastic.co/t/filebeat-not-reading-the-already-processed-log-file-again/177171>\
**Category:** Beats\
**Tags:** filebeat\
**Created:** [April 16, 2019, 11:47pm UTC](https://discuss.elastic.co/t/filebeat-not-reading-the-already-processed-log-file-again/177171 "2019-04-16T23:47:53Z")\
**Posts on this page:** 4\
**Page:** 1

<div class="post-metadata">

**Author:** ![SManorathna](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/smanorathna/32/42626_2.png) [@SManorathna](https://discuss.elastic.co/u/SManorathna)\
**Post date:** [April 16, 2019, 11:47pm UTC](https://discuss.elastic.co/t/filebeat-not-reading-the-already-processed-log-file-again/177171/1 "2019-04-16T23:47:53Z")

</div>

Hi,

I am reading a .log file using filebeat and I need the data to be output to the elasticsearch.

This is my filebeat configuration

```auto
#=========================== Filebeat inputs =============================
filebeat.inputs:
- type: log

  # Change to true to enable this input configuration.
  enabled: true

  # Paths that should be crawled and fetched. Glob based paths.
  paths:
    - ~/Documents/development/filebeat/*.log

#================================ Outputs =====================================
output.logstash:
  # The Logstash hosts
  hosts: ["localhost:5044"]

```

I have a log file placed in the ~/Documents/development/filebeat directory  
This is my logstash configuration

```auto
input {
    beats {
      host => "localhost"
      port => 5044
    }
}

filter {
// some filtering applies in here
}

output {
  elasticsearch {
    hosts => ["localhost:9200"]
    index => "transactions-%{+YYYY.MM.dd}"
}}

```

I have executed the process once and the log file entries were created as an index in the elasticsearch. Then I deleted the created index and now trying to recreate the index by restarting all the services (elasticsearch, logstash and filebeat)  
But the index is not creating again.

I even deleted the filebeat registry file and tried it again, but again the registry file will be created with the old data. I have even renamed the log files.  
What is the wrong thing that I am doing here ?

---

<div class="post-metadata">

**Author:** ![shaunak](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/shaunak/32/6643_2.png) [@shaunak](https://discuss.elastic.co/u/shaunak)\
**Post date:** [April 17, 2019, 5:04pm UTC](https://discuss.elastic.co/t/filebeat-not-reading-the-already-processed-log-file-again/177171/2 "2019-04-17T17:04:30Z")

</div>

> [@SManorathna](#):
>
> I even deleted the filebeat registry file and tried it again, but again the registry file will be created with the old data.

This seems odd. When you restart Filebeat after deleting the registry file, can you post the first 50 or so lines from the Filebeat log? **Please be sure to redact any sensitive information before posting.**

---

<div class="post-metadata">

**Author:** ![SManorathna](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/smanorathna/32/42626_2.png) [@SManorathna](https://discuss.elastic.co/u/SManorathna)\
**Post date:** [April 19, 2019, 4:47am UTC](https://discuss.elastic.co/t/filebeat-not-reading-the-already-processed-log-file-again/177171/3 "2019-04-19T04:47:28Z")

</div>

I have done a stupid mistake there. I was keep deleting the registry file when the logstash service is running. That's why it was not deleted properly. Stopping the service and then deleting works actually. My bad

---

<div class="post-metadata">

**Author:** ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)\
**Post date:** [May 17, 2019, 4:47am UTC](https://discuss.elastic.co/t/filebeat-not-reading-the-already-processed-log-file-again/177171/4 "2019-05-17T04:47:35Z")

</div>

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.