# FIlebeat doesn't reparse logs

**URL:** https://discuss.elastic.co/t/filebeat-doesnt-reparse-logs/46136
**Category:** Beats
**Tags:** filebeat
**Created:** [April 2, 2016, 2:42pm UTC](https://discuss.elastic.co/t/filebeat-doesnt-reparse-logs/46136 "2016-04-02T14:42:09Z")
**Posts on this page:** 12
**Page:** 1

<div class="post-metadata">

### Author: ![bronzelion](https://avatars.discourse-cdn.com/v4/letter/b/ebca7d/32.png) [@bronzelion](https://discuss.elastic.co/u/bronzelion)
#### Post date: [April 2, 2016, 2:42pm UTC](https://discuss.elastic.co/t/filebeat-doesnt-reparse-logs/46136/1 "2016-04-02T14:42:09Z")

</div>

Hey,  
I did read a couple of posts here but was not able to get the issue resolved.

I'm using filebeat-1.2.0(downloaded tar) and Logstash2.2.2. I want to force re-parse the files provided in the path section, but that doesn't seem to work.

I stopped file beat. I deleted the .filebeat registry. I start filebeat. I can see the paths I need appear in the registry file, but no parsing happens.

```auto
 prospectors:

paths: /var/*.log
ignore_older: 0

output:
logstash:
    # The Logstash hosts
    hosts: ["localhost:5044"]

```

Please let me know if i'm missing something here. Also, tail\_files is commented.

Running ./filebeat -v -d \* doesn't give any output

---

<div class="post-metadata">

### Author: ![steffens](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/steffens/32/79630_2.png) [@steffens](https://discuss.elastic.co/u/steffens)
#### Post date: [April 4, 2016, 11:38am UTC](https://discuss.elastic.co/t/filebeat-doesnt-reparse-logs/46136/2 "2016-04-04T11:38:34Z")

</div>

can you try `./filebeat -e -v -d '*'`

---

<div class="post-metadata">

### Author: ![bronzelion](https://avatars.discourse-cdn.com/v4/letter/b/ebca7d/32.png) [@bronzelion](https://discuss.elastic.co/u/bronzelion)
#### Post date: [April 4, 2016, 2:44pm UTC](https://discuss.elastic.co/t/filebeat-doesnt-reparse-logs/46136/3 "2016-04-04T14:44:27Z")

</div>

Hey Steffens,  
Trying that without -e didn't give any output on the console. Will try the other flag as well and post here if I find any change.

---

<div class="post-metadata">

### Author: ![bronzelion](https://avatars.discourse-cdn.com/v4/letter/b/ebca7d/32.png) [@bronzelion](https://discuss.elastic.co/u/bronzelion)
#### Post date: [April 4, 2016, 2:46pm UTC](https://discuss.elastic.co/t/filebeat-doesnt-reparse-logs/46136/4 "2016-04-04T14:46:21Z")

</div>

Also, can you thorw more light know where/what other registry files are used.  
I always deleted the .filebeat and touch \*.log and start filebeat again. Even this didn't allow me to re-parse files..

What I observed was, it was able to parse the directory for log( over 20k files) and get the offset info quickly

---

<div class="post-metadata">

### Author: ![steffens](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/steffens/32/79630_2.png) [@steffens](https://discuss.elastic.co/u/steffens)
#### Post date: [April 4, 2016, 4:18pm UTC](https://discuss.elastic.co/t/filebeat-doesnt-reparse-logs/46136/5 "2016-04-04T16:18:28Z")

</div>

registry file stores file meta-data like inode and offset from last lines being published. if deleting '.filebeat' is not working, maybe filebeat is writing the registry file to another location?

---

<div class="post-metadata">

### Author: ![bronzelion](https://avatars.discourse-cdn.com/v4/letter/b/ebca7d/32.png) [@bronzelion](https://discuss.elastic.co/u/bronzelion)
#### Post date: [April 4, 2016, 4:31pm UTC](https://discuss.elastic.co/t/filebeat-doesnt-reparse-logs/46136/6 "2016-04-04T16:31:42Z")

</div>

I can see the file getting created In the same location when I restart filebeat. So the registry file does get recreated. But it doesn't somehow re-parse the log files..And only iff new data is written to one of these files will I see an event in logstash

---

<div class="post-metadata">

### Author: ![bronzelion](https://avatars.discourse-cdn.com/v4/letter/b/ebca7d/32.png) [@bronzelion](https://discuss.elastic.co/u/bronzelion)
#### Post date: [April 4, 2016, 4:33pm UTC](https://discuss.elastic.co/t/filebeat-doesnt-reparse-logs/46136/7 "2016-04-04T16:33:51Z")

</div>

Or are there multiple places for a registry file? I haven't modified the block for registry file, just using the default

---

<div class="post-metadata">

### Author: ![bronzelion](https://avatars.discourse-cdn.com/v4/letter/b/ebca7d/32.png) [@bronzelion](https://discuss.elastic.co/u/bronzelion)
#### Post date: [April 4, 2016, 4:36pm UTC](https://discuss.elastic.co/t/filebeat-doesnt-reparse-logs/46136/8 "2016-04-04T16:36:52Z")

</div>

It is interesting coz the registry file just get recreated with the offsets and inode info for a huge log collection in a matter of seconds like it knew where it last left off from

---

<div class="post-metadata">

### Author: ![steffens](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/steffens/32/79630_2.png) [@steffens](https://discuss.elastic.co/u/steffens)
#### Post date: [April 4, 2016, 4:44pm UTC](https://discuss.elastic.co/t/filebeat-doesnt-reparse-logs/46136/9 "2016-04-04T16:44:16Z")

</div>

there is just one registry file used by filebeat. Without debug log I've no idea what's actually happening.

---

<div class="post-metadata">

### Author: ![bronzelion](https://avatars.discourse-cdn.com/v4/letter/b/ebca7d/32.png) [@bronzelion](https://discuss.elastic.co/u/bronzelion)
#### Post date: [April 4, 2016, 4:46pm UTC](https://discuss.elastic.co/t/filebeat-doesnt-reparse-logs/46136/10 "2016-04-04T16:46:13Z")

</div>

Also, can u throw more light on the ignore\_older parmater. If the requirement is to re-parse logs all the time,should that be set to a very large number or 0?

Next, we want to monitor a directory which keeps getting logs files in the order of thousands per day, what should be some explicit parameters like close\_older, which are good to have..

---

<div class="post-metadata">

### Author: ![bronzelion](https://avatars.discourse-cdn.com/v4/letter/b/ebca7d/32.png) [@bronzelion](https://discuss.elastic.co/u/bronzelion)
#### Post date: [April 4, 2016, 4:48pm UTC](https://discuss.elastic.co/t/filebeat-doesnt-reparse-logs/46136/11 "2016-04-04T16:48:07Z")

</div>

Let me give you repro tomorrow, after passing the debug flags. Like I had mentioned, tried using -d before and didn't see any output. Will re try and let you know..

---

<div class="post-metadata">

### Author: ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)
#### Post date: [July 5, 2017, 9:53pm UTC](https://discuss.elastic.co/t/filebeat-doesnt-reparse-logs/46136/12 "2017-07-05T21:53:50Z")

</div>


