# Filebeat sending old logs on restart

**URL:** <https://discuss.elastic.co/t/filebeat-sending-old-logs-on-restart/46189>\
**Category:** Beats\
**Tags:** filebeat\
**Created:** [April 4, 2016, 1:26am UTC](https://discuss.elastic.co/t/filebeat-sending-old-logs-on-restart/46189 "2016-04-04T01:26:20Z")\
**Posts on this page:** 7\
**Page:** 1

<div class="post-metadata">

**Author:** ![bbach](https://avatars.discourse-cdn.com/v4/letter/b/d26b3c/32.png) [@bbach](https://discuss.elastic.co/u/bbach)\
**Post date:** [April 4, 2016, 1:26am UTC](https://discuss.elastic.co/t/filebeat-sending-old-logs-on-restart/46189/1 "2016-04-04T01:26:20Z")

</div>

I upgraded about 50 servers to filebeat 1.2. Two of the servers seem to be sending lines from old log files (from log files that are over a year old). I stopped filebeat and added an "ignore\_older: 24h" line to to the prospector but it doesn't seem to make a difference. Am I misunderstanding what ignore\_older does? Thanks. -- Bud

---

<div class="post-metadata">

**Author:** ![steffens](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/steffens/32/79630_2.png) [@steffens](https://discuss.elastic.co/u/steffens)\
**Post date:** [April 4, 2016, 11:43am UTC](https://discuss.elastic.co/t/filebeat-sending-old-logs-on-restart/46189/2 "2016-04-04T11:43:46Z")

</div>

Hmmm.... ignore\_older should do the trick. Maybe files are still resend after setting ignore\_older due to new entries in registry file. Check and remove entries for these 2 log-files from registry file and see how things work out.

---

<div class="post-metadata">

**Author:** ![bbach](https://avatars.discourse-cdn.com/v4/letter/b/d26b3c/32.png) [@bbach](https://discuss.elastic.co/u/bbach)\
**Post date:** [April 4, 2016, 1:32pm UTC](https://discuss.elastic.co/t/filebeat-sending-old-logs-on-restart/46189/3 "2016-04-04T13:32:44Z")

</div>

@steffens, Is there a tool to use for the registry or just use an editor?

Also, I installed from the repo on Ubuntu and expected the registry to be in /var/lib/filebeat/registry but instead found it in /.filebeat, Is that "normal" ?

Thanks. -- Bud

---

<div class="post-metadata">

**Author:** ![steffens](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/steffens/32/79630_2.png) [@steffens](https://discuss.elastic.co/u/steffens)\
**Post date:** [April 4, 2016, 3:18pm UTC](https://discuss.elastic.co/t/filebeat-sending-old-logs-on-restart/46189/4 "2016-04-04T15:18:55Z")

</div>

you got filebeat from tar file?

registry file is plain json. I think it's one line of json per document (check in editor). If so, sed might do the trick.

---

<div class="post-metadata">

**Author:** ![bbach](https://avatars.discourse-cdn.com/v4/letter/b/d26b3c/32.png) [@bbach](https://discuss.elastic.co/u/bbach)\
**Post date:** [April 4, 2016, 5:41pm UTC](https://discuss.elastic.co/t/filebeat-sending-old-logs-on-restart/46189/5 "2016-04-04T17:41:46Z")

</div>

@steffens

> you got filebeat from tar file?

No, I used apt-get following:

> **[Repositories for APT and YUM | Filebeat Reference \[8.11\] | Elastic](https://www.elastic.co/guide/en/beats/filebeat/current/setup-repositories.html)**

On all on my servers (50ish) the registry appears to be /.filebeat

> registry file is plain json. I think it's one line of json per document (check in editor). If so, sed might do the trick.

So after some more investigation, it appears that _all_ of my servers were sending old logs.

On a test server, I stopped filebeat, deleted the registry file and restarted with "ignore\_older: 24h". Now the registry only has file created in the last 24 hours.

I'm going to clean up my indexes from yesterday and today, delete the registry on all the servers and restart with ignore\_older: 24h to get everything from the last 24 hours repopulated. I don't have any logs from the weekend to worry about so this should work for me.

-- Bud

---

<div class="post-metadata">

**Author:** ![bbach](https://avatars.discourse-cdn.com/v4/letter/b/d26b3c/32.png) [@bbach](https://discuss.elastic.co/u/bbach)\
**Post date:** [April 4, 2016, 11:59pm UTC](https://discuss.elastic.co/t/filebeat-sending-old-logs-on-restart/46189/6 "2016-04-04T23:59:36Z")

</div>

> [@bbach](#):
>
> I'm going to clean up my indexes from yesterday and today, delete the registry on all the servers and restart with ignore\_older: 24h to get everything from the last 24 hours repopulated. I don't have any logs from the weekend to worry about so this should work for me.

This worked fine. I just made sure filebeat was stopped on all the servers, deleted the documents from yesterday and today's index, deleted the filebeat registry file and then restarted filebeat. All the logs were repopulated properly. -- Bud

---

<div class="post-metadata">

**Author:** ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)\
**Post date:** [July 5, 2017, 9:53pm UTC](https://discuss.elastic.co/t/filebeat-sending-old-logs-on-restart/46189/7 "2017-07-05T21:53:49Z")

</div>


