# Filebeat filestream reads the files from last line after service restart

**URL:** <https://discuss.elastic.co/t/filebeat-filestream-reads-the-files-from-last-line-after-service-restart/368425>\
**Category:** Beats\
**Tags:** filebeat\
**Created:** [October 8, 2024, 9:45am UTC](https://discuss.elastic.co/t/filebeat-filestream-reads-the-files-from-last-line-after-service-restart/368425 "2024-10-08T09:45:12Z")\
**Posts on this page:** 5\
**Page:** 1

<div class="post-metadata">

**Author:** ![jack\_a](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/jack_a/32/133082_2.png) [@jack\_a](https://discuss.elastic.co/u/jack_a)\
**Post date:** [October 8, 2024, 9:45am UTC](https://discuss.elastic.co/t/filebeat-filestream-reads-the-files-from-last-line-after-service-restart/368425/1 "2024-10-08T09:45:12Z")

</div>

how can i setup filebeat so after each filebeat service restart it start reading the files from last line and stop it from reading the file from beginning.

i have a config like the following:

```auto
filebeat.inputs:
- type: filestream
  id: my-filestream-id
  paths:
    - /var/log/messages
    - /var/log/*.log

```

can i set `filebeat.registry.path: /dev/null` ? is there away to stop filebeat from reading old logs?

---

<div class="post-metadata">

**Author:** ![ashishtiwari1993](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/ashishtiwari1993/32/135241_2.png) [@ashishtiwari1993](https://discuss.elastic.co/u/ashishtiwari1993)\
**Post date:** [October 8, 2024, 10:28am UTC](https://discuss.elastic.co/t/filebeat-filestream-reads-the-files-from-last-line-after-service-restart/368425/2 "2024-10-08T10:28:23Z")

</div>

Hi Jack\_a, Filebeat, by default, reads new logs (except those that have already been sent to Elasticsearch)

From the [doc](https://www.elastic.co/guide/en/beats/filebeat/current/how-filebeat-works.html#_how_does_filebeat_keep_the_state_of_files) -

> _Filebeat keeps the state of each file and frequently flushes the state to disk in the registry file. The state is used to remember the last offset a harvester was reading from and to ensure all log lines are sent._

---

<div class="post-metadata">

**Author:** ![jack\_a](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/jack_a/32/133082_2.png) [@jack\_a](https://discuss.elastic.co/u/jack_a)\
**Post date:** [October 8, 2024, 10:43am UTC](https://discuss.elastic.co/t/filebeat-filestream-reads-the-files-from-last-line-after-service-restart/368425/3 "2024-10-08T10:43:16Z")

</div>

hi, thanks for the reply. what i want is the filebeat start reading from last line of a log file when it stars. assume i have a file that have the logs for the past 5 years and still being updated. i want when i start logstash it starts reading new logs (since filebeat start) and not start from beginning of the file and reading logs from 5 years ago.

---

<div class="post-metadata">

**Author:** ![ashishtiwari1993](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/ashishtiwari1993/32/135241_2.png) [@ashishtiwari1993](https://discuss.elastic.co/u/ashishtiwari1993)\
**Post date:** [October 8, 2024, 11:37am UTC](https://discuss.elastic.co/t/filebeat-filestream-reads-the-files-from-last-line-after-service-restart/368425/4 "2024-10-08T11:37:37Z")

</div>

1. The best approach is rotate the file and create a new one for new logs. Set [ignore\_older](https://www.elastic.co/guide/en/beats/filebeat/current/filebeat-input-log.html#filebeat-input-log-ignore-older) to not ingest previous file logs. Though if you can archive or move to different path, then you don't need to set ignore\_older.
2. You can give a try by setting [exclude\_lines](https://www.elastic.co/guide/en/beats/filebeat/current/filebeat-input-log.html#filebeat-input-log-exclude-lines). I haven't try but you can add regex on timestamp in your logs (specifically date or year) but you need to handle properly.

---

<div class="post-metadata">

**Author:** ![jack\_a](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/jack_a/32/133082_2.png) [@jack\_a](https://discuss.elastic.co/u/jack_a)\
**Post date:** [October 8, 2024, 11:58am UTC](https://discuss.elastic.co/t/filebeat-filestream-reads-the-files-from-last-line-after-service-restart/368425/5 "2024-10-08T11:58:02Z")

</div>

so i gues there is no direct approch. for example like what we can do with _ **logstash** _ and configure it like below

```
sincedb_path: /dev/null # Ignore read state on restart
start_position: end # Read from the end of the log file

```