# Filebeat injest same file repeatedly on an interval for logstash filter testing / development

**URL:** <https://discuss.elastic.co/t/filebeat-injest-same-file-repeatedly-on-an-interval-for-logstash-filter-testing-development/119660>\
**Category:** Beats\
**Created:** [February 13, 2018, 2:53pm UTC](https://discuss.elastic.co/t/filebeat-injest-same-file-repeatedly-on-an-interval-for-logstash-filter-testing-development/119660 "2018-02-13T14:53:17Z")\
**Posts on this page:** 4\
**Page:** 1

<div class="post-metadata">

**Author:** ![D-Sect](https://avatars.discourse-cdn.com/v4/letter/d/5daacb/32.png) [@D-Sect](https://discuss.elastic.co/u/D-Sect)\
**Post date:** [February 13, 2018, 2:53pm UTC](https://discuss.elastic.co/t/filebeat-injest-same-file-repeatedly-on-an-interval-for-logstash-filter-testing-development/119660/1 "2018-02-13T14:53:17Z")

</div>

Hello,

I am looking to cut the cycle time when developing and testing Logstash configuration file code with a filebeat input.

I am working on grok patterns, filters, etc and my current test cycle involved re-ingesting sample files with filebeat and observing the Logstash stdout. This is fine, but I had to clear the filebeat data registry to reingest the same file, and I had to stop and start filebeat.

I would like my testing cycle to be:

1. Filebeat injesting same sample log in N interval,
2. logstash config auto updating, emitting to stdout

#2 is good. I have not figured out how to get FB to injest the _same_ file on N interval.

I tried some filebeat mocking w/ the generator and stdin input filters. Neither were good. The generator just filled up the stdout because I was unable to "emit line on N interval". I tried a variety of stuff w/ stdin and could not get a basic string to logstash's stdout.

I am running the ELK stack @ 6.2.1.

ideally, I would periodically like to injest one sample log file in filebeat on N interval. I am relatively new to this stack, so it's not clicking yet.

It looks like scan\_frequency will set the N interval, but I am still unable to find the prospector setting which will reingest the same file. I have a feeling it may be the clean and ignore\_older settings?

---

<div class="post-metadata">

**Author:** ![Badger](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/badger/32/25190_2.png) [@Badger](https://discuss.elastic.co/u/Badger)\
**Post date:** [February 13, 2018, 3:29pm UTC](https://discuss.elastic.co/t/filebeat-injest-same-file-repeatedly-on-an-interval-for-logstash-filter-testing-development/119660/2 "2018-02-13T15:29:29Z")

</div>

If you are not committed to filebeat then you could use an exec input in logstash that runs cat every "interval" seconds.

---

<div class="post-metadata">

**Author:** ![adrisr](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/adrisr/32/25423_2.png) [@adrisr](https://discuss.elastic.co/u/adrisr)\
**Post date:** [February 13, 2018, 3:44pm UTC](https://discuss.elastic.co/t/filebeat-injest-same-file-repeatedly-on-an-interval-for-logstash-filter-testing-development/119660/3 "2018-02-13T15:44:46Z")

</div>

Filebeat doesn't have a facility to re-injest a file after a predefined interval.

However, you can trick it to believe it's a new file, if you delete the log file and copy it from an external location.

For example if you're watching `/var/log/*.log`, every N interval you can do:

```
rm /var/log/watched.log
cp source_dir/sample.log /var/log/watched.log

```

With this the file should be getting a new inode number everytime so filebeat is tricked into believe is a new file.

---

<div class="post-metadata">

**Author:** ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)\
**Post date:** [March 6, 2018, 2:53pm UTC](https://discuss.elastic.co/t/filebeat-injest-same-file-repeatedly-on-an-interval-for-logstash-filter-testing-development/119660/4 "2018-03-06T14:53:24Z")

</div>

This topic was automatically closed after 21 days. New replies are no longer allowed.
