# Filebeat: how to define a date pattern fo prospector's path?

**URL:** <https://discuss.elastic.co/t/filebeat-how-to-define-a-date-pattern-fo-prospectors-path/87697>\
**Category:** Beats\
**Created:** [May 31, 2017, 8:50am UTC](https://discuss.elastic.co/t/filebeat-how-to-define-a-date-pattern-fo-prospectors-path/87697 "2017-05-31T08:50:52Z")\
**Posts on this page:** 6\
**Page:** 1

<div class="post-metadata">

**Author:** ![asp](https://avatars.discourse-cdn.com/v4/letter/a/9fc348/32.png) [@asp](https://discuss.elastic.co/u/asp)\
**Post date:** [May 31, 2017, 8:50am UTC](https://discuss.elastic.co/t/filebeat-how-to-define-a-date-pattern-fo-prospectors-path/87697/1 "2017-05-31T08:50:52Z")

</div>

Hi,

I have logs which are named like mylog-20170531.log.  
The logs are pre-rotated at midnight UTC.

If I define path/mylog-\*.log It would read all logfiles, regardless of the date, if they have been created / modified within a specific time.

Is there a way to explicitly define something like /path/mylog-YYYYMMDD.log ?

Thanks, Andreas

---

<div class="post-metadata">

**Author:** ![exekias](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/exekias/32/28718_2.png) [@exekias](https://discuss.elastic.co/u/exekias)\
**Post date:** [May 31, 2017, 9:06am UTC](https://discuss.elastic.co/t/filebeat-how-to-define-a-date-pattern-fo-prospectors-path/87697/2 "2017-05-31T09:06:31Z")

</div>

Hi @asp,

As far as I know there is no way to filter that much, paths field supports glob like matching: [https://www.elastic.co/guide/en/beats/filebeat/current/configuration-filebeat-options.html#prospector-paths](https://www.elastic.co/guide/en/beats/filebeat/current/configuration-filebeat-options.html#prospector-paths)

Something you could do is creating a symlink to todays log file and read always from that one, what do you think?

---

<div class="post-metadata">

**Author:** ![asp](https://avatars.discourse-cdn.com/v4/letter/a/9fc348/32.png) [@asp](https://discuss.elastic.co/u/asp)\
**Post date:** [May 31, 2017, 9:22am UTC](https://discuss.elastic.co/t/filebeat-how-to-define-a-date-pattern-fo-prospectors-path/87697/3 "2017-05-31T09:22:04Z")

</div>

maybe some more details here are interesting for you. Maybe I do not really have a problem at all.

background:  
Our application has a problem, if it needs to append to a log \> 4GB when starting. If the application is already running and then it gets over the 4GB limit, there is no problem.

So our workaround is to **rename** the old log, then the application is creating a new logfile, which will be sipped to logstash.

I need to prevent filebeat to ship the renamed file to logstash, because the data is aready in elasticsearch.  
Or do I not need to worry about it? In the past we always managaed to rename to \*.log\_, so it did not match the pattern at all, but I would like to build a waterproof setup.

symlinking may be possible, but I think we will stay on the order to rename by adding a suffix to the logfile name which is not matching the pattern.

---

<div class="post-metadata">

**Author:** ![exekias](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/exekias/32/28718_2.png) [@exekias](https://discuss.elastic.co/u/exekias)\
**Post date:** [May 31, 2017, 9:35am UTC](https://discuss.elastic.co/t/filebeat-how-to-define-a-date-pattern-fo-prospectors-path/87697/4 "2017-05-31T09:35:49Z")

</div>

If you control file names best thing to do is rotating into paths that do not match the regexp, filebeat should ignore those

---

<div class="post-metadata">

**Author:** ![asp](https://avatars.discourse-cdn.com/v4/letter/a/9fc348/32.png) [@asp](https://discuss.elastic.co/u/asp)\
**Post date:** [May 31, 2017, 9:43am UTC](https://discuss.elastic.co/t/filebeat-how-to-define-a-date-pattern-fo-prospectors-path/87697/5 "2017-05-31T09:43:50Z")

</div>

good idea, unfortunately this would disrupt our backup strategy of logs on filesystem level 😉

But I think with adding a suffix which breaks the regex, it should be fine too. Thanks

---

<div class="post-metadata">

**Author:** ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)\
**Post date:** [June 21, 2017, 8:51am UTC](https://discuss.elastic.co/t/filebeat-how-to-define-a-date-pattern-fo-prospectors-path/87697/6 "2017-06-21T08:51:03Z")

</div>

This topic was automatically closed after 21 days. New replies are no longer allowed.
