# Filebeat paths logic

**URL:** <https://discuss.elastic.co/t/filebeat-paths-logic/68687>\
**Category:** Beats\
**Created:** [December 12, 2016, 9:50am UTC](https://discuss.elastic.co/t/filebeat-paths-logic/68687 "2016-12-12T09:50:43Z")\
**Posts on this page:** 7\
**Page:** 1

<div class="post-metadata">

**Author:** ![Arthur\_Francis](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/arthur_francis/32/13065_2.png) [@Arthur\_Francis](https://discuss.elastic.co/u/Arthur_Francis)\
**Post date:** [December 12, 2016, 9:50am UTC](https://discuss.elastic.co/t/filebeat-paths-logic/68687/1 "2016-12-12T09:50:43Z")

</div>

Hi,

Are there any other alternatives to using a wildcard (\*) in filebeat when defining a log path. My application generates a numeric directory tree based on its activity lifecycle 34..35..36 etc

The amount of directories can get pretty high (over a thousand) and using a wildcard makes beats extremely slow to start as it can take several hours to scan all the paths. This makes it very impractical if I have config changes for example.

---

<div class="post-metadata">

**Author:** ![magnusbaeck](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/magnusbaeck/32/44943_2.png) [@magnusbaeck](https://discuss.elastic.co/u/magnusbaeck)\
**Post date:** [December 12, 2016, 9:51am UTC](https://discuss.elastic.co/t/filebeat-paths-logic/68687/2 "2016-12-12T09:51:47Z")

</div>

Perhaps you can use a cron job or similar to periodically generate a new configuration file with more exact filename patterns?

---

<div class="post-metadata">

**Author:** ![Arthur\_Francis](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/arthur_francis/32/13065_2.png) [@Arthur\_Francis](https://discuss.elastic.co/u/Arthur_Francis)\
**Post date:** [December 12, 2016, 9:55am UTC](https://discuss.elastic.co/t/filebeat-paths-logic/68687/3 "2016-12-12T09:55:11Z")

</div>

Thanks @magnusbaeck I will give that a try.

Does the **ignore\_older** setting also includes directories? Will this work in omitting the older directories which have already been processed or does it still need to do the preliminary scan?

---

<div class="post-metadata">

**Author:** ![magnusbaeck](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/magnusbaeck/32/44943_2.png) [@magnusbaeck](https://discuss.elastic.co/u/magnusbaeck)\
**Post date:** [December 12, 2016, 9:55am UTC](https://discuss.elastic.co/t/filebeat-paths-logic/68687/4 "2016-12-12T09:55:59Z")

</div>

I'd expect `ignore_older` to only apply to files.

---

<div class="post-metadata">

**Author:** ![andrewkroh](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/andrewkroh/32/3784_2.png) [@andrewkroh](https://discuss.elastic.co/u/andrewkroh)\
**Post date:** [December 12, 2016, 2:23pm UTC](https://discuss.elastic.co/t/filebeat-paths-logic/68687/5 "2016-12-12T14:23:39Z")

</div>

Here is the matching that the filepath glob supports: [https://golang.org/pkg/path/filepath/#Match](https://golang.org/pkg/path/filepath/#Match)

You can test your patterns here: [https://play.golang.org/p/rJZII1vlJW](https://play.golang.org/p/rJZII1vlJW)

---

<div class="post-metadata">

**Author:** ![Arthur\_Francis](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/arthur_francis/32/13065_2.png) [@Arthur\_Francis](https://discuss.elastic.co/u/Arthur_Francis)\
**Post date:** [December 15, 2016, 1:32pm UTC](https://discuss.elastic.co/t/filebeat-paths-logic/68687/6 "2016-12-15T13:32:50Z")

</div>

@magnusbaeck thank you, I managed to write a script that generates new config with relevant directories and reloads filebeat.

Hopefully the config reload feature will come in soon

---

<div class="post-metadata">

**Author:** ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)\
**Post date:** [January 2, 2017, 9:51am UTC](https://discuss.elastic.co/t/filebeat-paths-logic/68687/7 "2017-01-02T09:51:20Z")

</div>

This topic was automatically closed after 21 days. New replies are no longer allowed.
