# Using wildcarded filenames in the logstash filter section **URL:** \ **Category:** Logstash\ **Created:** [April 24, 2018, 5:03pm UTC](https://discuss.elastic.co/t/using-wildcarded-filenames-in-the-logstash-filter-section/129356 "2018-04-24T17:03:35Z")\ **Posts on this page:** 9\ **Page:** 1
**Author:** ![camerodity](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/camerodity/32/29818_2.png) [@camerodity](https://discuss.elastic.co/u/camerodity)\ **Post date:** [April 24, 2018, 5:03pm UTC](https://discuss.elastic.co/t/using-wildcarded-filenames-in-the-logstash-filter-section/129356/1 "2018-04-24T17:03:35Z")
Is it possible to use wildcarded filenames in the filter plugin section of logstash? For example, I want to index Apache access logs for some of our webapps and since the apache access logs have the date as part of the active filename (localhost\_access\_log..txt, I am trying to use wildcards. I create index-names based on the source of the log, so in my filter section I have many source entries like: filter { if [source] == "/path/to/logfile" { mutate { add\_field =\> { 'app\_type' =\> 'some\_app\_type' } } } This works for all of my sources except the apache access logs with the wildcard. if [source] == "/path/to/apache/logs/localhost\_access\_log\*.txt" { mutate { add\_field =\> { 'app\_type' =\> 'some\_app\_type' } } } This filter doesnt seem to hit and the Access logs go into a default catchall index. In Kibana I can view the index and the log file entries just fine, even with the source showing up correctly as '/path/to/apache/logs/localhost\_access\_log..txt Any ideas on what I may be doing wrong? ---
**Author:** ![magnusbaeck](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/magnusbaeck/32/44943_2.png) [@magnusbaeck](https://discuss.elastic.co/u/magnusbaeck)\ **Post date:** [April 24, 2018, 5:05pm UTC](https://discuss.elastic.co/t/using-wildcarded-filenames-in-the-logstash-filter-section/129356/2 "2018-04-24T17:05:46Z")
> Is it possible to use wildcarded filenames in the filter plugin section of logstash? No, but you can perform regular expression matches, see [Accessing event data and fields | Logstash Reference [8.11] | Elastic](https://www.elastic.co/guide/en/logstash/current/event-dependent-configuration.html). ---
**Author:** ![camerodity](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/camerodity/32/29818_2.png) [@camerodity](https://discuss.elastic.co/u/camerodity)\ **Post date:** [April 24, 2018, 5:57pm UTC](https://discuss.elastic.co/t/using-wildcarded-filenames-in-the-logstash-filter-section/129356/3 "2018-04-24T17:57:55Z")
Thanks! I was able to get it to work with if [source] == "/path/to/apache/logs/localhost\_access\_log.%{YYYY-MM-dd}.txt" {..... ---
**Author:** ![camerodity](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/camerodity/32/29818_2.png) [@camerodity](https://discuss.elastic.co/u/camerodity)\ **Post date:** [April 24, 2018, 7:10pm UTC](https://discuss.elastic.co/t/using-wildcarded-filenames-in-the-logstash-filter-section/129356/4 "2018-04-24T19:10:30Z")
Ok, maybe I was wrong. Doesnt seem to be working. Is if [source] == "/path/to/apache/logs/localhost\_access\_log.%{YYYY-MM-dd}.txt" the correct syntax for a regex? ---
**Author:** ![magnusbaeck](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/magnusbaeck/32/44943_2.png) [@magnusbaeck](https://discuss.elastic.co/u/magnusbaeck)\ **Post date:** [April 24, 2018, 7:14pm UTC](https://discuss.elastic.co/t/using-wildcarded-filenames-in-the-logstash-filter-section/129356/5 "2018-04-24T19:14:55Z")
No, you need something like ``` if [source] =~ "/path/to/apache/logs/localhost_access_log.*\.txt$" { ``` but the best option is to add the desired fields and/or tags at the one place where all of this is known, namely in the input section. You have one input per log type anyway, right? ---
**Author:** ![camerodity](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/camerodity/32/29818_2.png) [@camerodity](https://discuss.elastic.co/u/camerodity)\ **Post date:** [April 24, 2018, 7:27pm UTC](https://discuss.elastic.co/t/using-wildcarded-filenames-in-the-logstash-filter-section/129356/6 "2018-04-24T19:27:08Z")
We are using filebeats for the input so I just have that one input: input { beats { port =\>"5044" type =\> "log" } } In the filter section, based on the app log source I add a field that is then used in the output section for creating the index. For example: filter { if [source] == '/app/log/app1.log' { mutate { add\_field =\> { 'app\_type' =\> 'app1'} } } if [source] == '/app/log/app2.log' { mutate { add\_field =\> { 'app\_type' =\> 'app2'} } } output { elasticsearch { hosts =\> [] index =\> "%{app\_type}-%{+YYYY.MM.dd}" } } ---
**Author:** ![magnusbaeck](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/magnusbaeck/32/44943_2.png) [@magnusbaeck](https://discuss.elastic.co/u/magnusbaeck)\ **Post date:** [April 24, 2018, 7:32pm UTC](https://discuss.elastic.co/t/using-wildcarded-filenames-in-the-logstash-filter-section/129356/7 "2018-04-24T19:32:19Z")
> We are using filebeats for the input so I just have that one input: Well, set the field and/or tags on the Filebeat side then. Same thing. The Logstash configuration shouldn't have an exact dependency to the file paths in the Filebeat configuration. ---
**Author:** ![camerodity](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/camerodity/32/29818_2.png) [@camerodity](https://discuss.elastic.co/u/camerodity)\ **Post date:** [April 24, 2018, 7:40pm UTC](https://discuss.elastic.co/t/using-wildcarded-filenames-in-the-logstash-filter-section/129356/8 "2018-04-24T19:40:24Z")
I tested adding a field on the Filebeat side e.g. fields: field\_name: apache\_access Then on the logstash side under filters if [field\_name] == 'apache\_access' {. Up til now, every single log from beats has a configuration in the filter section in order to apply grok filters and such. I'm sure this is not optimal as the current pipeline is growing large ---
**Author:** ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)\ **Post date:** [May 22, 2018, 7:40pm UTC](https://discuss.elastic.co/t/using-wildcarded-filenames-in-the-logstash-filter-section/129356/9 "2018-05-22T19:40:28Z")
This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.