Path params problem

[laywin]

hello! we recently have a demand which is that we want to get matched path params.

example:

we config filebeat path like: /var/log//.log, matched path is : /var/log/aa/bb.log

we want to get 'aa' and 'bb' params to be our index name.

logstash grok filter can achieve this,but instead,we want to use filebeat.

search for a help or suggest,thanks in advance!

[andrewkroh]

You would have to parse the source field in either Logstash or Ingest Node and then set the index based on that. You cannot parse the filename directly in Filebeat.

Depending on the number of permutations you have you could take a simpler approach. The example below would set the index based on a static field defined in the prospector config.

filebeat.prospectors:
- paths: ['/var/log/aa/*.log']
  fields: {index: aa}
- paths: ['/var/log/bb/*.log']
  fields: {index: bb}

output.elasticsearch:
  hosts: ['localhost:9200']
  index: '%{[fields.index]}-%{+yyyy.MM.dd}'

[laywin]

static field is not so flexible,because there are many directory in /var/log/,currently,we use logstash to achieve this,logstash parse source can also slove the problem. could filebeat provide a way to achieve this in the future,thus we will use logstash less often!

thanks your reply!