How to use filename as index wiht beats input

Elastic Stack Logstash
1

Hey!

How can I use the filename as an index when using beats input for multiple files?

My filebeat.yml looks currently like this:

filebeat.prospectors:
- type: log
  paths:
    - /path/${CURRENT_SERVER}/*log*
  fields:
     index:  ${CURRENT_SERVER}-${DATE_USED}
  close_eof: true
output.logstash:
  hosts: ["localhost:5044"]
output.console:
  enabled: false

and pipeline.conf like this:

input {

beats {
        port => "5044"
    }

}
filter {
    grok {
        match => { "message" => "%{LOGGERLEVEL:log}%{LOGTYPE:k}%{TIMESTAMP_ISO8601:datetime}%{GREEDYDATA:data}"}
    }
    date {
        match => ["datetime", "ISO8601"]
        timezone => TIMEZONE
    }

}
output {

    elasticsearch {
    	index => "%{[fields][index]}"
        hosts => ["localhost:9200"]
    }

}

recently this approach worked just fine when I had one file in a folder, but now I have two or three files in a folder. How can I extract the filename and use it as index?

2

The event should have a source field.

You can use this field index => "%{[source]}"

3

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.