Create different Indices for different filebeat folder location

Elastic Stack Logstash
1

Hi There,
I have different application logs sitting in different folders, for instance
../ABC/.logs
../XYZ/.logs

In this case filebeat should take generate events for multiple location and according to these location my indexes should be created in elasticsearch.

So for above scenario elasticsearch should have two indexes with name "abc" and "xyz".

What configuration shall we have and where?

My conf file currently has below details.

input {
beats {
port => "5044"
}
}
filter {
grok{
match => {"message" => "%{TIMESTAMP_ISO8601:timestamp} [%{NUMBER:number}] %{LOGLEVEL:loglevel} %{DATA:file} - (?(.|\r|\n))"}
}
date {
match => [ "timestamp", "YYYY-MM-dd HH:mm:ss,SSS"]
}
}
output {
elasticsearch {
hosts => [ "localhost:9200" ]
user => "elastic"
password => "&^&%"
index => "abc"
}
}

Thanks,
Sumit

2

You will need to parse the source path and then use that in the output section.

3

Thanks Mark. Can you provide some example, i am not able to understand how we need to set the two paths.
I guess, you are asking to change filebeat.yml file. Do you want me to add another path in paths section? If yes, then how logstash will identify for 2nd path which new indexes needs to create?

Thanks,
Sumit.

4

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.