How to define multiple indexes for different logs in Filebeat and Logstash?


#1

Hi, I have configured a log server, i.e. Filebeat -> Logstash -> Elasticsearch -> Kibana. I am wondering how to create separated indexes for different logs (fetched using Filebeat) in logstash (which were later passed onto elasticsearch), so that in kibana, I can define two indexes for them and discover them.

In my case, I have a few client servers (each of which is installed with filebeat) and a centralized log server (ELK). Each client server has different kinds of logs, e.g. redis.log, python logs, mongodb logs, that I like to sort them into different indexes and stored in elasticsearch.

Each client server also serves different purposes, e.g. databases, UIs, applications. Hence I also like to give them different names to indicate the sources of the indexes (by changing output index in filebeat.yml?). So redis.log fetched from a production server will have an index name like prod.redis.

cheers


(Mark Walkom) #2

Take a look at https://www.elastic.co/guide/en/logstash/current/event-dependent-configuration.html, it's exactly what you want to do.


(system) #3

This topic was automatically closed after 21 days. New replies are no longer allowed.