I have 4 log files from 4 completely different applications
For e.g. Log A, B, C, D
The log format of A AND B is same and
The log format of A, C, and D is different
Source of reading log A, B and C is filebeat => Logstash
And Source of reading D is UDP => Logstash
I want to create 4 different Indexes in ElasticSearch and would prefer to have 4 different pipelines to keep all the configuration separate
I'm not sure what you're asking.
I want to create 4 different Indexes in Elasticsearch
Why? Keep in mind that all shards have a fixed memory overhead so splitting your indexes in too many pieces isn't free.
and would prefer to have 4 different pipelines to keep all the configuration separate
Then just run four Logstash instances.
I/We need 4 different indexes as all 4 logs belongs to completely different applications
and we dont prefer to run/maintain multiple instance of Logstash as of now.
Thanks