Multiple indexnames from 1 filebeat instance with ilm

[Marcel_van_Zoggel]

I want to use filebeat on a server with 2 different logfolders. Each folder must go to a different indexname. This i can set via a custom field per input (fe. Field log_type) in combination with

output.elasticsearch.index: "index-%{[fields.log_type]:other}-%{+yyyy.MM.dd}"

For this to work i have to disable ILM.

I want to use ILM to setup different lifetime policy for the inputs.

How do i set this up ?

[RLPowellJr (Bob Powell)]

Since ILM doesn't work with the dynamic variable substitution, you may need to create a conditional. Assuming you have two log types, poplar and pine, you could try (note this isn't correct syntax/grammar, just an illustration of the idea):

if fields.log_type = poplar
then output.elasticsearch.index: "index-poplar"
else output.elasticsearch.index: "index-pine"

[Marcel27 (marcel)]

@RLPowellJr

output.elasticsearch.index

Thanks, but isn't output.elasticsearch.index ignored when ILM is enabled !?

[Marcel27 (marcel)]

Is 2 filebeat applications running on same machine an (the only) option ?
(when need for 2 different indexnames with ILM enabled) ?

[RLPowellJr (Bob Powell)]

I was pretty sure you could configure Filebeat to support what you want to do, but that doesn't seem possible. I confused myself because I'm using Logstash to route to different indices, as described at Elasticsearch output plugin | Logstash Reference [7.12] | Elastic

I don't know if two instances of Filebeat can run at the same time.