Multiple Filebeat Inputs to different Logstash ports

Elastic Stack Beats
1

Hi,
I am trying to send 3 total different logs from server A using filebeat, to server B running logstash on 3 different ports, based on different pipelines.

Server A,filebeat, file_1.log ---------> Server B, logstash with pipeline_1 on prot 5051
Server A,filebeat, file_2.log ---------> Server B, logstash with pipeline_2 on prot 6051
Server A,filebeat, file_3.log ---------> Server B, logstash with pipeline_3 on prot 7051

file_1.log <> file_2.log <> file_3.log (different log files on server A)
pipeline_1 <> pipeline_2 <> pipeline_3 (different conf files on server B)

Could I start 3 different filebeat instances on server A with 3 different filebeat.yml configurations (when installation is rpm based).

I really don't have much experience in this area. Could you please help me to resolve this problem.
I am using ELK 7.6

Many thanks,
rd

2

Hey!

What you are thinking of seems to be valid.
I found a similar post too explaining the reasoning of this approach a little bit: Conditional filebeat output to logstash

C.

3

Hi,
Thanks, I have solved my problem on this way (after reading documentation and from other Internet resources):

  • on the FILEBEAT side setup filebeat.yml to have references on my all log files (using filebeat.config.inputs attribute) and tags related wit every input file
  • on the LOGSTASH side setup one configuration file, now listening on on unique port and then
    inside one big configuration file use "if, elseif" based on the tags, so every time we have the execution of adequate part of pipline configuration (based on its tag)

Many thanks,
rd