Hi,
I am trying to send 3 total different logs from server A using filebeat, to server B running logstash on 3 different ports, based on different pipelines.
Server A,filebeat, file_1.log ---------> Server B, logstash with pipeline_1 on prot 5051
Server A,filebeat, file_2.log ---------> Server B, logstash with pipeline_2 on prot 6051
Server A,filebeat, file_3.log ---------> Server B, logstash with pipeline_3 on prot 7051
file_1.log <> file_2.log <> file_3.log (different log files on server A)
pipeline_1 <> pipeline_2 <> pipeline_3 (different conf files on server B)
Could I start 3 different filebeat instances on server A with 3 different filebeat.yml configurations (when installation is rpm based).
I really don't have much experience in this area. Could you please help me to resolve this problem.
I am using ELK 7.6
What you are thinking of seems to be valid.
I found a similar post too explaining the reasoning of this approach a little bit: Conditional filebeat output to logstash
Hi,
Thanks, I have solved my problem on this way (after reading documentation and from other Internet resources):
on the FILEBEAT side setup filebeat.yml to have references on my all log files (using filebeat.config.inputs attribute) and tags related wit every input file
on the LOGSTASH side setup one configuration file, now listening on on unique port and then
inside one big configuration file use "if, elseif" based on the tags, so every time we have the execution of adequate part of pipline configuration (based on its tag)
Apache, Apache Lucene, Apache Hadoop, Hadoop, HDFS and the yellow elephant
logo are trademarks of the
Apache Software Foundation
in the United States and/or other countries.