Filebeat multi-path configuration for send logs to Logstash

Elastic Stack Beats
1

Hi, I have one config file in filebeat, i am trying to add two paths for reading the logs from two different locations and under feilds i have created two directories for storing the values in logstash side.
Below is the config files.:

filebeat.inputs:
-  type: log
   enabled: true
   paths:
       -  "C:\\devops\\poms\\logs\\ *.log"
       - * "C:\\test\\logs1\\ *.log"
fields_under_root: true
fields:
dir1: application1
dir2: application2</em>

output.logstash:
    enabled: true
    hosts: ["localhost:5060"]

And logstash output is:

output {
stdout { codec => rubydebug }
file {
codec => line { format => "%{message}"}
#path => ["C:\Temp\junk\logs\testdoc.log-%{+YYYY.MM.dd}"]
#path => ["C:\Temp\junk\logs%{[dir]}%{[fileName]}-%{+YYYY.MM.dd}" && "C:\Temp\junk\logs1%{[dir]}%{[fileName]}-%{+YYYY.MM.dd}"]
path => "C:\Temp\junk\logs%{[dir1]}%{[fileName]}-%{+YYYY.MM.dd}"
}
file {
codec => line { format => "{message}" }
path => "C:\Temp\junk\logs1%{[dir2]}%{[fileName]}-%{+YYYY.MM.dd}"
}

..

When i execute this, in both directories only this path files 'C:\test\logs1\ *.log" 'are reading from filebeat side and writing output from logstash.
Would like to know best solution for this.
Thanks in Advance. :slight_smile:

2

Can you check if your Filebeat log has any messages about files matching C:\devops\poms\logs\*.log, particularly with the word harvester in them?

Shaunak

3

Also, is this configuration correct?

Should there be a * after the - on the second line?

Should there be a space before the *.log on both lines?

Shaunak

4

Hi Shaunak,
Thanks for your response, yes i have removed "*" in the second line.
I have able see out directories, it only reads in this location ""C:\test\logs1\ *.log"" files.

fields_under_root: true
fields:
dir1: application1
dir2: application2

As per above config, in output side, it is creating two directories but writing only one path files in both dir's like dir1 and dir2.
My requirement is like 1st path files write in dir1 and 2nd path files write in dir2.

Hope this time you may get a clear idea of my requirement.

5

Hi Friends,
Can you please anyone help me with the above scenario?

6

Hello Friends,
Please respond to my question if anyone having idea about this?

7

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.