Good morning, I have two configuration files in Logstash conf directory:
First file:
File: firewall.conf
** Logs from Firewall **
input {
file {
path => "/var/log/remote/firewall/*-firewall.log"
ignore_older => 60
tags => ["firewall-log"]
}
}
...
...
...
output {
if "firewall-log" in [tags] {
elasticsearch {
index => "firewall-log-%{+YYYY.MM}"
hosts => ["localhost: 9200"]
}
}
Second file:
** File: beats.conf **
** Logs from auditlogbeat and winlogbeat **
input {
beats {
port => 5044
}
}
output {
elasticsearch {
hosts => ["http://localhost:9200"]
index => "%{[@metadata][beat]}-%{[@metadata][version]}-%{+YYYY.MM}"
document_type => "%{[@metadata][type]}"
}
}
With this configuration, two indexes for the firewall data appear in Kibana. The first of them firewall-log-2019-07 with the data of the log file specified in the input section of firewall.conf. The funny thing is that a second index called %{[@metadata] [beat]}-{[@metadata] [version]}-%{2019-07} is created and with the same data as firewall-log-2019.07 index. I don't undestand the reason because in the input section of beats.conf I specified beats and listen from port 5044 and however Logstash is taking data from the firewall log file.
To solve the problem I have changed the output section in beats.conf file and I have added a if clause:
output {
if "beats_input_codec_plain_applied" in [tags] {
elasticsearch {
hosts => ["http://localhost:9200"]
index => "%{[@metadata][beat]}-%{[@metadata][version]}-%{+YYYY.MM}"
document_type => "%{[@metadata][type]}"
}
}
}
I don't know If this is a good solution, but this way the index %{[@metadata] [beat]}-{[@metadata] [version]}-% {2019-07} is not created and the data from the firewall log file is loaded only in the firewall-log-2019.07 index.
Thank you!!