I'm using Filebeat to collect custom logs and Metricbeat to monitor system (CPU, RAM, DISK SPACE). I'm able to do this without using if condition and if I'm using if conduction in the logstash filter then it is not working at all.
Below is my logstash beat-conf.conf file
input {
beats {
port => 5044
}
}
filter {
if [log_type] == "apache" {
grok {
match => { "message" => "^%{IP:CLIENT_IP} (?:-|%{USER:IDEN}) (?:-|%{USER:AUTH}) \[%{HTTPDATE:CREATED_ON}\] \"(?:%{WORD:REQUEST_METHOD} (?:/|%{NOTSPACE:REQUEST})(?: HTTP/%{NUMBER:HTTP_VERSION})?|-)\" %{NUMBER:RESPONSE_CODE}%{SPACE} (?:-|%{NOTSPACE:BYTE})%{SPACE}(?:-|%{NOTSPACE:EXECUTION_TIME})"}
add_field => {
"LOG_TYPES" => "apache-log"
}
overwrite => [ "message" ]
}
}
}
output {
if "apache" in [log_type] or [_type] in ["metricsets","metricbeat"] {
elasticsearch {
hosts => ["localhost:9200"]
manage_template => false
index => "metricbeat-index"
}
stdout { codec => rubydebug }
}
}
I want to use if conduction because in local I have only a limited source of the log. But in production, I have 4 machines and every machine is having different log pattern to another machine.