Logs that are pushed from filebeat aren't showing up in logstash.
My pipeline from logstash to elasticsearch works, as metric beat and heartbeat are getting pushed through, but my logs from filebeat are disappearing into thin air.
Here is my filebeat config:
filebeat.prospectors:
- paths:
- /usr/local/apache-tomcat-8.0.35/logs/*.log
tags: ["Company-Application"]
fields:
type: company-logs
- /usr/local/apache-tomcat-8.0.35/logs/*.log
output.logstash:
hosts: ["elk-server:10200"]
bulk_max_size: 1024
Logstash config:
input {
beats {
port => {{ logstash_beats_port }}
}
}
filter {
if [type] == "company-logs" {
json {
source => message
}
}
}
filter {
if "Company-Application" in [tags] {
json {
source => message
}
}
}
filter {
if [fields][log_type] == "company-logs" {
json {
source => message
}
}
}
output {
elasticsearch {
hosts => ["localhost:9200"]
index => "%{[@metadata][beat]}-%{+YYYY.MM.dd}"
document_type => "%{[@metadata][type]}"
}
}
I'm not getting any errors in the filebeat log or the logstash log. So I assume that this is an issue with how I am filtering the information, or something along those lines.