I'm using Logstash 2.1.1, ES 2.1.1, Filebeat 1.01, Winlogbeat 1.2.0 nightly build.
I'm deploy my system with model : Server (agent) -> LS-forward -> RabbitMQ-LS-Indexer-ES
I checked log in my server and have 70GB log/day. But i check Indices on ES, have 40GB log.
And i check on RabbitMQ, don't have message queue.
So, why do miss log ?
Here is my config in LS-forwarder.
input {
beats {
port => 5044
}
}
filter {
mutate {
add_field => {"beatname" => "%{[@metadata][beat]}"}
add_field => {"beattype" => "%{[@metadata][type]}"}
}
}
output {
rabbitmq {
exchange => "logstash"
exchange_type => "direct"
key => "logstash-key"
host => "10.1.6.244"
vhost => "ELK"
workers => 12
durable => true
persistent => true
port => 5677
user => "logstash"
password => "***"
}
}
In filebeat and winlogbeat. I set loadbalance with two LS-forwarder, workers = 4