Get miss log from server


(Tat Dat Pham) #1

I'm using Logstash 2.1.1, ES 2.1.1, Filebeat 1.01, Winlogbeat 1.2.0 nightly build.

I'm deploy my system with model : Server (agent) -> LS-forward -> RabbitMQ-LS-Indexer-ES

I checked log in my server and have 70GB log/day. But i check Indices on ES, have 40GB log.
And i check on RabbitMQ, don't have message queue.

So, why do miss log ?

Here is my config in LS-forwarder.

input {
  beats {
    port => 5044
  }
}
filter {
  mutate {
   add_field => {"beatname" => "%{[@metadata][beat]}"}
   add_field => {"beattype" => "%{[@metadata][type]}"}
  }
}
output {    
    rabbitmq {
        exchange => "logstash"
        exchange_type => "direct"
        key => "logstash-key"
        host => "10.1.6.244"
        vhost => "ELK"
        workers => 12
        durable => true
        persistent => true
        port => 5677
        user => "logstash"
        password => "***"

    }
}

In filebeat and winlogbeat. I set loadbalance with two LS-forwarder, workers = 4


(system) #2