[SOLVED]Slow pipeline

mikula · April 7, 2016, 9:09am

Recently (after upgrade to LS 2.3.0 and ES 2.3.0 my piplene slowed down from thousnads of entries indexed to around 50 per second...

I also started to use fingerprint plugin (twice per event) and one of figerprints is also newly my document id.
logstash.conf

gist.github.com

https://gist.github.com/samuraiii/af14ae427a8aaaf8ecd552efe4899378

logstash.conf

input {
        beats {
                port                    => 6023
                ssl                             => true
                ssl_certificate => "/etc/logstash/lumberjack.crt"
                ssl_key                 => "/etc/logstash/lumberjack.key"

        }
        ##      kvůli logstash nagios checku
        file {

This file has been truncated. show original

ES.yml

gist.github.com

https://gist.github.com/samuraiii/3c5da0c98ff5f073711182750016002a

Elasticsearch.yml

### MANAGED BY PUPPET ###
---
action: 
  destructive_requires_name: true
  disable_delete_all_indices: true
bootstrap: 
  mlockall: true
cluster: 
  name: elastic-farm
  routing:

This file has been truncated. show original

Any insights would by appreciated
AM

mikula · April 7, 2016, 4:04pm

There was some other problem in pipeline (kibana log, and other stuff)