[SOLVED]Slow pipeline

mikula · April 7, 2016, 9:09am

Recently (after upgrade to LS 2.3.0 and ES 2.3.0 my piplene slowed down from thousnads of entries indexed to around 50 per second...

I also started to use fingerprint plugin (twice per event) and one of figerprints is also newly my document id.
logstash.conf

gist.github.com

https://gist.github.com/samuraiii/af14ae427a8aaaf8ecd552efe4899378

logstash.conf

input {
        beats {
                port                    => 6023
                ssl                             => true
                ssl_certificate => "/etc/logstash/lumberjack.crt"
                ssl_key                 => "/etc/logstash/lumberjack.key"

        }
        ##      kvůli logstash nagios checku
        file {

This file has been truncated. show original

ES.yml

gist.github.com

https://gist.github.com/samuraiii/3c5da0c98ff5f073711182750016002a

Elasticsearch.yml

### MANAGED BY PUPPET ###
---
action: 
  destructive_requires_name: true
  disable_delete_all_indices: true
bootstrap: 
  mlockall: true
cluster: 
  name: elastic-farm
  routing:

This file has been truncated. show original

Any insights would by appreciated
AM

mikula · April 7, 2016, 4:04pm

There was some other problem in pipeline (kibana log, and other stuff)

Topic		Replies	Views
Logstash is slow for mulitiple pipelines Logstash	1	524	December 10, 2019
Pipelines Logstash	1	427	March 3, 2018
[SOLVED] Date filter slowing pipeline Logstash	5	886	July 6, 2017
Tracking slow pipelines Kibana	3	506	October 11, 2017
Slow indexing speed, possibly related to filebeat misconfiguration Beats docker , filebeat	5	1006	April 7, 2023

[SOLVED]Slow pipeline

Related topics