Redis, Logstash indexer, elasticsearch bottleneck

(Kiran Tella) #1

I have made a setup of ELK like below.
LS(reading from file) --> REDIS --> LS(indexer) --> ELS

I'm using a bench mark script from here to generate a 1M logs lines to measure performance.

  1. I'm monitoring redis list length with help of LLEN command
    /usr/local/bin/redis-cli -h LLEN "logstash-redis"
  2. I'm monitoring the records in ELS with ruby script which uses stats page, which prints COUNT for every second.

What i have observed is that REDIS LIST is empty very quick almost in a ~65 seconds. I can only see only ~60% which ~600K records in ELS.

I'm trying to find the bottleneck here.
Does that mean 400K records are still in LS-indexer queue or in ELS ? How can i speed the things up ?

Is it possible its to do with performance of ES indexing ?

(system) #2