Serious data delay in elasticsearch


#1

I use logstash as shipper output to redis then logstash (indexer) to elasticsearch.

per mins i have about 500,000 docs coming in for that index.

my cluster has 5 nodes, i can see the redis queue are growing larger and larger, how do I know it is indexer not fast enough or elasticsearch's issue?


#2

This become a bottle neck and makes me headache all the time.
the problem is I dont know how to troubleshoot such issues.


(system) #3