# Huge Time Delay between logstash and elasticsearch **URL:** \ **Category:** Logstash\ **Created:** [March 28, 2016, 7:08am UTC](https://discuss.elastic.co/t/huge-time-delay-between-logstash-and-elasticsearch/45560 "2016-03-28T07:08:37Z")\ **Posts on this page:** 6\ **Page:** 1
**Author:** ![ApoorvaLad](https://avatars.discourse-cdn.com/v4/letter/a/7ba0ec/32.png) [@ApoorvaLad](https://discuss.elastic.co/u/ApoorvaLad)\ **Post date:** [March 28, 2016, 7:08am UTC](https://discuss.elastic.co/t/huge-time-delay-between-logstash-and-elasticsearch/45560/1 "2016-03-28T07:08:37Z")
My current pipeline is: Rsyslog -\> Kafka -\> Logstash -\> ES (5 nodes) I see a huge time delay (Arnd 10hrs) between the logs processed by Kafka and the logs in my ES cluster. the input and output plugin in logstash looks like this: input { kafka { zk\_connect =\> 'host:port' topic\_id =\> 'abc' consumer\_threads=\> 50 codec =\> json } } Kafka currently has 50 partitions. output { elasticsearch { template =\> "/export/logstash\_new/elasticsearch-template.json" hosts =\>["host1","host2","host3","host4","host5"] template\_overwrite =\> true manage\_template =\> true codec=\>plain } } When I run my pipeline for a shorter duration (2 to 3 hours), no time delay is noticed. However, gradually, as the time increases, the delay increases too. How can I figure out where the problem currently resides? Is logstash failing to process data? Or is there a problem in the indexing of Elasticsearch? Current load of data is 6k messages per minute. (The load fluctuates) ---
**Author:** ![warkolm](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/warkolm/32/39224_2.png) [@warkolm](https://discuss.elastic.co/u/warkolm)\ **Post date:** [March 28, 2016, 7:40pm UTC](https://discuss.elastic.co/t/huge-time-delay-between-logstash-and-elasticsearch/45560/2 "2016-03-28T19:40:12Z")
Are you monitoring your kafka topic lengths, as well as the rest of the pipeline (CPU etc)? ---
**Author:** ![ApoorvaLad](https://avatars.discourse-cdn.com/v4/letter/a/7ba0ec/32.png) [@ApoorvaLad](https://discuss.elastic.co/u/ApoorvaLad)\ **Post date:** [March 30, 2016, 6:03am UTC](https://discuss.elastic.co/t/huge-time-delay-between-logstash-and-elasticsearch/45560/3 "2016-03-30T06:03:18Z")
Hi.. The problem got solved. The issue was in the indexing of Elasticsearch. There were 5 nodes in the cluster. Out of which 4 of them were data nodes and 1 was master as well as data node. In the logstash output plugin, I had given all 5 nodes. The issue was solved when I created a separate client node for the communication between Logstash . So now my ES cluster looks like this: 1 Client node (Role - Handing requests, load balancing) 1 Master node (Role - Cluster health management) 3 Data Nodes ---
**Author:** ![Christian\_Dahlqvist](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/christian_dahlqvist/32/4617_2.png) [@Christian\_Dahlqvist](https://discuss.elastic.co/u/Christian_Dahlqvist)\ **Post date:** [March 30, 2016, 6:40am UTC](https://discuss.elastic.co/t/huge-time-delay-between-logstash-and-elasticsearch/45560/4 "2016-03-30T06:40:24Z")
Running with a single master eligible node creates a single point of failure. You should look to have 3 master eligible nodes in the cluster (with minimum\_master\_nodes set to 2) in order to improve resiliency and availability. ---
**Author:** ![ApoorvaLad](https://avatars.discourse-cdn.com/v4/letter/a/7ba0ec/32.png) [@ApoorvaLad](https://discuss.elastic.co/u/ApoorvaLad)\ **Post date:** [March 30, 2016, 6:52am UTC](https://discuss.elastic.co/t/huge-time-delay-between-logstash-and-elasticsearch/45560/5 "2016-03-30T06:52:23Z")
Okay... Thanks .. ---
**Author:** ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)\ **Post date:** [July 6, 2017, 5:04am UTC](https://discuss.elastic.co/t/huge-time-delay-between-logstash-and-elasticsearch/45560/6 "2017-07-06T05:04:49Z")