# Delay in indexing the data

**URL:** <https://discuss.elastic.co/t/delay-in-indexing-the-data/213045>\
**Category:** Elasticsearch\
**Created:** [December 26, 2019, 6:09am UTC](https://discuss.elastic.co/t/delay-in-indexing-the-data/213045 "2019-12-26T06:09:33Z")\
**Posts on this page:** 9\
**Page:** 1

<div class="post-metadata">

**Author:** ![Ozil](https://avatars.discourse-cdn.com/v4/letter/o/a587f6/32.png) [@Ozil](https://discuss.elastic.co/u/Ozil)\
**Post date:** [December 26, 2019, 6:09am UTC](https://discuss.elastic.co/t/delay-in-indexing-the-data/213045/1 "2019-12-26T06:09:33Z")

</div>

Hi All  
I am currently using 9 nodes Elasticsearch cluster in that while indexing the data.their is 5 to 10 minutes of time delay is happening. Input data EPS is about 2000 and there is no data is within the queue. How can I eliminate this delay?

---

<div class="post-metadata">

**Author:** ![Christian\_Dahlqvist](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/christian_dahlqvist/32/4617_2.png) [@Christian\_Dahlqvist](https://discuss.elastic.co/u/Christian_Dahlqvist)\
**Post date:** [December 26, 2019, 8:30am UTC](https://discuss.elastic.co/t/delay-in-indexing-the-data/213045/2 "2019-12-26T08:30:44Z")

</div>

Where is the data coming from? How are you ingesting it? What does the flow from raw data to Elasticsearch look like?

---

<div class="post-metadata">

**Author:** ![Ozil](https://avatars.discourse-cdn.com/v4/letter/o/a587f6/32.png) [@Ozil](https://discuss.elastic.co/u/Ozil)\
**Post date:** [December 31, 2019, 8:49am UTC](https://discuss.elastic.co/t/delay-in-indexing-the-data/213045/3 "2019-12-31T08:49:38Z")

</div>

Hi Christian  
i am sending the data from an automated shipper.the logs are in json format.I am ingesting the data through logstash.the raw data send to the logstash then to Elastic cluster from there to UI

---

<div class="post-metadata">

**Author:** ![Christian\_Dahlqvist](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/christian_dahlqvist/32/4617_2.png) [@Christian\_Dahlqvist](https://discuss.elastic.co/u/Christian_Dahlqvist)\
**Post date:** [December 31, 2019, 10:03am UTC](https://discuss.elastic.co/t/delay-in-indexing-the-data/213045/4 "2019-12-31T10:03:41Z")

</div>

Have you identified where the delay comes from? Logstash by default sets the `@timestamp` field to the time the event is received, so you could compare this to the event timestamp to see if the delay occurs there or whether it is later. If the delay is later, have you perhaps got a custom refresh interval set for your indices?

---

<div class="post-metadata">

**Author:** ![Ozil](https://avatars.discourse-cdn.com/v4/letter/o/a587f6/32.png) [@Ozil](https://discuss.elastic.co/u/Ozil)\
**Post date:** [December 31, 2019, 2:21pm UTC](https://discuss.elastic.co/t/delay-in-indexing-the-data/213045/5 "2019-12-31T14:21:45Z")

</div>

no i haven't checked on that.but there is no queue is happening with in threat pool.i can see the delay through the UI. while close checking the time stamp with the current machine time i am seeing the delay.

---

<div class="post-metadata">

**Author:** ![Christian\_Dahlqvist](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/christian_dahlqvist/32/4617_2.png) [@Christian\_Dahlqvist](https://discuss.elastic.co/u/Christian_Dahlqvist)\
**Post date:** [December 31, 2019, 2:53pm UTC](https://discuss.elastic.co/t/delay-in-indexing-the-data/213045/6 "2019-12-31T14:53:26Z")

</div>

Try to identify where the delay occurs. Without that it is hard to speculate. How are you seeing the delay in the UI?

---

<div class="post-metadata">

**Author:** ![Ozil](https://avatars.discourse-cdn.com/v4/letter/o/a587f6/32.png) [@Ozil](https://discuss.elastic.co/u/Ozil)\
**Post date:** [January 1, 2020, 5:58am UTC](https://discuss.elastic.co/t/delay-in-indexing-the-data/213045/7 "2020-01-01T05:58:57Z")

</div>

The attached @timestamp is showing that 5 minutes delay from the actual time.  
sure i am trying to identify the issue

---

<div class="post-metadata">

**Author:** ![Christian\_Dahlqvist](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/christian_dahlqvist/32/4617_2.png) [@Christian\_Dahlqvist](https://discuss.elastic.co/u/Christian_Dahlqvist)\
**Post date:** [January 1, 2020, 7:18am UTC](https://discuss.elastic.co/t/delay-in-indexing-the-data/213045/8 "2020-01-01T07:18:18Z")

</div>

Can you show a sample document? What does your Logstash config look like? What type of shipper is sending data to Logstash? Are your hosts using NTP to synchronise time so there is no time drift that could contribute?

If the time difference between the event and the receive time stamp in Logstash is substantial, it could indicate an issue with the shipper, time drift between hosts or Logstash resource constraints.

---

<div class="post-metadata">

**Author:** ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)\
**Post date:** [January 29, 2020, 7:18am UTC](https://discuss.elastic.co/t/delay-in-indexing-the-data/213045/9 "2020-01-29T07:18:34Z")

</div>

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.
