# Data Drops

**URL:** <https://discuss.elastic.co/t/data-drops/177494>\
**Category:** Elasticsearch\
**Created:** [April 18, 2019, 2:56pm UTC](https://discuss.elastic.co/t/data-drops/177494 "2019-04-18T14:56:54Z")\
**Posts on this page:** 8\
**Page:** 1

<div class="post-metadata">

**Author:** ![aniaks](https://avatars.discourse-cdn.com/v4/letter/a/ac8455/32.png) [@aniaks](https://discuss.elastic.co/u/aniaks)\
**Post date:** [April 18, 2019, 2:56pm UTC](https://discuss.elastic.co/t/data-drops/177494/1 "2019-04-18T14:56:54Z")

</div>

We are seeing data drops intermittently while being written into Elastic. However, when we check for bulk exceptions it doesn't correlate to the data drop times.

is there any other place or logs to look for? any help to determine and analyze this situation would be helpful.

---

<div class="post-metadata">

**Author:** ![warkolm](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/warkolm/32/39224_2.png) [@warkolm](https://discuss.elastic.co/u/warkolm)\
**Post date:** [April 19, 2019, 9:40pm UTC](https://discuss.elastic.co/t/data-drops/177494/2 "2019-04-19T21:40:23Z")

</div>

Seeing them how?  
How is data being sent to Elasticsearch?  
Do you have Monitoring enabled?

---

<div class="post-metadata">

**Author:** ![aniaks](https://avatars.discourse-cdn.com/v4/letter/a/ac8455/32.png) [@aniaks](https://discuss.elastic.co/u/aniaks)\
**Post date:** [April 25, 2019, 3:31pm UTC](https://discuss.elastic.co/t/data-drops/177494/3 "2019-04-25T15:31:03Z")

</div>

we do not see anything in the logs. However we do notice some data being lost.  
We have monitoring enabled and it also doesn't show any signs of data loss. Which is strange, hence would be good to know if anyway to get this information.

---

<div class="post-metadata">

**Author:** ![Christian\_Dahlqvist](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/christian_dahlqvist/32/4617_2.png) [@Christian\_Dahlqvist](https://discuss.elastic.co/u/Christian_Dahlqvist)\
**Post date:** [April 25, 2019, 4:44pm UTC](https://discuss.elastic.co/t/data-drops/177494/4 "2019-04-25T16:44:02Z")

</div>

> [@warkolm](#):
>
> How is data being sent to Elasticsearch?

It would help if you provided some additional detail on how you are getting your data into Elasticsearch. Without this it is impossible to speculate about what is happening.

---

<div class="post-metadata">

**Author:** ![aniaks](https://avatars.discourse-cdn.com/v4/letter/a/ac8455/32.png) [@aniaks](https://discuss.elastic.co/u/aniaks)\
**Post date:** [April 25, 2019, 5:20pm UTC](https://discuss.elastic.co/t/data-drops/177494/5 "2019-04-25T17:20:53Z")

</div>

data sent through bulk upserts from our upstream system.

---

<div class="post-metadata">

**Author:** ![Christian\_Dahlqvist](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/christian_dahlqvist/32/4617_2.png) [@Christian\_Dahlqvist](https://discuss.elastic.co/u/Christian_Dahlqvist)\
**Post date:** [April 25, 2019, 5:42pm UTC](https://discuss.elastic.co/t/data-drops/177494/6 "2019-04-25T17:42:39Z")

</div>

Are you checking the bulk request response for any errors and retrying if you encounter any? How do you identify that data is being dropped?

---

<div class="post-metadata">

**Author:** ![aniaks](https://avatars.discourse-cdn.com/v4/letter/a/ac8455/32.png) [@aniaks](https://discuss.elastic.co/u/aniaks)\
**Post date:** [April 25, 2019, 6:25pm UTC](https://discuss.elastic.co/t/data-drops/177494/7 "2019-04-25T18:25:18Z")

</div>

there is no issues in the response to bulk request. However when we see the chart we see gaps making us believe the data being dropped. However none of the logs shows otherwise. Hence not sure how to confirm there are no issues on the Elastic that could cause data drop

---

<div class="post-metadata">

**Author:** ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)\
**Post date:** [May 23, 2019, 6:25pm UTC](https://discuss.elastic.co/t/data-drops/177494/8 "2019-05-23T18:25:20Z")

</div>

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.
