# Datafeed missing in ML job

**URL:** <https://discuss.elastic.co/t/datafeed-missing-in-ml-job/254343>\
**Category:** Kibana\
**Tags:** elastic-stack-machine-learning\
**Created:** [November 5, 2020, 1:57am UTC](https://discuss.elastic.co/t/datafeed-missing-in-ml-job/254343 "2020-11-05T01:57:32Z")\
**Posts on this page:** 6\
**Page:** 1

<div class="post-metadata">

**Author:** ![kvtang](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/kvtang/32/83160_2.png) [@kvtang](https://discuss.elastic.co/u/kvtang)\
**Post date:** [November 5, 2020, 1:57am UTC](https://discuss.elastic.co/t/datafeed-missing-in-ml-job/254343/1 "2020-11-05T01:57:32Z")

</div>

Hi all,

I always receive this warning on my jobs.

`Datafeed has missed 2 documents due to ingest latency.`

I have tried to increase the query\_delay but still the same.  
Any feedback?

Much appreciated!

---

<div class="post-metadata">

**Author:** ![lcawley](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/lcawley/32/58441_2.png) [@lcawley](https://discuss.elastic.co/u/lcawley)\
**Post date:** [November 5, 2020, 6:44pm UTC](https://discuss.elastic.co/t/datafeed-missing-in-ml-job/254343/2 "2020-11-05T18:44:24Z")

</div>

Hello! I think there is some good applicable advice in [Datafeed has missed documents due to ingest latency error](https://discuss.elastic.co/t/datafeed-has-missed-documents-due-to-ingest-latency-error/246016)

In particular, there are references there to

> **[Handling delayed data | Machine Learning in the Elastic Stack \[7.9\] | Elastic](https://www.elastic.co/guide/en/machine-learning/7.9/ml-delayed-data-detection.html)**

and [Calculating ingest lag and storing ingest time in Elasticsearch to improve observability | Elastic Blog](https://www.elastic.co/blog/calculating-ingest-lag-and-storing-ingest-time-in-elasticsearch-to-improve-observability)

Sophie also provided the following advice, which I think is applicable here too:

> Before you change your ingest, as a first step you could also try a manual validation that you have delayed data by running a search that will replicate what the delayed data check is trying to achieve. Assuming you have a `15m``bucket_span` and a `30m``query_delay` , create a date histogram search e.g. count of events every 15m from `now-90m` say. Manually refresh this periodically over the course of the next 90m and see if the counts change as time elapses. Pay particular attention to the counts from time buckets that are greater than 30m ago. If these are changing, this suggests an ingest latency.

---

<div class="post-metadata">

**Author:** ![kvtang](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/kvtang/32/83160_2.png) [@kvtang](https://discuss.elastic.co/u/kvtang)\
**Post date:** [November 6, 2020, 6:16am UTC](https://discuss.elastic.co/t/datafeed-missing-in-ml-job/254343/3 "2020-11-06T06:16:52Z")

</div>

Hi,

If I understand correctly, missing 2 documents seems "normal" since it is just a small number?

---

<div class="post-metadata">

**Author:** ![richcollier](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/richcollier/32/115035_2.png) [@richcollier](https://discuss.elastic.co/u/richcollier)\
**Post date:** [November 6, 2020, 12:00pm UTC](https://discuss.elastic.co/t/datafeed-missing-in-ml-job/254343/4 "2020-11-06T12:00:16Z")

</div>

Incorrect - ideally, you'd never want any documents to be missed. You need to either increase your `query_delay` so that you do not get missed documents or determine why your ingest pipeline is not keeping up with "real-time"

---

<div class="post-metadata">

**Author:** ![kvtang](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/kvtang/32/83160_2.png) [@kvtang](https://discuss.elastic.co/u/kvtang)\
**Post date:** [November 8, 2020, 4:32pm UTC](https://discuss.elastic.co/t/datafeed-missing-in-ml-job/254343/5 "2020-11-08T16:32:04Z")

</div>

For the ingest pipeline, is it at logstash layer or elasticsearch? And could it be my logstash having a complicated ruby filter?

---

<div class="post-metadata">

**Author:** ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)\
**Post date:** [December 6, 2020, 4:32pm UTC](https://discuss.elastic.co/t/datafeed-missing-in-ml-job/254343/6 "2020-12-06T16:32:06Z")

</div>

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.