Elastic Machine learning, Datafeed has missed xxx documents due to ingest latency

Elastic Stack Kibana
1

Hi,

I currenly have a APM server which running on cloud and receiving my php app transaction log.
Basiclly, request will keep sending to my php api endpoint.

And, I am using the APM generated index pattern log to feed my machine learning but no matter how long I set the delayed query, it still saying that there is docs missing due to injest laterncy.

I have no clue how I can fix this issue since I was hoping to have a real time datafeed, but now even I delay the query to 30 minutes and still having this issue.

1
11837×713 82.9 KB

2
2556×540 10.9 KB

3
3928×427 23.7 KB

update
I increased the query delay to 50mimutes, and still facing the same issue.
I wonder if this could be due to the spec of instance which use to do machine learning job
Here is some config indetails:

4
41826×778 95.2 KB

5
51570×419 44.8 KB

2

Hello @JasonREC ,

to troubleshoot, consider running the query from your datafeed and observe the difference between @timestamp and ingest_time. This will give you the feeling of what query_delay should be on your system.

3

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.