Kibana produces data view incrementally

[iamp3]

Hi, It looks like smth block search/view of metric data in real-time mode (from otel-collectors) and produces this data incrementally. For example, on 12 June I could check data till 01:00, but today at 13 June I could see data till 07:00 12 June.

I checked otel-collectors logs and looks no issue on data push to ELK.

ELK stack version: 8.13.2 (elastic cloud)

[Marco_Liberati]

Hi @iamp3

I would start to investigate whether the data is actually stored/retrieved from elasticsearch.
To check that, click on the top right of the first panel, on the ... and check the Inspect action: from there switch to Request and get the full request sent to ES and try that out in the DevTools

[iamp3]

Hi @Marco_Liberati, thanks for help!
I looked at the data\requests section and saw that we have periodic null values, after restarting otel-collectors there is usually a window of 10-15 minutes until the data comes in +- stable stream, after that the null values start appearing. Is this related to the otel-collectors configuration or could it be something on the APM events intake API side? I changed the batch size parameter to a higher value on the otel-collectors side and that seems to have helped a bit, but the problem with null values 15-20 minutes after otel-colletors start/restarts exists=(
In general you can see that periodically the data flow on one particular node is unstable, as I understand the data refresh process is every 30 seconds

UPD: looks like on otel-collectors side we got this (potential cause of null values), could you advice how we could fix it?

[Marco_Liberati]

Yes, that looks more a data collection problem rather than Kibana.
To to ask to Elasticsearch or Elastic Agent forum