Hi, It looks like smth block search/view of metric data in real-time mode (from otel-collectors) and produces this data incrementally. For example, on 12 June I could check data till 01:00, but today at 13 June I could see data till 07:00 12 June.
I checked otel-collectors logs and looks no issue on data push to ELK.
ELK stack version: 8.13.2 (elastic cloud)
Hi @iamp3
I would start to investigate whether the data is actually stored/retrieved from elasticsearch.
To check that, click on the top right of the first panel, on the ... and check the Inspect action: from there switch to Request and get the full request sent to ES and try that out in the DevTools
Hi @Marco_Liberati, thanks for help!
I looked at the data\requests section and saw that we have periodic null values, after restarting otel-collectors there is usually a window of 10-15 minutes until the data comes in +- stable stream, after that the null values start appearing. Is this related to the otel-collectors configuration or could it be something on the APM events intake API side? I changed the batch size parameter to a higher value on the otel-collectors side and that seems to have helped a bit, but the problem with null values 15-20 minutes after otel-colletors start/restarts exists=(
In general you can see that periodically the data flow on one particular node is unstable, as I understand the data refresh process is every 30 seconds
UPD: looks like on otel-collectors side we got this (potential cause of null values), could you advice how we could fix it?
Yes, that looks more a data collection problem rather than Kibana.
To to ask to Elasticsearch or Elastic Agent forum
© 2020. All Rights Reserved - Elasticsearch
Apache, Apache Lucene, Apache Hadoop, Hadoop, HDFS and the yellow elephant logo are trademarks of the Apache Software Foundation in the United States and/or other countries.
