# Kibana produces data view incrementally

**URL:** <https://discuss.elastic.co/t/kibana-produces-data-view-incrementally/361402>\
**Category:** Kibana\
**Tags:** kql-kibana-query-language, datastreams, data-views\
**Created:** [June 13, 2024, 12:09pm UTC](https://discuss.elastic.co/t/kibana-produces-data-view-incrementally/361402 "2024-06-13T12:09:12Z")\
**Posts on this page:** 4\
**Page:** 1

<div class="post-metadata">

**Author:** ![iamp3](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/iamp3/32/125051_2.png) [@iamp3](https://discuss.elastic.co/u/iamp3)\
**Post date:** [June 13, 2024, 12:09pm UTC](https://discuss.elastic.co/t/kibana-produces-data-view-incrementally/361402/1 "2024-06-13T12:09:12Z")

</div>

Hi, It looks like smth block search/view of metric data in real-time mode (from otel-collectors) and produces this data incrementally. For example, on 12 June I could check data till 01:00, but today at 13 June I could see data till 07:00 12 June.

 ![image](https://us1.discourse-cdn.com/elastic/original/3X/5/6/56ed2162510f28c5f05efcff04d76ef0a0ec9564.png)  
 ![image](https://us1.discourse-cdn.com/elastic/original/3X/c/5/c5bd6397e7eb64f6f0ea9c0b159b68d977091d0c.jpeg)

I checked otel-collectors logs and looks no issue on data push to ELK.

ELK stack version: 8.13.2 (elastic cloud)

---

<div class="post-metadata">

**Author:** ![Marco\_Liberati](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/marco_liberati/32/82953_2.png) [@Marco\_Liberati](https://discuss.elastic.co/u/Marco_Liberati)\
**Post date:** [June 13, 2024, 1:24pm UTC](https://discuss.elastic.co/t/kibana-produces-data-view-incrementally/361402/2 "2024-06-13T13:24:37Z")

</div>

Hi @iamp3

I would start to investigate whether the data is actually stored/retrieved from elasticsearch.  
To check that, click on the top right of the first panel, on the `...` and check the `Inspect` action: from there switch to `Request` and get the full request sent to ES and try that out in the DevTools

---

<div class="post-metadata">

**Author:** ![iamp3](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/iamp3/32/125051_2.png) [@iamp3](https://discuss.elastic.co/u/iamp3)\
**Post date:** [June 13, 2024, 2:23pm UTC](https://discuss.elastic.co/t/kibana-produces-data-view-incrementally/361402/3 "2024-06-13T14:23:43Z")

</div>

Hi @Marco_Liberati, thanks for help!  
I looked at the data\requests section and saw that we have periodic null values, after restarting otel-collectors there is usually a window of 10-15 minutes until the data comes in +- stable stream, after that the null values start appearing. Is this related to the otel-collectors configuration or could it be something on the APM events intake API side? I changed the batch size parameter to a higher value on the otel-collectors side and that seems to have helped a bit, but the problem with null values 15-20 minutes after otel-colletors start/restarts exists=(  
In general you can see that periodically the data flow on one particular node is unstable, as I understand the data refresh process is every 30 seconds

 ![image](https://us1.discourse-cdn.com/elastic/original/3X/5/6/56ba52b29596a3552eaa8b3edf20dafe2d6b7510.png)  
 ![image](https://us1.discourse-cdn.com/elastic/original/3X/4/3/43960332e651706a72bf207d5edac9ec8f33da5a.png)

UPD: looks like on otel-collectors side we got this (potential cause of null values), could you advice how we could fix it?

 ![image](https://us1.discourse-cdn.com/elastic/original/3X/c/c/cc5a97d7194f43b41147f0da3cb36e996eeb32d6.jpeg)

---

<div class="post-metadata">

**Author:** ![Marco\_Liberati](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/marco_liberati/32/82953_2.png) [@Marco\_Liberati](https://discuss.elastic.co/u/Marco_Liberati)\
**Post date:** [June 17, 2024, 9:43am UTC](https://discuss.elastic.co/t/kibana-produces-data-view-incrementally/361402/4 "2024-06-17T09:43:10Z")

</div>

Yes, that looks more a data collection problem rather than Kibana.  
To to ask to #Elasticsearch or #Elastic Agent forum
