# Not getting the latest logs in kibana

**URL:** https://discuss.elastic.co/t/not-getting-the-latest-logs-in-kibana/306167
**Category:** Kibana
**Created:** [June 1, 2022, 9:51pm UTC](https://discuss.elastic.co/t/not-getting-the-latest-logs-in-kibana/306167 "2022-06-01T21:51:33Z")
**Posts on this page:** 7
**Page:** 1

<div class="post-metadata">

### Author: ![Dev220](https://avatars.discourse-cdn.com/v4/letter/d/b5a626/32.png) [@Dev220](https://discuss.elastic.co/u/Dev220)
#### Post date: [June 1, 2022, 9:51pm UTC](https://discuss.elastic.co/t/not-getting-the-latest-logs-in-kibana/306167/1 "2022-06-01T21:51:33Z")

</div>

Hi Team,

I am new to kibana.  
For collecting the logs from my application pods I am using fluent-bit configuration as a pod.  
Both the Elasticsearch and kiban services are up and running.  
I am able to see the logs in kibana dashboard but not the latest ones.  
For some pods I am seeing 1day older logs in kibana dashboard although recent logs are present inside the pods.  
And for some other pods I am seeing logs in 30 minutes delay.

So my issue is I am unable to see the latest logs in Kibana dashboard.  
I have tried recreating the fluent bit pod by changing the fluent bit image.  
After 10 minutes of re-deploying the fluent bit pod I am still facing the same issue.

Can any body help me in this.  
Thank you

---

<div class="post-metadata">

### Author: ![stephenb](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/stephenb/32/40856_2.png) [@stephenb](https://discuss.elastic.co/u/stephenb)
#### Post date: [June 1, 2022, 10:06pm UTC](https://discuss.elastic.co/t/not-getting-the-latest-logs-in-kibana/306167/2 "2022-06-01T22:06:58Z")

</div>

Hi @Dev220 Welcome to the community.

At the macro level you may be overloading your elasticseach cluster... or perhaps you are overloading your fluent components?

But we can not know until you tell us a bit more about your cluster and the amount of logs you are shipping.

What is the size and configuration of your Elasticsearch cluster?

What kind / how much of HW / Storage / RAM / CPU?

About how many logs / minute are you shipping? What kind of logs?

How many Pods and fluent-bits (although we are not experts on that?)

Is it straight from fluent-bit -\> Elasticsearch or are there other components in the middile?

---

<div class="post-metadata">

### Author: ![Dev220](https://avatars.discourse-cdn.com/v4/letter/d/b5a626/32.png) [@Dev220](https://discuss.elastic.co/u/Dev220)
#### Post date: [June 2, 2022, 2:40pm UTC](https://discuss.elastic.co/t/not-getting-the-latest-logs-in-kibana/306167/3 "2022-06-02T14:40:27Z")

</div>

Hi stephenb

Please find below details which you have asked for.

Fluent bit we are running as a container on our EKS cluster

What is the size and configuration of your Elasticsearch cluster?  
There is no cluster for Elasticsearch we are maintaining our Elasticsearch on a standalone server.

What kind / how much of HW / Storage / RAM / CPU?  
0.5 CPU and 1GB of RAM for fluent bit

About how many logs / minute are you shipping? What kind of logs?  
Around 1GB logs /day we are shipping and application logs (text logs)

How many Pods and fluent-bits (although we are not experts on that?)  
2 environments and in each pod 17 pods  
apart from that 1 pod fluent bit in each env

Is it straight from fluent-bit -\> Elasticsearch or are there other components in the middile?  
Yes it is straight from fluent bit to Elasticsearch  
No middle components are there

Thanks

---

<div class="post-metadata">

### Author: ![stephenb](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/stephenb/32/40856_2.png) [@stephenb](https://discuss.elastic.co/u/stephenb)
#### Post date: [June 2, 2022, 11:05pm UTC](https://discuss.elastic.co/t/not-getting-the-latest-logs-in-kibana/306167/4 "2022-06-02T23:05:01Z")

</div>

> [@Dev220](#):
>
> There is no cluster for Elasticsearch we are maintaining our Elasticsearch on a standalone server.

Is it a single node? I wanted to know the specs on elasticsearch.. If elasticsearch is not big enough, it could be backing up.

If elasticsearch is standalone have you looked at the logs?

Is that 1 GB a day for each pod or a total shipped from all pods?

---

<div class="post-metadata">

### Author: ![Dev220](https://avatars.discourse-cdn.com/v4/letter/d/b5a626/32.png) [@Dev220](https://discuss.elastic.co/u/Dev220)
#### Post date: [June 7, 2022, 8:32pm UTC](https://discuss.elastic.co/t/not-getting-the-latest-logs-in-kibana/306167/5 "2022-06-07T20:32:53Z")

</div>

I wanted to know the specs on Elasticsearch.  
Ans-4 CPU , 16GB memory

If Elasticsearch is standalone have you looked at the logs?  
I am not sure what is the path for seeing the logs for Elasticsearch

Is that 1 GB a day for each pod or a total shipped from all pods?  
Ans - Total shipped logs from all the pods is around 1.8GB-2GB

---

<div class="post-metadata">

### Author: ![stephenb](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/stephenb/32/40856_2.png) [@stephenb](https://discuss.elastic.co/u/stephenb)
#### Post date: [June 7, 2022, 9:01pm UTC](https://discuss.elastic.co/t/not-getting-the-latest-logs-in-kibana/306167/6 "2022-06-07T21:01:39Z")

</div>

Looks like you should have sufficient resources... but it is hard to tell not sure if that is a Single node or multiple, if Single still should be fine.

I am still unclear where / how Elasticsearch is deployed... is it in the same k8s Cluster?  
If in K8s you should be able to log into the container and find the logs  
If on a host should be `/var/log/elasticsearch` or `journalctl`

I am not that familiar with fluent so if there is a problem there I am not the right one to answer.

You going to need to dig in and provide some additional detail so we can help... otherwise we are just guessing.

You could try filebeat 🙂

---

<div class="post-metadata">

### Author: ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)
#### Post date: [July 5, 2022, 9:02pm UTC](https://discuss.elastic.co/t/not-getting-the-latest-logs-in-kibana/306167/7 "2022-07-05T21:02:09Z")

</div>

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.
