I am new to kibana.
For collecting the logs from my application pods I am using fluent-bit configuration as a pod.
Both the Elasticsearch and kiban services are up and running.
I am able to see the logs in kibana dashboard but not the latest ones.
For some pods I am seeing 1day older logs in kibana dashboard although recent logs are present inside the pods.
And for some other pods I am seeing logs in 30 minutes delay.
So my issue is I am unable to see the latest logs in Kibana dashboard.
I have tried recreating the fluent bit pod by changing the fluent bit image.
After 10 minutes of re-deploying the fluent bit pod I am still facing the same issue.
Please find below details which you have asked for.
Fluent bit we are running as a container on our EKS cluster
What is the size and configuration of your Elasticsearch cluster?
There is no cluster for Elasticsearch we are maintaining our Elasticsearch on a standalone server.
What kind / how much of HW / Storage / RAM / CPU?
0.5 CPU and 1GB of RAM for fluent bit
About how many logs / minute are you shipping? What kind of logs?
Around 1GB logs /day we are shipping and application logs (text logs)
How many Pods and fluent-bits (although we are not experts on that?)
2 environments and in each pod 17 pods
apart from that 1 pod fluent bit in each env
Is it straight from fluent-bit -> Elasticsearch or are there other components in the middile?
Yes it is straight from fluent bit to Elasticsearch
No middle components are there
Looks like you should have sufficient resources... but it is hard to tell not sure if that is a Single node or multiple, if Single still should be fine.
I am still unclear where / how Elasticsearch is deployed... is it in the same k8s Cluster?
If in K8s you should be able to log into the container and find the logs
If on a host should be /var/log/elasticsearch or journalctl
I am not that familiar with fluent so if there is a problem there I am not the right one to answer.
You going to need to dig in and provide some additional detail so we can help... otherwise we are just guessing.
Apache, Apache Lucene, Apache Hadoop, Hadoop, HDFS and the yellow elephant
logo are trademarks of the
Apache Software Foundation
in the United States and/or other countries.