Logs shown in Kibana are behind the current time

Elastic Stack Beats
1

Hello,
I configured Filebeat, Logstash, ES and Kibana, to gather nginx-ingress logs from Kubernetes.
Since there's a lot of logs (10 hits in a second), the presented data in Kibana is good, but behind the current time.
For example , shown logs are for 17:05:24 but now is 17:15, and this delta is getting bigger, because of the amount of logs.
I tried to use scan_frequency and close_inactive in filebeat config

apiVersion: v1
kind: ConfigMap
metadata:
  name: filebeat-prospectors
  namespace: kube-system
  labels:
    k8s-app: filebeat
    kubernetes.io/cluster-service: "true"
data:
  kubernetes.yml: |-
    - type: docker
      containers.ids:
      - "*"
      processors:
        - add_kubernetes_metadata:
            in_cluster: true
      scan_frequency: 10s
      close_inactive: 1m

But it didn't really helps, any best practise for such case?

Thx!

2

The lag might be in Logstash or Elasticsearch. Filebeat tries to read files as fast as possible, but is subject to back-pressure from downstream systems. If Elasticsearch/Logstash can not hold up to the load generated by filebeat, they will force filebeat to slow down.

3

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.