Filebeat Ingests K8S Older Logs

Elastic Stack Beats
1

I'm using beats/filebeat-kubernetes.yaml at v7.8.1 · elastic/beats · GitHub to ingest K8S. There have been 2 issues:

  1. Filebeat picks up older logs when there is an other application deployment and generates old indices like filebeat-2021.03.01 etc. (index: filebeat-%{+YYYY.MM.dd}
  2. Filebeat misses some logs from the new pods unless I terminate Filebeat pods.

I think that the timestamp in filebeat-%{+YYYY.MM.dd} is from system rather than from the log file. Isn't it?

2

Could you share some sample logs with us? I suspect that there might be a timestamp conversion problem.

3

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.