Setup: Kubernetes cluster with filebeat-7.12.0 -> logstash-7.9. -> elasticsearch-7.9.1 -> kibana-7.9.1 Problem: we lose many logs often and sometimes in a massive way, as in we stop seeing logs from one or more pods (confirmed fact from other logging/alert mechanisms), we don't know where/how. …

Losing logs in ELK

warkolm (Mark Walkom) June 9, 2021, 12:26am 2

devopscanada:

Harvester could not be started on new file: ... Err: error setting up harvester: Harvester setup failed. Unexpected file opening error: file info is not identical with opened file. Aborting harvesting and retrying file later again

These seem pretty relevant.
Can you enable debug on one Filebeat pod and provide more logging?

Filebeat losing logs

Topic		Replies	Views
Suggestion improving filebeat performance Beats filebeat	3	1224	November 24, 2017
Throttling processing of log backfill using Filebeat Beats	11	3498	May 17, 2017
Filebeat traffic overwhelms ELK stack? Elasticsearch	1	730	February 14, 2017
Performance weird stuff Elasticsearch	13	874	September 25, 2020
Elasticagent - filebeat is still increasing memory Beats	4	668	January 30, 2024

Losing logs in ELK

Related topics