Hi, We are using filebeat to ship Zeek logs to logstash. We do this by defining a different filestream input for each type of log and in the input we define the path filebeat.yml path.home: /usr/share/filebeat path.data: /var/lib/filebeat_dns path.config: /etc/filebeat_top/filebeat_dns path.logs:…

I confirm I can reproduce that and indeed there is a bug in the way Filebeat is handling file truncation. It does not happen all the time and in your case @Dheeraj_Gupta it is triggered by identity reuse (either inode reuse or using file identity path as you described). I'll post more details about…

On restarting filebeat, every filestream paths input is deemed to be truncated

Elastic Stack Beats

Dheeraj_Gupta (Dheeraj) February 19, 2024, 11:26am 6

Thanks for looking at this @TiagoQueiroz

I feel this a filestream bug which is easy to run into if underlying filesystem is not EXT. I found this comment by you on an older (similar thread).

1 Like

Topic		Replies	Views
FIlebeat doesn't reparse logs Beats filebeat	11	2855	July 5, 2017
Filebeat beta1 resends random data upon every restart (registry file not updated properly?) Beats filebeat	14	2246	October 20, 2016
Filestream input sends duplicates events on restart and during operation Beats filebeat	34	1753	July 23, 2023
Filebeat duplicates events after restart Beats filebeat	1	603	February 7, 2022
Filebeat repeatedly sending old entries in log file Beats filebeat	3	3084	August 8, 2016

On restarting filebeat, every filestream paths input is deemed to be truncated

Related topics