Truncate command causes filebeat misbehave

Elastic Stack Beats
1

OS: Centos7
Filebeat: 6.6.2

We have a cronjob to reduce log size in case disk full. the command used is truncate. like:
truncate -s 15G /path/to/file

We found that filebeat will quickly eat up all memory in the system (16GB in our case) when log file got truncated. Restart filebeat wouldn't help (filebeat process RSS memory would reach 15GB in a few seconds), unless removed the registry file to make filebeat read log file from the beginning again.

Not sure if this is a bug, or is there any configure to suppress this issue

2

Could you please share the debug logs of Filebeat?
When Filebeat detects that the file has been truncated it start to read it from the beginning.

3

I didn't open the debug, and now i can't reproduce the issue.
I had tried to truncate a file to simulate the case. but as you mentioned, Filebeat would reset the offset to 0. I suspect that my log file might somehow damaged.

BTW, looks like Filebeat can't handle the truncate command well. As I was truncated the file to a smaller size but not wipe it to 0, it shouldn't read from the beginning again.

4

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.