Filebeat not properly recognising truncated file

in this post Filebeat and truncated files
@ruflin states that filebeat will detect when a file is truncated and start reading again.

I am on beats7.10 and centos 7

We are going to move all our logging to move and hup but for now I have some applications where we are using copytruncate

I see the following behaviour. After the log file is truncated I see a few lines in elasticsearch but then the data stops.

I looked in /var/lib/filebeat/registry/filebeat/log.json

I found the file in question. The inode is correct but the log offset is greater than the current filesize. So I wonder if filebeats got confused when the file was truncated and is waiting for the old log offset

config includes

scan_frequency: 10s
ignore_older: 24h

also paths does not match the rotated file

I see this in the debug log which seems ominous bit I have no idea why it is not recognising the inputs if that is the case.

2020-12-07T12:27:07.001Z        INFO    [crawler]       beater/crawler.go:71    Loading Inputs: 0
2020-12-07T12:27:07.001Z        DEBUG   [registrar]     registrar/registrar.go:140      Starting Registrar
2020-12-07T12:27:07.001Z        INFO    [crawler]       beater/crawler.go:108   Loading and starting Inputs completed. Enabled inputs: 0
2020-12-07T12:27:07.001Z        INFO    cfgfile/reload.go:164   Config reloader started
2020-12-07T12:27:07.001Z        INFO    cfgfile/reload.go:164   Config reloader started

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.