Filebeat 7.10 doesn't follow log while log rotation occur

[Mahoni]

filebeat 7.10 doesn't follow log while log rotation occur, log files inode number is changing but filebeat doesn't follow the log file;

     # ls -ltr /logs/user-app.tomcat/applicationLogs/tra*.log*
    -rw-r--r-- 1 user user 10485870 Nov 18 15:35 /logs/user-app.tomcat/applicationLogs/trace.log.10
    -rw-r--r-- 1 user user 10485761 Nov 18 16:09 /logs/user-app.tomcat/applicationLogs/trace.log.9
    -rw-r--r-- 1 user user 10485842 Nov 18 17:02 /logs/user-app.tomcat/applicationLogs/trace.log.8
    -rw-r--r-- 1 user user 10485801 Nov 19 04:45 /logs/user-app.tomcat/applicationLogs/trace.log.7
    -rw-r--r-- 1 user user 10485841 Nov 19 08:15 /logs/user-app.tomcat/applicationLogs/trace.log.6
    -rw-r--r-- 1 user user 10485776 Nov 19 09:07 /logs/user-app.tomcat/applicationLogs/trace.log.5
    -rw-r--r-- 1 user user 10485939 Nov 19 09:45 /logs/user-app.tomcat/applicationLogs/trace.log.4
    -rw-r--r-- 1 user user 10485803 Nov 19 10:26 /logs/user-app.tomcat/applicationLogs/trace.log.3
    -rw-r--r-- 1 user user 10485904 Nov 19 11:00 /logs/user-app.tomcat/applicationLogs/trace.log.2
    -rw-r--r-- 1 user user 10485766 Nov 19 11:35 /logs/user-app.tomcat/applicationLogs/trace.log.1
    -rw-r--r-- 1 user user  3304878 Nov 19 11:46 /logs/user-app.tomcat/applicationLogs/trace.log

filebeat.ylm config file is like this ;

    filebeat.inputs:
    - type: log
      enabled: false
      paths:
        - /var/log/*.log
    - type: log
      enabled: true
      paths:
        - /logs/user-app.tomcat/applicationLogs/trace.log*
        - /logs/user-app.tomcat/applicationLogs/yrucrem.log*
      exclude_files: ['\.gz$']
    [... ]

before the log rotation

     stat  /logs/user-app.tomcat/applicationLogs/trace.log | egrep "Inode"
    Device: fd08h/64776d    Inode: 98322       Links: 1

after the log rotation:

     # stat  /logs/user-app.tomcat/applicationLogs/trace.log | egrep "Inode|File"
    File: ‘/logs/user-app.tomcat/applicationLogs/trace.log’
    Device: fd08h/64776d    Inode: 98325       Links: 1

Once I restart filebeat service it is running again.

[shaunak (Shaunak Kashyap)]

Hi @Mahoni, welcome to the Elastic community forums!

Is this path a network mount by any chance?

Can you run lsof on the Filebeat process before and after the rotation and post the results here please?

Also, are you seeing any errors or warnings in the Filebeat log? If possible, could you post the Filebeat log entries from about 20 seconds before the file rotation to about 20 seconds after it?

Thanks,

Shaunak