Issue with logstash 8.17.4. Sincedb sometimes does not recognise the correct inode. After logstash restart it works

Elastic Stack Logstash
1

db.log rotates to db_backup.log.

input to logstash


input {
  file {
    path => "C:/Program Files (x86)/webserver/db/logs/db.log"
	exclude => "db_backup.log"
    type => "db"
    start_position => "beginning" 
    sincedb_path => "C:/setup/tools/logstash/sincedb_db.log"
    sincedb_clean_after => "4h"
    codec => multiline {
      pattern => "^\d{14}"  # Match lines starting with a 14-digit timestamp
      negate => true        # Lines NOT matching the pattern are part of the previous event
      what => "previous"    # Append non-matching lines to the previous event
      charset => "ISO-8859-1"
    }
  }
}

My sincedb entry before logstash restart

3336151636-103439-38731776 0 0 3743221 1746535009.097 C:/Program Files (x86)/webserver/db/logs/db.log
3336151636-103510-51970048 0 0 5255427 1746544213.982 C:/Program Files (x86)/webserver/db/logs/db.log
3336151636-103610-29032448 0 0 3033128 1746546108.149 C:/Program Files (x86)/webserver/db/logs/db.log

My sincedb entry after logstash restart

3336151636-103439-38731776 0 0 3743221 1746535009.097 C:/Program Files (x86)/webserver/db/logs/db.log
3336151636-103510-51970048 0 0 5255427 1746544213.982 C:/Program Files (x86)/webserver/db/logs/db.log
3336151636-103610-29032448 0 0 3033128 1746546108.149 C:/Program Files (x86)/webserver/db/logs/db.log
3336151636-103554-71303168 0 0 3896388 1746546378.764 C:/Program Files (x86)/webserver/db/logs/db.log   -- this is the right inode entry

db.log is pushed to elk. Without logstash restart. I want the correct inode to be updated. Please help

2

What OS and what filesystem type? The code suggests that it may not work on ReFS, which is used on the server OSes.

1 Like
3

This is the windows machine (cloud) and storage is the drive (EBS volumes)

4

That doesn't answer either of my questions. Which version of Windows, and what filesystem are you using on the EBS volumes? (In File Explorer, if you right click on the disk and select properties, the filesystem type will be labelled "File system".)

1 Like
5

Windows 2016 and NTFS .

6

Do you mean Windows Server 2016?

1 Like
7

I meant Microsoft windows server 2016.

8

@Badger Do we have issue with inode for windows server 2016 and NTFS filessystem. If yes is there any alternate method to solve this issue?

9

There is nothing in the comments in the source code that suggests the input would not work for an NTFS filesystem.

10

thanks for the update. The output of sincedb file before restart and after restart says that logstash is not able to get the right file identifier when the file rotation happens.