Input file contains tailing buffer - cannot input


(Johnny Farrell) #1

Hi All,

I've got a log file from an application which ends with a tailing buffer. When I try to input it into Logstash it doesn't read the file. At the end of each day the log file saves without the buffer and a new log is started for the new day.
The flat log files from the previous days input correctly but the new file doesn't.

I can't find any way to ignore the tailing buffer on the log file.

Can anyone help?


(Magnus Bäck) #2

What do mean by "tailing buffer"?


(Johnny Farrell) #3

There is a 4 mb memory block which is maintained at the end of the current days log file. Its full of NULLNULLNULL values. When logstash reaches this log line it stops working. When the file is next updated, logstash doesn't detect the update to the file and nothing is input into Logstash.


(Magnus Bäck) #4

Oh. How creative with 4 MB of nulls. Yes, I'd expect Logstash to not like that at all. I'm afraid I don't have any good suggestions.


(system) #5