Hi,
I'm attempting to parse authentication logs using an elk stack, it was all up and running one week. Logstash was able to process all of the logs placed within the directory, both .log and .log.1 files. Then after re-indexing the data it has appeared that logstash will not recognise any .log files only .log.1 etc.
I've uploaded part of the file config that deals with the input path.
file {
path => "C:/LOGS/**/.log"
type => "Authentication"
start_position => beginning
close_older => "1 hour"
sincedb_path => "NUL"
codec => plain {
charset => "Windows-1252"
}
}
When setting the output to console using rubydebug. Usually logstash starts up then stops on this line, as it can't seem to find any of the .log files.
[2019-04-04T11:11:46,873][INFO ][logstash.agent ] Successfully started Logstash API endpoint {:port=>9600}
I managed to get an error message eventually when it failed to process a .log file.
[2019-04-04T11:15:35,047][WARN ][filewatch.tailmode.processor] >>> Rotation In Progress - inode change detected and original content is not fully read, file is closed and path points to new content {"watched_file details"=>"<FileWatch::WatchedFile: @filename='AuthenticationService.log', @state='rotation_in_progress', @recent_states='[:watched, :watched]', @bytes_read='0', @bytes_unread='0', current_size='13950782', last_stat_size='13950782', file_open?='false', @initial=false, @sincedb_key='unknown 0 0'>"}
Anyone know anything about this issue? as it would be greatly appreciated.
Edit:
After trying to find more information on this matter, I enabled sincedb to see what its trying to process. and this is the result.
unknown 0 0 0 1554687598.275 C:/LOGS/AuthenticationService.log
612407792-715744-983040 0 0 40960059 1554687592.116 C:/LOGS/AuthenticationService.log.1
Whilst one log file gets an inode identifier the .log doesn't.
--SOLVED--
Turns out theres a known bug with this issue.
This post here allowed me to fix the issue.