File Discovery in Logstash

[thorsten1]

Hello,

my Logstash Server has an Upload-Directory where other Users can upload their Log-Files.
Logstash is configured to listen to all files in this directory.

  input {
       file {
        path => "/opt/logupload/*.log"
        start_position => "beginning"
        stat_interval => 5
        codec => ...
        }
      }

When I have uploaded a new file, nothing happen's in Logstash. But when I stop my Logstash-Service it begin to parse the file.

Do you have an idea, why the parsing only starts when the service will be stopped?

[thorsten1]

Did I use the wrong Plugin?

[Nico-DF]

It used to happend to me when I used multiline plugin. It waits until the end match, but when you stop it, it will push the event regardless of the end matched or not.

Do you have any aggregation or multiline that might cause this?

[thorsten1]

Yes I use multiline codec in this way:

   codec => multiline {
     pattern => "^%{MONTH}%{SPACE}%{MONTHDAY}|^%{HOUR}:%{MINUTE}:%{SECOND}"
     negate => true
     what => previous
   }
   ...