Multiline codec reading entire log file as one line

Elastic Stack Logstash
1

Hello,

I'm having an issue with my multiline codec working correctly. Most of the lines in my file are a single line but some of them do have large stack traces in them at some points. When I run my config file the entire log is going into one message, it is not separating each line by its datestamp.

input {      
  file {
    
    path => "C:\Users\e-team\Documents\ELK\logs\coordinatorNode_0\logs\deployment2016-09-01T10.44.56.445-0500.log"

codec => multiline {
      # Grok pattern names are valid! :)
      pattern => "^%{DATESTAMP} "
      negate => true
      what => previous
    }
  
    start_position => "beginning"
  }
}

abbreviated filter section to the part I'm having the issue in:

filter{
if [path] =~ "deployment"{
      mutate { replace => { "type" => "DeployLog" } }

      mutate {
    gsub => [ "message", "\n", "" ]
   }

    grok{
      match => { "message" => "%{DATESTAMP:timestamp} %{LOGLEVEL:severity}\s* \[%{NOTSPACE:package}\] %{GREEDYDATA:message}"
      }


      match => { "message" => "(?m)%{DATESTAMP:timestamp} %{LOGLEVEL:severity}\s* \[%{NOTSPACE:package}\] %{GREEDYDATA:message}"
      overwrite => ["message"] 
      }
      
    }
   
    date {
    match => [ "timestamp" , "yy-MM-dd HH:mm:ss,SSS"]
    target => "@timestamp"
    }
  }
}

does this all look correct?

some sample log lines - the last line is an abbreviated stack trace

2016-09-01 10:44:56,445 INFO  [com.ServerStateLogger] Worker 60493 is Unsuspended
2016-09-01 10:44:56,445 INFO  [com.ServerStateLogger] Worker 49696 is Unsuspended
2016-09-01 10:44:56,445 INFO  [com.ServerStateLogger] --------------------------------------------------------------------
2016-09-01 10:44:56,445 INFO  [comServerStateLogger] Queue State Summary
2016-09-01 10:45:11,460 ERROR [com.ServerStateLogger] Error attempting to log cluster state.
java.util.concurrent.TimeoutException: Futures timed out after [15 seconds]
	at scala.concurrent.impl.Promise$DefaultPromise.ready(Promise.scala:219)
	at scala.concurrent.impl.Promise$DefaultPromise.result(Promise.scala:223)
	at scala.concurrent.Await$$anonfun$result$1.apply(package.scala:116)
	at scala.concurrent.BlockContext$DefaultBlockContext$.blockOn(BlockContext.scala:53)
2

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.