Multiline grok filter not working with specific log

Hello,
I have logs that i have to process that look something like this


As far as my knowledge goes I have to set up logstash to support multiline logs so I created the following configuration

input {
	file {
		path => "/home/user/Desktop/Logs/**/**"
		start_position => "beginning"
		codec => multiline {
			pattern => "^\[ %{LOGLEVEL:log_level}\]"
			negate => true
			what => "previous"
	        }
	}
}

filter {
	grok {
		match => { "path" => "/(?<file_name>[^/]+).log" }
	}
	grok {
        match => {
            "message" => '(?m)\[ %{LOGLEVEL:log_level}\] %{DATE_EU:date}. %{TIME:time} \(%{DATA:class}:%{GREEDYDATA:operation}\) Date and time: %{GREEDYDATA:full_date_and_time}\| miliseconds: %{NUMBER:millis}'
        }
    }
}

output {
	elasticsearch { 
		hosts => ["localhost:9200"] 
		index => "%{file_name}"
	}
  	stdout { codec => rubydebug }
}

As far as the log pattern goes, it matches the logs in the screenshot (Checked it on Kibana Grok Debugger). I want logstash to consider line 1 2 and 3 as one event, 4,5,6 as one event etc. Essentially when it sees [ ] to consider it as a whole separate event, and to consider everything that follows as the same event until it sees another [ ]. Any ideas why this isn't working and what I could do to make it work?
Thanks a lot in advance!

The events flushed by the multiline codec will include the line separators, so you will probably need %{GREEDYDATA:operation}\)\nDate and time:.

That was it, thanks a lot!

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.