Hello,
I have logs that i have to process that look something like this
As far as my knowledge goes I have to set up logstash to support multiline logs so I created the following configuration
input {
file {
path => "/home/user/Desktop/Logs/**/**"
start_position => "beginning"
codec => multiline {
pattern => "^\[ %{LOGLEVEL:log_level}\]"
negate => true
what => "previous"
}
}
}
filter {
grok {
match => { "path" => "/(?<file_name>[^/]+).log" }
}
grok {
match => {
"message" => '(?m)\[ %{LOGLEVEL:log_level}\] %{DATE_EU:date}. %{TIME:time} \(%{DATA:class}:%{GREEDYDATA:operation}\) Date and time: %{GREEDYDATA:full_date_and_time}\| miliseconds: %{NUMBER:millis}'
}
}
}
output {
elasticsearch {
hosts => ["localhost:9200"]
index => "%{file_name}"
}
stdout { codec => rubydebug }
}
As far as the log pattern goes, it matches the logs in the screenshot (Checked it on Kibana Grok Debugger). I want logstash to consider line 1 2 and 3 as one event, 4,5,6 as one event etc. Essentially when it sees [ ] to consider it as a whole separate event, and to consider everything that follows as the same event until it sees another [ ]. Any ideas why this isn't working and what I could do to make it work?
Thanks a lot in advance!