How to handle multi line logs in logstash

Elastic Stack Logstash
1

Hi

I have such logs, i am not sure how handle this multi line logs, as you can see. its not consistent, we have multi lines at times and single lines at times.
please help, how to parse them

2018-10-22 15:32:24.407 DEBUG [task-detail-service-dev,9d3c34d063a7c5fe,646dff3d87b59c21,true] 13287 --- [askExecutor-438] CCMessagePublisher

2018-10-22 15:32:24.407 DEBUG [task-detail-service-dev,9d3c34d063a7c5fe,646dff3d87b59c21,true] 13287 --- [askExecutor-438] CCMessagePublisher : Publishing message some message to seome_event_centre:
taskInstanceId: "111c0242-d5e7-11e8-b966-0050568b4ef6"
eventType: "TaskVariablesUpdated"
variables {
key: "SOME_KEY_1"
value {
stringValue: "SOME_VALUE_1"
}
}
variables {
key: "SOME_KEY_2"
value {
stringValue: "SOME_VALUE_2"
}
}

I am using below conf

input { stdin { } }

filter {
grok {
patterns_dir => ["./patterns"]
match => { "message" => "%{WORD:log_level} [%{GREEDYDATA:local_address},%{POSTFIX_QUEUEID:queue_id},%{POSTFIX_QUEUEID:spanId},%{WORD:random_entry}]" }
}

date {
match => [ "timestamp" , "dd/MMM/yyyy:HH:mm:ss Z" ]
}
}

output {
stdout { codec => rubydebug }
}

2

use the multiline codec when importing, normally you would use the timestamp as the pattern to start a new event.

3

See https://www.elastic.co/guide/en/logstash/current/multiline.html.

4

thanks it worked

5

thanks a lot, it works

6

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.