Logstash grok pattern issue

[mugil1988 (Mugilvannan L)]

Hi All,

I am trying to configure grok patter to parse my application logs.

Part of the logs is not getting displayed in kibana.

Sample logs :

2019-09-10T23:04:21,584 | INFO | ://dedupresquest | _route1 | ID-ESB-Dev--in-1567014689815-54-2532 | {"txnid":"SME-LOS20180912122512345","source":"SME-LOS","timestamp":"10-2019-09 17-50-21","Type":"Source Response","Payload":{
"status" : "Success",
"message" : "Created Successfully"
},"status":"success"}

Grok Pattern :

(?m)%{TIMESTAMP_ISO8601:timestamp}%{SPACE}\|%{SPACE}%{WORD:LogLevel}%{SPACE}\|%{SPACE}%{GREEDYDATA:Word} \|%{SPACE}%{WORD:Routename}%{SPACE}\|%{SPACE}%{GREEDYDATA:ID}%{SPACE}\|(?m)%{GREEDYDATA:payload}"

Kibana Display logs :

{"txnid":"SME-LOS20180912122512345","source":"SME-LOS","timestamp":"10-2019-09 17-50-21","Type":"Source Response","Payload":{

Kindly help!!!

Regards,
Mugil

[Badger]

If the log message is really spread across four lines you will need a multiline codec to join them back together before you grok them. You may be able to use a regexp that matches the timestamp to do that.

[mugil1988 (Mugilvannan L)]

Thanks for the reply.

I am using filebeat to harvest logs.

I am using multiline pattern & negate options in filebeat config.

Filebeat config

multiline.pattern: '^%{TIMESTAMP_ISO8601}'
multiline.negate: false
multiline.match: after

Still facing same issue.

Do i need to change anything in filebeat config?

Regards,
Mugil