How to parse different log in logstash configuration

Hello to all,

i have a problem with activemq logos i have created different types of grok,
but the problem is that sometimes the message is different with additional fields
and consequently it is parsed incorrectly, is there a way to parse dynamically?
Are there any alternative methods to Grok?
I would be interested in parsing the fields, Timestamp, Evet_type (INFO, WARN, ERROR), Event, action, Username.

Log example:

2021-10-07 13:38:47,157 | INFO | [#id:xxXXXXXX] [#event:done pass control to super.addConnection(context,info)] [#arg:action:addDestination] [#arg:username:CCCCCCC] [#arg:client-id:ID:CCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCC] [#arg:connection-id:ID:CCCCCCCCCCCCCCCCCCCCCCCCCC] [#arg:destination:topic://CCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCC | ActiveMQ Transport: tcp:///127.0.0.1:57140@61616

2021-10-07 13:38:47,140 | INFO | [#id:xxxxxx] [#event:doing pass control to super.addConnection(context,info)] [#arg:username:xxxxxxx] [#arg:client-id:ID:xxxxxxxxxxxxxxxxxxxxxxxxxxxx] [#arg:client-ip:tcp://127.0.0.1:57140] [#arg:connection-id:ID:xxxxxxxxxxxx.xxxxxxxxx.xxxxx.xxxxxxxx] | xxxxxxxxxxx.xxxxxxxxx.xxxxxxxxx.xxxxxxx.xxxx| ActiveMQ Transport: tcp:///127.0.0.1:57140@61616

These are just two examples but there are many other different ones.

Thanks in advance

Not sure how dynamic all your logs are but have you tried doing it this way?

filter {
 grok {
  break_on_match => false
  match => { "message" => "^%{TIMESTAMP_ISO8601:timestamp} | %{LOGLEVEL:loglevel} |" }
  match => { "message" => "event:%{DATA:event}]" }
  match => { "message" => "action:%{DATA:action}]" }
  match => { "message" => "username:%{DATA:username}]" }
 } 
} 
1 Like

Hi Aaron,

thanks for your help, it works!

Thank you very much