Correct parsing Syslog message to json

Elastic Stack Logstash
1

Hi Folks!

I'm trying to parse the following message from mcafee proxy syslog inside my logstash pipeline:

<30>Feb 24 9:33:45 mwg-n3 mwg-n3: x-message="{"DateTime":"2023-02-22 14:03:44.927","MWG_Source":"mwg-n3.foo.de","MWG_ClientIP":"1.2.3.4","MWG_UserDisplayName":"TESTUSERA","MWG_URLCat":"Content Server","MWG_AppName":"","MWG_URLReputation":"Unverified","MWG_StatusCode":"200","MWG_URLHost":"testhost.com","MWG_URLPath":"/this/isa/a/test/path.img","MWG_URL":"https://testhost.com/this/isa/a/test/path.img?w=56&h=56&q=60&m=6&f=jpg&u=t","MWG_BytesToClient":"2265","MWG_UserAgent":"Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/109.0.0.0 Safari/537.36 Edg/109.0.1518.78","MWG_BlockID":"0","MWG_BlockReason":"","MWG_RequestMethod":"GET","MWG_Antimalware":"","MWG_HeaderReferer":"https://www.somereferrer.com/","MWG_MediaType":"image/jpeg","MWG_AuthDomain":"bmi-net.local","MWG_Cache":"TCP_MISS","MWG_UserNameRAW":"TESTTESTTEST@BMI-NET.LOCAL","MWG_BytesFromServer":"2383","MWG_BytesToServer":"74","MWG_BytesFromClient":"73","MWG_DST_IP":"123.456.789.123"}"

I just need all information from the beginning of x-message:

DateTime: 2023-02-22 14:03:44.927
MWG_Source: mwg-n3.foo.de
...
...
until
MWG_DST_IP: 123.456.789.123

How can I do this? I was trying to use Grok but I'm pretty sure that something is wrong in my filters configuration...

input {
 tcp {
    host => "192.168.1.101"
    port => 1514
  }
}
filter {
 grok {
  match => { "message" => "%{SYSLOGBASE} %{GREEDYDATA:message}" }
  overwrite => [ "message" ]
 }
 mutate {
  remove_field => ["@timestamp", "@version", "event", "process", "timestamp", "host"]
 }
}
output {
 stdout {}
}
2

I suggest using dissect to parse the prefix and then using a json filter to parse the rest. Something similar to this (except that uses kv rather than json).

3

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.