Mcafee DAM syslog parsing Logstash

[HawkUser]

Hi Community,

i'm looking for logstash idea how to parse the next content:
<15>#externalId=0#rt=1XXXX1#cs1=SaXXM XX-XXXXXX#cs1Label=XXX#dst=XX.XX.xx.xx#src=xx.xx.xx.xxx#duser=xxx_xxxx#suser=#shost=XXX-XXXXXXX#dproc=xxxxx#act=SELECT#cs2=SELECT XXXX,XXX,XXXX,XXXX,XXXXX FROM xxxxx.dbo.xxxxx WHERE xxxxx \\= @X1 AND (xxxxx is null) ORDER BY xxxxx ASC ,xxxxx ASC ,xxxxx ASC#cs3=#cs3Label=AccessedObjects#cs2Label=SqlStatement\u0000

i hashed the real values but the columns are real, also the delimiter is by request i customize the sender to delim using "#",
i tried the following config:

input {
  tcp {
    port => 5002
    type => syslog
    tags => ["dam"]
  }
  udp {
    port => 5002
    type => syslog
    tags => ["dam"]
  }
}

filter {
  if "dam" in [tags] {
    ruby {
        code => "
            fieldArray = event.get('message').split('#')
            for field in fieldArray
                field = field.delete('#')
                result = field.split('=',1)
                temp_key = 'key_' + result[0]
                temp_value = 'value_' + result[1]
                event.set(temp_key, temp_value)
            end

        "
    }
  }
}

i tried lot of trick even using KV, i believe i'm doing something wrong

[2017-01-29T00:04:10,785][ERROR][logstash.filters.ruby ] Ruby exception occurred: can't convert nil into String

Thanks in advance.

[Christian_Dahlqvist]

You should be able to use a grok filter to capture the start of the message before the key-value string and store the rest into a variable. You can then use the kv filter with custom field separator ('#') to parse the key-value list in this variable before using a mutate filter to it.

[HawkUser]

Hi,

Thank you for the reply, believe it or not I tried pretty much everything
Is there any chance there is sample?

Thanks

[magnusbaeck (Magnus Bäck)]

Show us what you have right now and it'll be easier for us to point you in the right direction.

[HawkUser]

Hi,
i mention it above, the configuration and the data sample
thanks!

[magnusbaeck (Magnus Bäck)]

I thought you had actually tried using grok and kv as Christian mentioned. You're looking for something like this:

filter {
  grok {
    match => ["message", "^<%{INT}>#%{GREEDYDATA:kv}"]
  }
  kv {
    source => "kv"
    field_split => "#"
  }
}