Parsing IBM Integration Bus logs with Grok

[mladen (Mladen Stankovic)]

Hello,

searching the internet I didn't find the right grok filter for IBM integration bus logs. At the end I decided to try by may self. Filter is working but I have opinion that it can be better. Below you can find my grok filter with log.

Mar 28 12:12:01 localhost IIB[5979]: IBM Integration Bus v10008 (INDEV2.default) [Thread 21750] (Msg 1/1) BIP2153I: About to 'Start' an integration server.

%{SYSLOGTIMESTAMP} %{HOSTNAME} %{WORD:SYSTEM}\[%{BASE10NUM:PID}]: %{WORD:DONTNEED1} %{WORD:DONTNEED2} %{WORD:DONTNEED3} %{WORD:DONTNEED4} \(%{WORD:NODE}.%{WORD:EG}\) \[%{WORD:DONTNEED5} %{BASE10NUM:DONTNEED6}] \(%{WORD:DONTNEED7} %{NOTSPACE}\) %{WORD:CODE}: %{GREEDYDATA:MESSAGE}

My questions are as follows:

Could you please give me some hints to improve my filter?
Is it possible for example to take three words with spaces end put in one field? For example:
IBM Integration Bus
I have some words that is not useful for me. Can I skip them?

I am new in this so don't be harsh on me .

[yaauie (Ry Biesemeyer)]

@mladen

%{WORD:DONTNEED1}

If you don't need it, don't capture it to a variable

%{WORD}

@mladen

Is it possible for example to take three words with spaces end put in one field? For example:

IBM Integration Bus

I have some words that is not useful for me.

Is is always "three words with spaces", or is it always literally IBM Integration Bus?

If it is the latter, it is easier and more efficient to just include the literal bit in your Grok pattern:

%{SYSLOGTIMESTAMP} %{HOSTNAME} %{WORD:SYSTEM}\[%{BASE10NUM:PID}]: IBM Integration Bus %{WORD} \(%{WORD:NODE}.%{WORD:EG}\) \[%{WORD} %{BASE10NUM}] \(%{WORD} %{NOTSPACE}\) %{WORD:CODE}: %{GREEDYDATA:MESSAGE}

[yaauie (Ry Biesemeyer)]

And, you probably will want to add the beginning-of-line anchor ^ to the very beginning of your pattern if we know that there isn't going to be random junk before the timestamp.

Without it, when the pattern matcher fails, it will start over from the second letter, fail again, start over with the third letter, fail again, and so-on, making the failure to match pretty computationally expensive.

[mladen (Mladen Stankovic)]

@yaauie

IBM Integration Bus

Always is IBM Integration Bus so I can use it . Thank you very much for your answers, it help me a lot. Also I learn something new

BR,
Mladen