Provided Grok patterns do not match data in the input

Elastic Stack Logstash
1

Hi,
Trying to create my own grok patterns, I'm using the following with the Grok Debugger :
Sample data :
[ 4812 6032][15 Feb 18:23:15][LdManInit] Loading Library in Load mode
Grok Pattern :
%{IDLOG:whom} %{TIMEST:when} %{LOGCAT:what}
Custom patterns :
IDLOG ^\[.([0-9]{4,6}.[0-9]{4,6})\] TIMEST \[([0-9]{1,2}.[a-zA-Z]{3}.*[0-9])\] LOGCAT \]\[(.[a-zA-Z]*.)\]|\]\[\]

Which returns the error above.
But if I'm using only one pattern (IDLOG, TIMEST, LOGCAT) at onces, they work perfectly...

Any idea that could help me to debug this ?
Many thanks in advance,
Chris

2

You pattern has spaces between the three items, your message does not. Also, TIMEST has consumed the ] after the timestamp, so LOGCAT does not have anything to match the "\]" that it starts with. This works...

input { generator { count => 1 lines => [ '[ 4812 6032][15 Feb 18:23:15][LdManInit] Loading Library in Load mode' ] } }
filter {
    grok {
        pattern_definitions => {
            IDLOG => "^\[.([0-9]{4,6}.[0-9]{4,6})\]"
            TIMEST => "\[([0-9]{1,2}.[a-zA-Z]{3}.*[0-9])\]"
            LOGCAT => "\[(.[a-zA-Z]*.)\]|\]\[\]"
        }
        match => { "message" => "%{IDLOG:whom}%{TIMEST:when}%{LOGCAT:what}" }
    }
}
3

Hi, thanks a lot indeed for this rapid answer.

You are right, I started so (without spaces) but still without spaces, the debug screen does not work.

I will try now using directly logstash, does it means that the debugger works differently from the routine ?

Chris

4

I have certainly seen cases where kibana and the Heroku debugger parse things differently to logstash (kibana does multiline matching of GREEDYDATA differently, for example). I do not use them, instead I debug grok patterns using grok as detailed here.

5

Thanks a lot, it worked perfectly... no more use of Grok Debugger :slight_smile:

6

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.