Is positive lookbehind supported in Logstash at all?

[stillfreem]

Hi there all, i started with Logstash the other day and currently getting to know the tool.
I have a situation here.

Consider the below Baraccuda FW log entry

2018 01 30 13:12:21 Security +01:00 Block: type=FWD|proto=UDP|srcIF=eth0|srcIP=10.17.34.12|srcPort=54915|srcMAC=18:db:f2:13:ca:9c|dstIP=10.17.34.255|dstPort=54915|dstService=|dstIF=|rule=BLOCKALL|info=Block by Rule|srcNAT=0.0.0.0|dstNAT=0.0.0.0|duration=0|count=1|receivedBytes=0|sentBytes=0|receivedPackets=0|sentPackets=0|user=|protocol=|application=|target=|content=|urlcat=`

I use Grok Debugger for testing

My grok regex so far is as follows

 %{YEAR} %{MONTHNUM} %{MONTHDAY} %{TIME} %{WORD:type_of_log} (?<tz>\+\d+:\d+) (?<action>\w+(?=\:\s))(?<tbd>\S) (?<type>(?<=w+=)\\w+\|)

What i want is to capture FWD string only when it's after type=.
In essence capturing "type=FWD" but only displaying FWD from the log entry above.
However this did not work for me although I tested this on regex101.com and it worked there.

My positive lookahead worked as charm though matching Block: but only displaying Block without the semicolon : and the space - (?\w+(?=:\s)

Could you please help me!

[Badger]

Personally I would approach a log like this using

    dissect { mapping => { "message" => "%{[@metadata][ts]} %{+[@metadata][ts]} %{+[@metadata][ts]} %{+[@metadata][ts]} %{word1} %{word2} %{word3} %{[@metadata][restOfLine]}" } }
    date { match => [ "[@metadata][ts]", "YYYY MM dd HH:mm:ss" ] }
    kv { field_split => "|" value_split => "=" }

That said, to answer the question: yes, positive lookbehind is supported

grok { match => { "message" => "(?<=type=)(?<foo>\w+)" } }

will result in

       "foo" => "FWD",

[stillfreem]

Much appreciated Badger. I'll need some time to read about dissect and comprehend it but really thank you very much.

Otherwise, I also thought that positive lookbehind is supported.
However, when using the Online grok Debugger, it seems that it did not work.
Does that mean that the problem might be in the online debugger itself?

As a proof, please, check the below images.
First picture shows that the grok pattern works so far and it yields results.

However, the second (and third) picture shows a valid positive lookbehind
Firstly, I only typed the field name (second pic) and it worked as my field name is TYPE and this is reflected in the output below (third pic)

However, if I try to apply a positive lookbehind it doesnt work as you can see below

However, I figure this out - A + quantifier inside a lookbehind makes it non-fixed width hence the Compile error

Then I used character class and it works on Regex101

And again in the online grok debugger didn't work but this time no errors just tells me that there are no matches?

I fully agree with you that your method is much better but nevertheless wanted to iron this out as it really bugs me

[Badger]

@stillfreem

Does that mean that the problem might be in the online debugger itself?

I never use anything other than grok to develop grok patterns because the online grok debugger and even kibana can interpret patterns differently to grok. See here for more detail.

[stillfreem]

Your awesome @Badger
Many thanks.
Stay safe