Extracting string from log path

[Bkumar (Bunty)]

Hi,
As per requirement, i can't put the tag to filebeat to filter log path at logstash end. So only option left with me to filter the log path on the basis of string.
Here is the sample log path-
/opt/tomcat/instances/dev/HSE/ms-app/log
/opt/tomcat/instances/dev/B07-STAT/service/log
/opt/tomcat/instances/stg/A96-MR/service/log
/opt/tomcar/instances/stg/R2C/service/log

From the above log path, I want to filter the log path which contains the string A96-MR at logstash.
Please help me in filtering on the basis of string.
Thanks

[Bkumar (Bunty)]

I tried using regex, but not sure how it will work on logstash

[Badger]

You could try using grok

grok { match => { "someField" => "/%{WORD:anotherField}/[^/]+/log$" } }

[Bkumar (Bunty)]

Thanks @Badger for quick reply

To be more precise, it should be like as per my requirement:

grok { match => { "path" => "/%{WORD:A96-MR}/[^/]+/logs$" } }

[Badger]

Actually I had that regexp wrong, because WORD does not match hyphen.

grok { match => { "path" => "/(?<anotherField>[^/]+)/[^/]+/log$" } }

[Bkumar (Bunty)]

Hi @Badger I am getting grokparsefailure tag after applying this filter. I am not preety sure about the root cause. Trying to explain use case again
I have almost 30 different log path as a input to logstash and grok need to parse only those path which contains the string "A96-MR" in the log path and send only these log path to output.

[Badger]

I tested it in logstash and that grok works if [path] contains "/opt/tomcat/instances/stg/A96-MR/service/log" or any of the other values you show. To drop events where the extracted value is not A96-MR you could check the extracted value

if [anotherField] != "A96-MR" { drop {} }

If you use

output { stdout { codec => rubydebug } }

then what does an event look like?

[Bkumar (Bunty)]

Hi @Badger
It is working indeed. But we don't want to drop any path. The path which contains A96-MR will go to one table and remaining path will go to other table.

[Badger]

OK, so use a conditional in the output section.

[Bkumar (Bunty)]

Thanks alot @Badger It worked for me