Grok escape

Elastic Stack Logstash
1

Below is the source I have --

"source" => "/apps/home/cache22/logs/room_description.16-02-01_17.log"

and my match currently is --

match => ["source","%{GREEDYDATA}/%{GREEDYDATA:filename}.log"]

But I need to also grok for "cache22" (this part keeps changing always in source) along with filename.

2

Is the position of (in this case) catch22 always the same within the path? That is, is it always the third path component from the end? Or should we instead count from the beginning of the string.

3

Do you need to capture the word "cache22" and store it in a separate field or you need to "grok" the third position of your "source" field?

4

My source is like --

either
/apps/home/cache22/logs/sixt/rse.16-02-02_09.log

or

/apps/home/cache22/logs/rse.16-02-02_09.log

In my source, cache22 is always the third occurence and the string keeps changing to different values like "cache22" or "hotels" or "rental" etc..but it is always after "/apps/home"

The other thing is there might or might not be a subdirectory after "/logs" in the source... My scenario is like I am streaming log files(.log) under "/logs" and in some of the applications there are additional (.log) inside subdirectories under "/logs" and these subdirectory names keep changine.

So, I need to grok for both "cache22" and also "sixt" (if it exists) else its fine.

5

In my source, cache22 is always the third occurence and the string keeps changing to different values like "cache22" or "hotels" or "rental" etc..but it is always after "/apps/home"

That's quite easy. The expression

^/apps/home/(?<whatever>[^/]+)/

will capture the third path component (one or more characters except slash) into the whatever field.

The other thing is there might or might not be a subdirectory after "/logs" in the source... My scenario is like I am streaming log files(.log) under "/logs" and in some of the applications there are additional (.log) inside subdirectories under "/logs" and these subdirectory names keep changine.

I don't follow exactly, but

^/apps/home/(?<whatever>[^/]+)/logs/%{GREEDYDATA:filename}

extracts whatever just like above, followed by "logs/", and then it captures the rest of the string. I'll leave any final adjustments as an exercise.

Please take note that none of the requirements in your second post was present in your original post.