Using field name other than "message" in grok match

Elastic Stack Elasticsearch
1

i have log like below, in line 2 , i have stored custom filter value in field "method"...in line 3 , i want to use "method" field and apply it another custom filter and store this result in method 2...i don't want to change "message" field ...but its not working ...does grok match only works on "message" field...how to match pattern on user defined fields without changing default "message" field.

INFO:root:Filesystem backup started at 2020-11-24 04:30:06
INFO:root:Backup method is volume enabled
INFO:root:Filesystem backup ended

grok{
match => { "message" =>"(?<Start_Time>Filesystem backup started.)"} (line 1)
match => { "message" => "(?Backup method is.)"} (line 2)
match => { "method" =>"(?:\w+$)"} (line 3)
match => { "message" =>"(?<End_Time>Filesystem backup started.)"}

2

In grok, message is the entire event that it reads, you can't change that.

Is this a multiline log entry?

3

I need to change grok and then use a user defined variable in place of message ...if I use message it reads event but if message is replaced let's say with a field method, grpk wont read anything

4

If this is your message;

INFO:root:Filesystem backup started at 2020-11-24 04:30:06
INFO:root:Backup method is volume enabled
INFO:root:Filesystem backup ended

Then you might want something like this;

grok {
  match => {
    "message" => "%{LOGLEVEL:level}:%{USERNAME:user}:%{GREEDYDATA:log}"
  }
}

That will capture the relevant sections of your entire log line. Once that is extracted then you can run another grok on the log field that is extracted and figure out what method was used.

What you have won't work because you haven't extracted anything into the method field for grok to match.

5

sorry , i was not able to explain what i am trying to ask or achieve.

my question was this , if we match a line like below in grok using custom pattern and store it in lets say, "Start_Time"
grok{
match => { "message" =>"(?<Start_Time>Filesystem backup started.)"}
Next , is it possible/allowed to use , this "Start_Time" field ,in place of "message" to match another regex and then store this result in "Variable"2
match => { "Start_Time" =>"(?myregex.)"}

6

Yes, that's exactly what I was saying :slight_smile:

7

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.