Grok Pattern/Expression for Multiline log from kibana (icingabeat index)

[shailesh]

Hello Team - I am trying to process some data from icinagbeat index from Kibana- so that i can visualize the output from icinagbeat - for that reason i am writing the Grok pattern - but the problem is when i write the pattern for one by one single line it all works fine - but the issue is i am unable to parse all the lines together - posting the output from Kibana for which i am trying to get the Grok pattern for
Totalopenfiledescriptors - is XXXX
Thresholdvalue for openfiledescriptors - is XXXX
processesrunning-html5client - XX
Total Sessions on a stitcher - is XX

• The above 4 lines are the output lines from icingabeat index for which i am trying to write the pattern for -
  And the Grok Pattern i came up with is
  Totalopenfiledescriptors - is %{NUMBER:Total_open_filedescriptors}, Thresholdvalue for openfiledescriptors - is %{NUMBER:Thresholdvalue_for_openfiledescriptors}, processesrunning-html5client - %{NUMBER:processesrunning-html5client}, Total Sessions on a stitcher - is %{NUMBER:Total_Sessions_on_stitcher}

i am trying to process all the lines together - please let us know if there is a way ? or if i am not going in the correct direction -
Thanks in Advance
@shailesh

[Cad]

Hi,

You have 4 grok pattern, one for each line, but you seperated them with ', ' why ?
Replace each ', ' by a [\n] resolve your error.

Cad.

[shailesh]

Hello Cad - Thank you so much for the response - I am attaching a Screenshot - am i doing something wrong here ?

[Cad]

Hi,

The pattern is not working because of the ", at the end.
I must specify that the pattern your trying to use work if all the data come at the same time.
If the values come line by line, you have to split all the pattern like this :

#One pattern per line
grok {
  match => { "message" => "Totalopenfiledescriptors - is %{NUMBER:Total_open_filedescriptors}",
                          "Thresholdvalue for openfiledescriptors - is %{NUMBER:Thresholdvalue_for_openfiledescriptors}",
                          "processesrunning-html5client - %{NUMBER:processesrunning-html5client}",
                          "Total Sessions on a stitcher - is %{NUMBER:Total_Sessions_on_stitcher}",

           }
}

With this configuration, patterns are sequentialy tested until one fit the data in input (so one line of your complete data). Once one pattern is correct, the pipeline leave the grok filter without testing the next patterns.

More about grok here.

Cad.

[shailesh]

Thanks Chad for the Response - I did fix it my converting the log message to a single line log message - Thank you for the help