Can long grok filter line be extended into multiple lines?

Elastic Stack Logstash
1
  • is there a continuation syntax - some of my filters are stretching beyond 150 characters - would be nice if that could be extended into second line ...

Thanks,

2

Have you seen https://www.elastic.co/guide/en/logstash/current/plugins-codecs-multiline.html?

3

yes I did - that merges multiple lines from the input file into one event ... I would like to break long lines from logstash.conf - like very long grok filter line, can it be folded into multiple filter line - talking about the grok code / - message filter itself, not the data ... Example:

match => [ "message" => "%{TIMESTAMP_ISO8601:zxk_mysql_timestamp} %{NUMBER:zxk_num_xxx} [%{MYSQLWARNING2:zxk_mysql_msgtype}] %{GREEDYDATA:zxk_warning_rest}",

Could I break the line after say 100 characters and extended it into second line:

match => [ "message" => "%{TIMESTAMP_ISO8601:ses_mysql_timestamp} %{NUMBER:ses_num_xxx} -
[%{MYSQLWARNING2:ses_mysql_msgtype}] %{GREEDYDATA:ses_warning_rest}",

using some kind of continuation character - I used '-' in the example here ... but that did not work in logstash

could not find it in the syntax anywhere ...

4

Oh, right.

No you cannot.

5

ok - thanks - too bad though - filters easily get very long and I need 30 inch monitor for those ...

6

You can do something like this if you have multiple patterns;

filter {
  grok { match => { "message" => [ "Duration: %{NUMBER:duration}", "Speed: %{NUMBER:speed}" ],
                                 [ "Something: %{NUMBER:duration}", "Somethingelse: %{NUMBER:speed}" ],
                                 [ "Foo: %{NUMBER:duration}", "Bar: %{NUMBER:speed}" ]
                                 }
  }
}
7

true - but if any one pattern out of those three is 200 characters long then the editor wraps the line ... to the far left ... not very readable ...

I think I can define my own patterns that cover multiple tokens into one - then store that into patterns file - but makes it less readable since one needs to look into the pattern file to see what the short pattern is ... that will make match line short but at the cost of having to look into pattern file ... not too bad - just though that there may be a way syntactically to continue one line into the next ...

8

.. I have filter lines that break the line into 10+ fields ... those get very long ...