Transform and simplify long regex with dissect or grok?

Elastic Stack Logstash
1

Hello,
I have some long and heavy regex and i wonder if i can simplify and gain in term of performances by using dissect
here are some examples

  1. "^<%{NONNEGINT:syslog_priority:int}>1 (?:-|%{TIMESTAMP_ISO8601:syslog_timestamp}) (?:-|%{SYSLOGHOST:syslog_hostname}) (?:%{PROG:syslog_process}|-) (?:-|%{POSINT:syslog_uid}) (?:-|%{WORD:syslog_message_id}) (?:-|(?<syslog_structured_data>([.*?[^\]])+))(?:%{SPACE}%{GREEDYDATA:syslog_message}|)$

  2. "^<%{NONNEGINT:syslog_priority:int}>(1 )?(%{TIMESTAMP_ISO8601:syslog_timestamp} %{SYSLOGHOST:syslog_hostname}|%{SYSLOGHOST:syslog_hostname}: (\d{4} )?%{SYSLOGTIMESTAMP:syslog_timestamp}) (?<syslog_process>\b[\w._/%-]+\b)(?:[%{POSINT:syslog_uid}])?:?%{SPACE}%{GREEDYDATA:syslog_message}$",*

3)"^(<%{NONNEGINT:syslog_priority:int}>)?(\d{4} )?%{SYSLOGTIMESTAMP:syslog_timestamp} %{SYSLOGHOST:syslog_hostname} (?<syslog_process>\b[\w._/%-]+\b)(?:[%{POSINT:syslog_uid}])?:?%{SPACE}%{GREEDYDATA:syslog_message}$",

thank you for your help

2

Most of the time you can replace your grok with a similar dissect filter, but how they work is completely different.

Grok will validate the data type, dissect is just positional, it will not validate anything, it will parse by the position.

If your logs have always the same structure and the values are always in the same place, then you can use dissect.

Sometimes you can also combine different parsers like grok, dissect, kv, json etc.

Can you share sample messages for your patterns?

1 Like
3

This looks like the syslog format, which is more suitable grok in case you have optional fields.
Is this 3 types of grok patterns in the same matching or you have 3 different syslog sources and the grok patterns?

4

the sources are differents.
In the pipeline from which these grok come, the output is an index elk.
I wonder if it would be more logic to use http filter in that case and write directly in the index?

5

Not sure how this would help as you would still need to parse the message.

6

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.