Determining reason for stuck grok filter

Elastic Stack Logstash
4

That first pattern has 9 DATA patterns in it. It is going to be very expensive for it to determine that it does not match a field. Whereever possible, replace DATA with something else. If a field cannot contain a space then replace %{DATA:cipher} with

(?<cipher>[^ ]+)

and so on. Ideally you should only have one DATA or GREEDYDATA and it should be the last item in the pattern.

Also, most importantly, anchor your patterns. Read this elastic blog entry. If you are matching the whole of a line there is a massive performance difference between the cost of

%{FASP_DATE:timestamp} %{WORD:hostname} ascp[%{POSINT:pid}]: LOG FASP Session Params uuid=%{UUID:uuid} userid=%{NUMBER:userid} user="%{WORD:remote_user}"

and

^%{FASP_DATE:timestamp} %{WORD:hostname} ascp[%{POSINT:pid}]: LOG FASP Session Params uuid=%{UUID:uuid} userid=%{NUMBER:userid} user="%{WORD:remote_user}"

when they do not match.

Also, grok looks like the wrong approach to me. I would try using to grok with this

^%{FASP_DATE:timestamp} %{WORD:hostname} ascp[%{POSINT:pid}]: LOG FASP Session %{WORD:operation} %{GREEDYDATA:restOfLine}

and then use a kv filter to parse restOfLine.

1 Like