Logstash Grok Filter patteren

Elastic Stack Logstash
1

Hi All,

I want to write Grok Filter for 100 Unique type of logs. Currently i have written for syslog and one log of our application. So i need to write 99 more grok filter. Can you please let me know how i can make it easy.

As per my knowledge i need to write 99 grok filter and need to append them with earlier 2 filter. But it is very complicated way...Please suggest some better way to complete my task.

Thanks in Advance.

Below is my current Grok Filter:

  filter {
   grok {
   match => { "message" => [ "%{SYSLOGTIMESTAMP:syslog_timestamp %{SYSLOGHOST:syslog_hostname} %{DATA:syslog_program}(?:\[%{POSINT:syslog_pid}\])?: %{GREEDYDATA:syslog_message}", "(?<loglevel>\w\,)\s\[(?<datetime>[\d\-\w\:\.\s\#]+)\]\s\s(?<loglevel2>[\w]+)\s\--\s\:\s\[(?<session-id>[\d\-\w]+)\]\s(?<message-brew>[\w\s\d\(\:\.\)]+)" ] }
}
}
2

A single grok filter can list multiple expressions so you don't need 100 grok filters. See the example in the grok filter documentation.

If you don't want your configuration or logs to get garbled when posting here make sure you format it as preformatted text using Markdown notation or the </> toolbar button.

3

Means, you are saying that i can continue as i have written 2 regex which are pasted above. I can continue with same procedure. Please confirm.

4

Oh, right. Didn't spot that you actually have two expressions in that filter. Yeah, that's fine.

5

But if i have 100 unique logs then it will be little bit lengthy procedure...Can we use any other method?

6

Maybe, but without knowing what the logs look like it's impossible to tell.

7

Thank you Magnusbaeck. Once i have the logs from our developer team, i will provide some sample logs.

8

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.