Need help in grok pattern

Elastic Stack Logstash
1
  1. How do I drop certain words in between my log lines? I dont want to write a parser for them or not useful.

E.g. Below log

29-Apr-2018 12:01:19.760 rpz: info: client 192.168.5.103#60868 (abc.com): rpz QNAME Local-Data rewrite abc.com via abc.com.malware.trap

I want to drop

  • rpz:
  • info:
  • client
  • Local-Data

  1. And can you suggest what is the correct method for matching Date at the beginning

  2. Also how do I match entire word when there are special characters?
    e.g. Local-Data
    Here are my Grok patterns

%{MONTHDAY:day}-%{MONTH:month}-%{YEAR:year} %{TIME:time} %{WORD:rpz}: %{WORD:log_category}: %{WORD:client} %{IPV4:clientipaddr}#%{NUMBER:src_port} (%{HOSTNAME:qdomain}): %{WORD:rpz2} %{WORD:dnsfw_method} %{WORD:loc_data-}

2

How do I drop certain words in between my log lines? I dont want to write a parser for them or not useful.

You still have to match them but you don't have to capture their contents into a field, i.e. use %{WORD} or similar.

And can you suggest what is the correct method for matching Date at the beginning

Should work:

(?<timestamp>%{MONTHDAY}-%{MONTH}-%{YEAR} %{TIME})

Also how do I match entire word when there are special characters?

In many cases NOTSPACE or some other exclusionary pattern that matches any character up to a certain boundary in more useful than an inclusionary pattern that specifies which characters to match.

3

Ok for NOTSPACE then what should be the pattern for Local-Data? Since my pattern is only matching till Local i.e. %{WORD:loc_data}

4

NOTSPACE will match a sequence of non-whitespace characters while WORD matches sequences of characters that occur on words (which, perhaps oddly, doesn't include hyphens). If this doesn't answer your question I don't understand what you're asking.

5

Great that worked!! Thanks for your reply.

6

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.