How to build grok filter

[smam]

Hello,
I want to understand how to build grok filter since other posts did not make it clear enough for me.

Here are example lines I want to send:

30,05/10/22,07:30:27,DNS Update Request,<ip>,<name>,,,0,6,,,,,,,,,0
11,05/10/22,07:30:27,Renew,<ip>,<name>,04EA56A998EA,,303150361,0,,,,<MAC>.0,,,,0
31,05/10/22,07:30:27,DNS Update Failed,<IP>,<name>,,0,6,,,,,,,,,9004

Here is the filter i built:

input {
  file {
        path => [ "/etc/filebeat/dhcp/test.log" ]
        add_field => { "testvalue" => "Shipped with logstash :)" }
        start_position => "beginning"
  }
}

filter {
  grok{
        match => { "message" => "%{code} %{DATE} %{TIME} %{EVENT} %{IP} %{NAME}" }
  }
  if "DNS" in [EVENT] {
        drop {
                                                                    
        }
  }
}


output {
  elasticsearch {
    hosts => "https://elasticsearch:9200"
    cacert => '/ca.crt'
    user => 'logstash'
    password => ?
    index => "logstash-%{+yyyy.MM.dd}-dhcp"
  }
}

I would like to filter the log lines in a way, in which it is represented as:

Code: x
Date: x
Time: x
IP: x
Name: x

Because currently it is one line(1:1 the same line as in the log file), which makes it hard to filter and work with

[Rios]

Make your life easier, use csv filter plugin.

[ibra_013 (Ibra13)]

Hi,

It will be like this using Grok, but there is other options like dissect or csv

this link is your friend for grok.

filter {
  grok{
        match => { "message" => "%{NUMBER:code},%{DATE:Date},%{TIME:Time},%{DATA:EVENT},%{IP:Ip},%{WORD:NAME},%{GREEDYDATA:Rest}" }
  }
  if "DNS" in [EVENT] {
        drop {
                                                                    
        }
  }
}

[smam]

Hello, thank you. The pattern you provided "does not match the input data" but I got a feeling for how to construct the filter. With you showing me to assign Datatypes and seperate them by, in this case, ",". I will now try to make it work and then see if the csv filter makes it easier. Thank you!

[ibra_013 (Ibra13)]

Hi,

in what does not match the input data?

[smam]

If I paste the lines into "Sample data" and your filter into "Grok Pattern":
After clicking "Simulate" I get a pop-up saying: Provided Grok patterns do not match data in the input

[ibra_013 (Ibra13)]

Hi,

i used this input data ;

30,05/10/22,07:30:27,DNS Update Request,10.0.0.0,user,,0,6,,,,,,,,,0

[smam]

Hmm strange..I do not know why its working for you and not for me..

Even if I paste your input data, I get the same error

[smam]

This should be right?

[ibra_013 (Ibra13)]

Hi,

Grok debugger is used only to check the grok part, what you are trying to do on the screenshot is on the Logstash.

[smam]

I see. I made it work just fine with csv, but I will try to also understand logstash grok filter. Thank you