How to remove event with "'<nil>'" value on an IP type field

Elastic Stack Logstash
1

Hello,

I am trying to drop events every time the field [dns.resolved_ip] is ''. I have tried multiple approachs but without successs.

"reason"=>"failed to parse field [dns.resolved_ip] of type [ip] in document with id 'G5mrqYABOI1CdVDindTb'. Preview of field's value: ''", "caused_by"=>{"type"=>"illegal_argument_exception", "reason"=>"'' is not an IP string literal."

Two filters i tried to use:

filter {
    if [agent][type] == "packetbeat" {
        if ![dns][resolved_ip] {
           drop{}
        }
    }
}

and i also have tried:

filter {
    if [agent][type] == "packetbeat" {
        if [dns][resolved_ip] == "<nil>" { #i also tried with "'<nil>'"
           drop{}
        }
    }
}

Do you have any guesses?
Thank you,

2

Preview of field's value: ''" says the value is '' not <nil>.

Try just if [dns][resolved_ip] == "" to match that value.

3

I don't know why but i probably deleted the correct logstash log.

Preview of field's value: '<nil>'", "caused_by"=>{"type"=>"illegal_argument_exception", "reason"=>"'<nil>' is not an IP string literal."

The above log is the correct one.

4

What you have is correct.

Maybe the if [agent][type] == "packetbeat" condition isn't being met?

input {
  generator {
    lines => [ '{ "msg": "<nil>" }' ]
    count => 1
    codec => json
  }
}
filter {
  if [msg] == "<nil>" {
    mutate { add_tag => "condition met" }
  }

}
output {
  stdout { codec =>  "json_lines" }
}

Output

{
    "msg": "<nil>",
    "@timestamp": "2022-05-09T18:07:02.875Z",
    "tags": [
        "condition met"
    ]
}
1 Like
5

The condition if [agent][type] == "packetbeat" is being met, since i have a remove_fields and it is working. I can see the documents being indexed on elastic without the fields i removed.
Like this:

filter {
    if [agent][type] == "packetbeat" {
        if [dns][resolved_ip] == "<nil>" {
          drop {}
        }
        mutate {
           remove_field => ["[dns][additionals_count]", "[dns][opt][udp_size]"]
        }
    }
}
6

The packetbeat documentation says dns.resolved_ip is an array. Does it work if you use this?

    if [dns][resolved_ip][0] == "<nil>" {
1 Like
7

Your solution seems to work! (from the time interval that i am checking the logs, they didn't appear again)
Thank you very much for both of your answers.

8

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.