How to remove event with "'<nil>'" value on an IP type field

Hello,

I am trying to drop events every time the field [dns.resolved_ip] is ''. I have tried multiple approachs but without successs.

"reason"=>"failed to parse field [dns.resolved_ip] of type [ip] in document with id 'G5mrqYABOI1CdVDindTb'. Preview of field's value: ''", "caused_by"=>{"type"=>"illegal_argument_exception", "reason"=>"'' is not an IP string literal."

Two filters i tried to use:

filter {
    if [agent][type] == "packetbeat" {
        if ![dns][resolved_ip] {
           drop{}
        }
    }
}

and i also have tried:

filter {
    if [agent][type] == "packetbeat" {
        if [dns][resolved_ip] == "<nil>" { #i also tried with "'<nil>'"
           drop{}
        }
    }
}

Do you have any guesses?
Thank you,

Preview of field's value: ''" says the value is '' not <nil>.

Try just if [dns][resolved_ip] == "" to match that value.

I don't know why but i probably deleted the correct logstash log.

Preview of field's value: '<nil>'", "caused_by"=>{"type"=>"illegal_argument_exception", "reason"=>"'<nil>' is not an IP string literal."

The above log is the correct one.

What you have is correct.

Maybe the if [agent][type] == "packetbeat" condition isn't being met?

input {
  generator {
    lines => [ '{ "msg": "<nil>" }' ]
    count => 1
    codec => json
  }
}
filter {
  if [msg] == "<nil>" {
    mutate { add_tag => "condition met" }
  }

}
output {
  stdout { codec =>  "json_lines" }
}

Output

{
    "msg": "<nil>",
    "@timestamp": "2022-05-09T18:07:02.875Z",
    "tags": [
        "condition met"
    ]
}
1 Like

The condition if [agent][type] == "packetbeat" is being met, since i have a remove_fields and it is working. I can see the documents being indexed on elastic without the fields i removed.
Like this:

filter {
    if [agent][type] == "packetbeat" {
        if [dns][resolved_ip] == "<nil>" {
          drop {}
        }
        mutate {
           remove_field => ["[dns][additionals_count]", "[dns][opt][udp_size]"]
        }
    }
}

The packetbeat documentation says dns.resolved_ip is an array. Does it work if you use this?

    if [dns][resolved_ip][0] == "<nil>" {
1 Like

Your solution seems to work! (from the time interval that i am checking the logs, they didn't appear again)
Thank you very much for both of your answers.

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.