DNS Filter wont work

[elasticcp]

Hey i tried to resolve a hostname in to a DNS.
My Input data Looks like this:

> {
>     "query_status": "ok",
>     "urls": [
>         {
>             "id": "223622",
>             "urlhaus_reference": "https:\/\/urlhaus.abuse.ch\/url\/223622\/",
>             "url": "http:\/\/45.61.49.78\/razor\/r4z0r.mips",
>             "url_status": "offline",
>             "host": "45.61.49.78",
>             "date_added": "2019-08-10 09:02:05 UTC",
>             "threat": "malware_download",
>             "blacklists": {
>                 "gsb": "not listed",
>                 "spamhaus_dbl": "not listed",
>                 "surbl": "not listed"
>             },
>             "reporter": "zbetcheckin",
>             "larted": "true",
>             "tags": [
>                 "elf"
>             ]
>         },
>         {
>             "id": "223621",
>             "urlhaus_reference": "https:\/\/urlhaus.abuse.ch\/url\/223621\/",
>             "url": "http:\/\/45.61.49.78\/razor\/r4z0r.sh4",
>             "url_status": "offline",
>             "host": "45.61.49.78",
>             "date_added": "2019-08-10 09:02:03 UTC",
>             "threat": "malware_download",
>             "blacklists": {
>                 "gsb": "not listed",
>                 "spamhaus_dbl": "not listed",
>                 "surbl": "not listed"
>             },
>             "reporter": "zbetcheckin",
>             "larted": "true",
>             "tags": [
>                 "elf",
>                 "mirai"
>             ]
>         }
>     ]
> }

and my DNS filter.

dns
{
resolve => ["[urls][host]"]
action => "replace"

}

So what i want is to replace all Host names with a IP with the DNS Filter.
For Example: "host" : "eBay.com" => "host" : "IP of the Host"
I dont get a error message or whatever it just dont resolve all hosts.
Maybe some of you know why not.

thank you

[Badger]

@elasticcp

resolve => ["[urls][*][host]"]

That will not work. The resolve option will accept an array of strings, but I think you are going to have to use ruby to build such an array. That will allow you to get an array of IP addresses, but does not modify the url field. You could write more ruby to do that.

[elasticcp]

Yes i think i could use a loop to iterate the Array and put the iterated value instead of "*" …
Where can i learn About ruby the best.
Is logstash using Jruby or Ruby.

[Badger]

Actually there is a problem. You can build a list of hostnames using

    ruby {
        code => '
            hostlist = []
            u = event.get("urls")
            u.each_index { |x|
                hostlist << u[x]["host"]
            }
            event.set("hostlist", hostlist)
        '
    }

but there is no way to tell the dns filter to use it. It accepts fields that are arrays but only if they have a single entry. It will not iterate over the array.

If the urls field has a limited length you could use

    dns {
        resolve => ["[urls][0][host]", "[urls][1][host]", "[urls][2][host]"]
        action => "replace"
    }

but you will get a lot of noise in the logs from

[WARN ][logstash.filters.dns     ][main] DNS filter could not resolve missing field {:field=>"[urls][2][host]"}

[elasticcp]

@Badger

hostlist =

The Array length is not over 1000 but it can be less.. but i think it is not a good idea to resolve all ip's separately like this

Is there Maybe a other way of resolving host names to IP'S, Maybe it is possible for ruby script without using Default DNS plugin

[Badger]

I expect you could re-purpose the code from the dns filter.