Match against blacklist

Elastic Stack Logstash
1

I have DNS logs which I'd like to match against a blacklist.
I've imported the dns logs via a grok filter.

Can anyone point me in the right direction to match my parsed URL field with a blacklist in some way?

2

Perhaps the translate filter can help? It allows you to look up and replace field values from an external dictionary file that's refreshed at regular (and configurable) intervals. I think it only does exact matches, though.

3

Looks to be in the right direction, though there's no need to replace.
Matching and adding a tag is enough, or not matching and drop.

4

Sure, but you can implement match-and-tag or match-and-drop semantics using that filter.

5

Okey, but wouldn't the filter still need something to be replaced to be successful?
My dictionary (blacklist) only hold one "version".

6

Copy the hostname field (or whatever) to a new field. Have the translate filter translate that field to a a fixed string. Then check if the translated field is equal to that fixed string.

7

Something like this?

filter {
translate {
field => "URL"
destination => "matched"
dictionary => [
"blacklisted1.url", "Yes",
"blacklisted2.url", "Yes",
"blacklisted3.url", "Yes"
]
}
}

And then use a YAML-file with "dictionary_path" for a large blacklist?

8

Yeah, something like that should be fine.