Match against blacklist


#1

I have DNS logs which I'd like to match against a blacklist.
I've imported the dns logs via a grok filter.

Can anyone point me in the right direction to match my parsed URL field with a blacklist in some way?


(Magnus Bäck) #2

Perhaps the translate filter can help? It allows you to look up and replace field values from an external dictionary file that's refreshed at regular (and configurable) intervals. I think it only does exact matches, though.


#3

Looks to be in the right direction, though there's no need to replace.
Matching and adding a tag is enough, or not matching and drop.


(Magnus Bäck) #4

Sure, but you can implement match-and-tag or match-and-drop semantics using that filter.


#5

Okey, but wouldn't the filter still need something to be replaced to be successful?
My dictionary (blacklist) only hold one "version".


(Magnus Bäck) #6

Copy the hostname field (or whatever) to a new field. Have the translate filter translate that field to a a fixed string. Then check if the translated field is equal to that fixed string.


#7

Something like this?

filter {
translate {
field => "URL"
destination => "matched"
dictionary => [
"blacklisted1.url", "Yes",
"blacklisted2.url", "Yes",
"blacklisted3.url", "Yes"
]
}
}

And then use a YAML-file with "dictionary_path" for a large blacklist?


(Magnus Bäck) #8

Yeah, something like that should be fine.


(system) #9