How effective translate field in logstash

Hi,

I have a set of threat intel feeds i want to check whether those blacklisted ip occurs in my logs... i tried using translate field it works fine...i want to know whether we can compare billions of blacklisted ip using translate field or is there any limitation

Thanks

With a translate filter it keeps a hash of all the entries in memory, so having billions of entries will require a large amount of memory, but if you have the memory it should be OK.

A cidr filter might be a better fit, depending on the details on the set of IPs.

ok thanks.... but again cidr filter uses only three fields a particular address,network,network path so as per my understanding i should define all the ip in a file and use the network_path field which points to that file ...

filter {
cidr {
add_tag => [ "linklocal" ]
address => [ "%{clientip}" ]
network_path => [ "/opt/blacklistedip" ]
}
}

and one more doubt is there any other method to combine threat intel and elastic

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.