I have a set of threat intel feeds i want to check whether those blacklisted ip occurs in my logs... i tried using translate field it works fine...i want to know whether we can compare billions of blacklisted ip using translate field or is there any limitation
With a translate filter it keeps a hash of all the entries in memory, so having billions of entries will require a large amount of memory, but if you have the memory it should be OK.
A cidr filter might be a better fit, depending on the details on the set of IPs.
ok thanks.... but again cidr filter uses only three fields a particular address,network,network path so as per my understanding i should define all the ip in a file and use the network_path field which points to that file ...
Apache, Apache Lucene, Apache Hadoop, Hadoop, HDFS and the yellow elephant
logo are trademarks of the
Apache Software Foundation
in the United States and/or other countries.