# Splunk lookup file equivalent **URL:** \ **Category:** Elasticsearch\ **Created:** [August 20, 2019, 11:34pm UTC](https://discuss.elastic.co/t/splunk-lookup-file-equivalent/196015 "2019-08-20T23:34:50Z")\ **Posts on this page:** 4\ **Page:** 1
**Author:** ![wengsengtam](https://avatars.discourse-cdn.com/v4/letter/w/4da419/32.png) [@wengsengtam](https://discuss.elastic.co/u/wengsengtam)\ **Post date:** [August 20, 2019, 11:34pm UTC](https://discuss.elastic.co/t/splunk-lookup-file-equivalent/196015/1 "2019-08-20T23:34:50Z")
Hello Elastic Forum members, I have been exploring the different possible data enrichments that are available in Splunk. However, I am trying to understand if there is a plugin or module that can be added to Elastic to create and manage lookup files, like the lookup editor in Splunk, that comes with Elastic. I also want to know if there is a way to put a lookup file into Elastic so that it can be used in data enrichment and searches. It will be needed for creating alerting. Are there other alternatives to store this lookup data in other tools, like a database, and link it with Elastic via JDBC? For example: > type,source,ip > C2, Test1, 192.168.1.13 > C2,"Test1,192.168.1.14 > TOR,Test2,192.168.2.23 > TOR,Test2,192.168.2.24 > Malware,Test1,192.168.1.69 I know I can put a single list of items and use a TERMS query clause. However, I have not yet finished experimenting to see if it supports wildcards and/or nest fields in the document to allow for more structured data. ``` PUT iplist/_doc/black2 { "ip": ["192.168.1.13","192.168.1.14","192.168.2.23","192.168.2.24","192.168.1.69"] } GET iplist/_search PUT traffic/_doc/_bulk { "index":{}} {"seq":1, "sip" : "192.168.8.2", "dip" : "192.168.1.13"} { "index":{}} {"seq":2, "sip" : "192.168.8.4", "dip" : "192.168.1.14"} { "index":{}} {"seq":3, "sip" : "192.168.8.8", "dip" : "192.168.2.23"} { "index":{}} {"seq":4, "sip" : "192.168.8.8", "dip" : "192.168.2.23"} { "index":{}} {"seq":5, "sip" : "192.168.8.2", "dip" : "192.168.1.13"} { "index":{}} {"seq":6, "sip" : "192.168.8.4", "dip" : "192.168.1.14"} { "index":{}} {"seq":7, "sip" : "192.168.1.69", "dip" : "192.168.8.2"} { "index":{}} {"seq":8, "sip" : "192.168.1.69", "dip" : "192.168.8.2"} GET traffic/_search GET traffic/_search?pretty { "query": { "terms": { "dip": { "index":"iplist", "id": "black2", "path":"ip" } } } } ``` Best Regards, Weng Seng. ---
**Author:** ![Christian\_Dahlqvist](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/christian_dahlqvist/32/4617_2.png) [@Christian\_Dahlqvist](https://discuss.elastic.co/u/Christian_Dahlqvist)\ **Post date:** [August 21, 2019, 5:18am UTC](https://discuss.elastic.co/t/splunk-lookup-file-equivalent/196015/2 "2019-08-21T05:18:19Z")
When working with the Elastic Stack this is generally done at index time. Please see [this blog post](https://www.elastic.co/blog/schema-on-write-vs-schema-on-read) for a more detailed discussion. ---
**Author:** ![wengsengtam](https://avatars.discourse-cdn.com/v4/letter/w/4da419/32.png) [@wengsengtam](https://discuss.elastic.co/u/wengsengtam)\ **Post date:** [August 21, 2019, 10:36am UTC](https://discuss.elastic.co/t/splunk-lookup-file-equivalent/196015/3 "2019-08-21T10:36:25Z")
> [@Christian\_Dahlqvist](#): > > When working with the Elastic Stack this is generally done at index time. Please see [this blog post](https://www.elastic.co/blog/schema-on-write-vs-schema-on-read) for a more detailed discussion. The ability to build/maintain dynamic whitelists and blacklists is not entirely covered by the schema-on-write or schema-on-read discussion. Even if the whitelists and/or blacklists are processed at ingestion, instead of search time, there needs to be a way to build and maintain them. Splunk has a way to maintain lookup files out of the box. I would like to know if there is an equivalent in Elastic. The current course material I have taken does not cover logstash or ingestion pipelines. I will try to find more course material on ingestion pipelines to see if they can be used to generate alerts, or if alerting is still based on search time as implied by this document. > **[Kibana Alerting: Alerts & Actions for Elasticsearch data](https://www.elastic.co/kibana/alerting)** > > Monitor all of your alerts in one place inside Kibana with the alerting and actions framework for Elasticsearch. Detect changes and anomalies in your logging, APM, and SIEM data. Do you have any examples of how ingestion pipelines are used to process firewall logs for Threat Intel hits, including retrospective searches covering events from the past. Do you have examples of how whitelists and blacklists might be implemented for IDS alerts? Where some alerts might be dynamically whitelisted for specific combinations of source/destination/signature id? My attempts to find these examples via Google have not been successful. I guess I am not the only person pondering this question, since Yelp has written ElastAlert to " \* “Match when a certain field matches a blacklist/whitelist” ( `blacklist` and `whitelist` type)". ([ElastAlert - Easy & Flexible Alerting With Elasticsearch — ElastAlert 0.0.1 documentation](https://elastalert.readthedocs.io/en/latest/elastalert.html#configuration)) Thanks, Weng Seng. ---
**Author:** ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)\ **Post date:** [September 18, 2019, 10:36am UTC](https://discuss.elastic.co/t/splunk-lookup-file-equivalent/196015/4 "2019-09-18T10:36:26Z")
This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.