# Exclusion List for Watcher

**URL:** <https://discuss.elastic.co/t/exclusion-list-for-watcher/224953>\
**Category:** Elasticsearch\
**Tags:** elastic-stack-alerting\
**Created:** [March 25, 2020, 7:34am UTC](https://discuss.elastic.co/t/exclusion-list-for-watcher/224953 "2020-03-25T07:34:37Z")\
**Posts on this page:** 7\
**Page:** 1

<div class="post-metadata">

**Author:** ![ankitsynX](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/ankitsynx/32/64535_2.png) [@ankitsynX](https://discuss.elastic.co/u/ankitsynX)\
**Post date:** [March 25, 2020, 7:34am UTC](https://discuss.elastic.co/t/exclusion-list-for-watcher/224953/1 "2020-03-25T07:34:38Z")

</div>

Hi,

I am trying to make an index which could be used as a static list of fields with data that will be used as exclusions in watcher alerts.

At the moment, I am doing that using exclude in the watcher. But for a larger list is it possible to be done using some sort of lookup. That can be used as a whitelist or blacklist in the watcher.

Also, will the use lookup list significant impact on performance. What would be the best way of doing it if any.

Thanks in advance!

Ankit

---

<div class="post-metadata">

**Author:** ![spinscale](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/spinscale/32/25011_2.png) [@spinscale](https://discuss.elastic.co/u/spinscale)\
**Post date:** [March 30, 2020, 1:49pm UTC](https://discuss.elastic.co/t/exclusion-list-for-watcher/224953/2 "2020-03-30T13:49:03Z")

</div>

How about the [Terms Lookup Query](https://www.elastic.co/guide/en/elasticsearch/reference/7.6/query-dsl-terms-query.html#query-dsl-terms-lookup)? Would that help you?

---

<div class="post-metadata">

**Author:** ![ankitsynX](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/ankitsynx/32/64535_2.png) [@ankitsynX](https://discuss.elastic.co/u/ankitsynX)\
**Post date:** [March 31, 2020, 9:36am UTC](https://discuss.elastic.co/t/exclusion-list-for-watcher/224953/3 "2020-03-31T09:36:49Z")

</div>

Hi Alex,

This is useful, but putting it in a watch condition to compare against matching terms from search in the watch is something I am trying to figure out.

Maybe I need to look at it in a different way. Probably matching arrays in the watch search against the lookup index. Any leads if you could share to compare specific keys of one doc to another index's.

Thanks,  
Ankit

---

<div class="post-metadata">

**Author:** ![spinscale](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/spinscale/32/25011_2.png) [@spinscale](https://discuss.elastic.co/u/spinscale)\
**Post date:** [March 31, 2020, 11:15am UTC](https://discuss.elastic.co/t/exclusion-list-for-watcher/224953/4 "2020-03-31T11:15:16Z")

</div>

You could also use a chain input, then execute a search to look up those filter values, and use that response in the second search for exclusion.

---

<div class="post-metadata">

**Author:** ![ankitsynX](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/ankitsynx/32/64535_2.png) [@ankitsynX](https://discuss.elastic.co/u/ankitsynX)\
**Post date:** [April 21, 2020, 11:10am UTC](https://discuss.elastic.co/t/exclusion-list-for-watcher/224953/5 "2020-04-21T11:10:24Z")

</div>

Hi Alex,

I am trying to implement a List lookup functionality using the same approach, but somehow painless is not working for me. The triple quotes as mentioned in documentation for condition context doesn't work on watcher.

```auto
{
"trigger": {
  "schedule": {
    "interval": "1m"
    }
  },

"input": {
    "chain": {
      "inputs": [ 
        {
          "first": {    
            "search": {
            "request": {
              "search_type": "query_then_fetch",
              "indices": ["packetbeat-*"],
              "body": {
                "query": {
                      "bool": {
                        "filter": [
                            
                            { "range": {
                            "@timestamp": {"gte": "now-5m"}
                                }
                            }
                    ]
                  }  
                },
                "size": 0,
                "aggs": {
                    "source_ip":{
                        "terms" :{
                            "field": "source.ip"
                        }
                    }
                }

              }
            }
            }
          }  
        },      
        {
          "second": {
            "search": {
            "request": {
              "search_type": "query_then_fetch",
              "indices": ["ioc_list"],
              "body": {
                "query": {
                  "match_all": {}
                },
                "size": 0,
                "aggs": {
                    "ioc_ip":{
                        "terms" :{
                            "field": "ip"
                        }
                    }
                }
              }

            }
            }
          }
        }    
      ]
    }     
},

"condition": {
    "script": 
          """
              int matches = 0;
              int var = 0;
              for (int i=0; i < ctx.payload.first.aggregations.source_ip.buckets.length; i++) {
                  for ( int j=0; i < ctx.payload.first.aggregations.ioc_ip.buckets.length; j++ ) {
                    if (ctx.payload.first.aggregations.source_ip.buckets.[i].key == ctx.payload.second.aggregations.ioc_ip.buckets.[j].key) {
                      matches++ ;
                      var = i;
                    }
                }      
              }        
              if ( matches >= 1) {
                  return ctx.payload.first.aggregations.source_ip.buckets.[var].key;
              }       
          """
        
       
}

```

---

<div class="post-metadata">

**Author:** ![ankitsynX](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/ankitsynx/32/64535_2.png) [@ankitsynX](https://discuss.elastic.co/u/ankitsynX)\
**Post date:** [April 24, 2020, 4:10pm UTC](https://discuss.elastic.co/t/exclusion-list-for-watcher/224953/6 "2020-04-24T16:10:46Z")

</div>

Just to give this a closure for anyone who goes down this path:

**Issue 1 :** Version 7.5.2 and below does not support triple quotes in script in the Kibana Watcher UI condition context, it's a reported bug on Github and probably would be fixed in 7.7. Workaround was to use watcher API to test and write the watch

**Issue 2:** The script had typos and return type was incorrect. Updated working script is below:

```auto
"condition": {
    "script": 
          """
              int matches = 0;
              for (int i=0; i < ctx.payload.first.aggregations.source_ip.buckets.size(); i++) {
                  for ( int j=0; j < ctx.payload.second.aggregations.ioc_ip.buckets.size(); j++ ) {
                    if (ctx.payload.first.aggregations.source_ip.buckets[i].key == ctx.payload.second.aggregations.ioc_ip.buckets[j].key) {
                      matches += 1;
                    }
                }      
              }        
              if ( matches >= 1) {
                  return true;
              }       
          """
        
       
}

```

Glad was able to find a solution after going around for a month (not full time ofcourse) for a feature which is very crucial in security analytics !  
Code could be better I agree, I am not a coder by profession but a security practitioner. Feel free to give suggestions.

Cheers!

---

<div class="post-metadata">

**Author:** ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)\
**Post date:** [May 22, 2020, 4:10pm UTC](https://discuss.elastic.co/t/exclusion-list-for-watcher/224953/7 "2020-05-22T16:10:51Z")

</div>

Closed on May 22, 2020, 4:10pm UTC

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.