# Filtering duplicate fields in Error Watchers

**URL:** <https://discuss.elastic.co/t/filtering-duplicate-fields-in-error-watchers/302710>\
**Category:** Kibana\
**Tags:** elastic-stack-alerting\
**Created:** [April 19, 2022, 11:10am UTC](https://discuss.elastic.co/t/filtering-duplicate-fields-in-error-watchers/302710 "2022-04-19T11:10:35Z")\
**Posts on this page:** 2\
**Page:** 1

<div class="post-metadata">

**Author:** ![HughEvans](https://avatars.discourse-cdn.com/v4/letter/h/e99b99/32.png) [@HughEvans](https://discuss.elastic.co/u/HughEvans)\
**Post date:** [April 19, 2022, 11:10am UTC](https://discuss.elastic.co/t/filtering-duplicate-fields-in-error-watchers/302710/1 "2022-04-19T11:10:35Z")

</div>

I'm trying to filter out duplicates hits in the following error watcher. Currently when there are multiple hits in the same namespace the output looks like this:

````auto
"body": "{\"link_names\": \"1\", \"username\": \"Kibana - ida-identity-api Full GC Watcher minor to major\", \"text\": \"\nNamespace(s): foo foo foo \", \"icon_emoji\": \":put_litter_in_its_place:\" }"```

````

I want to remove the duplicate namespaces so it looks like this:

```auto
"body": "{\"link_names\": \"1\", \"username\": \"Kibana - ida-identity-api Full GC Watcher minor to major\", \"text\": \"\nNamespace(s): foo \", \"icon_emoji\": \":put_litter_in_its_place:\" }"

```

I tried to add a condition that would filter out duplicates but I get the "internal server error message and no other feedback". Is anyone able to explain to me what the right approach for this would be? Thank you

````auto
  "trigger": {
    "schedule": {
      "interval": "10m"
    }
  },
  "input": {
    "search": {
      "request": {
        "search_type": "query_then_fetch",
        "indices": [
          "eue*"
        ],
        "types": [],
        "body": {
          "query": {
            "bool": {
              "must": [
                {
                  "match_phrase": {
                    "fields.application": "api"
                  }
                },
                {
                  "match_phrase": {
                    "json.message": {
                      "query": "tenured"
                    }
                  }
                },
                {
                  "match_phrase": {
                    "json.message": {
                      "query": "GC"
                    }
                  }
                }
              ],
              "filter": {
                "range": {
                  "@timestamp": {
                    "gt": "now-100m",
                    "lt": "now"
                  }
                }
              }
            }
          },
          "aggs": {
            "namespaces": {
              "aggs": {
                "namespaces_field": {
                  "terms": {
                    "field": "_source.fields.namespace"
                  }
                }
              },
              "nested": {
                "path": "_source.fields"
              }
            }
          }
        }
      }
    }
  },
  "condition": {
    "script" : "return ctx.payload.hits.hits.filter((value,index) => ctx.payload.hits.hits.indexOf(value) === index);"
  },
  "actions": {
    "NotifySlack": {
      "webhook": {
        "scheme": "https",
        "host": "hooks.slack.com",
        "port": 443,
        "method": "post",
        "path": "services/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX",
        "params": {},
        "headers": {
          "Content-type": "application/json"
        },
        "body": "{\"link_names\": \"1\", \"username\": \"Kibana - ida-identity-api Full GC Watcher minor to major\", \"text\": \"\nNamespace(s): {{#ctx.payload.hits.hits}} {{_source.fields.namespace}} {{/ctx.payload.hits.hits}}\n \", \"icon_emoji\": \":put_litter_in_its_place:\" }"
      }
    }
  }
}```
````

---

<div class="post-metadata">

**Author:** ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)\
**Post date:** [May 17, 2022, 11:11am UTC](https://discuss.elastic.co/t/filtering-duplicate-fields-in-error-watchers/302710/2 "2022-05-17T11:11:29Z")

</div>

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.