# Query Help: must\_not keeps matching document

**URL:** <https://discuss.elastic.co/t/query-help-must-not-keeps-matching-document/122762>\
**Category:** Elasticsearch\
**Created:** [March 6, 2018, 6:01pm UTC](https://discuss.elastic.co/t/query-help-must-not-keeps-matching-document/122762 "2018-03-06T18:01:50Z")\
**Posts on this page:** 5\
**Page:** 1

<div class="post-metadata">

**Author:** ![aaronloesattv](https://avatars.discourse-cdn.com/v4/letter/a/3d9bf3/32.png) [@aaronloesattv](https://discuss.elastic.co/u/aaronloesattv)\
**Post date:** [March 6, 2018, 6:01pm UTC](https://discuss.elastic.co/t/query-help-must-not-keeps-matching-document/122762/1 "2018-03-06T18:01:50Z")

</div>

I'm trying to write a watcher alert that triggers when any other watcher alert fails to execute. However, due to a bug, we're getting a lot of failed watch executions because a given watch is executed at the same time by multiple nodes resulting in a version mismatch exception. Until this bug is fixed and we've upgraded versions, i've updated our alert to exclude watch execution failures of this nature. However, what i think should be a working query, is not excluding the failed executions. i've narrowed it down to the "must\_not" clause not kicking out documents that contain the term.

Here is a document that should be excluded (cleaned up for simplicity):

```
{
  "_index": ".watcher-history-7-2018.03.06",
  "_type": "doc",
  "_id": "My_Failing_Test_Alert_624fd320-47a9-476c-87bb-7fa90037e3cf-2018-03-06T16:36:28.323Z",
  "_score": null,
  "_source": {
    "watch_id": "My_Failing_Test_Alert",
    "node": "hlvBERyKR5-XvzTE-Ud44g",
    "state": "failed",
    "status": {
      ... snip ...
    },
    "trigger_event": {
      "type": "schedule",
      "triggered_time": "2018-03-06T16:36:28.323Z",
      "schedule": {
        "scheduled_time": "2018-03-06T16:36:28.190Z"
      }
    },
    "input": {
      ... snip ...
    },
    "condition": {
      ... snip ...
    },
    "metadata": {
      ... snip ...
    },
    "result": {
      ... snip ...
    },
    "exception": {
      "type": "version_conflict_engine_exception",
      "reason": "[doc][My_Failing_Test_Alert]: version conflict, current version [22262] is different than the one provided [22261]",
      "index_uuid": "YR1UZaG-Rf6PEC82jg4YLg",
      "shard": "0",
      "index": ".watches",
      ... snip ...
    }
  },
  "sort": [
    1520354188323
  ]
}

```

Here is my explain query informing me that the document matches:

```
GET .watcher-history-7-2018.03.06/doc/My_Failing_Test_Alert_624fd320-47a9-476c-87bb-7fa90037e3cf-2018-03-06T16:36:28.323Z/_explain
{
  "query": {
    "constant_score": {
      "filter": {
        "bool": {
          "must_not": [
            {
              "term": {
                "exception.type": "version_conflict_engine_exception"
              }
            }
          ],
          "must": [
            {
              "term": {
                "state": "failed"
              }
            },
            {
              "range": {
                "trigger_event.triggered_time": {
                  "gte": "now-2h"
                }
              }
            }
          ]
        }
      }
    }
  }
}

```

And the output of the explain query:

```
{
  "_index": ".watcher-history-7-2018.03.06",
  "_type": "doc",
  "_id": "My_Failing_Test_Alert_624fd320-47a9-476c-87bb-7fa90037e3cf-2018-03-06T16:36:28.323Z",
  "matched": true,
  "explanation": {
    "value": 1,
    "description": "sum of:",
    "details": [
      {
        "value": 1,
        "description": "ConstantScore(+state:failed +trigger_event.triggered_time:[1520351559689 TO 9223372036854775807] -exception.type:version_conflict_engine_exception)",
        "details": []
      },
      {
        "value": 0,
        "description": "match on required clause, product of:",
        "details": [
          {
            "value": 0,
            "description": "# clause",
            "details": []
          },
          {
            "value": 1,
            "description": "DocValuesFieldExistsQuery [field=_primary_term]",
            "details": []
          }
        ]
      }
    ]
  }
}

```

Even if i just reduce the explain query down to just the "must\_not" it still matches:

```
GET .watcher-history-7-2018.03.06/doc/My_Failing_Test_Alert_624fd320-47a9-476c-87bb-7fa90037e3cf-2018-03-06T16:36:28.323Z/_explain
{
  "query": {
    "constant_score": {
      "filter": {
        "bool": {
          "must_not": [
            {
              "term": {
                "exception.type": "version_conflict_engine_exception"
              }
            }
          ]
        }
      }
    }
  }
}

```

And the output:

```
{
  "_index": ".watcher-history-7-2018.03.06",
  "_type": "doc",
  "_id": "My_Failing_Test_Alert_624fd320-47a9-476c-87bb-7fa90037e3cf-2018-03-06T16:36:28.323Z",
  "matched": true,
  "explanation": {
    "value": 1,
    "description": "sum of:",
    "details": [
      {
        "value": 1,
        "description": "ConstantScore(-exception.type:version_conflict_engine_exception +*:*)",
        "details": []
      },
      {
        "value": 0,
        "description": "match on required clause, product of:",
        "details": [
          {
            "value": 0,
            "description": "# clause",
            "details": []
          },
          {
            "value": 1,
            "description": "DocValuesFieldExistsQuery [field=_primary_term]",
            "details": []
          }
        ]
      }
    ]
  }
}

```

The question is what am i not understanding about the query DSL thats preventing me from excluding documents that match a term? I am fairly new to querying with ES but according to this document, this should work.

[https://www.elastic.co/guide/en/elasticsearch/guide/master/combining-filters.html](https://www.elastic.co/guide/en/elasticsearch/guide/master/combining-filters.html)

Thank you for your help

---

<div class="post-metadata">

**Author:** ![aaronloesattv](https://avatars.discourse-cdn.com/v4/letter/a/3d9bf3/32.png) [@aaronloesattv](https://discuss.elastic.co/u/aaronloesattv)\
**Post date:** [March 6, 2018, 7:53pm UTC](https://discuss.elastic.co/t/query-help-must-not-keeps-matching-document/122762/2 "2018-03-06T19:53:32Z")

</div>

is this because the mapping created for the watcher history index has disabled indexing on the exception object?

```
... snip ...
"properties": {
  "condition": {
    "type": "object",
    "enabled": false
  },
  "exception": {
    "type": "object",
    "enabled": false
  },
  "input": {
    "type": "object",
    "enabled": false
  },
... snip ...

```

if so, is there anything in the explain output that would have informed me of this? Also, is there an appropriate way for us to modify the built in templates? Meaning, if i modify the .watch-history-7 index template, will it get overwritten? And lets say that i eventually upgrade where a new index template is made for .watch-history-8, how would i get in front of that to configure it before it gets created?

---

<div class="post-metadata">

**Author:** ![aaronloesattv](https://avatars.discourse-cdn.com/v4/letter/a/3d9bf3/32.png) [@aaronloesattv](https://discuss.elastic.co/u/aaronloesattv)\
**Post date:** [March 6, 2018, 7:54pm UTC](https://discuss.elastic.co/t/query-help-must-not-keeps-matching-document/122762/3 "2018-03-06T19:54:43Z")

</div>

i guess the other way for me to deal with this is through painless looking at the data instead of relying on the querying. This would mean that i would have to do my own aggregations in code which seems possibly a bit overkill.

---

<div class="post-metadata">

**Author:** ![aaronloesattv](https://avatars.discourse-cdn.com/v4/letter/a/3d9bf3/32.png) [@aaronloesattv](https://discuss.elastic.co/u/aaronloesattv)\
**Post date:** [March 6, 2018, 9:17pm UTC](https://discuss.elastic.co/t/query-help-must-not-keeps-matching-document/122762/4 "2018-03-06T21:17:33Z")

</div>

i've also tried using a script query but i for the life of me cannot get it to execute

```
GET .watcher-history-*/_search
{
  "query": {
    "constant_score": {
      "filter": {
        "bool": {
          "must": [
            {
              "term": {
                "state": "failed"
              }
            },
            {
              "range": {
                "trigger_event.triggered_time": {
                  "gte": "now-2h",
                  "lt": "now"
                }
              }
            },
            {
              "script": {
                "script" : {
                    "source": "!doc['exception.type'].value.equals(\"version_conflict_engine_exception\")"
                 }
              }
            }
          ]
        }
      }
    }
  },
  "sort": [
    {
      "trigger_event.triggered_time": {
        "order": "desc"
      }
    }
  ],
  "aggs": {
    "group_by_watch_id": {
      "terms": {
        "field": "watch_id"
      }
    }
  }
}

```

but that only throws an exception

```
{
  "took": 12,
  "timed_out": false,
  "_shards": {
    "total": 35,
    "successful": 34,
    "skipped": 0,
    "failed": 1,
    "failures": [
      {
        "shard": 0,
        "index": ".watcher-history-7-2018.03.06",
        "node": "Top1lr1HRBeVVr8xT95CzQ",
        "reason": {
          "type": "script_exception",
          "reason": "runtime error",
          "script_stack": [
            "org.elasticsearch.search.lookup.LeafDocLookup.get(LeafDocLookup.java:81)",
            "org.elasticsearch.search.lookup.LeafDocLookup.get(LeafDocLookup.java:39)",
            """!doc['exception.type'].value.equals("version_conflict_engine_exception")""",
            " ^---- HERE"
          ],
          "script": """!doc['exception.type'].value.equals("version_conflict_engine_exception")""",
          "lang": "painless",
          "caused_by": {
            "type": "illegal_argument_exception",
            "reason": "No field found for [exception.type] in mapping with types []"
          }
        }
      }
    ]
  },
  "hits": {
    "total": 0,
    "max_score": null,
    "hits": []
  },
  "aggregations": {
    "group_by_watch_id": {
      "doc_count_error_upper_bound": 0,
      "sum_other_doc_count": 0,
      "buckets": []
    }
  }
}
```

---

<div class="post-metadata">

**Author:** ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)\
**Post date:** [April 3, 2018, 9:18pm UTC](https://discuss.elastic.co/t/query-help-must-not-keeps-matching-document/122762/5 "2018-04-03T21:18:22Z")

</div>

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.