# Nested Field exists check not working as expected

**URL:** https://discuss.elastic.co/t/nested-field-exists-check-not-working-as-expected/356859
**Category:** Elasticsearch
**Tags:** eql-elastic-query-language
**Created:** [April 5, 2024, 11:28am UTC](https://discuss.elastic.co/t/nested-field-exists-check-not-working-as-expected/356859 "2024-04-05T11:28:43Z")
**Posts on this page:** 9
**Page:** 1

<div class="post-metadata">

### Author: ![Sankar\_S](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/sankar_s/32/112213_2.png) [@Sankar\_S](https://discuss.elastic.co/u/Sankar_S)
#### Post date: [April 5, 2024, 11:28am UTC](https://discuss.elastic.co/t/nested-field-exists-check-not-working-as-expected/356859/1 "2024-04-05T11:28:43Z")

</div>

```auto
GET index1/_search
{
  "query": {
    "bool": {
      "must_not": [
        {
          "nested": {
            "path": "publish_details",
            "query": {
              "bool": {
                "must_not": [
                  {
                    "exists": {
                      "field": "publish_details.environment"
                    }
                  }
                ]
              }
            }
          }
        }
      ]
    }
  }
}

```

Query2

```auto
GET index1/_search
{
  "query": {
    "bool": {
      "must_not": [
        {
          "nested": {
            "path": "publish_details",
            "query": {
              "bool": {
                "must": [
                  {
                    "exists": {
                      "field": "publish_details.environment"
                    }
                  }
                ]
              }
            }
          }
        }
      ]
    }
  }
}

```

Both gives the same result. I was expecting must exists and must\_not exists to give mutually exclusive results but it is not.  
Please let me know how both gives the same results? and clarify do i need to specify the field path in exists query? ie., publish\_details.environment vs environement usage

---

<div class="post-metadata">

### Author: ![Christian\_Dahlqvist](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/christian_dahlqvist/32/4617_2.png) [@Christian\_Dahlqvist](https://discuss.elastic.co/u/Christian_Dahlqvist)
#### Post date: [April 6, 2024, 10:45am UTC](https://discuss.elastic.co/t/nested-field-exists-check-not-working-as-expected/356859/2 "2024-04-06T10:45:08Z")

</div>

Each nested sub-document is stored as a separate document behind the scenes and evaluated individually. It is therefore possible for a document to have some sub-documents that match and others that do not match the clause. All sub-documents do not need to match, so a document could match both examples.

Take this simple sample document as an example:

```auto
{
  "publish_details": [
    {
      "clause": "1",
      "environment": "A"
    },
    {
      "clause": "2"
    }
  ]
}

```

The first nested document matches the exists clause while the second does not. When you are using a nested sub clause I believe you are looking for any match so I suspect this document should match both your queries. If you however removed one of the clauses it should only match one.

---

<div class="post-metadata">

### Author: ![Sankar\_S](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/sankar_s/32/112213_2.png) [@Sankar\_S](https://discuss.elastic.co/u/Sankar_S)
#### Post date: [April 8, 2024, 9:22am UTC](https://discuss.elastic.co/t/nested-field-exists-check-not-working-as-expected/356859/3 "2024-04-08T09:22:31Z")

</div>

In My case both the conditions returns same set of documents, I used must in one query must\_not in another query.

---

<div class="post-metadata">

### Author: ![Christian\_Dahlqvist](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/christian_dahlqvist/32/4617_2.png) [@Christian\_Dahlqvist](https://discuss.elastic.co/u/Christian_Dahlqvist)
#### Post date: [April 8, 2024, 9:39am UTC](https://discuss.elastic.co/t/nested-field-exists-check-not-working-as-expected/356859/4 "2024-04-08T09:39:51Z")

</div>

Yes, I know. The nested clause is evaluated against all nested documents. In the example I provided above one will match and one will not match for both of your queries. As only one need to match you get the same result for both queries.

If you index the following documents into a test index you should be able to see what I mean:

```auto
{
  "publish_details": [
    {
      "clause": "1",
      "environment": "A"
    },
    {
      "clause": "2"
    }
  ]
}

{
  "publish_details": [
    {
      "clause": "1",
      "environment": "A"
    }
  ]
}

{
  "publish_details": [
    {
      "clause": "2"
    }
  ]
}

```

When you run your queries I would expect you to get 2 results for each query.

---

<div class="post-metadata">

### Author: ![smalleyd](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/smalleyd/32/135694_2.png) [@smalleyd](https://discuss.elastic.co/u/smalleyd)
#### Post date: [July 1, 2024, 7:22pm UTC](https://discuss.elastic.co/t/nested-field-exists-check-not-working-as-expected/356859/5 "2024-07-01T19:22:53Z")

</div>

I have the same problem. In my sample there are documents that have a completely empty/null nested field and the documents still are not found with must\_not:[nested ... exists]. The reverse works, though.

---

<div class="post-metadata">

### Author: ![Sankar\_S](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/sankar_s/32/112213_2.png) [@Sankar\_S](https://discuss.elastic.co/u/Sankar_S)
#### Post date: [July 22, 2024, 12:04pm UTC](https://discuss.elastic.co/t/nested-field-exists-check-not-working-as-expected/356859/6 "2024-07-22T12:04:25Z")

</div>

Hello David,  
did you find any solution?

---

<div class="post-metadata">

### Author: ![smalleyd](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/smalleyd/32/135694_2.png) [@smalleyd](https://discuss.elastic.co/u/smalleyd)
#### Post date: [July 22, 2024, 1:02pm UTC](https://discuss.elastic.co/t/nested-field-exists-check-not-working-as-expected/356859/7 "2024-07-22T13:02:43Z")

</div>

Sort of. I added a non-nested field on the parent document that is a count of the number of nested records. It's a hack but seems to be the only workaround for this issue.

---

<div class="post-metadata">

### Author: ![Sankar\_S](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/sankar_s/32/112213_2.png) [@Sankar\_S](https://discuss.elastic.co/u/Sankar_S)
#### Post date: [July 22, 2024, 2:22pm UTC](https://discuss.elastic.co/t/nested-field-exists-check-not-working-as-expected/356859/8 "2024-07-22T14:22:11Z")

</div>

using aggregation API? Performance impact would be there right?

---

<div class="post-metadata">

### Author: ![smalleyd](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/smalleyd/32/135694_2.png) [@smalleyd](https://discuss.elastic.co/u/smalleyd)
#### Post date: [July 22, 2024, 3:28pm UTC](https://discuss.elastic.co/t/nested-field-exists-check-not-working-as-expected/356859/9 "2024-07-22T15:28:37Z")

</div>

No, it's a denormalized field that gets stored whenever a document is created or updated.
