Best way to query nested objects in indexes where they don't exist

Elastic Stack Elasticsearch
1

Right now it seems like I need to create a new nested property and add that to every index I have. Otherwise the "[nested] failed to find nested object under path [MY_OBJECT_PATH]"

This topic which was closed:

and this issue applies:

I want to make an OR query by the nested object property or a different property, and have the query not fail if the nested object doesn't exist yet.

How do I do that?

2

The key to what you want to do is the ignore_unmapped option of the nested query. It tells Elasticsearch to ignore indexes where the nested path has not been mapped, rather than return an error for those indexes. You can make use of that to separately query indexes with and without a nested type.

Let's say you have two indexes. "my_index1" in which a nested type my_nested_object has been mapped, and "my_index2" where it has not:

PUT my_index1
{
  "mappings": {
    "_doc": {
      "properties": {
        "my_nested_object": {
          "type": "nested",
          "properties": {
            "foo": {
              "type": "keyword"
            }
          }
        },
        "bar": {
          "type": "keyword"
        }
      }
    }
  }
}

PUT my_index2
{
  "mappings": {
    "_doc": {
      "properties": {
        "bar": {
          "type": "keyword"
        }
      }
    }
  }
}

Now, let's index a document in each of these two indexes:

PUT my_index1/_doc/1
{
  "my_nested_object": {
    "foo": "123"
  },
  "bar": "456"
}

PUT my_index2/_doc/2
{
  "bar": "456"
}

We can now query across these two indexes using a nested query with ignore_unmapped set to true to silently ignore the index where I do not have a nested object my_nested_object:

GET my_index1,my_index2/_search
{
  "query": {
    "nested": {
      "path": "my_nested_object",
      "ignore_unmapped": true,
      "query": {
        "match": {
          "my_nested_object.foo": "123"
        }
      }
    }
  }
}

I can also do the reverse: query only those indexes that do not have a nested object my_nested_object. I can use the exists query for that, which I wrap in a bool query's must_not:

GET my_index1,my_index2/_search
{
  "query": {
    "bool": {
      "must_not": [
        {
          "nested": {
            "path": "my_nested_object",
            "ignore_unmapped": true,
            "query": {
              "exists": {
                "field": "my_nested_object"
              }
            }
          }
        }
      ],
      "must": [
        {
          "match": {
            "bar": "456"
          }
        }
      ]
    }
  }
}

Both queries can be combined to get your desired OR-like behavior by wrapping them in a bool query's should clause:

GET my_index1,my_index2/_search
{
  "query": {
    "bool": {
      "should": [
        {
          "nested": {
            "path": "my_nested_object",
            "ignore_unmapped": true,
            "query": {
              "match": {
                "my_nested_object.foo": "123"
              }
            }
          }
        },
        {
          "bool": {
            "must_not": [
              {
                "nested": {
                  "path": "my_nested_object",
                  "ignore_unmapped": true,
                  "query": {
                    "exists": {
                      "field": "my_nested_object"
                    }
                  }
                }
              }
            ],
            "must": [
              {
                "match": {
                  "bar": "456"
                }
              }
            ]
          }
        }
      ]
    }
  }
}

It's not pretty, but it works :smiley:

3

I'm still getting the same problem. I'm on Elastic Search version 6.0. Is that a new feature?

4

Maybe you can share your mapping and the query that you are trying to execute?

5

Thanks for helping, I figured out what is going on. Basically I tried to run the nested query on a mapping that already exists and is NOT nested. I would have expected it to not give that error, but it still does. I was hoping to convert the property from the current mapping to nested, but it seems I will have to use two different fields.

For instance if you had

PUT my_index2/_doc/2
{
  "foo": "123",
  "bar": "456"
}

Then I would expect the subsequent nested search to fail.

6

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.