# Nested query, missing inner hits

**URL:** <https://discuss.elastic.co/t/nested-query-missing-inner-hits/41038>\
**Category:** Elasticsearch\
**Created:** [February 5, 2016, 9:43am UTC](https://discuss.elastic.co/t/nested-query-missing-inner-hits/41038 "2016-02-05T09:43:46Z")\
**Posts on this page:** 5\
**Page:** 1

<div class="post-metadata">

**Author:** ![Benjamin\_Gathmann](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/benjamin_gathmann/32/7561_2.png) [@Benjamin\_Gathmann](https://discuss.elastic.co/u/Benjamin_Gathmann)\
**Post date:** [February 5, 2016, 9:43am UTC](https://discuss.elastic.co/t/nested-query-missing-inner-hits/41038/1 "2016-02-05T09:43:46Z")

</div>

I have observed a strange behavior for nested queries which I do not understand. Here is an example query:

```
{
    "query" : {
        "nested" : {
            "path" : "first.nested.path",
            "query" : {
                "bool" : {
                    "filter" : [{
                            "term" : {
                                "first.nested.path.general.something" : "ABC"
                            }
                        }, {
                            "nested" : {
                                "path" : "first.nested.path.second.path",
                                "query" : {
                                    "bool" : {
                                        "filter" : [{
                                                "term" : {
                                                    "first.nested.path.second.path.value" : "Hello"
                                                }
                                            }, {
                                                "term" : {
                                                    "first.nested.path.second.path.value2" : "World"
                                                }
                                            }
                                        ]
                                    }
                                }
                            }
                        }
                    ]
                }
            }
        }
    },
    "inner_hits" : {
        "kids" : {
            "path" : {
                "first.nested.path" : {
                    "query" : {
                        "term" : {
                            "first.nested.path.general.something" : "ABC"
                        }
                    },
                    "_source" : false,
                    "fielddata_fields" : ["first.nested.path.general.something"],
                    "inner_hits" : {
                        "kids" : {
                            "path" : {
                                "first.nested.path.second.path" : {
                                    "query" : {
                                        "bool" : {
                                            "filter" : [{
                                                    "term" : {
                                                        "first.nested.path.second.path.value" : "Hello"
                                                    }
                                                }, {
                                                    "term" : {
                                                        "first.nested.path.second.path.value2" : "World"
                                                    }
                                                }
                                            ]
                                        }
                                    },
                                    "_source" : false,
                                    "fielddata_fields" : ["first.nested.path.second.path.value", "first.nested.path.second.path.value2"]
                                }
                            }
                        }
                    }
                }
            }
        }
    }
}

```

This query returns empty inner\_hits for some results. But how is this possible?  
I have checked the original data and found that the filters apply (so it is correct that the result is returned), but for some reason the inner\_hits are not shown.  
What are possible reasons for this?

I am also not quite sure about how to include query-\>bool-\>filter-\>term in the inner\_hits definition. Should it be:

```
 "query" : {
                            "term" : {

```

(like above)  
OR:

```
"query" : {
                        "filtered": {
                            "filter": {
                                "term" : {

```

?

In general, this part of the query DSL is pretty awkward (see also:

> <https://github.com/elastic/elasticsearch/pull/16143#issuecomment-177973852>

---

<div class="post-metadata">

**Author:** ![Benjamin\_Gathmann](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/benjamin_gathmann/32/7561_2.png) [@Benjamin\_Gathmann](https://discuss.elastic.co/u/Benjamin_Gathmann)\
**Post date:** [February 10, 2016, 11:58am UTC](https://discuss.elastic.co/t/nested-query-missing-inner-hits/41038/2 "2016-02-10T11:58:19Z")

</div>

To better illustrate this behavior, I have created this Gist:

> <https://gist.github.com/abulhol/d951aef61c3ff4225306>

  
and the kind of data I receive for SOME of the hits:  

> <https://gist.github.com/abulhol/a5ae6e01e2927c320f61>

Does anybody have an idea what is going on?  
Is it a bug? Or is something wrong with my syntax?

---

<div class="post-metadata">

**Author:** ![Benjamin\_Gathmann](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/benjamin_gathmann/32/7561_2.png) [@Benjamin\_Gathmann](https://discuss.elastic.co/u/Benjamin_Gathmann)\
**Post date:** [February 10, 2016, 1:28pm UTC](https://discuss.elastic.co/t/nested-query-missing-inner-hits/41038/3 "2016-02-10T13:28:12Z")

</div>

Here is the probable answer (but not solution yet) to my question:  
It seems like ES stumbles over "null" values for objects when it retrieves inner\_hits. I have added an example to my Gist here (you can see that the object "f" is null in the second "first" entry :

> <https://gist.github.com/abulhol/d951aef61c3ff4225306#file-example-json>

I am not sure what to do about this problem. It looks like a bug to me - I guess ES should check for null values when iterating over the documents. What is the way forward?

---

<div class="post-metadata">

**Author:** ![Benjamin\_Gathmann](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/benjamin_gathmann/32/7561_2.png) [@Benjamin\_Gathmann](https://discuss.elastic.co/u/Benjamin_Gathmann)\
**Post date:** [February 12, 2016, 4:51pm UTC](https://discuss.elastic.co/t/nested-query-missing-inner-hits/41038/4 "2016-02-12T16:51:37Z")

</div>

It turned out the problem lies somewhere else.  
It has to do with the pretty insane syntax for top level inner hits. Here is a related issue on Github:

> <https://github.com/elastic/elasticsearch/issues/13064>

The point is that if I have multiple nesting, I have to repeat a query several times, i.e. not only in the deepest path, but also in all intermediate paths in between.  
Otherwise, the query will return all inner\_hits for the intermediate level, even those that do not have a matching nested object inside that matches the actual query.

Here is what my example query needs to look like:

> <https://gist.github.com/abulhol/0869e5c418b77f97e752>

As @RanadeepPolavarapu writes:  
"Downside that It is more verbose, because the query needs to be repeated on each nested level."

I would be very happy if something could be done to make the syntax a little less "verbose".

---

<div class="post-metadata">

**Author:** ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)\
**Post date:** [July 5, 2017, 11:16pm UTC](https://discuss.elastic.co/t/nested-query-missing-inner-hits/41038/5 "2017-07-05T23:16:47Z")

</div>


