# Query on fields of owning document inside nested query

**URL:** https://discuss.elastic.co/t/query-on-fields-of-owning-document-inside-nested-query/58369
**Category:** Elasticsearch
**Created:** [August 18, 2016, 3:44pm UTC](https://discuss.elastic.co/t/query-on-fields-of-owning-document-inside-nested-query/58369 "2016-08-18T15:44:49Z")
**Posts on this page:** 12
**Page:** 1

<div class="post-metadata">

### Author: ![rainbowalex](https://avatars.discourse-cdn.com/v4/letter/r/b2d939/32.png) [@rainbowalex](https://discuss.elastic.co/u/rainbowalex)
#### Post date: [August 18, 2016, 3:44pm UTC](https://discuss.elastic.co/t/query-on-fields-of-owning-document-inside-nested-query/58369/1 "2016-08-18T15:44:49Z")

</div>

Is it not possible to query on fields of the owning documents inside a nested query? All I can find is 'reverse nested aggregation', I guess I need a 'reversed nested query' but that doesn't seem to exist. Can someone prove me wrong or confirm that this doesn't exist?

Thanks!

P.S. My current workaround will be copying the parent document fields I need to every nested document. Ugh :s

---

<div class="post-metadata">

### Author: ![Mark\_Harwood](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/mark_harwood/32/10538_2.png) [@Mark\_Harwood](https://discuss.elastic.co/u/Mark_Harwood)
#### Post date: [August 18, 2016, 4:16pm UTC](https://discuss.elastic.co/t/query-on-fields-of-owning-document-inside-nested-query/58369/2 "2016-08-18T16:16:03Z")

</div>

Nested queries effectively lie about which Lucene documents they have matched - they test properties of nested docs but ultimately report back the match as the root doc. They can be combined with root-level query clauses using the `bool` query.  
See slide 8 here for a visual representation of how this query logic works: [http://www.slideshare.net/MarkHarwood/proposal-for-nested-document-support-in-lucene](http://www.slideshare.net/MarkHarwood/proposal-for-nested-document-support-in-lucene)  
(It is implemented slightly differently in Lucene with the order of owner and nested docs reversed but the principle is the same)

---

<div class="post-metadata">

### Author: ![rainbowalex](https://avatars.discourse-cdn.com/v4/letter/r/b2d939/32.png) [@rainbowalex](https://discuss.elastic.co/u/rainbowalex)
#### Post date: [August 18, 2016, 4:45pm UTC](https://discuss.elastic.co/t/query-on-fields-of-owning-document-inside-nested-query/58369/3 "2016-08-18T16:45:38Z")

</div>

Thanks for the quick reply Mark. I was aware of the internals, as I understand that's why a nested context (the query) is needed to access those fields in the first place.

Unfortunately, a bool query combining nested queries is not the same as a single nested query containing a bool query. The former may have different nested documents matching different clauses, resulting in a match even if no single nested document fulfills the entire query.

---

<div class="post-metadata">

### Author: ![Mark\_Harwood](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/mark_harwood/32/10538_2.png) [@Mark\_Harwood](https://discuss.elastic.co/u/Mark_Harwood)
#### Post date: [August 18, 2016, 5:06pm UTC](https://discuss.elastic.co/t/query-on-fields-of-owning-document-inside-nested-query/58369/4 "2016-08-18T17:06:23Z")

</div>

I think I need example docs and queries to make this more concrete...

---

<div class="post-metadata">

### Author: ![rainbowalex](https://avatars.discourse-cdn.com/v4/letter/r/b2d939/32.png) [@rainbowalex](https://discuss.elastic.co/u/rainbowalex)
#### Post date: [August 18, 2016, 5:52pm UTC](https://discuss.elastic.co/t/query-on-fields-of-owning-document-inside-nested-query/58369/5 "2016-08-18T17:52:27Z")

</div>

Sure. Given these documents:

```
{_id: 1, nested: [{name: ["John", "Smith"]}]}
{_id: 2, nested: [{name: ["John", "Doe"]}, {name: ["Jim", "Smith"]}]}

```

If I wanted to match documents with a single nested document containing John AND Smith, it's clear that:

```
{and: [{nested: {name: "John"}}, {nested: {name: "Smith"}}]}

```

Will match the second doc. That's what I meant by "different nested documents matching different clauses". The solution in this case is obvious, just rewrite the query, bringing the boolean combination inside the nested query:

```
{nested: {and: [{name: "John"}, {name: "Smith"}]}}

```

But that is not always possible. Once you start combining clauses that concern the root document with clauses that concern a nested document no amount of reordering can bring the nested document clauses neatly together without changing semantics. The simplest case I can come up with:

```
(<root clause 1> AND <nested clause 1>) OR (<root clause 2> AND <nested clause 2>)

```

Where I want nested clauses 1 & 2 to apply to the _same nested document_.

The ideal solution (I think) would be a 'reverse\_nested' query. I could then wrap the entire expression in a single nested query, and the root clauses each in a reverse nested query.

---

<div class="post-metadata">

### Author: ![Mark\_Harwood](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/mark_harwood/32/10538_2.png) [@Mark\_Harwood](https://discuss.elastic.co/u/Mark_Harwood)
#### Post date: [August 18, 2016, 5:59pm UTC](https://discuss.elastic.co/t/query-on-fields-of-owning-document-inside-nested-query/58369/6 "2016-08-18T17:59:31Z")

</div>

> [@rainbowalex](#):
>
> Where I want nested clauses 1 & 2 to apply to the same nested document.

I was OK understanding the problem up to this point. The top-level query is a bool OR so nested clauses 1 and 2 are not required to match together?

---

<div class="post-metadata">

### Author: ![rainbowalex](https://avatars.discourse-cdn.com/v4/letter/r/b2d939/32.png) [@rainbowalex](https://discuss.elastic.co/u/rainbowalex)
#### Post date: [August 18, 2016, 6:27pm UTC](https://discuss.elastic.co/t/query-on-fields-of-owning-document-inside-nested-query/58369/7 "2016-08-18T18:27:17Z")

</div>

Eh.. Right 🙂

That's just incidental though, if you invert the OR's and AND's you get the problem I described:

```
(<root clause 1> OR <nested clause 1>) AND (<root clause 2> OR <nested clause 2>)
```

---

<div class="post-metadata">

### Author: ![Mark\_Harwood](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/mark_harwood/32/10538_2.png) [@Mark\_Harwood](https://discuss.elastic.co/u/Mark_Harwood)
#### Post date: [August 18, 2016, 6:40pm UTC](https://discuss.elastic.co/t/query-on-fields-of-owning-document-inside-nested-query/58369/8 "2016-08-18T18:40:33Z")

</div>

So these are the valid combos you accept?  
rc1 + rc2  
rc1 + nc2  
rc2 + nc1  
nc1 + nc2 (but only on the same nested doc)

That seems unusual but you could write this as

((rc1 OR rc2) AND (nested(nc1) OR nested(nc2) )) OR nested(nc1 OR nc2)

---

<div class="post-metadata">

### Author: ![rainbowalex](https://avatars.discourse-cdn.com/v4/letter/r/b2d939/32.png) [@rainbowalex](https://discuss.elastic.co/u/rainbowalex)
#### Post date: [August 18, 2016, 8:03pm UTC](https://discuss.elastic.co/t/query-on-fields-of-owning-document-inside-nested-query/58369/9 "2016-08-18T20:03:00Z")

</div>

> That seems unusual

Yeah it's an edgy case in a pretty complicated app, isolated and simplified, but it does happen in real life.

Anyway, I don't think your rewrite is entirely correct.

```auto
>>> (r1, n1, r2, n2) = (True, True, False, False)
>>> (r1 or n1) and (r2 or n2)
False
>>> ((r1 or r2) and (n1 or n2)) or (n1 or n2)
True

```

There probably is a correct way to rewrite it, but I have to deal with arbitrary nesting of and/or/not anyway. I will try and see if I can come up with an algorithm for rewriting the queries. Otherwise; I will have to duplicate the necessary fields of the root document in the nested document.

In either case, I think having `reverse_query` might be of value. I will write up a clearer example and make a feature request.

---

<div class="post-metadata">

### Author: ![Mark\_Harwood](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/mark_harwood/32/10538_2.png) [@Mark\_Harwood](https://discuss.elastic.co/u/Mark_Harwood)
#### Post date: [August 19, 2016, 7:40am UTC](https://discuss.elastic.co/t/query-on-fields-of-owning-document-inside-nested-query/58369/10 "2016-08-19T07:40:28Z")

</div>

> [@rainbowalex](#):
>
> Anyway, I don't think your rewrite is entirely correct.

Yeah, if my list of acceptable combos was correct I think the pseudo code for the related query is this:

((rc1 OR rc2) AND (nested(nc1) OR nested(nc2) )) OR nested(nc1 **AND** nc2)

---

<div class="post-metadata">

### Author: ![rainbowalex](https://avatars.discourse-cdn.com/v4/letter/r/b2d939/32.png) [@rainbowalex](https://discuss.elastic.co/u/rainbowalex)
#### Post date: [August 22, 2016, 10:47am UTC](https://discuss.elastic.co/t/query-on-fields-of-owning-document-inside-nested-query/58369/11 "2016-08-22T10:47:45Z")

</div>

I have created a [feature request](https://github.com/elastic/elasticsearch/issues/20101)

---

<div class="post-metadata">

### Author: ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)
#### Post date: [July 5, 2017, 10:26pm UTC](https://discuss.elastic.co/t/query-on-fields-of-owning-document-inside-nested-query/58369/12 "2017-07-05T22:26:12Z")

</div>


