# Query all fields in an embedded document?

**URL:** <https://discuss.elastic.co/t/query-all-fields-in-an-embedded-document/5784>\
**Category:** Elasticsearch\
**Created:** [November 6, 2011, 8:38pm UTC](https://discuss.elastic.co/t/query-all-fields-in-an-embedded-document/5784 "2011-11-06T20:38:26Z")\
**Posts on this page:** 11\
**Page:** 1

<div class="post-metadata">

**Author:** ![Nick\_Hoffman](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/nick_hoffman/32/1872_2.png) [@Nick\_Hoffman](https://discuss.elastic.co/u/Nick_Hoffman)\
**Post date:** [November 6, 2011, 8:38pm UTC](https://discuss.elastic.co/t/query-all-fields-in-an-embedded-document/5784/1 "2011-11-06T20:38:26Z")

</div>

Is there a way to query all of the fields in an embedded document?

For example, if you have documents that're structured like this:  
{ first\_name: "Bob", interests: [ {name: "dogs", level: 5}, {topic: "dogs",  
degree: "strong"} ] }

Can you build a query that searches all documents in the "interests" field  
whose value is "dogs"?

---

<div class="post-metadata">

**Author:** ![arien](https://avatars.discourse-cdn.com/v4/letter/a/e0b2c6/32.png) [@arien](https://discuss.elastic.co/u/arien)\
**Post date:** [November 7, 2011, 5:12am UTC](https://discuss.elastic.co/t/query-all-fields-in-an-embedded-document/5784/2 "2011-11-07T05:12:03Z")

</div>

> **[Elasticsearch Platform — Find real-time answers at scale](https://www.elastic.co)**
>
> Power insights and outcomes with the Elasticsearch Platform and AI. See into your data and find answers that matter with enterprise solutions designed to help you build, observe, and protect. Try Elasticsearch free today.

aliases on indices using filters may help you.

On Mon, Nov 7, 2011 at 2:08 AM, Nick Hoffman [nick@deadorange.com](mailto:nick@deadorange.com) wrote:

> Is there a way to query all of the fields in an embedded document?
> 
> For example, if you have documents that're structured like this:  
> { first\_name: "Bob", interests: [ {name: "dogs", level: 5}, {topic:  
> "dogs", degree: "strong"} ] }
> 
> Can you build a query that searches all documents in the "interests" field  
> whose value is "dogs"?

---

<div class="post-metadata">

**Author:** ![Nick\_Hoffman](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/nick_hoffman/32/1872_2.png) [@Nick\_Hoffman](https://discuss.elastic.co/u/Nick_Hoffman)\
**Post date:** [November 7, 2011, 5:21am UTC](https://discuss.elastic.co/t/query-all-fields-in-an-embedded-document/5784/3 "2011-11-07T05:21:24Z")

</div>

Interesting. Thanks, arien. An index alias will work if you know the name  
of every field. Unfortunately, the fields that I'm dealing with are  
arbitrary.

---

<div class="post-metadata">

**Author:** ![Clinton\_Gormley](https://avatars.discourse-cdn.com/v4/letter/c/50afbb/32.png) [@Clinton\_Gormley](https://discuss.elastic.co/u/Clinton_Gormley)\
**Post date:** [November 7, 2011, 10:08am UTC](https://discuss.elastic.co/t/query-all-fields-in-an-embedded-document/5784/4 "2011-11-07T10:08:56Z")

</div>

Hi Nick

On Sun, 2011-11-06 at 12:38 -0800, Nick Hoffman wrote:

> Is there a way to query all of the fields in an embedded document?

> For example, if you have documents that're structured like this:  
> { first\_name: "Bob", interests: [ {name: "dogs", level: 5}, {topic:  
> "dogs", degree: "strong"} ] }
> 
> Can you build a query that searches all documents in the "interests"  
> field whose value is "dogs"?

You want to look at nested fields, and nested queries/filters

> **[Elasticsearch Platform — Find real-time answers at scale](https://www.elastic.co)**
>
> Power insights and outcomes with the Elasticsearch Platform and AI. See into your data and find answers that matter with enterprise solutions designed to help you build, observe, and protect. Try Elasticsearch free today.

> **[Elasticsearch Platform — Find real-time answers at scale](https://www.elastic.co)**
>
> Power insights and outcomes with the Elasticsearch Platform and AI. See into your data and find answers that matter with enterprise solutions designed to help you build, observe, and protect. Try Elasticsearch free today.

> **[Elasticsearch Platform — Find real-time answers at scale](https://www.elastic.co)**
>
> Power insights and outcomes with the Elasticsearch Platform and AI. See into your data and find answers that matter with enterprise solutions designed to help you build, observe, and protect. Try Elasticsearch free today.

---

<div class="post-metadata">

**Author:** ![Nick\_Hoffman](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/nick_hoffman/32/1872_2.png) [@Nick\_Hoffman](https://discuss.elastic.co/u/Nick_Hoffman)\
**Post date:** [November 8, 2011, 5:12am UTC](https://discuss.elastic.co/t/query-all-fields-in-an-embedded-document/5784/5 "2011-11-08T05:12:20Z")

</div>

Thanks for the push in the right direction, Clint. I really appreciate it.  
After some reading and research, I found a basic example posted by kimchy:

> <https://gist.github.com/kimchy/1108683>

I modified that example to fit the problem that I'm trying to  
solve. Unfortunately, I couldn't figure out how to search for a value  
across all of the nested fields without referencing each nested field  
individually. Is this possible?

Here's a gist of what I've got so far. Note that the query at the end  
returns no results. I'm not sure how to work \_all into it, or if that's  
even possible.

> <https://gist.github.com/nickhoffman/0b62966c116fd6eb6212>

I tried these, but they all resulted in an invalid query:  
"must": [{ "\_all": { "comments.\_all": "this is text" } }]  
"must": [{ "\_all": { "comments._": "this is text" } }]  
"must": [{ "_": { "comments.\_all": "this is text" } }]  
"must": [{ "_": { "comments._": "this is text" } }]

Cheers,  
Nick

---

<div class="post-metadata">

**Author:** ![vineeth\_mohan](https://avatars.discourse-cdn.com/v4/letter/v/bc79bd/32.png) [@vineeth\_mohan](https://discuss.elastic.co/u/vineeth_mohan)\
**Post date:** [November 8, 2011, 5:24am UTC](https://discuss.elastic.co/t/query-all-fields-in-an-embedded-document/5784/6 "2011-11-08T05:24:35Z")

</div>

I have also discovered the same rabbit hole .  
But then seeing that there is a new document created for each name value  
pair , scares me a lot.  
As number of such name value pair that can come is not limited and can  
shoot to any big number.  
Is there a way i can use array type to create name value pairs and  
search/facet using them ?

Is there a better solution here ??

On Tue, Nov 8, 2011 at 10:42 AM, Nick Hoffman [nick@deadorange.com](mailto:nick@deadorange.com) wrote:

> Thanks for the push in the right direction, Clint. I really appreciate it.  
> After some reading and research, I found a basic example posted by kimchy:  
> [gist:1108683 · GitHub](https://gist.github.com/1108683)
> 
> I modified that example to fit the problem that I'm trying to  
> solve. Unfortunately, I couldn't figure out how to search for a value  
> across all of the nested fields without referencing each nested field  
> individually. Is this possible?
> 
> Here's a gist of what I've got so far. Note that the query at the end  
> returns no results. I'm not sure how to work \_all into it, or if that's  
> even possible.  
> [Nested Query on arbitrary fields · GitHub](https://gist.github.com/0b62966c116fd6eb6212)
> 
> I tried these, but they all resulted in an invalid query:  
> "must": [{ "\_all": { "comments.\_all": "this is text" } }]  
> "must": [{ "\_all": { "comments._": "this is text" } }]  
> "must": [{ "_": { "comments.\_all": "this is text" } }]  
> "must": [{ "_": { "comments._": "this is text" } }]
> 
> Cheers,  
> Nick

---

<div class="post-metadata">

**Author:** ![Clinton\_Gormley](https://avatars.discourse-cdn.com/v4/letter/c/50afbb/32.png) [@Clinton\_Gormley](https://discuss.elastic.co/u/Clinton_Gormley)\
**Post date:** [November 8, 2011, 11:08am UTC](https://discuss.elastic.co/t/query-all-fields-in-an-embedded-document/5784/7 "2011-11-08T11:08:00Z")

</div>

Hi Nick

> I modified that example to fit the problem that I'm trying to solve.  
> Unfortunately, I couldn't figure out how to search for a value across  
> all of the nested fields without referencing each nested field  
> individually. Is this possible?

As an example, I'm going to use this doc:

{ text: "foo",  
attrib: [  
{ color: "red", active: true },  
{ color: "blue", active: false }  
]  
}

If 'attrib' is field-type 'object', then internally, this doc would look  
something like this:

{ text: ["foo"],  
attrib.color: ["red","blue"],  
attrib.active: [true, false]  
}

If 'attrib' is field-type 'nested', then internally, this doc would be  
stored as 3 separate docs, something like this:

{ text: ["foo"] }  
{ color: ["red"], active: true},  
{ color: ["blue"], active: false},

...plus something to tie docs 2 & 3 to the main doc.

Now, consider this query:

color == 'red' and active == false

In the first (object type) example, the values for the 'attrib' are  
flattened out, so the above clause will be true.

In the second (nested type) example, each doc is considered separately,  
so the above clause will be false.

So you should use the 'nested' field-type only when you need to run  
queries/filters that depend on the each sub-object being considered  
separately.

## Further configuration:

By default, the nested properties are not visible in the parent objects.  
By which I mean: a query for 'attrib.color' will only work as a nested  
query/filter.

However, if you set 'include\_in\_parent' (ie direct parent object) or  
'include\_in\_root' (topmost object) then the nested values will be copied  
into the parent or root object, in the same way as demonstrated in the  
first example.

Only the root object has an \_all field. By default, values in the  
nested objects ARE included in the \_all field.

So: two choices

1. query the topmost \_all field
2. use a nested query with a bool or dismax query to query each  
field in your nested doc.

For instance, using your example docs:

QUERY THE \_ALL FIELD:

curl -XGET '[http://127.0.0.1:9200/test/tweet/\_search?pretty=1](http://127.0.0.1:9200/test/tweet/_search?pretty=1)' -d '  
{  
"query" : {  
"text" : {  
"\_all" : "jack stuff"  
}  
}  
}  
'

NESTED QUERY OF ALL FIELDS:

curl -XGET '[http://127.0.0.1:9200/test/tweet/\_search?pretty=1](http://127.0.0.1:9200/test/tweet/_search?pretty=1)' -d '  
{  
"query" : {  
"nested" : {  
"query" : {  
"bool" : {  
"should" : [  
{  
"text" : {  
"comments.username" : "jack stuff"  
}  
},  
{  
"text" : {  
"comments.content" : "jack stuff"  
}  
}  
]  
}  
},  
"path" : "comments"  
}  
}  
}  
'

ALSO NOTE: In your example, you are querying 'this', which is a  
stopword, and would thus never return any results.

clint

---

<div class="post-metadata">

**Author:** ![kimchy](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/kimchy/32/44952_2.png) [@kimchy](https://discuss.elastic.co/u/kimchy)\
**Post date:** [November 9, 2011, 10:58am UTC](https://discuss.elastic.co/t/query-all-fields-in-an-embedded-document/5784/8 "2011-11-09T10:58:29Z")

</div>

So, you are after searching all fields within a specific nested object  
element, you can't do that without actually specifying those fields. The  
\_all field can help, but it only has one "aspect" to it, and you can  
include / exclude fields from all "once". Nested docs will not help, since  
they do not have \_all field for each nested doc.

On Tue, Nov 8, 2011 at 7:12 AM, Nick Hoffman [nick@deadorange.com](mailto:nick@deadorange.com) wrote:

> Thanks for the push in the right direction, Clint. I really appreciate it.  
> After some reading and research, I found a basic example posted by kimchy:  
> [gist:1108683 · GitHub](https://gist.github.com/1108683)
> 
> I modified that example to fit the problem that I'm trying to  
> solve. Unfortunately, I couldn't figure out how to search for a value  
> across all of the nested fields without referencing each nested field  
> individually. Is this possible?
> 
> Here's a gist of what I've got so far. Note that the query at the end  
> returns no results. I'm not sure how to work \_all into it, or if that's  
> even possible.  
> [Nested Query on arbitrary fields · GitHub](https://gist.github.com/0b62966c116fd6eb6212)
> 
> I tried these, but they all resulted in an invalid query:  
> "must": [{ "\_all": { "comments.\_all": "this is text" } }]  
> "must": [{ "\_all": { "comments._": "this is text" } }]  
> "must": [{ "_": { "comments.\_all": "this is text" } }]  
> "must": [{ "_": { "comments._": "this is text" } }]
> 
> Cheers,  
> Nick

---

<div class="post-metadata">

**Author:** ![Nick\_Hoffman](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/nick_hoffman/32/1872_2.png) [@Nick\_Hoffman](https://discuss.elastic.co/u/Nick_Hoffman)\
**Post date:** [November 15, 2011, 10:35pm UTC](https://discuss.elastic.co/t/query-all-fields-in-an-embedded-document/5784/9 "2011-11-15T22:35:32Z")

</div>

Wow, Clint. That was one heck of an explanation. Thank you! I really  
appreciate your help.

Kimchy, your concise response was very helpful, too.

I have a much better understanding of ES and nested fields now, thanks to  
you guys. If you ever need help with MongoDB, Mongoid, Ruby, or Rails, just  
holler!

---

<div class="post-metadata">

**Author:** ![my3sons](https://avatars.discourse-cdn.com/v4/letter/m/e274bd/32.png) [@my3sons](https://discuss.elastic.co/u/my3sons)\
**Post date:** [March 22, 2012, 12:40pm UTC](https://discuss.elastic.co/t/query-all-fields-in-an-embedded-document/5784/10 "2012-03-22T12:40:37Z")

</div>

Hi Guys,

So based on Nick's example, if he also wanted to query on "name" field of the root object (i.e. where name="jane"), while still including the nested query on "comments", is that possible? If so, how should that query be constructed. I am facing similar problem and have yet to find a solution.

Thanks!

---

<div class="post-metadata">

**Author:** ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)\
**Post date:** [July 6, 2017, 3:35am UTC](https://discuss.elastic.co/t/query-all-fields-in-an-embedded-document/5784/11 "2017-07-06T03:35:11Z")

</div>