# Nested vs. multi-valued for facet performance

**URL:** https://discuss.elastic.co/t/nested-vs-multi-valued-for-facet-performance/8043
**Category:** Elasticsearch
**Created:** [June 11, 2012, 3:14pm UTC](https://discuss.elastic.co/t/nested-vs-multi-valued-for-facet-performance/8043 "2012-06-11T15:14:46Z")
**Posts on this page:** 4
**Page:** 1

<div class="post-metadata">

### Author: ![Andrew\_Clegg](https://avatars.discourse-cdn.com/v4/letter/a/a88e4f/32.png) [@Andrew\_Clegg](https://discuss.elastic.co/u/Andrew_Clegg)
#### Post date: [June 11, 2012, 3:14pm UTC](https://discuss.elastic.co/t/nested-vs-multi-valued-for-facet-performance/8043/1 "2012-06-11T15:14:46Z")

</div>

Hi,

I've heard in one or two places, like this thread (2nd post):

[https://groups.google.com/group/elasticsearch/browse\_thread/thread/78f2600ad069cabb/19a688a270fa80cf?lnk=raot](https://groups.google.com/group/elasticsearch/browse_thread/thread/78f2600ad069cabb/19a688a270fa80cf?lnk=raot)

that using nested documents instead of multi-valued fields can reduce  
memory requirements for faceting by quite a lot.

But I wasn't sure if I understood it what it meant.

Is it just replacing documents like this:

{  
"user" : "bart",  
"friends" : ["homer", "maggie", "marge"]  
}

with something like this?

{  
"user" : "bart",  
"friends" : [  
{ "user" : "homer" },  
{ "user" : "maggie" },  
{ "user" : "marge" }  
]  
}

If it is that simple, why does anyone use list-valued fields at all when  
this is more efficient? Is there any other downside (apart from bigger JSON  
messages)?

(If it makes a difference, our application is write-heavy and largely  
facet-based)

Can you add a field with nested documents at indexing time, or do you have  
to map it in advance manually?

Many thanks!

Andrew.

---

<div class="post-metadata">

### Author: ![Andrew\_Clegg](https://avatars.discourse-cdn.com/v4/letter/a/a88e4f/32.png) [@Andrew\_Clegg](https://discuss.elastic.co/u/Andrew_Clegg)
#### Post date: [June 13, 2012, 4:07pm UTC](https://discuss.elastic.co/t/nested-vs-multi-valued-for-facet-performance/8043/2 "2012-06-13T16:07:16Z")

</div>

Appending this discussion from IRC for the benefit of future searchers --  
thanks Clinton.

4:46 PM \<clintongormley[https://irccloud.com/#!/irc.freenode.net:6667/clintongormley](https://irccloud.com/#!/irc.freenode.net:6667/clintongormley)

> so...  
> 4:47 PM \<clintongormley[https://irccloud.com/#!/irc.freenode.net:6667/clintongormley](https://irccloud.com/#!/irc.freenode.net:6667/clintongormley)  
> for multi-value fields, eg { tag: ['foo','bar']}  
> 4:47 PM \<clintongormley[https://irccloud.com/#!/irc.freenode.net:6667/clintongormley](https://irccloud.com/#!/irc.freenode.net:6667/clintongormley)  
> is has to create an array based on the max number of terms in that field  
> 4:48 PM \<clintongormley[https://irccloud.com/#!/irc.freenode.net:6667/clintongormley](https://irccloud.com/#!/irc.freenode.net:6667/clintongormley)  
> so if it needs to create an array with 10 values (where max values per  
> field = 1), then for one doc that has 5 values, it creates 10 x 5  
> 4:48 PM \<andrewclegg[https://irccloud.com/#!/irc.freenode.net:6667/andrewclegg](https://irccloud.com/#!/irc.freenode.net:6667/andrewclegg)  
> ouch  
> 4:48 PM \<clintongormley[https://irccloud.com/#!/irc.freenode.net:6667/clintongormley](https://irccloud.com/#!/irc.freenode.net:6667/clintongormley)  
> yes  
> 4:49 PM \<clintongormley[https://irccloud.com/#!/irc.freenode.net:6667/clintongormley](https://irccloud.com/#!/irc.freenode.net:6667/clintongormley)  
> so the 1% of docs with many values can greatly increase memory usage  
> 4:49 PM \<andrewclegg[https://irccloud.com/#!/irc.freenode.net:6667/andrewclegg](https://irccloud.com/#!/irc.freenode.net:6667/andrewclegg)  
> so am I thinking along right lines here:  
> [Redirecting to Google Groups](https://groups.google.com/forum/#!topic/elasticsearch/lIcFPQ2HoQc)  
> 4:49 PM \<andrewclegg[https://irccloud.com/#!/irc.freenode.net:6667/andrewclegg](https://irccloud.com/#!/irc.freenode.net:6667/andrewclegg)  
> apologies for bumping via irc 😉  
> 4:49 PM \<clintongormley[https://irccloud.com/#!/irc.freenode.net:6667/clintongormley](https://irccloud.com/#!/irc.freenode.net:6667/clintongormley)  
> not quite  
> 4:49 PM \<clintongormley[https://irccloud.com/#!/irc.freenode.net:6667/clintongormley](https://irccloud.com/#!/irc.freenode.net:6667/clintongormley)  
> those would be type: "object"  
> 4:49 PM \<clintongormley[https://irccloud.com/#!/irc.freenode.net:6667/clintongormley](https://irccloud.com/#!/irc.freenode.net:6667/clintongormley)  
> you have to make them type: "nested"  
> 4:50 PM \<clintongormley[https://irccloud.com/#!/irc.freenode.net:6667/clintongormley](https://irccloud.com/#!/irc.freenode.net:6667/clintongormley)  
> which actually indexes them internally as SEPARATE documents  
> 4:50 PM \<clintongormley[https://irccloud.com/#!/irc.freenode.net:6667/clintongormley](https://irccloud.com/#!/irc.freenode.net:6667/clintongormley)  
> but because each of these docs has only one value per field, the memory  
> usage in this case is much lower  
> 4:50 PM \<clintongormley[https://irccloud.com/#!/irc.freenode.net:6667/clintongormley](https://irccloud.com/#!/irc.freenode.net:6667/clintongormley)  
> i hadn't thought about that consequence before  
> 4:50 PM \<andrewclegg[https://irccloud.com/#!/irc.freenode.net:6667/andrewclegg](https://irccloud.com/#!/irc.freenode.net:6667/andrewclegg)  
> ah ok. do you know if it's possible to set that up via the default  
> mappings somehow? or do you have to map explicitly for every field where  
> you're doing that?  
> 4:51 PM \<clintongormley[https://irccloud.com/#!/irc.freenode.net:6667/clintongormley](https://irccloud.com/#!/irc.freenode.net:6667/clintongormley)  
> nested docs have their own overhead (eg performance - they're separate  
> docs after all)  
> 4:51 PM \<clintongormley[https://irccloud.com/#!/irc.freenode.net:6667/clintongormley](https://irccloud.com/#!/irc.freenode.net:6667/clintongormley)  
> so i'd advise doing it only where you need to  
> 4:52 PM \<clintongormley[https://irccloud.com/#!/irc.freenode.net:6667/clintongormley](https://irccloud.com/#!/irc.freenode.net:6667/clintongormley)  
> also, the way you query your docs changes as well  
> 4:52 PM \<clintongormley[https://irccloud.com/#!/irc.freenode.net:6667/clintongormley](https://irccloud.com/#!/irc.freenode.net:6667/clintongormley)  
> there are special nested queries/filters/facets  
> 4:52 PM \<andrewclegg[https://irccloud.com/#!/irc.freenode.net:6667/andrewclegg](https://irccloud.com/#!/irc.freenode.net:6667/andrewclegg)  
> ok, thanks  
> 4:52 PM \<clintongormley[https://irccloud.com/#!/irc.freenode.net:6667/clintongormley](https://irccloud.com/#!/irc.freenode.net:6667/clintongormley)  
> np

On Monday, 11 June 2012 16:14:46 UTC+1, Andrew Clegg wrote:

> Hi,
> 
> I've heard in one or two places, like this thread (2nd post):
> 
> [https://groups.google.com/group/elasticsearch/browse\_thread/thread/78f2600ad069cabb/19a688a270fa80cf?lnk=raot](https://groups.google.com/group/elasticsearch/browse_thread/thread/78f2600ad069cabb/19a688a270fa80cf?lnk=raot)
> 
> that using nested documents instead of multi-valued fields can reduce  
> memory requirements for faceting by quite a lot.
> 
> But I wasn't sure if I understood it what it meant.
> 
> Is it just replacing documents like this:
> 
> {  
> "user" : "bart",  
> "friends" : ["homer", "maggie", "marge"]  
> }
> 
> with something like this?
> 
> {  
> "user" : "bart",  
> "friends" : [  
> { "user" : "homer" },  
> { "user" : "maggie" },  
> { "user" : "marge" }  
> ]  
> }
> 
> If it is that simple, why does anyone use list-valued fields at all when  
> this is more efficient? Is there any other downside (apart from bigger JSON  
> messages)?
> 
> (If it makes a difference, our application is write-heavy and largely  
> facet-based)
> 
> Can you add a field with nested documents at indexing time, or do you have  
> to map it in advance manually?
> 
> Many thanks!
> 
> Andrew.

---

<div class="post-metadata">

### Author: ![Jack\_Chen](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/jack_chen/32/2464_2.png) [@Jack\_Chen](https://discuss.elastic.co/u/Jack_Chen)
#### Post date: [October 11, 2012, 3:03am UTC](https://discuss.elastic.co/t/nested-vs-multi-valued-for-facet-performance/8043/3 "2012-10-11T03:03:54Z")

</div>

I found this really useful in diagnosing why our facet performance was  
horrible for one of our indexes. Thanks for posting this!

Hopefully there are plans to make this better without having to resort to  
nested documents. I'm experimenting with indexing a string with tags  
delimited by spaces and using the whitespace analyzer to see if this helps.

Appending this discussion from IRC for the benefit of future searchers --

> thanks Clinton.
> 
> 4:46 PM \<clintongormley[https://irccloud.com/#!/irc.freenode.net:6667/clintongormley](https://irccloud.com/#!/irc.freenode.net:6667/clintongormley)
> 
> > so...  
> > 4:47 PM \<clintongormley[https://irccloud.com/#!/irc.freenode.net:6667/clintongormley](https://irccloud.com/#!/irc.freenode.net:6667/clintongormley)  
> > for multi-value fields, eg { tag: ['foo','bar']}  
> > 4:47 PM \<clintongormley[https://irccloud.com/#!/irc.freenode.net:6667/clintongormley](https://irccloud.com/#!/irc.freenode.net:6667/clintongormley)  
> > is has to create an array based on the max number of terms in that field  
> > 4:48 PM \<clintongormley[https://irccloud.com/#!/irc.freenode.net:6667/clintongormley](https://irccloud.com/#!/irc.freenode.net:6667/clintongormley)  
> > so if it needs to create an array with 10 values (where max values per  
> > field = 1), then for one doc that has 5 values, it creates 10 x 5  
> > 4:48 PM \<andrewclegg[https://irccloud.com/#!/irc.freenode.net:6667/andrewclegg](https://irccloud.com/#!/irc.freenode.net:6667/andrewclegg)  
> > ouch  
> > 4:48 PM \<clintongormley[https://irccloud.com/#!/irc.freenode.net:6667/clintongormley](https://irccloud.com/#!/irc.freenode.net:6667/clintongormley)  
> > yes  
> > 4:49 PM \<clintongormley[https://irccloud.com/#!/irc.freenode.net:6667/clintongormley](https://irccloud.com/#!/irc.freenode.net:6667/clintongormley)  
> > so the 1% of docs with many values can greatly increase memory usage  
> > 4:49 PM \<andrewclegg[https://irccloud.com/#!/irc.freenode.net:6667/andrewclegg](https://irccloud.com/#!/irc.freenode.net:6667/andrewclegg)  
> > so am I thinking along right lines here:  
> > [Redirecting to Google Groups](https://groups.google.com/forum/#!topic/elasticsearch/lIcFPQ2HoQc)  
> > 4:49 PM \<andrewclegg[https://irccloud.com/#!/irc.freenode.net:6667/andrewclegg](https://irccloud.com/#!/irc.freenode.net:6667/andrewclegg)  
> > apologies for bumping via irc 😉  
> > 4:49 PM \<clintongormley[https://irccloud.com/#!/irc.freenode.net:6667/clintongormley](https://irccloud.com/#!/irc.freenode.net:6667/clintongormley)  
> > not quite  
> > 4:49 PM \<clintongormley[https://irccloud.com/#!/irc.freenode.net:6667/clintongormley](https://irccloud.com/#!/irc.freenode.net:6667/clintongormley)  
> > those would be type: "object"  
> > 4:49 PM \<clintongormley[https://irccloud.com/#!/irc.freenode.net:6667/clintongormley](https://irccloud.com/#!/irc.freenode.net:6667/clintongormley)  
> > you have to make them type: "nested"  
> > 4:50 PM \<clintongormley[https://irccloud.com/#!/irc.freenode.net:6667/clintongormley](https://irccloud.com/#!/irc.freenode.net:6667/clintongormley)  
> > which actually indexes them internally as SEPARATE documents  
> > 4:50 PM \<clintongormley[https://irccloud.com/#!/irc.freenode.net:6667/clintongormley](https://irccloud.com/#!/irc.freenode.net:6667/clintongormley)  
> > but because each of these docs has only one value per field, the memory  
> > usage in this case is much lower  
> > 4:50 PM \<clintongormley[https://irccloud.com/#!/irc.freenode.net:6667/clintongormley](https://irccloud.com/#!/irc.freenode.net:6667/clintongormley)  
> > i hadn't thought about that consequence before  
> > 4:50 PM \<andrewclegg[https://irccloud.com/#!/irc.freenode.net:6667/andrewclegg](https://irccloud.com/#!/irc.freenode.net:6667/andrewclegg)  
> > ah ok. do you know if it's possible to set that up via the default  
> > mappings somehow? or do you have to map explicitly for every field where  
> > you're doing that?  
> > 4:51 PM \<clintongormley[https://irccloud.com/#!/irc.freenode.net:6667/clintongormley](https://irccloud.com/#!/irc.freenode.net:6667/clintongormley)  
> > nested docs have their own overhead (eg performance - they're separate  
> > docs after all)  
> > 4:51 PM \<clintongormley[https://irccloud.com/#!/irc.freenode.net:6667/clintongormley](https://irccloud.com/#!/irc.freenode.net:6667/clintongormley)  
> > so i'd advise doing it only where you need to  
> > 4:52 PM \<clintongormley[https://irccloud.com/#!/irc.freenode.net:6667/clintongormley](https://irccloud.com/#!/irc.freenode.net:6667/clintongormley)  
> > also, the way you query your docs changes as well  
> > 4:52 PM \<clintongormley[https://irccloud.com/#!/irc.freenode.net:6667/clintongormley](https://irccloud.com/#!/irc.freenode.net:6667/clintongormley)  
> > there are special nested queries/filters/facets  
> > 4:52 PM \<andrewclegg[https://irccloud.com/#!/irc.freenode.net:6667/andrewclegg](https://irccloud.com/#!/irc.freenode.net:6667/andrewclegg)  
> > ok, thanks  
> > 4:52 PM \<clintongormley[https://irccloud.com/#!/irc.freenode.net:6667/clintongormley](https://irccloud.com/#!/irc.freenode.net:6667/clintongormley)  
> > np
> 
> On Monday, 11 June 2012 16:14:46 UTC+1, Andrew Clegg wrote:
> 
> > Hi,
> > 
> > I've heard in one or two places, like this thread (2nd post):
> > 
> > [https://groups.google.com/group/elasticsearch/browse\_thread/thread/78f2600ad069cabb/19a688a270fa80cf?lnk=raot](https://groups.google.com/group/elasticsearch/browse_thread/thread/78f2600ad069cabb/19a688a270fa80cf?lnk=raot)
> > 
> > that using nested documents instead of multi-valued fields can reduce  
> > memory requirements for faceting by quite a lot.
> > 
> > But I wasn't sure if I understood it what it meant.
> > 
> > Is it just replacing documents like this:
> > 
> > {  
> > "user" : "bart",  
> > "friends" : ["homer", "maggie", "marge"]  
> > }
> > 
> > with something like this?
> > 
> > {  
> > "user" : "bart",  
> > "friends" : [  
> > { "user" : "homer" },  
> > { "user" : "maggie" },  
> > { "user" : "marge" }  
> > ]  
> > }
> > 
> > If it is that simple, why does anyone use list-valued fields at all when  
> > this is more efficient? Is there any other downside (apart from bigger JSON  
> > messages)?
> > 
> > (If it makes a difference, our application is write-heavy and largely  
> > facet-based)
> > 
> > Can you add a field with nested documents at indexing time, or do you  
> > have to map it in advance manually?
> > 
> > Many thanks!
> > 
> > Andrew.

--

---

<div class="post-metadata">

### Author: ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)
#### Post date: [July 6, 2017, 3:09am UTC](https://discuss.elastic.co/t/nested-vs-multi-valued-for-facet-performance/8043/4 "2017-07-06T03:09:22Z")

</div>