# Memory issues with facets
**URL:** https://discuss.elastic.co/t/memory-issues-with-facets/7971
**Category:** Elasticsearch
**Created:** [June 4, 2012, 4:55pm UTC](https://discuss.elastic.co/t/memory-issues-with-facets/7971 "2012-06-04T16:55:58Z")
**Posts on this page:** 5
**Page:** 1
### Author:  [@slushi](https://discuss.elastic.co/u/slushi)
#### Post date: [June 4, 2012, 4:55pm UTC](https://discuss.elastic.co/t/memory-issues-with-facets/7971/1 "2012-06-04T16:55:58Z")
I am trying to use facets to implement autocomplete behavior (try to
autocomplete the search term as the user types) but I have been having
memory issues. I have an single node index with 2 shards, each with about
450K documents and 150K terms in the field being used for autocomplete.
After starting the server, I tried doing my facet search which is something
like this:
{ "size" : 0,
"query" : {
"prefix" : { "f" : "foo" }
},
"facets" : {
"autocomplete" : {
"terms" : {
"field" : "f",
"regex" : "foo.\*",
"regex\_flags" : "DOTALL",
"size" : 5
}
}
}
}
the bigdesk cache graph showed the field cache jumping up to 800,000,000.
This seems high, these are basically tags so they should be short strings.
I saw this thread[https://groups.google.com/forum/?fromgroups#!topic/elasticsearch/Mdh8hN04c2c](https://groups.google.com/forum/?fromgroups#!topic/elasticsearch/Mdh8hN04c2c) which
is probably relevant as the field is indeed multivalued. I will try to
switch to nested documents and see if there is any improvement. A few
additional questions:
1. The docs say that by default field cache entries never expire. I
assume the cache is still kept coherent with the index, so if docs are
updated/deleted/expired the relevant cache entries are evicted?
2. Does the number of shards/node increase field cache memory usage?
I.e., would memory usage go down if i used one shard rather than 2?
3. I have a specific field just to support autocomplete, would it be
better to simply split out the field into its own index rather than use
nested docs?
4. Does this approach (facets for autocomplete) make sense given the
size of the field? I was using solr before which had a way (terms
component [http://wiki.apache.org/solr/TermsComponent](http://wiki.apache.org/solr/TermsComponent)) to directly
access the index terms, this seems much more efficient than what I am doing
here. Would it make more sense to build a separate index for autocomplete
by periodically mining the top terms from the main index and then doing
"normal" prefix queries to get a suggestion?
Thanks!
---
### Author:  [@kimchy](https://discuss.elastic.co/u/kimchy)
#### Post date: [June 8, 2012, 10:03pm UTC](https://discuss.elastic.co/t/memory-issues-with-facets/7971/2 "2012-06-08T22:03:38Z")
Using facets for autocomplete usually does not make sense, search the
mailing list on how to do it with edge ngrams.
On Mon, Jun 4, 2012 at 6:55 PM, slushi [kireetreddy@gmail.com](mailto:kireetreddy@gmail.com) wrote:
> I am trying to use facets to implement autocomplete behavior (try to
> autocomplete the search term as the user types) but I have been having
> memory issues. I have an single node index with 2 shards, each with about
> 450K documents and 150K terms in the field being used for autocomplete.
> After starting the server, I tried doing my facet search which is something
> like this:
>
> { "size" : 0,
> "query" : {
> "prefix" : { "f" : "foo" }
> },
> "facets" : {
> "autocomplete" : {
> "terms" : {
> "field" : "f",
> "regex" : "foo.\*",
> "regex\_flags" : "DOTALL",
> "size" : 5
> }
> }
> }
> }
>
> the bigdesk cache graph showed the field cache jumping up to 800,000,000.
> This seems high, these are basically tags so they should be short strings.
> I saw this thread[https://groups.google.com/forum/?fromgroups#!topic/elasticsearch/Mdh8hN04c2c](https://groups.google.com/forum/?fromgroups#!topic/elasticsearch/Mdh8hN04c2c) which
> is probably relevant as the field is indeed multivalued. I will try to
> switch to nested documents and see if there is any improvement. A few
> additional questions:
>
> 1. The docs say that by default field cache entries never expire. I
> assume the cache is still kept coherent with the index, so if docs are
> updated/deleted/expired the relevant cache entries are evicted?
> 2. Does the number of shards/node increase field cache memory usage?
> I.e., would memory usage go down if i used one shard rather than 2?
> 3. I have a specific field just to support autocomplete, would it be
> better to simply split out the field into its own index rather than use
> nested docs?
> 4. Does this approach (facets for autocomplete) make sense given the
> size of the field? I was using solr before which had a way (terms
> component [http://wiki.apache.org/solr/TermsComponent](http://wiki.apache.org/solr/TermsComponent)) to directly
> access the index terms, this seems much more efficient than what I am doing
> here. Would it make more sense to build a separate index for autocomplete
> by periodically mining the top terms from the main index and then doing
> "normal" prefix queries to get a suggestion?
>
> Thanks!
---
### Author:  [@slushi](https://discuss.elastic.co/u/slushi)
#### Post date: [August 14, 2012, 5:02pm UTC](https://discuss.elastic.co/t/memory-issues-with-facets/7971/3 "2012-08-14T17:02:57Z")
I was planning to try ngrams as well. One issue I am unsure about is how to
boost popular terms in the ngram index. The nice thing about the solr
TermsComponent or facets is I can suggest common terms as the user types
instead of the "closest" term. If I build an autocompletion index while I
am building my main index, it's not clear to me how to easily boost terms
according to index popularity.
On Friday, June 8, 2012 6:03:38 PM UTC-4, kimchy wrote:
> Using facets for autocomplete usually does not make sense, search the
> mailing list on how to do it with edge ngrams.
>
> On Mon, Jun 4, 2012 at 6:55 PM, slushi \<[kiree...@gmail.com](mailto:kiree...@gmail.com) \\>wrote:
>
> > I am trying to use facets to implement autocomplete behavior (try to
> > autocomplete the search term as the user types) but I have been having
> > memory issues. I have an single node index with 2 shards, each with about
> > 450K documents and 150K terms in the field being used for autocomplete.
> > After starting the server, I tried doing my facet search which is something
> > like this:
> >
> > { "size" : 0,
> > "query" : {
> > "prefix" : { "f" : "foo" }
> > },
> > "facets" : {
> > "autocomplete" : {
> > "terms" : {
> > "field" : "f",
> > "regex" : "foo.\*",
> > "regex\_flags" : "DOTALL",
> > "size" : 5
> > }
> > }
> > }
> > }
> >
> > the bigdesk cache graph showed the field cache jumping up to 800,000,000.
> > This seems high, these are basically tags so they should be short strings.
> > I saw this thread[https://groups.google.com/forum/?fromgroups#!topic/elasticsearch/Mdh8hN04c2c](https://groups.google.com/forum/?fromgroups#!topic/elasticsearch/Mdh8hN04c2c) which
> > is probably relevant as the field is indeed multivalued. I will try to
> > switch to nested documents and see if there is any improvement. A few
> > additional questions:
> >
> > 1. The docs say that by default field cache entries never expire. I
> > assume the cache is still kept coherent with the index, so if docs are
> > updated/deleted/expired the relevant cache entries are evicted?
> > 2. Does the number of shards/node increase field cache memory usage?
> > I.e., would memory usage go down if i used one shard rather than 2?
> > 3. I have a specific field just to support autocomplete, would it be
> > better to simply split out the field into its own index rather than use
> > nested docs?
> > 4. Does this approach (facets for autocomplete) make sense given the
> > size of the field? I was using solr before which had a way (terms
> > component [http://wiki.apache.org/solr/TermsComponent](http://wiki.apache.org/solr/TermsComponent)) to directly
> > access the index terms, this seems much more efficient than what I am doing
> > here. Would it make more sense to build a separate index for autocomplete
> > by periodically mining the top terms from the main index and then doing
> > "normal" prefix queries to get a suggestion?
> >
> > Thanks!
--
---
### Author:  [@Ludovic\_Fleury](https://discuss.elastic.co/u/Ludovic_Fleury)
#### Post date: [August 17, 2012, 2:00pm UTC](https://discuss.elastic.co/t/memory-issues-with-facets/7971/4 "2012-08-17T14:00:39Z")
Hey, I'm new to ES but I've got the same need:
I've got a list of place with a city and I wanted to provide an
autocomplete feature on the facets list.
I can't see how to do that without facets (because it's an aggregation of
unique value for a field in my index type). Any hint ?
Thanks
Le samedi 9 juin 2012 00:03:38 UTC+2, kimchy a écrit :
> Using facets for autocomplete usually does not make sense, search the
> mailing list on how to do it with edge ngrams.
>
> On Mon, Jun 4, 2012 at 6:55 PM, slushi \<[kiree...@gmail.com](mailto:kiree...@gmail.com) \\>wrote:
>
> > I am trying to use facets to implement autocomplete behavior (try to
> > autocomplete the search term as the user types) but I have been having
> > memory issues. I have an single node index with 2 shards, each with about
> > 450K documents and 150K terms in the field being used for autocomplete.
> > After starting the server, I tried doing my facet search which is something
> > like this:
> >
> > { "size" : 0,
> > "query" : {
> > "prefix" : { "f" : "foo" }
> > },
> > "facets" : {
> > "autocomplete" : {
> > "terms" : {
> > "field" : "f",
> > "regex" : "foo.\*",
> > "regex\_flags" : "DOTALL",
> > "size" : 5
> > }
> > }
> > }
> > }
> >
> > the bigdesk cache graph showed the field cache jumping up to 800,000,000.
> > This seems high, these are basically tags so they should be short strings.
> > I saw this thread[https://groups.google.com/forum/?fromgroups#!topic/elasticsearch/Mdh8hN04c2c](https://groups.google.com/forum/?fromgroups#!topic/elasticsearch/Mdh8hN04c2c) which
> > is probably relevant as the field is indeed multivalued. I will try to
> > switch to nested documents and see if there is any improvement. A few
> > additional questions:
> >
> > 1. The docs say that by default field cache entries never expire. I
> > assume the cache is still kept coherent with the index, so if docs are
> > updated/deleted/expired the relevant cache entries are evicted?
> > 2. Does the number of shards/node increase field cache memory usage?
> > I.e., would memory usage go down if i used one shard rather than 2?
> > 3. I have a specific field just to support autocomplete, would it be
> > better to simply split out the field into its own index rather than use
> > nested docs?
> > 4. Does this approach (facets for autocomplete) make sense given the
> > size of the field? I was using solr before which had a way (terms
> > component [http://wiki.apache.org/solr/TermsComponent](http://wiki.apache.org/solr/TermsComponent)) to directly
> > access the index terms, this seems much more efficient than what I am doing
> > here. Would it make more sense to build a separate index for autocomplete
> > by periodically mining the top terms from the main index and then doing
> > "normal" prefix queries to get a suggestion?
> >
> > Thanks!
--
---
### Author:  [@system](https://discuss.elastic.co/u/system)
#### Post date: [July 6, 2017, 3:16am UTC](https://discuss.elastic.co/t/memory-issues-with-facets/7971/5 "2017-07-06T03:16:04Z")