# Terms Facet never returns a response when the number of documents increases past tens of thousands

**URL:** https://discuss.elastic.co/t/terms-facet-never-returns-a-response-when-the-number-of-documents-increases-past-tens-of-thousands/14941
**Category:** Elasticsearch
**Created:** [December 18, 2013, 9:51pm UTC](https://discuss.elastic.co/t/terms-facet-never-returns-a-response-when-the-number-of-documents-increases-past-tens-of-thousands/14941 "2013-12-18T21:51:02Z")
**Posts on this page:** 5
**Page:** 1

<div class="post-metadata">

### Author: ![tbrianjones](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/tbrianjones/32/1402_2.png) [@tbrianjones](https://discuss.elastic.co/u/tbrianjones)
#### Post date: [December 18, 2013, 9:51pm UTC](https://discuss.elastic.co/t/terms-facet-never-returns-a-response-when-the-number-of-documents-increases-past-tens-of-thousands/14941/1 "2013-12-18T21:51:02Z")

</div>

I'm using the Terms Facet with Elasticsearch V0.20.2. The server has 8 x  
Intel Xeon E5-2680 v2 processors and 15GB of memory.

My Terms Facet queries work great as long as the number of documents in the  
index is small ( eg. less than 20,000 ). When the system hits more,  
pushing into the hundreds of thousands or millions of documents, my Terms  
Facets never return results. Watching the server, I initially see a few  
Java processes using a lot of CPU, but within a few seconds, this is  
reduced to a half dozen processes each using ~2% cpu. I never see memory  
usage increase on the server as a result of these queries. When these  
queries fail to return results, they also sometimes seem to "freeze"  
Elasticsearch and I often have to restart the ES server or even reboot the  
physical server to get ES back online for other simple queries.

The fields I'm trying to facet exist for nearly every document and can have  
anywhere from 0 to hundreds of different values across the dataset. All  
values are text strings and I'm using a custom analyzer that reduces them  
to lowercase. I realize that increasing the number of potential values in  
a field will dramatically increase the resources needed for the Terms Facet  
Query. In testing, I would expect some of the smaller fields should work  
fine even at scale with millions of documents.

Questions:

1.) My test field ( industries ), can have no more than 32 unique values.  
Each document could have none or all 32 values. Each value can be from 10  
to 100 characters of text. This Terms Facet never returns a result at  
scale. Any thoughts on what is happening? Is my setup flawed?

1. Will I ever be able to run a facet on a field that can have millions of  
unique text values? I have some data analysis cases like this where I'd  
like to use Elasticsearch Facetting.

3.) Would reducing the fields I'm faceting on to integers ( and then  
translating back to text outside ES ) make a big difference in performance  
and required resources?

Test Query:

curl -X POST "http://remote\_host:9200/companies/company/\_search?pretty=true"  
-d '  
{  
"query" : {  
"match\_all" : { }  
},  
"facets" : {  
"industries" : {  
"terms" : {  
"field" : "industries.term.keyword\_lowercase",  
"size" : 100  
}  
}  
},  
"size" : 0  
}  
'

Index Configuration:

{  
"index" : {  
"number\_of\_shards" : 5,  
"number\_of\_replicas" : 1,  
"analysis" : {  
"analyzer" : {  
"default" : {  
"tokenizer" : "standard",  
"filter" : ["standard", "word\_delimiter", "lowercase", "stop"]  
},  
"html\_strip" : {  
"tokenizer" : "standard",  
"filter" : ["standard", "word\_delimiter", "lowercase", "stop"],  
"char\_filter" : "html\_strip"  
},  
"keyword\_lowercase" : {  
"tokenizer" : "keyword",  
"filter" : "lowercase"  
}  
}  
}  
}  
}

Company Document Mapping:

\*\* i've removed irrelevant fields

{  
"company" : {  
"type" : "object",  
"include\_in\_all" : false,  
"path" : "full",  
"dynamic" : "strict",  
"properties" : {  
"name" : {  
"type" : "multi\_field",  
"fields" : {  
"name" : { "type" : "string", "index" : "analyzed", "include\_in\_all" :  
"true", "boost" : 10.0 },  
"keyword\_lowercase" : { "type" : "string", "index" : "analyzed", "analyzer"  
: "keyword\_lowercase", "include\_in\_all" : "false" }  
}  
},  
"description" : { "type" : "string", "index" : "analyzed", "include\_in\_all"  
: "true", "boost" : 6.0 },  
"industries" : {  
"type" : "nested",  
"include\_in\_root" : true,  
"properties" : {  
"term" : {  
"type" : "multi\_field",  
"fields" : {  
"term" : { "type" : "string", "index" : "analyzed", "include\_in\_all" :  
true, "boost" : 3.0 },  
"keyword\_lowercase" : { "type" : "string", "index" : "analyzed", "analyzer"  
: "keyword\_lowercase" }  
}  
},  
"description" : { "type" : "string", "index" : "analyzed", "include\_in\_all"  
: true },  
"score" : { "type" : "integer" },  
"verified" : { "type" : "boolean" }  
}  
}  
}  
}  
}

--  
You received this message because you are subscribed to the Google Groups "elasticsearch" group.  
To unsubscribe from this group and stop receiving emails from it, send an email to [elasticsearch+unsubscribe@googlegroups.com](mailto:elasticsearch+unsubscribe@googlegroups.com).  
To view this discussion on the web visit [https://groups.google.com/d/msgid/elasticsearch/3e608b31-8569-49d3-b9fa-20d3a1e4a597%40googlegroups.com](https://groups.google.com/d/msgid/elasticsearch/3e608b31-8569-49d3-b9fa-20d3a1e4a597%40googlegroups.com).  
For more options, visit [https://groups.google.com/groups/opt\_out](https://groups.google.com/groups/opt_out).

---

<div class="post-metadata">

### Author: ![spinscale](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/spinscale/32/25011_2.png) [@spinscale](https://discuss.elastic.co/u/spinscale)
#### Post date: [December 19, 2013, 3:00pm UTC](https://discuss.elastic.co/t/terms-facet-never-returns-a-response-when-the-number-of-documents-increases-past-tens-of-thousands/14941/2 "2013-12-19T15:00:13Z")

</div>

Hey,

can you test with a more recent version of elasticsearch first? There were  
some dramatic improvements regarding facetting.  
Also, you should explain your setup a bit more. Facetting can need a lot of  
memory with lots of documents as it uses so-called fielddata, so you should  
configure and monitor elasticsearch appropriately.

See

> **[Elasticsearch Platform — Find real-time answers at scale](https://www.elastic.co)**
>
> Power insights and outcomes with the Elasticsearch Platform and AI. See into your data and find answers that matter with enterprise solutions designed to help you build, observe, and protect. Try Elasticsearch free today.

> **[Elasticsearch Platform — Find real-time answers at scale](https://www.elastic.co)**
>
> Power insights and outcomes with the Elasticsearch Platform and AI. See into your data and find answers that matter with enterprise solutions designed to help you build, observe, and protect. Try Elasticsearch free today.

--Alex

On Wed, Dec 18, 2013 at 10:51 PM, Brian Jones [tbrianjones@gmail.com](mailto:tbrianjones@gmail.com) wrote:

> I'm using the Terms Facet with Elasticsearch V0.20.2. The server has 8 x  
> Intel Xeon E5-2680 v2 processors and 15GB of memory.
> 
> My Terms Facet queries work great as long as the number of documents in  
> the index is small ( eg. less than 20,000 ). When the system hits more,  
> pushing into the hundreds of thousands or millions of documents, my Terms  
> Facets never return results. Watching the server, I initially see a few  
> Java processes using a lot of CPU, but within a few seconds, this is  
> reduced to a half dozen processes each using ~2% cpu. I never see memory  
> usage increase on the server as a result of these queries. When these  
> queries fail to return results, they also sometimes seem to "freeze"  
> Elasticsearch and I often have to restart the ES server or even reboot the  
> physical server to get ES back online for other simple queries.
> 
> The fields I'm trying to facet exist for nearly every document and can  
> have anywhere from 0 to hundreds of different values across the dataset.  
> All values are text strings and I'm using a custom analyzer that reduces  
> them to lowercase. I realize that increasing the number of potential  
> values in a field will dramatically increase the resources needed for the  
> Terms Facet Query. In testing, I would expect some of the smaller fields  
> should work fine even at scale with millions of documents.
> 
> Questions:
> 
> 1.) My test field ( industries ), can have no more than 32 unique values.  
> Each document could have none or all 32 values. Each value can be from 10  
> to 100 characters of text. This Terms Facet never returns a result at  
> scale. Any thoughts on what is happening? Is my setup flawed?
> 
> 1. Will I ever be able to run a facet on a field that can have millions of  
> unique text values? I have some data analysis cases like this where I'd  
> like to use Elasticsearch Facetting.
> 
> 3.) Would reducing the fields I'm faceting on to integers ( and then  
> translating back to text outside ES ) make a big difference in performance  
> and required resources?
> 
> Test Query:
> 
> curl -X POST "  
> http://remote\_host:9200/companies/company/\_search?pretty=true" -d '  
> {  
> "query" : {  
> "match\_all" : { }  
> },  
> "facets" : {  
> "industries" : {  
> "terms" : {  
> "field" : "industries.term.keyword\_lowercase",  
> "size" : 100  
> }  
> }  
> },  
> "size" : 0  
> }  
> '
> 
> Index Configuration:
> 
> {  
> "index" : {  
> "number\_of\_shards" : 5,  
> "number\_of\_replicas" : 1,  
> "analysis" : {  
> "analyzer" : {  
> "default" : {  
> "tokenizer" : "standard",  
> "filter" : ["standard", "word\_delimiter", "lowercase", "stop"]  
> },  
> "html\_strip" : {  
> "tokenizer" : "standard",  
> "filter" : ["standard", "word\_delimiter", "lowercase", "stop"],  
> "char\_filter" : "html\_strip"  
> },  
> "keyword\_lowercase" : {  
> "tokenizer" : "keyword",  
> "filter" : "lowercase"  
> }  
> }  
> }  
> }  
> }
> 
> Company Document Mapping:
> 
> \*\* i've removed irrelevant fields
> 
> {  
> "company" : {  
> "type" : "object",  
> "include\_in\_all" : false,  
> "path" : "full",  
> "dynamic" : "strict",  
> "properties" : {  
> "name" : {  
> "type" : "multi\_field",  
> "fields" : {  
> "name" : { "type" : "string", "index" : "analyzed", "include\_in\_all" :  
> "true", "boost" : 10.0 },  
> "keyword\_lowercase" : { "type" : "string", "index" : "analyzed",  
> "analyzer" : "keyword\_lowercase", "include\_in\_all" : "false" }  
> }  
> },  
> "description" : { "type" : "string", "index" : "analyzed",  
> "include\_in\_all" : "true", "boost" : 6.0 },  
> "industries" : {  
> "type" : "nested",  
> "include\_in\_root" : true,  
> "properties" : {  
> "term" : {  
> "type" : "multi\_field",  
> "fields" : {  
> "term" : { "type" : "string", "index" : "analyzed", "include\_in\_all" :  
> true, "boost" : 3.0 },  
> "keyword\_lowercase" : { "type" : "string", "index" : "analyzed",  
> "analyzer" : "keyword\_lowercase" }  
> }  
> },  
> "description" : { "type" : "string", "index" : "analyzed",  
> "include\_in\_all" : true },  
> "score" : { "type" : "integer" },  
> "verified" : { "type" : "boolean" }  
> }  
> }  
> }  
> }  
> }
> 
> --  
> You received this message because you are subscribed to the Google Groups  
> "elasticsearch" group.  
> To unsubscribe from this group and stop receiving emails from it, send an  
> email to [elasticsearch+unsubscribe@googlegroups.com](mailto:elasticsearch+unsubscribe@googlegroups.com).  
> To view this discussion on the web visit  
> [https://groups.google.com/d/msgid/elasticsearch/3e608b31-8569-49d3-b9fa-20d3a1e4a597%40googlegroups.com](https://groups.google.com/d/msgid/elasticsearch/3e608b31-8569-49d3-b9fa-20d3a1e4a597%40googlegroups.com)  
> .  
> For more options, visit [https://groups.google.com/groups/opt\_out](https://groups.google.com/groups/opt_out).

--  
You received this message because you are subscribed to the Google Groups "elasticsearch" group.  
To unsubscribe from this group and stop receiving emails from it, send an email to [elasticsearch+unsubscribe@googlegroups.com](mailto:elasticsearch+unsubscribe@googlegroups.com).  
To view this discussion on the web visit [https://groups.google.com/d/msgid/elasticsearch/CAGCwEM80zPKeTo%3DrXEBinoatkZmX%2BbWqhx2itE4tuCBg87NEwQ%40mail.gmail.com](https://groups.google.com/d/msgid/elasticsearch/CAGCwEM80zPKeTo%3DrXEBinoatkZmX%2BbWqhx2itE4tuCBg87NEwQ%40mail.gmail.com).  
For more options, visit [https://groups.google.com/groups/opt\_out](https://groups.google.com/groups/opt_out).

---

<div class="post-metadata">

### Author: ![Ivan](https://avatars.discourse-cdn.com/v4/letter/i/df788c/32.png) [@Ivan](https://discuss.elastic.co/u/Ivan)
#### Post date: [December 19, 2013, 5:16pm UTC](https://discuss.elastic.co/t/terms-facet-never-returns-a-response-when-the-number-of-documents-increases-past-tens-of-thousands/14941/3 "2013-12-19T17:16:25Z")

</div>

I completely agree about upgrading. Elasticsearch 0.90 introduced numerous  
memory improvements, including one issue that directly affects you. With  
the previous versions (0.20 and prior), high cardinality fields, such as  
your industry field, would use inefficient data structures to load the  
faceted values. Your situation would be greatly improved with 0.90+. Easily.

In terms of your last point, I would use numerical values whenever  
possible. My taxonomy is known in advance, so I do client side lookups for  
numerical values. I do not have statistics for how much of an improvement  
it is, but you can do the simple math of how much can be saved . In Java,  
ints are 4 bytes, while each character in UTF-8 (assuming you are using  
unicode) can be 1 to 6 bytes. It all adds up.

Cheers,

Ivan

On Thu, Dec 19, 2013 at 7:00 AM, Alexander Reelsen [alr@spinscale.de](mailto:alr@spinscale.de) wrote:

> Hey,
> 
> can you test with a more recent version of elasticsearch first? There were  
> some dramatic improvements regarding facetting.  
> Also, you should explain your setup a bit more. Facetting can need a lot  
> of memory with lots of documents as it uses so-called fielddata, so you  
> should configure and monitor elasticsearch appropriately.
> 
> See
> 
> [Elasticsearch Platform — Find real-time answers at scale | Elastic](http://www.elasticsearch.org/guide/en/elasticsearch/reference/current/setup-configuration.html)
> 
> [Elasticsearch Platform — Find real-time answers at scale | Elastic](http://www.elasticsearch.org/guide/en/elasticsearch/reference/current/cluster-nodes-stats.html#field-data)
> 
> --Alex
> 
> On Wed, Dec 18, 2013 at 10:51 PM, Brian Jones [tbrianjones@gmail.com](mailto:tbrianjones@gmail.com)wrote:
> 
> > I'm using the Terms Facet with Elasticsearch V0.20.2. The server has 8 x  
> > Intel Xeon E5-2680 v2 processors and 15GB of memory.
> > 
> > My Terms Facet queries work great as long as the number of documents in  
> > the index is small ( eg. less than 20,000 ). When the system hits more,  
> > pushing into the hundreds of thousands or millions of documents, my Terms  
> > Facets never return results. Watching the server, I initially see a few  
> > Java processes using a lot of CPU, but within a few seconds, this is  
> > reduced to a half dozen processes each using ~2% cpu. I never see memory  
> > usage increase on the server as a result of these queries. When these  
> > queries fail to return results, they also sometimes seem to "freeze"  
> > Elasticsearch and I often have to restart the ES server or even reboot the  
> > physical server to get ES back online for other simple queries.
> > 
> > The fields I'm trying to facet exist for nearly every document and can  
> > have anywhere from 0 to hundreds of different values across the dataset.  
> > All values are text strings and I'm using a custom analyzer that reduces  
> > them to lowercase. I realize that increasing the number of potential  
> > values in a field will dramatically increase the resources needed for the  
> > Terms Facet Query. In testing, I would expect some of the smaller fields  
> > should work fine even at scale with millions of documents.
> > 
> > Questions:
> > 
> > 1.) My test field ( industries ), can have no more than 32 unique values.  
> > Each document could have none or all 32 values. Each value can be from 10  
> > to 100 characters of text. This Terms Facet never returns a result at  
> > scale. Any thoughts on what is happening? Is my setup flawed?
> > 
> > 1. Will I ever be able to run a facet on a field that can have millions  
> > of unique text values? I have some data analysis cases like this where I'd  
> > like to use Elasticsearch Facetting.
> > 
> > 3.) Would reducing the fields I'm faceting on to integers ( and then  
> > translating back to text outside ES ) make a big difference in performance  
> > and required resources?
> > 
> > Test Query:
> > 
> > curl -X POST "  
> > http://remote\_host:9200/companies/company/\_search?pretty=true" -d '  
> > {  
> > "query" : {  
> > "match\_all" : { }  
> > },  
> > "facets" : {  
> > "industries" : {  
> > "terms" : {  
> > "field" : "industries.term.keyword\_lowercase",  
> > "size" : 100  
> > }  
> > }  
> > },  
> > "size" : 0  
> > }  
> > '
> > 
> > Index Configuration:
> > 
> > {  
> > "index" : {  
> > "number\_of\_shards" : 5,  
> > "number\_of\_replicas" : 1,  
> > "analysis" : {  
> > "analyzer" : {  
> > "default" : {  
> > "tokenizer" : "standard",  
> > "filter" : ["standard", "word\_delimiter", "lowercase", "stop"]  
> > },  
> > "html\_strip" : {  
> > "tokenizer" : "standard",  
> > "filter" : ["standard", "word\_delimiter", "lowercase", "stop"],  
> > "char\_filter" : "html\_strip"  
> > },  
> > "keyword\_lowercase" : {  
> > "tokenizer" : "keyword",  
> > "filter" : "lowercase"  
> > }  
> > }  
> > }  
> > }  
> > }
> > 
> > Company Document Mapping:
> > 
> > \*\* i've removed irrelevant fields
> > 
> > {  
> > "company" : {  
> > "type" : "object",  
> > "include\_in\_all" : false,  
> > "path" : "full",  
> > "dynamic" : "strict",  
> > "properties" : {  
> > "name" : {  
> > "type" : "multi\_field",  
> > "fields" : {  
> > "name" : { "type" : "string", "index" : "analyzed", "include\_in\_all" :  
> > "true", "boost" : 10.0 },  
> > "keyword\_lowercase" : { "type" : "string", "index" : "analyzed",  
> > "analyzer" : "keyword\_lowercase", "include\_in\_all" : "false" }  
> > }  
> > },  
> > "description" : { "type" : "string", "index" : "analyzed",  
> > "include\_in\_all" : "true", "boost" : 6.0 },  
> > "industries" : {  
> > "type" : "nested",  
> > "include\_in\_root" : true,  
> > "properties" : {  
> > "term" : {  
> > "type" : "multi\_field",  
> > "fields" : {  
> > "term" : { "type" : "string", "index" : "analyzed", "include\_in\_all" :  
> > true, "boost" : 3.0 },  
> > "keyword\_lowercase" : { "type" : "string", "index" : "analyzed",  
> > "analyzer" : "keyword\_lowercase" }  
> > }  
> > },  
> > "description" : { "type" : "string", "index" : "analyzed",  
> > "include\_in\_all" : true },  
> > "score" : { "type" : "integer" },  
> > "verified" : { "type" : "boolean" }  
> > }  
> > }  
> > }  
> > }  
> > }
> > 
> > --  
> > You received this message because you are subscribed to the Google Groups  
> > "elasticsearch" group.  
> > To unsubscribe from this group and stop receiving emails from it, send an  
> > email to [elasticsearch+unsubscribe@googlegroups.com](mailto:elasticsearch+unsubscribe@googlegroups.com).  
> > To view this discussion on the web visit  
> > [https://groups.google.com/d/msgid/elasticsearch/3e608b31-8569-49d3-b9fa-20d3a1e4a597%40googlegroups.com](https://groups.google.com/d/msgid/elasticsearch/3e608b31-8569-49d3-b9fa-20d3a1e4a597%40googlegroups.com)  
> > .  
> > For more options, visit [https://groups.google.com/groups/opt\_out](https://groups.google.com/groups/opt_out).
> 
> --  
> You received this message because you are subscribed to the Google Groups  
> "elasticsearch" group.  
> To unsubscribe from this group and stop receiving emails from it, send an  
> email to [elasticsearch+unsubscribe@googlegroups.com](mailto:elasticsearch+unsubscribe@googlegroups.com).  
> To view this discussion on the web visit  
> [https://groups.google.com/d/msgid/elasticsearch/CAGCwEM80zPKeTo%3DrXEBinoatkZmX%2BbWqhx2itE4tuCBg87NEwQ%40mail.gmail.com](https://groups.google.com/d/msgid/elasticsearch/CAGCwEM80zPKeTo%3DrXEBinoatkZmX%2BbWqhx2itE4tuCBg87NEwQ%40mail.gmail.com)  
> .
> 
> For more options, visit [https://groups.google.com/groups/opt\_out](https://groups.google.com/groups/opt_out).

--  
You received this message because you are subscribed to the Google Groups "elasticsearch" group.  
To unsubscribe from this group and stop receiving emails from it, send an email to [elasticsearch+unsubscribe@googlegroups.com](mailto:elasticsearch+unsubscribe@googlegroups.com).  
To view this discussion on the web visit [https://groups.google.com/d/msgid/elasticsearch/CALY%3DcQBa14uQ3De4XPCv%2BRP6qxBxKcHZi6A-j09Qi759Ru0X7w%40mail.gmail.com](https://groups.google.com/d/msgid/elasticsearch/CALY%3DcQBa14uQ3De4XPCv%2BRP6qxBxKcHZi6A-j09Qi759Ru0X7w%40mail.gmail.com).  
For more options, visit [https://groups.google.com/groups/opt\_out](https://groups.google.com/groups/opt_out).

---

<div class="post-metadata">

### Author: ![tbrianjones](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/tbrianjones/32/1402_2.png) [@tbrianjones](https://discuss.elastic.co/u/tbrianjones)
#### Post date: [January 7, 2014, 1:54am UTC](https://discuss.elastic.co/t/terms-facet-never-returns-a-response-when-the-number-of-documents-increases-past-tens-of-thousands/14941/4 "2014-01-07T01:54:49Z")

</div>

I built a fresh server with Elasticsearch V0.90.9 and all my problems went  
away without making a single change to my index or code. It's running on a  
much smaller server ( CPU and MEMORY ) than my previous install as well  
with no problems.

The new install even loaded an index that was originally created by a  
server running 0.2.x from Amazon S3.

This is great. Elasticsearch impresses and surprises me again.

I will probably still investigate converting the text i'm Facetting on to  
integer id's that I'll convert back to strings for users on the app end of  
things. It sounds like this will further increase performance.

On Thursday, December 19, 2013 7:00:13 AM UTC-8, Alexander Reelsen wrote:

> Hey,
> 
> can you test with a more recent version of elasticsearch first? There were  
> some dramatic improvements regarding facetting.  
> Also, you should explain your setup a bit more. Facetting can need a lot  
> of memory with lots of documents as it uses so-called fielddata, so you  
> should configure and monitor elasticsearch appropriately.
> 
> See
> 
> [Elasticsearch Platform — Find real-time answers at scale | Elastic](http://www.elasticsearch.org/guide/en/elasticsearch/reference/current/setup-configuration.html)
> 
> [Elasticsearch Platform — Find real-time answers at scale | Elastic](http://www.elasticsearch.org/guide/en/elasticsearch/reference/current/cluster-nodes-stats.html#field-data)
> 
> --Alex
> 
> On Wed, Dec 18, 2013 at 10:51 PM, Brian Jones \<[tbria...@gmail.com](mailto:tbria...@gmail.com)\<javascript:\>
> 
> > wrote:
> 
> > I'm using the Terms Facet with Elasticsearch V0.20.2. The server has 8 x  
> > Intel Xeon E5-2680 v2 processors and 15GB of memory.
> > 
> > My Terms Facet queries work great as long as the number of documents in  
> > the index is small ( eg. less than 20,000 ). When the system hits more,  
> > pushing into the hundreds of thousands or millions of documents, my Terms  
> > Facets never return results. Watching the server, I initially see a few  
> > Java processes using a lot of CPU, but within a few seconds, this is  
> > reduced to a half dozen processes each using ~2% cpu. I never see memory  
> > usage increase on the server as a result of these queries. When these  
> > queries fail to return results, they also sometimes seem to "freeze"  
> > Elasticsearch and I often have to restart the ES server or even reboot the  
> > physical server to get ES back online for other simple queries.
> > 
> > The fields I'm trying to facet exist for nearly every document and can  
> > have anywhere from 0 to hundreds of different values across the dataset.  
> > All values are text strings and I'm using a custom analyzer that reduces  
> > them to lowercase. I realize that increasing the number of potential  
> > values in a field will dramatically increase the resources needed for the  
> > Terms Facet Query. In testing, I would expect some of the smaller fields  
> > should work fine even at scale with millions of documents.
> > 
> > Questions:
> > 
> > 1.) My test field ( industries ), can have no more than 32 unique values.  
> > Each document could have none or all 32 values. Each value can be from 10  
> > to 100 characters of text. This Terms Facet never returns a result at  
> > scale. Any thoughts on what is happening? Is my setup flawed?
> > 
> > 1. Will I ever be able to run a facet on a field that can have millions  
> > of unique text values? I have some data analysis cases like this where I'd  
> > like to use Elasticsearch Facetting.
> > 
> > 3.) Would reducing the fields I'm faceting on to integers ( and then  
> > translating back to text outside ES ) make a big difference in performance  
> > and required resources?
> > 
> > Test Query:
> > 
> > curl -X POST "  
> > http://remote\_host:9200/companies/company/\_search?pretty=true" -d '  
> > {  
> > "query" : {  
> > "match\_all" : { }  
> > },  
> > "facets" : {  
> > "industries" : {  
> > "terms" : {  
> > "field" : "industries.term.keyword\_lowercase",  
> > "size" : 100  
> > }  
> > }  
> > },  
> > "size" : 0  
> > }  
> > '
> > 
> > Index Configuration:
> > 
> > {  
> > "index" : {  
> > "number\_of\_shards" : 5,  
> > "number\_of\_replicas" : 1,  
> > "analysis" : {  
> > "analyzer" : {  
> > "default" : {  
> > "tokenizer" : "standard",  
> > "filter" : ["standard", "word\_delimiter", "lowercase", "stop"]  
> > },  
> > "html\_strip" : {  
> > "tokenizer" : "standard",  
> > "filter" : ["standard", "word\_delimiter", "lowercase", "stop"],  
> > "char\_filter" : "html\_strip"  
> > },  
> > "keyword\_lowercase" : {  
> > "tokenizer" : "keyword",  
> > "filter" : "lowercase"  
> > }  
> > }  
> > }  
> > }  
> > }
> > 
> > Company Document Mapping:
> > 
> > \*\* i've removed irrelevant fields
> > 
> > {  
> > "company" : {  
> > "type" : "object",  
> > "include\_in\_all" : false,  
> > "path" : "full",  
> > "dynamic" : "strict",  
> > "properties" : {  
> > "name" : {  
> > "type" : "multi\_field",  
> > "fields" : {  
> > "name" : { "type" : "string", "index" : "analyzed", "include\_in\_all" :  
> > "true", "boost" : 10.0 },  
> > "keyword\_lowercase" : { "type" : "string", "index" : "analyzed",  
> > "analyzer" : "keyword\_lowercase", "include\_in\_all" : "false" }  
> > }  
> > },  
> > "description" : { "type" : "string", "index" : "analyzed",  
> > "include\_in\_all" : "true", "boost" : 6.0 },  
> > "industries" : {  
> > "type" : "nested",  
> > "include\_in\_root" : true,  
> > "properties" : {  
> > "term" : {  
> > "type" : "multi\_field",  
> > "fields" : {  
> > "term" : { "type" : "string", "index" : "analyzed", "include\_in\_all" :  
> > true, "boost" : 3.0 },  
> > "keyword\_lowercase" : { "type" : "string", "index" : "analyzed",  
> > "analyzer" : "keyword\_lowercase" }  
> > }  
> > },  
> > "description" : { "type" : "string", "index" : "analyzed",  
> > "include\_in\_all" : true },  
> > "score" : { "type" : "integer" },  
> > "verified" : { "type" : "boolean" }  
> > }  
> > }  
> > }  
> > }  
> > }
> > 
> > --  
> > You received this message because you are subscribed to the Google Groups  
> > "elasticsearch" group.  
> > To unsubscribe from this group and stop receiving emails from it, send an  
> > email to [elasticsearc...@googlegroups.com](mailto:elasticsearc...@googlegroups.com) \<javascript:\>.  
> > To view this discussion on the web visit  
> > [https://groups.google.com/d/msgid/elasticsearch/3e608b31-8569-49d3-b9fa-20d3a1e4a597%40googlegroups.com](https://groups.google.com/d/msgid/elasticsearch/3e608b31-8569-49d3-b9fa-20d3a1e4a597%40googlegroups.com)  
> > .  
> > For more options, visit [https://groups.google.com/groups/opt\_out](https://groups.google.com/groups/opt_out).

--  
You received this message because you are subscribed to the Google Groups "elasticsearch" group.  
To unsubscribe from this group and stop receiving emails from it, send an email to [elasticsearch+unsubscribe@googlegroups.com](mailto:elasticsearch+unsubscribe@googlegroups.com).  
To view this discussion on the web visit [https://groups.google.com/d/msgid/elasticsearch/a9d95122-398a-42c4-968f-ea40f4b678e6%40googlegroups.com](https://groups.google.com/d/msgid/elasticsearch/a9d95122-398a-42c4-968f-ea40f4b678e6%40googlegroups.com).  
For more options, visit [https://groups.google.com/groups/opt\_out](https://groups.google.com/groups/opt_out).

---

<div class="post-metadata">

### Author: ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)
#### Post date: [July 6, 2017, 1:58am UTC](https://discuss.elastic.co/t/terms-facet-never-returns-a-response-when-the-number-of-documents-increases-past-tens-of-thousands/14941/5 "2017-07-06T01:58:08Z")

</div>


