# Field cache limits ignored

**URL:** <https://discuss.elastic.co/t/field-cache-limits-ignored/9569>\
**Category:** Elasticsearch\
**Created:** [November 5, 2012, 10:58pm UTC](https://discuss.elastic.co/t/field-cache-limits-ignored/9569 "2012-11-05T22:58:44Z")\
**Posts on this page:** 10\
**Page:** 1

<div class="post-metadata">

**Author:** ![Logan\_2](https://avatars.discourse-cdn.com/v4/letter/l/c2a13f/32.png) [@Logan\_2](https://discuss.elastic.co/u/Logan_2)\
**Post date:** [November 5, 2012, 10:58pm UTC](https://discuss.elastic.co/t/field-cache-limits-ignored/9569/1 "2012-11-05T22:58:44Z")

</div>

I'm having problems controlling OOME errors on my 6 node CentOS 5.6  
cluster. I am currently running elasticsearch-0.19.8 using the service  
wrapper and java version "1.6.0\_25". I have set the following  
index.cache.field settings in bin/service/elasticsearch.conf and when that  
failed config/elasticsearch.yml but they seem to be ignored as I never see  
any field cache evictions in bigdesk and field cache will eventually eat up  
all the heap memory when certain searches are performed.

index.cache.field.max\_size: 1000  
index.cache.field.expire: 5m  
index.cache.field.type: soft

I'm not entirely sure that filed.type: soft isn't working as sometimes the  
field cache will drop to zero after a GC but it seems to only work when the  
cluster is idle. But field.expire: and max\_size: definitely seem to have  
no effect.

Am I going about this the right way? What config should I be setting these  
values in? Is there a good way to verify what settings are currently in  
effect on the cluster?

Logan

--

---

<div class="post-metadata">

**Author:** ![Igor\_Motov](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/igor_motov/32/45193_2.png) [@Igor\_Motov](https://discuss.elastic.co/u/Igor_Motov)\
**Post date:** [November 6, 2012, 12:19am UTC](https://discuss.elastic.co/t/field-cache-limits-ignored/9569/2 "2012-11-06T00:19:38Z")

</div>

The index.cache.field.max\_size and index.cache.field.expire settings are  
applicable only to the "resident" cache type. The "soft" cache type is  
garbage collected in response to memory demand. If all heap is getting used  
even with soft cache it might be a good indication that you simply don't  
have enough memory on the nodes to perform these queries. What type of  
queries are causing OOM? Are you doing any faceted searches on multivalued  
fields?

On Monday, November 5, 2012 5:58:44 PM UTC-5, Logan wrote:

> I'm having problems controlling OOME errors on my 6 node CentOS 5.6  
> cluster. I am currently running elasticsearch-0.19.8 using the service  
> wrapper and java version "1.6.0\_25". I have set the following  
> index.cache.field settings in bin/service/elasticsearch.conf and when that  
> failed config/elasticsearch.yml but they seem to be ignored as I never see  
> any field cache evictions in bigdesk and field cache will eventually eat up  
> all the heap memory when certain searches are performed.
> 
> index.cache.field.max\_size: 1000  
> index.cache.field.expire: 5m  
> index.cache.field.type: soft
> 
> I'm not entirely sure that filed.type: soft isn't working as sometimes the  
> field cache will drop to zero after a GC but it seems to only work when the  
> cluster is idle. But field.expire: and max\_size: definitely seem to have  
> no effect.
> 
> Am I going about this the right way? What config should I be setting these  
> values in? Is there a good way to verify what settings are currently in  
> effect on the cluster?
> 
> Logan

--

---

<div class="post-metadata">

**Author:** ![Sushant\_Shankar](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/sushant_shankar/32/2652_2.png) [@Sushant\_Shankar](https://discuss.elastic.co/u/Sushant_Shankar)\
**Post date:** [November 6, 2012, 12:34am UTC](https://discuss.elastic.co/t/field-cache-limits-ignored/9569/3 "2012-11-06T00:34:12Z")

</div>

Hi Igor,

It is possible that we do not have enough memory for this. The odd thing is  
that we're not using facets. We are issuing CPU-intensive custom script  
queries (as we need to compute operations like the sum of different fields).

Thanks,  
Sushant

On Monday, November 5, 2012 4:19:38 PM UTC-8, Igor Motov wrote:

> The index.cache.field.max\_size and index.cache.field.expire settings are  
> applicable only to the "resident" cache type. The "soft" cache type is  
> garbage collected in response to memory demand. If all heap is getting used  
> even with soft cache it might be a good indication that you simply don't  
> have enough memory on the nodes to perform these queries. What type of  
> queries are causing OOM? Are you doing any faceted searches on multivalued  
> fields?
> 
> On Monday, November 5, 2012 5:58:44 PM UTC-5, Logan wrote:
> 
> > I'm having problems controlling OOME errors on my 6 node CentOS 5.6  
> > cluster. I am currently running elasticsearch-0.19.8 using the service  
> > wrapper and java version "1.6.0\_25". I have set the following  
> > index.cache.field settings in bin/service/elasticsearch.conf and when that  
> > failed config/elasticsearch.yml but they seem to be ignored as I never see  
> > any field cache evictions in bigdesk and field cache will eventually eat up  
> > all the heap memory when certain searches are performed.
> > 
> > index.cache.field.max\_size: 1000  
> > index.cache.field.expire: 5m  
> > index.cache.field.type: soft
> > 
> > I'm not entirely sure that filed.type: soft isn't working as sometimes  
> > the field cache will drop to zero after a GC but it seems to only work when  
> > the cluster is idle. But field.expire: and max\_size: definitely seem to  
> > have no effect.
> > 
> > Am I going about this the right way? What config should I be setting  
> > these values in? Is there a good way to verify what settings are currently  
> > in effect on the cluster?
> > 
> > Logan

--

---

<div class="post-metadata">

**Author:** ![Igor\_Motov](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/igor_motov/32/45193_2.png) [@Igor\_Motov](https://discuss.elastic.co/u/Igor_Motov)\
**Post date:** [November 6, 2012, 12:47am UTC](https://discuss.elastic.co/t/field-cache-limits-ignored/9569/4 "2012-11-06T00:47:19Z")

</div>

Are you using document fields (structures like doc['my\_field']) to access  
the data?

On Monday, November 5, 2012 7:34:12 PM UTC-5, Sushant Shankar wrote:

> Hi Igor,
> 
> It is possible that we do not have enough memory for this. The odd thing  
> is that we're not using facets. We are issuing CPU-intensive custom script  
> queries (as we need to compute operations like the sum of different fields).
> 
> Thanks,  
> Sushant
> 
> On Monday, November 5, 2012 4:19:38 PM UTC-8, Igor Motov wrote:
> 
> > The index.cache.field.max\_size and index.cache.field.expire settings are  
> > applicable only to the "resident" cache type. The "soft" cache type is  
> > garbage collected in response to memory demand. If all heap is getting used  
> > even with soft cache it might be a good indication that you simply don't  
> > have enough memory on the nodes to perform these queries. What type of  
> > queries are causing OOM? Are you doing any faceted searches on multivalued  
> > fields?
> > 
> > On Monday, November 5, 2012 5:58:44 PM UTC-5, Logan wrote:
> > 
> > > I'm having problems controlling OOME errors on my 6 node CentOS 5.6  
> > > cluster. I am currently running elasticsearch-0.19.8 using the service  
> > > wrapper and java version "1.6.0\_25". I have set the following  
> > > index.cache.field settings in bin/service/elasticsearch.conf and when that  
> > > failed config/elasticsearch.yml but they seem to be ignored as I never see  
> > > any field cache evictions in bigdesk and field cache will eventually eat up  
> > > all the heap memory when certain searches are performed.
> > > 
> > > index.cache.field.max\_size: 1000  
> > > index.cache.field.expire: 5m  
> > > index.cache.field.type: soft
> > > 
> > > I'm not entirely sure that filed.type: soft isn't working as sometimes  
> > > the field cache will drop to zero after a GC but it seems to only work when  
> > > the cluster is idle. But field.expire: and max\_size: definitely seem to  
> > > have no effect.
> > > 
> > > Am I going about this the right way? What config should I be setting  
> > > these values in? Is there a good way to verify what settings are currently  
> > > in effect on the cluster?
> > > 
> > > Logan

--

---

<div class="post-metadata">

**Author:** ![Sushant\_Shankar](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/sushant_shankar/32/2652_2.png) [@Sushant\_Shankar](https://discuss.elastic.co/u/Sushant_Shankar)\
**Post date:** [November 6, 2012, 1:00am UTC](https://discuss.elastic.co/t/field-cache-limits-ignored/9569/5 "2012-11-06T01:00:22Z")

</div>

Yes we are using several calls to structures like this in a custom script,  
e.g.  
'doc['f1'] + doc['f2] .. \>= 2',  
often in addition to filter that can have up to 1000 terms.

On Monday, November 5, 2012 4:47:19 PM UTC-8, Igor Motov wrote:

> Are you using document fields (structures like doc['my\_field']) to access  
> the data?
> 
> On Monday, November 5, 2012 7:34:12 PM UTC-5, Sushant Shankar wrote:
> 
> > Hi Igor,
> > 
> > It is possible that we do not have enough memory for this. The odd thing  
> > is that we're not using facets. We are issuing CPU-intensive custom script  
> > queries (as we need to compute operations like the sum of different fields).
> > 
> > Thanks,  
> > Sushant
> > 
> > On Monday, November 5, 2012 4:19:38 PM UTC-8, Igor Motov wrote:
> > 
> > > The index.cache.field.max\_size and index.cache.field.expire settings  
> > > are applicable only to the "resident" cache type. The "soft" cache type is  
> > > garbage collected in response to memory demand. If all heap is getting used  
> > > even with soft cache it might be a good indication that you simply don't  
> > > have enough memory on the nodes to perform these queries. What type of  
> > > queries are causing OOM? Are you doing any faceted searches on multivalued  
> > > fields?
> > > 
> > > On Monday, November 5, 2012 5:58:44 PM UTC-5, Logan wrote:
> > > 
> > > > I'm having problems controlling OOME errors on my 6 node CentOS 5.6  
> > > > cluster. I am currently running elasticsearch-0.19.8 using the service  
> > > > wrapper and java version "1.6.0\_25". I have set the following  
> > > > index.cache.field settings in bin/service/elasticsearch.conf and when that  
> > > > failed config/elasticsearch.yml but they seem to be ignored as I never see  
> > > > any field cache evictions in bigdesk and field cache will eventually eat up  
> > > > all the heap memory when certain searches are performed.
> > > > 
> > > > index.cache.field.max\_size: 1000  
> > > > index.cache.field.expire: 5m  
> > > > index.cache.field.type: soft
> > > > 
> > > > I'm not entirely sure that filed.type: soft isn't working as sometimes  
> > > > the field cache will drop to zero after a GC but it seems to only work when  
> > > > the cluster is idle. But field.expire: and max\_size: definitely seem to  
> > > > have no effect.
> > > > 
> > > > Am I going about this the right way? What config should I be setting  
> > > > these values in? Is there a good way to verify what settings are currently  
> > > > in effect on the cluster?
> > > > 
> > > > Logan

--

---

<div class="post-metadata">

**Author:** ![Logan\_2](https://avatars.discourse-cdn.com/v4/letter/l/c2a13f/32.png) [@Logan\_2](https://discuss.elastic.co/u/Logan_2)\
**Post date:** [November 6, 2012, 1:15am UTC](https://discuss.elastic.co/t/field-cache-limits-ignored/9569/6 "2012-11-06T01:15:15Z")

</div>

Should I bet setting index.cache.field.\* settings in  
bin/service/elasticsearch.conf or config/elasticsearch.yml?

On Monday, November 5, 2012 6:00:22 PM UTC-7, Sushant Shankar wrote:

> Yes we are using several calls to structures like this in a custom script,  
> e.g.  
> 'doc['f1'] + doc['f2] .. \>= 2',  
> often in addition to filter that can have up to 1000 terms.
> 
> On Monday, November 5, 2012 4:47:19 PM UTC-8, Igor Motov wrote:
> 
> > Are you using document fields (structures like doc['my\_field']) to access  
> > the data?
> > 
> > On Monday, November 5, 2012 7:34:12 PM UTC-5, Sushant Shankar wrote:
> > 
> > > Hi Igor,
> > > 
> > > It is possible that we do not have enough memory for this. The odd thing  
> > > is that we're not using facets. We are issuing CPU-intensive custom script  
> > > queries (as we need to compute operations like the sum of different fields).
> > > 
> > > Thanks,  
> > > Sushant
> > > 
> > > On Monday, November 5, 2012 4:19:38 PM UTC-8, Igor Motov wrote:
> > > 
> > > > The index.cache.field.max\_size and index.cache.field.expire settings  
> > > > are applicable only to the "resident" cache type. The "soft" cache type is  
> > > > garbage collected in response to memory demand. If all heap is getting used  
> > > > even with soft cache it might be a good indication that you simply don't  
> > > > have enough memory on the nodes to perform these queries. What type of  
> > > > queries are causing OOM? Are you doing any faceted searches on multivalued  
> > > > fields?
> > > > 
> > > > On Monday, November 5, 2012 5:58:44 PM UTC-5, Logan wrote:
> > > > 
> > > > > I'm having problems controlling OOME errors on my 6 node CentOS 5.6  
> > > > > cluster. I am currently running elasticsearch-0.19.8 using the service  
> > > > > wrapper and java version "1.6.0\_25". I have set the following  
> > > > > index.cache.field settings in bin/service/elasticsearch.conf and when that  
> > > > > failed config/elasticsearch.yml but they seem to be ignored as I never see  
> > > > > any field cache evictions in bigdesk and field cache will eventually eat up  
> > > > > all the heap memory when certain searches are performed.
> > > > > 
> > > > > index.cache.field.max\_size: 1000  
> > > > > index.cache.field.expire: 5m  
> > > > > index.cache.field.type: soft
> > > > > 
> > > > > I'm not entirely sure that filed.type: soft isn't working as sometimes  
> > > > > the field cache will drop to zero after a GC but it seems to only work when  
> > > > > the cluster is idle. But field.expire: and max\_size: definitely seem to  
> > > > > have no effect.
> > > > > 
> > > > > Am I going about this the right way? What config should I be setting  
> > > > > these values in? Is there a good way to verify what settings are currently  
> > > > > in effect on the cluster?
> > > > > 
> > > > > Logan

--

---

<div class="post-metadata">

**Author:** ![Igor\_Motov](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/igor_motov/32/45193_2.png) [@Igor\_Motov](https://discuss.elastic.co/u/Igor_Motov)\
**Post date:** [November 6, 2012, 1:22am UTC](https://discuss.elastic.co/t/field-cache-limits-ignored/9569/7 "2012-11-06T01:22:51Z")

</div>

When elasticsearch sees the call for doc['f1'] it caches all values for  
this field to improve performance of the future calls. If you don't have  
enough memory to cache all these values, you might want to consider  
replacing document fields with stored fields (\_field['f1']) or even source  
(\_source['f1']). It will be slower than using document fields, especially  
if you call this script on a large number of documents, but it will have  
much lighter memory requirements.

index.cache.field.\* should be in config/elasticsearch.yml

On Monday, November 5, 2012 8:15:15 PM UTC-5, Logan wrote:

> Should I bet setting index.cache.field.\* settings in  
> bin/service/elasticsearch.conf or config/elasticsearch.yml?
> 
> On Monday, November 5, 2012 6:00:22 PM UTC-7, Sushant Shankar wrote:
> 
> > Yes we are using several calls to structures like this in a custom  
> > script, e.g.  
> > 'doc['f1'] + doc['f2] .. \>= 2',  
> > often in addition to filter that can have up to 1000 terms.
> > 
> > On Monday, November 5, 2012 4:47:19 PM UTC-8, Igor Motov wrote:
> > 
> > > Are you using document fields (structures like doc['my\_field']) to  
> > > access the data?
> > > 
> > > On Monday, November 5, 2012 7:34:12 PM UTC-5, Sushant Shankar wrote:
> > > 
> > > > Hi Igor,
> > > > 
> > > > It is possible that we do not have enough memory for this. The odd  
> > > > thing is that we're not using facets. We are issuing CPU-intensive custom  
> > > > script queries (as we need to compute operations like the sum of different  
> > > > fields).
> > > > 
> > > > Thanks,  
> > > > Sushant
> > > > 
> > > > On Monday, November 5, 2012 4:19:38 PM UTC-8, Igor Motov wrote:
> > > > 
> > > > > The index.cache.field.max\_size and index.cache.field.expire settings  
> > > > > are applicable only to the "resident" cache type. The "soft" cache type is  
> > > > > garbage collected in response to memory demand. If all heap is getting used  
> > > > > even with soft cache it might be a good indication that you simply don't  
> > > > > have enough memory on the nodes to perform these queries. What type of  
> > > > > queries are causing OOM? Are you doing any faceted searches on multivalued  
> > > > > fields?
> > > > > 
> > > > > On Monday, November 5, 2012 5:58:44 PM UTC-5, Logan wrote:
> > > > > 
> > > > > > I'm having problems controlling OOME errors on my 6 node CentOS 5.6  
> > > > > > cluster. I am currently running elasticsearch-0.19.8 using the service  
> > > > > > wrapper and java version "1.6.0\_25". I have set the following  
> > > > > > index.cache.field settings in bin/service/elasticsearch.conf and when that  
> > > > > > failed config/elasticsearch.yml but they seem to be ignored as I never see  
> > > > > > any field cache evictions in bigdesk and field cache will eventually eat up  
> > > > > > all the heap memory when certain searches are performed.
> > > > > > 
> > > > > > index.cache.field.max\_size: 1000  
> > > > > > index.cache.field.expire: 5m  
> > > > > > index.cache.field.type: soft
> > > > > > 
> > > > > > I'm not entirely sure that filed.type: soft isn't working as  
> > > > > > sometimes the field cache will drop to zero after a GC but it seems to only  
> > > > > > work when the cluster is idle. But field.expire: and  
> > > > > > max\_size: definitely seem to have no effect.
> > > > > > 
> > > > > > Am I going about this the right way? What config should I be setting  
> > > > > > these values in? Is there a good way to verify what settings are currently  
> > > > > > in effect on the cluster?
> > > > > > 
> > > > > > Logan

--

---

<div class="post-metadata">

**Author:** ![Logan\_2](https://avatars.discourse-cdn.com/v4/letter/l/c2a13f/32.png) [@Logan\_2](https://discuss.elastic.co/u/Logan_2)\
**Post date:** [November 6, 2012, 1:52am UTC](https://discuss.elastic.co/t/field-cache-limits-ignored/9569/8 "2012-11-06T01:52:18Z")

</div>

Thanks a ton Igor. I switched to resident and put the index.cache.field  
settings in elasticsearch.yml and behold they work! It's amazing how one  
bad assumption can cause weeks of pain and head scratching. Thanks again.

Logan

On Monday, November 5, 2012 6:22:51 PM UTC-7, Igor Motov wrote:

> When elasticsearch sees the call for doc['f1'] it caches all values for  
> this field to improve performance of the future calls. If you don't have  
> enough memory to cache all these values, you might want to consider  
> replacing document fields with stored fields (\_field['f1']) or even source  
> (\_source['f1']). It will be slower than using document fields, especially  
> if you call this script on a large number of documents, but it will have  
> much lighter memory requirements.
> 
> index.cache.field.\* should be in config/elasticsearch.yml
> 
> On Monday, November 5, 2012 8:15:15 PM UTC-5, Logan wrote:
> 
> > Should I bet setting index.cache.field.\* settings in  
> > bin/service/elasticsearch.conf or config/elasticsearch.yml?
> > 
> > On Monday, November 5, 2012 6:00:22 PM UTC-7, Sushant Shankar wrote:
> > 
> > > Yes we are using several calls to structures like this in a custom  
> > > script, e.g.  
> > > 'doc['f1'] + doc['f2] .. \>= 2',  
> > > often in addition to filter that can have up to 1000 terms.
> > > 
> > > On Monday, November 5, 2012 4:47:19 PM UTC-8, Igor Motov wrote:
> > > 
> > > > Are you using document fields (structures like doc['my\_field']) to  
> > > > access the data?
> > > > 
> > > > On Monday, November 5, 2012 7:34:12 PM UTC-5, Sushant Shankar wrote:
> > > > 
> > > > > Hi Igor,
> > > > > 
> > > > > It is possible that we do not have enough memory for this. The odd  
> > > > > thing is that we're not using facets. We are issuing CPU-intensive custom  
> > > > > script queries (as we need to compute operations like the sum of different  
> > > > > fields).
> > > > > 
> > > > > Thanks,  
> > > > > Sushant
> > > > > 
> > > > > On Monday, November 5, 2012 4:19:38 PM UTC-8, Igor Motov wrote:
> > > > > 
> > > > > > The index.cache.field.max\_size and index.cache.field.expire settings  
> > > > > > are applicable only to the "resident" cache type. The "soft" cache type is  
> > > > > > garbage collected in response to memory demand. If all heap is getting used  
> > > > > > even with soft cache it might be a good indication that you simply don't  
> > > > > > have enough memory on the nodes to perform these queries. What type of  
> > > > > > queries are causing OOM? Are you doing any faceted searches on multivalued  
> > > > > > fields?
> > > > > > 
> > > > > > On Monday, November 5, 2012 5:58:44 PM UTC-5, Logan wrote:
> > > > > > 
> > > > > > > I'm having problems controlling OOME errors on my 6 node CentOS 5.6  
> > > > > > > cluster. I am currently running elasticsearch-0.19.8 using the service  
> > > > > > > wrapper and java version "1.6.0\_25". I have set the following  
> > > > > > > index.cache.field settings in bin/service/elasticsearch.conf and when that  
> > > > > > > failed config/elasticsearch.yml but they seem to be ignored as I never see  
> > > > > > > any field cache evictions in bigdesk and field cache will eventually eat up  
> > > > > > > all the heap memory when certain searches are performed.
> > > > > > > 
> > > > > > > index.cache.field.max\_size: 1000  
> > > > > > > index.cache.field.expire: 5m  
> > > > > > > index.cache.field.type: soft
> > > > > > > 
> > > > > > > I'm not entirely sure that filed.type: soft isn't working as  
> > > > > > > sometimes the field cache will drop to zero after a GC but it seems to only  
> > > > > > > work when the cluster is idle. But field.expire: and  
> > > > > > > max\_size: definitely seem to have no effect.
> > > > > > > 
> > > > > > > Am I going about this the right way? What config should I be setting  
> > > > > > > these values in? Is there a good way to verify what settings are currently  
> > > > > > > in effect on the cluster?
> > > > > > > 
> > > > > > > Logan

--

---

<div class="post-metadata">

**Author:** ![Sushant\_Shankar](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/sushant_shankar/32/2652_2.png) [@Sushant\_Shankar](https://discuss.elastic.co/u/Sushant_Shankar)\
**Post date:** [November 6, 2012, 4:49pm UTC](https://discuss.elastic.co/t/field-cache-limits-ignored/9569/9 "2012-11-06T16:49:59Z")

</div>

Yes, we tried \_field and \_source, but went with doc as it was faster.  
Thanks for pointing out the increased memory requirements on doc accesses.

On Monday, November 5, 2012 5:22:51 PM UTC-8, Igor Motov wrote:

> When elasticsearch sees the call for doc['f1'] it caches all values for  
> this field to improve performance of the future calls. If you don't have  
> enough memory to cache all these values, you might want to consider  
> replacing document fields with stored fields (\_field['f1']) or even source  
> (\_source['f1']). It will be slower than using document fields, especially  
> if you call this script on a large number of documents, but it will have  
> much lighter memory requirements.
> 
> index.cache.field.\* should be in config/elasticsearch.yml
> 
> On Monday, November 5, 2012 8:15:15 PM UTC-5, Logan wrote:
> 
> > Should I bet setting index.cache.field.\* settings in  
> > bin/service/elasticsearch.conf or config/elasticsearch.yml?
> > 
> > On Monday, November 5, 2012 6:00:22 PM UTC-7, Sushant Shankar wrote:
> > 
> > > Yes we are using several calls to structures like this in a custom  
> > > script, e.g.  
> > > 'doc['f1'] + doc['f2] .. \>= 2',  
> > > often in addition to filter that can have up to 1000 terms.
> > > 
> > > On Monday, November 5, 2012 4:47:19 PM UTC-8, Igor Motov wrote:
> > > 
> > > > Are you using document fields (structures like doc['my\_field']) to  
> > > > access the data?
> > > > 
> > > > On Monday, November 5, 2012 7:34:12 PM UTC-5, Sushant Shankar wrote:
> > > > 
> > > > > Hi Igor,
> > > > > 
> > > > > It is possible that we do not have enough memory for this. The odd  
> > > > > thing is that we're not using facets. We are issuing CPU-intensive custom  
> > > > > script queries (as we need to compute operations like the sum of different  
> > > > > fields).
> > > > > 
> > > > > Thanks,  
> > > > > Sushant
> > > > > 
> > > > > On Monday, November 5, 2012 4:19:38 PM UTC-8, Igor Motov wrote:
> > > > > 
> > > > > > The index.cache.field.max\_size and index.cache.field.expire settings  
> > > > > > are applicable only to the "resident" cache type. The "soft" cache type is  
> > > > > > garbage collected in response to memory demand. If all heap is getting used  
> > > > > > even with soft cache it might be a good indication that you simply don't  
> > > > > > have enough memory on the nodes to perform these queries. What type of  
> > > > > > queries are causing OOM? Are you doing any faceted searches on multivalued  
> > > > > > fields?
> > > > > > 
> > > > > > On Monday, November 5, 2012 5:58:44 PM UTC-5, Logan wrote:
> > > > > > 
> > > > > > > I'm having problems controlling OOME errors on my 6 node CentOS 5.6  
> > > > > > > cluster. I am currently running elasticsearch-0.19.8 using the service  
> > > > > > > wrapper and java version "1.6.0\_25". I have set the following  
> > > > > > > index.cache.field settings in bin/service/elasticsearch.conf and when that  
> > > > > > > failed config/elasticsearch.yml but they seem to be ignored as I never see  
> > > > > > > any field cache evictions in bigdesk and field cache will eventually eat up  
> > > > > > > all the heap memory when certain searches are performed.
> > > > > > > 
> > > > > > > index.cache.field.max\_size: 1000  
> > > > > > > index.cache.field.expire: 5m  
> > > > > > > index.cache.field.type: soft
> > > > > > > 
> > > > > > > I'm not entirely sure that filed.type: soft isn't working as  
> > > > > > > sometimes the field cache will drop to zero after a GC but it seems to only  
> > > > > > > work when the cluster is idle. But field.expire: and  
> > > > > > > max\_size: definitely seem to have no effect.
> > > > > > > 
> > > > > > > Am I going about this the right way? What config should I be setting  
> > > > > > > these values in? Is there a good way to verify what settings are currently  
> > > > > > > in effect on the cluster?
> > > > > > > 
> > > > > > > Logan

--

---

<div class="post-metadata">

**Author:** ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)\
**Post date:** [July 6, 2017, 3:05am UTC](https://discuss.elastic.co/t/field-cache-limits-ignored/9569/10 "2017-07-06T03:05:56Z")

</div>


