# Unnecessary Cache Eviction Explained

**URL:** <https://discuss.elastic.co/t/unnecessary-cache-eviction-explained/19915>\
**Category:** Elasticsearch\
**Created:** [September 22, 2014, 2:30pm UTC](https://discuss.elastic.co/t/unnecessary-cache-eviction-explained/19915 "2014-09-22T14:30:44Z")\
**Posts on this page:** 11\
**Page:** 1

<div class="post-metadata">

**Author:** ![Philippe\_Laflamme](https://avatars.discourse-cdn.com/v4/letter/p/7ab992/32.png) [@Philippe\_Laflamme](https://discuss.elastic.co/u/Philippe_Laflamme)\
**Post date:** [September 22, 2014, 2:30pm UTC](https://discuss.elastic.co/t/unnecessary-cache-eviction-explained/19915/1 "2014-09-22T14:30:44Z")

</div>

Hi,

I've recently posted a question regarding mysterious field data cache  
eviction[1]: ES is evicting field data cache entries even though it's  
nowhere near the limit I've set.

In an unrelated post, someone found the root cause of this problem[2]: the  
maximum cache size specified is actually split evenly between Guava's cache  
partitions. ES configures Guava to use 16 partitions[3], which means that  
any given field data inserted will actually have a maximum size of  
indices.fielddata.cache.size / 16

In my case, I configured the cache size to be 10GB, but saw eviction at  
very low cache usage (1.5GB in some cases). This is because at least one of  
the cache partitions hit its maximum size of 625MB.

Obviously, the short-term solution is to increase the field data cache  
size, but this will require that we overcommit by quite a bit in order to  
have partitions with a sensible size for our most frequent queries.

Until Guava provides a way to have a global maximum size instead of a  
per-partition size (as mentioned by Craig in his post), it would be nice to  
have a handle on the number of partitions created for this cache. If I set  
this to 2, for example, I'm still allowing 2 threads to write to this cache  
concurrently without having to overcommit my global field data cache size  
(at least not by much).

Anyone have another idea about how to deal with this?

Cheers,  
Philippe  
[1] [https://groups.google.com/d/msg/elasticsearch/54XzFO44yJI/sd7Fm-WcrPcJ](https://groups.google.com/d/msg/elasticsearch/54XzFO44yJI/sd7Fm-WcrPcJ)  
[2] [https://groups.google.com/d/msg/elasticsearch/42qrpYRJvsU/jhl3UZZG5sQJ](https://groups.google.com/d/msg/elasticsearch/42qrpYRJvsU/jhl3UZZG5sQJ)  
[3] [https://github.com/elasticsearch/elasticsearch/blob/master/src/main/java/org/elasticsearch/indices/fielddata/cache/IndicesFieldDataCache.java#L77](https://github.com/elasticsearch/elasticsearch/blob/master/src/main/java/org/elasticsearch/indices/fielddata/cache/IndicesFieldDataCache.java#L77)

--  
You received this message because you are subscribed to the Google Groups "elasticsearch" group.  
To unsubscribe from this group and stop receiving emails from it, send an email to [elasticsearch+unsubscribe@googlegroups.com](mailto:elasticsearch+unsubscribe@googlegroups.com).  
To view this discussion on the web visit [https://groups.google.com/d/msgid/elasticsearch/87d08d3a-b7e9-4d7a-b7c5-238cfa5e355f%40googlegroups.com](https://groups.google.com/d/msgid/elasticsearch/87d08d3a-b7e9-4d7a-b7c5-238cfa5e355f%40googlegroups.com).  
For more options, visit [https://groups.google.com/d/optout](https://groups.google.com/d/optout).

---

<div class="post-metadata">

**Author:** ![craigwi](https://avatars.discourse-cdn.com/v4/letter/c/cdc98d/32.png) [@craigwi](https://discuss.elastic.co/u/craigwi)\
**Post date:** [September 22, 2014, 4:02pm UTC](https://discuss.elastic.co/t/unnecessary-cache-eviction-explained/19915/2 "2014-09-22T16:02:28Z")

</div>

Hi Philippe,

Over the weekend I coded up a simple change to the Guava size-based  
eviction algorithm to fix this. With my proposed change there are no API  
changes and it works as a drop in replacement in ES. As you probably know  
ES renames and compiles in the Guava libraries so actually deploying a new  
build of Guava requires rebuilding ES.

The approach I took was to allow segments to grow larger than  
maxSegmentWeight such that the total cache size remains below the overall  
maxWeight. Then, if eviction within one segment doesn't reduce the cache  
weight to below maxWeight, I find the largest segment and evict from  
there. I use tryLock() so that if another thread is already using that  
segment, eviction will happen as new values are loaded there. Like I said,  
simple.

Perhaps we can work together to review my change, make improvements on it,  
and get it submitted?

Craig.

On Monday, September 22, 2014 7:30:44 AM UTC-7, Philippe Laflamme wrote:

> Hi,
> 
> I've recently posted a question regarding mysterious field data cache  
> eviction[1]: ES is evicting field data cache entries even though it's  
> nowhere near the limit I've set.
> 
> In an unrelated post, someone found the root cause of this problem[2]: the  
> maximum cache size specified is actually split evenly between Guava's cache  
> partitions. ES configures Guava to use 16 partitions[3], which means that  
> any given field data inserted will actually have a maximum size of  
> indices.fielddata.cache.size / 16
> 
> In my case, I configured the cache size to be 10GB, but saw eviction at  
> very low cache usage (1.5GB in some cases). This is because at least one of  
> the cache partitions hit its maximum size of 625MB.
> 
> Obviously, the short-term solution is to increase the field data cache  
> size, but this will require that we overcommit by quite a bit in order to  
> have partitions with a sensible size for our most frequent queries.
> 
> Until Guava provides a way to have a global maximum size instead of a  
> per-partition size (as mentioned by Craig in his post), it would be nice to  
> have a handle on the number of partitions created for this cache. If I set  
> this to 2, for example, I'm still allowing 2 threads to write to this cache  
> concurrently without having to overcommit my global field data cache size  
> (at least not by much).
> 
> Anyone have another idea about how to deal with this?
> 
> Cheers,  
> Philippe  
> [1] [https://groups.google.com/d/msg/elasticsearch/54XzFO44yJI/sd7Fm-WcrPcJ](https://groups.google.com/d/msg/elasticsearch/54XzFO44yJI/sd7Fm-WcrPcJ)  
> [2] [https://groups.google.com/d/msg/elasticsearch/42qrpYRJvsU/jhl3UZZG5sQJ](https://groups.google.com/d/msg/elasticsearch/42qrpYRJvsU/jhl3UZZG5sQJ)  
> [3]  
> [https://github.com/elasticsearch/elasticsearch/blob/master/src/main/java/org/elasticsearch/indices/fielddata/cache/IndicesFieldDataCache.java#L77](https://github.com/elasticsearch/elasticsearch/blob/master/src/main/java/org/elasticsearch/indices/fielddata/cache/IndicesFieldDataCache.java#L77)

--  
You received this message because you are subscribed to the Google Groups "elasticsearch" group.  
To unsubscribe from this group and stop receiving emails from it, send an email to [elasticsearch+unsubscribe@googlegroups.com](mailto:elasticsearch+unsubscribe@googlegroups.com).  
To view this discussion on the web visit [https://groups.google.com/d/msgid/elasticsearch/56e71573-7b1a-47da-ae07-d8775804f104%40googlegroups.com](https://groups.google.com/d/msgid/elasticsearch/56e71573-7b1a-47da-ae07-d8775804f104%40googlegroups.com).  
For more options, visit [https://groups.google.com/d/optout](https://groups.google.com/d/optout).

---

<div class="post-metadata">

**Author:** ![Felipe\_Hummel](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/felipe_hummel/32/1263_2.png) [@Felipe\_Hummel](https://discuss.elastic.co/u/Felipe_Hummel)\
**Post date:** [September 22, 2014, 5:24pm UTC](https://discuss.elastic.co/t/unnecessary-cache-eviction-explained/19915/3 "2014-09-22T17:24:57Z")

</div>

It seems I had the same  
problem: [Redirecting to Google Groups](https://groups.google.com/forum/#!topic/elasticsearch/2wJG5L9A8cs)

On Monday, September 22, 2014 1:02:29 PM UTC-3, Craig Wittenberg wrote:

> Hi Philippe,
> 
> Over the weekend I coded up a simple change to the Guava size-based  
> eviction algorithm to fix this. With my proposed change there are no API  
> changes and it works as a drop in replacement in ES. As you probably know  
> ES renames and compiles in the Guava libraries so actually deploying a new  
> build of Guava requires rebuilding ES.
> 
> The approach I took was to allow segments to grow larger than  
> maxSegmentWeight such that the total cache size remains below the overall  
> maxWeight. Then, if eviction within one segment doesn't reduce the cache  
> weight to below maxWeight, I find the largest segment and evict from  
> there. I use tryLock() so that if another thread is already using that  
> segment, eviction will happen as new values are loaded there. Like I said,  
> simple.
> 
> Perhaps we can work together to review my change, make improvements on it,  
> and get it submitted?
> 
> Craig.
> 
> On Monday, September 22, 2014 7:30:44 AM UTC-7, Philippe Laflamme wrote:
> 
> > Hi,
> > 
> > I've recently posted a question regarding mysterious field data cache  
> > eviction[1]: ES is evicting field data cache entries even though it's  
> > nowhere near the limit I've set.
> > 
> > In an unrelated post, someone found the root cause of this problem[2]:  
> > the maximum cache size specified is actually split evenly between Guava's  
> > cache partitions. ES configures Guava to use 16 partitions[3], which means  
> > that any given field data inserted will actually have a maximum size of  
> > indices.fielddata.cache.size / 16
> > 
> > In my case, I configured the cache size to be 10GB, but saw eviction at  
> > very low cache usage (1.5GB in some cases). This is because at least one of  
> > the cache partitions hit its maximum size of 625MB.
> > 
> > Obviously, the short-term solution is to increase the field data cache  
> > size, but this will require that we overcommit by quite a bit in order to  
> > have partitions with a sensible size for our most frequent queries.
> > 
> > Until Guava provides a way to have a global maximum size instead of a  
> > per-partition size (as mentioned by Craig in his post), it would be nice to  
> > have a handle on the number of partitions created for this cache. If I set  
> > this to 2, for example, I'm still allowing 2 threads to write to this cache  
> > concurrently without having to overcommit my global field data cache size  
> > (at least not by much).
> > 
> > Anyone have another idea about how to deal with this?
> > 
> > Cheers,  
> > Philippe  
> > [1]  
> > [https://groups.google.com/d/msg/elasticsearch/54XzFO44yJI/sd7Fm-WcrPcJ](https://groups.google.com/d/msg/elasticsearch/54XzFO44yJI/sd7Fm-WcrPcJ)  
> > [2]  
> > [https://groups.google.com/d/msg/elasticsearch/42qrpYRJvsU/jhl3UZZG5sQJ](https://groups.google.com/d/msg/elasticsearch/42qrpYRJvsU/jhl3UZZG5sQJ)  
> > [3]  
> > [https://github.com/elasticsearch/elasticsearch/blob/master/src/main/java/org/elasticsearch/indices/fielddata/cache/IndicesFieldDataCache.java#L77](https://github.com/elasticsearch/elasticsearch/blob/master/src/main/java/org/elasticsearch/indices/fielddata/cache/IndicesFieldDataCache.java#L77)

--  
You received this message because you are subscribed to the Google Groups "elasticsearch" group.  
To unsubscribe from this group and stop receiving emails from it, send an email to [elasticsearch+unsubscribe@googlegroups.com](mailto:elasticsearch+unsubscribe@googlegroups.com).  
To view this discussion on the web visit [https://groups.google.com/d/msgid/elasticsearch/07cfb705-1f13-40d5-b7b2-c5b84e328ddb%40googlegroups.com](https://groups.google.com/d/msgid/elasticsearch/07cfb705-1f13-40d5-b7b2-c5b84e328ddb%40googlegroups.com).  
For more options, visit [https://groups.google.com/d/optout](https://groups.google.com/d/optout).

---

<div class="post-metadata">

**Author:** ![Philippe\_Laflamme](https://avatars.discourse-cdn.com/v4/letter/p/7ab992/32.png) [@Philippe\_Laflamme](https://discuss.elastic.co/u/Philippe_Laflamme)\
**Post date:** [September 22, 2014, 8:43pm UTC](https://discuss.elastic.co/t/unnecessary-cache-eviction-explained/19915/4 "2014-09-22T20:43:55Z")

</div>

That sounds great! I'd be happy to take a look at your change and possibly  
do some testing locally. Is this hosted somewhere?

Cheers,  
Philippe

On Mon, Sep 22, 2014 at 12:02 PM, Craig Wittenberg [craigwi@microsoft.com](mailto:craigwi@microsoft.com)  
wrote:

> Hi Philippe,
> 
> Over the weekend I coded up a simple change to the Guava size-based  
> eviction algorithm to fix this. With my proposed change there are no API  
> changes and it works as a drop in replacement in ES. As you probably know  
> ES renames and compiles in the Guava libraries so actually deploying a new  
> build of Guava requires rebuilding ES.
> 
> The approach I took was to allow segments to grow larger than  
> maxSegmentWeight such that the total cache size remains below the overall  
> maxWeight. Then, if eviction within one segment doesn't reduce the cache  
> weight to below maxWeight, I find the largest segment and evict from  
> there. I use tryLock() so that if another thread is already using that  
> segment, eviction will happen as new values are loaded there. Like I said,  
> simple.
> 
> Perhaps we can work together to review my change, make improvements on it,  
> and get it submitted?
> 
> Craig.
> 
> On Monday, September 22, 2014 7:30:44 AM UTC-7, Philippe Laflamme wrote:
> 
> > Hi,
> > 
> > I've recently posted a question regarding mysterious field data cache  
> > eviction[1]: ES is evicting field data cache entries even though it's  
> > nowhere near the limit I've set.
> > 
> > In an unrelated post, someone found the root cause of this problem[2]:  
> > the maximum cache size specified is actually split evenly between Guava's  
> > cache partitions. ES configures Guava to use 16 partitions[3], which means  
> > that any given field data inserted will actually have a maximum size of  
> > indices.fielddata.cache.size / 16
> > 
> > In my case, I configured the cache size to be 10GB, but saw eviction at  
> > very low cache usage (1.5GB in some cases). This is because at least one of  
> > the cache partitions hit its maximum size of 625MB.
> > 
> > Obviously, the short-term solution is to increase the field data cache  
> > size, but this will require that we overcommit by quite a bit in order to  
> > have partitions with a sensible size for our most frequent queries.
> > 
> > Until Guava provides a way to have a global maximum size instead of a  
> > per-partition size (as mentioned by Craig in his post), it would be nice to  
> > have a handle on the number of partitions created for this cache. If I set  
> > this to 2, for example, I'm still allowing 2 threads to write to this cache  
> > concurrently without having to overcommit my global field data cache size  
> > (at least not by much).
> > 
> > Anyone have another idea about how to deal with this?
> > 
> > Cheers,  
> > Philippe  
> > [1] [https://groups.google.com/d/msg/elasticsearch/54XzFO44yJI/](https://groups.google.com/d/msg/elasticsearch/54XzFO44yJI/)  
> > sd7Fm-WcrPcJ  
> > [2] [https://groups.google.com/d/msg/elasticsearch/42qrpYRJvsU/](https://groups.google.com/d/msg/elasticsearch/42qrpYRJvsU/)  
> > jhl3UZZG5sQJ  
> > [3] [GitHub - elastic/elasticsearch: Free and Open, Distributed, RESTful Search Engine](https://github.com/elasticsearch/elasticsearch/)  
> > blob/master/src/main/java/org/elasticsearch/indices/fielddata/cache/  
> > IndicesFieldDataCache.java#L77
> > 
> > --  
> > You received this message because you are subscribed to the Google Groups  
> > "elasticsearch" group.  
> > To unsubscribe from this group and stop receiving emails from it, send an  
> > email to [elasticsearch+unsubscribe@googlegroups.com](mailto:elasticsearch+unsubscribe@googlegroups.com).  
> > To view this discussion on the web visit  
> > [https://groups.google.com/d/msgid/elasticsearch/56e71573-7b1a-47da-ae07-d8775804f104%40googlegroups.com](https://groups.google.com/d/msgid/elasticsearch/56e71573-7b1a-47da-ae07-d8775804f104%40googlegroups.com)  
> > [https://groups.google.com/d/msgid/elasticsearch/56e71573-7b1a-47da-ae07-d8775804f104%40googlegroups.com?utm\_medium=email&utm\_source=footer](https://groups.google.com/d/msgid/elasticsearch/56e71573-7b1a-47da-ae07-d8775804f104%40googlegroups.com?utm_medium=email&utm_source=footer)  
> > .
> 
> For more options, visit [https://groups.google.com/d/optout](https://groups.google.com/d/optout).

--  
You received this message because you are subscribed to the Google Groups "elasticsearch" group.  
To unsubscribe from this group and stop receiving emails from it, send an email to [elasticsearch+unsubscribe@googlegroups.com](mailto:elasticsearch+unsubscribe@googlegroups.com).  
To view this discussion on the web visit [https://groups.google.com/d/msgid/elasticsearch/CAKixPJJkTn\_FLK5yoOGc%2B1FsRVQrvav\_U7VDpsxXVzjVgKkonw%40mail.gmail.com](https://groups.google.com/d/msgid/elasticsearch/CAKixPJJkTn_FLK5yoOGc%2B1FsRVQrvav_U7VDpsxXVzjVgKkonw%40mail.gmail.com).  
For more options, visit [https://groups.google.com/d/optout](https://groups.google.com/d/optout).

---

<div class="post-metadata">

**Author:** ![Philippe\_Laflamme](https://avatars.discourse-cdn.com/v4/letter/p/7ab992/32.png) [@Philippe\_Laflamme](https://discuss.elastic.co/u/Philippe_Laflamme)\
**Post date:** [September 22, 2014, 8:44pm UTC](https://discuss.elastic.co/t/unnecessary-cache-eviction-explained/19915/5 "2014-09-22T20:44:10Z")

</div>

Yes, that sounds exactly like the same problem.

On Mon, Sep 22, 2014 at 1:24 PM, Felipe Hummel [felipehummel@gmail.com](mailto:felipehummel@gmail.com)  
wrote:

> It seems I had the same problem:  
> [Redirecting to Google Groups](https://groups.google.com/forum/#!topic/elasticsearch/2wJG5L9A8cs)
> 
> On Monday, September 22, 2014 1:02:29 PM UTC-3, Craig Wittenberg wrote:
> 
> > Hi Philippe,
> > 
> > Over the weekend I coded up a simple change to the Guava size-based  
> > eviction algorithm to fix this. With my proposed change there are no API  
> > changes and it works as a drop in replacement in ES. As you probably know  
> > ES renames and compiles in the Guava libraries so actually deploying a new  
> > build of Guava requires rebuilding ES.
> > 
> > The approach I took was to allow segments to grow larger than  
> > maxSegmentWeight such that the total cache size remains below the overall  
> > maxWeight. Then, if eviction within one segment doesn't reduce the cache  
> > weight to below maxWeight, I find the largest segment and evict from  
> > there. I use tryLock() so that if another thread is already using that  
> > segment, eviction will happen as new values are loaded there. Like I said,  
> > simple.
> > 
> > Perhaps we can work together to review my change, make improvements on  
> > it, and get it submitted?
> > 
> > Craig.
> > 
> > On Monday, September 22, 2014 7:30:44 AM UTC-7, Philippe Laflamme wrote:
> > 
> > > Hi,
> > > 
> > > I've recently posted a question regarding mysterious field data cache  
> > > eviction[1]: ES is evicting field data cache entries even though it's  
> > > nowhere near the limit I've set.
> > > 
> > > In an unrelated post, someone found the root cause of this problem[2]:  
> > > the maximum cache size specified is actually split evenly between Guava's  
> > > cache partitions. ES configures Guava to use 16 partitions[3], which means  
> > > that any given field data inserted will actually have a maximum size of  
> > > indices.fielddata.cache.size / 16
> > > 
> > > In my case, I configured the cache size to be 10GB, but saw eviction at  
> > > very low cache usage (1.5GB in some cases). This is because at least one of  
> > > the cache partitions hit its maximum size of 625MB.
> > > 
> > > Obviously, the short-term solution is to increase the field data cache  
> > > size, but this will require that we overcommit by quite a bit in order to  
> > > have partitions with a sensible size for our most frequent queries.
> > > 
> > > Until Guava provides a way to have a global maximum size instead of a  
> > > per-partition size (as mentioned by Craig in his post), it would be nice to  
> > > have a handle on the number of partitions created for this cache. If I set  
> > > this to 2, for example, I'm still allowing 2 threads to write to this cache  
> > > concurrently without having to overcommit my global field data cache size  
> > > (at least not by much).
> > > 
> > > Anyone have another idea about how to deal with this?
> > > 
> > > Cheers,  
> > > Philippe  
> > > [1] [https://groups.google.com/d/msg/elasticsearch/54XzFO44yJI/](https://groups.google.com/d/msg/elasticsearch/54XzFO44yJI/)  
> > > sd7Fm-WcrPcJ  
> > > [2] [https://groups.google.com/d/msg/elasticsearch/42qrpYRJvsU/](https://groups.google.com/d/msg/elasticsearch/42qrpYRJvsU/)  
> > > jhl3UZZG5sQJ  
> > > [3] [GitHub - elastic/elasticsearch: Free and Open, Distributed, RESTful Search Engine](https://github.com/elasticsearch/elasticsearch/)  
> > > blob/master/src/main/java/org/elasticsearch/indices/fielddata/cache/  
> > > IndicesFieldDataCache.java#L77
> > > 
> > > --  
> > > You received this message because you are subscribed to the Google Groups  
> > > "elasticsearch" group.  
> > > To unsubscribe from this group and stop receiving emails from it, send an  
> > > email to [elasticsearch+unsubscribe@googlegroups.com](mailto:elasticsearch+unsubscribe@googlegroups.com).  
> > > To view this discussion on the web visit  
> > > [https://groups.google.com/d/msgid/elasticsearch/07cfb705-1f13-40d5-b7b2-c5b84e328ddb%40googlegroups.com](https://groups.google.com/d/msgid/elasticsearch/07cfb705-1f13-40d5-b7b2-c5b84e328ddb%40googlegroups.com)  
> > > [https://groups.google.com/d/msgid/elasticsearch/07cfb705-1f13-40d5-b7b2-c5b84e328ddb%40googlegroups.com?utm\_medium=email&utm\_source=footer](https://groups.google.com/d/msgid/elasticsearch/07cfb705-1f13-40d5-b7b2-c5b84e328ddb%40googlegroups.com?utm_medium=email&utm_source=footer)  
> > > .
> 
> For more options, visit [https://groups.google.com/d/optout](https://groups.google.com/d/optout).

--  
You received this message because you are subscribed to the Google Groups "elasticsearch" group.  
To unsubscribe from this group and stop receiving emails from it, send an email to [elasticsearch+unsubscribe@googlegroups.com](mailto:elasticsearch+unsubscribe@googlegroups.com).  
To view this discussion on the web visit [https://groups.google.com/d/msgid/elasticsearch/CAKixPJJ2aVS8mXrUfk%3D7k%3Do7wNahPm60cwFPCdqEZt3DcWqkHw%40mail.gmail.com](https://groups.google.com/d/msgid/elasticsearch/CAKixPJJ2aVS8mXrUfk%3D7k%3Do7wNahPm60cwFPCdqEZt3DcWqkHw%40mail.gmail.com).  
For more options, visit [https://groups.google.com/d/optout](https://groups.google.com/d/optout).

---

<div class="post-metadata">

**Author:** ![craigwi](https://avatars.discourse-cdn.com/v4/letter/c/cdc98d/32.png) [@craigwi](https://discuss.elastic.co/u/craigwi)\
**Post date:** [September 22, 2014, 9:27pm UTC](https://discuss.elastic.co/t/unnecessary-cache-eviction-explained/19915/6 "2014-09-22T21:27:56Z")

</div>

You can find my proposed changes at [https://code.google.com/r/craigwi-guava/](https://code.google.com/r/craigwi-guava/).  
Comments welcome.

I'm having trouble compiling the tests and so haven't run them yet.

Craig.

--  
You received this message because you are subscribed to the Google Groups "elasticsearch" group.  
To unsubscribe from this group and stop receiving emails from it, send an email to [elasticsearch+unsubscribe@googlegroups.com](mailto:elasticsearch+unsubscribe@googlegroups.com).  
To view this discussion on the web visit [https://groups.google.com/d/msgid/elasticsearch/0f0bca16-7c75-4f25-91ac-dc512f63e12e%40googlegroups.com](https://groups.google.com/d/msgid/elasticsearch/0f0bca16-7c75-4f25-91ac-dc512f63e12e%40googlegroups.com).  
For more options, visit [https://groups.google.com/d/optout](https://groups.google.com/d/optout).

---

<div class="post-metadata">

**Author:** ![otisg](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/otisg/32/492_2.png) [@otisg](https://discuss.elastic.co/u/otisg)\
**Post date:** [September 23, 2014, 4:27am UTC](https://discuss.elastic.co/t/unnecessary-cache-eviction-explained/19915/7 "2014-09-23T04:27:47Z")

</div>

Hi,

It sounds like every single ES deployment out there suffers from this, or  
am I missing something? Is there an ES issue where this could be tracked  
(even if the problem in in Guava)?

## Thanks, Otis

Monitoring \* Alerting \* Anomaly Detection \* Centralized Log Management  
Solr & Elasticsearch Support \* [http://sematext.com/](http://sematext.com/)

On Monday, September 22, 2014 5:27:56 PM UTC-4, Craig Wittenberg wrote:

> You can find my proposed changes at  
> [Google Code Archive - Long-term storage for Google Code Project Hosting.](https://code.google.com/r/craigwi-guava/). Comments welcome.
> 
> I'm having trouble compiling the tests and so haven't run them yet.
> 
> Craig.

--  
You received this message because you are subscribed to the Google Groups "elasticsearch" group.  
To unsubscribe from this group and stop receiving emails from it, send an email to [elasticsearch+unsubscribe@googlegroups.com](mailto:elasticsearch+unsubscribe@googlegroups.com).  
To view this discussion on the web visit [https://groups.google.com/d/msgid/elasticsearch/b941d15b-1904-40b1-883b-063995e7bbfb%40googlegroups.com](https://groups.google.com/d/msgid/elasticsearch/b941d15b-1904-40b1-883b-063995e7bbfb%40googlegroups.com).  
For more options, visit [https://groups.google.com/d/optout](https://groups.google.com/d/optout).

---

<div class="post-metadata">

**Author:** ![craigwi](https://avatars.discourse-cdn.com/v4/letter/c/cdc98d/32.png) [@craigwi](https://discuss.elastic.co/u/craigwi)\
**Post date:** [September 23, 2014, 1:56pm UTC](https://discuss.elastic.co/t/unnecessary-cache-eviction-explained/19915/8 "2014-09-23T13:56:30Z")

</div>

I opened issue #7836:  
[Fix unnecessary cache evictions when setting indices.fielddata.cache.size · Issue #7836 · elastic/elasticsearch · GitHub](https://github.com/elasticsearch/elasticsearch/issues/7836).

On Monday, September 22, 2014 9:27:47 PM UTC-7, Otis Gospodnetic wrote:

> Hi,
> 
> It sounds like every single ES deployment out there suffers from this, or  
> am I missing something? Is there an ES issue where this could be tracked  
> (even if the problem in in Guava)?
> 
> ## Thanks, Otis
> 
> Monitoring \* Alerting \* Anomaly Detection \* Centralized Log Management  
> Solr & Elasticsearch Support \* [http://sematext.com/](http://sematext.com/)
> 
> On Monday, September 22, 2014 5:27:56 PM UTC-4, Craig Wittenberg wrote:
> 
> > You can find my proposed changes at  
> > [Google Code Archive - Long-term storage for Google Code Project Hosting.](https://code.google.com/r/craigwi-guava/). Comments welcome.
> > 
> > I'm having trouble compiling the tests and so haven't run them yet.
> > 
> > Craig.

--  
You received this message because you are subscribed to the Google Groups "elasticsearch" group.  
To unsubscribe from this group and stop receiving emails from it, send an email to [elasticsearch+unsubscribe@googlegroups.com](mailto:elasticsearch+unsubscribe@googlegroups.com).  
To view this discussion on the web visit [https://groups.google.com/d/msgid/elasticsearch/69532927-3b07-41cd-9734-3c50fe10ced4%40googlegroups.com](https://groups.google.com/d/msgid/elasticsearch/69532927-3b07-41cd-9734-3c50fe10ced4%40googlegroups.com).  
For more options, visit [https://groups.google.com/d/optout](https://groups.google.com/d/optout).

---

<div class="post-metadata">

**Author:** ![Ivan](https://avatars.discourse-cdn.com/v4/letter/i/df788c/32.png) [@Ivan](https://discuss.elastic.co/u/Ivan)\
**Post date:** [September 23, 2014, 3:00pm UTC](https://discuss.elastic.co/t/unnecessary-cache-eviction-explained/19915/9 "2014-09-23T15:00:27Z")

</div>

Otis, from what I understand, the default size for the cache is unbounded,  
so cache eviction should not occur due to inconsistent range checks in the  
default case.

--  
Ivan

On Mon, Sep 22, 2014 at 9:27 PM, Otis Gospodnetic \<  
[otis.gospodnetic@gmail.com](mailto:otis.gospodnetic@gmail.com)\> wrote:

> Hi,
> 
> It sounds like every single ES deployment out there suffers from this, or  
> am I missing something? Is there an ES issue where this could be tracked  
> (even if the problem in in Guava)?
> 
> ## Thanks, Otis
> 
> Monitoring \* Alerting \* Anomaly Detection \* Centralized Log Management  
> Solr & Elasticsearch Support \* [http://sematext.com/](http://sematext.com/)
> 
> On Monday, September 22, 2014 5:27:56 PM UTC-4, Craig Wittenberg wrote:
> 
> > You can find my proposed changes at [Google Code Archive - Long-term storage for Google Code Project Hosting.](https://code.google.com/r/)  
> > craigwi-guava/. Comments welcome.
> > 
> > I'm having trouble compiling the tests and so haven't run them yet.
> > 
> > Craig.
> 
> --  
> You received this message because you are subscribed to the Google Groups  
> "elasticsearch" group.  
> To unsubscribe from this group and stop receiving emails from it, send an  
> email to [elasticsearch+unsubscribe@googlegroups.com](mailto:elasticsearch+unsubscribe@googlegroups.com).  
> To view this discussion on the web visit  
> [https://groups.google.com/d/msgid/elasticsearch/b941d15b-1904-40b1-883b-063995e7bbfb%40googlegroups.com](https://groups.google.com/d/msgid/elasticsearch/b941d15b-1904-40b1-883b-063995e7bbfb%40googlegroups.com)  
> [https://groups.google.com/d/msgid/elasticsearch/b941d15b-1904-40b1-883b-063995e7bbfb%40googlegroups.com?utm\_medium=email&utm\_source=footer](https://groups.google.com/d/msgid/elasticsearch/b941d15b-1904-40b1-883b-063995e7bbfb%40googlegroups.com?utm_medium=email&utm_source=footer)  
> .
> 
> For more options, visit [https://groups.google.com/d/optout](https://groups.google.com/d/optout).

--  
You received this message because you are subscribed to the Google Groups "elasticsearch" group.  
To unsubscribe from this group and stop receiving emails from it, send an email to [elasticsearch+unsubscribe@googlegroups.com](mailto:elasticsearch+unsubscribe@googlegroups.com).  
To view this discussion on the web visit [https://groups.google.com/d/msgid/elasticsearch/CALY%3DcQCbmnsUXY5HMkqzMBw\_fC20EnQwe9qgftROvHsN%2B881GA%40mail.gmail.com](https://groups.google.com/d/msgid/elasticsearch/CALY%3DcQCbmnsUXY5HMkqzMBw_fC20EnQwe9qgftROvHsN%2B881GA%40mail.gmail.com).  
For more options, visit [https://groups.google.com/d/optout](https://groups.google.com/d/optout).

---

<div class="post-metadata">

**Author:** ![Philippe\_Laflamme](https://avatars.discourse-cdn.com/v4/letter/p/7ab992/32.png) [@Philippe\_Laflamme](https://discuss.elastic.co/u/Philippe_Laflamme)\
**Post date:** [September 23, 2014, 4:22pm UTC](https://discuss.elastic.co/t/unnecessary-cache-eviction-explained/19915/10 "2014-09-23T16:22:15Z")

</div>

Indeed, only instances with a value (greater than 0) specified for  
indices.fielddata.cache.size are affected. This is what triggers the use of  
Guava's eviction-based-on-size feature[1]

Philippe  
[1]  
[https://github.com/elasticsearch/elasticsearch/blob/master/src/main/java/org/elasticsearch/indices/fielddata/cache/IndicesFieldDataCache.java#L74](https://github.com/elasticsearch/elasticsearch/blob/master/src/main/java/org/elasticsearch/indices/fielddata/cache/IndicesFieldDataCache.java#L74)

On Tue, Sep 23, 2014 at 11:00 AM, Ivan Brusic [ivan@brusic.com](mailto:ivan@brusic.com) wrote:

> Otis, from what I understand, the default size for the cache is unbounded,  
> so cache eviction should not occur due to inconsistent range checks in the  
> default case.
> 
> --  
> Ivan
> 
> On Mon, Sep 22, 2014 at 9:27 PM, Otis Gospodnetic \<  
> [otis.gospodnetic@gmail.com](mailto:otis.gospodnetic@gmail.com)\> wrote:
> 
> > Hi,
> > 
> > It sounds like every single ES deployment out there suffers from this, or  
> > am I missing something? Is there an ES issue where this could be tracked  
> > (even if the problem in in Guava)?
> > 
> > ## Thanks, Otis
> > 
> > Monitoring \* Alerting \* Anomaly Detection \* Centralized Log Management  
> > Solr & Elasticsearch Support \* [http://sematext.com/](http://sematext.com/)
> > 
> > On Monday, September 22, 2014 5:27:56 PM UTC-4, Craig Wittenberg wrote:
> > 
> > > You can find my proposed changes at [Google Code Archive - Long-term storage for Google Code Project Hosting.](https://code.google.com/r/)  
> > > craigwi-guava/. Comments welcome.
> > > 
> > > I'm having trouble compiling the tests and so haven't run them yet.
> > > 
> > > Craig.
> > 
> > --  
> > You received this message because you are subscribed to the Google Groups  
> > "elasticsearch" group.  
> > To unsubscribe from this group and stop receiving emails from it, send an  
> > email to [elasticsearch+unsubscribe@googlegroups.com](mailto:elasticsearch+unsubscribe@googlegroups.com).  
> > To view this discussion on the web visit  
> > [https://groups.google.com/d/msgid/elasticsearch/b941d15b-1904-40b1-883b-063995e7bbfb%40googlegroups.com](https://groups.google.com/d/msgid/elasticsearch/b941d15b-1904-40b1-883b-063995e7bbfb%40googlegroups.com)  
> > [https://groups.google.com/d/msgid/elasticsearch/b941d15b-1904-40b1-883b-063995e7bbfb%40googlegroups.com?utm\_medium=email&utm\_source=footer](https://groups.google.com/d/msgid/elasticsearch/b941d15b-1904-40b1-883b-063995e7bbfb%40googlegroups.com?utm_medium=email&utm_source=footer)  
> > .
> > 
> > For more options, visit [https://groups.google.com/d/optout](https://groups.google.com/d/optout).
> 
> --  
> You received this message because you are subscribed to the Google Groups  
> "elasticsearch" group.  
> To unsubscribe from this group and stop receiving emails from it, send an  
> email to [elasticsearch+unsubscribe@googlegroups.com](mailto:elasticsearch+unsubscribe@googlegroups.com).  
> To view this discussion on the web visit  
> [https://groups.google.com/d/msgid/elasticsearch/CALY%3DcQCbmnsUXY5HMkqzMBw\_fC20EnQwe9qgftROvHsN%2B881GA%40mail.gmail.com](https://groups.google.com/d/msgid/elasticsearch/CALY%3DcQCbmnsUXY5HMkqzMBw_fC20EnQwe9qgftROvHsN%2B881GA%40mail.gmail.com)  
> [https://groups.google.com/d/msgid/elasticsearch/CALY%3DcQCbmnsUXY5HMkqzMBw\_fC20EnQwe9qgftROvHsN%2B881GA%40mail.gmail.com?utm\_medium=email&utm\_source=footer](https://groups.google.com/d/msgid/elasticsearch/CALY%3DcQCbmnsUXY5HMkqzMBw_fC20EnQwe9qgftROvHsN%2B881GA%40mail.gmail.com?utm_medium=email&utm_source=footer)  
> .
> 
> For more options, visit [https://groups.google.com/d/optout](https://groups.google.com/d/optout).

--  
You received this message because you are subscribed to the Google Groups "elasticsearch" group.  
To unsubscribe from this group and stop receiving emails from it, send an email to [elasticsearch+unsubscribe@googlegroups.com](mailto:elasticsearch+unsubscribe@googlegroups.com).  
To view this discussion on the web visit [https://groups.google.com/d/msgid/elasticsearch/CAKixPJ%2BAiriZJKE\_M%2BJ2NqWU3riyPcfAonfCgCLOzX\_mCK%2BJsw%40mail.gmail.com](https://groups.google.com/d/msgid/elasticsearch/CAKixPJ%2BAiriZJKE_M%2BJ2NqWU3riyPcfAonfCgCLOzX_mCK%2BJsw%40mail.gmail.com).  
For more options, visit [https://groups.google.com/d/optout](https://groups.google.com/d/optout).

---

<div class="post-metadata">

**Author:** ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)\
**Post date:** [July 6, 2017, 1:00am UTC](https://discuss.elastic.co/t/unnecessary-cache-eviction-explained/19915/11 "2017-07-06T01:00:24Z")

</div>