Hello, I have 8GB dedicated for heap, 20% heap is dedicated for filters
cache. When clear filter cache from api it clears ~4GB memory from heap
(way more then filters cache). Is it normal, or I have some problems with
cached filters? We are using warmers, but this should not be a reason...
When you cache filters, there are two things that can take memory: the
cached bitsets and the filter itself (which is used as a key). And while we
have APIs to estimate the memory usage for bitsets, we do not have anything
for filters unfortunately. So if you happen to have lots of small segments
on your machine or if you use filters that can have a high memory footprint
(like terms filters with lots of terms) then the memory usage of your
filter cache might be underestimated because of the filters which are not
taken into account.
Hello, I have 8GB dedicated for heap, 20% heap is dedicated for filters
cache. When clear filter cache from api it clears ~4GB memory from heap
(way more then filters cache). Is it normal, or I have some problems with
cached filters? We are using warmers, but this should not be a reason...
Thank you for answer. We had lot's of cache evictions so we increased cache
size. Should I use custom cache keys and try to make it smaller? What
cache keys are generated by default? Is it full filter query dsl string?
2014 m. lapkritis 29 d., šeštadienis 13:01:59 UTC+2, Adrien Grand rašė:
When you cache filters, there are two things that can take memory: the
cached bitsets and the filter itself (which is used as a key). And while we
have APIs to estimate the memory usage for bitsets, we do not have anything
for filters unfortunately. So if you happen to have lots of small segments
on your machine or if you use filters that can have a high memory footprint
(like terms filters with lots of terms) then the memory usage of your
filter cache might be underestimated because of the filters which are not
taken into account.
On Fri, Nov 28, 2014 at 9:58 AM, Audrius Bugas <audriu...@hometogo.com
<javascript:>> wrote:
Hello, I have 8GB dedicated for heap, 20% heap is dedicated for filters
cache. When clear filter cache from api it clears ~4GB memory from heap
(way more then filters cache). Is it normal, or I have some problems with
cached filters? We are using warmers, but this should not be a reason...
Cache keys are pretty much the input filter by default. Specifying your own
keys can certainly help.
On Dec 1, 2014 7:05 AM, "Audrius Bugas" bugas.audrius@gmail.com wrote:
Thank you for answer. We had lot's of cache evictions so we increased
cache size. Should I use custom cache keys and try to make it smaller?
What cache keys are generated by default? Is it full filter query dsl
string?
2014 m. lapkritis 29 d., šeštadienis 13:01:59 UTC+2, Adrien Grand rašė:
When you cache filters, there are two things that can take memory: the
cached bitsets and the filter itself (which is used as a key). And while we
have APIs to estimate the memory usage for bitsets, we do not have anything
for filters unfortunately. So if you happen to have lots of small segments
on your machine or if you use filters that can have a high memory footprint
(like terms filters with lots of terms) then the memory usage of your
filter cache might be underestimated because of the filters which are not
taken into account.
By the way, 20% is a bit high for a filter cache size. Recent versions of
elasticsearch have a default value of 10% (http://www.elasticsearch.org/
guide/en/elasticsearch/reference/current/index-modules-cache.html) which
should already allow for caching lots of caches.
Hello, I have 8GB dedicated for heap, 20% heap is dedicated for filters
cache. When clear filter cache from api it clears ~4GB memory from heap
(way more then filters cache). Is it normal, or I have some problems with
cached filters? We are using warmers, but this should not be a reason...
Apache, Apache Lucene, Apache Hadoop, Hadoop, HDFS and the yellow elephant
logo are trademarks of the
Apache Software Foundation
in the United States and/or other countries.