Throttling fixed bitset cache

Elastic Stack Elasticsearch
1

Hello,

Is there any way to throttle fixed bitset cache (in 1.7 or at least in 2.x)?
I have a client cluster whose 8 GB heap memory is saturated by fixed bitset cache after five minutes.

The cluster avoid OutOfMemoryException by setting index.load_fixed_bitset_filters_eagerly to false and clearing cache every 5 minutes, which is not a durable situation.

Other caches (cache field and cache filter) are OK.

I know that the principal reason is due to massive usage of nested fields in filters and sort using neasted_filter as in https://github.com/elastic/elasticsearch/issues/10224 and that the best solution is to migrate to Elasticsearch 2.2.

But I would like to be able to throttle this cache. First as a temporary solution, the time to do the migration since the latter requires time and eviction on fixed bitset cache should be better than periodically clearing all caches. Then, because I don't want to prevent node crashes in case migration to 2.2 would not fix the problem.

Thank you.

2

There was a bug that caused this cache to duplicate its content, maybe you are affected? https://github.com/elastic/elasticsearch/pull/15836

3

That might be a cause since their real version is 1.7.3.

By the way I support your idea to read fixed bitset from the disk.
It would be a good idea.

Thank you, Adrien.

4

You're welcome Jérôme!