Elasticsearch field cache always filled up and prevents other query to run


(Olivier B) #1

Hi all,

I set up an index containing 500.000 with million nested documents and I
run big queries with a lot of aggregation. As expected this is using a LOT
of memory and I easily have an OutOfMemory exception if a user perform a
query for too many documents
So I configured elasticsearch with the folloting:

indices.fielddata.cache.size: 20%
indices.fielddata.breaker.limit: 20%
indices.fielddata.breaker.overhead: 1.2
indices.fielddata.cache.expiry: 1m
indices.cache.filter.size: 20%
indices.cache.filter.expire: 1m

(I've two nodes with one index running on m3.xlarge EC2 instances with 5
shards per node HEAP size is 8G (half of total RAM)).

Now I've well this beautiful error message when a user is performing a big
query:

CircuitBreakingException[Data too large, data would be larger than limit of
[2570007347] bytes

If I check the nodes, the field cache reached its maximum size (1.6G) as
expected. Then I can easily returned an error message to the user telling
him to downsize the scope of its query and it should be fine.

But... This will never let me run a query anymore!

The cache is filled up for ever and all query, even basic small search
(without aggregation) will be responded with the CircuitBreakingException.

The only way I found so far is manually clearing the cache. Of course this
is not an acceptable solution for production.

*How can I make sure this cache will never be continuously filled up? *

How can I automatically remove from the cache unused data?

Does indices.fielddata.cache.expiry config param actually work?

Thanks all!

--
You received this message because you are subscribed to the Google Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email to elasticsearch+unsubscribe@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/elasticsearch/72f68e09-c670-494c-af77-328e37e4aa0f%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.


(system) #2