Hi,
Adrien, Thanks for your response.
From what I understood, both from your response and the cache document, Is it fair to say that there is very less memory utilized during write ? (i.e no in memory operations, data is directly flushed to the disk) however, searching across several indices or types might involve loading of the indices/types for catering to the search query ?
So, at any given point of the time, the state of the cache is dependent on the previous queries (search) that are made to the cluster. Is that correct ? Can you please confirm ?
Thanks,
Ashwin Sathya
Date: Mon, 9 Sep 2013 23:05:35 +0200
Subject: Re: Types memory and loading/unloading
From: adrien.grand@elasticsearch.com
To: elasticsearch@googlegroups.com
Hi,
There is no document outlining the internals of memory usage, but most of time, memory is mainly used for filter caching and field data. The filter cache size is configurable[1] and although having a big filter cache will improve response times, an Elasticsearch instance with little memory for the filter cache size can work just fine. It is a bit more complicated for field data. Field data is used whenever you need to sort or facet or a field and is very expensive to load (it can take several seconds on large indices), so you usually don't want entries to be evicted and would rather like to cache data for all the segments of your index in main memory. A general recommendation is to not limit its size.
About the types, there is no memory overhead about moving to different types over time, resources used for the old types will be deleted as segments get merged (which happens in the background when you update your index).
[1] http://www.elasticsearch.org/guide/reference/index-modules/cache/
On Mon, Sep 9, 2013 at 6:20 PM, R Ashwin Sathya ashwin.sathya@outlook.com wrote:
Hi,
Is there a document outlining the internals of memory usage ?
My particular question is this. When I am actively writing to one of the types, (I move from one type to next incrementally over time) Does the previous committed types stay in memory (some part of it, like the metadata, etc) Is there a way I can unload to free memory ?
Thanks,
Ashwin Sathya
--
You received this message because you are subscribed to the Google Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email to elasticsearch+unsubscribe@googlegroups.com.
For more options, visit https://groups.google.com/groups/opt_out.
--
Adrien Grand
--
You received this message because you are subscribed to the Google Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email to elasticsearch+unsubscribe@googlegroups.com.
For more options, visit https://groups.google.com/groups/opt_out.
--
You received this message because you are subscribed to the Google Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email to elasticsearch+unsubscribe@googlegroups.com.
For more options, visit https://groups.google.com/groups/opt_out.