Hi All, I have a doubt about the heap size in elasticsearch. I have read the definitive guide of elasticsearch. Still I am not clear why it needs so much heap size. The segments(inverted index) are stored in off heap space and Lucene also runs in off heap space. So at the time of aggregation and sorting also it should use the off heap memory right?? But it is written there that for these intensive operations a lot of heap memory is needed. And for in-memory data structure heap memory is needed. What in-memory data structure are these??
The segments(inverted index) are stored in off heap space and Lucene also runs in off heap space.
That's only partially true. This applies only to some of the Lucene data structures (doc values) that can be accessed in that way. Other data structures are loaded into the JVM heap (field data, FSTs, ...)
So at the time of aggregation and sorting also it should use the off heap memory right??
no, that's done on the JVM heap.
Do you have any particular concerns about finding the right heap size for your Elasticsearch instance or is it more of a general question?
1 Like
No it's not a general question. Actually i am configuring Elasticsearch for my organization. There are so many other processes running apart from elasticsearch. So of course i want to save memory as well as i don't want elasticsearch be hindered by any memory shortage issue. If i get to know the exact objective of heap space then it will be easy and reasonable for me to allot heap space at the time of configuration.
So can you please elaborate the usefulness of heap memory in elasticsearch.
Determining the amount of heap that is required by ES for your use case is usually an empirical process. There are so many factors playing a role, depending on the data that is queried, the type of queries performed and the query load. The following talk from Elastic{ON} last year gives an overview on how to approach this.
This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.