Memory Optimization

Hi,

I have a doubt. when I start elastic search it's taking memory around 510MB
.now I created around 7,00,000 records in four indexes then it's taking
around 3.7GB memory.
but when i remove all four indexes memory it's not free.(means it does not
come to around 510MB).

My doubt is why Memory is not free after deleting the index.

Please help.

Thanks
Rajesh

--
You received this message because you are subscribed to the Google Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email to elasticsearch+unsubscribe@googlegroups.com.
For more options, visit https://groups.google.com/groups/opt_out.

Hi Raj,

How much memory did you give to the JVM? It's very common for JVMs to not
give back memory as soon as memory is not used anymore. They assume that if
memory was needed at some point, it might be needed again soon, so they
prefer keeping unused memory reserved instead of paying the cost to give
memory back to the OS and then allocate it again a few minutes later.

--
Adrien Grand

--
You received this message because you are subscribed to the Google Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email to elasticsearch+unsubscribe@googlegroups.com.
For more options, visit https://groups.google.com/groups/opt_out.

Hi,
Thanks for reply.
I have given 4GB memory to the JVM.
How can I free memory after deleting index.
because I need that memory to other purpose. and any other way to save ram
space?

Thanks
Rajesh

On Friday, May 24, 2013 12:41:07 PM UTC+5:30, Adrien Grand wrote:

Hi Raj,

How much memory did you give to the JVM? It's very common for JVMs to not
give back memory as soon as memory is not used anymore. They assume that if
memory was needed at some point, it might be needed again soon, so they
prefer keeping unused memory reserved instead of paying the cost to give
memory back to the OS and then allocate it again a few minutes later.

--
Adrien Grand

--
You received this message because you are subscribed to the Google Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email to elasticsearch+unsubscribe@googlegroups.com.
For more options, visit https://groups.google.com/groups/opt_out.

Hi,

On Fri, May 24, 2013 at 11:52 AM, raj rajesh.elasticsearch@gmail.comwrote:

I have given 4GB memory to the JVM.
How can I free memory after deleting index.
because I need that memory to other purpose. and any other way to save ram
space?

Some JVMs such as the Oracle one support options to make the JVM give
memory back to the OS more aggressively, see -XX:MaxHeapFreeRatio and
-XX:MinHeapFreeRatio[1] for example (try lowering the default values).
Please however note that this will bring a performance penalty since the
JVM will need to perform more round-trips to the OS to allocate and release
memory.

[1]
http://www.oracle.com/technetwork/java/javase/tech/vmoptions-jsp-140102.html

--
Adrien Grand

--
You received this message because you are subscribed to the Google Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email to elasticsearch+unsubscribe@googlegroups.com.
For more options, visit https://groups.google.com/groups/opt_out.

Well... how long has the memory been consumed? A GC (Garbage Collection)
will be triggered depending on JVM options, like running out of permgen or
oldgen. There is no harm in memory being consumed up to the limit set in
the jvm options, so I wouldn't worry. If you run out of heap space, then
worry.

On Friday, May 24, 2013 1:55:13 AM UTC-4, raj wrote:

Hi,

I have a doubt. when I start elastic search it's taking memory around
510MB .now I created around 7,00,000 records in four indexes then it's
taking around 3.7GB memory.
but when i remove all four indexes memory it's not free.(means it does not
come to around 510MB).

My doubt is why Memory is not free after deleting the index.

Please help.

Thanks
Rajesh

--
You received this message because you are subscribed to the Google Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email to elasticsearch+unsubscribe@googlegroups.com.
For more options, visit https://groups.google.com/groups/opt_out.

How to you give 4GB to JVM? Please show your JVM options.

You must not use -Xmx and -Xms which is used by elasticsearch start
scripts by default to enable the rare occasion of releasing heap memory
back to OS. The heap is only one part of the memory in use and only the
younger parts of the heap may be released back, which is very rare in
long-running applications like ES.

The default strategy of ES memory is reserving the heap memory at once
at startup time. It is most efficient to operate on already allocated
memory with the vast amount of large objects on the ES heap. If you want
to avoid ES memory strategy, ES performance will become a challenge
because heap memory will immediately allocated again after having it
released back to OS, just to continue other operations. All you will see
is additional burden on the Java VM to handle memory requests.

Note, in the world of inverted indexing, assuming you can save memory by
deleting an index is plain wrong. It's the dictionary (the list of
unique terms in the documents and their frequencies) that dominates the
volume of allocated space. You would have to delete all indexes to
achieve memory saving. And a better method to delete all indexes is just
not using ES.

Jörg

Am 24.05.13 11:52, schrieb raj:

Hi,
Thanks for reply.
I have given 4GB memory to the JVM.
How can I free memory after deleting index.
because I need that memory to other purpose. and any other way to save
ram space?

Thanks
Rajesh

On Friday, May 24, 2013 12:41:07 PM UTC+5:30, Adrien Grand wrote:

Hi Raj,

How much memory did you give to the JVM? It's very common for JVMs
to not give back memory as soon as memory is not used anymore.
They assume that if memory was needed at some point, it might be
needed again soon, so they prefer keeping unused memory reserved
instead of paying the cost to give memory back to the OS and then
allocate it again a few minutes later.

-- 
Adrien Grand

--
You received this message because you are subscribed to the Google
Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send
an email to elasticsearch+unsubscribe@googlegroups.com.
For more options, visit https://groups.google.com/groups/opt_out.

--
You received this message because you are subscribed to the Google Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email to elasticsearch+unsubscribe@googlegroups.com.
For more options, visit https://groups.google.com/groups/opt_out.