High VIRT on ElasticSearch Java Process


(stefano ruggiero) #1

Hi all,

i would like to understand why my ElasticSearch Node is display me that 45
gb of VIRT memory is used even if the system is pretty cool...

as i see if i run some query with kiaban the load averege goes suddenly to
4 or 8 but the physical memory still ok and neither swap is used.

Thanks for all explantion and supports...

https://lh3.googleusercontent.com/-1UteIHXWN_I/UwJAZgC6xMI/AAAAAAAAABQ/4YdhzQ21tbI/s1600/VIRT_Usage.png

--
You received this message because you are subscribed to the Google Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email to elasticsearch+unsubscribe@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/elasticsearch/462ca065-f293-4cde-8545-f98e8764d31a%40googlegroups.com.
For more options, visit https://groups.google.com/groups/opt_out.


(Jörg Prante) #2

You must not be worried about the VIRT column in the top display.

In the VIRT column, there is no real memory diaplsyed, but all requests for
virtual address space which is not allocated is shown.

You can VIRT think of all kind of memory requests like direct buffers the
Java JVM wanted to allocate in their lifetime. Only parts of this address
space is active, it's the RSS column.

Jörg

On Mon, Feb 17, 2014 at 6:02 PM, stefano.secure@gmail.com wrote:

Hi all,

i would like to understand why my ElasticSearch Node is display me that 45
gb of VIRT memory is used even if the system is pretty cool...

as i see if i run some query with kiaban the load averege goes suddenly to
4 or 8 but the physical memory still ok and neither swap is used.

Thanks for all explantion and supports...

https://lh3.googleusercontent.com/-1UteIHXWN_I/UwJAZgC6xMI/AAAAAAAAABQ/4YdhzQ21tbI/s1600/VIRT_Usage.png

--
You received this message because you are subscribed to the Google Groups
"elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an
email to elasticsearch+unsubscribe@googlegroups.com.
To view this discussion on the web visit
https://groups.google.com/d/msgid/elasticsearch/462ca065-f293-4cde-8545-f98e8764d31a%40googlegroups.com
.
For more options, visit https://groups.google.com/groups/opt_out.

--
You received this message because you are subscribed to the Google Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email to elasticsearch+unsubscribe@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/elasticsearch/CAKdsXoHabxjSAbosqYV%3DjJKCD5f6eVO1Er%3DrOp381_ZhDFj3yA%40mail.gmail.com.
For more options, visit https://groups.google.com/groups/opt_out.


(stefano ruggiero) #3

thanks Jorg for the faster anwer.

but iam not able to understand why if i have enought RAM and iam on FC Disk
if i run some query in Kibana i see a very high Load Averege about 8 or 10,
and i perform query on 100 milion documents..

Il giorno lunedì 17 febbraio 2014 18:25:26 UTC+1, Jörg Prante ha scritto:

You must not be worried about the VIRT column in the top display.

In the VIRT column, there is no real memory diaplsyed, but all requests
for virtual address space which is not allocated is shown.

You can VIRT think of all kind of memory requests like direct buffers the
Java JVM wanted to allocate in their lifetime. Only parts of this address
space is active, it's the RSS column.

Jörg

On Mon, Feb 17, 2014 at 6:02 PM, <stefano...@gmail.com <javascript:>>wrote:

Hi all,

i would like to understand why my ElasticSearch Node is display me that
45 gb of VIRT memory is used even if the system is pretty cool...

as i see if i run some query with kiaban the load averege goes suddenly
to 4 or 8 but the physical memory still ok and neither swap is used.

Thanks for all explantion and supports...

https://lh3.googleusercontent.com/-1UteIHXWN_I/UwJAZgC6xMI/AAAAAAAAABQ/4YdhzQ21tbI/s1600/VIRT_Usage.png

--
You received this message because you are subscribed to the Google Groups
"elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an
email to elasticsearc...@googlegroups.com <javascript:>.
To view this discussion on the web visit
https://groups.google.com/d/msgid/elasticsearch/462ca065-f293-4cde-8545-f98e8764d31a%40googlegroups.com
.
For more options, visit https://groups.google.com/groups/opt_out.

--
You received this message because you are subscribed to the Google Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email to elasticsearch+unsubscribe@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/elasticsearch/6b6f8c8f-161a-4eca-b91d-c9a8d6772763%40googlegroups.com.
For more options, visit https://groups.google.com/groups/opt_out.


(Jörg Prante) #4

Load is a different issue, it is related to the number of processes running
and waiting for the CPU.

What about CPU peaks? Is it just a single query that causes CPU peaks? How
long does the query take to complete?

Can you see something in the logs?

Jörg

--
You received this message because you are subscribed to the Google Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email to elasticsearch+unsubscribe@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/elasticsearch/CAKdsXoE2Q9CU8Beu%2BFt5YffxnxX3FqZdBC7B94t5gkPoR%3Dfz6Q%40mail.gmail.com.
For more options, visit https://groups.google.com/groups/opt_out.


(stefano ruggiero) #5

Hi jorg,

let me reproduce the usses and then collects all the information needed for
a better examplantion.

as i saw the simple count query for 10 milion events overload my Load
Average even if my quad core still not at 100% , the query ends in about 10
seconds and i cant find any past errors in logs.

iam running ES 1.0.0 with 4 nodes

Il giorno lunedì 17 febbraio 2014 19:57:02 UTC+1, Jörg Prante ha scritto:

Load is a different issue, it is related to the number of processes
running and waiting for the CPU.

What about CPU peaks? Is it just a single query that causes CPU peaks? How
long does the query take to complete?

Can you see something in the logs?

Jörg

--
You received this message because you are subscribed to the Google Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email to elasticsearch+unsubscribe@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/elasticsearch/d493a00f-fd51-44db-998e-5794b26d0f1e%40googlegroups.com.
For more options, visit https://groups.google.com/groups/opt_out.


(system) #6