Optimize elasticsearch / JVM

Elastic Stack Elasticsearch
2

How much heap are you giving to ES? With this many requests, if your setup
is not falling over it is probably not garbage collect related because that
would result in very noticable delays/unavailability of es. 32GB should be
a good value given how much memory you have. Also, you probably want to use
doc_values in your mapping so that you can utilize the os file cache and
move some of the memory pressure on the heap. You seem to have plenty of
ram, so your entire dataset should easily fit in RAM.

Also, don't use G1 for elasticsearch. There are known issues with that
particular garbage collector in combination with lucene. CMS is the best
option for ES.

500% of 20 cores doesn't sound that bad; you'd max them out at 4000%.
Still, it would be nice to know what it is doing. In any case, you might
want to try out marvel to find out where your setup is bottlenecked. Also,
you might want to consider scaling horizontally instead of vertically. Many
smaller servers can be nicer than one big one.

On Wednesday, January 28, 2015 at 2:09:22 PM UTC+1, Oto Iashvili wrote:

Hi

I have a website for classified. For this I'm using elasticsearch,
postgres and rails on a same ubuntu 14.04 dedicated server, with 256go of
RAM and 20 cores, 40 threads .

I have 10 indexes on elasticsearch, each have default numbers of shardes
(5). They have between 1000 and 400 000 classifieds dependings on which
index.
approximatly 5000 requests per minute, 2/3 making an elasticsearch request.

according to htop, jvm is using around 500% of CPU
I try different options, I reduce number of shardes per index, I also try
to change JAVA_OPTS of followed

#JAVA_OPTS="$JAVA_OPTS -XX:+UseParNewGC"
#JAVA_OPTS="$JAVA_OPTS -XX:+UseConcMarkSweepGC"

#JAVA_OPTS="$JAVA_OPTS -XX:CMSInitiatingOccupancyFraction=75"
#JAVA_OPTS="$JAVA_OPTS -XX:+UseCMSInitiatingOccupancyOnly"

JAVA_OPTS="$JAVA_OPTS -XX:+UseG1GC"

but it doesnt seems to change anything.

so to questions :

  • when you change any setting on elasticsearch, and then restart, should
    the improvement (if any) be visible immediatly or can it arrive a bit later
    thanks to cache or any thing else ?
  • can any one help me to find good configuration for JVM / elasticsearch
    so it will not take that many ressources

--
You received this message because you are subscribed to the Google Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email to elasticsearch+unsubscribe@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/elasticsearch/49d08b16-276c-4639-8c1b-26cb5d74530f%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.