Hi,
I am currently having trouble with fairly slow and intensive queries
causing excessive load on my elasticsearch cluster and I would like to know
people's opinions on ways to mitigate or prevent that excessive load.
We attempt about 50 of these slow queries per second, and they take an
average of 300ms which adds up to more than we can process,, causing
excessive load and sometimes causing elasticsearch to become non-responsive.
The slow queries are all low priority, and we have other, high priority
queries running on the index. Slow queries could take 10 seconds to process
for all we care, and we'd rather have them fail than cause excessive load
on the cluster.
Is there a way to give these queries a lower priority and to force them to
use no more than a certain percentage of the cluster's resources? Or is it
possible to refuse certain types of queries if elasticsearch is under
excessive load?
I am also curious if people have thoughts on what could improve the
throughput of these queries based on the information given below. I can
give more details about the structure of the queries themselves if
necessary.
The cluster is made of two machines (both have 16 CPU cores and let
elasticsearch use 15G of memory) running ES 0.90.12 with the G1 garbage
collector. I began experiencing higher load with this setup than before
when I upgraded to 0.90.12 from 0.90.0 beta (which was using the CMS
collector with the default settings), however, several other changes were
made at the same time and it isn't yet fully clear whether either the
change of version or the change of GC is responsible, or whether it was
simply a coincidence. Some thoughts on that would be appreciated.
The index has 5 shards and one replica (both machines each have a version
of all shards), is a couple gigs in size and contains a couple million
documents.
Thanks!
--
You received this message because you are subscribed to the Google Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email to elasticsearch+unsubscribe@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/elasticsearch/38e8c947-7dff-4eab-9cc0-b492e20a769e%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.