Hi list,
I'm having some troubles with prioritization between insertions and
requesting on an Elascticsearch "log" cluster of 6 data nodes and 2 master
data-less. The data nodes are 64GB (JVM has 32GB heap allocated)
and CPU Intel Xeon E5-2620.
The client is doing a request on two indices
(index-2014.04.10 and index-2014.04.11) that have the following
topology :
green index-2014.04.10 10 2 33185941 2099195 121.7gb 40.5gb
green index-2014.04.11 10 2 33552398 2312089 123.8gb 41.2gb
The request is looking at some strings existing in the message field like
message:foobar. The message field as an average length of 1000 chars.
Logstash is the one feeding Elasticsearch with elasticsearch_http
output plugin and 10 worker threads. The cluster inserts at an average
of 1000 events per second (and on peak has been seen inserting at avg 8000
eps).
While I understand the previous request will generate some significant
I/O on the nodes, I would like to be assured that insertion requests
will always be served first even if my search request will have a
somehow "bad" response time.
The direct effect of the insertion drops can be seen from kibana on the
attached file.
Do you have any advices to to set up some QoS on search/insertion requests ?
Cheers,
-- Mathieu
--
You received this message because you are subscribed to the Google Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email to elasticsearch+unsubscribe@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/elasticsearch/c314145e-4507-49f9-b1aa-b1dcb1c67456%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.