There are many users working with the application. Some of them are online
users, that should have their requests served in real time and some of them
are workers that are preparing reports and their requests can wait. At the
moment, all the requests arriving to elasticsearch are queued at the same
queue and that sometimes results in online users not getting their request
served as soon as possible or even getting rejected when the queue is full
of workers' requests. That could be solved by adding separate Thread Pool
and Queue for 'low priority search'. Same would apply for other operations
(get, bulk etc).
Are there any plans or any chance for this kind of feature being
implemented in Elasticsearch?
You could set up a special node for "low priority" search, with "slim"
thread pool settings, and forward client calls to it respectively.
Jörg
On Mon, Mar 24, 2014 at 3:08 PM, Paweł Młynarczyk zwarios@gmail.com wrote:
There are many users working with the application. Some of them are online
users, that should have their requests served in real time and some of them
are workers that are preparing reports and their requests can wait. At the
moment, all the requests arriving to elasticsearch are queued at the same
queue and that sometimes results in online users not getting their request
served as soon as possible or even getting rejected when the queue is full
of workers' requests. That could be solved by adding separate Thread Pool
and Queue for 'low priority search'. Same would apply for other operations
(get, bulk etc).
Are there any plans or any chance for this kind of feature being
implemented in Elasticsearch?
I did some tests and it appears that the lowest queue size between the data
nodes is the effective one (as far as I am concerned the search requests
are spread across all nodes with data) and queue sizes in non data nodes
are ignored (I guess that it is not ignored but redirecting search requests
to data nodes omits storing them in queue). So the given solution is not
working. Or am I missing something?
Paweł Młynarczyk
W dniu poniedziałek, 24 marca 2014 15:17:22 UTC+1 użytkownik Jörg Prante
napisał:
You could set up a special node for "low priority" search, with "slim"
thread pool settings, and forward client calls to it respectively.
Jörg
On Mon, Mar 24, 2014 at 3:08 PM, Paweł Młynarczyk <zwa...@gmail.com<javascript:>
wrote:
There are many users working with the application. Some of them are
online users, that should have their requests served in real time and some
of them are workers that are preparing reports and their requests can wait.
At the moment, all the requests arriving to elasticsearch are queued at the
same queue and that sometimes results in online users not getting their
request served as soon as possible or even getting rejected when the queue
is full of workers' requests. That could be solved by adding separate
Thread Pool and Queue for 'low priority search'. Same would apply for other
operations (get, bulk etc).
Are there any plans or any chance for this kind of feature being
implemented in Elasticsearch?
Apache, Apache Lucene, Apache Hadoop, Hadoop, HDFS and the yellow elephant
logo are trademarks of the
Apache Software Foundation
in the United States and/or other countries.