Queue Size 5000 exceeded error (Elastic Search Optimization)

Hello All,

I am nilesh makwana. I have set up elastic search for my web site which is
world wide event discovery portal.We use elasticsearch for search
operations. I am experiencing issues in elastic search, It stops working
randomly. Qbox client gives error "all shards failed". I looked up the
issue and increased thread pull queue size to 5000. Still server stops
randomly. I test sever performance using
Android Benchmark. Server can not handle 150 simultaneous request. It stops
every time I run script using ab. What should I do? I except server to at
least handle 1000 concurrent requests.

--
You received this message because you are subscribed to the Google Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email to elasticsearch+unsubscribe@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/elasticsearch/93c566aa-02a1-46a5-9bcf-d61c40b2a054%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

Please describe your custom settings, and examine the server logs. You may
have encountered a server error because you have changed the settings.

Jörg

On Mon, Dec 22, 2014 at 12:44 PM, nilesh makwana make.naleswave@gmail.com
wrote:

Hello All,

I am nilesh makwana. I have set up Elasticsearch for my web site which
is world wide event discovery portal.We use elasticsearch for search
operations. I am experiencing issues in Elasticsearch, It stops working
randomly. Qbox client gives error "all shards failed". I looked up the
issue and increased thread pull queue size to 5000. Still server stops
randomly. I test sever performance using
Android Benchmark. Server can not handle 150 simultaneous request. It
stops every time I run script using ab. What should I do? I except server
to at least handle 1000 concurrent requests.

--
You received this message because you are subscribed to the Google Groups
"elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an
email to elasticsearch+unsubscribe@googlegroups.com.
To view this discussion on the web visit
https://groups.google.com/d/msgid/elasticsearch/93c566aa-02a1-46a5-9bcf-d61c40b2a054%40googlegroups.com
https://groups.google.com/d/msgid/elasticsearch/93c566aa-02a1-46a5-9bcf-d61c40b2a054%40googlegroups.com?utm_medium=email&utm_source=footer
.
For more options, visit https://groups.google.com/d/optout.

--
You received this message because you are subscribed to the Google Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email to elasticsearch+unsubscribe@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/elasticsearch/CAKdsXoERF_6wLNEyWuQi-%2Bq7J6FiW09Qu3H6v%3DC1McCc1qFjBQ%40mail.gmail.com.
For more options, visit https://groups.google.com/d/optout.

Here is elasticsearch.yml file for elasticsearch service. Logs are showing
queue size error as I told earlier. I don't have logs at time I first time
changed configurations. After that I am tweaking configurations to get
optimized service.

On Mon, Dec 22, 2014 at 6:42 PM, joergprante@gmail.com <
joergprante@gmail.com> wrote:

Please describe your custom settings, and examine the server logs. You may
have encountered a server error because you have changed the settings.

Jörg

On Mon, Dec 22, 2014 at 12:44 PM, nilesh makwana <make.naleswave@gmail.com

wrote:

Hello All,

I am nilesh makwana. I have set up Elasticsearch for my web site which
is world wide event discovery portal.We use elasticsearch for search
operations. I am experiencing issues in Elasticsearch, It stops working
randomly. Qbox client gives error "all shards failed". I looked up the
issue and increased thread pull queue size to 5000. Still server stops
randomly. I test sever performance using
Android Benchmark. Server can not handle 150 simultaneous request. It
stops every time I run script using ab. What should I do? I except server
to at least handle 1000 concurrent requests.

--
You received this message because you are subscribed to the Google Groups
"elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an
email to elasticsearch+unsubscribe@googlegroups.com.
To view this discussion on the web visit
https://groups.google.com/d/msgid/elasticsearch/93c566aa-02a1-46a5-9bcf-d61c40b2a054%40googlegroups.com
https://groups.google.com/d/msgid/elasticsearch/93c566aa-02a1-46a5-9bcf-d61c40b2a054%40googlegroups.com?utm_medium=email&utm_source=footer
.
For more options, visit https://groups.google.com/d/optout.

--
You received this message because you are subscribed to a topic in the
Google Groups "elasticsearch" group.
To unsubscribe from this topic, visit
https://groups.google.com/d/topic/elasticsearch/fERgjfFt9l4/unsubscribe.
To unsubscribe from this group and all its topics, send an email to
elasticsearch+unsubscribe@googlegroups.com.
To view this discussion on the web visit
https://groups.google.com/d/msgid/elasticsearch/CAKdsXoERF_6wLNEyWuQi-%2Bq7J6FiW09Qu3H6v%3DC1McCc1qFjBQ%40mail.gmail.com
https://groups.google.com/d/msgid/elasticsearch/CAKdsXoERF_6wLNEyWuQi-%2Bq7J6FiW09Qu3H6v%3DC1McCc1qFjBQ%40mail.gmail.com?utm_medium=email&utm_source=footer
.

For more options, visit https://groups.google.com/d/optout.

--
You received this message because you are subscribed to the Google Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email to elasticsearch+unsubscribe@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/elasticsearch/CAGDNVTfgaY51j3_uWW0qW9zfARdA8-GexbBJJwtAbm2dmpNarQ%40mail.gmail.com.
For more options, visit https://groups.google.com/d/optout.

Please remove

threadpool.search.queue_size: 5000 threadpool.search.type: fixed
threadpool.bulk.queue_size: 5000 threadpool.bulk.type: fixed
threadpool.index.queue_size: 5000 threadpool.index.type: fixed

These settings way too large and will bog down your system.

Jörg

On Tue, Dec 23, 2014 at 10:56 AM, nilesh makwana make.naleswave@gmail.com
wrote:

Here is elasticsearch.yml file for elasticsearch service. Logs are showing
queue size error as I told earlier. I don't have logs at time I first time
changed configurations. After that I am tweaking configurations to get
optimized service.

On Mon, Dec 22, 2014 at 6:42 PM, joergprante@gmail.com <
joergprante@gmail.com> wrote:

Please describe your custom settings, and examine the server logs. You
may have encountered a server error because you have changed the settings.

Jörg

On Mon, Dec 22, 2014 at 12:44 PM, nilesh makwana <
make.naleswave@gmail.com> wrote:

Hello All,

I am nilesh makwana. I have set up Elasticsearch for my web site which
is world wide event discovery portal.We use elasticsearch for search
operations. I am experiencing issues in Elasticsearch, It stops working
randomly. Qbox client gives error "all shards failed". I looked up the
issue and increased thread pull queue size to 5000. Still server stops
randomly. I test sever performance using
Android Benchmark. Server can not handle 150 simultaneous request. It
stops every time I run script using ab. What should I do? I except server
to at least handle 1000 concurrent requests.

--
You received this message because you are subscribed to the Google
Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send
an email to elasticsearch+unsubscribe@googlegroups.com.
To view this discussion on the web visit
https://groups.google.com/d/msgid/elasticsearch/93c566aa-02a1-46a5-9bcf-d61c40b2a054%40googlegroups.com
https://groups.google.com/d/msgid/elasticsearch/93c566aa-02a1-46a5-9bcf-d61c40b2a054%40googlegroups.com?utm_medium=email&utm_source=footer
.
For more options, visit https://groups.google.com/d/optout.

--
You received this message because you are subscribed to a topic in the
Google Groups "elasticsearch" group.
To unsubscribe from this topic, visit
https://groups.google.com/d/topic/elasticsearch/fERgjfFt9l4/unsubscribe.
To unsubscribe from this group and all its topics, send an email to
elasticsearch+unsubscribe@googlegroups.com.
To view this discussion on the web visit
https://groups.google.com/d/msgid/elasticsearch/CAKdsXoERF_6wLNEyWuQi-%2Bq7J6FiW09Qu3H6v%3DC1McCc1qFjBQ%40mail.gmail.com
https://groups.google.com/d/msgid/elasticsearch/CAKdsXoERF_6wLNEyWuQi-%2Bq7J6FiW09Qu3H6v%3DC1McCc1qFjBQ%40mail.gmail.com?utm_medium=email&utm_source=footer
.

For more options, visit https://groups.google.com/d/optout.

--
You received this message because you are subscribed to the Google Groups
"elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an
email to elasticsearch+unsubscribe@googlegroups.com.
To view this discussion on the web visit
https://groups.google.com/d/msgid/elasticsearch/CAGDNVTfgaY51j3_uWW0qW9zfARdA8-GexbBJJwtAbm2dmpNarQ%40mail.gmail.com
https://groups.google.com/d/msgid/elasticsearch/CAGDNVTfgaY51j3_uWW0qW9zfARdA8-GexbBJJwtAbm2dmpNarQ%40mail.gmail.com?utm_medium=email&utm_source=footer
.

For more options, visit https://groups.google.com/d/optout.

--
You received this message because you are subscribed to the Google Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email to elasticsearch+unsubscribe@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/elasticsearch/CAKdsXoGWzhoHqLVg09Czzk61uZCmbjUcR7aOJqoFw1mKEc2OkA%40mail.gmail.com.
For more options, visit https://groups.google.com/d/optout.

I removed all those lines. After that I use this command ab -n 500 -c 500
http://localhost/es_curl.php to benchmark elasticsearch. It works till 300
concurrent requests. After that server stops. We have 2 indexes. 1 is
having 1.1 million documents, 2nd is having 10K documents.

What could be possible issues?

-Nilesh Makwana

On Tue, Dec 23, 2014 at 4:28 PM, joergprante@gmail.com <
joergprante@gmail.com> wrote:

Please remove

threadpool.search.queue_size: 5000 threadpool.search.type: fixed
threadpool.bulk.queue_size: 5000 threadpool.bulk.type: fixed
threadpool.index.queue_size: 5000 threadpool.index.type: fixed

These settings way too large and will bog down your system.

Jörg

On Tue, Dec 23, 2014 at 10:56 AM, nilesh makwana <make.naleswave@gmail.com

wrote:

Here is elasticsearch.yml file for elasticsearch service. Logs are
showing queue size error as I told earlier. I don't have logs at time I
first time changed configurations. After that I am tweaking configurations
to get optimized service.

On Mon, Dec 22, 2014 at 6:42 PM, joergprante@gmail.com <
joergprante@gmail.com> wrote:

Please describe your custom settings, and examine the server logs. You
may have encountered a server error because you have changed the settings.

Jörg

On Mon, Dec 22, 2014 at 12:44 PM, nilesh makwana <
make.naleswave@gmail.com> wrote:

Hello All,

I am nilesh makwana. I have set up Elasticsearch for my web site
which is world wide event discovery portal.We use elasticsearch for search
operations. I am experiencing issues in Elasticsearch, It stops working
randomly. Qbox client gives error "all shards failed". I looked up the
issue and increased thread pull queue size to 5000. Still server stops
randomly. I test sever performance using
Android Benchmark. Server can not handle 150 simultaneous request. It
stops every time I run script using ab. What should I do? I except server
to at least handle 1000 concurrent requests.

--
You received this message because you are subscribed to the Google
Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send
an email to elasticsearch+unsubscribe@googlegroups.com.
To view this discussion on the web visit
https://groups.google.com/d/msgid/elasticsearch/93c566aa-02a1-46a5-9bcf-d61c40b2a054%40googlegroups.com
https://groups.google.com/d/msgid/elasticsearch/93c566aa-02a1-46a5-9bcf-d61c40b2a054%40googlegroups.com?utm_medium=email&utm_source=footer
.
For more options, visit https://groups.google.com/d/optout.

--
You received this message because you are subscribed to a topic in the
Google Groups "elasticsearch" group.
To unsubscribe from this topic, visit
https://groups.google.com/d/topic/elasticsearch/fERgjfFt9l4/unsubscribe.
To unsubscribe from this group and all its topics, send an email to
elasticsearch+unsubscribe@googlegroups.com.
To view this discussion on the web visit
https://groups.google.com/d/msgid/elasticsearch/CAKdsXoERF_6wLNEyWuQi-%2Bq7J6FiW09Qu3H6v%3DC1McCc1qFjBQ%40mail.gmail.com
https://groups.google.com/d/msgid/elasticsearch/CAKdsXoERF_6wLNEyWuQi-%2Bq7J6FiW09Qu3H6v%3DC1McCc1qFjBQ%40mail.gmail.com?utm_medium=email&utm_source=footer
.

For more options, visit https://groups.google.com/d/optout.

--
You received this message because you are subscribed to the Google Groups
"elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an
email to elasticsearch+unsubscribe@googlegroups.com.
To view this discussion on the web visit
https://groups.google.com/d/msgid/elasticsearch/CAGDNVTfgaY51j3_uWW0qW9zfARdA8-GexbBJJwtAbm2dmpNarQ%40mail.gmail.com
https://groups.google.com/d/msgid/elasticsearch/CAGDNVTfgaY51j3_uWW0qW9zfARdA8-GexbBJJwtAbm2dmpNarQ%40mail.gmail.com?utm_medium=email&utm_source=footer
.

For more options, visit https://groups.google.com/d/optout.

--
You received this message because you are subscribed to a topic in the
Google Groups "elasticsearch" group.
To unsubscribe from this topic, visit
https://groups.google.com/d/topic/elasticsearch/fERgjfFt9l4/unsubscribe.
To unsubscribe from this group and all its topics, send an email to
elasticsearch+unsubscribe@googlegroups.com.
To view this discussion on the web visit
https://groups.google.com/d/msgid/elasticsearch/CAKdsXoGWzhoHqLVg09Czzk61uZCmbjUcR7aOJqoFw1mKEc2OkA%40mail.gmail.com
https://groups.google.com/d/msgid/elasticsearch/CAKdsXoGWzhoHqLVg09Czzk61uZCmbjUcR7aOJqoFw1mKEc2OkA%40mail.gmail.com?utm_medium=email&utm_source=footer
.

For more options, visit https://groups.google.com/d/optout.

--
You received this message because you are subscribed to the Google Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email to elasticsearch+unsubscribe@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/elasticsearch/CAGDNVTe1%3D45J2_arZCDOyCSbpzOKuV6kWPkoT7cPRc36nPBfDA%40mail.gmail.com.
For more options, visit https://groups.google.com/d/optout.

What do you mean by "after that server stops"?

You give very few details but in the general case, you overwhelm your
hardware, and the easiest remedy is to change queries, indexing, or add
nodes.

Jörg

On Tue, Dec 23, 2014 at 12:39 PM, nilesh makwana make.naleswave@gmail.com
wrote:

I removed all those lines. After that I use this command ab -n 500 -c 500
http://localhost/es_curl.php to benchmark elasticsearch. It works till
300 concurrent requests. After that server stops. We have 2 indexes. 1 is
having 1.1 million documents, 2nd is having 10K documents.

What could be possible issues?

-Nilesh Makwana

On Tue, Dec 23, 2014 at 4:28 PM, joergprante@gmail.com <
joergprante@gmail.com> wrote:

Please remove

threadpool.search.queue_size: 5000 threadpool.search.type: fixed
threadpool.bulk.queue_size: 5000 threadpool.bulk.type: fixed
threadpool.index.queue_size: 5000 threadpool.index.type: fixed

These settings way too large and will bog down your system.

Jörg

On Tue, Dec 23, 2014 at 10:56 AM, nilesh makwana <
make.naleswave@gmail.com> wrote:

Here is elasticsearch.yml file for elasticsearch service. Logs are
showing queue size error as I told earlier. I don't have logs at time I
first time changed configurations. After that I am tweaking configurations
to get optimized service.

On Mon, Dec 22, 2014 at 6:42 PM, joergprante@gmail.com <
joergprante@gmail.com> wrote:

Please describe your custom settings, and examine the server logs. You
may have encountered a server error because you have changed the settings.

Jörg

On Mon, Dec 22, 2014 at 12:44 PM, nilesh makwana <
make.naleswave@gmail.com> wrote:

Hello All,

I am nilesh makwana. I have set up Elasticsearch for my web site
which is world wide event discovery portal.We use elasticsearch for search
operations. I am experiencing issues in Elasticsearch, It stops working
randomly. Qbox client gives error "all shards failed". I looked up the
issue and increased thread pull queue size to 5000. Still server stops
randomly. I test sever performance using
Android Benchmark. Server can not handle 150 simultaneous request. It
stops every time I run script using ab. What should I do? I except server
to at least handle 1000 concurrent requests.

--
You received this message because you are subscribed to the Google
Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send
an email to elasticsearch+unsubscribe@googlegroups.com.
To view this discussion on the web visit
https://groups.google.com/d/msgid/elasticsearch/93c566aa-02a1-46a5-9bcf-d61c40b2a054%40googlegroups.com
https://groups.google.com/d/msgid/elasticsearch/93c566aa-02a1-46a5-9bcf-d61c40b2a054%40googlegroups.com?utm_medium=email&utm_source=footer
.
For more options, visit https://groups.google.com/d/optout.

--
You received this message because you are subscribed to a topic in the
Google Groups "elasticsearch" group.
To unsubscribe from this topic, visit
https://groups.google.com/d/topic/elasticsearch/fERgjfFt9l4/unsubscribe
.
To unsubscribe from this group and all its topics, send an email to
elasticsearch+unsubscribe@googlegroups.com.
To view this discussion on the web visit
https://groups.google.com/d/msgid/elasticsearch/CAKdsXoERF_6wLNEyWuQi-%2Bq7J6FiW09Qu3H6v%3DC1McCc1qFjBQ%40mail.gmail.com
https://groups.google.com/d/msgid/elasticsearch/CAKdsXoERF_6wLNEyWuQi-%2Bq7J6FiW09Qu3H6v%3DC1McCc1qFjBQ%40mail.gmail.com?utm_medium=email&utm_source=footer
.

For more options, visit https://groups.google.com/d/optout.

--
You received this message because you are subscribed to the Google
Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send
an email to elasticsearch+unsubscribe@googlegroups.com.
To view this discussion on the web visit
https://groups.google.com/d/msgid/elasticsearch/CAGDNVTfgaY51j3_uWW0qW9zfARdA8-GexbBJJwtAbm2dmpNarQ%40mail.gmail.com
https://groups.google.com/d/msgid/elasticsearch/CAGDNVTfgaY51j3_uWW0qW9zfARdA8-GexbBJJwtAbm2dmpNarQ%40mail.gmail.com?utm_medium=email&utm_source=footer
.

For more options, visit https://groups.google.com/d/optout.

--
You received this message because you are subscribed to a topic in the
Google Groups "elasticsearch" group.
To unsubscribe from this topic, visit
https://groups.google.com/d/topic/elasticsearch/fERgjfFt9l4/unsubscribe.
To unsubscribe from this group and all its topics, send an email to
elasticsearch+unsubscribe@googlegroups.com.
To view this discussion on the web visit
https://groups.google.com/d/msgid/elasticsearch/CAKdsXoGWzhoHqLVg09Czzk61uZCmbjUcR7aOJqoFw1mKEc2OkA%40mail.gmail.com
https://groups.google.com/d/msgid/elasticsearch/CAKdsXoGWzhoHqLVg09Czzk61uZCmbjUcR7aOJqoFw1mKEc2OkA%40mail.gmail.com?utm_medium=email&utm_source=footer
.

For more options, visit https://groups.google.com/d/optout.

--
You received this message because you are subscribed to the Google Groups
"elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an
email to elasticsearch+unsubscribe@googlegroups.com.
To view this discussion on the web visit
https://groups.google.com/d/msgid/elasticsearch/CAGDNVTe1%3D45J2_arZCDOyCSbpzOKuV6kWPkoT7cPRc36nPBfDA%40mail.gmail.com
https://groups.google.com/d/msgid/elasticsearch/CAGDNVTe1%3D45J2_arZCDOyCSbpzOKuV6kWPkoT7cPRc36nPBfDA%40mail.gmail.com?utm_medium=email&utm_source=footer
.

For more options, visit https://groups.google.com/d/optout.

--
You received this message because you are subscribed to the Google Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email to elasticsearch+unsubscribe@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/elasticsearch/CAKdsXoGQkhcVVWXMbeUkAJZ%2BNdN6w0_yGmHVoPf5EUCsNS6_bQ%40mail.gmail.com.
For more options, visit https://groups.google.com/d/optout.