Delete by query fails often with HTTP 503

Elastic Stack Elasticsearch
1

Hi,

We often get failures when using the delete by query API. The response is
an HTTP 503 with a body like this:

{"_indices": {"myindex": {"_shards": {"successful": 2, "failed": 58,
"total": 60}}}}

Is there a way to figure out what is causing this error? It seems to mostly
happen when the search cluster is busy.

Thomas

--
You received this message because you are subscribed to the Google Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email to elasticsearch+unsubscribe@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/elasticsearch/f8c84eaf-79b9-4f4e-9b26-732d11544fb9%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

2

Do you have lots of shards on just a few nodes? Delete by query is handled
by the index thread pool, but those threads are shared across all shards
on a node. Delete by query can produce a large number of changes, which
can fill up the thread pool queue and result in rejections.

You can either just (a) retry or (b) increase the queue size for the
index thread pool (which will use more memory as more delete requests
will need to be queued)

See

clint

On 18 March 2014 08:13, Thomas S. thomas.st@gmail.com wrote:

Hi,

We often get failures when using the delete by query API. The response is
an HTTP 503 with a body like this:

{"_indices": {"myindex": {"_shards": {"successful": 2, "failed": 58,
"total": 60}}}}

Is there a way to figure out what is causing this error? It seems to
mostly happen when the search cluster is busy.

Thomas

--
You received this message because you are subscribed to the Google Groups
"elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an
email to elasticsearch+unsubscribe@googlegroups.com.
To view this discussion on the web visit
https://groups.google.com/d/msgid/elasticsearch/f8c84eaf-79b9-4f4e-9b26-732d11544fb9%40googlegroups.comhttps://groups.google.com/d/msgid/elasticsearch/f8c84eaf-79b9-4f4e-9b26-732d11544fb9%40googlegroups.com?utm_medium=email&utm_source=footer
.
For more options, visit https://groups.google.com/d/optout.

--
You received this message because you are subscribed to the Google Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email to elasticsearch+unsubscribe@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/elasticsearch/CAPt3XKTOAAc0ov6YrKGj1Fgkxkh%2BA5%2Bya1LKSfXjX5gVtbb9xA%40mail.gmail.com.
For more options, visit https://groups.google.com/d/optout.

3

Thanks Clint,

We have two nodes with 60 shards per node. I will increase the queue size.
Hopefully this will reduce the amount of rejections.

Thomas

On Tuesday, March 18, 2014 6:11:27 PM UTC+1, Clinton Gormley wrote:

Do you have lots of shards on just a few nodes? Delete by query is handled
by the index thread pool, but those threads are shared across all shards
on a node. Delete by query can produce a large number of changes, which
can fill up the thread pool queue and result in rejections.

You can either just (a) retry or (b) increase the queue size for the
index thread pool (which will use more memory as more delete requests
will need to be queued)

See
Elasticsearch Platform — Find real-time answers at scale | Elastic

clint

On 18 March 2014 08:13, Thomas S. <thom...@gmail.com <javascript:>> wrote:

Hi,

We often get failures when using the delete by query API. The response is
an HTTP 503 with a body like this:

{"_indices": {"myindex": {"_shards": {"successful": 2, "failed": 58,
"total": 60}}}}

Is there a way to figure out what is causing this error? It seems to
mostly happen when the search cluster is busy.

Thomas

--
You received this message because you are subscribed to the Google Groups
"elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an
email to elasticsearc...@googlegroups.com <javascript:>.
To view this discussion on the web visit
https://groups.google.com/d/msgid/elasticsearch/f8c84eaf-79b9-4f4e-9b26-732d11544fb9%40googlegroups.comhttps://groups.google.com/d/msgid/elasticsearch/f8c84eaf-79b9-4f4e-9b26-732d11544fb9%40googlegroups.com?utm_medium=email&utm_source=footer
.
For more options, visit https://groups.google.com/d/optout.

--
You received this message because you are subscribed to the Google Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email to elasticsearch+unsubscribe@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/elasticsearch/b815184a-8382-4b25-8a54-b98753f6cbb4%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.