Elasticsearch error docs


(Alexis Okuwa) #1

I know that the team is working on more documents but it would be great to
have more error related documents what types of errors I should be able to
expect. If there are and what they are for things like index and bulk and
search. Really building out the error handling of my application for cases
where a shard is down or the whole database is down and being able to
dectect those errors seems really useful.

--
You received this message because you are subscribed to the Google Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email to elasticsearch+unsubscribe@googlegroups.com.
For more options, visit https://groups.google.com/groups/opt_out.


(Alexander Reelsen) #2

Hey,

can you explain what you are trying to achieve? You should not use bulk or
search requests for first hand error detection and parse those errors, but
rather put a good monitoring in place which uses the cluster/nodes stats
APIs in order to find out if your cluster is in a good shape.

If you are missing something, just be more specific and we can try to
improve.

--Alex

On Sun, Jul 14, 2013 at 11:07 AM, Alexis Okuwa wojonstech@gmail.com wrote:

I know that the team is working on more documents but it would be great to
have more error related documents what types of errors I should be able to
expect. If there are and what they are for things like index and bulk and
search. Really building out the error handling of my application for cases
where a shard is down or the whole database is down and being able to
dectect those errors seems really useful.

--
You received this message because you are subscribed to the Google Groups
"elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an
email to elasticsearch+unsubscribe@googlegroups.com.
For more options, visit https://groups.google.com/groups/opt_out.

--
You received this message because you are subscribed to the Google Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email to elasticsearch+unsubscribe@googlegroups.com.
For more options, visit https://groups.google.com/groups/opt_out.


(Alexis Okuwa) #3

I do have monitoring in place to know the health and status of my cluster
but if my code is running and es crashes then i plan to queue writes to
rabbit mq. I am finding it hard to find documentation on different types of
errors I will get from elasticsearch on errors in bulk or search or index
operations.

On Sun, Jul 14, 2013 at 11:15 PM, Alexander Reelsen alr@spinscale.dewrote:

Hey,

can you explain what you are trying to achieve? You should not use bulk or
search requests for first hand error detection and parse those errors, but
rather put a good monitoring in place which uses the cluster/nodes stats
APIs in order to find out if your cluster is in a good shape.

If you are missing something, just be more specific and we can try to
improve.

--Alex

On Sun, Jul 14, 2013 at 11:07 AM, Alexis Okuwa wojonstech@gmail.comwrote:

I know that the team is working on more documents but it would be great
to have more error related documents what types of errors I should be able
to expect. If there are and what they are for things like index and bulk
and search. Really building out the error handling of my application for
cases where a shard is down or the whole database is down and being able to
dectect those errors seems really useful.

--
You received this message because you are subscribed to the Google Groups
"elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an
email to elasticsearch+unsubscribe@googlegroups.com.

For more options, visit https://groups.google.com/groups/opt_out.

--
You received this message because you are subscribed to a topic in the
Google Groups "elasticsearch" group.
To unsubscribe from this topic, visit
https://groups.google.com/d/topic/elasticsearch/a21rQ_IvK5M/unsubscribe.
To unsubscribe from this group and all its topics, send an email to
elasticsearch+unsubscribe@googlegroups.com.
For more options, visit https://groups.google.com/groups/opt_out.

--
Enjoy,
Alexis Okuwa
WojonsTech
424.835.1223

--
You received this message because you are subscribed to the Google Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email to elasticsearch+unsubscribe@googlegroups.com.
For more options, visit https://groups.google.com/groups/opt_out.


(Jörg Prante) #4

For bulk indexing, you may receive

  • syntax errors, if you use HTTP and the bulk format is not correct.

  • mapping errors, when a document contains illegal JSON syntax,
    Elasticsearch will refuse to process this document.

  • timeout errors, when the node you use for indexing can't contact other
    nodes in time.

  • resource errors, if the node is running out if resources you will get
    all kind of nasty errors (OOM, file descriptors etc.)

For search operations, there are a lot more error possibilities
regarding to the type of operation.

I'm not sure how this error type listing can help you. You must catch
every exception on the bulk response. If you encounter one, you should
stop indexing no matter what happend.

Jörg

Am 15.07.13 08:31, schrieb Alexis Okuwa:

I am finding it hard to find documentation on different types of
errors I will get from elasticsearch on errors in bulk or search or
index operations.

--
You received this message because you are subscribed to the Google Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email to elasticsearch+unsubscribe@googlegroups.com.
For more options, visit https://groups.google.com/groups/opt_out.


(Alexis Okuwa) #5

I am looking for really how i can spot that there was an error. So i know
to queue the insert (assuming its important data). For searches I will just
log them as failed but for bulk insert and bulk do i just check the status
or can it return a 200 but have a differnt type of error in the doc

On Mon, Jul 15, 2013 at 3:23 AM, Jörg Prante joergprante@gmail.com wrote:

For bulk indexing, you may receive

  • syntax errors, if you use HTTP and the bulk format is not correct.

  • mapping errors, when a document contains illegal JSON syntax,
    Elasticsearch will refuse to process this document.

  • timeout errors, when the node you use for indexing can't contact other
    nodes in time.

  • resource errors, if the node is running out if resources you will get
    all kind of nasty errors (OOM, file descriptors etc.)

For search operations, there are a lot more error possibilities regarding
to the type of operation.

I'm not sure how this error type listing can help you. You must catch
every exception on the bulk response. If you encounter one, you should stop
indexing no matter what happend.

Jörg

Am 15.07.13 08:31, schrieb Alexis Okuwa:

I am finding it hard to find documentation on different types of errors I

will get from elasticsearch on errors in bulk or search or index
operations.

--
You received this message because you are subscribed to a topic in the
Google Groups "elasticsearch" group.
To unsubscribe from this topic, visit https://groups.google.com/d/**
topic/elasticsearch/a21rQ_**IvK5M/unsubscribehttps://groups.google.com/d/topic/elasticsearch/a21rQ_IvK5M/unsubscribe
.
To unsubscribe from this group and all its topics, send an email to
elasticsearch+unsubscribe@**googlegroups.comelasticsearch%2Bunsubscribe@googlegroups.com
.
For more options, visit https://groups.google.com/**groups/opt_outhttps://groups.google.com/groups/opt_out
.

--
Enjoy,
Alexis Okuwa
WojonsTech
424.835.1223

--
You received this message because you are subscribed to the Google Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email to elasticsearch+unsubscribe@googlegroups.com.
For more options, visit https://groups.google.com/groups/opt_out.


(Clinton Gormley) #6

For bulk requests you need to check the response for each request. If it
has an error element, then it failed. Each request is independent of the
other requests

clint

On 15 July 2013 12:36, Alexis Okuwa wojonstech@gmail.com wrote:

I am looking for really how i can spot that there was an error. So i know
to queue the insert (assuming its important data). For searches I will just
log them as failed but for bulk insert and bulk do i just check the status
or can it return a 200 but have a differnt type of error in the doc

On Mon, Jul 15, 2013 at 3:23 AM, Jörg Prante joergprante@gmail.comwrote:

For bulk indexing, you may receive

  • syntax errors, if you use HTTP and the bulk format is not correct.

  • mapping errors, when a document contains illegal JSON syntax,
    Elasticsearch will refuse to process this document.

  • timeout errors, when the node you use for indexing can't contact other
    nodes in time.

  • resource errors, if the node is running out if resources you will get
    all kind of nasty errors (OOM, file descriptors etc.)

For search operations, there are a lot more error possibilities regarding
to the type of operation.

I'm not sure how this error type listing can help you. You must catch
every exception on the bulk response. If you encounter one, you should stop
indexing no matter what happend.

Jörg

Am 15.07.13 08:31, schrieb Alexis Okuwa:

I am finding it hard to find documentation on different types of errors

I will get from elasticsearch on errors in bulk or search or index
operations.

--
You received this message because you are subscribed to a topic in the
Google Groups "elasticsearch" group.
To unsubscribe from this topic, visit https://groups.google.com/d/**
topic/elasticsearch/a21rQ_**IvK5M/unsubscribehttps://groups.google.com/d/topic/elasticsearch/a21rQ_IvK5M/unsubscribe
.
To unsubscribe from this group and all its topics, send an email to
elasticsearch+unsubscribe@**googlegroups.comelasticsearch%2Bunsubscribe@googlegroups.com
.

For more options, visit https://groups.google.com/**groups/opt_outhttps://groups.google.com/groups/opt_out
.

--
Enjoy,
Alexis Okuwa
WojonsTech
424.835.1223

--
You received this message because you are subscribed to the Google Groups
"elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an
email to elasticsearch+unsubscribe@googlegroups.com.
For more options, visit https://groups.google.com/groups/opt_out.

--
You received this message because you are subscribed to the Google Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email to elasticsearch+unsubscribe@googlegroups.com.
For more options, visit https://groups.google.com/groups/opt_out.


(Jörg Prante) #7

Just a hint where to find the bulk reponses:

  • in HTTP, always 200 is returned (OK) for a bulk request. Check the
    HTTP response body for a JSON document which should contain the response
    of each doc in the bulk.

  • in Java, there is a class BulkResponse, it is availabe in the async
    BulkRequest API call.

Jörg

Am 15.07.13 14:19, schrieb Clinton Gormley:

For bulk requests you need to check the response for each request. If
it has an error element, then it failed. Each request is independent
of the other requests

--
You received this message because you are subscribed to the Google Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email to elasticsearch+unsubscribe@googlegroups.com.
For more options, visit https://groups.google.com/groups/opt_out.


(patrik.ragnarsson) #8

On Monday, July 15, 2013 12:23:37 PM UTC+2, Jörg Prante wrote:

I'm not sure how this error type listing can help you. You must catch
every exception on the bulk response. If you encounter one, you should
stop indexing no matter what happend.

I can see how a list of all the error types can help.

It's not true that you have stop indexing no matter what happened. Bulk
requests with the create action returns conflict errors, but that might be
okay for you and you want to ignore those errors. On other errors you might
want to re-queue the request for some reason (e.g. your
threadpool.bulk.queue_size was too low and you got something like
"EsRejectedExecutionException[rejected execution (queue capacity 50)").

--
You received this message because you are subscribed to the Google Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email to elasticsearch+unsubscribe@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/elasticsearch/16f25cde-df7e-4d60-a603-5995212a2dc8%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.


(system) #9