I know that the team is working on more documents but it would be great to
have more error related documents what types of errors I should be able to
expect. If there are and what they are for things like index and bulk and
search. Really building out the error handling of my application for cases
where a shard is down or the whole database is down and being able to
dectect those errors seems really useful.
can you explain what you are trying to achieve? You should not use bulk or
search requests for first hand error detection and parse those errors, but
rather put a good monitoring in place which uses the cluster/nodes stats
APIs in order to find out if your cluster is in a good shape.
If you are missing something, just be more specific and we can try to
improve.
I know that the team is working on more documents but it would be great to
have more error related documents what types of errors I should be able to
expect. If there are and what they are for things like index and bulk and
search. Really building out the error handling of my application for cases
where a shard is down or the whole database is down and being able to
dectect those errors seems really useful.
I do have monitoring in place to know the health and status of my cluster
but if my code is running and es crashes then i plan to queue writes to
rabbit mq. I am finding it hard to find documentation on different types of
errors I will get from elasticsearch on errors in bulk or search or index
operations.
On Sun, Jul 14, 2013 at 11:15 PM, Alexander Reelsen alr@spinscale.dewrote:
Hey,
can you explain what you are trying to achieve? You should not use bulk or
search requests for first hand error detection and parse those errors, but
rather put a good monitoring in place which uses the cluster/nodes stats
APIs in order to find out if your cluster is in a good shape.
If you are missing something, just be more specific and we can try to
improve.
I know that the team is working on more documents but it would be great
to have more error related documents what types of errors I should be able
to expect. If there are and what they are for things like index and bulk
and search. Really building out the error handling of my application for
cases where a shard is down or the whole database is down and being able to
dectect those errors seems really useful.
--
You received this message because you are subscribed to the Google Groups
"elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an
email to elasticsearch+unsubscribe@googlegroups.com.
syntax errors, if you use HTTP and the bulk format is not correct.
mapping errors, when a document contains illegal JSON syntax,
Elasticsearch will refuse to process this document.
timeout errors, when the node you use for indexing can't contact other
nodes in time.
resource errors, if the node is running out if resources you will get
all kind of nasty errors (OOM, file descriptors etc.)
For search operations, there are a lot more error possibilities
regarding to the type of operation.
I'm not sure how this error type listing can help you. You must catch
every exception on the bulk response. If you encounter one, you should
stop indexing no matter what happend.
Jörg
Am 15.07.13 08:31, schrieb Alexis Okuwa:
I am finding it hard to find documentation on different types of
errors I will get from elasticsearch on errors in bulk or search or
index operations.
I am looking for really how i can spot that there was an error. So i know
to queue the insert (assuming its important data). For searches I will just
log them as failed but for bulk insert and bulk do i just check the status
or can it return a 200 but have a differnt type of error in the doc
syntax errors, if you use HTTP and the bulk format is not correct.
mapping errors, when a document contains illegal JSON syntax,
Elasticsearch will refuse to process this document.
timeout errors, when the node you use for indexing can't contact other
nodes in time.
resource errors, if the node is running out if resources you will get
all kind of nasty errors (OOM, file descriptors etc.)
For search operations, there are a lot more error possibilities regarding
to the type of operation.
I'm not sure how this error type listing can help you. You must catch
every exception on the bulk response. If you encounter one, you should stop
indexing no matter what happend.
Jörg
Am 15.07.13 08:31, schrieb Alexis Okuwa:
I am finding it hard to find documentation on different types of errors I
will get from elasticsearch on errors in bulk or search or index
operations.
For bulk requests you need to check the response for each request. If it
has an error element, then it failed. Each request is independent of the
other requests
I am looking for really how i can spot that there was an error. So i know
to queue the insert (assuming its important data). For searches I will just
log them as failed but for bulk insert and bulk do i just check the status
or can it return a 200 but have a differnt type of error in the doc
syntax errors, if you use HTTP and the bulk format is not correct.
mapping errors, when a document contains illegal JSON syntax,
Elasticsearch will refuse to process this document.
timeout errors, when the node you use for indexing can't contact other
nodes in time.
resource errors, if the node is running out if resources you will get
all kind of nasty errors (OOM, file descriptors etc.)
For search operations, there are a lot more error possibilities regarding
to the type of operation.
I'm not sure how this error type listing can help you. You must catch
every exception on the bulk response. If you encounter one, you should stop
indexing no matter what happend.
Jörg
Am 15.07.13 08:31, schrieb Alexis Okuwa:
I am finding it hard to find documentation on different types of errors
I will get from elasticsearch on errors in bulk or search or index
operations.
in HTTP, always 200 is returned (OK) for a bulk request. Check the
HTTP response body for a JSON document which should contain the response
of each doc in the bulk.
in Java, there is a class BulkResponse, it is availabe in the async
BulkRequest API call.
Jörg
Am 15.07.13 14:19, schrieb Clinton Gormley:
For bulk requests you need to check the response for each request. If
it has an error element, then it failed. Each request is independent
of the other requests
On Monday, July 15, 2013 12:23:37 PM UTC+2, Jörg Prante wrote:
I'm not sure how this error type listing can help you. You must catch
every exception on the bulk response. If you encounter one, you should
stop indexing no matter what happend.
I can see how a list of all the error types can help.
It's not true that you have stop indexing no matter what happened. Bulk
requests with the create action returns conflict errors, but that might be
okay for you and you want to ignore those errors. On other errors you might
want to re-queue the request for some reason (e.g. your
threadpool.bulk.queue_size was too low and you got something like
"EsRejectedExecutionException[rejected execution (queue capacity 50)").
Apache, Apache Lucene, Apache Hadoop, Hadoop, HDFS and the yellow elephant
logo are trademarks of the
Apache Software Foundation
in the United States and/or other countries.