# High failure rates on REST indexing **URL:** \ **Category:** Elasticsearch\ **Created:** [February 13, 2010, 8:00pm UTC](https://discuss.elastic.co/t/high-failure-rates-on-rest-indexing/2807 "2010-02-13T20:00:22Z")\ **Posts on this page:** 7\ **Page:** 1
**Author:** ![Tim\_Dysinger\_2\_3](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/tim_dysinger_2_3/32/3365_2.png) [@Tim\_Dysinger\_2\_3](https://discuss.elastic.co/u/Tim_Dysinger_2_3)\ **Post date:** [February 13, 2010, 8:00pm UTC](https://discuss.elastic.co/t/high-failure-rates-on-rest-indexing/2807/1 "2010-02-13T20:00:22Z")
I am rapidly pushing 750k documents to one elasticsearch node (localhost). About 3% of them fail to push over rest. The json is fine. I've pushed all these to CouchDB before. The console just fills up with this [11:55:54,225][WARN][http.netty] [Reilly, Ben] Caught exception while handling client http trafic java.lang.ClassCastException: org.jboss.netty.handler.codec.http.DefaultHttpChunk cannot be cast to org.jboss.netty.handler.codec.http.HttpRequest at org.elasticsearch.http.netty.HttpRequestHandler.messageReceived(HttpRequestHandler.java: 39) at org.jboss.netty.channel.Channels.fireMessageReceived(Channels.java: 391) at org.jboss.netty.handler.codec.replay.ReplayingDecoder.unfoldAndfireMessageReceived(ReplayingDecoder.java: 499) at org.jboss.netty.handler.codec.replay.ReplayingDecoder.callDecode(ReplayingDecoder.java: 490) at org.jboss.netty.handler.codec.replay.ReplayingDecoder.messageReceived(ReplayingDecoder.java: 427) at org.jboss.netty.handler.timeout.ReadTimeoutHandler.messageReceived(ReadTimeoutHandler.java: 156) at org.elasticsearch.http.netty.OpenChannelsHandler.handleUpstream(OpenChannelsHandler.java: 49) at org.jboss.netty.channel.Channels.fireMessageReceived(Channels.java: 345) at org.jboss.netty.channel.Channels.fireMessageReceived(Channels.java: 332) at org.jboss.netty.channel.socket.nio.NioWorker.read(NioWorker.java: 323) at org.jboss.netty.channel.socket.nio.NioWorker.processSelectedKeys(NioWorker.java: 275) at org.jboss.netty.channel.socket.nio.NioWorker.run(NioWorker.java: 196) at org.jboss.netty.util.internal.IoWorkerRunnable.run(IoWorkerRunnable.java: 46) at java.util.concurrent.ThreadPoolExecutor $Worker.runTask(ThreadPoolExecutor.java:886) at java.util.concurrent.ThreadPoolExecutor $Worker.run(ThreadPoolExecutor.java:908) at java.lang.Thread.run(Thread.java:619) ---
**Author:** ![kimchy](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/kimchy/32/44952_2.png) [@kimchy](https://discuss.elastic.co/u/kimchy)\ **Post date:** [February 13, 2010, 8:19pm UTC](https://discuss.elastic.co/t/high-failure-rates-on-rest-indexing/2807/2 "2010-02-13T20:19:33Z")
Hey, This happens basically because there is no support for http chunk yet, which I guess the http client you use does for large requests. Is there a chance that you can disable chunking for now? Which http client are you using? -shay.banon On 2/13/10, Tim Dysinger [dysinger@gmail.com](mailto:dysinger@gmail.com) wrote: > I am rapidly pushing 750k documents to one elasticsearch node > (localhost). About 3% of them fail to push over rest. The json is > fine. I've pushed all these to CouchDB before. > > The console just fills up with this > > [11:55:54,225][WARN][http.netty] [Reilly, Ben] Caught > exception while handling client http trafic > java.lang.ClassCastException: > org.jboss.netty.handler.codec.http.DefaultHttpChunk cannot be cast to > org.jboss.netty.handler.codec.http.HttpRequest > at > org.elasticsearch.http.netty.HttpRequestHandler.messageReceived(HttpRequestHandler.java: > 39) > at org.jboss.netty.channel.Channels.fireMessageReceived(Channels.java: > 391) > at > org.jboss.netty.handler.codec.replay.ReplayingDecoder.unfoldAndfireMessageReceived(ReplayingDecoder.java: > 499) > at > org.jboss.netty.handler.codec.replay.ReplayingDecoder.callDecode(ReplayingDecoder.java: > 490) > at > org.jboss.netty.handler.codec.replay.ReplayingDecoder.messageReceived(ReplayingDecoder.java: > 427) > at > org.jboss.netty.handler.timeout.ReadTimeoutHandler.messageReceived(ReadTimeoutHandler.java: > 156) > at > org.elasticsearch.http.netty.OpenChannelsHandler.handleUpstream(OpenChannelsHandler.java: > 49) > at org.jboss.netty.channel.Channels.fireMessageReceived(Channels.java: > 345) > at org.jboss.netty.channel.Channels.fireMessageReceived(Channels.java: > 332) > at org.jboss.netty.channel.socket.nio.NioWorker.read(NioWorker.java: > 323) > at > org.jboss.netty.channel.socket.nio.NioWorker.processSelectedKeys(NioWorker.java: > 275) > at org.jboss.netty.channel.socket.nio.NioWorker.run(NioWorker.java: > 196) > at > org.jboss.netty.util.internal.IoWorkerRunnable.run(IoWorkerRunnable.java: > 46) > at java.util.concurrent.ThreadPoolExecutor > $Worker.runTask(ThreadPoolExecutor.java:886) > at java.util.concurrent.ThreadPoolExecutor > $Worker.run(ThreadPoolExecutor.java:908) > at java.lang.Thread.run(Thread.java:619) -- Sent from my mobile device ---
**Author:** ![Tim\_Dysinger\_2\_3](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/tim_dysinger_2_3/32/3365_2.png) [@Tim\_Dysinger\_2\_3](https://discuss.elastic.co/u/Tim_Dysinger_2_3)\ **Post date:** [February 13, 2010, 8:28pm UTC](https://discuss.elastic.co/t/high-failure-rates-on-rest-indexing/2807/3 "2010-02-13T20:28:35Z")
Using this client [GitHub - francois/rest-client: Simple REST client for Ruby, inspired by microframework syntax for specifying actions.](http://github.com/francois/rest-client) I am not explicitly doing chunking but it's possible it's doing this behind the scenes on larger documents. On Feb 13, 12:19 pm, Shay Banon [shay.ba...@elasticsearch.com](mailto:shay.ba...@elasticsearch.com) wrote: > Hey, > > This happens basically because there is no support for http chunk yet, > which I guess the http client you use does for large requests. > > Is there a chance that you can disable chunking for now? Which http > client are you using? > > -shay.banon > > On 2/13/10, Tim Dysinger [dysin...@gmail.com](mailto:dysin...@gmail.com) wrote: > > > I am rapidly pushing 750k documents to one elasticsearch node > > (localhost). About 3% of them fail to push over rest. The json is > > fine. I've pushed all these to CouchDB before. > > > The console just fills up with this > > > [11:55:54,225][WARN][http.netty] [Reilly, Ben] Caught > > exception while handling client http trafic > > java.lang.ClassCastException: > > org.jboss.netty.handler.codec.http.DefaultHttpChunk cannot be cast to > > org.jboss.netty.handler.codec.http.HttpRequest > > at > > org.elasticsearch.http.netty.HttpRequestHandler.messageReceived(HttpRequest Handler.java: > > 39) > > at org.jboss.netty.channel.Channels.fireMessageReceived(Channels.java: > > 391) > > at > > org.jboss.netty.handler.codec.replay.ReplayingDecoder.unfoldAndfireMessageR eceived(ReplayingDecoder.java: > > 499) > > at > > org.jboss.netty.handler.codec.replay.ReplayingDecoder.callDecode(ReplayingD ecoder.java: > > 490) > > at > > org.jboss.netty.handler.codec.replay.ReplayingDecoder.messageReceived(Repla yingDecoder.java: > > 427) > > at > > org.jboss.netty.handler.timeout.ReadTimeoutHandler.messageReceived(ReadTime outHandler.java: > > 156) > > at > > org.elasticsearch.http.netty.OpenChannelsHandler.handleUpstream(OpenChannel sHandler.java: > > 49) > > at org.jboss.netty.channel.Channels.fireMessageReceived(Channels.java: > > 345) > > at org.jboss.netty.channel.Channels.fireMessageReceived(Channels.java: > > 332) > > at org.jboss.netty.channel.socket.nio.NioWorker.read(NioWorker.java: > > 323) > > at > > org.jboss.netty.channel.socket.nio.NioWorker.processSelectedKeys(NioWorker. java: > > 275) > > at org.jboss.netty.channel.socket.nio.NioWorker.run(NioWorker.java: > > 196) > > at > > org.jboss.netty.util.internal.IoWorkerRunnable.run(IoWorkerRunnable.java: > > 46) > > at java.util.concurrent.ThreadPoolExecutor > > $Worker.runTask(ThreadPoolExecutor.java:886) > > at java.util.concurrent.ThreadPoolExecutor > > $Worker.run(ThreadPoolExecutor.java:908) > > at java.lang.Thread.run(Thread.java:619) > > -- > Sent from my mobile device ---
**Author:** ![kimchy](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/kimchy/32/44952_2.png) [@kimchy](https://discuss.elastic.co/u/kimchy)\ **Post date:** [February 13, 2010, 9:46pm UTC](https://discuss.elastic.co/t/high-failure-rates-on-rest-indexing/2807/4 "2010-02-13T21:46:32Z")
Support for HTTP chunks has been added to master: [Issues · elastic/elasticsearch · GitHub](http://github.com/elasticsearch/elasticsearch/issues/closed#issue/8). You can get it here: [http://www.elasticsearch.com/download/](http://www.elasticsearch.com/download/) (with instructions on how to build it). -shay.banon On Sat, Feb 13, 2010 at 10:28 PM, Tim Dysinger [dysinger@gmail.com](mailto:dysinger@gmail.com) wrote: > Using this client [GitHub - francois/rest-client: Simple REST client for Ruby, inspired by microframework syntax for specifying actions.](http://github.com/francois/rest-client) > > I am not explicitly doing chunking but it's possible it's doing this > behind the scenes on larger documents. > > On Feb 13, 12:19 pm, Shay Banon [shay.ba...@elasticsearch.com](mailto:shay.ba...@elasticsearch.com) wrote: > > > Hey, > > > > This happens basically because there is no support for http chunk yet, > > which I guess the http client you use does for large requests. > > > > Is there a chance that you can disable chunking for now? Which http > > client are you using? > > > > -shay.banon > > > > On 2/13/10, Tim Dysinger [dysin...@gmail.com](mailto:dysin...@gmail.com) wrote: > > > > > I am rapidly pushing 750k documents to one elasticsearch node > > > (localhost). About 3% of them fail to push over rest. The json is > > > fine. I've pushed all these to CouchDB before. > > > > > The console just fills up with this > > > > > [11:55:54,225][WARN][http.netty] [Reilly, Ben] Caught > > > exception while handling client http trafic > > > java.lang.ClassCastException: > > > org.jboss.netty.handler.codec.http.DefaultHttpChunk cannot be cast to > > > org.jboss.netty.handler.codec.http.HttpRequest > > > at > > org.elasticsearch.http.netty.HttpRequestHandler.messageReceived(HttpRequest > Handler.java: > > > > 1. > > > > > > at > > > org.jboss.netty.channel.Channels.fireMessageReceived(Channels.java: > > > 391) > > > at > > org.jboss.netty.handler.codec.replay.ReplayingDecoder.unfoldAndfireMessageR > eceived(ReplayingDecoder.java: > > > > 1. > > > > > > at > > org.jboss.netty.handler.codec.replay.ReplayingDecoder.callDecode(ReplayingD > ecoder.java: > > > > 1. > > > > > > at > > org.jboss.netty.handler.codec.replay.ReplayingDecoder.messageReceived(Repla > yingDecoder.java: > > > > 1. > > > > > > at > > org.jboss.netty.handler.timeout.ReadTimeoutHandler.messageReceived(ReadTime > outHandler.java: > > > > 1. > > > > > > at > > org.elasticsearch.http.netty.OpenChannelsHandler.handleUpstream(OpenChannel > sHandler.java: > > > > 1. > > > > > > at > > > org.jboss.netty.channel.Channels.fireMessageReceived(Channels.java: > > > 345) > > > at > > > org.jboss.netty.channel.Channels.fireMessageReceived(Channels.java: > > > 332) > > > at org.jboss.netty.channel.socket.nio.NioWorker.read(NioWorker.java: > > > 323) > > > at > > org.jboss.netty.channel.socket.nio.NioWorker.processSelectedKeys(NioWorker. > java: > > > > 1. > > > > > > at org.jboss.netty.channel.socket.nio.NioWorker.run(NioWorker.java: > > > 196) > > > at > > org.jboss.netty.util.internal.IoWorkerRunnable.run(IoWorkerRunnable.java: > > > > 1. > > > > > > at java.util.concurrent.ThreadPoolExecutor > > > $Worker.runTask(ThreadPoolExecutor.java:886) > > > at java.util.concurrent.ThreadPoolExecutor > > > $Worker.run(ThreadPoolExecutor.java:908) > > > at java.lang.Thread.run(Thread.java:619) > > > > -- > > Sent from my mobile device ---
**Author:** ![kimchy](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/kimchy/32/44952_2.png) [@kimchy](https://discuss.elastic.co/u/kimchy)\ **Post date:** [February 14, 2010, 12:24am UTC](https://discuss.elastic.co/t/high-failure-rates-on-rest-indexing/2807/5 "2010-02-14T00:24:32Z")
By the way, it would be interesting the know the numbers you are getting. If you can, also post the environment you are running on (os, number of CPUs), the client type, number of clients (concurrent), typical document size and content, elasticsearch configuration (both node level and index level), cpu taken by client and by elasticsearch, and anything else you find useful. -shay.banon On Sat, Feb 13, 2010 at 11:46 PM, Shay Banon [shay.banon@elasticsearch.com](mailto:shay.banon@elasticsearch.com)wrote: > Support for HTTP chunks has been added to master: > [Issues · elastic/elasticsearch · GitHub](http://github.com/elasticsearch/elasticsearch/issues/closed#issue/8). You > can get it here: [http://www.elasticsearch.com/download/](http://www.elasticsearch.com/download/) (with instructions > on how to build it). > > -shay.banon > > On Sat, Feb 13, 2010 at 10:28 PM, Tim Dysinger [dysinger@gmail.com](mailto:dysinger@gmail.com) wrote: > > > Using this client [GitHub - francois/rest-client: Simple REST client for Ruby, inspired by microframework syntax for specifying actions.](http://github.com/francois/rest-client) > > > > I am not explicitly doing chunking but it's possible it's doing this > > behind the scenes on larger documents. > > > > On Feb 13, 12:19 pm, Shay Banon [shay.ba...@elasticsearch.com](mailto:shay.ba...@elasticsearch.com) wrote: > > > > > Hey, > > > > > > This happens basically because there is no support for http chunk yet, > > > which I guess the http client you use does for large requests. > > > > > > Is there a chance that you can disable chunking for now? Which http > > > client are you using? > > > > > > -shay.banon > > > > > > On 2/13/10, Tim Dysinger [dysin...@gmail.com](mailto:dysin...@gmail.com) wrote: > > > > > > > I am rapidly pushing 750k documents to one elasticsearch node > > > > (localhost). About 3% of them fail to push over rest. The json is > > > > fine. I've pushed all these to CouchDB before. > > > > > > > The console just fills up with this > > > > > > > [11:55:54,225][WARN][http.netty] [Reilly, Ben] Caught > > > > exception while handling client http trafic > > > > java.lang.ClassCastException: > > > > org.jboss.netty.handler.codec.http.DefaultHttpChunk cannot be cast to > > > > org.jboss.netty.handler.codec.http.HttpRequest > > > > at > > > > org.elasticsearch.http.netty.HttpRequestHandler.messageReceived(HttpRequest > > Handler.java: > > > > > > 1. > > > > > > > > at > > > > org.jboss.netty.channel.Channels.fireMessageReceived(Channels.java: > > > > 391) > > > > at > > > > org.jboss.netty.handler.codec.replay.ReplayingDecoder.unfoldAndfireMessageR > > eceived(ReplayingDecoder.java: > > > > > > 1. > > > > > > > > at > > > > org.jboss.netty.handler.codec.replay.ReplayingDecoder.callDecode(ReplayingD > > ecoder.java: > > > > > > 1. > > > > > > > > at > > > > org.jboss.netty.handler.codec.replay.ReplayingDecoder.messageReceived(Repla > > yingDecoder.java: > > > > > > 1. > > > > > > > > at > > > > org.jboss.netty.handler.timeout.ReadTimeoutHandler.messageReceived(ReadTime > > outHandler.java: > > > > > > 1. > > > > > > > > at > > > > org.elasticsearch.http.netty.OpenChannelsHandler.handleUpstream(OpenChannel > > sHandler.java: > > > > > > 1. > > > > > > > > at > > > > org.jboss.netty.channel.Channels.fireMessageReceived(Channels.java: > > > > 345) > > > > at > > > > org.jboss.netty.channel.Channels.fireMessageReceived(Channels.java: > > > > 332) > > > > at > > > > org.jboss.netty.channel.socket.nio.NioWorker.read(NioWorker.java: > > > > 323) > > > > at > > > > org.jboss.netty.channel.socket.nio.NioWorker.processSelectedKeys(NioWorker. > > java: > > > > > > 1. > > > > > > > > at org.jboss.netty.channel.socket.nio.NioWorker.run(NioWorker.java: > > > > 196) > > > > at > > > > org.jboss.netty.util.internal.IoWorkerRunnable.run(IoWorkerRunnable.java: > > > > > > 1. > > > > > > > > at java.util.concurrent.ThreadPoolExecutor > > > > $Worker.runTask(ThreadPoolExecutor.java:886) > > > > at java.util.concurrent.ThreadPoolExecutor > > > > $Worker.run(ThreadPoolExecutor.java:908) > > > > at java.lang.Thread.run(Thread.java:619) > > > > > > -- > > > Sent from my mobile device ---
**Author:** ![Tim\_Dysinger\_2\_3](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/tim_dysinger_2_3/32/3365_2.png) [@Tim\_Dysinger\_2\_3](https://discuss.elastic.co/u/Tim_Dysinger_2_3)\ **Post date:** [February 14, 2010, 1:57am UTC](https://discuss.elastic.co/t/high-failure-rates-on-rest-indexing/2807/6 "2010-02-14T01:57:25Z")
Thanks. It works now. On Feb 13, 1:46 pm, Shay Banon [shay.ba...@elasticsearch.com](mailto:shay.ba...@elasticsearch.com) wrote: > Support for HTTP chunks has been added to master:[Issues · elastic/elasticsearch · GitHub](http://github.com/elasticsearch/elasticsearch/issues/closed#issue/8). You can > get it here:[http://www.elasticsearch.com/download/](http://www.elasticsearch.com/download/)(with instructions on > how to build it). > > -shay.banon > > On Sat, Feb 13, 2010 at 10:28 PM, Tim Dysinger [dysin...@gmail.com](mailto:dysin...@gmail.com) wrote: > > > Using this clienthttp://github.com/francois/rest-client > > > I am not explicitly doing chunking but it's possible it's doing this > > behind the scenes on larger documents. > > > On Feb 13, 12:19 pm, Shay Banon [shay.ba...@elasticsearch.com](mailto:shay.ba...@elasticsearch.com) wrote: > > > > > Hey, > > > > This happens basically because there is no support for http chunk yet, > > > which I guess the http client you use does for large requests. > > > > Is there a chance that you can disable chunking for now? Which http > > > client are you using? > > > > -shay.banon > > > > On 2/13/10, Tim Dysinger [dysin...@gmail.com](mailto:dysin...@gmail.com) wrote: > > > > > I am rapidly pushing 750k documents to one elasticsearch node > > > > (localhost). About 3% of them fail to push over rest. The json is > > > > fine. I've pushed all these to CouchDB before. > > > > > The console just fills up with this > > > > > [11:55:54,225][WARN][http.netty] [Reilly, Ben] Caught > > > > exception while handling client http trafic > > > > java.lang.ClassCastException: > > > > org.jboss.netty.handler.codec.http.DefaultHttpChunk cannot be cast to > > > > org.jboss.netty.handler.codec.http.HttpRequest > > > > at > > > org.elasticsearch.http.netty.HttpRequestHandler.messageReceived(HttpRequest > > Handler.java: > > > > > > 1. > > > > > > > > at > > > > org.jboss.netty.channel.Channels.fireMessageReceived(Channels.java: > > > > 391) > > > > at > > > org.jboss.netty.handler.codec.replay.ReplayingDecoder.unfoldAndfireMessageR > > eceived(ReplayingDecoder.java: > > > > > > 1. > > > > > > > > at > > > org.jboss.netty.handler.codec.replay.ReplayingDecoder.callDecode(ReplayingD > > ecoder.java: > > > > > > 1. > > > > > > > > at > > > org.jboss.netty.handler.codec.replay.ReplayingDecoder.messageReceived(Repla > > yingDecoder.java: > > > > > > 1. > > > > > > > > at > > > org.jboss.netty.handler.timeout.ReadTimeoutHandler.messageReceived(ReadTime > > outHandler.java: > > > > > > 1. > > > > > > > > at > > > org.elasticsearch.http.netty.OpenChannelsHandler.handleUpstream(OpenChannel > > sHandler.java: > > > > > > 1. > > > > > > > > at > > > > org.jboss.netty.channel.Channels.fireMessageReceived(Channels.java: > > > > 345) > > > > at > > > > org.jboss.netty.channel.Channels.fireMessageReceived(Channels.java: > > > > 332) > > > > at org.jboss.netty.channel.socket.nio.NioWorker.read(NioWorker.java: > > > > 323) > > > > at > > > org.jboss.netty.channel.socket.nio.NioWorker.processSelectedKeys(NioWorker. > > java: > > > > > > 1. > > > > > > > > at org.jboss.netty.channel.socket.nio.NioWorker.run(NioWorker.java: > > > > 196) > > > > at > > > org.jboss.netty.util.internal.IoWorkerRunnable.run(IoWorkerRunnable.java: > > > > > > 1. > > > > > > > > at java.util.concurrent.ThreadPoolExecutor > > > > $Worker.runTask(ThreadPoolExecutor.java:886) > > > > at java.util.concurrent.ThreadPoolExecutor > > > > $Worker.run(ThreadPoolExecutor.java:908) > > > > at java.lang.Thread.run(Thread.java:619) > > > > -- > > > Sent from my mobile device ---
**Author:** ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)\ **Post date:** [July 6, 2017, 4:25am UTC](https://discuss.elastic.co/t/high-failure-rates-on-rest-indexing/2807/7 "2017-07-06T04:25:49Z")