# Connection time out for indexing request - ES 1.0.2 **URL:** \ **Category:** Elasticsearch\ **Created:** [March 14, 2017, 9:39am UTC](https://discuss.elastic.co/t/connection-time-out-for-indexing-request-es-1-0-2/78485 "2017-03-14T09:39:47Z")\ **Posts on this page:** 7\ **Page:** 1
**Author:** ![bneelima84](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/bneelima84/32/28009_2.png) [@bneelima84](https://discuss.elastic.co/u/bneelima84)\ **Post date:** [March 14, 2017, 9:39am UTC](https://discuss.elastic.co/t/connection-time-out-for-indexing-request-es-1-0-2/78485/1 "2017-03-14T09:39:47Z")
We are facing timeouts while trying to add a document to the index intermittently. We are using elasticsearch 1.0.2 in embedded mode; 2 nodes are configured as data nodes. Any ideas on what could cause connection timeouts intermittently. Also the below stacktrace is observed after 15 min delay of issuing the request; Why is it not honoring the 1m timeout that's configured by default ? ``` org.elasticsearch.action.UnavailableShardsException: [xyz][0] [2] shardIt, [2] active : Timeout waiting for [-869752], request: index {[xyz] at org.elasticsearch.action.support.replication.TransportShardReplicationOperationAction$AsyncShardOperationAction.raiseTimeoutFailure(TransportShardReplicationOperationAction.java:548) at org.elasticsearch.action.support.replication.TransportShardReplicationOperationAction$AsyncShardOperationAction.retry(TransportShardReplicationOperationAction.java:496) at org.elasticsearch.action.support.replication.TransportShardReplicationOperationAction$AsyncShardOperationAction$2.handleException(TransportShardReplicationOperationAction.java:466) at org.elasticsearch.transport.TransportService$Adapter$2$1.run(TransportService.java:316) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617) at java.lang.Thread.run(Thread.java:745) ``` Configuration details: Server being used: JBoss 6.4 EAP Heap size: 4GB JVM Information: Vendor: Oracle Corporation, JVM version: 1.8.0\_77 VM Name: Java HotSpot(TM) 64-Bit Server VM(build 1.8.0\_77-b03) Host OS Information: OS: Linux, version: 2.6.32-642.3.1.el6.x86\_64 Architecture: amd64 ---
**Author:** ![bneelima84](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/bneelima84/32/28009_2.png) [@bneelima84](https://discuss.elastic.co/u/bneelima84)\ **Post date:** [March 15, 2017, 3:51am UTC](https://discuss.elastic.co/t/connection-time-out-for-indexing-request-es-1-0-2/78485/2 "2017-03-15T03:51:36Z")
When we tried with only one index node, this behavior is observed again the the case where - Index request comes from a client node But if the indexing request is submitted to the index node, we don't see any issues. One more observation is while adding each document, we try to delete it first and then re-add it. When the request comes from a client node (non-data node), the delete-by-query request log is seen on the index node, however the subsequent addition to index log is not displayed and gets timed out. The same scenario on index node displays both delete-by-query and index request being executed. Any ideas on why the index request seems to take longer time while trying to communicate from other node ? ---
**Author:** ![bneelima84](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/bneelima84/32/28009_2.png) [@bneelima84](https://discuss.elastic.co/u/bneelima84)\ **Post date:** [March 16, 2017, 5:15am UTC](https://discuss.elastic.co/t/connection-time-out-for-indexing-request-es-1-0-2/78485/3 "2017-03-16T05:15:29Z")
Seems similar to [Random node disconnects - Java.io.IOException: Connection timed out](https://discuss.elastic.co/t/random-node-disconnects-java-io-ioexception-connection-timed-out/33151) ---
**Author:** ![dadoonet](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/dadoonet/32/137187_2.png) [@dadoonet](https://discuss.elastic.co/u/dadoonet)\ **Post date:** [March 16, 2017, 8:50am UTC](https://discuss.elastic.co/t/connection-time-out-for-indexing-request-es-1-0-2/78485/4 "2017-03-16T08:50:12Z")
At least try with a supported version like 2.4 or better 5.2. I don't think we can really help on so old ones. If you can reproduce such behavior on recent versions we'll be happy to help. ---
**Author:** ![bneelima84](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/bneelima84/32/28009_2.png) [@bneelima84](https://discuss.elastic.co/u/bneelima84)\ **Post date:** [March 22, 2017, 3:38am UTC](https://discuss.elastic.co/t/connection-time-out-for-indexing-request-es-1-0-2/78485/5 "2017-03-22T03:38:54Z")
We tried to turn off scatter-gather I/O but that didn't help us; When we looked at the TCP dumps, there was a packet loss happening in between the nodes. Modifying TCP\_KEEPALIVE settings has helped in our scenario. ---
**Author:** ![dadoonet](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/dadoonet/32/137187_2.png) [@dadoonet](https://discuss.elastic.co/u/dadoonet)\ **Post date:** [March 22, 2017, 6:05am UTC](https://discuss.elastic.co/t/connection-time-out-for-indexing-request-es-1-0-2/78485/6 "2017-03-22T06:05:59Z")
Thanks for sharing your findings! That can help others. ---
**Author:** ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)\ **Post date:** [April 19, 2017, 6:06am UTC](https://discuss.elastic.co/t/connection-time-out-for-indexing-request-es-1-0-2/78485/7 "2017-04-19T06:06:10Z")
This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.