# Need Help --- How to Handle "org.elasticsearch.transport.ConnectTransportException" in Java Application **URL:** \ **Category:** Elasticsearch\ **Created:** [August 7, 2013, 7:59am UTC](https://discuss.elastic.co/t/need-help-how-to-handle-org-elasticsearch-transport-connecttransportexception-in-java-application/13102 "2013-08-07T07:59:49Z")\ **Posts on this page:** 4\ **Page:** 1
**Author:** ![mohammad\_rafi\_g](https://avatars.discourse-cdn.com/v4/letter/m/58956e/32.png) [@mohammad\_rafi\_g](https://discuss.elastic.co/u/mohammad_rafi_g)\ **Post date:** [August 7, 2013, 7:59am UTC](https://discuss.elastic.co/t/need-help-how-to-handle-org-elasticsearch-transport-connecttransportexception-in-java-application/13102/1 "2013-08-07T07:59:49Z")
Hi to All, We have 4 nodes[1 Master Node+1 client Node + 2 Data Nodes] and the settings of each node is: each node heap size is 5gb["ES\_MIN\_MEM" and "ES\_MAX\_MEM" are 5gb] and in elasticsearch.yml file bootstrap.mlockall: true index.store.compress.stored: true index.refresh\_interval: 20s indices.memory.index\_buffer\_ size: 50% index.translog.flush\_threshold\_ops: 50000 threadpool.bulk.type: fixed threadpool.bulk.size: 90 ``` The 4 nodes are running under same cluster, and every thing is ``` working fine for bulk loading. But When we are trying to query the elasticsearch data, while a bulk loading happening in background, one node got down out of 4 nodes. and we got the following Exception at java application log file., ................................................................................................................................................................ 05.08.2013 18:18:00 DEBUG (Log4jESLogger.java:99) - [Thing] failed to connect to node [[#transport#-3][inet[/10.14.133.89:9313]]], removed from nodes list org.elasticsearch.transport.ConnectTransportException: [][inet[/ xxx.xxx.xxx.xxx:9304 [http://10.14.133.89:9313](http://10.14.133.89:9313)]] connect\_timeout[30s] at org.elasticsearch.transport.netty.NettyTransport.connectToChannels(NettyTransport.java:683) at org.elasticsearch.transport.netty.NettyTransport.connectToNode(NettyTransport.java:604) at org.elasticsearch.transport.netty.NettyTransport.connectToNode(NettyTransport.java:574) at org.elasticsearch.transport.TransportService.connectToNode(TransportService.java:127) at org.elasticsearch.client.transport.TransportClientNodesService$SimpleNodeSampler.sample(TransportClientNodesService.java:302) at org.elasticsearch.client.transport.TransportClientNodesService$ScheduledNodeSampler.run(TransportClientNodesService.java:281) at java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) at java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) at java.lang.Thread.run(Unknown Source) Caused by: java.net.ConnectException: connection timed out at org.elasticsearch.common.netty.channel.socket.nio.NioClientBoss.processConnectTimeout(NioClientBoss.java:136) at org.elasticsearch.common.netty.channel.socket.nio.NioClientBoss.process(NioClientBoss.java:82) at org.elasticsearch.common.netty.channel.socket.nio.AbstractNioSelector.run(AbstractNioSelector.java:312) at org.elasticsearch.common.netty.channel.socket.nio.NioClientBoss.run(NioClientBoss.java:41) at org.elasticsearch.common.netty.util.ThreadRenamingRunnable.run(ThreadRenamingRunnable.java:108) at org.elasticsearch.common.netty.util.internal.DeadLockProofWorker$1.run(DeadLockProofWorker.java:42) ... 3 more ................................................................................................................................................................ And at elasticsearch logs we found the following... at Data Node: "java.lang.OutOfMemoryError: Java heap space" and " java.io.StreamCorruptedException" at Master Node: "org.elasticsearch.transport.NodeDisconnectedException", "org.elasticsearch.indices.IndexMissingException: [20130730] missing" and "org.elasticsearch.index.IndexShardMissingException: [20130727][6] missing" ................................................................................................................................................................ Now i want to handle this "org.elasticsearch.transport.ConnectTransportException" Exception in my java Application. so i did the following.. //in java application this Exception is triggered by this Client[when node is got down]object for every 5 seconds, so i added try-catch block for this Client object. Client client = null; try{ //here we are assigning the TransportClient object[of data nodes]. client = getClient(EDRPropertyBean.getMapClients()); }catch(org.elasticsearch.transport.ConnectTransportException cTE){ log.error("Some Exception message"); // here we have Exception Handle Logic } But This Exception Handling is not working, still we are getting the same Exception instead of our Exception Handle Logic. 1. Please Tell me What I have to Do for Catch the "org.elasticsearch.transport.ConnectTransportException" Exception so that I will run my Exception Handle Logic to handle the Node-Down[means, node is down]. and also 2. Please tell me if any other settings require for these nodes to handle the both querying and bulk loading at a time. Thanks and Regards G Md Rafi -- You received this message because you are subscribed to the Google Groups "elasticsearch" group. To unsubscribe from this group and stop receiving emails from it, send an email to [elasticsearch+unsubscribe@googlegroups.com](mailto:elasticsearch+unsubscribe@googlegroups.com). For more options, visit [https://groups.google.com/groups/opt\_out](https://groups.google.com/groups/opt_out). ---
**Author:** ![spinscale](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/spinscale/32/25011_2.png) [@spinscale](https://discuss.elastic.co/u/spinscale)\ **Post date:** [August 7, 2013, 8:26am UTC](https://discuss.elastic.co/t/need-help-how-to-handle-org-elasticsearch-transport-connecttransportexception-in-java-application/13102/2 "2013-08-07T08:26:53Z")
Hey, the problem is not the ConnectException, but the OutOfMemoryError entry. This means you fired a query which resulted in loading too much data into the memory of the JVM. The simplest solution is to give more memory to the JVM if you have it. The more tedious solution is to analyze your query and see if you are doing something wrong (like facetting on an analyzed field for example). Hard to tell without more information. Hope this helps --Alex On Wed, Aug 7, 2013 at 9:59 AM, rafi [me.mrafi.en@gmail.com](mailto:me.mrafi.en@gmail.com) wrote: > Hi to All, > We have 4 nodes[1 Master Node+1 client Node + 2 Data Nodes] and > the settings of each node is: > each node heap size is 5gb["ES\_MIN\_MEM" and "ES\_MAX\_MEM" > are 5gb] > and in elasticsearch.yml file > bootstrap.mlockall: true > index.store.compress.stored: true > index.refresh\_interval: 20s > indices.memory.index\_buffer\_ > \*\*size: 50% > index.translog.flush\_\*\*threshold\_ops: 50000 > threadpool.bulk.type: fixed > threadpool.bulk.size: 90 > > ``` > The 4 nodes are running under same cluster, and every thing is > > ``` > > working fine for bulk loading. But > When we are trying to query the elasticsearch data, while a bulk > loading happening in background, one node got down out of 4 nodes. and we > got the following Exception at java application log file., > .............................. **..............................** > .............................. **..............................** > ..............................\*\*.......... > 05.08.2013 18:18:00 DEBUG (Log4jESLogger.java:99) - [Thing] failed to > connect to node [[#transport#-3][inet[/10.14.\*\*133.89:9313]]], removed > from nodes list > org.elasticsearch.transport.\*\*ConnectTransportException: [inet[/ > xxx.xxx.xxx.xxx:9304 [http://10.14.133.89:9313](http://10.14.133.89:9313)]] connect\_timeout[30s] > at org.elasticsearch.transport. **netty.NettyTransport.** > connectToChannels(\*\*NettyTransport.java:683) > at org.elasticsearch.transport. **netty.NettyTransport.** > connectToNode(NettyTransport.\*\*java:604) > at org.elasticsearch.transport. **netty.NettyTransport.** > connectToNode(NettyTransport.\*\*java:574) > at org.elasticsearch.transport.\*\*TransportService.**connectToNode(** > TransportService.java:127) > at org.elasticsearch.client.\*\*transport.\*\*TransportClientNodesService$ > \*\*SimpleNodeSampler.sample(\*\*TransportClientNodesService.\*\*java:302) > at org.elasticsearch.client.\*\*transport.\*\*TransportClientNodesService$ > \*\*ScheduledNodeSampler.run(\*\*TransportClientNodesService.\*\*java:281) > at java.util.concurrent.\*\*ThreadPoolExecutor.runWorker(\*\*Unknown > Source) > at java.util.concurrent.\*\*ThreadPoolExecutor$Worker.run(\*\*Unknown > Source) > at java.lang.Thread.run(Unknown Source) > Caused by: java.net.ConnectException: connection timed out > at org.elasticsearch.common. **netty.channel.socket.nio.** > NioClientBoss.\*\*processConnectTimeout(\*\*NioClientBoss.java:136) > at org.elasticsearch.common. **netty.channel.socket.nio.** > NioClientBoss.process(\*\*NioClientBoss.java:82) > at org.elasticsearch.common. **netty.channel.socket.nio.** > AbstractNioSelector.run(\*\*AbstractNioSelector.java:312) > at org.elasticsearch.common. **netty.channel.socket.nio.** > NioClientBoss.run(\*\*NioClientBoss.java:41) > at org.elasticsearch.common.**netty.util.ThreadRenamingRunnable.run( > ThreadRenamingRunnable.java:108) > at org.elasticsearch.common.netty.util.internal. > DeadLockProofWorker$1.run(DeadLockProofWorker.java:42) > ... 3 more > ............................................................ > ............................................................ > ..............................**.......... > And at elasticsearch logs we found the following... > at Data Node: "java.lang.OutOfMemoryError: Java heap space" and > " [java.io](http://java.io).\*\*StreamCorruptedException" > at Master Node: "org.elasticsearch.transport.\*\*NodeDisconnectedException", > > ``` > "org.elasticsearch.indices.**IndexMissingException: > > ``` > > [20130730] missing" and > "org.elasticsearch.index.**IndexShardMissingException: > [20130727][6] missing" > ..............................**..............................\*\* > .............................. **..............................** > ..............................\*\*.......... > > Now i want to handle this "org.elasticsearch.transport.**ConnectTransportException" > Exception in my java Application. > so i did the following.. > //in java application this Exception is triggered by this Client[when node > is got down]object for every 5 seconds, so i added try-catch block for this > Client object. > Client client = null; > try{ > //here we are assigning the TransportClient object[of data > nodes]. > client = getClient(EDRPropertyBean.** > getMapClients()); \*\* > }catch(org.elasticsearch.\*\*transport.\*\*ConnectTransportException > cTE){ > log.error("Some Exception message"); > // here we have Exception Handle Logic > } > But This Exception Handling is not working, still we are getting the same > Exception instead of our Exception Handle Logic. > > 1. Please Tell me What I have to Do for Catch the > "org.elasticsearch.transport.\*\*ConnectTransportException" Exception > so that I will run my Exception Handle Logic to handle the > Node-Down[means, node is down]. and also > 2. Please tell me if any other settings require for these nodes to handle > the both querying and bulk loading at a time. > > Thanks and Regards > G Md Rafi > > -- > You received this message because you are subscribed to the Google Groups > "elasticsearch" group. > To unsubscribe from this group and stop receiving emails from it, send an > email to [elasticsearch+unsubscribe@googlegroups.com](mailto:elasticsearch+unsubscribe@googlegroups.com). > For more options, visit [https://groups.google.com/groups/opt\_out](https://groups.google.com/groups/opt_out). -- You received this message because you are subscribed to the Google Groups "elasticsearch" group. To unsubscribe from this group and stop receiving emails from it, send an email to [elasticsearch+unsubscribe@googlegroups.com](mailto:elasticsearch+unsubscribe@googlegroups.com). For more options, visit [https://groups.google.com/groups/opt\_out](https://groups.google.com/groups/opt_out). ---
**Author:** ![mohammad\_rafi\_g](https://avatars.discourse-cdn.com/v4/letter/m/58956e/32.png) [@mohammad\_rafi\_g](https://discuss.elastic.co/u/mohammad_rafi_g)\ **Post date:** [August 7, 2013, 9:13am UTC](https://discuss.elastic.co/t/need-help-how-to-handle-org-elasticsearch-transport-connecttransportexception-in-java-application/13102/3 "2013-08-07T09:13:11Z")
Hi Alex., ``` Thanks for Replying.. ``` As per you Reply, > The more tedious solution is to analyze your query and see if you are > doing something wrong (like facetting on an analyzed field for example). > Hard to tell without more information. > What We have done is, we tested the load balance and > performance of the elasticsearch documents. > For this we used jmeter. we run this test with 100 threads for > 300seconds. > We Queried all indices[20 indices with 500 million documets]. We ran > this jmeter query test when bulk index operation is going on. ``` Here we used simple query with routing. like: ``` http://:///\_search?q=:&routing=&pretty=true ``` We need to handle the index operation with out any data loss when ``` any data node got down. So fro this, by handling the "ConnectTransportException" or "ConnectException" in java application[this java application loads the data into elasticsearch] so that when ever a node got down we need to stop the indexing operation. ``` I have tried to catch these exceptions but i am not able to catch ``` these Exceptions. 1. Please tell me, if any one know, how to catch this Exceptions[ "ConnectTransportException" or "ConnectException"] in java application. > the problem is not the ConnectException, but the OutOfMemoryError entry. > This means you fired a query which resulted in loading too much data into > the memory of the JVM > Yes, the node got down due to OutofMemory Error. As of now, > we have 5gb for each node. > 2. please tell me, for indexing and querying on huge data[say, above > 100 million douments] what amount of jvm should i give to each node?[we > have 128gb of RAM] Thanks and Regards G Md Rafi -- You received this message because you are subscribed to the Google Groups "elasticsearch" group. To unsubscribe from this group and stop receiving emails from it, send an email to [elasticsearch+unsubscribe@googlegroups.com](mailto:elasticsearch+unsubscribe@googlegroups.com). For more options, visit [https://groups.google.com/groups/opt\_out](https://groups.google.com/groups/opt_out). ---
**Author:** ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)\ **Post date:** [July 6, 2017, 2:22am UTC](https://discuss.elastic.co/t/need-help-how-to-handle-org-elasticsearch-transport-connecttransportexception-in-java-application/13102/4 "2017-07-06T02:22:28Z")