# How to debug NoNodeAvailableException **URL:** \ **Category:** Elasticsearch\ **Created:** [August 4, 2014, 8:31am UTC](https://discuss.elastic.co/t/how-to-debug-nonodeavailableexception/19063 "2014-08-04T08:31:02Z")\ **Posts on this page:** 5\ **Page:** 1
**Author:** ![Dennis\_de\_Boer](https://avatars.discourse-cdn.com/v4/letter/d/cab0a1/32.png) [@Dennis\_de\_Boer](https://discuss.elastic.co/u/Dennis_de_Boer)\ **Post date:** [August 4, 2014, 8:31am UTC](https://discuss.elastic.co/t/how-to-debug-nonodeavailableexception/19063/1 "2014-08-04T08:31:02Z")
Hi all, Hope you can give me some pointers on this topic. I'm trying to figure out what is going wrong in my setup/config but I cannot figure it out. I have two servers. Server A hosts a public website with the elasticsearch index. Server B retrieves XML productfeeds , parses the feeds, and adds/deletes/updates these products into the ES index on server A using a TransportClient. The problem: - this process takes place on 3:00 am (supposedly a quite time) - 9 out of 10 days I'll get a NoNodeAvailableException on a random point during the indexing of the records. - When I run the process during daytime (e.g. 10:00 am), everything works fine. My guess: In the access logs I see that a lot of bots are crawling my site around 3:00 am. Since the exception occurs randomly and only when the site is busy, it has to be a threading/connection problem My question: How can I debug this problem to figure out if it is a threading/jvm/memory/connection problem. I want to see some actual proof instead of guessing around. Any debug settings or plugins I can try to monitor the ES nodes? Any tips or pointers are most appreciated. Dennis -- You received this message because you are subscribed to the Google Groups "elasticsearch" group. To unsubscribe from this group and stop receiving emails from it, send an email to [elasticsearch+unsubscribe@googlegroups.com](mailto:elasticsearch+unsubscribe@googlegroups.com). To view this discussion on the web visit [https://groups.google.com/d/msgid/elasticsearch/dd42e89b-2c28-4991-a5ec-f79f80e05311%40googlegroups.com](https://groups.google.com/d/msgid/elasticsearch/dd42e89b-2c28-4991-a5ec-f79f80e05311%40googlegroups.com). For more options, visit [https://groups.google.com/d/optout](https://groups.google.com/d/optout). ---
**Author:** ![jprante](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/jprante/32/44941_2.png) [@jprante](https://discuss.elastic.co/u/jprante)\ **Post date:** [August 4, 2014, 8:50am UTC](https://discuss.elastic.co/t/how-to-debug-nonodeavailableexception/19063/2 "2014-08-04T08:50:14Z")
Can you show your code? Jörg On Mon, Aug 4, 2014 at 10:31 AM, Dennis de Boer [datdeboer@gmail.com](mailto:datdeboer@gmail.com) wrote: > Hi all, > > Hope you can give me some pointers on this topic. I'm trying to figure out > what is going wrong in my setup/config but I cannot figure it out. > > I have two servers. Server A hosts a public website with the elasticsearch > index. > Server B retrieves XML productfeeds , parses the feeds, and > adds/deletes/updates these products into the ES index on server A using a > TransportClient. > > The problem: > > - this process takes place on 3:00 am (supposedly a quite time) > - 9 out of 10 days I'll get a NoNodeAvailableException on a random point > during the indexing of the records. > - When I run the process during daytime (e.g. 10:00 am), everything works > fine. > > My guess: > In the access logs I see that a lot of bots are crawling my site around > 3:00 am. Since the exception occurs randomly and only when the site is > busy, it has to be a threading/connection problem > > My question: > How can I debug this problem to figure out if it is a > threading/jvm/memory/connection problem. I want to see some actual proof > instead of guessing around. > Any debug settings or plugins I can try to monitor the ES nodes? > > Any tips or pointers are most appreciated. > Dennis > > -- > You received this message because you are subscribed to the Google Groups > "elasticsearch" group. > To unsubscribe from this group and stop receiving emails from it, send an > email to [elasticsearch+unsubscribe@googlegroups.com](mailto:elasticsearch+unsubscribe@googlegroups.com). > To view this discussion on the web visit > [https://groups.google.com/d/msgid/elasticsearch/dd42e89b-2c28-4991-a5ec-f79f80e05311%40googlegroups.com](https://groups.google.com/d/msgid/elasticsearch/dd42e89b-2c28-4991-a5ec-f79f80e05311%40googlegroups.com) > [https://groups.google.com/d/msgid/elasticsearch/dd42e89b-2c28-4991-a5ec-f79f80e05311%40googlegroups.com?utm\_medium=email&utm\_source=footer](https://groups.google.com/d/msgid/elasticsearch/dd42e89b-2c28-4991-a5ec-f79f80e05311%40googlegroups.com?utm_medium=email&utm_source=footer) > . > For more options, visit [https://groups.google.com/d/optout](https://groups.google.com/d/optout). -- You received this message because you are subscribed to the Google Groups "elasticsearch" group. To unsubscribe from this group and stop receiving emails from it, send an email to [elasticsearch+unsubscribe@googlegroups.com](mailto:elasticsearch+unsubscribe@googlegroups.com). To view this discussion on the web visit [https://groups.google.com/d/msgid/elasticsearch/CAKdsXoEfu4V2oXb6nUPyn3\_OXyFf18a8VwsJm3dXyV7iJS%2BgnA%40mail.gmail.com](https://groups.google.com/d/msgid/elasticsearch/CAKdsXoEfu4V2oXb6nUPyn3_OXyFf18a8VwsJm3dXyV7iJS%2BgnA%40mail.gmail.com). For more options, visit [https://groups.google.com/d/optout](https://groups.google.com/d/optout). ---
**Author:** ![Dennis\_de\_Boer](https://avatars.discourse-cdn.com/v4/letter/d/cab0a1/32.png) [@Dennis\_de\_Boer](https://discuss.elastic.co/u/Dennis_de_Boer)\ **Post date:** [August 4, 2014, 10:08am UTC](https://discuss.elastic.co/t/how-to-debug-nonodeavailableexception/19063/3 "2014-08-04T10:08:46Z")
Sure, along with some additional info. - I use the Java API within a Grails application - I use Elasticsearch version 0.90.5 ### code to create a Transportclient. Code is executed on server B, pointing to ES instance on Server A I tried to increase the timeout to check if this would help anything.....which isn't the case. Settings settings = ImmutableSettings.settingsBuilder() .put("cluster.name", "prodcluster") .put("discovery.zen.ping.multicast.enabled", false) .put("discovery.zen.ping.unicast.hosts", ["10.0.184.47"]) .put("client.transport.ping\_timeout","25s") .put("client.transport.nodes\_sampler\_interval","25s") .build(); ``` Client transportClient = new TransportClient(settings) .addTransportAddress(new ``` InetSocketTransportAddress("10.0.184.47", 9300)); ### then a lot of feed parsing magic happens, e.g. checking which products are new/updated and converting them into a structure which I can use for my application _Every_ record is then inserted using this piece of code UpdateResponse response = client.prepareUpdate(indexName, documentTypeName, product.internal\_id) .setDoc(product) .setUpsert(newProduct) .setId(product.internal\_id) .execute() .actionGet(); This update function is called more then 100.000 times. Once for every product in all the product feeds. Sometimes after ~2000 or ~3000 records I receive a NoNodeAvailableException Op maandag 4 augustus 2014 10:31:02 UTC+2 schreef Dennis de Boer: > Hi all, > > Hope you can give me some pointers on this topic. I'm trying to figure out > what is going wrong in my setup/config but I cannot figure it out. > > I have two servers. Server A hosts a public website with the elasticsearch > index. > Server B retrieves XML productfeeds , parses the feeds, and > adds/deletes/updates these products into the ES index on server A using a > TransportClient. > > The problem: > > - this process takes place on 3:00 am (supposedly a quite time) > - 9 out of 10 days I'll get a NoNodeAvailableException on a random point > during the indexing of the records. > - When I run the process during daytime (e.g. 10:00 am), everything works > fine. > > My guess: > In the access logs I see that a lot of bots are crawling my site around > 3:00 am. Since the exception occurs randomly and only when the site is > busy, it has to be a threading/connection problem > > My question: > How can I debug this problem to figure out if it is a > threading/jvm/memory/connection problem. I want to see some actual proof > instead of guessing around. > Any debug settings or plugins I can try to monitor the ES nodes? > > Any tips or pointers are most appreciated. > Dennis -- You received this message because you are subscribed to the Google Groups "elasticsearch" group. To unsubscribe from this group and stop receiving emails from it, send an email to [elasticsearch+unsubscribe@googlegroups.com](mailto:elasticsearch+unsubscribe@googlegroups.com). To view this discussion on the web visit [https://groups.google.com/d/msgid/elasticsearch/1d206366-b13e-479e-8bbe-d79b867ad64b%40googlegroups.com](https://groups.google.com/d/msgid/elasticsearch/1d206366-b13e-479e-8bbe-d79b867ad64b%40googlegroups.com). For more options, visit [https://groups.google.com/d/optout](https://groups.google.com/d/optout). ---
**Author:** ![Ivan](https://avatars.discourse-cdn.com/v4/letter/i/df788c/32.png) [@Ivan](https://discuss.elastic.co/u/Ivan)\ **Post date:** [August 4, 2014, 5:49pm UTC](https://discuss.elastic.co/t/how-to-debug-nonodeavailableexception/19063/4 "2014-08-04T17:49:13Z")
You should switch to using bulk indexing instead of indexing an individual documents. Also, considering switching off the refresh interval (set it to -1) for the duration of your bulk indexing. Cheers, Ivan On Mon, Aug 4, 2014 at 3:08 AM, Dennis de Boer [datdeboer@gmail.com](mailto:datdeboer@gmail.com) wrote: > Sure, along with some additional info. > > - I use the Java API within a Grails application > - I use Elasticsearch version 0.90.5 > > ### code to create a Transportclient. > > Code is executed on server B, pointing to ES instance on Server A > I tried to increase the timeout to check if this would help > anything.....which isn't the case. > > Settings settings = ImmutableSettings.settingsBuilder() > .put("cluster.name", "prodcluster") > .put("discovery.zen.ping.multicast.enabled", false) > .put("discovery.zen.ping.unicast.hosts", ["10.0.184.47"]) > .put("client.transport.ping\_timeout","25s") > .put("client.transport.nodes\_sampler\_interval","25s") > .build(); > > ``` > Client transportClient = new TransportClient(settings) > .addTransportAddress(new > > ``` > > InetSocketTransportAddress("10.0.184.47", 9300)); > > ### then a lot of feed parsing magic happens, e.g. checking which products > > are new/updated and converting them into a structure which I can use for my > application > > _Every_ record is then inserted using this piece of code > > UpdateResponse response = client.prepareUpdate(indexName, > documentTypeName, product.internal\_id) > .setDoc(product) > .setUpsert(newProduct) > .setId(product.internal\_id) > .execute() > .actionGet(); > > This update function is called more then 100.000 times. Once for every > product in all the product feeds. > Sometimes after ~2000 or ~3000 records I receive a NoNodeAvailableException > > Op maandag 4 augustus 2014 10:31:02 UTC+2 schreef Dennis de Boer: > > > Hi all, > > > > Hope you can give me some pointers on this topic. I'm trying to figure > > out what is going wrong in my setup/config but I cannot figure it out. > > > > I have two servers. Server A hosts a public website with the > > elasticsearch index. > > Server B retrieves XML productfeeds , parses the feeds, and > > adds/deletes/updates these products into the ES index on server A using a > > TransportClient. > > > > The problem: > > > > - this process takes place on 3:00 am (supposedly a quite time) > > - 9 out of 10 days I'll get a NoNodeAvailableException on a random point > > during the indexing of the records. > > - When I run the process during daytime (e.g. 10:00 am), everything works > > fine. > > > > My guess: > > In the access logs I see that a lot of bots are crawling my site around > > 3:00 am. Since the exception occurs randomly and only when the site is > > busy, it has to be a threading/connection problem > > > > My question: > > How can I debug this problem to figure out if it is a > > threading/jvm/memory/connection problem. I want to see some actual proof > > instead of guessing around. > > Any debug settings or plugins I can try to monitor the ES nodes? > > > > Any tips or pointers are most appreciated. > > Dennis > > > > -- > > You received this message because you are subscribed to the Google Groups > > "elasticsearch" group. > > To unsubscribe from this group and stop receiving emails from it, send an > > email to [elasticsearch+unsubscribe@googlegroups.com](mailto:elasticsearch+unsubscribe@googlegroups.com). > > To view this discussion on the web visit > > [https://groups.google.com/d/msgid/elasticsearch/1d206366-b13e-479e-8bbe-d79b867ad64b%40googlegroups.com](https://groups.google.com/d/msgid/elasticsearch/1d206366-b13e-479e-8bbe-d79b867ad64b%40googlegroups.com) > > [https://groups.google.com/d/msgid/elasticsearch/1d206366-b13e-479e-8bbe-d79b867ad64b%40googlegroups.com?utm\_medium=email&utm\_source=footer](https://groups.google.com/d/msgid/elasticsearch/1d206366-b13e-479e-8bbe-d79b867ad64b%40googlegroups.com?utm_medium=email&utm_source=footer) > > . > > For more options, visit [https://groups.google.com/d/optout](https://groups.google.com/d/optout). -- You received this message because you are subscribed to the Google Groups "elasticsearch" group. To unsubscribe from this group and stop receiving emails from it, send an email to [elasticsearch+unsubscribe@googlegroups.com](mailto:elasticsearch+unsubscribe@googlegroups.com). To view this discussion on the web visit [https://groups.google.com/d/msgid/elasticsearch/CALY%3DcQDGHFGWXT2OYDsY8cSiTSRUnc%2BSM8zq6rx0Dgif52tC1Q%40mail.gmail.com](https://groups.google.com/d/msgid/elasticsearch/CALY%3DcQDGHFGWXT2OYDsY8cSiTSRUnc%2BSM8zq6rx0Dgif52tC1Q%40mail.gmail.com). For more options, visit [https://groups.google.com/d/optout](https://groups.google.com/d/optout). ---
**Author:** ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)\ **Post date:** [July 6, 2017, 1:10am UTC](https://discuss.elastic.co/t/how-to-debug-nonodeavailableexception/19063/5 "2017-07-06T01:10:54Z")