Randomly no response for search query


(Timmy Ziesenhenne) #1

We use elasticsearch since a couple of weeks and there are over 160 million
of entries separated in a couple of indexes splitted by month (e.g.
index_2014_01,index_2014_02,....).
Sometimes if we send a bulk query requests to the elasticsearch nodes for
some requests there is no responses and no timeouts. There is no chance to
reproduce this problem because it happens randomly and also on querys wich
works seconds before with a minimum response time. We started to debug down
to the elasticsearch transport client without success. We noticed that
there was a request without response (Breakpoints where hooked in
response-listener, request-listener and failure-listener) - even after a
long wait time there was no response or failure. There is no exception or
log-output which indicates that problem except that after a time there is a
"disconnected from node" message.
There is not much load on the elasticsearch cluster and on our firewall all
ports from 9200 to 9400 are open

Our application structure:
TOMCAT 7 (JAVAEE-7u25) --> spring-data-elasticsearch-1.0.0.M2 -->
elasticsearch-1.1.0(1.0.0 with same issue) ----------> elasticsearch (8
nodes) version 1.0.0 (JAVA-7u25)

This issue happens randomly and we have no idea where it comes from.

--
You received this message because you are subscribed to the Google Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email to elasticsearch+unsubscribe@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/elasticsearch/ae9c972b-6fae-4cf2-8a0c-d43143a09ac2%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.


(Alexander Reelsen) #2

Hey,

very hard to tell what has been happening here without more information.
There has been a fix for this added recently, see

However this only happens when you get rejected requests in your thread
pools, is this the case? You can check with the nodes stats API.

--Alex

On Mon, Apr 28, 2014 at 12:03 PM, Timmy Ziesenhenne <
timmy.ziesenhenne@gmail.com> wrote:

We use elasticsearch since a couple of weeks and there are over 160
million of entries separated in a couple of indexes splitted by month (e.g.
index_2014_01,index_2014_02,....).
Sometimes if we send a bulk query requests to the elasticsearch nodes for
some requests there is no responses and no timeouts. There is no chance to
reproduce this problem because it happens randomly and also on querys wich
works seconds before with a minimum response time. We started to debug down
to the elasticsearch transport client without success. We noticed that
there was a request without response (Breakpoints where hooked in
response-listener, request-listener and failure-listener) - even after a
long wait time there was no response or failure. There is no exception or
log-output which indicates that problem except that after a time there is a
"disconnected from node" message.
There is not much load on the elasticsearch cluster and on our firewall
all ports from 9200 to 9400 are open

Our application structure:
TOMCAT 7 (JAVAEE-7u25) --> spring-data-elasticsearch-1.0.0.M2 -->
elasticsearch-1.1.0(1.0.0 with same issue) ----------> elasticsearch (8
nodes) version 1.0.0 (JAVA-7u25)

This issue happens randomly and we have no idea where it comes from.

--
You received this message because you are subscribed to the Google Groups
"elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an
email to elasticsearch+unsubscribe@googlegroups.com.
To view this discussion on the web visit
https://groups.google.com/d/msgid/elasticsearch/ae9c972b-6fae-4cf2-8a0c-d43143a09ac2%40googlegroups.comhttps://groups.google.com/d/msgid/elasticsearch/ae9c972b-6fae-4cf2-8a0c-d43143a09ac2%40googlegroups.com?utm_medium=email&utm_source=footer
.
For more options, visit https://groups.google.com/d/optout.

--
You received this message because you are subscribed to the Google Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email to elasticsearch+unsubscribe@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/elasticsearch/CAGCwEM-udpsEDHXpSb%3DKAB6y1y31dwT5eSz9jbFhk5Z-zmhq3Q%40mail.gmail.com.
For more options, visit https://groups.google.com/d/optout.


(system) #3