TransportClient timeout / webserver configuration - JAVA Api

roblangenfeld · March 10, 2014, 1:42pm

Hello,

I'm developing a tomcat webserver application that uses ElasticSearch 1.0
(Java API). There is a client facing desktop application that communicates
with the server so all the code for ElasticSearch is on that one instance
and it is used by all our clients. With that being said I am running into
this issue: After initializing a new TransportClient object and performing
some operation on it, there is a chance that i could sit idle for a very
long time. When does sit idle for a long time it gets this error:

Mar 08, 2014 1:15:37 AM org.elasticsearch.client.transport

INFO: [Elven] failed to get node info for
[#transport#-1][WIN7-113-00726][inet[/159.140.213.87:9300]],
disconnecting...

org.elasticsearch.transport.RemoteTransportException:
[Server_Dev1][inet[/159.140.213.87:9300]][cluster/nodes/info]

Caused by: java.lang.NullPointerException

at org.elasticsearch.http.HttpInfo.writeTo(HttpInfo.java:82)

at
org.elasticsearch.action.admin.cluster.node.info.NodeInfo.writeTo(NodeInfo.java:301)

at
org.elasticsearch.action.admin.cluster.node.info.NodesInfoResponse.writeTo(NodesInfoResponse.java:63)

at
org.elasticsearch.transport.netty.NettyTransportChannel.sendResponse(NettyTransportChannel.java:83)

at
org.elasticsearch.action.support.nodes.TransportNodesOperationAction$TransportHandler$1.onResponse(TransportNodesOperationAction.java:244)

at
org.elasticsearch.action.support.nodes.TransportNodesOperationAction$TransportHandler$1.onResponse(TransportNodesOperationAction.java:239)

at
org.elasticsearch.action.support.nodes.TransportNodesOperationAction$AsyncAction.finishHim(TransportNodesOperationAction.java:225)

at
org.elasticsearch.action.support.nodes.TransportNodesOperationAction$AsyncAction.onOperation(TransportNodesOperationAction.java:200)

at
org.elasticsearch.action.support.nodes.TransportNodesOperationAction$AsyncAction.access$900(TransportNodesOperationAction.java:102)

at
org.elasticsearch.action.support.nodes.TransportNodesOperationAction$AsyncAction$2.run(TransportNodesOperationAction.java:146)

at
java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)

at
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)

at java.lang.Thread.run(Thread.java:744)

Is there any way to prevent this from happening? I know the ideal situation
would be that after every request the transport client is closed. But since
it lives on a webserver with lots of search requests coming in, we would
ideally like it to stay open because it takes 3-4 seconds for a transport
client to initialize and we are going for speed here.

Also since we are having one central server to handle all search and index
requests, can the TransportClient handle multiple simultaneous requests
from different users at the same time? We just want to make sure that we
are doing this correctly.

--
You received this message because you are subscribed to the Google Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email to elasticsearch+unsubscribe@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/elasticsearch/cb489c01-1e1a-400f-8d46-95db9ba782e3%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

Thomas_Bolis · March 10, 2014, 1:46pm

As per the documentation Client is threadsafe and is suggested by the
elasticsearch team to use the same client instance across your app.
Considering your exception above you might need to look your configuration
first (like cluster name and host/port) and you should use port 9300 for
the Java API. Finally, check whether you are under a firewall or something
and you block 9300 port.

Hope it helps
Thomas

On Monday, 10 March 2014 15:42:25 UTC+2, Robert Langenfeld wrote:

Hello,

I'm developing a tomcat webserver application that uses Elasticsearch 1.0
(Java API). There is a client facing desktop application that communicates
with the server so all the code for Elasticsearch is on that one instance
and it is used by all our clients. With that being said I am running into
this issue: After initializing a new TransportClient object and performing
some operation on it, there is a chance that i could sit idle for a very
long time. When does sit idle for a long time it gets this error:

Mar 08, 2014 1:15:37 AM org.elasticsearch.client.transport

INFO: [Elven] failed to get node info for
[#transport#-1][WIN7-113-00726][inet[/159.140.213.87:9300]],
disconnecting...

org.elasticsearch.transport.RemoteTransportException:
[Server_Dev1][inet[/159.140.213.87:9300]][cluster/nodes/info]

Caused by: java.lang.NullPointerException

at org.elasticsearch.http.HttpInfo.writeTo(HttpInfo.java:82)

at
org.elasticsearch.action.admin.cluster.node.info.NodeInfo.writeTo(NodeInfo.java:301)

at
org.elasticsearch.action.admin.cluster.node.info.NodesInfoResponse.writeTo(NodesInfoResponse.java:63)

at
org.elasticsearch.transport.netty.NettyTransportChannel.sendResponse(NettyTransportChannel.java:83)

at
org.elasticsearch.action.support.nodes.TransportNodesOperationAction$TransportHandler$1.onResponse(TransportNodesOperationAction.java:244)

at
org.elasticsearch.action.support.nodes.TransportNodesOperationAction$TransportHandler$1.onResponse(TransportNodesOperationAction.java:239)

at
org.elasticsearch.action.support.nodes.TransportNodesOperationAction$AsyncAction.finishHim(TransportNodesOperationAction.java:225)

at
org.elasticsearch.action.support.nodes.TransportNodesOperationAction$AsyncAction.onOperation(TransportNodesOperationAction.java:200)

at
org.elasticsearch.action.support.nodes.TransportNodesOperationAction$AsyncAction.access$900(TransportNodesOperationAction.java:102)

at
org.elasticsearch.action.support.nodes.TransportNodesOperationAction$AsyncAction$2.run(TransportNodesOperationAction.java:146)

at
java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)

at
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)

at java.lang.Thread.run(Thread.java:744)

Is there any way to prevent this from happening? I know the ideal
situation would be that after every request the transport client is closed.
But since it lives on a webserver with lots of search requests coming in,
we would ideally like it to stay open because it takes 3-4 seconds for a
transport client to initialize and we are going for speed here.

Also since we are having one central server to handle all search and index
requests, can the TransportClient handle multiple simultaneous requests
from different users at the same time? We just want to make sure that we
are doing this correctly.

--
You received this message because you are subscribed to the Google Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email to elasticsearch+unsubscribe@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/elasticsearch/d625d553-2ed5-456a-9180-7b423874b43e%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

roblangenfeld · March 10, 2014, 5:18pm

Thomas,

We can connect and do queries and index processes just fine. The problem we
are experiencing is that after a long time of idle (15 minutes or so) doing
nothing, when we go to perform another operation this occurs.

On Monday, March 10, 2014 8:46:53 AM UTC-5, Thomas wrote:

As per the documentation Client is threadsafe and is suggested by the
elasticsearch team to use the same client instance across your app.
Considering your exception above you might need to look your configuration
first (like cluster name and host/port) and you should use port 9300 for
the Java API. Finally, check whether you are under a firewall or something
and you block 9300 port.

Hope it helps
Thomas

On Monday, 10 March 2014 15:42:25 UTC+2, Robert Langenfeld wrote:

Hello,

I'm developing a tomcat webserver application that uses Elasticsearch 1.0
(Java API). There is a client facing desktop application that communicates
with the server so all the code for Elasticsearch is on that one instance
and it is used by all our clients. With that being said I am running into
this issue: After initializing a new TransportClient object and performing
some operation on it, there is a chance that i could sit idle for a very
long time. When does sit idle for a long time it gets this error:

Mar 08, 2014 1:15:37 AM org.elasticsearch.client.transport

INFO: [Elven] failed to get node info for
[#transport#-1][WIN7-113-00726][inet[/159.140.213.87:9300]],
disconnecting...

org.elasticsearch.transport.RemoteTransportException:
[Server_Dev1][inet[/159.140.213.87:9300]][cluster/nodes/info]

Caused by: java.lang.NullPointerException

at org.elasticsearch.http.HttpInfo.writeTo(HttpInfo.java:82)

at
org.elasticsearch.action.admin.cluster.node.info.NodeInfo.writeTo(NodeInfo.java:301)

at
org.elasticsearch.action.admin.cluster.node.info.NodesInfoResponse.writeTo(NodesInfoResponse.java:63)

at
org.elasticsearch.transport.netty.NettyTransportChannel.sendResponse(NettyTransportChannel.java:83)

at
org.elasticsearch.action.support.nodes.TransportNodesOperationAction$TransportHandler$1.onResponse(TransportNodesOperationAction.java:244)

at
org.elasticsearch.action.support.nodes.TransportNodesOperationAction$TransportHandler$1.onResponse(TransportNodesOperationAction.java:239)

at
org.elasticsearch.action.support.nodes.TransportNodesOperationAction$AsyncAction.finishHim(TransportNodesOperationAction.java:225)

at
org.elasticsearch.action.support.nodes.TransportNodesOperationAction$AsyncAction.onOperation(TransportNodesOperationAction.java:200)

at
org.elasticsearch.action.support.nodes.TransportNodesOperationAction$AsyncAction.access$900(TransportNodesOperationAction.java:102)

at
org.elasticsearch.action.support.nodes.TransportNodesOperationAction$AsyncAction$2.run(TransportNodesOperationAction.java:146)

at
java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)

at
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)

at java.lang.Thread.run(Thread.java:744)

Is there any way to prevent this from happening? I know the ideal
situation would be that after every request the transport client is closed.
But since it lives on a webserver with lots of search requests coming in,
we would ideally like it to stay open because it takes 3-4 seconds for a
transport client to initialize and we are going for speed here.

Also since we are having one central server to handle all search and
index requests, can the TransportClient handle multiple simultaneous
requests from different users at the same time? We just want to make sure
that we are doing this correctly.

--
You received this message because you are subscribed to the Google Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email to elasticsearch+unsubscribe@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/elasticsearch/760f6a9f-a917-494e-9328-c1b3e9877cae%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

jprante · March 10, 2014, 7:54pm

Check the DNS of the ES servers. Netty initializes DNS services to get the
publish address, which is used for setting the bound address, and this
network call seems to take a long time (>30secs) so the bound address is
null for a while.

If your DNS or network is misconfigured, a DNS call will stall for a long
time. During this time, no node ping request/response should occur, since
this will trigger an NPE in NettyHttpServerTransport.

You could also increase the default node ping timeout
discovery.zen.ping.timeout which is by default 30 secs (already very high).

Another fix might be, I have all my ES servers names/IPs also registered in
/etc/hosts, so OS-based server name resolution can not play tricks on Java.

Jörg

On Mon, Mar 10, 2014 at 6:18 PM, Robert Langenfeld
roblangenfeld@gmail.comwrote:

Thomas,

We can connect and do queries and index processes just fine. The problem
we are experiencing is that after a long time of idle (15 minutes or so)
doing nothing, when we go to perform another operation this occurs.

On Monday, March 10, 2014 8:46:53 AM UTC-5, Thomas wrote:

As per the documentation Client is threadsafe and is suggested by the
elasticsearch team to use the same client instance across your app.
Considering your exception above you might need to look your configuration
first (like cluster name and host/port) and you should use port 9300 for
the Java API. Finally, check whether you are under a firewall or something
and you block 9300 port.

Hope it helps
Thomas

On Monday, 10 March 2014 15:42:25 UTC+2, Robert Langenfeld wrote:

Hello,

I'm developing a tomcat webserver application that uses Elasticsearch
1.0 (Java API). There is a client facing desktop application that
communicates with the server so all the code for Elasticsearch is on that
one instance and it is used by all our clients. With that being said I am
running into this issue: After initializing a new TransportClient object
and performing some operation on it, there is a chance that i could sit
idle for a very long time. When does sit idle for a long time it gets this
error:

Mar 08, 2014 1:15:37 AM org.elasticsearch.client.transport

INFO: [Elven] failed to get node info for [#transport#-1][WIN7-113-
00726][inet[/159.140.213.87:9300]], disconnecting...

org.elasticsearch.transport.RemoteTransportException:
[Server_Dev1][inet[/159.140.213.87:9300]][cluster/nodes/info]

Caused by: java.lang.NullPointerException

at org.elasticsearch.http.HttpInfo.writeTo(HttpInfo.java:82)

at org.elasticsearch.action.admin.cluster.node.info.
NodeInfo.writeTo(NodeInfo.java:301)

at org.elasticsearch.action.admin.cluster.node.info.
NodesInfoResponse.writeTo(NodesInfoResponse.java:63)

at org.elasticsearch.transport.netty.NettyTransportChannel.sendResponse(
NettyTransportChannel.java:83)

at org.elasticsearch.action.support.nodes.TransportNodesOperationAction$
TransportHandler$1.onResponse(TransportNodesOperationAction.java:244)

at org.elasticsearch.action.support.nodes.TransportNodesOperationAction$
TransportHandler$1.onResponse(TransportNodesOperationAction.java:239)

at org.elasticsearch.action.support.nodes.TransportNodesOperationAction$
AsyncAction.finishHim(TransportNodesOperationAction.java:225)

at org.elasticsearch.action.support.nodes.TransportNodesOperationAction$
AsyncAction.onOperation(TransportNodesOperationAction.java:200)

at org.elasticsearch.action.support.nodes.TransportNodesOperationAction$
AsyncAction.access$900(TransportNodesOperationAction.java:102)

at org.elasticsearch.action.support.nodes.TransportNodesOperationAction$
AsyncAction$2.run(TransportNodesOperationAction.java:146)

at java.util.concurrent.ThreadPoolExecutor.runWorker(
ThreadPoolExecutor.java:1145)

at java.util.concurrent.ThreadPoolExecutor$Worker.run(
ThreadPoolExecutor.java:615)

at java.lang.Thread.run(Thread.java:744)

Is there any way to prevent this from happening? I know the ideal
situation would be that after every request the transport client is closed.
But since it lives on a webserver with lots of search requests coming in,
we would ideally like it to stay open because it takes 3-4 seconds for a
transport client to initialize and we are going for speed here.

Also since we are having one central server to handle all search and
index requests, can the TransportClient handle multiple simultaneous
requests from different users at the same time? We just want to make sure
that we are doing this correctly.

--
You received this message because you are subscribed to the Google Groups
"elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an
email to elasticsearch+unsubscribe@googlegroups.com.
To view this discussion on the web visit
https://groups.google.com/d/msgid/elasticsearch/760f6a9f-a917-494e-9328-c1b3e9877cae%40googlegroups.com https://groups.google.com/d/msgid/elasticsearch/760f6a9f-a917-494e-9328-c1b3e9877cae%40googlegroups.com?utm_medium=email&utm_source=footer
.

For more options, visit https://groups.google.com/d/optout.

--
You received this message because you are subscribed to the Google Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email to elasticsearch+unsubscribe@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/elasticsearch/CAKdsXoHUYejnLU_4OwkwrPnLZX%3D%3DBoBiKHvbaGO48FmrMfZQ%3DQ%40mail.gmail.com.
For more options, visit https://groups.google.com/d/optout.